From oden at cs.wisc.edu Thu Feb 2 12:36:44 1989 From: oden at cs.wisc.edu (Greg Oden) Date: Thu, 2 Feb 89 11:36:44 CST Subject: Faculty position Message-ID: <8902021736.AA29380@ai.cs.wisc.edu> The Department of Psychology of the University of Wisconsin anticipates making the appointment of an Assistant Professor in the area of cognitive science effective August 1989. We are most interested in people working with connectionist models but welcome applications from experimental psychologists with strong research records in any area of cognitive science. To apply, send your vita and have three letters of recommendation sent to: New Personnel Committee Department of Psychology University of Wisconsin Madison, WI 53706. Applications received after March 1, 1989 may not be given full consideration. The University of Wisconsin is an Affirmative Action/Equal Opportunity Employer. From jose at tractatus.bellcore.com Thu Feb 2 13:52:25 1989 From: jose at tractatus.bellcore.com (Stephen J Hanson) Date: Thu, 2 Feb 89 13:52:25 EST Subject: NIPS CALL FOR PAPERS Message-ID: <8902021852.AA12552@tractatus.bellcore.com> CALL FOR PAPERS IEEE Conference on Neural Information Processing Systems - Natural and Synthetic - Monday, November 27 -- Thursday November 30, 1989 Denver, Colorado This is the third meeting of a high quality, relatively small, inter-disciplinary conference which brings together neuroscientists, engineers, computer scientists, cognitive scientists, physicists, and mathematicians interested in all aspects of neural processing and computation. Several days of focussed workshops will follow at a nearby ski area. Major categories and examples of subcategories for papers are the following: 1. Neuroscience: Neurobiological models of development, cellular information processing, synaptic function, learning, and memory. Studies and analyses of neurobiological systems and development of neurophysiological recording tools. 2. Architecture Design: Design and evaluation of net architectures to perform cognitive or behavioral functions and to implement conventional algorithms. Data representation; static networks and dynamic networks that can process or generate pattern sequences. 3. Learning Theory Models of learning; training paradigms for static and dynamic networks; analysis of capability, generalization, complexity, and scaling. 4. Applications: Applications to signal processing, vision, speech, motor control, robotics, knowledge representation, cognitive modelling and adaptive systems. 5. Implementation and Simulation: VLSI or optical implementations of hardware neural nets. Practical issues for simulations and simulation tools. Technical Program: Plenary, contributed, and poster sessions will be held. There will be no parallel sessions. The full text of presented papers will be published. Submission Procedures: Original research contributions are solicited, and will be refereed by experts in the respective disciplines. Authors should submit four copies of a 1000-word (or less) summary and four copies of a single-page 50-100 word abstract clearly stating their results by May 30, 1989. Indicate preference for oral or poster presentation and specify which of the above five broad categories and, if appropriate, sub- categories (for example, Learning Theory: Complexity, or Applications: Speech) best applies to your paper. Indicate presentation preference and category information at the bottom of each abstract page and after each summary. Failure to do so will delay processing of your submission. Mail submissions to Kathy Hibbard, NIPS89 Local Committee, Engineering Center, Campus Box 425, Boulder, CO, 80309-0425. DEADLINE FOR SUMMARIES ABSTRACTS IS MAY 30, 1989 From mesard at BBN.COM Fri Feb 3 14:18:51 1989 From: mesard at BBN.COM (mesard@BBN.COM) Date: Fri, 03 Feb 89 14:18:51 -0500 Subject: Post-processing of neural net output (SUMMARY) Message-ID: About a month ago, I asked for information about post-processing of output activation of a (trained or semi-trained) network solving a classification task. My specific interest was what additional information can be extracted from the output vector, and what techniques are being used to improve performance and/or adjust the classification criteria (i.e., how the output is interpreted). I've been thinking about how Signal Detection Theory (SDT; cf. Green and Swets, 1966) could be applied to NN classification systems. Three areas I am concerned about are: 1) Typically interpretation of a net's classifications ignores the cost/payoff matrix associated with the classification decision. SDT provides a way to take this into account. 2) A "point-5 threshold interpretation" of output vectors is in some sense arbitrary given (1) and because it may have developed a "bias" (predisposition) towards producing a particular response (or responses) as an artifact of its training. 3) The standard interpretation does not take into account the a priori probability (likelihood) of an input of a particular type being observed. SDT may also provide an interesting way to compare two networks. Specifically, the d' ("D-prime") measure and the ROC (receiver operating characteristic) curves which have been successfully used to analyze human decision making, may be quite useful in understanding NN behavior. --- The enclosed summary covers only responses that addressed these specific issues. (The 19 messages I received totaled 27.5K. This summary is just under 8K. I endeavored to preserve all the non-redundant information and citations.) Thanks to all who replied. -- void Wayne_Mesard(); Mesard at BBN.COM Bolt Beranek and Newman, Cambridge, MA -- Summary of citation respondents: ------- -- -------- ------------ The following two papers discuss interpretation of multi-layer perceptron outputs using probabilistic or entropy-like formulations @TECHREPORT{Bourlard88, AUTHOR = "H. Bourlard and C. J. Wellekens", YEAR = "1988", TITLE = "Links Between {M}arkov Models and Multilayer Perceptrons", INSTITUTION = "Philips Research Laboratory", MONTH = "October", NUMBER = "Manuscript M 263", ADDRESS = "Brussels, Belgium" } @INPROCEEDINGS{Golden88, AUTHOR = "R. M. Golden", TITLE = "Probabilistic Characterization of Neural Model Computations", EDITOR = "D. Anderson", BOOKTITLE = "Neural Information Processing Systems", PUBLISHER = "American Institute of Physics", YEAR = "1988", ADDRESS = "New York", PAGES = "310-316" } Geoffrey Hinton (and others) cites Hinton, G. E. (1987) "Connectionist Learning Procedures", CMU-CS-87-115 (version 2) as a review of some post-processing techniques. He said that this tech report will eventually appear in the AI journal. He also says: The central idea is that any gradient descent learning procedure works just fine if the "neural net" has a non-adaptive post processing stage which is invertible -- i.e. it must be possible to back-propagate the difference between the desired and actual outputs through the post processing. [...] The most sophisticated post-processing I know of is Herve Bourlard's use of dynamic time warping to map the output of a net onto a desired string of elements. The error is back-propagated through the best time warp to get error derivatives for the detection of the individual elements in the sequence. The paper by Kaplan and Johnson in the 1988 ICNN Proceedings addressed the problem. A couple of people Michael Jordan has done interesting work in the area of post-processing, but no citations were provided. (His work from 2-3 years ago does discuss interpretation of output when trained with "don't care"s in the target vector. I don't know if this is what they were referring to.) "Best Guess" ---- ----- This involves looking at the set of valid output vectors, V(), and the observed output, O, and interpreting O as V(i) where i minimizes |V(i) - O| . For one-unit-on-the-rest-off output vectors, this is the same thing as taking the one with the largest activation, but when classifying along multiple dimensions simultaneously, this technique may be quite useful. ---- J.E. Roberts sent me a paper by A.P. Doohovskoy called "Metatemplates," presented at ICASSP Dallas, 1987 (no, I don't know what that is). He (Roberts) suggests using "a trained or semi-trained neural net to produce one 'typical' output for each type of input class. These vectors would be saved as 'metatemplates'." Then classification can be done by comparing (via Euclidian distance or dot product) observed output vectors with the metatemplates (where the closest metatemplate wins). This is uses the information from the entire network output vector for classification. Probability Measures ----------- -------- Terry Sejnowski writes: The value of an output unit is highly correlated with the confidence of a binary categorization. In our study of predicting protein secondary structure (Qian and Sejnowski, J. Molec. Biol., 202, 865-884) we have trained a network to perform a three-way classification. Recently we have found that the real value of the output unit is highly correlated with the probability of correct classification of new, testing sequences. Thus, 25% of the sequences could be predicted correctly with 80% or greater probability even though the average performance on the training set was only 64%. The highest value among the output units is also highly correlated with the difference between the largest and second largest values. We are preparing a paper for publication on these results. --- Mark Gluck writes: In our recent JEP:General paper (Gluck & Bower, 1988) we showed how the activations could be converted to choice probabilities using an exponential ratio function. This leads to good quantitative fits to human choice performance both at asymptote and during learning. --- Tony Robinson states that the summed squared difference between the actual output vector and the relevant target vector provides a measure of the probability of belonging to each class [in a one-bit-on-others-off output set]. [See "Best Guess" above.] Confidence Measures ---------- -------- John Denker says: Yes, we've been using the activation level of the runner-up neurons to provide confidence information in our character recognizer for some time. The work was reported at the last San Diego mtg and at the last Denver mtg. --- Mike Rossen describes the speech recognition system that he and Jim Anderson are working on. The target vectors are real-valued. With each phoneme represented by several units with activation on [-1, 1]: Our retrieval method is a discretized dynamical system in which system output is fed back into the system using appropriate feedback and decay parameters. Our scoring method is based on an average activation threshold, but the number of iterations the -> system takes to reach this threshold -- the system reaction time -- -> serves as a confidence measure. [He also reports on intra-layer connections on the outputs (otherwise, he's using a vanilla feedforward net) which sounds like a groovy idea, although it seems to me that this would have pros and cons in his application.] After the feedforward network is trained, connections AMONG THE OUTPUT UNITS are trained. this "post-processing" reduces both omission and confusion errors by the system. Some preliminary results of the speech model are reported in: Rossen, M.L., Niles, L.T., Tajchman, G.N., Bush, M.A., & Anderson, J.A. (1988). Training methods for a connectionist model of CV syllable recognition. Proceedings of the Second Annual International Conference on Neural Networks, 239-246. Rossen, M.L., Niles, L.T., Tajchman, G.N., Bush, M.A., Anderson, J.A., & Blumstein, S.E. (1988). A connectionist model for consonant-vowel syllable recognition. ICASSP-88, 59-66. Improving Discriminability --------- ---------------- Ralph Linsker says: You may be interested in an issue related, but not identical, to the one you raised; namely, how can one tailor the network's response so that the output optimally discriminates among the set of input vectors, i.e. so that the output provides maximum information about what the input vector was? This is addressed in: R. Linsker, Computer 21(3)105-117 (March 1988); and in my papers in the 1987 and 1988 Denver NIPS conferences. The quantity being maximized is the Shannon information rate (from input to output), or equivalently the average mutual information between input and output. --- Dave Burr refers to D. J. Burr, "Experiments with a Connectionist Text Reader," Proc. ICNN-87, pp. IV717-IV724, San Diego, CA, June 1987. Which describes a post-processing routine which assigns a score to every word in an English dictionary by summing log compressed activations. -=-=-=-=-=-=-=-=-=- From honavar at cs.wisc.edu Fri Feb 3 14:31:37 1989 From: honavar at cs.wisc.edu (A Buggy AI Program) Date: Fri, 3 Feb 89 13:31:37 CST Subject: Workshop on Optimization and Neural Nets Message-ID: <8902031931.AA06314@goat.cs.wisc.edu> Subject: Call For Papers : Neural Nets & Optimization. CALL FOR PAPERS TENCON '89 (IEEE Region 10 Conference) SESSION ON OPTIMIZATION AND NEURAL NETWORKS November 22 -- 24, 1989 Bombay, India Under the auspices of the IEEE, the session organizers invite submission of papers for a session on "Optimization and Neural Networks". This session will focus on the interrelationship of neural networks and optimization problems. Neural networks can be seen to be related to optimization in two distinct ways: + As an adaptive neural network learns from examples, the convergence of its weights solves an optimiza- tion problem. + A large class of networks , even with constant we- ights , solves optimization problems as they settle from initial to final state. The areas of interest include but are not limited to: + Combinatorial optimization + Continuous optimization + Sensor integration ( when posed as an optimization problem) + Mean Field Annealing + Stochastic Relaxation Depending on the number and quality of the responses,this ses- sion may be split into multiple sessions, with one part focus- ing on optimizing the weight-determination process in adaptive nets,and the second one on using those nets to solve other pro blems. Prospective authors should submit two copies of an extended ab stract (not exceeding 5 pages , double spaced) of their papers to either of the organizers by March 31, 1989. Authors will be notified of acceptance or rejection by May 15,1989.Photo-ready copy of the complete paper (not exceeding 25 pages double-spa- ced) must be received by Jul 15,1989 for inclusion in the pro- ceedings which will be published by the IEEE and distributed at the symposium. Session Organizers Dr. Wesley E. Snyder / Mr. Harish P. Hiriyannaiah Dept of Electrical and Computer Engineering North Carolina State University Raleigh, NC 27695-7911, USA Telephone: (919)-737-2336 FAX: (919)-737-7382 email: {wes,harish}@ecelet.ncsu.edu -- (Internet) mcnc!ece-csc!{wes,harish} -- (UUCP) -- From djb at flash.bellcore.com Fri Feb 3 17:16:13 1989 From: djb at flash.bellcore.com (David J Burr) Date: Fri, 3 Feb 89 17:16:13 EST Subject: Post-processing of neural net output Message-ID: <8902032216.AA03217@flash.bellcore.com> The following soon-to-appear paper uses signal detection theory (Green and Swets, 1966) to compare various methods for vowel recognition. Among the methods compared are linear prediction, weighted cepstra and neural nets, with both bark scale and linear frequency normalization. Requests should be emailed to: cak at bellcore.com C. A. Kamm, L. A. Streeter, Y. Kane-Esrig, and D. J. Burr, "Comparing Performance of Spectral Distance Measures and Neural Network Methods for Vowel Recognition," Computers, Speech and Language (to appear). From harnad at Princeton.EDU Sun Feb 5 13:00:12 1989 From: harnad at Princeton.EDU (Stevan Harnad) Date: Sun, 5 Feb 89 13:00:12 EST Subject: Feature Detection, Symbolic Rules and Connectionism Message-ID: <8902051800.AA01398@dictus.Princeton.EDU> I am redirecting to connectionists a segment of an ongoing discussion of categorization on comp.ai that seems to have taken a connectionistic turn. I think it will all be understandable from context. The issue concerns whether category representations are "nonclassical" (i.e., with membership a matter of degree, and no features that provide necessary and sufficient conditions for assigning membership) or "classical" (i.e., with all-or-none membership, assigned on the basis of features that do provide necessary and sufficient conditions). I am arguing against the former and for the latter. Connectionism seems to have slipped in as a way of having features yet not-having them too, so to speak, and the discussion has touched base with the familiar question of whether or not connectionist representations are really representational or ruleful: anwst at cisunx.UUCP (Anders N. Weinstein) of Univ. of Pittsburgh, Comp & Info Sys wrote: " I think Harnad errs... that reliable categorization *must* be " interestingly describable as application of some (perhaps complex) rule " in "featurese" (for some appropriate set of detectable features)... " Limiting ourselves (as I think we must) to quick and automatic " observational classification... If... the effects of context on such tasks " are minimal... there must be within us some isolable module which can " take sensory input and produce a one bit yes-or-no output for category " membership... But how does it follow that such a device must be " describable as applying some *rule*? Any physical object in the world " could be treated as a recognition device for something by interpreting " some of its states as "inputs" and some as "yes-or-no responses." But " intuitively, it looks like not every such machine is usefully described " as applying a rule in this way. In particular, this certainly doesn't " seem a natural way of describing connectionist pattern recognizers. So " why couldn't it turn out that there is just no simpler description of " the "rule" for certain category membership than: whatever a machine of " a certain type recognizes? For the points I have been trying to make it does not matter whether or not the internal basis for a machine's feature-detecting and categorizing success is described by us as a "rule" (though I suspect it can always be described that way). It does not even matter whether or not the internal basis consists of an explicit representation of a symbolic rule that is actually "applied" (in fact, according to my theory, such symbolic representations of categories would first have to be grounded in prior nonsymbolic representations). A connectionist feature-detector would be perfectly fine with me; I even suggest in my book that that would be a natural (and circumscribed) role for a connectionist module to play in a category representation system (if it can actually deliver the goods). To rehabilitate the "classical" view I've been trying to rescue from well over a decade of red herrings and incoherent criticism all I need to re-establish is that where there is reliable, correct, all-or-none categorization performance, there must surely exist detectable features in the input that are actually detected by the categorizing device as a ("necessary and sufficient") basis for its successful categorization performance. I think this should be self-evident to anyone who is mindful of the obvious facts about our categorization performance capacity and is not in the grip of a California theory (and does not believe in magic). The so-called "classical" view is only that features must EXIST in the inputs that we are manifestly able to sort and label, and that these features are actually DETECTED and USED to generate our successful performance. The classical view is not committed to internal representations of rules symbolically describing the features in "featurese" or operating on symbolic descriptions of features. That's another issue. (According to my own theory, symbolic "featurese" itself, like all abstract category labels in the "language of thought," must first be grounded in nonsymbolic, sensory categories and their nonsymbolic, sensory features.) [By the way, I don't think there's really a problem with sorting out which devices are actually categorizing and which ones aren't. Do you, really? That sounds like a philosopher's problem only. (If what you're worried about is whether the categorizer really has a mind, then apply my Total Turing Test -- require it to have ALL of our robotic and linguistic capacities.) Nor does "whatever a machine of a certain type recognizes" sound like a satisfactory answer to the "question of how in fact our neural machinery functions to enable us to so classify things." You have to say what features it detects, and HOW.] [Related to the last point, Greg Lee (lee at uhccux.uhcc.hawaii.edu), University of Hawaii, had added, concerning connectionist feature-detectors: "If you don't understand how the machine works, how can you give a rule?" I agree that the actual workings of connectionist black boxes need more analysis, but to a first approximation the answer to the question of how they work (if and when they work) is: "they learn features by sampling inputs, with feedback about miscategorization, `using' back-prop and the delta rule." And that's certainly a lot better than nothing. A fuller analysis would require specifying what features they're detecting, and how they arrived at them on the available data, as constrained by back-prop and the delta rule. There's no need whatsoever for any rules to be explicitly "represented" in order to account fully for their success, however. -- In any case, connectionist black boxes apparently do not settle the classical/nonclassical matter one way or the other, as evidenced by the fact that there seems to be ample room for them in both nonclassical approaches (e.g., Lakoff's) and classical ones (e.g., mine).] " [We must distinguish] the normative question of which things are " *correctly* classified as birds or even numbers, and the descriptive " question of how in fact our neural machinery functions to enable us to " so classify things. I agree also with Harnad that psychology ought to " keep its focus on the latter and not the former of these questions. A kind of "correctness" factor does figure in the second question too: To model how people categorize things we have to have data on what inputs they categorize as members of what categories, according to what constraints on MIScategorization. However, it's certainly not an ontological correctness that's at issue, i.e., we're not concerned with what the things people categorize really ARE "sub specie aeternitatis": We're just concerned with what people get provisionally right and wrong, under the constraints of the sample they've encountered so far and the feedback they've so far received from the consequences of miscategorization. I also see no reason to limit our discussion to "quick, automatic, observational" categorization; it applies just as much to slow perceptual pattern learning and, with proper grounding, to abstract, nonperceptual categorization too (although here is where explicitly represented symbolic rules [in "featurese"?] do play more of a role, according to my grounding theory). And I think context effects are rarely "minimal": All categorization is provisional and approximate, dependent on the context of confusable alternatives so far sampled, and the consequences (so far) of miscategorizing them. Stevan Harnad harnad at confidence.princeton.edu harnad at pucc.bitnet From pablo at june.cs.washington.edu Tue Feb 7 18:59:55 1989 From: pablo at june.cs.washington.edu (David Cohn) Date: Tue, 7 Feb 89 15:59:55 PST Subject: Ummm... a request Message-ID: <8902072359.AA12054@june.cs.washington.edu> First off, could someone please add me to the mailing list? (Thanks) With that out of the way: I'm with the Computer Sci. Dept. at U. Washington. My research interests are in formal learning theory, specifically how it can be applied to the study of the capabilities and limitations of neural networks. Research in our department is rather heavily weighted on the computational complexity side of the problem with relatively little interest (other than myself) on the neural network side. With the blessings of my advisor, Dr. Richard Ladner, I'm interested in spending the summer attempting to "cross-pollinate" some ideas in learning and neural networks. I would like to be able to work with some research group apart from UW to learn their approaches to the problems and hopefully contribute some of my own. Could anyone refer me to some likely targets? I actually have quite a bit of experience hacking neural networks, but am looking for a research group studying more theoretical aspects of the problem of neural net learning. (I can send detailed background, references and c.v. to anyone interested.) Thanks, David "Pablo" Cohn (pablo at cs.washington.edu) Dept. of Comp. Sci., FR-35 University of Washington (206) 543-7798 days Seattle, WA 98195 From netlist at psych.Stanford.EDU Wed Feb 8 10:06:25 1989 From: netlist at psych.Stanford.EDU (Mark Gluck) Date: Wed, 8 Feb 89 07:06:25 PST Subject: (Thurs. 2/9): Carver Mead on Neural VLSI Message-ID: Stanford University Interdisciplinary Colloquium Series: Adaptive Networks and their Applications Feb. 9th (Thursday, 3:30pm): ----------------------------- -- Note new room: 380-380X -- ******************************************************************************** VLSI Models of Neural Networks CARVER MEAD Moore Professor of Computer Science Calif. Inst. of Technology Pasadena, CA 91125 (818) 356 -6841 ******************************************************************************** Abstract Semiconductor technology has evolved to the point where chips containing a million transistors can be fabricated without defects. If a small number of defects can be tolerated, this number is increased by two orders of magnitude. Devices now being fabricated on an experimental basis have shown that another two orders of magnitude are possible. The inescapable conclusion is that wafers containing 10**10 devices, of which only a vanishing fraction are defective, will be in production within a few years. This level of complexity is well below that required for higher cortical functions, but is already sufficient to solve lower level perception tasks. This remarkable technology has made possible a new discipline: Synthetic Neurobiology. The thesis of this discipline is that it is not possible, even in principle, to claim a full understanding of a system unless one is able to build one that functions properly. This principle is already well accepted in mollecular biology, and more recently in genetics. It is hoped that the approach will soon join the traditional descriptive and analytical foundations of neurobiology. Small examples using current technology to attack problems in early vision and hearing will be described. Additional Information ---------------------- Location: Room 380-380X, which can be reached through the lower level between the Psychology and Mathematical Sciences buildings. Technical Level: These talks will be technically oriented and are intended for persons actively working in related areas. They are not intended for the newcomer seeking general introductory material. Mailing lists: To be added to the network mailing list, netmail to netlist at psych.stanford.edu. For additional information, or contact Mark Gluck (gluck at psych.stanford.edu). Co-Sponsored by: Departments of Electrical Engineering (B. Widrow) and Psychology (D. Rumelhart, M. Pavel, M. Gluck), Stanford Univ. From rsun at cs.brandeis.edu Tue Feb 7 11:14:34 1989 From: rsun at cs.brandeis.edu (Ron Sun) Date: Tue, 7 Feb 89 11:14:34 est Subject: No subject Message-ID: I am currently doing research on modeling biological neural networks according to the accurately identified connectivity patterns found by biologists. I will appreciate any pointers to the published papers, ongoing research or just philosophical thoughts on the subject, esp. regarding the following questions: 1) which model (existing or to be invented) can best describe real neural networks? 2) what are the criteria for measuring accuracy of a model in terms of its emergent behavior and network dynamics? 3) In which level of abstraction, should wee try to model biological neural networks in order to advance our understanding of neural networks in general? Please send response to rsun%cs.brandeis.edu at relay.cs.net Ron Sun Brandeis University CS Dept Waltham, MA 02254 From harish at ecelet.ncsu.edu Wed Feb 8 17:11:50 1989 From: harish at ecelet.ncsu.edu (Harish Hiriyannaiah) Date: Wed, 8 Feb 89 17:11:50 EST Subject: Call for Papers - TENCON '89. Message-ID: <8902082211.AA01004@ecelet.ncsu.edu> CALL FOR PAPERS TENCON '89 (IEEE Region 10 Conference) SESSION ON OPTIMIZATION AND NEURAL NETWORKS November 22 -- 24, 1989 Bombay, India Under the auspices of the IEEE, the session organizers invite submission of papers for a session on "Optimization and Neural Networks". This session will focus on the interrelationship of neural networks and optimization problems. Neural networks can be seen to be related to optimization in two distinct ways: + As an adaptive neural network learns from examples, the convergence of its weights solves an optimiza- tion problem. + A large class of networks , even with constant we- ights , solves optimization problems as they settle from initial to final state. The areas of interest include but are not limited to: + Combinatorial optimization + Continuous optimization + Sensor integration ( when posed as an optimization problem) + Mean Field Annealing + Stochastic Relaxation Depending on the number and quality of the responses,this ses- sion may be split into multiple sessions, with one part focus- ing on optimizing the weight-determination process in adaptive nets,and the second one on using those nets to solve other pro blems. Prospective authors should submit two copies of an extended ab stract (not exceeding 5 pages , double spaced) of their papers to either of the organizers by March 31, 1989. Authors will be notified of acceptance or rejection by May 15,1989.Photo-ready copy of the complete paper (not exceeding 25 pages double-spa- ced) must be received by Jul 15,1989 for inclusion in the pro- ceedings which will be published by the IEEE and distributed at the symposium. Session Organizers Dr. Wesley E. Snyder / Mr. Harish P. Hiriyannaiah Dept of Electrical and Computer Engineering North Carolina State University Raleigh, NC 27695-7911, USA Telephone: (919)-737-2336 FAX: (919)-737-7382 email: {wes,harish}@ecelet.ncsu.edu -- (Internet) mcnc!ece-csc!{wes,harish} -- (UUCP) From hlogan at watdcs.UWaterloo.ca Thu Feb 9 11:37:10 1989 From: hlogan at watdcs.UWaterloo.ca (Harry M. Logan) Date: Thu, 9 Feb 89 11:37:10 EST Subject: No subject Message-ID: Dear list owner, I should be grateful if you can add my name to the list of subscribers of Connectionists. My name is Harry M. Logan, and the e-mail address is: hlogan at watdcs.UWaterloo.ca Thank you for your consideration of this matter. Sincerely yours, Harry Logan From mel at cougar.ccsr.uiuc.edu Thu Feb 9 13:26:34 1989 From: mel at cougar.ccsr.uiuc.edu (Bartlett Mel) Date: Thu, 9 Feb 89 12:26:34 CST Subject: thesis/tech report Message-ID: <8902091826.AA24151@cougar.ccsrsun> The following thesis/TR is now available--about 50% of it is dedicated to relations to traditional methods in robotics, and to psychological and biological issues... MURPHY: A Neurally-Inspired Connectionist Approach to Learning and Performance in Vision-Based Robot Motion Planning Bartlett W. Mel Center for Complex Systems Research Beckman Institute, University of Illinois Many aspects of intelligent animal behavior require an understanding of the complex spatial relationships between the body and its parts and the coordinate systems of the external world. This thesis deals specifically with the problem of guiding a multi-link arm to a visual target in the presence of obstacles. A simple vision-based kinematic controller and motion planner based on a connectionist network architecture has been developed, called MURPHY. The physical setup consists of a video camera and a Rhino XR-3 robot arm with three joints that move in the image plane of the camera. We assume no a priori model of arm kinematics or of the imaging characteristics of the camera/visual system, and no sophisticated built-in algorithms for obstacle avoidance. Instead, MURPHY builds a model of his arm through a combination of physical and ``mental'' practice, and then uses simple heuristic search with mental images of his arm to solve visually-guided reaching problems in the presence of obstacles whose traditional algorithmic solutions are extremely complex. MURPHY differs from previous approaches to robot motion-planning primarily in his use of an explicit full-visual-field representation of the workspace. Several other aspects of MURPHY's design are unusual, including the sigma-pi synaptic learning rule, the teacherless training paradigm, and the integration of sequential control within an otherwise connectionist architecture. In concluding sections we outline a series of strong correspondences between the representations and algorithms used by MURPHY, and the psychology, physiology, and neural bases for the programming and control of directed, voluntary arm movements in humans and animals. You can write to me: mel at complex.ccsr.uiuc.edu, or judi jr at complex.ccsr.uiuc.edu. Out computers go down on Feb. 13 for 2 days, so if you want one then, call (217)244-4250 instead. -Bartlett Mel From jose at tractatus.bellcore.com Thu Feb 9 13:16:17 1989 From: jose at tractatus.bellcore.com (Stephen J Hanson) Date: Thu, 9 Feb 89 13:16:17 EST Subject: NIPS latex version PLEASE FORMAT, PRINT and POST Message-ID: <8902091816.AA06682@tractatus.bellcore.com> \documentstyle[11pt]{article} %% set sizes to fill page with small margins \setlength{\headheight}{0in} \setlength{\headsep}{0in} \setlength{\topmargin}{-0.25in} \setlength{\textwidth}{6.5in} \setlength{\textheight}{9.5in} \setlength{\oddsidemargin}{0.0in} \setlength{\evensidemargin}{0.0in} \setlength{\footheight}{0.0in} \setlength{\footskip}{0.25in} \begin{document} \pagestyle{empty} \Huge \begin{center} {\bf CALL FOR PAPERS\\} \Large IEEE Conference on\\ \LARGE {\bf Neural Information Processing Systems\\ - Natural and Synthetic -\\} \bigskip \Large Monday, November 27 -- Thursday November 30, 1989\\ Denver, Colorado\\ \end{center} \medskip \large \noindent This is the third meeting of a high quality, relatively small, inter-disciplinary conference which brings together neuroscientists, engineers, computer scientists, cognitive scientists, physicists, and mathematicians interested in all aspects of neural processing and computation. Several days of focussed workshops will follow at a nearby ski area. Major categories and examples of subcategories for papers are the following: \begin{quote} \small \begin{description} \item[{\bf 1. Neuroscience:}] Neurobiological models of development, cellular information processing, synaptic function, learning, and memory. Studies and analyses of neurobiological systems and development of neurophysiological recording tools. \item[{\bf 2. Architecture Design:}] Design and evaluation of net architectures to perform cognitive or behavioral functions and to implement conventional algorithms. Data representation; static networks and dynamic networks that can process or generate pattern sequences. \item[{\bf 3. Learning Theory:}] Models of learning; training paradigms for static and dynamic networks; analysis of capability, generalization, complexity, and scaling. \item[{\bf 4. Applications:}] Applications to signal processing, vision, speech, motor control, robotics, knowledge representation, cognitive modelling and adaptive systems. \item[{\bf 5. Implementation and Simulation:}] VLSI or optical implementations of hardware neural nets. Practical issues for simulations and simulation tools. \end{description} \end{quote} \large \smallskip \noindent {\bf Technical Program:} Plenary, contributed, and poster sessions will be held. There will be no parallel sessions. The full text of presented papers will be published. \medskip \noindent {\bf Submission Procedures:} Original research contributions are solicited, and will be refereed by experts in the respective disciplines. Authors should submit four copies of a 1000-word (or less) summary and four copies of a single-page 50-100 word abstract clearly stating their results by May 30, 1989. Indicate preference for oral or poster presentation and specify which of the above five broad categories and, if appropriate, sub-categories (for example, {\em Learning Theory: Complexity}, or {\em Applications: Speech}) best applies to your paper. Indicate presentation preference and category information at the bottom of each abstract page and after each summary. Failure to do so will delay processing of your submission. Mail submissions to Kathie Hibbard, NIPS89 Local Committee, Engineering Center, Campus Box 425, Boulder, CO, 80309-0425. \medskip \noindent {\bf Organizing Committee}\\ \small \noindent {Scott Kirkpatrick, IBM Research, General Chairman; Richard Lippmann, MIT Lincoln Labs, Program Chairman; Kristina Johnson, University of Colorado, Treasurer; Stephen J. Hanson, Bellcore, Publicity Chairman; David S. Touretzky, Carnegie-Mellon, Publications Chairman; Kathie Hibbard, University of Colorado, Local Arrangements; Alex Waibel, Carnegie-Mellon, Workshop Chairman; Howard Wachtel, University of Colorado, Workshop Local Arrangements; Edward C. Posner, Caltech, IEEE Liaison; James Bower, Caltech, Neurosciences Liaison; Larry Jackel, AT\&T Bell Labs, APS Liaison} \begin{center} \large {\bf DEADLINE FOR SUMMARIES \& ABSTRACTS IS MAY 30, 1989}\\ \end{center} \begin{flushright} Please Post \end{flushright} \end{document} From KINSELLAJ at vax1.nihel.ie Thu Feb 9 12:35:00 1989 From: KINSELLAJ at vax1.nihel.ie (KINSELLAJ@vax1.nihel.ie) Date: Thu, 9 Feb 89 17:35 GMT Subject: Identity Mappings Message-ID: John A. Kinsella Mathematics Dept., University of Limerick, Limerick, IRELAND KINSELLAJ at VAX1.NIHEL.IE The strategy "identity mapping", namely training a feedforward network to reproduce its input was (to the best of my knowledge) suggested by Geoffrey Hinton and applied in a paper by J.L. Elman & D. Zipser "Learning the hidden structure of speech". It is not clear to me, however, that this approach can do more than aid in the selection of the salient features of the data set. In other words what use is a network which has been trained as an identity mapping on (say) a vision problem? Certainly one can "strip off" the output layer & weights and by a simple piece of linear algebra determine the appropriate weights to transform the hidden layer states into output states corresponding to the salient features mentioned above. It would appear, though, that this is almost as expensive a procedure computationally as training the network as well as being numerically unstable with respect to the subset of the training set selected for the purpose. I would appreciate any comments on these remarks and in particular references to relevant publised material, John Kinsella From kanderso at DINO.BBN.COM Fri Feb 10 15:52:34 1989 From: kanderso at DINO.BBN.COM (kanderso@DINO.BBN.COM) Date: Fri, 10 Feb 89 15:52:34 -0500 Subject: Weight Decay In-Reply-To: Your message of Wed, 25 Jan 89 15:13:58 -0500. <8901252012.AA00971@neural.UUCP> Message-ID: To: att!cs.cmu.edu!connectionists Subject: Re: Weight Decay Reply-To: yann at neural.att.com Date: Wed, 25 Jan 89 15:13:58 -0500 From: Yann le Cun Consider a single layer linear network with N inputs. When the number of training pattern is smaller than N , the set of solutions (in weight space) is a proper linear subspace. adding weight decay will select the minimum norm solution in this subspace (if the weight decay coefficient is decreased with time). The minimum norm solution happens to be the solution given by the pseudo-inverse technique (cf Kohonen), and the solution which optimally cancels out uncorrelated zero mean additive noise on the input. - Yann Le Cun I think this needs some clarification. Your linear network problem is Aw = d, where A is an N x M matrix of input patterns, w is an M x 1 vector of weights, and d is an Nx1 vector of outputs. In the case you described, N < M, and w is underdetermined, ie there are many solutions. The pseudoinverse solution, w, is the one of all solutions that mimimizes |w|^2, ie any other solution will be longer. In the case where N > M and A is full rank, the pseudo-inverse minimizes |d - Aw|^2, ie it is the least squares solution. In the general case, where A is not full rank, the pseudoinverse solution minizes both (1) |d - Aw|^2 and (2) |w|^2. In an iterative network application, a learning step typically minimizes (1) while adding weight decay minimizes (2) at the same time. Another way to say this is that it trys to find a w that minimizes the error subject to the constraint that w is bounded to some length. That length is determined by the weight decay coefficient you use. In general, it would seem wrong to let the weight decay coefficient go to zero, since then you will wind up at the least squares solution which may not be what you want. k From brp at sim.berkeley.edu Fri Feb 10 19:53:47 1989 From: brp at sim.berkeley.edu (bruce raoul parnas) Date: Fri, 10 Feb 89 16:53:47 PST Subject: mailing list Message-ID: <8902110053.AA20168@sim.berkeley.edu> hi, i would like to be placed on the connectionist neural nets mailing list that you distribute. thanx, Bruce Parnas brp at sim.berkeley.edu From hinton at ai.toronto.edu Fri Feb 10 22:49:34 1989 From: hinton at ai.toronto.edu (Geoffrey Hinton) Date: Fri, 10 Feb 89 22:49:34 EST Subject: Identity Mappings In-Reply-To: Your message of Thu, 09 Feb 89 12:35:00 -0500. Message-ID: <89Feb10.224948est.10802@ephemeral.ai.toronto.edu> The potential advantage of using "encoder" networks is that the code in the middle can be developed without any supervision. If the output and hidden units are non-linear, the codes do NOT just span the same subspace as the principal components. The difference between a linear approach like principal components and a non-linear approach is especially significant if there is more than one hidden layer. If the codes from several encoder networks are then used as the input vector for a "higher level" network, one can get a multilayer, modular, unsupervised learning procedure that should scale up better to really large problems. Ballard (AAAI proceedings, 1987) has investigated this approach for a simple problem and has introduced the interesting idea that as the learning proceeds, the central code of each encoder module should give greater weight to the error feedback coming from higher level modules that use this code as input and less weight to the error feedback coming from the output of the code's own module. However, to the best of my knowledge, nobody has yet shown that it really works well for a hard task. One problem, pointed out by Steve Nowlan, is that the codes formed in a bottleneck tend to "encrypt" the information in a compact form that is not necessarily helpful for further processing. It may be worth exploring encoders in nets with many hidden layers that are given inputs from real domains, but my own current view is that to achieve modular unsupervised learning we probably need to optimize some other function which does not simply ensure good reconstruction of the input vector. Geoff Hinton From Zipser%cogsci at ucsd.edu Sat Feb 11 13:49:00 1989 From: Zipser%cogsci at ucsd.edu (Zipser%cogsci@ucsd.edu) Date: Sat, 11 Feb 89 10:49 PST Subject: Identity Mappings In-Reply-To: <89Feb10.224948est.10802@ephemeral.ai.toronto.edu> Message-ID: <890211104950.2.ZIPSER@BUGS.ucsd> Geoff, Perhaps of interest is that in our work with identity mapping of speech, the hidden layer spontaneously learned to represent vowels and consonants in separate groups of units. Within these groups the individual sounds seemed quite compactly coded. Maybe the ease with which we are able to identify the distinct features used to recognize whole items depends on the kind of coding they have in our hidden layers. David Zipser From ST401843%BROWNVM.BITNET at VMA.CC.CMU.EDU Mon Feb 13 00:47:00 1989 From: ST401843%BROWNVM.BITNET at VMA.CC.CMU.EDU (thanasis kehagias) Date: Mon, 13 Feb 89 00:47:00 EST Subject: Connection between Hidden Markov Models and Connectionist Networks Message-ID: the following paper explores the connection between Hidden Markov Models and Connectionist networks. anybody interested in a copy, email me. if you have a TeX setup i will send you the dvi file. else give me your physical mail address. OPTIMAL CONTROL FOR TRAINING THE MISSING LINK BETWEEN HIDDEN MARKOV MODELS AND CONNECTIONIST NETWORKS by Athanasios Kehagias Division of Applied Mathematics Brown University Providence, RI 02912 ABSTRACT For every Hidden Markov Modl there is a set of forward probabilities that need to be computed for both the recognition and training problem . These probabilties are computed recursively and hence the computation can be performed by a multistage , feedforward network that we will call Hidden Markov Model Network (HMMN). This network has exactly the same architecture as the standard Connectionist Network(CN). Furthermore, training a Hidden Markov Model is equivalent to optimizing a function of the HMMN; training a CN is equivalent to minimizing a function of the CN. Due to the multistage feedforward architecture, both problems can be seen as Optimal Control problems. By applying standard Optimal Control techniques, we discover in both problems that certain back propagating quantities (backward probabilities for HMMN, backward propogated errors for CN) are of crucial importance for the solution. So HMMN's and CN's are similar both in architecture and training. ************** i was influenced in this research by the work of H. Bourlard and C. C. Wellekens (the HMM- CN connection) and Y. leCun (Optimal Control applications in CN's). as i was finishing my aper i received a message by J.N. Hwang saying that he and S.Y. Kung have written a paper that includes similar results. Thanasis Kehagias From alexis%yummy at gateway.mitre.org Mon Feb 13 08:43:05 1989 From: alexis%yummy at gateway.mitre.org (alexis%yummy@gateway.mitre.org) Date: Mon, 13 Feb 89 08:43:05 EST Subject: Job Opportunity at MITRE Message-ID: <8902131343.AA02002@marzipan.mitre.org> The MITRE Corporation is looking for technical staff for their expanding neural network effort. MITRE's neural network program currently includes both IR&D and sponsored work in areas ranging from performance analysis, learning algorithms, pattern recognition, and simulation/implementation. The ideal candidate would have the following qualifications: 1. 2-4 years experience in the area of neural networks. 2. Strong background in traditional signal processing with an emphasis on detection and classification theory. 3. Experienced programmer in C/Unix. Experience in graphics (X11/NeWS), scientific programming, symbolic programming, and fast hardware (array and parallel processors) are pluses. 4. US citizenship required. Interested canidates should send resumes to: Garry Jacyna The MITRE Corporation M.S. Z406 7525 Colshire Drive McLean, Va. 22102 USA From postmast at watdcs.UWaterloo.ca Mon Feb 13 13:05:25 1989 From: postmast at watdcs.UWaterloo.ca (DCS Postmaster) Date: Mon, 13 Feb 89 13:05:25 EST Subject: Subscription Request Message-ID: POSTMASTer is sending this request on behalf of one of our users who seems to be having trouble getting mail through to the list-owner. Please consider the following mail, sent on behalf of Prof. Logan. thank you .. walter mccutchan Deptartment of Computer Services Postmaster University of Waterloo, Waterloo, Ontarion -------- Dear list owner, I should be grateful if you can add my name in the list of suscriptors of Connectionists. My name is Harry M. Logan, and my e-mail address is: Thank you for your consideration in this matter. Sincerely yours, Harry Logan From ROB%BGERUG51.BITNET at VMA.CC.CMU.EDU Mon Feb 13 12:54:00 1989 From: ROB%BGERUG51.BITNET at VMA.CC.CMU.EDU (Rob A. Vingerhoeds / Ghent State University) Date: Mon, 13 Feb 89 12:54 N Subject: Neural Networks Seminar, 25 april 1989, Ghent, Belgium Message-ID: BIRA SEMINAR ON NEURAL NETWORKS 25 APRIL 1989 International Congress Centre Ghent BELGIUM BIRA (Belgian Institute for Control Engineering and Automation) is organising a seminar on the state of the art in Neural Networks. The central theme will be "When and how will neural networks become applicable for industry". To be able to give a good and reliable verdict to this theme, some of the most important and leading scientists in this fasci- nating area have been invited to present a lecture at the seminar and take part in a panel discussion. The following schedule is foreseen: 8.30 - 9.00 Registration 9.00 - 9.15 Opening on behalf of BIRA Prof. L. Boullart Ghent State University 9.15 - 10.00 Introduction to the domain Prof. Fogelman Soulie Universite de Paris V 10.00 - 10.30 coffee 10.30 - 11.30 Theoretical Backgrounds and Mathematical Models Prof. B. Kosko University of Southern California 11.30 - 12.00 Special dedicated hardware (probably the French representative of Hecht-Nielsen Neurocomputers) 12.00 - 14.00 lunch / exhibition 14.00 - 15.00 Application in Robotics Dr. David Handelman Princeton 15.00 - 16.00 Application in Image Processing and Pattern Recognition (Neocognitron) Dr. S. Miyake ATR 16.00 - 16.30 tea 16.30 - 17.15 panel discussion over the central theme 17.15 - 17.30 closing and conclusions The seminar will be held in the same period as the famous Flanders Technology International (F.T.I.) exhibition is held. This exhibition is for both representatives from industry and for other interested people very interesting and going to both the seminar and the exhibition is double interesting. It is possible to obtain a ticket for F.T.I. at a reduced price, when attending the seminar. Please indicate, whether you would like to get a ticket, when sending in a letter or an e-mail message. Prices: members of BIRA : 12.500 BEF others : 15.000 BEF universities : 7.500 BEF If you intent to attent our seminar, you can either send a letter to the BIRA coordinator (adress follows) or an e-mail message to one of us. We will fill you in on the details as soon as possible. Rob Vingerhoeds Leo Vercauteren BIRA Coordinator: L. Pauwels BIRA-secretariaat Het Ingenieurshuis Desguinlei 214 2018 Antwerpen Belgium telefax: +32-3-216-06-89 (attn. BIRA L. Pauwels) From jose at tractatus.bellcore.com Tue Feb 14 17:22:22 1989 From: jose at tractatus.bellcore.com (Stephen J Hanson) Date: Tue, 14 Feb 89 17:22:22 EST Subject: NIPS POST-MEETING WORKSHOPS Message-ID: <8902142222.AA12893@tractatus.bellcore.com> NIPS-89 POST-CONFERENCE WORKSHOPS DECEMBER 1-2, 1989 REQUEST FOR PROPOSALS Following the regular NIPS program, workshops on current topics in Neural Information Processing will be held on December 1 and 2, 1989, at a ski resort near Denver. Proposals by qualified individuals interested in chairing one of these workshops are solicited. Past topics have included: Rules and Connectionist Models; Speech, Neural Networks and Hidden Markov Models; Imaging Techniques in Neurobiology; Computational Complexity Issues; Fault Tolerance in Neural Networks; Benchmarking and Comparing Neural Network Applications; Architectural Issues; Fast Training Techniques. The format of the workshops is informal. Beyond reporting on past research, their goal is to provide a forum for scientists actively working in the field to freely discuss current issues of concern and interest. Sessions will meet in the morning and in the afternoon of both days, with free time in between for ongoing individual exchange or outdoor activities. Specific open and/or controversial issues are encouraged and preferred as workshop topics. Individuals interested in chairing a workshop must propose a topic of current interest and must be willing to accept responsibility for their group's discussion. Discussion leaders' responsibilities include: arrange brief informal presentations by experts working on this topic, moderate or lead the discussion; and report its high points, findings and conclusions to the group during evening plenary sessions. Submission Procedure: Interested parties should submit a short proposal for a workshop of interest by May 30, 1989. Proposals should include a title and a short description of what the workshop is to address and accomplish. It should state why the topic is of interest or controversial, why it should be discussed and what the targeted group of participants is. In addition, please send a brief resume of the prospective workshop chair, list of publications and evidence of scholarship in the field of interest. Mail submissions to: Kathie Hibbard NIPS89 Local Committee Engineering Center Campus Box 425 Boulder, CO, 80309-0425 Name, mailing address, phone number, and e-mail net address (if applicable) should be on all submissions. Workshop Organizing Committee: Alex Waibel, Carnegie-Mellon, Workshop Chairman; Howard Wachtel, University of Colorado, Workshop Local Arrangements; Kathie Hibbard, University of Colorado, NIPS General Local Arrangements; PROPOSALS MUST BE RECEIVED BY MAY 30, 1989. From krulwich-bruce at YALE.ARPA Thu Feb 16 11:08:38 1989 From: krulwich-bruce at YALE.ARPA (Bruce Krulwich) Date: Thu, 16 Feb 89 11:08:38 EST Subject: Identity Mappings In-Reply-To: Geoffrey Hinton , Fri, 10 Feb 89 22:49:34 EST Message-ID: <8902161607.AA02472@ELI.CS.YALE.EDU> Geoff Hinton wrote recently: The potential advantage of using "encoder" networks is that the code in the middle can be developed without any supervision. ... If the codes from several encoder networks are then used as the input vector for a "higher level" network, one can get a multilayer, modular, unsupervised learning procedure that should scale up better to really large problems. This brings out a point I've wanted to discuss for a while: Are "encoder nets" any better than, say, competitive learning (or recirculation, or maybe GMax?) for a task such as this?? It seems to me that feed-forward I/O nets are the wrong model for learning correlations, especially if the encodings themselves are what is going to be used for further computation. More generally and to the point, could it be that the success in backprop (in applications and analysis) has resulted in stagnation by tying people to the idea of feed-forward nets?? Bruce Krulwich krulwich at cs.yale.edu ------- From linhf at ester.ecn.purdue.edu Fri Feb 17 10:27:23 1989 From: linhf at ester.ecn.purdue.edu (Han-Fei Lin) Date: Fri, 17 Feb 89 10:27:23 EST Subject: Tech Report Announcement Message-ID: <8902171527.AA00797@ester.ecn.purdue.edu> Would you please send me the copies, my address is: Han-Fei Lin School of Chemical Engineering Purdue University West Lafayette, IN 47906 Thank you! From smk at flash.bellcore.com Fri Feb 17 09:45:33 1989 From: smk at flash.bellcore.com (Selma M Kaufman) Date: Fri, 17 Feb 89 09:45:33 EST Subject: No subject Message-ID: <8902171445.AA15011@flash.bellcore.com> Subject: Preprint Available - Performance of a Stochastic Learning Microchip Performance of a Stochastic Learning Microchip Joshua Alspector, Bhusan Gupta, and Robert B. Allen We have fabricated a test chip in 2 micron CMOS that can perform supervised learning in a manner similar to the Boltzmann machine. Patterns can be presented to it at 100,000 per second. The chip learns to solve the XOR problem in a few milliseconds. We also have demonstrated the capability to do unsupervised competitive learning with it. The functions of the chip components are exam- ined and the performance is assessed. For copies contact: Selma Kaufman, smk at flash.bellcore.com From movellan at garnet.berkeley.edu Fri Feb 17 13:11:40 1989 From: movellan at garnet.berkeley.edu (movellan@garnet.berkeley.edu) Date: Fri, 17 Feb 89 10:11:40 pst Subject: Noise resistance Message-ID: <8902171811.AA26238@garnet.berkeley.edu> I am interested in ANY information regarding NOISE RESISTANCE in BP and other connectionist learning algorithms. In return I will organize the information and I will send it back to all the contributors. You may include REFERENCES (theoretical treatments, applications ...) as well as HANDS-ON EXPERIENCE (explain in detail the phenomena you encounter or the procedure you use for improving noise resistance). Please send your mail directly to (no reply command): movellan at garnet.berkeley.edu Use "noise" as subject name. Sincerely, - Javier R. Movellan. From pollack at cis.ohio-state.edu Mon Feb 20 17:01:04 1989 From: pollack at cis.ohio-state.edu (Jordan B. Pollack) Date: Mon, 20 Feb 89 17:01:04 EST Subject: worldwide circulation Message-ID: <8902202201.AA00387@toto.cis.ohio-state.edu> **DO NOT FORWARD TO ANY BBOARDS** **DO NOT FORWARD TO ANY BBOARDS** I do recall a discussion a couple of years ago (started by Paul Smolensky) of how we were going to circulate our Tech Reports FREE to our colleagues on this mailing list. But I do not recall a public discussion that postings on the CONNECTIONISTS mailing list can get placed on at least two widely broadcasted bulletin boards, COMP.AI.NEURAL-NETS and NEURON-DIGEST. So my offer of free NIPS preprints to the extended connectionist summer school mailing list, has a new life, banging around the electronic BBoards of the "Free" world, netting requests from West Germany, Australia, South Korea, etc. The number of requests has just surpassed 100. Not that I mind the unexpected hundreds of dollars in postage and copying, I just want to make folks aware of what they are getting into with offers of free tech reports by post. (My next TR will be a Postscript file available by anonymous FTP.) Jordan **DO NOT FORWARD TO ANY BBOARDS** **DO NOT FORWARD TO ANY BBOARDS** From Dave.Touretzky at B.GP.CS.CMU.EDU Mon Feb 20 21:05:54 1989 From: Dave.Touretzky at B.GP.CS.CMU.EDU (Dave.Touretzky@B.GP.CS.CMU.EDU) Date: Mon, 20 Feb 89 21:05:54 EST Subject: worldwide circulation In-Reply-To: Your message of Mon, 20 Feb 89 17:01:04 -0500. <8902202201.AA00387@toto.cis.ohio-state.edu> Message-ID: <4734.604029954@DST.BOLTZ.CS.CMU.EDU> The following are the policies on redistribution of messages from the CONNECTIONISTS list, and related issues: 1. The moderator of Neuron Digest is a subscriber to CONNECTIONISTS, and, with my permission, extracts public announcements of upcoming conferences, new tech reports, and talk abstracts for reposting to Neuron Digest, which in turn shows up on comp.ai.neural-nets. He eliminates from these messages any reference to the CONNECTIONISTS list before reposting them. He DOES NOT repost any technical discussions from this list, only public announcements to which no reply is expected. 2. If you want to be sure some announcement of yours is not redistributed to other lists, start off with a "**DO NOT FORWARD TO ANY BBOARDS**" line as Jordan Pollack has done. Your wishes will be respected. 3. Subscription to CONNECTIONISTS is restricted to people actively engaged in neural net research. Some sites maintain local redistribution lists; it is the responsibility of the local list maintainer to see that this subscription policy is adhered to. We have had no problems so far. 4. Conferences and workshops may be announced on this list AT MOST twice: once to send out a call for papers, and once to remind non-authors about the registration deadline. A flood of repetitive announcements about the same conference is not welcome here. 5. Random requests for trivial info (Does anybody know Frank Rosenblatt's net address?) should be sent to connectionists-request at cs.cmu.edu, not to the entire list. Some of our overseas subscribers pay hard cash for every kbyte of messages they receive; let's keep the noise level to a minimum. 6. To respond to the author of a message on the connectionists list, e.g., to order a copy of his or her new tech report, use the "mail" command, not the "reply" command. Otherwise you will end up sending your message to the entire list, which REALLY annoys some people. The rest of us will just laugh at you behind your back. 7. Do not EVER tell a friend about connectionists at cs.cmu.edu. Tell him or her only about connectionists-request at cs.cmu.edu. This will save your friend much public embarassment if he/she tries to subscribe. -- Dave PS: This list has grown by an order of magnitude since its creation. It now contains nearly 450 addresses, including two dozen local redistribution sites in the US, Canada, UK, Germany, Greece, Australia, and Japan. So even without any Neuron Digest rebroadcasting, announcements posted here will be widely read. PPS: a public service announcement: the collected papers of the 1988 NIPS conference have gone to the printer. The book contains 91 papers and 4 invited talk summaries. Total length is about 830 pages. Printing, binding, packing, and shipping take 5 weeks, so copies will start issuing from the Morgan Kaufmann warehouse in early April. The correct citation for papers in this volume is: D. S. Touretzky (ed.) Advances in Neural Information Processing Systems 1. San Mateo, CA: Morgan Kauffman, 1989. [Collected papers of the IEEE Conference on Neural Information Processing Systems - Natural and Synthetic, Denver, Nov. 28-Dec. 1, 1988.] From netlist at psych.Stanford.EDU Mon Feb 20 23:11:03 1989 From: netlist at psych.Stanford.EDU (Mark Gluck) Date: Mon, 20 Feb 89 20:11:03 PST Subject: (Tues. 2/21): Pierre Baldi on Space & Time in Neural Nets Message-ID: Stanford University Interdisciplinary Colloquium Series: Adaptive Networks and their Applications Feb. 21th (Tuesday, 3:30pm): ----------------------------- -- Note new room: 380-380X -- ******************************************************************************** On space and time in neural networks: the role of oscillations. PIERRE BALDI Jet Propulsion Lab 198-330 4800 Oak Grove Drive Pasadena, CA 91109 ******************************************************************************** Co-Sponsored by: Departments of Electrical Engineering (B. Widrow) and Psychology (D. Rumelhart, M. Pavel, M. Gluck), Stanford Univ. From mv10801 at uc.msc.umn.edu Tue Feb 21 16:39:29 1989 From: mv10801 at uc.msc.umn.edu (mv10801@uc.msc.umn.edu) Date: Tue, 21 Feb 89 15:39:29 CST Subject: Conf. on VISION & 3-D REPRESENTATION Message-ID: <8902212139.AA06188@uc.msc.umn.edu> Conference on VISION AND THREE-DIMENSIONAL REPRESENTATION May 24-26, 1989 University of Minnesota Minneapolis, Minnesota The appearance of the three dimensional world from images pro- jected on our two dimensional retinas is immediate, effortless, and compelling. Despite the vigor of research in vision over the past two decades, questions remain about the nature of three di- mensional representations and the use of those representations for recognition and action. What information is gathered? How is it integrated and structured? How is the information used in higher level perceptual tasks? This conference will bring togeth- er nineteen prominent scientists to address these questions from neurophysiological, psychological, and computational perspec- tives. The conference is sponsored by the Air Force Office of Scientific Research and the University of Minnesota College of Liberal Arts in cooperation with the Departments of Psychology, Computer Science, Electrical Engineering, Child Development, and the Center for Research in Learning, Perception, and Cognition. Conference Speakers and Titles: ------------------------------- Albert Yonas, Institute of Child Development, University of Minnesota "Development of Depth Perception" Leslie G. Ungerleider, NIMH Laboratory of Neuropsychology "Cortical Pathways for the Analysis of Form, Space, and Motion: Three Streams of Visual Processing" James Todd, Psychology, Brandeis University "Perception of 3D Structure from Motion" William B. Thompson, Computer Science, University of Minnesota "Analyzing Visual Motion -- Spatial Organization at Surface Boundaries" Kent Stevens, Computer Science, University of Oregon "The Reconstruction of Continuous Surfaces from Stereo Measurements and Monocular Inferences" Eric Schwartz, Neurophysiology, New York University "Binocular Representation of the Visual Field in Primate Cortex" Ken Nakayama, Smith-Kettlewell Eye Institute "Occlusion Constraints and the Encoding of Color, Form, Motion, and Depth" Jittendra Malik, Computer Science, University of California, Berkeley "Representing Constraints for Inferring 3D Scene Structure from Monocular Views" David Lowe, Computer Science, University of British Columbia "What Must We Know to Recognize Something" Margaret Livingstone, Harvard Medical School "Separate Processing of Form, Color, Movement, and Depth: Anatomy, Physiology, Art, and Illusion" Stephen Kosslyn, Psychology, Harvard University "Components of High-Level Vision" J. J. Koenderink, Rijksuniversiteit Utrecht Fysisch Laboratorium "Affine Shape from Motion" Ramesh Jain, Electrical Engineering and Computer Science, Univ. of Michigan "3D Recognition from Range Imagery" Melvin Goodale, Psychology, University of Western Ontario "Depth Cues and Distance Estimation: The Calibration of Ballistic Movements" Bruce Goldstein, Psychology, University of Pittsburgh "A Perceptual Approach to Art: Comments on the Art Exhibition" John Foley, Psychology, University of California, Santa Barbara "Binocular Space Perception" Martin Fischler, SRI International "Representation and the Scene Modeling Problem" Patrick Cavanagh, Psychology, University of Montreal "How 3D Are We?" Irving Biederman, Psychology, University of Minnesota "Viewpoint Invariant Primitives as a Basis for Human Object Recognition" An art exhibit reflecting the theme of the conference will be held at the Coffman Gallery, Coffman Memorial Union, throughout May. Registration: ------------- The conference fee is $30 ($15 for current students). This fee includes program materials, refreshments, and Wednesday's recep- tion. Conference enrollment is limited, so early registration is recommended. Location/Parking: ----------------- The conference will be held in room 3-180 Electrical Engineering and Computer Science Building, University of Minnesota, Minneapo- lis. Parking is available nearby in the Harvard Street Ramp, 216 Harvard Street SE. A map indicating building and parking loca- tions will be sent to registrants. Accommodations: --------------- A block of rooms has been reserved at the Radisson University Hotel. Rates are $68 (plus tax) for double or single occupancy. To make reservations, contact the hotel at (612) 379-8888 and refer to the program title to obtain these special rates. Reser- vations must be made by April 9. For Further Information, Contact: Program: Jo Nichols, Center for Research in Learning Perception and Cognition, (612) 625-9367 Registration: Char Greenwald, Professional Development and Conference Services, (612) 625-1520 Organizing Chairpersons: Gordon Legge, Department of Psychology, (612) 625-0846, legge at eye.psych.umn.edu Lee Zimmerman, Department of Electrical Engineering, (612) 625-8544, lzimmerm at umn-ai.umn-cs.cs.umn.edu _______________________________________________________________________ Registration Form: Please duplicate for additional registrants. 54-38LB Vision and Three Dimensional Representation May 24-26, 1989 University of Minnesota Name _______________________________________________ Address ____________________________________________ ____________________________________________ ____________________________________________ Telephone Day:_____________ Evening:______________ Position ___________________________________________ Affiliation ________________________________________ ____ I enclose $30 general registration. ____ I enclose $15 current student registration. Student I.D. number __________ ____ The above fee will be provided by the University of Minnesota Department budget number __________ Please make check or money order payable to the University of Minnesota. Mail to: Registrar Professional Development and Conference Services University of Minnesota 338 Nolte Center 315 Pillsbury Drive S.E. Minneapolis, MN 55455-0139 Registration should be received by May 15. -------------------------------------------------------------------------- From mmdf at relay.cs.net Thu Feb 23 07:20:22 1989 From: mmdf at relay.cs.net (RELAY Mail System (MMDF)) Date: Thu, 23 Feb 89 7:20:22 EST Subject: Failed mail (msg.aa02554) Message-ID: After 7 days (150 hours), your message could not be fully delivered. It failed to be received by the following address(es): connectionists at CS.CMU.EDU (host: cs.cmu.edu) (queue: smtpmx) Problems usually are due to service interruptions at the receiving machine. Less often, they are caused by the communication system. Your message follows: Received: from relay2.cs.net by RELAY.CS.NET id aa02554; 17 Feb 89 1:05 EST Received: from ncr by RELAY.CS.NET id ab17414; 17 Feb 89 1:01 EST Received: by ncr-fc.FtCollins.NCR.com on Thu, 16 Feb 89 13:16:30 MST Date: Thu, 16 Feb 89 13:16:30 MST Message-Id: <8902162016.AA08393 at ncr-fc.FtCollins.NCR.com> To: connectionists%cs.cmu.edu at ncr-sd.sandiego.ncr.com Subject: Please return me to the discussion group From: <@ncrlnk.Dayton.NCR.COM:ncr-sd!ncr-fc!avery> X-pmdf-warning: 1 format errors detected at ncrlnk.Dayton.NCR.COM I am suffering from withdrawl symptoms. I need to see at least one connectionist fight,I mean, discussion a day or I will go crazy. Please see what someone can do about getting me back on the mail group. Jim Avery NCR Microelectronics 2001 Danfield Ct Ft. Collins, Colo. 80525 (303) 223-5100 avery%ncr-fc at ncr-sd.sandiego.ncr.com From risto at CS.UCLA.EDU Thu Feb 23 17:15:22 1989 From: risto at CS.UCLA.EDU (Risto Miikkulainen) Date: Thu, 23 Feb 89 14:15:22 PST Subject: Tech Report Announcement Message-ID: <8902232215.AA24410@oahu.cs.ucla.edu> [ Please send requests to valerie at cs.ucla.edu ] A Modular Neural Network Architecture for Sequential Paraphrasing of Script-Based Stories Risto Miikkulainen and Michael G. Dyer Artificial Intelligence Laboratory Computer Science Department University of California, Los Angeles, CA 90024 Abstract We have applied sequential recurrent neural networks to a fairly high-level cognitive task, i.e. paraphrasing script-based stories. Using hierarchically organized modular subnetworks, which are trained separately and in parallel, the complexity of the task is reduced by effectively dividing it into subgoals. The system uses sequential natural language input and output, and develops its own I/O representations for the words. The representations are stored in an external global lexicon, and they are adjusted in the course of training by all four subnetworks simultaneously, according to the FGREP-method. By concatenating a unique identification with the resulting representation, an arbitrary number of instances of the same word type can be created and used in the stories. The system is able to produce a fully expanded paraphrase of the story from only a few sentences, i.e. the unmentioned events are inferred. The word instances are correctly bound to their roles, and simple plausible inferences of the variable content of the story are made in the process. From Shastri at cis.upenn.edu Sun Feb 26 20:58:00 1989 From: Shastri at cis.upenn.edu (Lokendra Shastri) Date: Sun, 26 Feb 89 20:58 EST Subject: Rules and Variables in a Connectionist Reasoning System Message-ID: <8902271350.AA07571@central.cis.upenn.edu> Technical report announcement, please send requests to glenda at cis.upenn.edu A Connectionist System for Rule Based Reasoning with Multi-Place Predicates and Variables Lokendra Shastri and Venkat Ajjanagadde Computer and Information Science Department University of Pennsylvania Philadelphia, PA 19104 MS-CIS-8906 LINC LAB 141 Abstract McCarthy has observed that the representational power of most connectionist systems is restricted to unary predicates applied to a fixed object. More recently, Fodor and Pylyshyn have made a sweeping claim that connectionist systems cannot incorporate systematicity and compositionality. These comments suggest that representing structured knowledge in a connectionist network and using this knowledge in a systematic way is considered difficult if not impossible. The work reported in this paper demonstrates that a connectionist system can not only represent structured knowledge and display systematic behavior, but it can also do so with extreme efficiency. The paper describes a connectionist system that can represent knowledge expressed as rules and facts involving multi-place predicates (i.e., n-ary relations), and draw limited, but sound, inferences based on this knowledge. The system is extremely efficient - in fact, optimal, as it draws conclusions in time proportional to the length of the proof. It is observed that representing and reasoning with structured knowledge requires a solution to the variable binding problem. A solution to this problem using a multi-phase clock is proposed. The solution allows the system to maintain and propagate an arbitrary number of variable bindings during the reasoning process. The work also identifies constraints on the structure of inferential dependencies and the nature of quantification in individual rules that are required for efficient reasoning. These constraints may eventually help in modeling the remarkable human ability of performing certain inferences with extreme efficiency. From Shastri at cis.upenn.edu Sun Feb 26 18:45:00 1989 From: Shastri at cis.upenn.edu (Lokendra Shastri) Date: Sun, 26 Feb 89 18:45 EST Subject: Rules and variables in a connectionist system Message-ID: <8902271350.AA07563@central.cis.upenn.edu> From oden at cs.wisc.edu Thu Feb 2 12:36:44 1989 From: oden at cs.wisc.edu (Greg Oden) Date: Thu, 2 Feb 89 11:36:44 CST Subject: Faculty position Message-ID: <8902021736.AA29380@ai.cs.wisc.edu> The Department of Psychology of the University of Wisconsin anticipates making the appointment of an Assistant Professor in the area of cognitive science effective August 1989. We are most interested in people working with connectionist models but welcome applications from experimental psychologists with strong research records in any area of cognitive science. To apply, send your vita and have three letters of recommendation sent to: New Personnel Committee Department of Psychology University of Wisconsin Madison, WI 53706. Applications received after March 1, 1989 may not be given full consideration. The University of Wisconsin is an Affirmative Action/Equal Opportunity Employer. From jose at tractatus.bellcore.com Thu Feb 2 13:52:25 1989 From: jose at tractatus.bellcore.com (Stephen J Hanson) Date: Thu, 2 Feb 89 13:52:25 EST Subject: NIPS CALL FOR PAPERS Message-ID: <8902021852.AA12552@tractatus.bellcore.com> CALL FOR PAPERS IEEE Conference on Neural Information Processing Systems - Natural and Synthetic - Monday, November 27 -- Thursday November 30, 1989 Denver, Colorado This is the third meeting of a high quality, relatively small, inter-disciplinary conference which brings together neuroscientists, engineers, computer scientists, cognitive scientists, physicists, and mathematicians interested in all aspects of neural processing and computation. Several days of focussed workshops will follow at a nearby ski area. Major categories and examples of subcategories for papers are the following: 1. Neuroscience: Neurobiological models of development, cellular information processing, synaptic function, learning, and memory. Studies and analyses of neurobiological systems and development of neurophysiological recording tools. 2. Architecture Design: Design and evaluation of net architectures to perform cognitive or behavioral functions and to implement conventional algorithms. Data representation; static networks and dynamic networks that can process or generate pattern sequences. 3. Learning Theory Models of learning; training paradigms for static and dynamic networks; analysis of capability, generalization, complexity, and scaling. 4. Applications: Applications to signal processing, vision, speech, motor control, robotics, knowledge representation, cognitive modelling and adaptive systems. 5. Implementation and Simulation: VLSI or optical implementations of hardware neural nets. Practical issues for simulations and simulation tools. Technical Program: Plenary, contributed, and poster sessions will be held. There will be no parallel sessions. The full text of presented papers will be published. Submission Procedures: Original research contributions are solicited, and will be refereed by experts in the respective disciplines. Authors should submit four copies of a 1000-word (or less) summary and four copies of a single-page 50-100 word abstract clearly stating their results by May 30, 1989. Indicate preference for oral or poster presentation and specify which of the above five broad categories and, if appropriate, sub- categories (for example, Learning Theory: Complexity, or Applications: Speech) best applies to your paper. Indicate presentation preference and category information at the bottom of each abstract page and after each summary. Failure to do so will delay processing of your submission. Mail submissions to Kathy Hibbard, NIPS89 Local Committee, Engineering Center, Campus Box 425, Boulder, CO, 80309-0425. DEADLINE FOR SUMMARIES ABSTRACTS IS MAY 30, 1989 From mesard at BBN.COM Fri Feb 3 14:18:51 1989 From: mesard at BBN.COM (mesard@BBN.COM) Date: Fri, 03 Feb 89 14:18:51 -0500 Subject: Post-processing of neural net output (SUMMARY) Message-ID: About a month ago, I asked for information about post-processing of output activation of a (trained or semi-trained) network solving a classification task. My specific interest was what additional information can be extracted from the output vector, and what techniques are being used to improve performance and/or adjust the classification criteria (i.e., how the output is interpreted). I've been thinking about how Signal Detection Theory (SDT; cf. Green and Swets, 1966) could be applied to NN classification systems. Three areas I am concerned about are: 1) Typically interpretation of a net's classifications ignores the cost/payoff matrix associated with the classification decision. SDT provides a way to take this into account. 2) A "point-5 threshold interpretation" of output vectors is in some sense arbitrary given (1) and because it may have developed a "bias" (predisposition) towards producing a particular response (or responses) as an artifact of its training. 3) The standard interpretation does not take into account the a priori probability (likelihood) of an input of a particular type being observed. SDT may also provide an interesting way to compare two networks. Specifically, the d' ("D-prime") measure and the ROC (receiver operating characteristic) curves which have been successfully used to analyze human decision making, may be quite useful in understanding NN behavior. --- The enclosed summary covers only responses that addressed these specific issues. (The 19 messages I received totaled 27.5K. This summary is just under 8K. I endeavored to preserve all the non-redundant information and citations.) Thanks to all who replied. -- void Wayne_Mesard(); Mesard at BBN.COM Bolt Beranek and Newman, Cambridge, MA -- Summary of citation respondents: ------- -- -------- ------------ The following two papers discuss interpretation of multi-layer perceptron outputs using probabilistic or entropy-like formulations @TECHREPORT{Bourlard88, AUTHOR = "H. Bourlard and C. J. Wellekens", YEAR = "1988", TITLE = "Links Between {M}arkov Models and Multilayer Perceptrons", INSTITUTION = "Philips Research Laboratory", MONTH = "October", NUMBER = "Manuscript M 263", ADDRESS = "Brussels, Belgium" } @INPROCEEDINGS{Golden88, AUTHOR = "R. M. Golden", TITLE = "Probabilistic Characterization of Neural Model Computations", EDITOR = "D. Anderson", BOOKTITLE = "Neural Information Processing Systems", PUBLISHER = "American Institute of Physics", YEAR = "1988", ADDRESS = "New York", PAGES = "310-316" } Geoffrey Hinton (and others) cites Hinton, G. E. (1987) "Connectionist Learning Procedures", CMU-CS-87-115 (version 2) as a review of some post-processing techniques. He said that this tech report will eventually appear in the AI journal. He also says: The central idea is that any gradient descent learning procedure works just fine if the "neural net" has a non-adaptive post processing stage which is invertible -- i.e. it must be possible to back-propagate the difference between the desired and actual outputs through the post processing. [...] The most sophisticated post-processing I know of is Herve Bourlard's use of dynamic time warping to map the output of a net onto a desired string of elements. The error is back-propagated through the best time warp to get error derivatives for the detection of the individual elements in the sequence. The paper by Kaplan and Johnson in the 1988 ICNN Proceedings addressed the problem. A couple of people Michael Jordan has done interesting work in the area of post-processing, but no citations were provided. (His work from 2-3 years ago does discuss interpretation of output when trained with "don't care"s in the target vector. I don't know if this is what they were referring to.) "Best Guess" ---- ----- This involves looking at the set of valid output vectors, V(), and the observed output, O, and interpreting O as V(i) where i minimizes |V(i) - O| . For one-unit-on-the-rest-off output vectors, this is the same thing as taking the one with the largest activation, but when classifying along multiple dimensions simultaneously, this technique may be quite useful. ---- J.E. Roberts sent me a paper by A.P. Doohovskoy called "Metatemplates," presented at ICASSP Dallas, 1987 (no, I don't know what that is). He (Roberts) suggests using "a trained or semi-trained neural net to produce one 'typical' output for each type of input class. These vectors would be saved as 'metatemplates'." Then classification can be done by comparing (via Euclidian distance or dot product) observed output vectors with the metatemplates (where the closest metatemplate wins). This is uses the information from the entire network output vector for classification. Probability Measures ----------- -------- Terry Sejnowski writes: The value of an output unit is highly correlated with the confidence of a binary categorization. In our study of predicting protein secondary structure (Qian and Sejnowski, J. Molec. Biol., 202, 865-884) we have trained a network to perform a three-way classification. Recently we have found that the real value of the output unit is highly correlated with the probability of correct classification of new, testing sequences. Thus, 25% of the sequences could be predicted correctly with 80% or greater probability even though the average performance on the training set was only 64%. The highest value among the output units is also highly correlated with the difference between the largest and second largest values. We are preparing a paper for publication on these results. --- Mark Gluck writes: In our recent JEP:General paper (Gluck & Bower, 1988) we showed how the activations could be converted to choice probabilities using an exponential ratio function. This leads to good quantitative fits to human choice performance both at asymptote and during learning. --- Tony Robinson states that the summed squared difference between the actual output vector and the relevant target vector provides a measure of the probability of belonging to each class [in a one-bit-on-others-off output set]. [See "Best Guess" above.] Confidence Measures ---------- -------- John Denker says: Yes, we've been using the activation level of the runner-up neurons to provide confidence information in our character recognizer for some time. The work was reported at the last San Diego mtg and at the last Denver mtg. --- Mike Rossen describes the speech recognition system that he and Jim Anderson are working on. The target vectors are real-valued. With each phoneme represented by several units with activation on [-1, 1]: Our retrieval method is a discretized dynamical system in which system output is fed back into the system using appropriate feedback and decay parameters. Our scoring method is based on an average activation threshold, but the number of iterations the -> system takes to reach this threshold -- the system reaction time -- -> serves as a confidence measure. [He also reports on intra-layer connections on the outputs (otherwise, he's using a vanilla feedforward net) which sounds like a groovy idea, although it seems to me that this would have pros and cons in his application.] After the feedforward network is trained, connections AMONG THE OUTPUT UNITS are trained. this "post-processing" reduces both omission and confusion errors by the system. Some preliminary results of the speech model are reported in: Rossen, M.L., Niles, L.T., Tajchman, G.N., Bush, M.A., & Anderson, J.A. (1988). Training methods for a connectionist model of CV syllable recognition. Proceedings of the Second Annual International Conference on Neural Networks, 239-246. Rossen, M.L., Niles, L.T., Tajchman, G.N., Bush, M.A., Anderson, J.A., & Blumstein, S.E. (1988). A connectionist model for consonant-vowel syllable recognition. ICASSP-88, 59-66. Improving Discriminability --------- ---------------- Ralph Linsker says: You may be interested in an issue related, but not identical, to the one you raised; namely, how can one tailor the network's response so that the output optimally discriminates among the set of input vectors, i.e. so that the output provides maximum information about what the input vector was? This is addressed in: R. Linsker, Computer 21(3)105-117 (March 1988); and in my papers in the 1987 and 1988 Denver NIPS conferences. The quantity being maximized is the Shannon information rate (from input to output), or equivalently the average mutual information between input and output. --- Dave Burr refers to D. J. Burr, "Experiments with a Connectionist Text Reader," Proc. ICNN-87, pp. IV717-IV724, San Diego, CA, June 1987. Which describes a post-processing routine which assigns a score to every word in an English dictionary by summing log compressed activations. -=-=-=-=-=-=-=-=-=- From honavar at cs.wisc.edu Fri Feb 3 14:31:37 1989 From: honavar at cs.wisc.edu (A Buggy AI Program) Date: Fri, 3 Feb 89 13:31:37 CST Subject: Workshop on Optimization and Neural Nets Message-ID: <8902031931.AA06314@goat.cs.wisc.edu> Subject: Call For Papers : Neural Nets & Optimization. CALL FOR PAPERS TENCON '89 (IEEE Region 10 Conference) SESSION ON OPTIMIZATION AND NEURAL NETWORKS November 22 -- 24, 1989 Bombay, India Under the auspices of the IEEE, the session organizers invite submission of papers for a session on "Optimization and Neural Networks". This session will focus on the interrelationship of neural networks and optimization problems. Neural networks can be seen to be related to optimization in two distinct ways: + As an adaptive neural network learns from examples, the convergence of its weights solves an optimiza- tion problem. + A large class of networks , even with constant we- ights , solves optimization problems as they settle from initial to final state. The areas of interest include but are not limited to: + Combinatorial optimization + Continuous optimization + Sensor integration ( when posed as an optimization problem) + Mean Field Annealing + Stochastic Relaxation Depending on the number and quality of the responses,this ses- sion may be split into multiple sessions, with one part focus- ing on optimizing the weight-determination process in adaptive nets,and the second one on using those nets to solve other pro blems. Prospective authors should submit two copies of an extended ab stract (not exceeding 5 pages , double spaced) of their papers to either of the organizers by March 31, 1989. Authors will be notified of acceptance or rejection by May 15,1989.Photo-ready copy of the complete paper (not exceeding 25 pages double-spa- ced) must be received by Jul 15,1989 for inclusion in the pro- ceedings which will be published by the IEEE and distributed at the symposium. Session Organizers Dr. Wesley E. Snyder / Mr. Harish P. Hiriyannaiah Dept of Electrical and Computer Engineering North Carolina State University Raleigh, NC 27695-7911, USA Telephone: (919)-737-2336 FAX: (919)-737-7382 email: {wes,harish}@ecelet.ncsu.edu -- (Internet) mcnc!ece-csc!{wes,harish} -- (UUCP) -- From djb at flash.bellcore.com Fri Feb 3 17:16:13 1989 From: djb at flash.bellcore.com (David J Burr) Date: Fri, 3 Feb 89 17:16:13 EST Subject: Post-processing of neural net output Message-ID: <8902032216.AA03217@flash.bellcore.com> The following soon-to-appear paper uses signal detection theory (Green and Swets, 1966) to compare various methods for vowel recognition. Among the methods compared are linear prediction, weighted cepstra and neural nets, with both bark scale and linear frequency normalization. Requests should be emailed to: cak at bellcore.com C. A. Kamm, L. A. Streeter, Y. Kane-Esrig, and D. J. Burr, "Comparing Performance of Spectral Distance Measures and Neural Network Methods for Vowel Recognition," Computers, Speech and Language (to appear). From harnad at Princeton.EDU Sun Feb 5 13:00:12 1989 From: harnad at Princeton.EDU (Stevan Harnad) Date: Sun, 5 Feb 89 13:00:12 EST Subject: Feature Detection, Symbolic Rules and Connectionism Message-ID: <8902051800.AA01398@dictus.Princeton.EDU> I am redirecting to connectionists a segment of an ongoing discussion of categorization on comp.ai that seems to have taken a connectionistic turn. I think it will all be understandable from context. The issue concerns whether category representations are "nonclassical" (i.e., with membership a matter of degree, and no features that provide necessary and sufficient conditions for assigning membership) or "classical" (i.e., with all-or-none membership, assigned on the basis of features that do provide necessary and sufficient conditions). I am arguing against the former and for the latter. Connectionism seems to have slipped in as a way of having features yet not-having them too, so to speak, and the discussion has touched base with the familiar question of whether or not connectionist representations are really representational or ruleful: anwst at cisunx.UUCP (Anders N. Weinstein) of Univ. of Pittsburgh, Comp & Info Sys wrote: " I think Harnad errs... that reliable categorization *must* be " interestingly describable as application of some (perhaps complex) rule " in "featurese" (for some appropriate set of detectable features)... " Limiting ourselves (as I think we must) to quick and automatic " observational classification... If... the effects of context on such tasks " are minimal... there must be within us some isolable module which can " take sensory input and produce a one bit yes-or-no output for category " membership... But how does it follow that such a device must be " describable as applying some *rule*? Any physical object in the world " could be treated as a recognition device for something by interpreting " some of its states as "inputs" and some as "yes-or-no responses." But " intuitively, it looks like not every such machine is usefully described " as applying a rule in this way. In particular, this certainly doesn't " seem a natural way of describing connectionist pattern recognizers. So " why couldn't it turn out that there is just no simpler description of " the "rule" for certain category membership than: whatever a machine of " a certain type recognizes? For the points I have been trying to make it does not matter whether or not the internal basis for a machine's feature-detecting and categorizing success is described by us as a "rule" (though I suspect it can always be described that way). It does not even matter whether or not the internal basis consists of an explicit representation of a symbolic rule that is actually "applied" (in fact, according to my theory, such symbolic representations of categories would first have to be grounded in prior nonsymbolic representations). A connectionist feature-detector would be perfectly fine with me; I even suggest in my book that that would be a natural (and circumscribed) role for a connectionist module to play in a category representation system (if it can actually deliver the goods). To rehabilitate the "classical" view I've been trying to rescue from well over a decade of red herrings and incoherent criticism all I need to re-establish is that where there is reliable, correct, all-or-none categorization performance, there must surely exist detectable features in the input that are actually detected by the categorizing device as a ("necessary and sufficient") basis for its successful categorization performance. I think this should be self-evident to anyone who is mindful of the obvious facts about our categorization performance capacity and is not in the grip of a California theory (and does not believe in magic). The so-called "classical" view is only that features must EXIST in the inputs that we are manifestly able to sort and label, and that these features are actually DETECTED and USED to generate our successful performance. The classical view is not committed to internal representations of rules symbolically describing the features in "featurese" or operating on symbolic descriptions of features. That's another issue. (According to my own theory, symbolic "featurese" itself, like all abstract category labels in the "language of thought," must first be grounded in nonsymbolic, sensory categories and their nonsymbolic, sensory features.) [By the way, I don't think there's really a problem with sorting out which devices are actually categorizing and which ones aren't. Do you, really? That sounds like a philosopher's problem only. (If what you're worried about is whether the categorizer really has a mind, then apply my Total Turing Test -- require it to have ALL of our robotic and linguistic capacities.) Nor does "whatever a machine of a certain type recognizes" sound like a satisfactory answer to the "question of how in fact our neural machinery functions to enable us to so classify things." You have to say what features it detects, and HOW.] [Related to the last point, Greg Lee (lee at uhccux.uhcc.hawaii.edu), University of Hawaii, had added, concerning connectionist feature-detectors: "If you don't understand how the machine works, how can you give a rule?" I agree that the actual workings of connectionist black boxes need more analysis, but to a first approximation the answer to the question of how they work (if and when they work) is: "they learn features by sampling inputs, with feedback about miscategorization, `using' back-prop and the delta rule." And that's certainly a lot better than nothing. A fuller analysis would require specifying what features they're detecting, and how they arrived at them on the available data, as constrained by back-prop and the delta rule. There's no need whatsoever for any rules to be explicitly "represented" in order to account fully for their success, however. -- In any case, connectionist black boxes apparently do not settle the classical/nonclassical matter one way or the other, as evidenced by the fact that there seems to be ample room for them in both nonclassical approaches (e.g., Lakoff's) and classical ones (e.g., mine).] " [We must distinguish] the normative question of which things are " *correctly* classified as birds or even numbers, and the descriptive " question of how in fact our neural machinery functions to enable us to " so classify things. I agree also with Harnad that psychology ought to " keep its focus on the latter and not the former of these questions. A kind of "correctness" factor does figure in the second question too: To model how people categorize things we have to have data on what inputs they categorize as members of what categories, according to what constraints on MIScategorization. However, it's certainly not an ontological correctness that's at issue, i.e., we're not concerned with what the things people categorize really ARE "sub specie aeternitatis": We're just concerned with what people get provisionally right and wrong, under the constraints of the sample they've encountered so far and the feedback they've so far received from the consequences of miscategorization. I also see no reason to limit our discussion to "quick, automatic, observational" categorization; it applies just as much to slow perceptual pattern learning and, with proper grounding, to abstract, nonperceptual categorization too (although here is where explicitly represented symbolic rules [in "featurese"?] do play more of a role, according to my grounding theory). And I think context effects are rarely "minimal": All categorization is provisional and approximate, dependent on the context of confusable alternatives so far sampled, and the consequences (so far) of miscategorizing them. Stevan Harnad harnad at confidence.princeton.edu harnad at pucc.bitnet From pablo at june.cs.washington.edu Tue Feb 7 18:59:55 1989 From: pablo at june.cs.washington.edu (David Cohn) Date: Tue, 7 Feb 89 15:59:55 PST Subject: Ummm... a request Message-ID: <8902072359.AA12054@june.cs.washington.edu> First off, could someone please add me to the mailing list? (Thanks) With that out of the way: I'm with the Computer Sci. Dept. at U. Washington. My research interests are in formal learning theory, specifically how it can be applied to the study of the capabilities and limitations of neural networks. Research in our department is rather heavily weighted on the computational complexity side of the problem with relatively little interest (other than myself) on the neural network side. With the blessings of my advisor, Dr. Richard Ladner, I'm interested in spending the summer attempting to "cross-pollinate" some ideas in learning and neural networks. I would like to be able to work with some research group apart from UW to learn their approaches to the problems and hopefully contribute some of my own. Could anyone refer me to some likely targets? I actually have quite a bit of experience hacking neural networks, but am looking for a research group studying more theoretical aspects of the problem of neural net learning. (I can send detailed background, references and c.v. to anyone interested.) Thanks, David "Pablo" Cohn (pablo at cs.washington.edu) Dept. of Comp. Sci., FR-35 University of Washington (206) 543-7798 days Seattle, WA 98195 From netlist at psych.Stanford.EDU Wed Feb 8 10:06:25 1989 From: netlist at psych.Stanford.EDU (Mark Gluck) Date: Wed, 8 Feb 89 07:06:25 PST Subject: (Thurs. 2/9): Carver Mead on Neural VLSI Message-ID: Stanford University Interdisciplinary Colloquium Series: Adaptive Networks and their Applications Feb. 9th (Thursday, 3:30pm): ----------------------------- -- Note new room: 380-380X -- ******************************************************************************** VLSI Models of Neural Networks CARVER MEAD Moore Professor of Computer Science Calif. Inst. of Technology Pasadena, CA 91125 (818) 356 -6841 ******************************************************************************** Abstract Semiconductor technology has evolved to the point where chips containing a million transistors can be fabricated without defects. If a small number of defects can be tolerated, this number is increased by two orders of magnitude. Devices now being fabricated on an experimental basis have shown that another two orders of magnitude are possible. The inescapable conclusion is that wafers containing 10**10 devices, of which only a vanishing fraction are defective, will be in production within a few years. This level of complexity is well below that required for higher cortical functions, but is already sufficient to solve lower level perception tasks. This remarkable technology has made possible a new discipline: Synthetic Neurobiology. The thesis of this discipline is that it is not possible, even in principle, to claim a full understanding of a system unless one is able to build one that functions properly. This principle is already well accepted in mollecular biology, and more recently in genetics. It is hoped that the approach will soon join the traditional descriptive and analytical foundations of neurobiology. Small examples using current technology to attack problems in early vision and hearing will be described. Additional Information ---------------------- Location: Room 380-380X, which can be reached through the lower level between the Psychology and Mathematical Sciences buildings. Technical Level: These talks will be technically oriented and are intended for persons actively working in related areas. They are not intended for the newcomer seeking general introductory material. Mailing lists: To be added to the network mailing list, netmail to netlist at psych.stanford.edu. For additional information, or contact Mark Gluck (gluck at psych.stanford.edu). Co-Sponsored by: Departments of Electrical Engineering (B. Widrow) and Psychology (D. Rumelhart, M. Pavel, M. Gluck), Stanford Univ. From rsun at cs.brandeis.edu Tue Feb 7 11:14:34 1989 From: rsun at cs.brandeis.edu (Ron Sun) Date: Tue, 7 Feb 89 11:14:34 est Subject: No subject Message-ID: I am currently doing research on modeling biological neural networks according to the accurately identified connectivity patterns found by biologists. I will appreciate any pointers to the published papers, ongoing research or just philosophical thoughts on the subject, esp. regarding the following questions: 1) which model (existing or to be invented) can best describe real neural networks? 2) what are the criteria for measuring accuracy of a model in terms of its emergent behavior and network dynamics? 3) In which level of abstraction, should wee try to model biological neural networks in order to advance our understanding of neural networks in general? Please send response to rsun%cs.brandeis.edu at relay.cs.net Ron Sun Brandeis University CS Dept Waltham, MA 02254 From harish at ecelet.ncsu.edu Wed Feb 8 17:11:50 1989 From: harish at ecelet.ncsu.edu (Harish Hiriyannaiah) Date: Wed, 8 Feb 89 17:11:50 EST Subject: Call for Papers - TENCON '89. Message-ID: <8902082211.AA01004@ecelet.ncsu.edu> CALL FOR PAPERS TENCON '89 (IEEE Region 10 Conference) SESSION ON OPTIMIZATION AND NEURAL NETWORKS November 22 -- 24, 1989 Bombay, India Under the auspices of the IEEE, the session organizers invite submission of papers for a session on "Optimization and Neural Networks". This session will focus on the interrelationship of neural networks and optimization problems. Neural networks can be seen to be related to optimization in two distinct ways: + As an adaptive neural network learns from examples, the convergence of its weights solves an optimiza- tion problem. + A large class of networks , even with constant we- ights , solves optimization problems as they settle from initial to final state. The areas of interest include but are not limited to: + Combinatorial optimization + Continuous optimization + Sensor integration ( when posed as an optimization problem) + Mean Field Annealing + Stochastic Relaxation Depending on the number and quality of the responses,this ses- sion may be split into multiple sessions, with one part focus- ing on optimizing the weight-determination process in adaptive nets,and the second one on using those nets to solve other pro blems. Prospective authors should submit two copies of an extended ab stract (not exceeding 5 pages , double spaced) of their papers to either of the organizers by March 31, 1989. Authors will be notified of acceptance or rejection by May 15,1989.Photo-ready copy of the complete paper (not exceeding 25 pages double-spa- ced) must be received by Jul 15,1989 for inclusion in the pro- ceedings which will be published by the IEEE and distributed at the symposium. Session Organizers Dr. Wesley E. Snyder / Mr. Harish P. Hiriyannaiah Dept of Electrical and Computer Engineering North Carolina State University Raleigh, NC 27695-7911, USA Telephone: (919)-737-2336 FAX: (919)-737-7382 email: {wes,harish}@ecelet.ncsu.edu -- (Internet) mcnc!ece-csc!{wes,harish} -- (UUCP) From hlogan at watdcs.UWaterloo.ca Thu Feb 9 11:37:10 1989 From: hlogan at watdcs.UWaterloo.ca (Harry M. Logan) Date: Thu, 9 Feb 89 11:37:10 EST Subject: No subject Message-ID: Dear list owner, I should be grateful if you can add my name to the list of subscribers of Connectionists. My name is Harry M. Logan, and the e-mail address is: hlogan at watdcs.UWaterloo.ca Thank you for your consideration of this matter. Sincerely yours, Harry Logan From mel at cougar.ccsr.uiuc.edu Thu Feb 9 13:26:34 1989 From: mel at cougar.ccsr.uiuc.edu (Bartlett Mel) Date: Thu, 9 Feb 89 12:26:34 CST Subject: thesis/tech report Message-ID: <8902091826.AA24151@cougar.ccsrsun> The following thesis/TR is now available--about 50% of it is dedicated to relations to traditional methods in robotics, and to psychological and biological issues... MURPHY: A Neurally-Inspired Connectionist Approach to Learning and Performance in Vision-Based Robot Motion Planning Bartlett W. Mel Center for Complex Systems Research Beckman Institute, University of Illinois Many aspects of intelligent animal behavior require an understanding of the complex spatial relationships between the body and its parts and the coordinate systems of the external world. This thesis deals specifically with the problem of guiding a multi-link arm to a visual target in the presence of obstacles. A simple vision-based kinematic controller and motion planner based on a connectionist network architecture has been developed, called MURPHY. The physical setup consists of a video camera and a Rhino XR-3 robot arm with three joints that move in the image plane of the camera. We assume no a priori model of arm kinematics or of the imaging characteristics of the camera/visual system, and no sophisticated built-in algorithms for obstacle avoidance. Instead, MURPHY builds a model of his arm through a combination of physical and ``mental'' practice, and then uses simple heuristic search with mental images of his arm to solve visually-guided reaching problems in the presence of obstacles whose traditional algorithmic solutions are extremely complex. MURPHY differs from previous approaches to robot motion-planning primarily in his use of an explicit full-visual-field representation of the workspace. Several other aspects of MURPHY's design are unusual, including the sigma-pi synaptic learning rule, the teacherless training paradigm, and the integration of sequential control within an otherwise connectionist architecture. In concluding sections we outline a series of strong correspondences between the representations and algorithms used by MURPHY, and the psychology, physiology, and neural bases for the programming and control of directed, voluntary arm movements in humans and animals. You can write to me: mel at complex.ccsr.uiuc.edu, or judi jr at complex.ccsr.uiuc.edu. Out computers go down on Feb. 13 for 2 days, so if you want one then, call (217)244-4250 instead. -Bartlett Mel From jose at tractatus.bellcore.com Thu Feb 9 13:16:17 1989 From: jose at tractatus.bellcore.com (Stephen J Hanson) Date: Thu, 9 Feb 89 13:16:17 EST Subject: NIPS latex version PLEASE FORMAT, PRINT and POST Message-ID: <8902091816.AA06682@tractatus.bellcore.com> \documentstyle[11pt]{article} %% set sizes to fill page with small margins \setlength{\headheight}{0in} \setlength{\headsep}{0in} \setlength{\topmargin}{-0.25in} \setlength{\textwidth}{6.5in} \setlength{\textheight}{9.5in} \setlength{\oddsidemargin}{0.0in} \setlength{\evensidemargin}{0.0in} \setlength{\footheight}{0.0in} \setlength{\footskip}{0.25in} \begin{document} \pagestyle{empty} \Huge \begin{center} {\bf CALL FOR PAPERS\\} \Large IEEE Conference on\\ \LARGE {\bf Neural Information Processing Systems\\ - Natural and Synthetic -\\} \bigskip \Large Monday, November 27 -- Thursday November 30, 1989\\ Denver, Colorado\\ \end{center} \medskip \large \noindent This is the third meeting of a high quality, relatively small, inter-disciplinary conference which brings together neuroscientists, engineers, computer scientists, cognitive scientists, physicists, and mathematicians interested in all aspects of neural processing and computation. Several days of focussed workshops will follow at a nearby ski area. Major categories and examples of subcategories for papers are the following: \begin{quote} \small \begin{description} \item[{\bf 1. Neuroscience:}] Neurobiological models of development, cellular information processing, synaptic function, learning, and memory. Studies and analyses of neurobiological systems and development of neurophysiological recording tools. \item[{\bf 2. Architecture Design:}] Design and evaluation of net architectures to perform cognitive or behavioral functions and to implement conventional algorithms. Data representation; static networks and dynamic networks that can process or generate pattern sequences. \item[{\bf 3. Learning Theory:}] Models of learning; training paradigms for static and dynamic networks; analysis of capability, generalization, complexity, and scaling. \item[{\bf 4. Applications:}] Applications to signal processing, vision, speech, motor control, robotics, knowledge representation, cognitive modelling and adaptive systems. \item[{\bf 5. Implementation and Simulation:}] VLSI or optical implementations of hardware neural nets. Practical issues for simulations and simulation tools. \end{description} \end{quote} \large \smallskip \noindent {\bf Technical Program:} Plenary, contributed, and poster sessions will be held. There will be no parallel sessions. The full text of presented papers will be published. \medskip \noindent {\bf Submission Procedures:} Original research contributions are solicited, and will be refereed by experts in the respective disciplines. Authors should submit four copies of a 1000-word (or less) summary and four copies of a single-page 50-100 word abstract clearly stating their results by May 30, 1989. Indicate preference for oral or poster presentation and specify which of the above five broad categories and, if appropriate, sub-categories (for example, {\em Learning Theory: Complexity}, or {\em Applications: Speech}) best applies to your paper. Indicate presentation preference and category information at the bottom of each abstract page and after each summary. Failure to do so will delay processing of your submission. Mail submissions to Kathie Hibbard, NIPS89 Local Committee, Engineering Center, Campus Box 425, Boulder, CO, 80309-0425. \medskip \noindent {\bf Organizing Committee}\\ \small \noindent {Scott Kirkpatrick, IBM Research, General Chairman; Richard Lippmann, MIT Lincoln Labs, Program Chairman; Kristina Johnson, University of Colorado, Treasurer; Stephen J. Hanson, Bellcore, Publicity Chairman; David S. Touretzky, Carnegie-Mellon, Publications Chairman; Kathie Hibbard, University of Colorado, Local Arrangements; Alex Waibel, Carnegie-Mellon, Workshop Chairman; Howard Wachtel, University of Colorado, Workshop Local Arrangements; Edward C. Posner, Caltech, IEEE Liaison; James Bower, Caltech, Neurosciences Liaison; Larry Jackel, AT\&T Bell Labs, APS Liaison} \begin{center} \large {\bf DEADLINE FOR SUMMARIES \& ABSTRACTS IS MAY 30, 1989}\\ \end{center} \begin{flushright} Please Post \end{flushright} \end{document} From KINSELLAJ at vax1.nihel.ie Thu Feb 9 12:35:00 1989 From: KINSELLAJ at vax1.nihel.ie (KINSELLAJ@vax1.nihel.ie) Date: Thu, 9 Feb 89 17:35 GMT Subject: Identity Mappings Message-ID: John A. Kinsella Mathematics Dept., University of Limerick, Limerick, IRELAND KINSELLAJ at VAX1.NIHEL.IE The strategy "identity mapping", namely training a feedforward network to reproduce its input was (to the best of my knowledge) suggested by Geoffrey Hinton and applied in a paper by J.L. Elman & D. Zipser "Learning the hidden structure of speech". It is not clear to me, however, that this approach can do more than aid in the selection of the salient features of the data set. In other words what use is a network which has been trained as an identity mapping on (say) a vision problem? Certainly one can "strip off" the output layer & weights and by a simple piece of linear algebra determine the appropriate weights to transform the hidden layer states into output states corresponding to the salient features mentioned above. It would appear, though, that this is almost as expensive a procedure computationally as training the network as well as being numerically unstable with respect to the subset of the training set selected for the purpose. I would appreciate any comments on these remarks and in particular references to relevant publised material, John Kinsella From kanderso at DINO.BBN.COM Fri Feb 10 15:52:34 1989 From: kanderso at DINO.BBN.COM (kanderso@DINO.BBN.COM) Date: Fri, 10 Feb 89 15:52:34 -0500 Subject: Weight Decay In-Reply-To: Your message of Wed, 25 Jan 89 15:13:58 -0500. <8901252012.AA00971@neural.UUCP> Message-ID: To: att!cs.cmu.edu!connectionists Subject: Re: Weight Decay Reply-To: yann at neural.att.com Date: Wed, 25 Jan 89 15:13:58 -0500 From: Yann le Cun Consider a single layer linear network with N inputs. When the number of training pattern is smaller than N , the set of solutions (in weight space) is a proper linear subspace. adding weight decay will select the minimum norm solution in this subspace (if the weight decay coefficient is decreased with time). The minimum norm solution happens to be the solution given by the pseudo-inverse technique (cf Kohonen), and the solution which optimally cancels out uncorrelated zero mean additive noise on the input. - Yann Le Cun I think this needs some clarification. Your linear network problem is Aw = d, where A is an N x M matrix of input patterns, w is an M x 1 vector of weights, and d is an Nx1 vector of outputs. In the case you described, N < M, and w is underdetermined, ie there are many solutions. The pseudoinverse solution, w, is the one of all solutions that mimimizes |w|^2, ie any other solution will be longer. In the case where N > M and A is full rank, the pseudo-inverse minimizes |d - Aw|^2, ie it is the least squares solution. In the general case, where A is not full rank, the pseudoinverse solution minizes both (1) |d - Aw|^2 and (2) |w|^2. In an iterative network application, a learning step typically minimizes (1) while adding weight decay minimizes (2) at the same time. Another way to say this is that it trys to find a w that minimizes the error subject to the constraint that w is bounded to some length. That length is determined by the weight decay coefficient you use. In general, it would seem wrong to let the weight decay coefficient go to zero, since then you will wind up at the least squares solution which may not be what you want. k From brp at sim.berkeley.edu Fri Feb 10 19:53:47 1989 From: brp at sim.berkeley.edu (bruce raoul parnas) Date: Fri, 10 Feb 89 16:53:47 PST Subject: mailing list Message-ID: <8902110053.AA20168@sim.berkeley.edu> hi, i would like to be placed on the connectionist neural nets mailing list that you distribute. thanx, Bruce Parnas brp at sim.berkeley.edu From hinton at ai.toronto.edu Fri Feb 10 22:49:34 1989 From: hinton at ai.toronto.edu (Geoffrey Hinton) Date: Fri, 10 Feb 89 22:49:34 EST Subject: Identity Mappings In-Reply-To: Your message of Thu, 09 Feb 89 12:35:00 -0500. Message-ID: <89Feb10.224948est.10802@ephemeral.ai.toronto.edu> The potential advantage of using "encoder" networks is that the code in the middle can be developed without any supervision. If the output and hidden units are non-linear, the codes do NOT just span the same subspace as the principal components. The difference between a linear approach like principal components and a non-linear approach is especially significant if there is more than one hidden layer. If the codes from several encoder networks are then used as the input vector for a "higher level" network, one can get a multilayer, modular, unsupervised learning procedure that should scale up better to really large problems. Ballard (AAAI proceedings, 1987) has investigated this approach for a simple problem and has introduced the interesting idea that as the learning proceeds, the central code of each encoder module should give greater weight to the error feedback coming from higher level modules that use this code as input and less weight to the error feedback coming from the output of the code's own module. However, to the best of my knowledge, nobody has yet shown that it really works well for a hard task. One problem, pointed out by Steve Nowlan, is that the codes formed in a bottleneck tend to "encrypt" the information in a compact form that is not necessarily helpful for further processing. It may be worth exploring encoders in nets with many hidden layers that are given inputs from real domains, but my own current view is that to achieve modular unsupervised learning we probably need to optimize some other function which does not simply ensure good reconstruction of the input vector. Geoff Hinton From Zipser%cogsci at ucsd.edu Sat Feb 11 13:49:00 1989 From: Zipser%cogsci at ucsd.edu (Zipser%cogsci@ucsd.edu) Date: Sat, 11 Feb 89 10:49 PST Subject: Identity Mappings In-Reply-To: <89Feb10.224948est.10802@ephemeral.ai.toronto.edu> Message-ID: <890211104950.2.ZIPSER@BUGS.ucsd> Geoff, Perhaps of interest is that in our work with identity mapping of speech, the hidden layer spontaneously learned to represent vowels and consonants in separate groups of units. Within these groups the individual sounds seemed quite compactly coded. Maybe the ease with which we are able to identify the distinct features used to recognize whole items depends on the kind of coding they have in our hidden layers. David Zipser From ST401843%BROWNVM.BITNET at VMA.CC.CMU.EDU Mon Feb 13 00:47:00 1989 From: ST401843%BROWNVM.BITNET at VMA.CC.CMU.EDU (thanasis kehagias) Date: Mon, 13 Feb 89 00:47:00 EST Subject: Connection between Hidden Markov Models and Connectionist Networks Message-ID: the following paper explores the connection between Hidden Markov Models and Connectionist networks. anybody interested in a copy, email me. if you have a TeX setup i will send you the dvi file. else give me your physical mail address. OPTIMAL CONTROL FOR TRAINING THE MISSING LINK BETWEEN HIDDEN MARKOV MODELS AND CONNECTIONIST NETWORKS by Athanasios Kehagias Division of Applied Mathematics Brown University Providence, RI 02912 ABSTRACT For every Hidden Markov Modl there is a set of forward probabilities that need to be computed for both the recognition and training problem . These probabilties are computed recursively and hence the computation can be performed by a multistage , feedforward network that we will call Hidden Markov Model Network (HMMN). This network has exactly the same architecture as the standard Connectionist Network(CN). Furthermore, training a Hidden Markov Model is equivalent to optimizing a function of the HMMN; training a CN is equivalent to minimizing a function of the CN. Due to the multistage feedforward architecture, both problems can be seen as Optimal Control problems. By applying standard Optimal Control techniques, we discover in both problems that certain back propagating quantities (backward probabilities for HMMN, backward propogated errors for CN) are of crucial importance for the solution. So HMMN's and CN's are similar both in architecture and training. ************** i was influenced in this research by the work of H. Bourlard and C. C. Wellekens (the HMM- CN connection) and Y. leCun (Optimal Control applications in CN's). as i was finishing my aper i received a message by J.N. Hwang saying that he and S.Y. Kung have written a paper that includes similar results. Thanasis Kehagias From alexis%yummy at gateway.mitre.org Mon Feb 13 08:43:05 1989 From: alexis%yummy at gateway.mitre.org (alexis%yummy@gateway.mitre.org) Date: Mon, 13 Feb 89 08:43:05 EST Subject: Job Opportunity at MITRE Message-ID: <8902131343.AA02002@marzipan.mitre.org> The MITRE Corporation is looking for technical staff for their expanding neural network effort. MITRE's neural network program currently includes both IR&D and sponsored work in areas ranging from performance analysis, learning algorithms, pattern recognition, and simulation/implementation. The ideal candidate would have the following qualifications: 1. 2-4 years experience in the area of neural networks. 2. Strong background in traditional signal processing with an emphasis on detection and classification theory. 3. Experienced programmer in C/Unix. Experience in graphics (X11/NeWS), scientific programming, symbolic programming, and fast hardware (array and parallel processors) are pluses. 4. US citizenship required. Interested canidates should send resumes to: Garry Jacyna The MITRE Corporation M.S. Z406 7525 Colshire Drive McLean, Va. 22102 USA From postmast at watdcs.UWaterloo.ca Mon Feb 13 13:05:25 1989 From: postmast at watdcs.UWaterloo.ca (DCS Postmaster) Date: Mon, 13 Feb 89 13:05:25 EST Subject: Subscription Request Message-ID: POSTMASTer is sending this request on behalf of one of our users who seems to be having trouble getting mail through to the list-owner. Please consider the following mail, sent on behalf of Prof. Logan. thank you .. walter mccutchan Deptartment of Computer Services Postmaster University of Waterloo, Waterloo, Ontarion -------- Dear list owner, I should be grateful if you can add my name in the list of suscriptors of Connectionists. My name is Harry M. Logan, and my e-mail address is: Thank you for your consideration in this matter. Sincerely yours, Harry Logan From ROB%BGERUG51.BITNET at VMA.CC.CMU.EDU Mon Feb 13 12:54:00 1989 From: ROB%BGERUG51.BITNET at VMA.CC.CMU.EDU (Rob A. Vingerhoeds / Ghent State University) Date: Mon, 13 Feb 89 12:54 N Subject: Neural Networks Seminar, 25 april 1989, Ghent, Belgium Message-ID: BIRA SEMINAR ON NEURAL NETWORKS 25 APRIL 1989 International Congress Centre Ghent BELGIUM BIRA (Belgian Institute for Control Engineering and Automation) is organising a seminar on the state of the art in Neural Networks. The central theme will be "When and how will neural networks become applicable for industry". To be able to give a good and reliable verdict to this theme, some of the most important and leading scientists in this fasci- nating area have been invited to present a lecture at the seminar and take part in a panel discussion. The following schedule is foreseen: 8.30 - 9.00 Registration 9.00 - 9.15 Opening on behalf of BIRA Prof. L. Boullart Ghent State University 9.15 - 10.00 Introduction to the domain Prof. Fogelman Soulie Universite de Paris V 10.00 - 10.30 coffee 10.30 - 11.30 Theoretical Backgrounds and Mathematical Models Prof. B. Kosko University of Southern California 11.30 - 12.00 Special dedicated hardware (probably the French representative of Hecht-Nielsen Neurocomputers) 12.00 - 14.00 lunch / exhibition 14.00 - 15.00 Application in Robotics Dr. David Handelman Princeton 15.00 - 16.00 Application in Image Processing and Pattern Recognition (Neocognitron) Dr. S. Miyake ATR 16.00 - 16.30 tea 16.30 - 17.15 panel discussion over the central theme 17.15 - 17.30 closing and conclusions The seminar will be held in the same period as the famous Flanders Technology International (F.T.I.) exhibition is held. This exhibition is for both representatives from industry and for other interested people very interesting and going to both the seminar and the exhibition is double interesting. It is possible to obtain a ticket for F.T.I. at a reduced price, when attending the seminar. Please indicate, whether you would like to get a ticket, when sending in a letter or an e-mail message. Prices: members of BIRA : 12.500 BEF others : 15.000 BEF universities : 7.500 BEF If you intent to attent our seminar, you can either send a letter to the BIRA coordinator (adress follows) or an e-mail message to one of us. We will fill you in on the details as soon as possible. Rob Vingerhoeds Leo Vercauteren BIRA Coordinator: L. Pauwels BIRA-secretariaat Het Ingenieurshuis Desguinlei 214 2018 Antwerpen Belgium telefax: +32-3-216-06-89 (attn. BIRA L. Pauwels) From jose at tractatus.bellcore.com Tue Feb 14 17:22:22 1989 From: jose at tractatus.bellcore.com (Stephen J Hanson) Date: Tue, 14 Feb 89 17:22:22 EST Subject: NIPS POST-MEETING WORKSHOPS Message-ID: <8902142222.AA12893@tractatus.bellcore.com> NIPS-89 POST-CONFERENCE WORKSHOPS DECEMBER 1-2, 1989 REQUEST FOR PROPOSALS Following the regular NIPS program, workshops on current topics in Neural Information Processing will be held on December 1 and 2, 1989, at a ski resort near Denver. Proposals by qualified individuals interested in chairing one of these workshops are solicited. Past topics have included: Rules and Connectionist Models; Speech, Neural Networks and Hidden Markov Models; Imaging Techniques in Neurobiology; Computational Complexity Issues; Fault Tolerance in Neural Networks; Benchmarking and Comparing Neural Network Applications; Architectural Issues; Fast Training Techniques. The format of the workshops is informal. Beyond reporting on past research, their goal is to provide a forum for scientists actively working in the field to freely discuss current issues of concern and interest. Sessions will meet in the morning and in the afternoon of both days, with free time in between for ongoing individual exchange or outdoor activities. Specific open and/or controversial issues are encouraged and preferred as workshop topics. Individuals interested in chairing a workshop must propose a topic of current interest and must be willing to accept responsibility for their group's discussion. Discussion leaders' responsibilities include: arrange brief informal presentations by experts working on this topic, moderate or lead the discussion; and report its high points, findings and conclusions to the group during evening plenary sessions. Submission Procedure: Interested parties should submit a short proposal for a workshop of interest by May 30, 1989. Proposals should include a title and a short description of what the workshop is to address and accomplish. It should state why the topic is of interest or controversial, why it should be discussed and what the targeted group of participants is. In addition, please send a brief resume of the prospective workshop chair, list of publications and evidence of scholarship in the field of interest. Mail submissions to: Kathie Hibbard NIPS89 Local Committee Engineering Center Campus Box 425 Boulder, CO, 80309-0425 Name, mailing address, phone number, and e-mail net address (if applicable) should be on all submissions. Workshop Organizing Committee: Alex Waibel, Carnegie-Mellon, Workshop Chairman; Howard Wachtel, University of Colorado, Workshop Local Arrangements; Kathie Hibbard, University of Colorado, NIPS General Local Arrangements; PROPOSALS MUST BE RECEIVED BY MAY 30, 1989. From krulwich-bruce at YALE.ARPA Thu Feb 16 11:08:38 1989 From: krulwich-bruce at YALE.ARPA (Bruce Krulwich) Date: Thu, 16 Feb 89 11:08:38 EST Subject: Identity Mappings In-Reply-To: Geoffrey Hinton , Fri, 10 Feb 89 22:49:34 EST Message-ID: <8902161607.AA02472@ELI.CS.YALE.EDU> Geoff Hinton wrote recently: The potential advantage of using "encoder" networks is that the code in the middle can be developed without any supervision. ... If the codes from several encoder networks are then used as the input vector for a "higher level" network, one can get a multilayer, modular, unsupervised learning procedure that should scale up better to really large problems. This brings out a point I've wanted to discuss for a while: Are "encoder nets" any better than, say, competitive learning (or recirculation, or maybe GMax?) for a task such as this?? It seems to me that feed-forward I/O nets are the wrong model for learning correlations, especially if the encodings themselves are what is going to be used for further computation. More generally and to the point, could it be that the success in backprop (in applications and analysis) has resulted in stagnation by tying people to the idea of feed-forward nets?? Bruce Krulwich krulwich at cs.yale.edu ------- From linhf at ester.ecn.purdue.edu Fri Feb 17 10:27:23 1989 From: linhf at ester.ecn.purdue.edu (Han-Fei Lin) Date: Fri, 17 Feb 89 10:27:23 EST Subject: Tech Report Announcement Message-ID: <8902171527.AA00797@ester.ecn.purdue.edu> Would you please send me the copies, my address is: Han-Fei Lin School of Chemical Engineering Purdue University West Lafayette, IN 47906 Thank you! From smk at flash.bellcore.com Fri Feb 17 09:45:33 1989 From: smk at flash.bellcore.com (Selma M Kaufman) Date: Fri, 17 Feb 89 09:45:33 EST Subject: No subject Message-ID: <8902171445.AA15011@flash.bellcore.com> Subject: Preprint Available - Performance of a Stochastic Learning Microchip Performance of a Stochastic Learning Microchip Joshua Alspector, Bhusan Gupta, and Robert B. Allen We have fabricated a test chip in 2 micron CMOS that can perform supervised learning in a manner similar to the Boltzmann machine. Patterns can be presented to it at 100,000 per second. The chip learns to solve the XOR problem in a few milliseconds. We also have demonstrated the capability to do unsupervised competitive learning with it. The functions of the chip components are exam- ined and the performance is assessed. For copies contact: Selma Kaufman, smk at flash.bellcore.com From movellan at garnet.berkeley.edu Fri Feb 17 13:11:40 1989 From: movellan at garnet.berkeley.edu (movellan@garnet.berkeley.edu) Date: Fri, 17 Feb 89 10:11:40 pst Subject: Noise resistance Message-ID: <8902171811.AA26238@garnet.berkeley.edu> I am interested in ANY information regarding NOISE RESISTANCE in BP and other connectionist learning algorithms. In return I will organize the information and I will send it back to all the contributors. You may include REFERENCES (theoretical treatments, applications ...) as well as HANDS-ON EXPERIENCE (explain in detail the phenomena you encounter or the procedure you use for improving noise resistance). Please send your mail directly to (no reply command): movellan at garnet.berkeley.edu Use "noise" as subject name. Sincerely, - Javier R. Movellan. From pollack at cis.ohio-state.edu Mon Feb 20 17:01:04 1989 From: pollack at cis.ohio-state.edu (Jordan B. Pollack) Date: Mon, 20 Feb 89 17:01:04 EST Subject: worldwide circulation Message-ID: <8902202201.AA00387@toto.cis.ohio-state.edu> **DO NOT FORWARD TO ANY BBOARDS** **DO NOT FORWARD TO ANY BBOARDS** I do recall a discussion a couple of years ago (started by Paul Smolensky) of how we were going to circulate our Tech Reports FREE to our colleagues on this mailing list. But I do not recall a public discussion that postings on the CONNECTIONISTS mailing list can get placed on at least two widely broadcasted bulletin boards, COMP.AI.NEURAL-NETS and NEURON-DIGEST. So my offer of free NIPS preprints to the extended connectionist summer school mailing list, has a new life, banging around the electronic BBoards of the "Free" world, netting requests from West Germany, Australia, South Korea, etc. The number of requests has just surpassed 100. Not that I mind the unexpected hundreds of dollars in postage and copying, I just want to make folks aware of what they are getting into with offers of free tech reports by post. (My next TR will be a Postscript file available by anonymous FTP.) Jordan **DO NOT FORWARD TO ANY BBOARDS** **DO NOT FORWARD TO ANY BBOARDS** From Dave.Touretzky at B.GP.CS.CMU.EDU Mon Feb 20 21:05:54 1989 From: Dave.Touretzky at B.GP.CS.CMU.EDU (Dave.Touretzky@B.GP.CS.CMU.EDU) Date: Mon, 20 Feb 89 21:05:54 EST Subject: worldwide circulation In-Reply-To: Your message of Mon, 20 Feb 89 17:01:04 -0500. <8902202201.AA00387@toto.cis.ohio-state.edu> Message-ID: <4734.604029954@DST.BOLTZ.CS.CMU.EDU> The following are the policies on redistribution of messages from the CONNECTIONISTS list, and related issues: 1. The moderator of Neuron Digest is a subscriber to CONNECTIONISTS, and, with my permission, extracts public announcements of upcoming conferences, new tech reports, and talk abstracts for reposting to Neuron Digest, which in turn shows up on comp.ai.neural-nets. He eliminates from these messages any reference to the CONNECTIONISTS list before reposting them. He DOES NOT repost any technical discussions from this list, only public announcements to which no reply is expected. 2. If you want to be sure some announcement of yours is not redistributed to other lists, start off with a "**DO NOT FORWARD TO ANY BBOARDS**" line as Jordan Pollack has done. Your wishes will be respected. 3. Subscription to CONNECTIONISTS is restricted to people actively engaged in neural net research. Some sites maintain local redistribution lists; it is the responsibility of the local list maintainer to see that this subscription policy is adhered to. We have had no problems so far. 4. Conferences and workshops may be announced on this list AT MOST twice: once to send out a call for papers, and once to remind non-authors about the registration deadline. A flood of repetitive announcements about the same conference is not welcome here. 5. Random requests for trivial info (Does anybody know Frank Rosenblatt's net address?) should be sent to connectionists-request at cs.cmu.edu, not to the entire list. Some of our overseas subscribers pay hard cash for every kbyte of messages they receive; let's keep the noise level to a minimum. 6. To respond to the author of a message on the connectionists list, e.g., to order a copy of his or her new tech report, use the "mail" command, not the "reply" command. Otherwise you will end up sending your message to the entire list, which REALLY annoys some people. The rest of us will just laugh at you behind your back. 7. Do not EVER tell a friend about connectionists at cs.cmu.edu. Tell him or her only about connectionists-request at cs.cmu.edu. This will save your friend much public embarassment if he/she tries to subscribe. -- Dave PS: This list has grown by an order of magnitude since its creation. It now contains nearly 450 addresses, including two dozen local redistribution sites in the US, Canada, UK, Germany, Greece, Australia, and Japan. So even without any Neuron Digest rebroadcasting, announcements posted here will be widely read. PPS: a public service announcement: the collected papers of the 1988 NIPS conference have gone to the printer. The book contains 91 papers and 4 invited talk summaries. Total length is about 830 pages. Printing, binding, packing, and shipping take 5 weeks, so copies will start issuing from the Morgan Kaufmann warehouse in early April. The correct citation for papers in this volume is: D. S. Touretzky (ed.) Advances in Neural Information Processing Systems 1. San Mateo, CA: Morgan Kauffman, 1989. [Collected papers of the IEEE Conference on Neural Information Processing Systems - Natural and Synthetic, Denver, Nov. 28-Dec. 1, 1988.] From netlist at psych.Stanford.EDU Mon Feb 20 23:11:03 1989 From: netlist at psych.Stanford.EDU (Mark Gluck) Date: Mon, 20 Feb 89 20:11:03 PST Subject: (Tues. 2/21): Pierre Baldi on Space & Time in Neural Nets Message-ID: Stanford University Interdisciplinary Colloquium Series: Adaptive Networks and their Applications Feb. 21th (Tuesday, 3:30pm): ----------------------------- -- Note new room: 380-380X -- ******************************************************************************** On space and time in neural networks: the role of oscillations. PIERRE BALDI Jet Propulsion Lab 198-330 4800 Oak Grove Drive Pasadena, CA 91109 ******************************************************************************** Co-Sponsored by: Departments of Electrical Engineering (B. Widrow) and Psychology (D. Rumelhart, M. Pavel, M. Gluck), Stanford Univ. From mv10801 at uc.msc.umn.edu Tue Feb 21 16:39:29 1989 From: mv10801 at uc.msc.umn.edu (mv10801@uc.msc.umn.edu) Date: Tue, 21 Feb 89 15:39:29 CST Subject: Conf. on VISION & 3-D REPRESENTATION Message-ID: <8902212139.AA06188@uc.msc.umn.edu> Conference on VISION AND THREE-DIMENSIONAL REPRESENTATION May 24-26, 1989 University of Minnesota Minneapolis, Minnesota The appearance of the three dimensional world from images pro- jected on our two dimensional retinas is immediate, effortless, and compelling. Despite the vigor of research in vision over the past two decades, questions remain about the nature of three di- mensional representations and the use of those representations for recognition and action. What information is gathered? How is it integrated and structured? How is the information used in higher level perceptual tasks? This conference will bring togeth- er nineteen prominent scientists to address these questions from neurophysiological, psychological, and computational perspec- tives. The conference is sponsored by the Air Force Office of Scientific Research and the University of Minnesota College of Liberal Arts in cooperation with the Departments of Psychology, Computer Science, Electrical Engineering, Child Development, and the Center for Research in Learning, Perception, and Cognition. Conference Speakers and Titles: ------------------------------- Albert Yonas, Institute of Child Development, University of Minnesota "Development of Depth Perception" Leslie G. Ungerleider, NIMH Laboratory of Neuropsychology "Cortical Pathways for the Analysis of Form, Space, and Motion: Three Streams of Visual Processing" James Todd, Psychology, Brandeis University "Perception of 3D Structure from Motion" William B. Thompson, Computer Science, University of Minnesota "Analyzing Visual Motion -- Spatial Organization at Surface Boundaries" Kent Stevens, Computer Science, University of Oregon "The Reconstruction of Continuous Surfaces from Stereo Measurements and Monocular Inferences" Eric Schwartz, Neurophysiology, New York University "Binocular Representation of the Visual Field in Primate Cortex" Ken Nakayama, Smith-Kettlewell Eye Institute "Occlusion Constraints and the Encoding of Color, Form, Motion, and Depth" Jittendra Malik, Computer Science, University of California, Berkeley "Representing Constraints for Inferring 3D Scene Structure from Monocular Views" David Lowe, Computer Science, University of British Columbia "What Must We Know to Recognize Something" Margaret Livingstone, Harvard Medical School "Separate Processing of Form, Color, Movement, and Depth: Anatomy, Physiology, Art, and Illusion" Stephen Kosslyn, Psychology, Harvard University "Components of High-Level Vision" J. J. Koenderink, Rijksuniversiteit Utrecht Fysisch Laboratorium "Affine Shape from Motion" Ramesh Jain, Electrical Engineering and Computer Science, Univ. of Michigan "3D Recognition from Range Imagery" Melvin Goodale, Psychology, University of Western Ontario "Depth Cues and Distance Estimation: The Calibration of Ballistic Movements" Bruce Goldstein, Psychology, University of Pittsburgh "A Perceptual Approach to Art: Comments on the Art Exhibition" John Foley, Psychology, University of California, Santa Barbara "Binocular Space Perception" Martin Fischler, SRI International "Representation and the Scene Modeling Problem" Patrick Cavanagh, Psychology, University of Montreal "How 3D Are We?" Irving Biederman, Psychology, University of Minnesota "Viewpoint Invariant Primitives as a Basis for Human Object Recognition" An art exhibit reflecting the theme of the conference will be held at the Coffman Gallery, Coffman Memorial Union, throughout May. Registration: ------------- The conference fee is $30 ($15 for current students). This fee includes program materials, refreshments, and Wednesday's recep- tion. Conference enrollment is limited, so early registration is recommended. Location/Parking: ----------------- The conference will be held in room 3-180 Electrical Engineering and Computer Science Building, University of Minnesota, Minneapo- lis. Parking is available nearby in the Harvard Street Ramp, 216 Harvard Street SE. A map indicating building and parking loca- tions will be sent to registrants. Accommodations: --------------- A block of rooms has been reserved at the Radisson University Hotel. Rates are $68 (plus tax) for double or single occupancy. To make reservations, contact the hotel at (612) 379-8888 and refer to the program title to obtain these special rates. Reser- vations must be made by April 9. For Further Information, Contact: Program: Jo Nichols, Center for Research in Learning Perception and Cognition, (612) 625-9367 Registration: Char Greenwald, Professional Development and Conference Services, (612) 625-1520 Organizing Chairpersons: Gordon Legge, Department of Psychology, (612) 625-0846, legge at eye.psych.umn.edu Lee Zimmerman, Department of Electrical Engineering, (612) 625-8544, lzimmerm at umn-ai.umn-cs.cs.umn.edu _______________________________________________________________________ Registration Form: Please duplicate for additional registrants. 54-38LB Vision and Three Dimensional Representation May 24-26, 1989 University of Minnesota Name _______________________________________________ Address ____________________________________________ ____________________________________________ ____________________________________________ Telephone Day:_____________ Evening:______________ Position ___________________________________________ Affiliation ________________________________________ ____ I enclose $30 general registration. ____ I enclose $15 current student registration. Student I.D. number __________ ____ The above fee will be provided by the University of Minnesota Department budget number __________ Please make check or money order payable to the University of Minnesota. Mail to: Registrar Professional Development and Conference Services University of Minnesota 338 Nolte Center 315 Pillsbury Drive S.E. Minneapolis, MN 55455-0139 Registration should be received by May 15. -------------------------------------------------------------------------- From mmdf at relay.cs.net Thu Feb 23 07:20:22 1989 From: mmdf at relay.cs.net (RELAY Mail System (MMDF)) Date: Thu, 23 Feb 89 7:20:22 EST Subject: Failed mail (msg.aa02554) Message-ID: After 7 days (150 hours), your message could not be fully delivered. It failed to be received by the following address(es): connectionists at CS.CMU.EDU (host: cs.cmu.edu) (queue: smtpmx) Problems usually are due to service interruptions at the receiving machine. Less often, they are caused by the communication system. Your message follows: Received: from relay2.cs.net by RELAY.CS.NET id aa02554; 17 Feb 89 1:05 EST Received: from ncr by RELAY.CS.NET id ab17414; 17 Feb 89 1:01 EST Received: by ncr-fc.FtCollins.NCR.com on Thu, 16 Feb 89 13:16:30 MST Date: Thu, 16 Feb 89 13:16:30 MST Message-Id: <8902162016.AA08393 at ncr-fc.FtCollins.NCR.com> To: connectionists%cs.cmu.edu at ncr-sd.sandiego.ncr.com Subject: Please return me to the discussion group From: <@ncrlnk.Dayton.NCR.COM:ncr-sd!ncr-fc!avery> X-pmdf-warning: 1 format errors detected at ncrlnk.Dayton.NCR.COM I am suffering from withdrawl symptoms. I need to see at least one connectionist fight,I mean, discussion a day or I will go crazy. Please see what someone can do about getting me back on the mail group. Jim Avery NCR Microelectronics 2001 Danfield Ct Ft. Collins, Colo. 80525 (303) 223-5100 avery%ncr-fc at ncr-sd.sandiego.ncr.com From risto at CS.UCLA.EDU Thu Feb 23 17:15:22 1989 From: risto at CS.UCLA.EDU (Risto Miikkulainen) Date: Thu, 23 Feb 89 14:15:22 PST Subject: Tech Report Announcement Message-ID: <8902232215.AA24410@oahu.cs.ucla.edu> [ Please send requests to valerie at cs.ucla.edu ] A Modular Neural Network Architecture for Sequential Paraphrasing of Script-Based Stories Risto Miikkulainen and Michael G. Dyer Artificial Intelligence Laboratory Computer Science Department University of California, Los Angeles, CA 90024 Abstract We have applied sequential recurrent neural networks to a fairly high-level cognitive task, i.e. paraphrasing script-based stories. Using hierarchically organized modular subnetworks, which are trained separately and in parallel, the complexity of the task is reduced by effectively dividing it into subgoals. The system uses sequential natural language input and output, and develops its own I/O representations for the words. The representations are stored in an external global lexicon, and they are adjusted in the course of training by all four subnetworks simultaneously, according to the FGREP-method. By concatenating a unique identification with the resulting representation, an arbitrary number of instances of the same word type can be created and used in the stories. The system is able to produce a fully expanded paraphrase of the story from only a few sentences, i.e. the unmentioned events are inferred. The word instances are correctly bound to their roles, and simple plausible inferences of the variable content of the story are made in the process. From Shastri at cis.upenn.edu Sun Feb 26 20:58:00 1989 From: Shastri at cis.upenn.edu (Lokendra Shastri) Date: Sun, 26 Feb 89 20:58 EST Subject: Rules and Variables in a Connectionist Reasoning System Message-ID: <8902271350.AA07571@central.cis.upenn.edu> Technical report announcement, please send requests to glenda at cis.upenn.edu A Connectionist System for Rule Based Reasoning with Multi-Place Predicates and Variables Lokendra Shastri and Venkat Ajjanagadde Computer and Information Science Department University of Pennsylvania Philadelphia, PA 19104 MS-CIS-8906 LINC LAB 141 Abstract McCarthy has observed that the representational power of most connectionist systems is restricted to unary predicates applied to a fixed object. More recently, Fodor and Pylyshyn have made a sweeping claim that connectionist systems cannot incorporate systematicity and compositionality. These comments suggest that representing structured knowledge in a connectionist network and using this knowledge in a systematic way is considered difficult if not impossible. The work reported in this paper demonstrates that a connectionist system can not only represent structured knowledge and display systematic behavior, but it can also do so with extreme efficiency. The paper describes a connectionist system that can represent knowledge expressed as rules and facts involving multi-place predicates (i.e., n-ary relations), and draw limited, but sound, inferences based on this knowledge. The system is extremely efficient - in fact, optimal, as it draws conclusions in time proportional to the length of the proof. It is observed that representing and reasoning with structured knowledge requires a solution to the variable binding problem. A solution to this problem using a multi-phase clock is proposed. The solution allows the system to maintain and propagate an arbitrary number of variable bindings during the reasoning process. The work also identifies constraints on the structure of inferential dependencies and the nature of quantification in individual rules that are required for efficient reasoning. These constraints may eventually help in modeling the remarkable human ability of performing certain inferences with extreme efficiency. From Shastri at cis.upenn.edu Sun Feb 26 18:45:00 1989 From: Shastri at cis.upenn.edu (Lokendra Shastri) Date: Sun, 26 Feb 89 18:45 EST Subject: Rules and variables in a connectionist system Message-ID: <8902271350.AA07563@central.cis.upenn.edu>