From SAYEGH%IPFWCVAX.BITNET at VMA.CC.CMU.EDU Wed Nov 1 11:05:00 1989 From: SAYEGH%IPFWCVAX.BITNET at VMA.CC.CMU.EDU (SAYEGH%IPFWCVAX.BITNET@VMA.CC.CMU.EDU) Date: Wed, 1 Nov 89 11:05 EST Subject: NN conference. Message-ID: Second Conference on Neural Networks and Parallel Distributed Processing ------------------------------------------------------------------------ Indiana-Purdue University Fort Wayne Thursday November 9, 6 to 9 pm (Neff 101) Friday November 10, 6 to 9 pm (KT 132) Saturday November 11, 9 am to 1 pm (KT 132) Schedule: Thursday: -------- "Physics and Neural Networks" Dr. Samir Sayegh, Indiana-Purdue University. "An Engineering Introduction to Back Propagation" Dr. Allen Pugh, Indiana_Purdue University and Dr. Kirk Dunkelberger, Magnavox Electronics Company. "Computer Simulation of the Motor Nervous System of a Simple Invertebrate" Dr. Ernst Niebur, California Institute of Technology. Friday: ------ "Neural Networks in Vivo" Dr. Jeff Wilson, Indiana-Purdue University. "Designing Structured Neural Networks for Speech Recognition" Dr. Alex Waibel, Carnegie Mellon University and ATR Interpreting Telephony Research Labs (Japan). "Self Organization Applied to the Design of Neural Networks Architectures" Dr. Manoel Tenorio, Purdue University. Saturday: -------- Workshop: "Applying Connectionist Models to Speech and other Real World Signals." Dr. Alex Waibel. For more information: sayegh at ipfwcvax.bitnet sayegh at ed.ecn.purdue.edu FAX: (219) 481-6880 Voice: (219) 481-6157. From KENYON at tahoma.phys.washington.edu Mon Nov 6 17:55:00 1989 From: KENYON at tahoma.phys.washington.edu (KENYON@tahoma.phys.washington.edu) Date: Mon, 6 Nov 89 14:55 PST Subject: ROOMMATES FOR NIPS 89? Message-ID: Hello, Is anyone interested in sharing the costs of a room for the upcoming NIPS conference? Please drop me line if you are. I will try to distribute any additional responses I receive to other interested connectionist. The above applies to the workshop also. Thanks, Gar Kenyon (206) 543-7334 email kenyon at phast.phys.washington.edu From gmdzi!muehlen at uunet.UU.NET Tue Nov 7 09:55:10 1989 From: gmdzi!muehlen at uunet.UU.NET (Heinz Muehlenbein) Date: Tue, 7 Nov 89 13:55:10 -0100 Subject: CFP: Parallel Computing special issue on NNs Message-ID: <8911071255.AA22543@gmdzi.UUCP> Special issue on neural networks ----------------------------------- Dear colleagues, I am editing a special issue of the journal Parallel Computing on neural networks. The following topics will be covered ---introduction to NN ---simulation of NN's ---performance of NN's --- limitations of current NN's ---the next generation I am looking for papers describing the limitations of current NN's and/or give an outline of the next generation. In my opinion, the next generation of NN's will have the following features ( to mention only some important ones). They are -modular -recurrent -asynchronous Modular neural networks are networks which are composed out of subnetworks, which can be trained independently. A major problem is to find the modular structure which fits the specific application problem. I have proposed genetic neural networks as a longterm research topic. In these networks, the genes specify the network modules and their interconnection. By simulating the evolution process, these networks adapt to the application. I believe that many researchers are going into the same direction. Why not publishing it now? Please contact me by e-mail. Deadline for abstracts is the end of November. Deadline for the finished paper (length 10-15 pages) is January, 31. The issue will appear late summer 1990. -----Heinz Muehlenbein GMD P.O 1240 5205 Sankt Augustin 1 Germany muehlen at gmdzi.uucp From Clayton.Bridges at A.GP.CS.CMU.EDU Tue Nov 7 16:23:28 1989 From: Clayton.Bridges at A.GP.CS.CMU.EDU (Clay Bridges) Date: Tue, 07 Nov 89 16:23:28 EST Subject: Genetics and Connectionism Message-ID: <5568.626477008@A.GP.CS.CMU.EDU> While I may be a bit wet behind the ears, I feel obliged to say something about the recent spate of enthusiasm about combining some sort of genetic evolution with connectionist networks. I am making the assumption that genetic evolution implies some variation on genetic algorithms. I'll admit that combining these two naturally inspired paradigms appeals to one's intuition in a powerful way (It certainly appeals to mine), but I believe that we should be wary. As far as I know, connectionist networks are still not very well understood, at least with respect to recurrent, asynchronous, or modular networks. Genetic algorithms (GAs) are even less well understood than connectionism, in general. Thus, in combining these two fields, we create the _potential_ for a subfield with little or no guiding theory, and thus one where specious projects abound, successes go unexplained, and research efforts go wasted. This is not to say that I don't think that the combination should be explored. I harbor some nascent plans to do so myself. What I am saying is that we shouldn't expect magic when we combine connectionism and GAs. We should explore the combination tentatively, with an eye toward having explanations (i.e. theory) for simple things before we attempt more complex ones. Clay Bridges clay at cs.cmu.edu From caroly at bucasb.BU.EDU Tue Nov 7 14:14:51 1989 From: caroly at bucasb.BU.EDU (Carol Yanakakis) Date: Tue, 7 Nov 89 14:14:51 EST Subject: graduate study Message-ID: <8911071914.AA11971@bucasb.bu.edu> * * * * * * * * * * * * * * * * * * * * * * * * * * * * M.A. AND Ph.D. PROGRAM in * * * * COGNITIVE AND NEURAL SYSTEMS * * * * at BOSTON UNIVERSITY * * * * Gail Carpenter and * * Stephen Grossberg, Co-Directors * * * * * * * * * * * * * * * * * * * * * * * * * * * * Boston University offers a unique M.A. and Ph.D. program in Cognitive and Neural Systems. This program presents an integrated curriculum offering the full range of psychological, neurobiological, and computational concepts, models, and methods in the broad field variously called neural networks, connectionism, parallel distributed processing, and biological information processing, in which Boston University is an acknowledged leader. Each student is required to take an equal number of carefully selected courses in one or more core departments, such as psychology, biology, computer science, mathematics, or engineering. A limited number of full-time graduate research fellowships are expected to be available. ***> For application materials, write to: Admissions Office Graduate School, Boston University 705 Commonwealth Avenue Boston, MA 02215 requesting materials for the Cognitive and Neural Systems (CNS) Program, or call: (617) 353-2697. ***> For a CNS brochure describing the curriculum and degree requirements, write to: Carol Yanakakis, Coordinator CNS Graduate Program Center for Adaptive Systems Boston University 111 Cummington Street Boston, MA 02215 or reply to: caroly at bucasb.bu.edu NOTE: You must contact BOTH the University Admissions Office and the CNS Program Coordinator in order to receive all materials necessary for applying. From eric at mcc.com Tue Nov 7 17:17:23 1989 From: eric at mcc.com (Eric Hartman) Date: Tue, 7 Nov 89 16:17:23 CST Subject: TR available Message-ID: <8911072217.AA05101@hobbes.aca.mcc.com> The following technical report is available. Requests may be sent to eric at mcc.com or via physical mail to the MCC address below. MCC Technical Report Number: ACT-ST-272-89 Layered Neural Networks With Gaussian Hidden Units as Universal Approximators Eric Hartman, James D. Keeler, and Jacek M Kowalski Microelectronics and Computer Technology Corporation 3500 W. Balcones Center Dr. Austin, TX 78759-6509 Abstract: A neural network with a single layer of hidden units of gaussian type (radial basis functions) is proved to be a universal approximator for real-valued maps defined on convex, compact sets set of R^n. (Submitted to Neural Computation) From kolen-j at cis.ohio-state.edu Wed Nov 8 09:44:07 1989 From: kolen-j at cis.ohio-state.edu (john kolen) Date: Wed, 8 Nov 89 09:44:07 EST Subject: Universal Approximators Message-ID: <8911081444.AA01671@toto.cis.ohio-state.edu> Question: How important are "universal approximator" results? Hornik, Stinchcombe, and White [1] demonstrate that a single hidden layer that uses an arbitrary squashing function can appoximate any Borel measurable function (i.e. has a countable number of discontinuities). They do this by showing the functions computable by this class of networks is dense in the set of Borel measurable functions. Great, but so are polynomials, or any sigma-algebra over the input space for that matter. [1] K. Hornik, M. Stinchcombe, H. White. "Multi-Layer Feedforward Networks are Universal Approximators". in press, Neural Networks. ---------------------------------------------------------------------- John Kolen (kolen-j at cis.ohio-state.edu)|computer science - n. A field of study Computer & Info. Sci. Dept. |somewhere between numerology and The Ohio State Univeristy |astrology, lacking the formalism of the Columbus, Ohio 43210 (USA) |former and the popularity of the later. From risto at CS.UCLA.EDU Wed Nov 8 19:24:39 1989 From: risto at CS.UCLA.EDU (Risto Miikkulainen) Date: Wed, 8 Nov 89 16:24:39 -0800 Subject: Tech report: Script Recognition with Hierarchical Feature Maps Message-ID: <8911090024.AA06401@oahu.cs.ucla.edu> **********DO NOT FORWARD TO OTHER BBOARDS************** **********DO NOT FORWARD TO OTHER BBOARDS************** **********DO NOT FORWARD TO OTHER BBOARDS************** The following tech report is available by anonymous ftp from the connectionist tech report database at Ohio State: SCRIPT RECOGNITION WITH HIERARCHICAL FEATURE MAPS Risto Miikkulainen Artificial Intelligence Laboratory Computer Science Department University of California, Los Angeles, CA 90024 risto at cs.ucla.edu Technical Report UCLA-AI-89-10 The hierarchical feature map system recognizes an input story as an instance of a particular script by classifying it at three levels: scripts, tracks and role bindings. The recognition taxonomy, i.e. the breakdown of each script into the relevant tracks and roles, is extracted automatically and independently for each script from story examples in an unsupervised learning process. The process resembles human learning in that the differentiation of the most frequently encountered scripts become gradually the most detailed. The resulting structure is a hierachical pyramid of feature maps. The number of input lines and the self-organization time required are considerably reduced compared to ordinary single-level feature mapping. The system is capable of recognizing incomplete stories and recovering the missing events. --------------- Here's how to obtain a copy: unix> ftp cheops.cis.ohio-state.edu # (or ftp 128.146.8.62) Name (cheops.cis.ohio-state.edu:): anonymous Password (cheops.cis.ohio-state.edu:anonymous): neuron ftp> cd pub/neuroprose ftp> type binary ftp> get (remote-file) miikkulainen.hierarchical.ps.Z (local-file) foo.ps.Z 119599 bytes received in 7.37 seconds (16 Kbytes/sec) ftp> quit unix> uncompress foo.ps unix> lpr -P(your_local_postscript_printer) foo.ps From postech.postech.ac.kr!jhkim at RELAY.CS.NET Thu Nov 9 01:51:24 1989 From: postech.postech.ac.kr!jhkim at RELAY.CS.NET (JooHeon Kim) Date: Thu, 9 Nov 89 15:51:24+0900 Subject: First meeting Message-ID: <8911090651.AA18730@postech.postech.ac.kr> Hello. My name is JooHeon Kim, a graduate student and a researcher at POSTECH, (Pohang Institute of Science and Technology) in KOREA. My interested area is Connectionist Modeling. > Boltzmann Machine Learning > Learning with Distributed Representation My address is JooHeon Kim, Artificial Intelligence Lab., Dept. of Computer Science, POSTECH, Pohang City, P.O.Box 125, KOREA. e-mail : Thanks. From jhkim at postech.postech.ac.kr Thu Nov 9 06:47:05 1989 From: jhkim at postech.postech.ac.kr (jhkim@postech.postech.ac.kr) Date: Thu, 9 Nov 89 13:47:05 +0200 Subject: First meeting Message-ID: <8911091147.AA06250@ariadne.csi.forth.gr> Hello. My name is JooHeon Kim, a graduate student and a researcher at POSTECH, (Pohang Institute of Science and Technology) in KOREA. My interested area is Connectionist Modeling. > Boltzmann Machine Learning > Learning with Distributed Representation My address is JooHeon Kim, Artificial Intelligence Lab., Dept. of Computer Science, POSTECH, Pohang City, P.O.Box 125, KOREA. e-mail : Thanks. From ST401843%BROWNVM.BITNET at VMA.CC.CMU.EDU Thu Nov 9 10:54:56 1989 From: ST401843%BROWNVM.BITNET at VMA.CC.CMU.EDU (thanasis kehagias) Date: Thu, 09 Nov 89 10:54:56 EST Subject: recurrent net bibliography Message-ID: a while ago i posted asking for information on recurrent nets. i collected a lot of responses and compiled a bibliography, which i sent to the list manager of connectionists. here is his response: ------------------------------------------------------------------ (List manager) I've placed a copy of your message and bibliography (recurrent.bib) in the "bibliographies" subdirectory of the connectionists directory that is accessible via anonymous FTP (instructions included below). I would suggest you post a description of the bibliography to the list, telling people that it is accessible via FTP and offering to mail it to people who cannot access it. ------------------------------------------------------------------------------- How to FTP Files from the CONNECTIONISTS Archive ------------------------------------------------ 1. Open an FTP connection to host B.GP.CS.CMU.EDU (Internet address 128.2.242.8). 2. Login as user anonymous with password your username. 3. 'cd' directly to one of the following directories: /usr/connect/connectionists/archives /usr/connect/connectionists/bibliographies 4. The archives and bibliographies directories are the ONLY ones you can access. You can't even find out whether any other directories exist. If you are using the 'cd' command you must cd DIRECTLY into one of these two directories. Access will be denied to any others, including their parent directory. 5. The archives subdirectory contains back issues of the mailing list. Some bibliographies are in the bibliographies subdirectory. Problems? - contact us at "Connectionists-Request at cs.cmu.edu". ------------------------------------------------------- (this is thanasis again): so now you know there is a bibliography and how to access it. in the rest of the message i include a small description of what is in the bibliography ... ****** THIS IS NOT A COMPLETE BIBLIOGRAPHY !!! ******** a while ago i posted a request for information on recurrent nets. i got a lot of responses and i am posting back, as i had promised. the bibliography follows at the end of the message. a few words of explanation: i was not one hundred percent sure of what i was looking for. having seen the responses, i think a fair description of the list i am sending would be: "a personal study guide for the time-space problem in connectionist networks" . what i mean is the following: i would classify neural nets in two broad categories: (1) static nets, where the output of nodes at time t is not fed back to other nodes at time t+1, and (2) dynamic nets where the output of nodes at time t is fed back to other nodes at time t+1. however, some of the static nets attempt to capture temporal relationships, and they usually do this by including time delayed inputs, this is quite commomn for the speech researchers (and i have included some references to such STATIC work), as well as for the people doing the chaotic time series problem (no references included here). for people with signal processing background, this kind of static nets is similar to FIR filters. almost by definition, all the dynamic nets exhibit temporal behavior. however, many of these dynamic nets are used in a static way. e.g. Hopfield type nets are dynamic but they are most often used to store static patterns. they are designed explicitly so they will settle down to a steady state (no oscillations either). (but i must say there are some few exceptions to this). Boltzmann machines are similar: they are in constant thermal motion, but we are basically interested in their steady state. another clarification is important: the training phase involves dynamic behavior in many cases even for static nets. as we vary the weights, the behavior of the system changes in time. but again the basic qualification about this "dynamic" behavior is that we are usually interested in the equilibrium. after all these qualifications, there are some truly dynamic nets, (meaning they are designed so as to exhibit interesting non-equlibria) analogous to the IIR filters in the signal processing paradigm , and my impression is that they are becoming more and more popular. in my opinion they are the most interesting ones and maybe the future is in these truly dynamic nets. having said all of the above, let me add that i include in the following short bibliography almost everything that was sent to me. needless to say, i have not read everything that i include in the list. there is no attempt for completeness here, and omission of some work should not be taken to mean that i consider this work inferior or unimportant. in particular i need to make the following very clear: i included very little ART type work, even if ART architectures is an example of truly dynamic networks. the reason i did this is simply that i am not familiar with this work. the bibliography is very provisional. if you find it useful use it, if not useful, do not flame me. i have a rather arbitrary grouping, with sparse comments as to what each category has. maybe a more annotated version will come later. all the more reason to send me suggestions about additions or changes. and many many thanks to all the people who send me contributions. thanasis From josh at flash.bellcore.com Thu Nov 9 14:54:04 1989 From: josh at flash.bellcore.com (Joshua Alspector) Date: Thu, 9 Nov 89 14:54:04 EST Subject: NIPS VLSI workshop Message-ID: <8911091954.AA03643@flash.bellcore.com> The following is an announcement for one of the workshops to be held December 1-2, 1989 at the Keystone resort after the NIPS-89 conference. -------------------------------------------------- VLSI NEURAL NETWORKS: CURRENT MILESTONES AND FUTURE HORIZONS Moderators: Joshua Alspector and Daniel B. Schwartz Bell Communications Research GTE Laboratories, Inc. 445 South Street 40 Sylvan Road Morristown, NJ 07960-19910 Waltham, MA 02254 (201) 829-4342 (617) 466-2414 e-mail: josh at bellcore.com e-mail: dbs%gte.com at relay.cs.net This workshop will explore the potential and problems of VLSI implementations of neural network. Several speakers will discuss their implementation strategies and speculate about where their work may lead. Workshop attendees will then be encouraged to organize working groups to address specific issues raised in connection with the presentations. An example of a topic that has lead to contentious discussion in the past is the relative virtue of analog vs. digital implementations of neural networks. Some other possible topics include: o Architectural issues - synchronous or asynchronous - full time or multiplexed interconnect - local or global connectivity o Technological issues - neural network specific VLSI technololgies - design tools and methodologies - robustness/fault tolerance o Theoretical issues - model of analog computation - complexity As part of the working groups, we also expect to make contact with deeper issues such as the limits to VLSI complexity for neural networks, the nature of VLSI compatible neural network algorithms, and which neural network applications demand special purpose hardware. Speakers (besides the moderators) include: Jay Sage - MIT-Lincoln Lab Rod Goodman - Caltech Bernhard Boser - AT&T/Bell Labs Alan Kramer - UC Berkeley Jim Burr - Stanford Nelson Morgan - ICSI From gc at s16.csrd.uiuc.edu Thu Nov 9 12:39:53 1989 From: gc at s16.csrd.uiuc.edu (George Cybenko) Date: Thu, 9 Nov 89 11:39:53 CST Subject: "Universal Approximators" Message-ID: <8911091739.AA06184@s16.csrd.uiuc.edu> Recently John Kolen asked why "universal approximator" results are interesting. Universal approximation results are important because they say that the technique being used (sigmoidal or radial basis function networks) has no a priori limitations if complexity (size) of the network is not a constraint. Such results are network analogues of Church's Thesis. Results about universal approximation properties for a variety of network types can be found in : G. Cybenko, "Approximation by Superpositions of a Sigmoidal Function", Mathematics of Control, Signals and Systems (Springer), August 1989, Vol. 2, pp. 303-314. Since such properties are shared by polynomials, Fourier series, splines, etc. etc., an important question to ask is "What makes network approaches better"? Parallelism and the existence of training algorithms does not count as an answer because polynomials and Fourier series have similar parallelism and training algorithms. I have a recent report that scratches the surface of this question and proposes a notion of complexity of classification problems that attempts to capture the way in which sigmoidal networks might be better on a class of problems. The report is titled "Designing Neural Networks" and is currently issued as CSRD Report #934. Send me mail or comments if you are interested in such questions. George Cybenko Center for Supercomputing Research and Development University of Illinois at Urbana-Champaign From rik%cs at ucsd.edu Thu Nov 9 15:53:09 1989 From: rik%cs at ucsd.edu (Rik Belew) Date: Thu, 9 Nov 89 12:53:09 PST Subject: Combining connectionism with the GA Message-ID: <8911092053.AA20392@roland.UCSD.EDU> There's been too much Email ink spilled lately about potential interactions between evolutionary algorithms and connectionist networks for me not to throw my two cents in. I announced a new tech report, "Evolution, learning and culture: Computational metaphors for adaptive algorithms" on this list a couple of weeks ago and I don't know whether the recent storm of messages has anything to do with that report or not. Anyway, some of the things I'll say below are better said there. Let me begin with Hammerstrom's analysis, related in [rudnick at cse.ogc.edu 26 Oct 89]. His basic point seems to be that connectionist algorithms (he uses NetTalk as his example) take a long time, and putting an evolutionary outer loop around them can only make matters worse. And if we are satisfied with the results of a single backpropagation (BP) search and the evloutionary loop is doing nothing more than randomly repeating this experiment, he may be right. But only if. First, who today is satisfied with vanilla BP in feed-forward networks? (And I'm not just BP bashing; almost all of what I say applies equally to other gradient descent connectionist learning algorithms.) A great deal of current research is concerned with both simplifying such nets to more minimal structures, and elaborating the nets (e.g., with recurrence) to solve more difficult problems. Also, BP performance is known to be highly variable under stochastic variation. Consequently most investigators do use an 'outer-loop' iteration, using multiple restarts to improve their confidence in the solution found. The Genetic Algoritms (GAs) can help with these connectionist problems. (To clarify, when I talk of 'evolutionary algorithms' I have some variant of John Holland's Genetic Algorithm in mind because I believe it to be the most powerful. But there are other evolution- like algorithms (eg, Ackley, Fogel Jr. & Sr.) and these may also prove useful to connectionists.) Second, there is a considerable body of work that shows that evolutionary search is far from random. GA's are extremely effective sampling procedures, at least on some kinds of problems. (See Goldberg's book or the most recent GA proceedings for characterizations of what makes a problem 'GA-hard'.) Further, there are reasons to believe that connectionist nets are a good problem for the GA: the GLOBAL SAMPLING performed by the GA is very compimentary to the LOCAL SEARCH performed by gradient descent procedures like BP. Bridges complains that we are compounding ignorance when we try to consider hybrids of connectionist and GA algorithms [clay at cs.cmu.edu 7 Nov 89]. But we are beginning to understand the basic features of connectionist search (as function approximators, via analysis of internal structure,etc.), and there are substantial things known about the GA, too (e.g., Holland's Schema Theorem and its progeny). These foundations do suggest deliberate research strategies and immediately eliminate others (eg, some of the naive ways in which we might encode a network onto a GA string). There are a tremendous number of ways the techniques can be combined, with the GA as simply an outer loop around a conventional BP simulation being one of the least imaginative. For example, when used in conjunction with the GA there is a very good question as to how long each BP training cycle must be in order to provide a useful fitness measure for the GA. Preliminary results of ours suggest much shorter training times are possible. Similarly, use of the GA seems to tolerate quicker search (i.e., higher learning rate and momentum values) than typical. Another rich dimension is just how a GA bit string is related to a net, from a literal encoding of every real number weight at one extreme to complex developmental programs that build nets at the other. One feature of these hybrids that should not be underestimated is that the GA offers a very natural mechanism for introducing PARALLELISM into connectionist algorithms, since each individual in a population can be evaluated independently. We have had some success exploiting this parallelism in two implementations, one in a SIMD (Connection Machine) environment and one using a heterogeneous mess of distributed computers. Finally we shouldn't let computer science drive the entire agenda. Theoretical biology and evolutionary neurophysiology have some extremely important and unresolved questions that models like these can help to address, for example concerning complex, Lamarckian-like interactions between the learning of individuals and the evolution of species. (I think the Harp et al. simulation may be particularly useful to evolutionary neurophysiologists.) One of the things that makes connectionism most exciting is that the same class of systems that interest (some) mathematicians as new statistical techniques also interest neuroscientists as a theory to coordinate their data collection. I think GA/connectionist hybrids are important for similar reasons: they make sense as algorithms AND they make sense as models of natural phenomena. This note has been long on enthusiasm and short on specifics. But some results have already been reported, by me and others. And I expect there to be many new results reported at the upcoming sessions (at IJCNN in Washington and at the NIPS workshop) devoted to this topic. So watch this space. Richard K. Belew rik%cs at ucsd.edu Assistant Professor CSE Department (C-014) UCSD San Diego, CA 92093 619 / 534-2601 or 534-5288 (messages) From mike at bucasb.BU.EDU Fri Nov 10 00:57:00 1989 From: mike at bucasb.BU.EDU (Michael Cohen) Date: Fri, 10 Nov 89 00:57:00 EST Subject: NEURAL NET SUMMER SCHOOL AT WANG INSTITUTE Message-ID: <8911100557.AA24680@bucasb.bu.edu> BOSTON UNIVERSITY, A WORLD LEADER IN NEURAL NETWORK RESEARCH AND TECHNOLOGY, PRESENTS TWO MAJOR SCIENTIFIC EVENTS: MAY 6--11, 1990 NEURAL NETWORKS: FROM FOUNDATIONS TO APPLICATIONS A self-contained systematic course by leading neural architects who know the field as only its creators can. MAY 11--13, 1990 NEURAL NETWORKS FOR AUTOMATIC TARGET RECOGNITION An international research conference presenting INVITED and CONTRIBUTED papers, herewith solicited, on one of the most active research topics in science and technology today. SPONSORED BY THE CENTER FOR ADAPTIVE SYSTEMS AND THE WANG INSTITUTE OF BOSTON UNIVERSITY WITH PARTIAL SUPPORT FROM THE AIR FORCE OFFICE OF SCIENTIFIC RESEARCH ----------------------------------------------------------------------------- CALL FOR PAPERS --------------- NEURAL NETWORKS FOR AUTOMATIC TARGET RECOGNITION MAY 11--13, 1990 This research conference at the cutting edge of neural network science and technology will bring together leading experts in academe, government, and industry to present their latest results on automatic target recognition in invited lectures and contributed posters. Automatic target recognition is a key process in systems designed for vision and image processing, speech and time series prediction, adaptive pattern recognition, and adaptive sensory-motor control and robotics. It is one of the areas emphasized by the DARPA Neural Networks Program, and has attracted intense research activity around the world. Invited lecturers include: JOE BROWN, Martin Marietta, "Multi-Sensor ATR using Neural Nets" GAIL CARPENTER, Boston University, "Target Recognition by Adaptive Resonance: ART for ATR" NABIL FARHAT, University of Pennsylvania, "Bifurcating Networks for Target Recognition" STEPHEN GROSSBERG, Boston University, "Recent Results on Self-Organizing ATR Networks" ROBERT HECHT-NIELSEN, HNC, "Spatiotemporal Attention Focusing by Expectation Feedback" KEN JOHNSON, Hughes Aircraft, "The Application of Neural Networks to the Acquisition and Tracking of Maneuvering Tactical Targets in High Clutter IR Imagery" PAUL KOLODZY, MIT Lincoln Laboratory, "A Multi-Dimensional ATR System" MICHAEL KUPERSTEIN, Neurogen, "Adaptive Sensory-Motor Coordination using the INFANT Controller" YANN LECUN, AT&T Bell Labs, "Structured Back Propagation Networks for Handwriting Recognition" CHRISTOPHER SCOFIELD, Nestor, "Neural Network Automatic Target Recognition by Active and Passive Sonar Signals" STEVEN SIMMES, Science Applications International Co., "Massively Parallel Approaches to Automatic Target Recognition" ALEX WAIBEL, Carnegie Mellon University, "Patterns, Sequences and Variability: Advances in Connectionist Speech Recognition" ALLEN WAXMAN, MIT Lincoln Laboratory, "Invariant Learning and Recognition of 3D Objects from Temporal View Sequences" FRED WEINGARD, Booz-Allen and Hamilton, "Current Status and Results of Two Major Government Programs in Neural Network-Based ATR" BARBARA YOON, DARPA, "DARPA Artificial Neural Networks Technology Program: Automatic Target Recognition" ------------------------------------------------------ CALL FOR PAPERS---ATR POSTER SESSION: A featured poster session on ATR neural network research will be held on May 12, 1990. Attendees who wish to present a poster should submit 3 copies of an extended abstract (1 single-spaced page), postmarked by March 1, 1990, for refereeing. Include with the abstract the name, address, and telephone number of the corresponding author. Mail to: ATR Poster Session, Neural Networks Conference, Wang Institute of Boston University, 72 Tyng Road, Tyngsboro, MA 01879. Authors will be informed of abstract acceptance by March 31, 1990. SITE: The Wang Institute possesses excellent conference facilities on a beautiful 220-acre rustic setting. It is easily reached from Boston's Logan Airport and Route 128. REGISTRATION FEE: Regular attendee--$90; full-time student--$70. Registration fee includes admission to all lectures and poster session, one reception, two continental breakfasts, one lunch, one dinner, daily morning and afternoon coffee service. STUDENTS: Read below about FELLOWSHIP support. REGISTRATION: To register by telephone with VISA or MasterCard call (508) 649-9731 between 9:00AM--5:00PM (EST). To register by FAX, fill out the registration form and FAX back to (508) 649-6926. To register by mail, complete the registration form and mail with your full form of payment as directed. Make check payable in U.S. dollars to "Boston University". See below for Registration Form. To register by electronic mail, use the address "rosenber at bu-tyng.bu.edu". On-site registration on a space-available basis will take place from 1:00--5:00PM on Friday, May 11. A RECEPTION will be held from 3:00--5:00PM on Friday, May 11. LECTURES begin at 5:00PM on Friday, May 11 and conclude at 1:00PM on Sunday, May 13. ------------------------------------------------------------------------------ NEURAL NETWORKS: FROM FOUNDATIONS TO APPLICATIONS MAY 6--11, 1989 This in-depth, systematic, 5-day course is based upon the world's leading graduate curriculum in the technology, computation, mathematics, and biology of neural networks. Developed at the Center for Adaptive Systems (CAS) and the Graduate Program in Cognitive and Neural Systems (CNS) of Boston University, twenty-eight hours of the course will be taught by six CAS/CNS faculty. Three distinguished guest lecturers will present eight hours of the course. COURSE OUTLINE -------------- MAY 7, 1990 ----------- MORNING SESSION (PROFESSOR GROSSBERG) HISTORICAL OVERVIEW: Introduction to the binary, linear, and continuous-nonlinear streams of neural network research: McCulloch-Pitts, Rosenblatt, von Neumann; Anderson, Kohonen, Widrow; Hodgkin-Huxley, Hartline-Ratliff, Grossberg. CONTENT ADDRESSABLE MEMORY: Classification and analysis of neural network models for absolutely stable CAM. Models include: Cohen-Grossberg, additive, shunting, Brain-State-In-A-Box, Hopfield, Boltzmann Machine, McCulloch-Pitts, masking field, bidirectional associative memory. COMPETITIVE DECISION MAKING: Analysis of asynchronous variable-load parallel processing by shunting competitive networks; solution of noise-saturation dilemma; classification of feedforward networks: automatic gain control, ratio processing, Weber law, total activity normalization, noise suppression, pattern matching, edge detection, brightness constancy and contrast, automatic compensation for variable illumination or other background energy distortions; classification of feedback networks: influence of nonlinear feedback signals, notably sigmoid signals, on pattern transformation and memory storage, winner-take-all choices, partial memory compression, tunable filtering, quantization and normalization of total activity, emergent boundary segmentation; method of jumps for classifying globally consistent and inconsistent competitive decision schemes. ASSOCIATIVE LEARNING: Derivation of associative equations for short-term memory and long-term memory. Overview and analysis of associative outstars, instars, computational maps, avalanches, counterpropagation nets, adaptive bidrectional associative memories. Analysis of unbiased associative pattern learning by asynchronous parallel sampling channels; classification of associative learning laws. AFTERNOON SESSION (PROFESSORS JORDAN AND MINGOLLA) COMBINATORIAL OPTIMIZATION PERCEPTRONS: Adeline, Madeline, delta rule, gradient descent, adaptive statistical predictor, nonlinear separability. INTRODUCTION TO BACK PROPAGATION: Supervised learning of multidimensional nonlinear maps, NETtalk, image compression, robotic control. RECENT DEVELOPMENTS OF BACK PROPAGATION: This two-hour guest tutorial lecture will provide a systematic review of recent developments of the back propagation learning network, especially focussing on recurrent back propagation variations and applications to outstanding technological problems. EVENING SESSION: DISCUSSIONS WITH TUTORS MAY 8, 1990 ----------- MORNING SESSION (PROFESSORS CARPENTER AND GROSSBERG) ADAPTIVE PATTERN RECOGNITION: Adaptive filtering; contrast enhancement; competitive learning of recognition categories; adaptive vector quantization; self-organizing computational maps; statistical properties of adaptive weights; learning stability and causes of instability. INTRODUCTION TO ADAPTIVE RESONANCE THEORY: Absolutely stable recognition learning, role of learned top-down expectations; attentional priming; matching by 2/3 Rule; adaptive search; self-controlled hypothesis testing; direct access to globally optimal recognition code; control of categorical coarseness by attentional vigilance; comparison with relevant behavioral and brain data to emphasize biological basis of ART computations. ANALYSIS OF ART 1: Computational analysis of ART 1 architecture for self-organized real-time hypothesis testing, learning, and recognition of arbitrary sequences of binary input patterns. AFTERNOON SESSION (PROFESSOR CARPENTER) ANALYSIS OF ART 2: Computational analysis of ART 2 architecture for self-organized real-time hypothesis testing, learning, and recognition for arbitrary sequences of analog or binary input patterns. ANALYSIS OF ART 3: Computational analysis of ART 3 architecture for self-organized real-time hypothesis testing, learning, and recognition within distributed network hierarchies; role of chemical transmitter dynamics in forming a memory representation distinct from short-term memory and long-term memory; relationships to brain data concerning neuromodulators and synergetic ionic and transmitter interactions. SELF-ORGANIZATION OF INVARIANT PATTERN RECOGNITION CODES: Computational analysis of self-organizing ART architectures for recognizing noisy imagery undergoing changes in position, rotation, and size. NEOCOGNITION: Recognition and completion of images by hierarchical bottom-up filtering and top-down attentive feedback. EVENING SESSION: DISCUSSIONS WITH TUTORS MAY 9, 1990 ----------- MORNING SESSION (PROFESSORS GROSSBERG & MINGOLLA) VISION AND IMAGE PROCESSING: Introduction to Boundary Contour System for emergent segmentation and Feature Contour System for filling-in after compensation for variable illumination; image compression, orthogonalization, and reconstruction; multidimensional filtering, multiplexing, and fusion; coherent boundary detection, regularization, self-scaling, and completion; compensation for variable illumination sources, including artificial sensors (infrared sensors, laser radars); filling-in of surface color and form; 3-D form from shading, texture, stereo, and motion; parallel processing of static form and moving form; motion capture and induced motion; synthesis of static form and motion form representations. AFTERNOON SESSION (PROFESSORS BULLOCK, COHEN, & GROSSBERG) ADAPTIVE SENSORY-MOTOR CONTROL AND ROBOTICS: Overview of recent progress in adaptive sensory-motor control and related robotics research. Reaching to, grasping, and transporting objects of variable mass and form under visual guidance in a cluttered environment will be used as a target behavioral competence to clarify subproblems of real-time adaptive sensory-motor control. The balance of the tutorial will be spent detailing neural network modules that solve various subproblems. Topics include: Self-organizing networks for real-time control of eye movements, arm movements, and eye-arm coordination; learning of invariant body-centered target position maps; learning of intermodal associative maps; real-time trajectory formation; adaptive vector encoders; circular reactions between action and sensory feedback; adaptive control of variable speed movements; varieties of error signals; supportive behavioral and neural data; inverse kinematics; automatic compensation for unexpected perturbations; independent adaptive control of force and position; adaptive gain control by cerebellar learning; position-dependent sampling from spatial maps; predictive motor planning and execution. SPEECH PERCEPTION AND PRODUCTION: Hidden Markov models; self-organization of speech perception and production codes; eighth nerve Average Localized Synchrony Response; phoneme recognition by back propagation, time delay networks, and vector quantization. MAY 10, 1990 ------------ MORNING SESSION (PROFESSORS COHEN, GROSSBERG, & MERRILL) SPEECH PERCEPTION AND PRODUCTION: Disambiguation of coarticulated vowels and consonants; dynamics of working memory; multiple-scale adaptive coding by masking fields; categorical perception; phonemic restoration; contextual disambiguation of speech tokens; resonant completion and grouping of noisy variable-rate speech streams. REINFORCEMENT LEARNING AND PREDICTION: Recognition learning, reinforcement learning, and recall learning are the 3 R's of neural network learning. Reinforcement learning clarifies how external events interact with internal organismic requirements to trigger learning processes capable of focussing attention upon and generating appropriate actions towards motivationally desired goals. A neural network model will be derived to show how reinforcement learning and recall learning can self-organize in response to asynchronous series of significant and irrelevant events. These mechanisms also control selective forgetting of memories that are no longer predictive, adaptive timing of behavioral responses, and self-organization of goal directed problem solvers. AFTERNOON SESSION (PROFESSORS GROSSBERG & MERRILL AND DR. HECHT-NIELSEN) REINFORCEMENT LEARNING AND PREDICTION: Analysis of drive representations, adaptive critics, conditioned reinforcers, role of motivational feedback in focusing attention on predictive data; attentional blocking and unblocking; adaptively timed problem solving; synthesis of perception, recognition, reinforcement, recall, and robotics mechanisms into a total neural architecture; relationship to data about hypothalamus, hippocampus, neocortex, and related brain regions. RECENT DEVELOPMENTS IN THE NEUROCOMPUTER INDUSTRY: This two-hour guest tutorial will provide an overview of the growth and prospects of the burgeoning neurocomputer industry by one of its most important leaders. EVENING SESSION: DISCUSSIONS WITH TUTORS MAY 11, 1990 ------------ MORNING SESSION (DR. FAGGIN) VLSI IMPLEMENTATION OF NEURAL NETWORKS: This is a four-hour self-contained tutorial on the application and development of VLSI techniques for creating compact real-time chips embodying neural network designs for applications in technology. Review of neural networks from a hardware implementation perspective; hardware requirements and alternatives; dedicated digital implementation of neural networks; neuromorphic design methodology using VLSI CMOS technology; applications and performance of neuromorphic implementations; comparison of neuromorphic and digital hardware; future prospectus. ---------------------------------------------------------------------------- COURSE FACULTY FROM BOSTON UNIVERSITY ------------------------------------- STEPHEN GROSSBERG, Wang Professor of CNS, as well as Professor of Mathematics, Psychology, and Biomedical Engineering, is one of the world's leading neural network pioneers and most versatile neural architects; Founder and 1988 President of the International Neural Network Society (INNS); Founder and Co-Editor-in-Chief of the INNS journal "Neural Networks"; an editor of the journals "Neural Computation", "Cognitive Science", and "IEEE Expert"; Founder and Director of the Center for Adaptive Systems; General Chairman of the 1987 IEEE First International Conference on Neural Networks (ICNN); Chief Scientist of Hecht-Nielsen Neurocomputer Company (HNC); and one of the four technical consultants to the national DARPA Neural Network Study. He is author of 200 articles and books about neural networks, including "Neural Networks and Natural Intelligence" (MIT Press, 1988), "Neural Dynamics of Adaptive Sensory-Motor Control" (with Michael Kuperstein, Pergamon Press, 1989), "The Adaptive Brain, Volumes I and II" (Elsevier/North-Holland, 1987), "Studies of Mind and Brain" (Reidel Press, 1982), and the forthcoming "Pattern Recognition by Self-Organizing Neural Networks" (with Gail Carpenter). GAIL CARPENTER is Professor of Mathematics and CNS; Co-Director of the CNS Graduate Program; 1989 Vice President of the International Neural Network Society (INNS); Organization Chairman of the 1988 INNS annual meeting; Session Chairman at the 1989 and 1990 IEEE/INNS International Joint Conference on Neural Networks (IJCNN); one of four technical consultants to the national DARPA Neural Network Study; editor of the journals "Neural Networks", "Neural Computation", and "Neural Network Review"; and a member of the scientific advisory board of HNC. A leading neural architect, Carpenter is especially well-known for her seminal work on developing the adaptive resonance theory architectures (ART 1, ART 2, ART 3) for adaptive pattern recognition. MICHAEL COHEN, Associate Professor of Computer Science and CNS, is a leading architect of neural networks for content addressable memory (Cohen-Grossberg model), vision (Feature Contour System), and speech (Masking Fields); editor of "Neural Networks"; Session Chairman at the 1987 ICNN, and the 1989 IJCNN; and member of the DARPA Neural Network Study panel on Simulation/Emulation Tools and Techniques. ENNIO MINGOLLA, Assistant Professor of Psychology and CNS, is holder of one of the first patented neural network architectures for vision and image processing (Boundary Contour System); Co-Organizer of the 3rd Workshop on Human and Machine Vision in 1985; editor of the journals "Neural Networks" and "Ecological Psychology"; member of the DARPA Neural Network Study panel of Adaptive Knowledge Processing; consultant to E.I. duPont de Nemours, Inc.; Session Chairman for vision and image processing at the 1987 ICNN, and the 1988 INNS meetings. DANIEL BULLOCK, Assistant Professor of Psychology and CNS, is developer of neural network models for real-time adaptive sensory-motor control of arm movements and eye-arm coordination, notably the VITE and FLETE models for adaptive control of multi-joint trajectories; editor of "Neural Networks"; Session Chairman for adaptive sensory-motor control and robotics at the 1987 ICNN and the 1988 INNS meetings; invited speaker at the 1990 IJCNN. JOHN MERRILL, Assistant Professor of Mathematics and CNS, is developing neural network models for adaptive pattern recognition, speech recognition, reinforcement learning, and adaptive timing in problem solving behavior, after having received his Ph.D. in mathematics from the University of Wisconsin at Madison, and completing postdoctoral research in computer science and linguistics at Indiana University. GUEST LECTURERS --------------- FEDERICO FAGGIN is co-founder and president of Synaptics, Inc. Dr. Faggin developed the Silicon Gate Technology at Fairchild Semiconductor. He also designed the first commercial circuit using Silicon Gate Technology: the 3708, an 8-bit analog multiplexer. At Intel Corporation he was responsible for designing what was to become the first microprocessor---the 4000 family, also called MCS-4. He and Hal Feeney designed the 8008, the first 8-bit microprocessor introduced in 1972, and later Faggin conceived the 8080 and with M. Shima designed it. The 8080 was the first high-performance 8-bit microprocessor. At Zilog Inc., Faggin conceived the Z80 microprocessor family and directed the design of the Z80 CPU. Faggin also started Cygnet Technologies, which developed a voice and data communication peripheral for the personal computer. In 1986 Faggin co-founded Synaptics Inc., a company dedicated to the creation of a new type of VLSI hardware for artificial neural networks and other machine intelligence applications. Faggin is the recipient of the 1988 Marconi Fellowship Award for his contributions to the birth of the microprocessor. ROBERT HECHT-NIELSEN is co-founder and chairman of the Board of Directors of Hecht-Nielsen Neurocomputer Corporation (HNC), a pioneer in neurocomputer technology and the application of neural networks, and a recognized leader in the field. Prior to the formation of HNC, he founded and managed the neurocomputer development and neural network applications at TRW (1983--1986) and Motorola (1979--1983). He has been active in neural network technology and neurocomputers since 1961 and earned his Ph.D. in mathematics in 1974. He is currently a visiting lecturer in the Electrical Engineering Department at the University of California at San Diego, and is the author of influential technical reports and papers on neurocomputers, neural networks, pattern recognition, signal processing algorithms, and artificial intelligence. MICHAEL JORDAN is an Assistant Professor of Brain and Cognitive Sciences at MIT. One of the key developers of the recurrent back propagation algorithms, Professor Jordan's research is concerned with learning in recurrent networks and with the use of networks as forward models in planning and control. His interest in interdisciplinary research on neural networks is founded in his training for a Bachelors degree in Psychology, a Masters degree in Mathematics, and a Ph.D. in Cognitive Science from the University of California at San Diego. He was a postdoctoral researcher in Computer Science at the University of Massachusetts at Amherst before assuming his present position at MIT. ---------------------------------------------------------- REGISTRATION FEE: Regular attendee--$950; full-time student--$250. Registration fee includes five days of tutorials, course notebooks, one reception, five continental breakfasts, five lunches, four dinners, daily morning and afternoon coffee service, evening discussion sessions with leading neural architects. REGISTRATION: To register by telephone with VISA or MasterCard call (508) 649-9731 between 9:00AM--5:00PM (EST). To register by FAX, fill out the registration form and FAX back to (508) 649-6926. To register by mail, complete the registration form and mail with you full form of payment as directed. Make check payable in U.S. dollars to "Boston University". See below for Registration Form. To register by electronic mail, use the address "rosenber at bu-tyng.bu.edu". On-site registration on a space-available basis will take place from 2:00--7:00PM on Sunday, May 6 and from 7:00--8:00AM on Monday, May 7, 1990. A RECEPTION will be held from 4:00--7:00PM on Sunday, May 6. LECTURES begin at 8:00AM on Monday, May 7 and conclude at 12:30PM on Friday, May 11. STUDENT FELLOWSHIPS supporting travel, registration, and lodging for the Course and the Research Conference are available to full-time graduate students in a PhD program. Applications must be postmarked by March 1, 1990. Send curriculum vitae, a one-page essay describing your interest in neural networks, and a letter from a faculty advisor to: Student Fellowships, Neural Networks Course, Wang Institute of Boston University, 72 Tyng Road, Tyngsboro, MA 01879. CNS FELLOWSHIP FUND: Net revenues from the course will endow fellowships for Ph.D. candidates in the CNS Graduate Program. Corporate and individual gifts to endow CNS Fellowships are also welcome. Please write: Cognitive and Neural Systems Fellowship Fund, Center for Adaptive Systems, Boston University, 111 Cummington Street, Boston, MA 02215. ------------------------------------------------------------------------------ REGISTRATION FOR COURSE AND RESEARCH CONFERENCE Course: Neural Network Foundations and Applications, May 6--11, 1990 Research Conference: Neural Networks for Automatic Target Recognition, May 11--13, 1990 NAME: _________________________________________________________________ ORGANIZATION (for badge): _____________________________________________ MAILING ADDRESS: ______________________________________________________ ______________________________________________________ CITY/STATE/COUNTRY: ___________________________________________________ POSTAL/ZIP CODE: ______________________________________________________ TELEPHONE(S): _________________________________________________________ COURSE RESEARCH CONFERENCE ------ ------------------- [ ] regular attendee $950 [ ] regular attendee $90 [ ] full-time student $250 [ ] full-time student $70 (limited number of spaces) (limited number of spaces) [ ] Gift to CNS Fellowship Fund TOTAL PAYMENT: $________ FORM OF PAYMENT: [ ] check or money order (payable in U.S. dollars to Boston University) [ ] VISA [ ] MasterCard Card Number: ______________________________________________ Expiration Date: ______________________________________________ Signature: ______________________________________________ Please complete and mail to: Neural Networks Wang Institute of Boston University 72 Tyng Road Tyngsboro, MA 01879 USA To register by telephone, call: (508) 649-9731. HOTEL RESERVATIONS: Room blocks have been reserved at 3 hotels near the Wang Institute. Hotel names, rates, and telephone numbers are listed below. A shuttle bus will take attendees to and from the hotels for the Course and Research Conference. Attendees should make their own reservations by calling the hotel. The special conference rate applies only if you mention the name and dates of the meeting when making the reservations. Sheraton Tara Red Roof Inn Stonehedge Inn Nashua, NH Nashua, NH Tyngsboro, MA (603) 888-9970 (603) 888-1893 (508) 649-4342 $70/night+tax $39.95/night+tax $89/night+tax The hotels in Nashua are located approximately 5 miles from the Wang Institute. A shuttle bus will be provided. ------------------------------------------------------------------------------- From mike at bucasb.BU.EDU Fri Nov 10 00:57:00 1989 From: mike at bucasb.BU.EDU (Michael Cohen) Date: Fri, 10 Nov 89 00:57:00 EST Subject: NEURAL NET SUMMER SCHOOL AT WANG INSTITUTE Message-ID: <8911100557.AA24680@bucasb.bu.edu> BOSTON UNIVERSITY, A WORLD LEADER IN NEURAL NETWORK RESEARCH AND TECHNOLOGY, PRESENTS TWO MAJOR SCIENTIFIC EVENTS: MAY 6--11, 1990 NEURAL NETWORKS: FROM FOUNDATIONS TO APPLICATIONS A self-contained systematic course by leading neural architects who know the field as only its creators can. MAY 11--13, 1990 NEURAL NETWORKS FOR AUTOMATIC TARGET RECOGNITION An international research conference presenting INVITED and CONTRIBUTED papers, herewith solicited, on one of the most active research topics in science and technology today. SPONSORED BY THE CENTER FOR ADAPTIVE SYSTEMS AND THE WANG INSTITUTE OF BOSTON UNIVERSITY WITH PARTIAL SUPPORT FROM THE AIR FORCE OFFICE OF SCIENTIFIC RESEARCH ----------------------------------------------------------------------------- CALL FOR PAPERS --------------- NEURAL NETWORKS FOR AUTOMATIC TARGET RECOGNITION MAY 11--13, 1990 This research conference at the cutting edge of neural network science and technology will bring together leading experts in academe, government, and industry to present their latest results on automatic target recognition in invited lectures and contributed posters. Automatic target recognition is a key process in systems designed for vision and image processing, speech and time series prediction, adaptive pattern recognition, and adaptive sensory-motor control and robotics. It is one of the areas emphasized by the DARPA Neural Networks Program, and has attracted intense research activity around the world. Invited lecturers include: JOE BROWN, Martin Marietta, "Multi-Sensor ATR using Neural Nets" GAIL CARPENTER, Boston University, "Target Recognition by Adaptive Resonance: ART for ATR" NABIL FARHAT, University of Pennsylvania, "Bifurcating Networks for Target Recognition" STEPHEN GROSSBERG, Boston University, "Recent Results on Self-Organizing ATR Networks" ROBERT HECHT-NIELSEN, HNC, "Spatiotemporal Attention Focusing by Expectation Feedback" KEN JOHNSON, Hughes Aircraft, "The Application of Neural Networks to the Acquisition and Tracking of Maneuvering Tactical Targets in High Clutter IR Imagery" PAUL KOLODZY, MIT Lincoln Laboratory, "A Multi-Dimensional ATR System" MICHAEL KUPERSTEIN, Neurogen, "Adaptive Sensory-Motor Coordination using the INFANT Controller" YANN LECUN, AT&T Bell Labs, "Structured Back Propagation Networks for Handwriting Recognition" CHRISTOPHER SCOFIELD, Nestor, "Neural Network Automatic Target Recognition by Active and Passive Sonar Signals" STEVEN SIMMES, Science Applications International Co., "Massively Parallel Approaches to Automatic Target Recognition" ALEX WAIBEL, Carnegie Mellon University, "Patterns, Sequences and Variability: Advances in Connectionist Speech Recognition" ALLEN WAXMAN, MIT Lincoln Laboratory, "Invariant Learning and Recognition of 3D Objects from Temporal View Sequences" FRED WEINGARD, Booz-Allen and Hamilton, "Current Status and Results of Two Major Government Programs in Neural Network-Based ATR" BARBARA YOON, DARPA, "DARPA Artificial Neural Networks Technology Program: Automatic Target Recognition" ------------------------------------------------------ CALL FOR PAPERS---ATR POSTER SESSION: A featured poster session on ATR neural network research will be held on May 12, 1990. Attendees who wish to present a poster should submit 3 copies of an extended abstract (1 single-spaced page), postmarked by March 1, 1990, for refereeing. Include with the abstract the name, address, and telephone number of the corresponding author. Mail to: ATR Poster Session, Neural Networks Conference, Wang Institute of Boston University, 72 Tyng Road, Tyngsboro, MA 01879. Authors will be informed of abstract acceptance by March 31, 1990. SITE: The Wang Institute possesses excellent conference facilities on a beautiful 220-acre rustic setting. It is easily reached from Boston's Logan Airport and Route 128. REGISTRATION FEE: Regular attendee--$90; full-time student--$70. Registration fee includes admission to all lectures and poster session, one reception, two continental breakfasts, one lunch, one dinner, daily morning and afternoon coffee service. STUDENTS: Read below about FELLOWSHIP support. REGISTRATION: To register by telephone with VISA or MasterCard call (508) 649-9731 between 9:00AM--5:00PM (EST). To register by FAX, fill out the registration form and FAX back to (508) 649-6926. To register by mail, complete the registration form and mail with your full form of payment as directed. Make check payable in U.S. dollars to "Boston University". See below for Registration Form. To register by electronic mail, use the address "rosenber at bu-tyng.bu.edu". On-site registration on a space-available basis will take place from 1:00--5:00PM on Friday, May 11. A RECEPTION will be held from 3:00--5:00PM on Friday, May 11. LECTURES begin at 5:00PM on Friday, May 11 and conclude at 1:00PM on Sunday, May 13. ------------------------------------------------------------------------------ NEURAL NETWORKS: FROM FOUNDATIONS TO APPLICATIONS MAY 6--11, 1989 This in-depth, systematic, 5-day course is based upon the world's leading graduate curriculum in the technology, computation, mathematics, and biology of neural networks. Developed at the Center for Adaptive Systems (CAS) and the Graduate Program in Cognitive and Neural Systems (CNS) of Boston University, twenty-eight hours of the course will be taught by six CAS/CNS faculty. Three distinguished guest lecturers will present eight hours of the course. COURSE OUTLINE -------------- MAY 7, 1990 ----------- MORNING SESSION (PROFESSOR GROSSBERG) HISTORICAL OVERVIEW: Introduction to the binary, linear, and continuous-nonlinear streams of neural network research: McCulloch-Pitts, Rosenblatt, von Neumann; Anderson, Kohonen, Widrow; Hodgkin-Huxley, Hartline-Ratliff, Grossberg. CONTENT ADDRESSABLE MEMORY: Classification and analysis of neural network models for absolutely stable CAM. Models include: Cohen-Grossberg, additive, shunting, Brain-State-In-A-Box, Hopfield, Boltzmann Machine, McCulloch-Pitts, masking field, bidirectional associative memory. COMPETITIVE DECISION MAKING: Analysis of asynchronous variable-load parallel processing by shunting competitive networks; solution of noise-saturation dilemma; classification of feedforward networks: automatic gain control, ratio processing, Weber law, total activity normalization, noise suppression, pattern matching, edge detection, brightness constancy and contrast, automatic compensation for variable illumination or other background energy distortions; classification of feedback networks: influence of nonlinear feedback signals, notably sigmoid signals, on pattern transformation and memory storage, winner-take-all choices, partial memory compression, tunable filtering, quantization and normalization of total activity, emergent boundary segmentation; method of jumps for classifying globally consistent and inconsistent competitive decision schemes. ASSOCIATIVE LEARNING: Derivation of associative equations for short-term memory and long-term memory. Overview and analysis of associative outstars, instars, computational maps, avalanches, counterpropagation nets, adaptive bidrectional associative memories. Analysis of unbiased associative pattern learning by asynchronous parallel sampling channels; classification of associative learning laws. AFTERNOON SESSION (PROFESSORS JORDAN AND MINGOLLA) COMBINATORIAL OPTIMIZATION PERCEPTRONS: Adeline, Madeline, delta rule, gradient descent, adaptive statistical predictor, nonlinear separability. INTRODUCTION TO BACK PROPAGATION: Supervised learning of multidimensional nonlinear maps, NETtalk, image compression, robotic control. RECENT DEVELOPMENTS OF BACK PROPAGATION: This two-hour guest tutorial lecture will provide a systematic review of recent developments of the back propagation learning network, especially focussing on recurrent back propagation variations and applications to outstanding technological problems. EVENING SESSION: DISCUSSIONS WITH TUTORS MAY 8, 1990 ----------- MORNING SESSION (PROFESSORS CARPENTER AND GROSSBERG) ADAPTIVE PATTERN RECOGNITION: Adaptive filtering; contrast enhancement; competitive learning of recognition categories; adaptive vector quantization; self-organizing computational maps; statistical properties of adaptive weights; learning stability and causes of instability. INTRODUCTION TO ADAPTIVE RESONANCE THEORY: Absolutely stable recognition learning, role of learned top-down expectations; attentional priming; matching by 2/3 Rule; adaptive search; self-controlled hypothesis testing; direct access to globally optimal recognition code; control of categorical coarseness by attentional vigilance; comparison with relevant behavioral and brain data to emphasize biological basis of ART computations. ANALYSIS OF ART 1: Computational analysis of ART 1 architecture for self-organized real-time hypothesis testing, learning, and recognition of arbitrary sequences of binary input patterns. AFTERNOON SESSION (PROFESSOR CARPENTER) ANALYSIS OF ART 2: Computational analysis of ART 2 architecture for self-organized real-time hypothesis testing, learning, and recognition for arbitrary sequences of analog or binary input patterns. ANALYSIS OF ART 3: Computational analysis of ART 3 architecture for self-organized real-time hypothesis testing, learning, and recognition within distributed network hierarchies; role of chemical transmitter dynamics in forming a memory representation distinct from short-term memory and long-term memory; relationships to brain data concerning neuromodulators and synergetic ionic and transmitter interactions. SELF-ORGANIZATION OF INVARIANT PATTERN RECOGNITION CODES: Computational analysis of self-organizing ART architectures for recognizing noisy imagery undergoing changes in position, rotation, and size. NEOCOGNITION: Recognition and completion of images by hierarchical bottom-up filtering and top-down attentive feedback. EVENING SESSION: DISCUSSIONS WITH TUTORS MAY 9, 1990 ----------- MORNING SESSION (PROFESSORS GROSSBERG & MINGOLLA) VISION AND IMAGE PROCESSING: Introduction to Boundary Contour System for emergent segmentation and Feature Contour System for filling-in after compensation for variable illumination; image compression, orthogonalization, and reconstruction; multidimensional filtering, multiplexing, and fusion; coherent boundary detection, regularization, self-scaling, and completion; compensation for variable illumination sources, including artificial sensors (infrared sensors, laser radars); filling-in of surface color and form; 3-D form from shading, texture, stereo, and motion; parallel processing of static form and moving form; motion capture and induced motion; synthesis of static form and motion form representations. AFTERNOON SESSION (PROFESSORS BULLOCK, COHEN, & GROSSBERG) ADAPTIVE SENSORY-MOTOR CONTROL AND ROBOTICS: Overview of recent progress in adaptive sensory-motor control and related robotics research. Reaching to, grasping, and transporting objects of variable mass and form under visual guidance in a cluttered environment will be used as a target behavioral competence to clarify subproblems of real-time adaptive sensory-motor control. The balance of the tutorial will be spent detailing neural network modules that solve various subproblems. Topics include: Self-organizing networks for real-time control of eye movements, arm movements, and eye-arm coordination; learning of invariant body-centered target position maps; learning of intermodal associative maps; real-time trajectory formation; adaptive vector encoders; circular reactions between action and sensory feedback; adaptive control of variable speed movements; varieties of error signals; supportive behavioral and neural data; inverse kinematics; automatic compensation for unexpected perturbations; independent adaptive control of force and position; adaptive gain control by cerebellar learning; position-dependent sampling from spatial maps; predictive motor planning and execution. SPEECH PERCEPTION AND PRODUCTION: Hidden Markov models; self-organization of speech perception and production codes; eighth nerve Average Localized Synchrony Response; phoneme recognition by back propagation, time delay networks, and vector quantization. MAY 10, 1990 ------------ MORNING SESSION (PROFESSORS COHEN, GROSSBERG, & MERRILL) SPEECH PERCEPTION AND PRODUCTION: Disambiguation of coarticulated vowels and consonants; dynamics of working memory; multiple-scale adaptive coding by masking fields; categorical perception; phonemic restoration; contextual disambiguation of speech tokens; resonant completion and grouping of noisy variable-rate speech streams. REINFORCEMENT LEARNING AND PREDICTION: Recognition learning, reinforcement learning, and recall learning are the 3 R's of neural network learning. Reinforcement learning clarifies how external events interact with internal organismic requirements to trigger learning processes capable of focussing attention upon and generating appropriate actions towards motivationally desired goals. A neural network model will be derived to show how reinforcement learning and recall learning can self-organize in response to asynchronous series of significant and irrelevant events. These mechanisms also control selective forgetting of memories that are no longer predictive, adaptive timing of behavioral responses, and self-organization of goal directed problem solvers. AFTERNOON SESSION (PROFESSORS GROSSBERG & MERRILL AND DR. HECHT-NIELSEN) REINFORCEMENT LEARNING AND PREDICTION: Analysis of drive representations, adaptive critics, conditioned reinforcers, role of motivational feedback in focusing attention on predictive data; attentional blocking and unblocking; adaptively timed problem solving; synthesis of perception, recognition, reinforcement, recall, and robotics mechanisms into a total neural architecture; relationship to data about hypothalamus, hippocampus, neocortex, and related brain regions. RECENT DEVELOPMENTS IN THE NEUROCOMPUTER INDUSTRY: This two-hour guest tutorial will provide an overview of the growth and prospects of the burgeoning neurocomputer industry by one of its most important leaders. EVENING SESSION: DISCUSSIONS WITH TUTORS MAY 11, 1990 ------------ MORNING SESSION (DR. FAGGIN) VLSI IMPLEMENTATION OF NEURAL NETWORKS: This is a four-hour self-contained tutorial on the application and development of VLSI techniques for creating compact real-time chips embodying neural network designs for applications in technology. Review of neural networks from a hardware implementation perspective; hardware requirements and alternatives; dedicated digital implementation of neural networks; neuromorphic design methodology using VLSI CMOS technology; applications and performance of neuromorphic implementations; comparison of neuromorphic and digital hardware; future prospectus. ---------------------------------------------------------------------------- COURSE FACULTY FROM BOSTON UNIVERSITY ------------------------------------- STEPHEN GROSSBERG, Wang Professor of CNS, as well as Professor of Mathematics, Psychology, and Biomedical Engineering, is one of the world's leading neural network pioneers and most versatile neural architects; Founder and 1988 President of the International Neural Network Society (INNS); Founder and Co-Editor-in-Chief of the INNS journal "Neural Networks"; an editor of the journals "Neural Computation", "Cognitive Science", and "IEEE Expert"; Founder and Director of the Center for Adaptive Systems; General Chairman of the 1987 IEEE First International Conference on Neural Networks (ICNN); Chief Scientist of Hecht-Nielsen Neurocomputer Company (HNC); and one of the four technical consultants to the national DARPA Neural Network Study. He is author of 200 articles and books about neural networks, including "Neural Networks and Natural Intelligence" (MIT Press, 1988), "Neural Dynamics of Adaptive Sensory-Motor Control" (with Michael Kuperstein, Pergamon Press, 1989), "The Adaptive Brain, Volumes I and II" (Elsevier/North-Holland, 1987), "Studies of Mind and Brain" (Reidel Press, 1982), and the forthcoming "Pattern Recognition by Self-Organizing Neural Networks" (with Gail Carpenter). GAIL CARPENTER is Professor of Mathematics and CNS; Co-Director of the CNS Graduate Program; 1989 Vice President of the International Neural Network Society (INNS); Organization Chairman of the 1988 INNS annual meeting; Session Chairman at the 1989 and 1990 IEEE/INNS International Joint Conference on Neural Networks (IJCNN); one of four technical consultants to the national DARPA Neural Network Study; editor of the journals "Neural Networks", "Neural Computation", and "Neural Network Review"; and a member of the scientific advisory board of HNC. A leading neural architect, Carpenter is especially well-known for her seminal work on developing the adaptive resonance theory architectures (ART 1, ART 2, ART 3) for adaptive pattern recognition. MICHAEL COHEN, Associate Professor of Computer Science and CNS, is a leading architect of neural networks for content addressable memory (Cohen-Grossberg model), vision (Feature Contour System), and speech (Masking Fields); editor of "Neural Networks"; Session Chairman at the 1987 ICNN, and the 1989 IJCNN; and member of the DARPA Neural Network Study panel on Simulation/Emulation Tools and Techniques. ENNIO MINGOLLA, Assistant Professor of Psychology and CNS, is holder of one of the first patented neural network architectures for vision and image processing (Boundary Contour System); Co-Organizer of the 3rd Workshop on Human and Machine Vision in 1985; editor of the journals "Neural Networks" and "Ecological Psychology"; member of the DARPA Neural Network Study panel of Adaptive Knowledge Processing; consultant to E.I. duPont de Nemours, Inc.; Session Chairman for vision and image processing at the 1987 ICNN, and the 1988 INNS meetings. DANIEL BULLOCK, Assistant Professor of Psychology and CNS, is developer of neural network models for real-time adaptive sensory-motor control of arm movements and eye-arm coordination, notably the VITE and FLETE models for adaptive control of multi-joint trajectories; editor of "Neural Networks"; Session Chairman for adaptive sensory-motor control and robotics at the 1987 ICNN and the 1988 INNS meetings; invited speaker at the 1990 IJCNN. JOHN MERRILL, Assistant Professor of Mathematics and CNS, is developing neural network models for adaptive pattern recognition, speech recognition, reinforcement learning, and adaptive timing in problem solving behavior, after having received his Ph.D. in mathematics from the University of Wisconsin at Madison, and completing postdoctoral research in computer science and linguistics at Indiana University. GUEST LECTURERS --------------- FEDERICO FAGGIN is co-founder and president of Synaptics, Inc. Dr. Faggin developed the Silicon Gate Technology at Fairchild Semiconductor. He also designed the first commercial circuit using Silicon Gate Technology: the 3708, an 8-bit analog multiplexer. At Intel Corporation he was responsible for designing what was to become the first microprocessor---the 4000 family, also called MCS-4. He and Hal Feeney designed the 8008, the first 8-bit microprocessor introduced in 1972, and later Faggin conceived the 8080 and with M. Shima designed it. The 8080 was the first high-performance 8-bit microprocessor. At Zilog Inc., Faggin conceived the Z80 microprocessor family and directed the design of the Z80 CPU. Faggin also started Cygnet Technologies, which developed a voice and data communication peripheral for the personal computer. In 1986 Faggin co-founded Synaptics Inc., a company dedicated to the creation of a new type of VLSI hardware for artificial neural networks and other machine intelligence applications. Faggin is the recipient of the 1988 Marconi Fellowship Award for his contributions to the birth of the microprocessor. ROBERT HECHT-NIELSEN is co-founder and chairman of the Board of Directors of Hecht-Nielsen Neurocomputer Corporation (HNC), a pioneer in neurocomputer technology and the application of neural networks, and a recognized leader in the field. Prior to the formation of HNC, he founded and managed the neurocomputer development and neural network applications at TRW (1983--1986) and Motorola (1979--1983). He has been active in neural network technology and neurocomputers since 1961 and earned his Ph.D. in mathematics in 1974. He is currently a visiting lecturer in the Electrical Engineering Department at the University of California at San Diego, and is the author of influential technical reports and papers on neurocomputers, neural networks, pattern recognition, signal processing algorithms, and artificial intelligence. MICHAEL JORDAN is an Assistant Professor of Brain and Cognitive Sciences at MIT. One of the key developers of the recurrent back propagation algorithms, Professor Jordan's research is concerned with learning in recurrent networks and with the use of networks as forward models in planning and control. His interest in interdisciplinary research on neural networks is founded in his training for a Bachelors degree in Psychology, a Masters degree in Mathematics, and a Ph.D. in Cognitive Science from the University of California at San Diego. He was a postdoctoral researcher in Computer Science at the University of Massachusetts at Amherst before assuming his present position at MIT. ---------------------------------------------------------- REGISTRATION FEE: Regular attendee--$950; full-time student--$250. Registration fee includes five days of tutorials, course notebooks, one reception, five continental breakfasts, five lunches, four dinners, daily morning and afternoon coffee service, evening discussion sessions with leading neural architects. REGISTRATION: To register by telephone with VISA or MasterCard call (508) 649-9731 between 9:00AM--5:00PM (EST). To register by FAX, fill out the registration form and FAX back to (508) 649-6926. To register by mail, complete the registration form and mail with you full form of payment as directed. Make check payable in U.S. dollars to "Boston University". See below for Registration Form. To register by electronic mail, use the address "rosenber at bu-tyng.bu.edu". On-site registration on a space-available basis will take place from 2:00--7:00PM on Sunday, May 6 and from 7:00--8:00AM on Monday, May 7, 1990. A RECEPTION will be held from 4:00--7:00PM on Sunday, May 6. LECTURES begin at 8:00AM on Monday, May 7 and conclude at 12:30PM on Friday, May 11. STUDENT FELLOWSHIPS supporting travel, registration, and lodging for the Course and the Research Conference are available to full-time graduate students in a PhD program. Applications must be postmarked by March 1, 1990. Send curriculum vitae, a one-page essay describing your interest in neural networks, and a letter from a faculty advisor to: Student Fellowships, Neural Networks Course, Wang Institute of Boston University, 72 Tyng Road, Tyngsboro, MA 01879. CNS FELLOWSHIP FUND: Net revenues from the course will endow fellowships for Ph.D. candidates in the CNS Graduate Program. Corporate and individual gifts to endow CNS Fellowships are also welcome. Please write: Cognitive and Neural Systems Fellowship Fund, Center for Adaptive Systems, Boston University, 111 Cummington Street, Boston, MA 02215. ------------------------------------------------------------------------------ REGISTRATION FOR COURSE AND RESEARCH CONFERENCE Course: Neural Network Foundations and Applications, May 6--11, 1990 Research Conference: Neural Networks for Automatic Target Recognition, May 11--13, 1990 NAME: _________________________________________________________________ ORGANIZATION (for badge): _____________________________________________ MAILING ADDRESS: ______________________________________________________ ______________________________________________________ CITY/STATE/COUNTRY: ___________________________________________________ POSTAL/ZIP CODE: ______________________________________________________ TELEPHONE(S): _________________________________________________________ COURSE RESEARCH CONFERENCE ------ ------------------- [ ] regular attendee $950 [ ] regular attendee $90 [ ] full-time student $250 [ ] full-time student $70 (limited number of spaces) (limited number of spaces) [ ] Gift to CNS Fellowship Fund TOTAL PAYMENT: $________ FORM OF PAYMENT: [ ] check or money order (payable in U.S. dollars to Boston University) [ ] VISA [ ] MasterCard Card Number: ______________________________________________ Expiration Date: ______________________________________________ Signature: ______________________________________________ Please complete and mail to: Neural Networks Wang Institute of Boston University 72 Tyng Road Tyngsboro, MA 01879 USA To register by telephone, call: (508) 649-9731. HOTEL RESERVATIONS: Room blocks have been reserved at 3 hotels near the Wang Institute. Hotel names, rates, and telephone numbers are listed below. A shuttle bus will take attendees to and from the hotels for the Course and Research Conference. Attendees should make their own reservations by calling the hotel. The special conference rate applies only if you mention the name and dates of the meeting when making the reservations. Sheraton Tara Red Roof Inn Stonehedge Inn Nashua, NH Nashua, NH Tyngsboro, MA (603) 888-9970 (603) 888-1893 (508) 649-4342 $70/night+tax $39.95/night+tax $89/night+tax The hotels in Nashua are located approximately 5 miles from the Wang Institute. A shuttle bus will be provided. ------------------------------------------------------------------------------- From rudnick at cse.ogc.edu Fri Nov 10 13:38:27 1989 From: rudnick at cse.ogc.edu (Mike Rudnick) Date: Fri, 10 Nov 89 10:38:27 PST Subject: fault behaviour of NNs Message-ID: <8911101838.AA17796@cse.ogc.edu> I'm searching for references to work on the analysis and performance of neural net models with/under fault conditions (including VLSI models/implementations), eg, missing connections, frozen weights and activations, etc. I will post a summary of references received. As I'm doing this as a part of my search for a dissertation topic, I'm interested in making contact with anyone actively doing research in this area. Thanks, Mike From Dave.Touretzky at B.GP.CS.CMU.EDU Fri Nov 10 19:33:33 1989 From: Dave.Touretzky at B.GP.CS.CMU.EDU (Dave.Touretzky@B.GP.CS.CMU.EDU) Date: Fri, 10 Nov 89 19:33:33 EST Subject: tech report announcement Message-ID: <922.626747613@DST.BOLTZ.CS.CMU.EDU> A Connectionist Implementation of Cognitive Phonology Deirdre W. Wheeler [1] David S. Touretzky [2] [1] Department of Linguistics [2] School of Computer Science University of Pittsburgh Carnegie Mellon University Pittsburgh, PA 15260 Pittsburgh, PA 15213 Technical report number CMU-CS-89-144 ABSTRACT This paper reports on an initial implementation of Lakoff's theory of cognitive phonology in a connectionist network. Standard generative phonological theories require serial application of rules, which results in derivations with numerous intermediate states. This is incompatible with the connectionist goals of psychological and biological plausibility, and may also hinder learnability. Lakoff's theory of cognitive phonology offers a solution to some of these problems by providing an alternative way to think about derivations and ordered rules, and by eliminating the need for right-to-left iterative rule application. On the other hand, Lakoff's proposal presents certain computational difficulties due to its appeal to Harmony Theory. We present a reformulation of cognitive phonology using a novel clustering mechanism that completely eliminates iteration and permits an efficient feed-forward implementation. An earlier version of this paper was presented at the Berkeley Workshop on Constraints vs. Rules in Phonology, May 26-27, 1989. Note: the version of the system described in this report is considerably more refined than the version in the 1989 Cognitive Science Conference paper and the "Rules and Maps" tech report. The following abstract for a talk I've been giving recently at various institutions may better explain why this material should be of interest to cognitive scientists, not just linguists: A COMPUTATIONAL BASIS FOR PHONOLOGY Phonology is the study of the sound patterns of a language. It includes processes such as nasal assimilation, vowel harmony, tone shifting, and syllabification. The phonological structure of human languages is intricate, but it is also highly constrained and stunningly regular. The easy observability of phonological processes, their discrete, symbolic nature, and their rapid acquisition by very young children suggest that this may be a good domain in which to explore issues of rules and symbolic representations in the brain. In this talk I will give a brief sketch of George Lakoff's new theory of cognitive phonology, in which sequential derivations are eliminated by having all rules apply in parallel. I will then describe how our attempt to construct a connectionist implementation of the theory led us to revise it in significant ways. The architecture we developed resulted in a novel prediction of a constraint on insertion processes. Subsequent consultations with expert phonologists have so far confirmed this prediction. If correct, it represents the first step toward our long term goal of developing a computational explanation for why phonology looks the way it does. ---------------------------------------------------------------- HOW TO ORDER THIS REPORT: Copies of the report are available by writing to Ms. Catherine Copetas, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213. Email requests may be sent to copetas at cs.cmu.edu. Ask for report number CMU-CS-89-144. There is no charge for this report. From koch%HAMLET.BITNET at VMA.CC.CMU.EDU Fri Nov 10 20:40:38 1989 From: koch%HAMLET.BITNET at VMA.CC.CMU.EDU (Christof Koch) Date: Fri, 10 Nov 89 17:40:38 PST Subject: NEURAL NET SUMMER SCHOOL AT WANG INSTITUTE In-Reply-To: Your message <8911100557.AA24680@bucasb.bu.edu> dated 10-Nov-1989 Message-ID: <891110173624.2a605c2f@Hamlet.Caltech.Edu> Given that you guys have solved problems related to Brain Science, we would be infinitely grateful if you could provide us with a condensed version of the final result. Maybe ``42''? Signed: Not one of the World's Leading Expert's From dfausett at zach.fit.edu Sat Nov 11 10:47:51 1989 From: dfausett at zach.fit.edu ( Donald W. Fausett) Date: Sat, 11 Nov 89 10:47:51 EST Subject: E-mail Distribution List Message-ID: <8911111547.AA06552@zach.fit.edu> Please add me to your connectionists e-mail distribution list. Thanks. -- Don Fausett dfausett at zach.fit.edu From dfausett at zach.fit.edu Sat Nov 11 10:45:43 1989 From: dfausett at zach.fit.edu ( Donald W. Fausett) Date: Sat, 11 Nov 89 10:45:43 EST Subject: E-mail Distribution List Message-ID: <8911111545.AA06517@zach.fit.edu> Please add me to your connectionists e-mail distribution list. Thanks. -- Don Fausett dfausett at zach.fit.edu From khaines at GALILEO.ECE.CMU.EDU Sun Nov 12 16:35:26 1989 From: khaines at GALILEO.ECE.CMU.EDU (Karen Haines) Date: Sun, 12 Nov 89 16:35:26 EST Subject: IJCNN - Request for Volunteers Message-ID: <8911122135.AA02488@galileo.ece.cmu.edu> *************************************************************************** IJCNN - REQUEST FOR VOLUNTEERS *************************************************************************** This is the final call for volunteers to help at the IJCNN conference, to be held at the Omni Shorham Hotel in Washington D.C., on January 15-19, 1990. Full admittance to the conference and a copy of the proceedings is offered in exchange for your assistance throughout the conference. I would like to point out that student registration does not include proceedings. In general, each volunteer is expected to work one shift each day of the conference. Hours are approximately: AM shift - 7:00 am - Noon PM shift - Noon - 5:00 pm In addition, assistance may be required for the social events. There a re a few positions available, but I suggest that if you are interested you conatact ma as soon as possible. Below is a list of specific volunteer events. =================================== VOLUNTEER SCHEDULE OF EVENTS =================================== Sunday, January 14, 1990 ------------------------- 10am - 2pm Volunteer Shift Signup Registration time is based upon and Registration commitment date (i.e those whose commit earlier will get first choice ) 6pm - 7pm General Meeting **** Mandatory Meeting **** 7pm - 9pm Volunteer Welcome Party To sign up please contact: Karen Haines - Volunteer Coordinator 3138 Beechwood Blvd. Pittsburgh, PA 15217 office: (412) 268-3304 message: (412) 422-6026 email: khaines at galileo.ece.cmu.edu or, Nina Kowalski - Assistant Volunteer Coordinator 209 W. 29th St. FLR 2 Baltimore, MD 21211 message: (301) 889-0587 email: nina at alpha.ece.jhu.edu If you have further questions, please feel free to contact me. Thank you, Karen Haines IJCNN Volunteer Coordinator From Connectionists-Request at CS.CMU.EDU Mon Nov 13 11:29:54 1989 From: Connectionists-Request at CS.CMU.EDU (Connectionists-Request@CS.CMU.EDU) Date: Mon, 13 Nov 89 11:29:54 EST Subject: apparent FTP problems Message-ID: <846.626977794@B.GP.CS.CMU.EDU> When logging in to B.GP.CS.CMU.EDU via anonymous FTP, you may receive the following "error" message: Connected to B.GP.CS.CMU.EDU. 220 B.GP.CS.CMU.EDU FTP server (Version 4.105 of 18-Jun-89 19:22) ready. Name: anonymous 331 Guest login ok, send ident as password. Password: 230-[ AFS authentication failed: no password ] 230 Filenames can not have '/..' in them. ftp> Don't worry about it - you should still have access to the directories /usr/connect/connectionists/archives /usr/connect/connectionists/bibliographies If you still have trouble, send mail to "Connectionists-Request at cs.cmu.edu". David Plaut Connectionists-Request at cs.cmu.edu (ARPAnet) From eric at mcc.com Mon Nov 13 14:11:01 1989 From: eric at mcc.com (Eric Hartman) Date: Mon, 13 Nov 89 13:11:01 CST Subject: "Universal Approximators" Message-ID: <8911131911.AA04381@bird.aca.mcc.com> We agree with George Cybenko's response to John Kolnen and have some additional comments: The interest in sigmoidal and gaussian functions stems at least in part from their biological relevance; they are (much) more relevant than polynomials. Showing that neural networks serve as universal approximators is much like having an existence proof for a differential equation: you know the answer exists, but the theorem does not tell you how to find it. For that reason it is an important question in principle, but not necessarily in practice. Note that the answer could just as easily have been negative: there are several classes of functions that do not serve as universal approximators. (Take, e.g., functions from C_k trying to approximate functions from C_k+1.) If the answer was negative for neural networks, then we would have to think hard about why neural networks work so well. Jim Keeler and Eric Hartman From merrill at bucasb.BU.EDU Mon Nov 13 14:49:37 1989 From: merrill at bucasb.BU.EDU (John Merrill) Date: Mon, 13 Nov 89 14:49:37 EST Subject: "Universal Approximators" Message-ID: <8911131949.AA00642@bucasb.bu.edu> JK/EH> Jim Keeler and Eric Hartman JK/EH> The interest in sigmoidal and gaussian functions stems at JK/EH> least in part from their biological relevance; they are JK/EH> (much) more relevant than polynomials. That's honestly a debatable point. It may indeed be that sigmoids are more "biologically natural" than polynomials, but their use in a discrete-time system makes the difference hard to establish. The fact is that "real" neurons perform computations which are far more complicated than any kind of "take a dot product and squash" system; indeed, all the neurobiological evidence indicates that they do no such thing. JK/EH> Showing that neural networks serve as universal approximators JK/EH> is much like having an existence proof for a differential equation: JK/EH> you know the answer exists, but the theorem does not tell you JK/EH> how to find it. For that reason it is an important question JK/EH> in principle, but not necessarily in practice. By that standard, once any one universal approximator theorem had been established, no other would possess even the faintest semblance of interest. Since Borel approximation (in the sense of either l_2 or l_\infty) is easy to establish with sigmoidal networks alone, it seems to me that the results concerning (eg.) radial basis function would be hard to swallow. In fact, the radial basis function theorem gives somewhat better bounds on the *number* of intermediate nodes necessary, and, as a consequence, indicates that if you're only interested in approximation, you want to use RBF's. --- John From ST401843%BROWNVM.BITNET at VMA.CC.CMU.EDU Mon Nov 13 17:14:14 1989 From: ST401843%BROWNVM.BITNET at VMA.CC.CMU.EDU (thanasis kehagias) Date: Mon, 13 Nov 89 17:14:14 EST Subject: No subject Message-ID: ok, several people tried to ftp the recurrent nets bibliography and it did not work. i will send it out electronically. if somebody else tried and was able to ftp, like the manager's message implies, can you please let me know? thanasis From paul at NMSU.Edu Mon Nov 13 23:37:26 1989 From: paul at NMSU.Edu (paul@NMSU.Edu) Date: Mon, 13 Nov 89 21:37:26 MST Subject: No subject Message-ID: <8911140437.AA21121@NMSU.Edu> PRAGMATICS AI PRAGMATICS AI PRAGMATICS AI PRAGMATICS AI PRAGMATICS AI PRAGMATI PRAGMATICS AI PRAGMATICS AI PRAGMATICS AI PRAGMATICS AI PRAGMATICS AI PRAGMATI TO: Newsgroups: ar.ailist comp.ai.digest comp.ai.edu comp.ai.neural-nets comp.ai.nlang-know-rep comp.ai.shells comp.ai.vision news.announce.conferences news.announce.important sci.lang soc.celtic.culture Networks: AI List AI-NL NL-KR IRL-NET The Emigrant psy-net con-net nl-kr Local: NMSU CS NMSU CRL Journals: SIGART AISB Ack: SIGART RMSAI U S WEST CC: RMCAI-90 Program Committee RMCAI-90 Program Chairperson RMCAI-90 Local Arrangements Chairperson RMCAI-90 Organizing Committee RMCAI-90 Invited Speakers FROM: Organizing Committee RMCAI-90: Paul Mc Kevitt Yorick Wilks Research Scientist Director CRL CRL SUBJECT: Please post the following in your Laboratory/Department/Journal: Cut--------------------------------------------------------------------------- SUBJECT: Please post the following in your Laboratory/Department/Journal: CALL FOR PAPERS Pragmatics in Artificial Intelligence 5th Rocky Mountain Conference on Artificial Intelligence (RMCAI-90) Las Cruces, New Mexico, USA, June 28-30, 1990 PRAGMATICS PROBLEM: The problem of pragmatics in AI is one of developing theories, models, and implementations of systems that make effective use of contextual information to solve problems in changing environments. CONFERENCE GOAL: This conference will provide a forum for researchers from all subfields of AI to discuss the problem of pragmatics in AI. The implications that each area has for the others in tackling this problem are of particular interest. ACKNOWLEDGEMENTS: In cooperation with: Association for Computing Machinery (ACM) (pending approval) Special Interest Group in Artificial Intelligence (SIGART) (pending approval) U S WEST Advanced Technologies and the Rocky Mountain Society for Artificial Intelligence (RMSAI) With grants from: Association for Computing Machinery (ACM) Special Interest Group in Artificial Intelligence (SIGART) U S WEST Advanced Technologies and the Rocky Mountain Society for Artificial Intelligence (RMSAI) THE LAND OF ENCHANTMENT: Las Cruces, lies in THE LAND OF ENCHANTMENT (New Mexico), USA and is situated in the Rio Grande Corridor with the scenic Organ Mountains overlooking the city. The city is close to Mexico, Carlsbad Caverns, and White Sands National Monument. There are a number of Indian Reservations and Pueblos in the Land Of Enchantment and the cultural and scenic cities of Taos and Santa Fe lie to the north. New Mexico has an interesting mixture of Indian, Mexican and Spanish culture. There is quite a variation of Mexican and New Mexican food to be found here too. GENERAL INFORMATION: The Rocky Mountain Conference on Artificial Intelligence is a major regional forum in the USA for scientific exchange and presentation of AI research. The conference emphasizes discussion and informal interaction as well as presentations. The conference encourages the presentation of completed research, ongoing research, and preliminary investigations. Researchers from both within and outside the region are invited to participate. Some travel awards will be available for qualified applicants. FORMAT FOR PAPERS: Submitted papers should be double spaced and no more than 5 pages long. E-mail versions will not be accepted. Send 3 copies of your paper to: Paul Mc Kevitt, Program Chairperson, RMCAI-90, Computing Research Laboratory (CRL), Dept. 3CRL, Box 30001, New Mexico State University, Las Cruces, NM 88003-0001, USA. DEADLINES: Paper submission: March 1st, 1990 Pre-registration: April 1st, 1990 Notice of acceptance: May 1st, 1990 Final papers due: June 1st, 1990 LOCAL ARRANGEMENTS: Jennifer Griffiths, Local Arrangements Chairperson, RMCAI-90. (same postal address as above). INQUIRIES: Inquiries regarding conference brochure and registration form should be addressed to the Local Arrangements Chairperson. Inquiries regarding the conference program should be addressed to the program Chairperson. Local Arrangements Chairperson: E-mail: INTERNET: rmcai at nmsu.edu Phone: (+ 1 505)-646-5466 Fax: (+ 1 505)-646-6218. Program Chairperson: E-mail: INTERNET: paul at nmsu.edu Phone: (+ 1 505)-646-5109 Fax: (+ 1 505)-646-6218. TOPICS OF INTEREST: You are invited to submit a research paper addressing Pragmatics in AI , with any of the following orientations: Philosophy, Foundations and Methodology Knowledge Representation Neural Networks and Connectionism Genetic Algorithms, Emergent Computation, Nonlinear Systems Natural Language and Speech Understanding Problem Solving, Planning, Reasoning Machine Learning Vision and Robotics Applications INVITED SPEAKERS: The following researchers have agreed to speak at the conference (a number of others have been invited): Martin Casdagli, Los Alamos National Laboratory USA (Dynamical systems, Artificial neural nets, Applications) Arthur Cater, University College Dublin IRELAND (Robust Parsing) James Martin, University of Colorado at Boulder USA (Metaphor and Context) Derek Partridge, University of Exeter UK (Connectionism, Learning) Philip Stenton, Hewlett Packard UK (Natural Language Interfaces) PROGRAM COMMITTEE: John Barnden, New Mexico State University (Connectionism, Beliefs, Metaphor processing) Hans Brunner, U S WEST Advanced Technologies (Natural language interfaces, Dialogue interfaces) Martin Casdagli, Los Alamos National Laboratory (Dynamical systems, Artificial neural nets, Applications) Mike Coombs, New Mexico State University (Problem solving, Adaptive systems, Planning) Thomas Eskridge, Lockheed Missile and Space Co. (Analogy, Problem solving) Chris Fields, New Mexico State University (Neural networks, Nonlinear systems, Applications) Roger Hartley, New Mexico State University (Knowledge Representation, Planning, Problem Solving) Paul Mc Kevitt, New Mexico State University (Natural language interfaces, Dialogue modeling) Joe Pfeiffer, New Mexico State University (Computer Vision, Parallel architectures) Keith Phillips, University of Colorado at Colorado Springs (Computer vision, Mathematical modeling) Yorick Wilks, New Mexico State University (Natural language processing, Knowledge representation) Scott Wolff, U S WEST Advanced Technologies (Intelligent tutoring, User interface design, Cognitive modeling) REGISTRATION: Pre-Registration: Professionals $50.00; Students $30.00 (Pre-Registration cutoff date is April 1st 1990) Registration: Professionals $70.00; Students $50.00 (Copied proof of student status is required). Registration form (IN BLOCK CAPITALS). Enclose payment (personal checks and Eurochecks accepted). Send to the following address: Jennifer Griffiths, Local Arrangements Chairperson, RMCAI-90 Computing Research Laboratory Dept. 3CRL, Box 30001, NMSU Las Cruces, NM 88003-0001, USA. Name:_______________________________ E-mail_____________________________ Phone__________________________ Affiliation: ____________________________________________________ Fax: ____________________________________________________ Address: ____________________________________________________ ____________________________________________________ ____________________________________________________ COUNTRY__________________________________________ Organizing Committee RMCAI-90: Paul Mc Kevitt Yorick Wilks Research Scientist Director CRL CRL cut------------------------------------------------------------------------ From yann at lesun.att.com Tue Nov 14 10:10:15 1989 From: yann at lesun.att.com (Yann le Cun) Date: Tue, 14 Nov 89 10:10:15 -0500 Subject: "Universal Approximators" In-Reply-To: Your message of Mon, 13 Nov 89 14:49:37 -0500. Message-ID: <8911141510.AA08431@lesun.> John Merrill says: > In fact, the radial basis function theorem gives somewhat better > bounds on the *number* of intermediate nodes necessary, and, as a > consequence, indicates that if you're only interested in > approximation, you want to use RBF's. But you can build RBF's (or rather, multidimensional "bumps") with two layers of sigmoid units. And you just need O(n) units for n bumps. A third (linear) layer can sum up the bumps. So i guess the bound on the number of intermediate nodes is the same in both cases. You can prove the universality of 2 hidden layer structures that way (and the proof is a lot simpler than with 1 hidden layer). Yann Le Cun From merrill at bucasb.BU.EDU Tue Nov 14 12:30:26 1989 From: merrill at bucasb.BU.EDU (John Merrill) Date: Tue, 14 Nov 89 12:30:26 EST Subject: "Universal Approximators" In-Reply-To: Yann le Cun's message of Tue, 14 Nov 89 10:10:15 -0500 <8911141510.AA08431@lesun.> Message-ID: <8911141730.AA11580@bucasb.bu.edu> YlC> Yann le Cun JM> John Merrill (me) JM> In fact, the radial basis function theorem gives somewhat better JM> bounds on the *number* of intermediate nodes necessary, and, as a JM> consequence, indicates that if you're only interested in JM> approximation, you want to use RBF's. YlC> But you can build RBF's (or rather, multidimensional "bumps") YlC> with two layers of sigmoid units. And you just need O(n) units YlC> for n bumps. A third (linear) layer can sum up the bumps. YlC> So i guess the bound on the number of intermediate nodes is the YlC> same in both cases. Not quite. If your input lies in ${\bf R}^k$, it takes at least k units (in the lower hidden level) to build a single k-dimensional bump in the upper hidden level.(*) As a consequence, the network that this argument gives is wildly more computationally demanding than the original RBF network, since it's got to have $nk^2$ edges between the input layer and the first hidden layer, as well as $nk$ more edges between the two hidden layers, for a total edge count of $n (k^2 + k)$, as compared to an edge count of $nk$ for the RBF network. If $k = 1000$, this is a factor of $1000$ more costly (in terms of edges; it's actually less than that, since each RBF node needs an extra few multiplies.) YlC> You can prove the universality of 2 hidden layer structures that YlC> way (and the proof is a lot simpler than with 1 hidden layer). Absolutely, and it's the one I would actually teach if I taught one at all. --- John (*) It might seem that one would need 2k such units; in fact, by using k-dimensional simplices as the bump-shapes, k units suffice. From sontag at fermat.rutgers.edu Tue Nov 14 16:29:52 1989 From: sontag at fermat.rutgers.edu (Eduardo Sontag) Date: Tue, 14 Nov 89 16:29:52 EST Subject: number of hidden neurons Message-ID: <8911142129.AA02052@fermat.rutgers.edu> In the context of the discussions about numbers of hidden neurons, it seems relevant to point out a short note of mine to appear in Neural Computation, Number 4, entitled: SIGMOIDS DISTINGUISH MORE EFFICIENTLY THAN HEAVISIDES I prove (this is the abstract): Every dichotomy on a $2k$-point set in $\R^N$ can be implemented by a neural net with a single hidden layer containing $k$ sigmoidal neurons. If the neurons were of a hardlimiter (Heaviside) type, $2k-1$ would be in general needed. So one can save half as many neurons using sigmoids, a fact which might not be totally obvious at first sight (but which is indeed easy to prove). The first paragraph of the intro is: The main point of this note is to draw attention to the fact mentioned in the title, that sigmoids have different recognition capabilities than hard-limitting nonlinearities. One way to exhibit this difference is through a worst-case analysis in the context of binary classification, and this is done here. Results can also be obtained in terms of VC dimension, and work is in progress in that regard. (Added note: now established that in 2-d, VC dimension of k-neuron nets is 4k. This is not written up yet, though. I also have an example of a family of boolean functions that can be computed with a fixed number of sigmoidal hidden units but which --I conjecture-- needs a growing number of Heavisides.) -eduardo Eduardo D. Sontag (Phone: (201)932-3072; dept.: (201)932-2390) sontag at fermat.rutgers.edu ...!rutgers!fermat.rutgers.edu!sontag sontag at pisces.bitnet From yann at lesun.att.com Tue Nov 14 17:15:25 1989 From: yann at lesun.att.com (Yann le Cun) Date: Tue, 14 Nov 89 17:15:25 -0500 Subject: "Universal Approximators" In-Reply-To: Your message of Tue, 14 Nov 89 12:30:26 -0500. Message-ID: <8911142215.AA08568@lesun.> John Merrill says: >If your input lies in ${\bf R}^k$, it takes at least k units >(in the lower hidden level) to build a single k-dimensional bump in >the upper hidden level True, although, as you say, it is easier with 2k units. >As a consequence, the network that this >argument gives is wildly more computationally demanding than the >original RBF network, since it's got to have $nk^2$ edges between the >input layer and the first hidden layer Not true, since each of these 2k units only needs 2 incoming weights (not k) one for the bias, and one coming from one of the inputs (*). thus the total number of edges is 6nk, just 6 times bigger than regular RBF's. It can even be better than that (almost 2nk) if your bumps are regularly spaced since they can share the first level units. And you can back-propagate through the whole thing. -- Yann Le Cun (*) you might want k incoming weights if you absolutely need to have non symetric and rotated RBF's, but otherwise 2 is enough From jagota at cs.Buffalo.EDU Wed Nov 15 13:25:22 1989 From: jagota at cs.Buffalo.EDU (Arun Jagota) Date: Wed, 15 Nov 89 13:25:22 EST Subject: Universal Approximator Results Message-ID: <8911151825.AA23769@sybil.cs.Buffalo.EDU> A question was raised some time back that since universal approximator results establish no new bounds on computability (one result is as good as another in a computability sense), what then is their significance. Don't such results for k-hidden layer networks show, additionally, that the represented function can be _evaluated_ on a point in it's domain in (k+1) inner-product + k hidden-layer function eval steps on a suitable abstract machine. Doesn't that provide a strong result on the running time, as compared with Church's thesis which says, that any algorithm (effective procedure) can be programmed on a Turing m/c but doesn't put a bound on the running time. Arun Jagota jagota at cs.buffalo.edu [I don't wish to be accused of starting a fresh round of value-less discussions (if so perceived), so I prefer receiving responses by mail. I suggest using 'mail' instead of 'R' or 'r'] From tmb at ai.mit.edu Wed Nov 15 14:02:46 1989 From: tmb at ai.mit.edu (Thomas M. Breuel) Date: Wed, 15 Nov 89 14:02:46 EST Subject: Universal Approximator Results In-Reply-To: <8911151825.AA23769@sybil.cs.Buffalo.EDU> Message-ID: <8911151902.AA09129@rice-chex> Arun Jagota writes: > A question was raised some time back that since universal approximator > results establish no new bounds on computability (one result is as good > as another in a computability sense), what then is their significance. > Don't such results for k-hidden layer networks show, additionally, > that the represented function can be _evaluated_ on a point in it's > domain in > > (k+1) inner-product + k hidden-layer function eval > steps on a suitable abstract machine. > > Doesn't that provide a strong result on the running time, > as compared with Church's thesis which says, that any algorithm > (effective procedure) can be programmed on a Turing m/c but doesn't > put a bound on the running time. The claim is still true: "computability" says nothing about time complexity, space complexity, or parallel complexity. Therefore, from a computability point of view, it makes no difference whatsoever whether you prove that something is computable via networks or via a Turing machine, even if the Turing machine takes much longer. There are two other issues here: the "universal approximator" results talk about approximating a real function, not computing a real function, and, also, all the arithmetic here is real arithmetic, so a comparison between complexity on a Turing machine and the arithmetic complexity on the network is non-trivial. Thomas. From tmb at ai.mit.edu Wed Nov 15 14:06:56 1989 From: tmb at ai.mit.edu (Thomas M. Breuel) Date: Wed, 15 Nov 89 14:06:56 EST Subject: Universal Approximator Results In-Reply-To: <8911151825.AA23769@sybil.cs.Buffalo.EDU> Message-ID: <8911151906.AA09140@rice-chex> Arun Jagota writes: > [...] Oops--my reply should not have gone to the whole list. Apologies. (I would have worded it a little more carefully if I had intended to send it to the whole list). From kawahara at av-convex.ntt.jp Thu Nov 16 14:52:41 1989 From: kawahara at av-convex.ntt.jp (Hideki KAWAHARA) Date: Fri, 17 Nov 89 04:52:41+0900 Subject: News on JNNS(Japanese Neural Network Society) Message-ID: <8911161952.AA02921@av-convex.ntt.jp> -------------------------- JNNS (Japanese Neural Network Society) have delivered its first newsletter and started a mailing list -------------------------- Japanese Neural Network society, which was founded in July 1989, have delivered its first newsletter on 14 November 1989. Prof. Shiro Usui of Toyohasi University of Technology, who is in charge of editor in chief have also started a mailing list to encouredge discussions among active researchers in Japan. Prof. Usui and I would like to introduce the connectionists mailing list to JNNS's mailing list and to quit delivering to BBORD eventually. Electronic communications in Japan is still in its infancy. JUNET, the largest one, is a volunteer based (mainly UUCP) network. However, the number of researchers who are accessible to some electronic communication systems is increasing rapidly. I look forward to see some Japanese researchers to contribute this global electronic research community. Hideki Kawahara NTT Basic Research Labs. JAPAN. PS: JNNS President is Prof.Kunihiko Fukushima JNNS V.P. is Prof.Shun'ichi Amari If you need more detailes, please e-mail to: kawahara%siva.ntt.jp at RELAY.CS.NET . From munnari!extro.ucc.su.oz.au!root at uunet.UU.NET Fri Nov 17 14:00:27 1989 From: munnari!extro.ucc.su.oz.au!root at uunet.UU.NET (Admin) Date: Fri Nov 17 14:00:27 1989 Subject: This mail got misdirected ... Message-ID: <8911170303.914@munnari.oz.au> From Fri Nov 17 12:15 EST 1989 Date: Fri Nov 17 12:18:29 1989 From: mailer-daemon keeps mailer happy <> Apparently-To: MAILER-DAEMON >From ml-connectionists-request at Q.CS.CMU.EDU@murtoa.cs.mu.oz Fri Nov 17 09:24:23 1989 From: Hideki KAWAHARA Date: Fri, 17 Nov 89 04:52:41+0900 To: connectionists at CS.CMU.EDU Subject: News on JNNS(Japanese Neural Network Society) -------------------------- JNNS (Japanese Neural Network Society) have delivered its first newsletter and started a mailing list -------------------------- Japanese Neural Network society, which was founded in July 1989, have delivered its first newsletter on 14 November 1989. Prof. Shiro Usui of Toyohasi University of Technology, who is in charge of editor in chief have also started a mailing list to encouredge discussions among active researchers in Japan. Prof. Usui and I would like to introduce the connectionists mailing list to JNNS's mailing list and to quit delivering to BBORD eventually. Electronic communications in Japan is still in its infancy. JUNET, the largest one, is a volunteer based (mainly UUCP) network. However, the number of researchers who are accessible to some electronic communication systems is increasing rapidly. I look forward to see some Japanese researchers to contribute this global electronic research community. Hideki Kawahara NTT Basic Research Labs. JAPAN. PS: JNNS President is Prof.Kunihiko Fukushima JNNS V.P. is Prof.Shun'ichi Amari If you need more detailes, please e-mail to: kawahara%siva.ntt.jp at RELAY.CS.NET . From munnari!extro.ucc.su.oz.au!root at uunet.UU.NET Fri Nov 17 14:00:13 1989 From: munnari!extro.ucc.su.oz.au!root at uunet.UU.NET (Admin) Date: Fri Nov 17 14:00:13 1989 Subject: This mail got misdirected ... Message-ID: <8911170303.900@munnari.oz.au> From Fri Nov 17 12:14 EST 1989 Date: Fri Nov 17 12:18:15 1989 From: mailer-daemon keeps mailer happy <> Apparently-To: MAILER-DAEMON >From ml-connectionists-request at Q.CS.CMU.EDU@murtoa.cs.mu.oz Fri Nov 17 09:24:23 1989 From: Hideki KAWAHARA Date: Fri, 17 Nov 89 04:52:41+0900 To: connectionists at CS.CMU.EDU Subject: News on JNNS(Japanese Neural Network Society) -------------------------- JNNS (Japanese Neural Network Society) have delivered its first newsletter and started a mailing list -------------------------- Japanese Neural Network society, which was founded in July 1989, have delivered its first newsletter on 14 November 1989. Prof. Shiro Usui of Toyohasi University of Technology, who is in charge of editor in chief have also started a mailing list to encouredge discussions among active researchers in Japan. Prof. Usui and I would like to introduce the connectionists mailing list to JNNS's mailing list and to quit delivering to BBORD eventually. Electronic communications in Japan is still in its infancy. JUNET, the largest one, is a volunteer based (mainly UUCP) network. However, the number of researchers who are accessible to some electronic communication systems is increasing rapidly. I look forward to see some Japanese researchers to contribute this global electronic research community. Hideki Kawahara NTT Basic Research Labs. JAPAN. PS: JNNS President is Prof.Kunihiko Fukushima JNNS V.P. is Prof.Shun'ichi Amari If you need more detailes, please e-mail to: kawahara%siva.ntt.jp at RELAY.CS.NET . From nrandall%watdcs.UWaterloo.ca at VMA.CC.CMU.EDU Fri Nov 17 09:50:16 1989 From: nrandall%watdcs.UWaterloo.ca at VMA.CC.CMU.EDU (Neil Randall (ENGLISH)) Date: Fri, 17 Nov 89 09:50:16 EST Subject: E-Mail Distribution List Message-ID: If possible, please add me to the e-mail distribution list. I am one of the Rhetoric group in the Department of English at Waterloo. Thank you. Neil Randall From mohandes at ed.ecn.purdue.edu Fri Nov 17 10:15:04 1989 From: mohandes at ed.ecn.purdue.edu (Mohamed Mohandes) Date: Fri, 17 Nov 89 10:15:04 -0500 Subject: News on JNNS(Japanese Neural Network Society) Message-ID: <8911171515.AA03432@ed.ecn.purdue.edu> d From smk at flash.bellcore.com Fri Nov 17 11:38:14 1989 From: smk at flash.bellcore.com (Selma M Kaufman) Date: Fri, 17 Nov 89 11:38:14 EST Subject: No subject Message-ID: <8911171638.AA20706@flash.bellcore.com> Subject: Reprints Available Identifying and Discriminating Temporal Events with Connectionist Language Users Presented at: IEE Conference on Artificial Neural Networks (London, October, 1989) 284-286. Robert B. Allen and Selma M. Kaufman The "connectionist language user" paradigm is applied to several studies of the perception, processing, and description of events. In one study, a network was trained to discriminate the order with which objects appeared in a microworld. In a second study, networks were trained to recognize and describe sequences of events in the microworld using 'verbs'. In a third study 'plan recognition' was modeled. In the final study, networks answered questions that used verbs of possession. These results further strengthen the generality of the approach as a unified model of perception, action, and language. Back-Propagation as Computational Model of Gestalt Cognition: Evidence for a Halo Effect Presented at: Second International Symposium on Artificial Intelligence (Monterrey, Mexico, October 23-27, 1989). Robert B. Allen Connectionist networks show a distinctly different type of processing than ruled-based approaches. It is proposed that connectionist networks resemble the gestalt models of cognition that were popular in the 1950s. Moreover, a context effect known as the "halo effect", which is a hallmark of gestalt models of cognition, was modeled. The effect was confirmed when networks were required to generate valences assigned to objects which were presented in the context of other objects. For paper copies, contact: Selma Kaufman, 2M-356, Bellcore, 445 South St., Morristown, NJ 07960-1910. smk at flash.bellcore.com From pa1490%sdcc13 at ucsd.edu Fri Nov 17 14:08:47 1989 From: pa1490%sdcc13 at ucsd.edu (Dave Scotese) Date: Fri, 17 Nov 89 11:08:47 PST Subject: E-Mail Distribution List Message-ID: <8911171908.AA29998@sdcc13.UCSD.EDU> From THEPCAP%SELDC52.BITNET at VMA.CC.CMU.EDU Fri Nov 17 13:32:00 1989 From: THEPCAP%SELDC52.BITNET at VMA.CC.CMU.EDU (THEPCAP%SELDC52.BITNET@VMA.CC.CMU.EDU) Date: Fri, 17 Nov 89 19:32 +0100 Subject: TR available Message-ID: October 1989 LU TP 89-19 "TEACHERS AND CLASSES" WITH NEURAL NETWORKS Lars Gislen, Carsten Peterson and Bo Soderberg Department of Theoretical Physics, University of Lund Solvegatan 14A, S-22362 Lund, Sweden Submitted to International Journal of Neural Systems Abstract: A convenient mapping and an efficient algorithm for solving scheduling problems within the neural network paradigm is presented. It is based on a reduced encoding scheme and a mean field annealing prescription, which was recently successfully applied to TSP. Most scheduling problems are characterized by a set of hard and soft constraints. The prime target of this work is the hard constraints. In this domain the algorithm persistently finds legal solutions for quite difficult problems. We also make some exploratory investigations by adding soft constraints with very encouraging results. Our numerical studies cover problem sizes up to O(5*10^4) degrees of freedom with no parameter sensitivity. We stress the importance of adding certain extra terms to the energy functions which are redundant from the encoding point of view but beneficial when it comes to ignoring local minima and to stabilizing the good solutions in the annealing process. --------------------------------------- For copies of this report send requests to: THEPCAP at SELDC52. NOTICE: Those of you who requested our previous report, "A New Way of Mapping Optimization.... (LU TP 89-1), will automatically receive this one so no request is necessary. From harnad at clarity.Princeton.EDU Mon Nov 20 11:09:13 1989 From: harnad at clarity.Princeton.EDU (Stevan Harnad) Date: Mon, 20 Nov 89 11:09:13 EST Subject: What is a Symbol System? Message-ID: <8911201609.AA01142@psycho.Princeton.EDU> There has been some difference of opinion as to whether a connectionist network is or is not, or can or cannot be, a symbol system. To answer such questions, one must first settle on what a symbol system is. Here's my candidate: What is a symbol system? From Newell (1980) Pylyshyn (1984), Fodor (1987) and the classical work of Von Neumann, Turing, Goedel, Church, etc.(see Kleene 1969) on the foundations of computation, we can reconstruct the following definition: A symbol system is: (1) a set of arbitrary PHYSICAL TOKENS (scratches on paper, holes on a tape, events in a digital computer, etc.) that are (2) manipulated on the basis of EXPLICIT RULES that are (3) likewise physical tokens and STRINGS of tokens. The rule-governed symbol-token manipulation is based (4) purely on the SHAPE of the symbol tokens (not their "meaning"), i.e., it is purely SYNTACTIC, and consists of (5) RULEFULLY COMBINING and recombining symbol tokens. There are (6) primitive ATOMIC symbol tokens and (7) COMPOSITE symbol-token strings. The entire system and all its parts -- the atomic tokens, the composite tokens, the syntactic manipulations (both actual and possible) and the rules -- are all (8) SEMANTICALLY INTERPRETABLE: The syntax can be SYSTEMATICALLY assigned a meaning (e.g., as standing for objects, as describing states of affairs). According to proponents of the symbolic model of mind such as Fodor (1980) and Pylyshyn (1980, 1984), symbol-strings of this sort capture what mental phenomena such as thoughts and beliefs are. Symbolists emphasize that the symbolic level (for them, the mental level) is a natural functional level of its own, with ruleful regularities that are independent of their specific physical realizations. For symbolists, this implementation-independence is the critical difference between cognitive phenomena and ordinary physical phenomena and their respective explanations. This concept of an autonomous symbolic level also conforms to general foundational principles in the theory of computation and applies to all the work being done in symbolic AI, the branch of science that has so far been the most successful in generating (hence explaining) intelligent behavior. All eight of the properties listed above seem to be critical to this definition of symbolic. Many phenomena have some of the properties, but that does not entail that they are symbolic in this explicit, technical sense. It is not enough, for example, for a phenomenon to be INTERPRETABLE as rule-governed, for just about anything can be interpreted as rule-governed. A thermostat may be interpreted as following the rule: Turn on the furnace if the temperature goes below 70 degrees and turn it off if it goes above 70 degrees, yet nowhere in the thermostat is that rule explicitly represented. Wittgenstein (1953) emphasized the difference between EXPLICIT and IMPLICIT rules: It is not the same thing to "follow" a rule (explicitly) and merely to behave "in accordance with" a rule (implicitly). The critical difference is in the compositeness (7) and systematicity (8) criteria. The explicitly represented symbolic rule is part of a formal system, it is decomposable (unless primitive), its application and manipulation is purely formal (syntactic, shape-dependent), and the entire system must be semantically interpretable, not just the chunk in question. An isolated ("modular") chunk cannot be symbolic; being symbolic is a combinatory, systematic property. So the mere fact that a behavior is "interpretable" as ruleful does not mean that it is really governed by a symbolic rule. Semantic interpretability must be coupled with explicit representation (2), syntactic manipulability (4), and systematicity (8) in order to be symbolic. None of these criteria is arbitrary, and, as far as I can tell, if you weaken them, you lose the grip on what looks like a natural category and you sever the links with the formal theory of computation, leaving a sense of "symbolic" that is merely unexplicated metaphor (and probably differs from speaker to speaker). Any rival definitions, counterexamples, amplifications? Excerpted from: Harnad, S. (1990) The Symbol Grounding Problem. Physica D (in press) ----------------------------------------------------- References: Fodor, J. A. (1975) The language of thought. New York: Thomas Y. Crowell Fodor, J. A. (1987) Psychosemantics. Cambridge MA: MIT/Bradford. Fodor, J. A. & Pylyshyn, Z. W. (1988) Connectionism and cognitive architecture: A critical appraisal. Cognition 28: 3 - 71. Harnad, S. (1989) Minds, Machines and Searle. Journal of Theoretical and Experimental Artificial Intelligence 1: 5-25. Kleene, S. C. (1969) Formalized recursive functionals and formalized realizability. Providence, R.I.: American Mathematical Society. Newell, A. (1980) Physical Symbol Systems. Cognitive Science 4: 135-83. Pylyshyn, Z. W. (1980) Computation and cognition: Issues in the foundations of cognitive science. Behavioral and Brain Sciences 3: 111-169. Pylyshyn, Z. W. (1984) Computation and cognition. Cambridge MA: MIT/Bradford Turing, A. M. (1964) Computing machinery and intelligence. In: Minds and machines, A.R. Anderson (ed.), Engelwood Cliffs NJ: Prentice Hall. From skrzypek at CS.UCLA.EDU Mon Nov 20 16:53:15 1989 From: skrzypek at CS.UCLA.EDU (Dr. Josef Skrzypek) Date: Mon, 20 Nov 89 13:53:15 PST Subject: UCLA SFINX - "neural" net simulator Message-ID: <8911202153.AA04758@retina.cs.ucla.edu> The UCLA-SFINX, a "neural" network simulator is now in pub- lic domain. UCLA-SFINX (Structure and Function In Neural connec- tions) is an interactive neural network simulation environment designed to provide the investigative tools for studying the behavior of various neural structures. It was designed to easily express and simulate the highly regular patterns often found in large networks, but it is also general enough to model parallel systems of arbitrary interconnectivity. UCLA-SFINX is not based on any single neural network para- digm such as Backward Error Propagation (BEP) but rather enables users to simulate a wide variety of neural network models. UCLA- SFINX has been used to simulate neural networks for the segmenta- tion of images using textural cues, architectures for color and lightness constancy, script character recognition using BEP and others. It is all written in C, includes an X11 interface, and it has been ported to HP 9000 320/350 workstations running HP-UX, Sun workstations running SUNOS 3.5, IBM RT workstations running BSD 4.3, Ardent Titan workstations running Ardent UNIX Release 2.0, and VAX 8200's running Ultrix 2.2-1. To get UCLA-SFINX source code and documentation (in LaTeX format) follow the in- structions below: 1. To obtain UCLA-SFINX via the Internet: Sign and return the enclosed UCLA-SFINX License Agreement to the address below. We will send you a copy of the signed license agreement along with instructions on how to FTP a copy of UCLA-SFINX. If you have a PostScript printer, you should be able to produce your own copy of the manual. If you wish to obtain a hardcopy of the manual, return a check for $30 along with the license. 2. To obtain UCLA-SFINX on tape: Sign and return the enclosed UCLA-SFINX License Agreement to the address below. Return a check for $100 dollars along with the license, for a hardcopy of the manual and a copy of UCLA-SFINX on 1/4 inch cartridge tape (in tar format) read- able by a Sun 3 workstation. We will also send you a copy of the signed license agreement. Checks should be made payable to the Regents of the Univer- sity of California. If you have questions regarding any of the information discussed above send electronic mail to sfinx at retina.cs.ucla.edu or US mail to: UCLA Machine Perception Laboratory, Computer Science Department, 3532 Boelter Hall, Los Angeles, CA. 90024, USA. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cut here for license <<<<<<<<<<<<<<<<<<<<<<<<< THE REGENTS OF THE UNIVERSITY OF CALIFORNIA LOS ANGELES CAMPUS UCLA-SFINX LICENSE AGREEMENT This Agreement is entered into this___________of ____________________, 198__, by and between THE REGENTS OF THE UNIVERSITY OF CALIFORNIA, a California corporation, hereinafter called "University", and ________________________________________ _____________________________________, hereinafter called "Licensee." This Agreement is made with reference to the follow- ing: 1. DEFINITION "UCLA-SFINX" is a set of copyrighted, source code computer programs and any future modifications thereof delivered by University to Licensee, and any accompanying documentation provided by University. UCLA-SFINX is a general purpose software system for the development and evaluation of con- nectionist models. UCLA-SFINX is written for and operates on UNIX systems. 2. GRANT OF RIGHTS A. University grants to Licensee and Licensee accepts a non-exclusive, non-transferable license to use UCLA- SFINX solely for Licensee's non-commercial purposes. B. Such use may include the making of sufficient copies of UCLA-SFINX for the reasonable purposes of Licensee hereunder. All copies of UCLA-SFINX made by Licensee, in whole or in part, regardless of the form in which the Licensee may subsequently use it, and regardless of any modification which the Licensee may subsequently make to it are the property of University and no title to or ownership of such materials are transferred to Licensee hereunder. Licensee shall include on any such copies labels containing the name UCLA-SFINX, the University's copyright notice, and any other proprietary or restrictive notices appearing on the la- bel of the copy of UCLA-SFINX furnished to Licensee by University. 1 C. Such use may include the modification of UCLA-SFINX by Licensee. Such modified versions of UCLA-SFINX shall remain the property of University. D. Such use shall not include further distribution, or any action which may be construed as selling or licensing UCLA-SFINX to any person or entity. 3. ACKNOWLEDGMENT A. Licensee acknowledges that UCLA-SFINX has been developed for research purposes only. B. Licensee shall require its employees and students to acknowledge in writing their use of UCLA-SFINX when re- porting any research resulting from such use. The fol- lowing notice should be used: "UCLA-SFINX from UCLA MACHINE PERCEPTION LABORATORY." 4. WARRANTIES AND INDEMNIFICATION A. University warrants that is is the owner of all right, title, and interest in and to UCLA-SFINX, including all copyright pertaining thereto and subsisting therein. B. UCLA-SFINX is licensed "AS IS," and University dis- claims all warranties, express and implied, including but not limited to, the implied warranties of merchan- tability and fitness for a particular purpose. In no event will University be liable for any business ex- pense, machine down time, loss of profits, any inciden- tal, special, exemplary or consequential damages, or any claims or demands brought against Licensee. The entire risk as to the quality and performance of UCLA- SFINX is with Licensee. C. Licensee agrees to indemnify, defend, hold harmless, and defend University, its officers, employees and agents, against any and all claims, suits, losses, dam- ages, costs, fees, and expenses resulting from or aris- ing out of any use of UCLA-SFINX by Licensee. 5. TECHNICAL SUPPORT AND FEEDBACK A. University shall have no obligation to install, sup- port, maintain, or correct any defects in UCLA-SFINX. 2 B. Licensee agrees to notify University of any errors, functional problems, and any defects in performance discovered in UCLA-SFINX and of any fixes made by Licensee. Such notice will contain a full description of the problem, indicating in what circumstances it originated, and how it manifested itself. Technical matters and errors discovered in UCLA-SFINX may be com- municated as provided in Article 9 below or via elec- tronic mail to: sfinx at retina.cs.ucla.edu. 6. TERM AND TERMINATION A. The term of this Agreement is perpetual and shall be effective from the date of its signing by duly author- ized official of Licensee. B. Any failure of Licensee to comply with all terms and conditions of this Agreement shall result in its im- mediate termination. 7. SEVERABILITY If any of the provisions or portions of this Agreement are invalid under any applicable statute or rule of law, they are to the extent of such invalidity severable and shall not affect any other provision of this Agreement. 8. APPLICABLE LAW This Agreement shall be governed by the laws of the State of California. 9. NOTICE A. Any notice under this Agreement shall be in writing and mailed to the appropriate address given below: To University regarding this Agreement: The Regents of the University of California Office of Contract and Grant Administration University of California, Los Angeles 405 Hilgard Avenue Los Angeles, California 90024-1406 Attention: Dr. Enrique Riveros-Schafer 3 B. To University regarding technical matters: UCLA Machine Perception Laboratory 3532 Boelter Hall Computer Science Department Los Angeles, California 90024 Attention: Prof. Josef Skrzypek C. To Licensee: ____________________________________________________ ____________________________________________________ ____________________________________________________ ____________________________________________________ ____________________________________________________ 10. ENTIRETY This Agreement supersedes any previous communication and, when signed by both parties, constitutes the complete under- standing of the parties. No modification or waiver of any provisions hereof shall be valid unless in writing and signed by both parties. IN WITNESS THEREOF, the parties here to have caused this Agree- ment to be executed. LICENSEE THE REGENTS OF THE UNIVERSITY OF CALIFORNIA From Dave.Touretzky at B.GP.CS.CMU.EDU Tue Nov 21 03:46:47 1989 From: Dave.Touretzky at B.GP.CS.CMU.EDU (Dave.Touretzky@B.GP.CS.CMU.EDU) Date: Tue, 21 Nov 89 03:46:47 EST Subject: What is a Symbol System? In-Reply-To: Your message of Mon, 20 Nov 89 11:09:13 -0500. <8911201609.AA01142@psycho.Princeton.EDU> Message-ID: <9939.627641207@DST.BOLTZ.CS.CMU.EDU> Okay, I'll take a shot at responding to Stevan's query. Of the eight criteria he listed, I take exception to numbers 2 and 3, that rules must be EXPLICIT and expressed as STRINGS of tokens. In my recent work on phonology with Deirdre Wheeler (see "A connectionist implementation of cognitive phonology", tech report number CMU-CS-89-144), we define an architecture for manipulating sequences of phonemes. This architecture supports a small number of primitive operations like insertion, mutation, and deletion of phonemes. I claim that the rules for deciding WHEN to apply these primitives do not need to be represented explicitly or have a symbolic representation, in order for us to have a symbol system. It suffices that the rules' actions be combinations of the primitive actions our architecture provides. This is what distinguishes our phonological model from the Rumelhart and McClelland verb learning model. In their model, rules have no explicit representation, but in addition, rules operate directly on the phoneme sequence in a totally unconstrained way, mapping activation patterns to activation patterns; there are no primitive symbolic operations. Therefore their model is non-symbolic, as they themselves point out. ================ I also think the definition of symbol system Stevan describes is likely to prove so constrained that it rules out human cognition. This sort of symbol system appears to operate only by disassembling and recombining discrete structures according to explicit axioms. What about more implicit, continuous kinds of computation, like using spreading activation to access semantically related concepts in a net? How far the activation spreads depends on a number of things, like the branching factor of the semantic net, the weights on the links, and the amount of cognitive resources (what Charniak calls "zorch") available at the moment. (People reason differently when trying to do other things at the same time, as opposed to when they're relaxed and able to concentrate on a single task.) Of course, spreading activation can be SIMULATED on a symbol processing system, such as a Turing machine or digital computer, but this raises the very important issue of levels of representation. What the Physical Symbol System Hypothesis requires is that the primitive atomic symbol tokens have meaning IN THE DOMAIN of discourse we're modeling. Although a Turing machine can be made to simulate continuous computations at any level of precision desired, it can only do so by using its primitive atomic symbols in ways that have nothing to do with the semantic net it's trying to simulate. Instead its symbols are used to represent things like the individual bits in some floating point number. To play the Physical Symbol Systems game correctly in the semantic net case, you have to choose primitives corresponding to nodes and links. But in that case there doesn't seem to be room for continuous, non-compositional sorts of computations. Another problem I see with this definition of symbol system is that it doesn't say what it means in #5 to "rulefully combine" symbols. What about stochastic systems, like Boltzmann machines? They don't follow deterministic rules, but they do obey statistical ones. What about a multilayer perceptron, which could be described as one giant rule for mapping input patterns to output patterns? -- Dave From well!mitsu at apple.com Mon Nov 20 22:31:20 1989 From: well!mitsu at apple.com (Mitsuharu Hadeishi) Date: Mon, 20 Nov 89 19:31:20 pst Subject: What is a Symbol System? Message-ID: <8911210331.AA07032@well.sf.ca.us> This is an interesting question. First of all, I think it is clear that since a recurrent neural network can emulate any finite-state automaton that they are Turing equivalent goes almost without saying, so it is also clear that recurrent NNs should be capable of the symbolic-level processing of which you speak. First of all, however, I'd like to address the symbolist point of view that higher-level cognition is purely symbolic, irrespective of the implementation scheme. I submit this is patently absurd. Symbolic representations of thought are simply models of how we think, and quite crude models at that. They happen to have several redeeming qualities however, among them that they are simple, well-defined, and easy to manipulate. However, in truth, though it is clear that many operations (such as syntactic analysis of language) operate within the structure, at least in part, of symbolic processing, others go outside (such as understanding a subtle poem). In addition, there are many other forms of higher-level cognition, such as that which visual artists engage themselves in, which do not easily lend themselves to symbolic decomposition. I submit that even everyday actions and thoughts do not follow any strict symbolic decomposition, though to some degree of approximation they can be modelled *as though* they were following rules of some kind. I think the comparison between rule-based and analog systems is apt; however, in my opinion it is the analog systems which have the greater flexibility, or one might say economy of expression. That is to say, inasmuch as one can emulate one with the other they are equivalent, but given limitations on complexity and size I think it is clear the complex analog dynamical systems have the edge. The fact is that as a model for the world or how we think rule-based representations are sorely lacking. It is similar to trying to paint a landscape using polygons; one can do it, but it is not particularly well-suited for the task, except in very simple situations (or situations where the landscape happens to be man-made.) We should not confuse the map with the territory. Just because we happen to have this crude model for thinking, i.e., the symbolic model, does not mean that is *how* we think. We may even describe our decisions this way, but the intractability of AI problems except for very limited-domain applications indicates or suggests the weaknesses with our model. For example, natural language systems only work with extremely limited context. The fact that they do work at all is evidence that our symbolic models are not completely inadequate, however, that they are limited in domain suggests they are nonetheless mere approximations. Connectionist models, I believe, have much greater chance at capturing the true complexity of cognitive systems. In addition, the recent introduction of fuzzy reasoning and nonmonotonic logic are extensions of the symbolic model which certainly improve the situation, but also point out the main weaknesses with symbolic models of cognition. Symbolic models address only one aspect of the thinking process, perhaps not even the most important part. For example, a master chess player typically only considers about a hundred possible moves, yet can beat a computer program that considers tens of thousands of moves. The intractability of even more difficult problems than chess also points this out. Before the symbolic engine can be put into action, a great deal of pre-processing goes on which will likely not be best described in symbolic terms. Mitsu Hadeishi Open Mind 16110 S. Western Avenue Gardena, CA 90247 (213) 532-1654 (213) 327-4994 mitsu at well.sf.ca.us From netlist at psych.Stanford.EDU Tue Nov 21 10:43:41 1989 From: netlist at psych.Stanford.EDU (Mark Gluck) Date: Tue, 21 Nov 89 07:43:41 PST Subject: Stanford Adaptive Network Colloq: RICHARD SUTTON, Dec 4. Message-ID: Stanford University Interdisciplinary Colloquium Series: Adaptive Networks and their Applications December 4th (Monday, 3:45pm): Room 380-380C ******************************************************************************** DYNA: AN INTEGRATED ARCHITECTURE FOR LEARNING, PLANNING, AND REACTING Richard S. Sutton GTE Laboratories Incorporated ******************************************************************************** Abstract How should a robot decide what to do? The traditional answer in AI has been that it should deduce its best action in light of its current goals and world model, i.e., that it should _plan_. However, it is now widely recognized that planning's computational complexity makes it infeasible for rapid decision making and that its dependence on a complete and accurate world model also greatly limits its applicability. An alternative is to do the planning in advance and compile it into a set of rapid _reactions_, or situation-action rules, which are then used for real-time decision making. Yet a third approach is to _learn_ a good set of reactions by trial and error; this has the advantage that it eliminates all dependence on a world model. In this talk I present _Dyna_, a simple architecture integrating and permitting tradeoffs among all three approaches. Dyna is based on the old idea that planning is like trial-and-error learning from hypothetical experience. The theory of Dyna is based on the classical optimization technique of _dynamic_programming_, and on dynamic programming's relationship to reinforcement learning, to temporal-difference learning, and to AI methods for planning and search. In this talk, I summarize Dyna theory and present Dyna systems that learn from trial and error while they simultaneously learn a world model and use it to plan optimal action sequences. This work is an integration and extension of prior work by Barto, Watkins, and Whitehead. =========================================================================== GENERAL INFO: Location: Room 380-380C, which can be reached through the lower level between the Psychology and Mathematical Sciences buildings. Level: Technically oriented for persons working in related areas. Mailing lists: To be added to the network mailing list, netmail to netlist at psych.stanford.edu with "addme" as your subject header. Additional information: Contact Mark Gluck (gluck at psych.stanford.edu). From mclennan%MACLENNAN.CS.UTK.EDU at cs.utk.edu Tue Nov 21 13:29:24 1989 From: mclennan%MACLENNAN.CS.UTK.EDU at cs.utk.edu (mclennan%MACLENNAN.CS.UTK.EDU@cs.utk.edu) Date: Tue, 21 Nov 89 14:29:24 EDT Subject: What is a symbol system? Message-ID: <8911211929.AA22810@MACLENNAN.CS.UTK.EDU> Steve Harnad has invited rival definitions of the notion of a symbol system. I formulated the following (tentative) definition as a basis for discussion in a connectionism course I taught last year. After stating the definition I'll discuss some of the ways it differs from Harnad's. PROPERTIES OF DISCRETE SYMBOL SYSTEMS A. Tokens and Types 1. TOKENS can be unerringly separated from the background. 2. Tokens can be unambiguously classified as to TYPE. 3. There are a finite number of types. B. Formulas and Schemata 1. Tokens can be put into relationships with one another. 2. A FORMULA is an assemblage of interrelated tokens. 3. Formulas comprise a finite number of tokens. 4. Every formula results from a computation (see below) starting from a given token. 5. A SCHEMA is a class of relationships among tokens that depends only on the types of those tokens. 6. It can be unerringly determined whether a formula belongs to a given schema. C. Rules 1. Rules describe ANALYSIS and SYNTHESIS. 2. Analysis determines if a formula belongs to a given schema. 3. Synthesis constructs a formula belonging to a given schema. 4. It can be unerringly determined whether a rule applies to a given formula, and what schema will result from applying that rule to that formula. 5. A computational process is described by a finite set of rules. D. Computation 1. A COMPUTATION is the successive application of the rules to a given initial formula. 2. A computation comprises a finite number of rule appli- cations. COMPARISON WITH HARNAD'S DEFINITION 1. Note that my terminology is a little different from Steve's: his "atomic tokens" are my "tokens", his "composite tokens" are my "formulas". He refers to the "shape" of tokens, whereas I distinguish the "type" of an (atomic) token from the "schema" of a formula (composite token). 2. So far as I can see, Steve's definition does not include anything corresponding to my A.1, A.2, B.6 and C.4. There are all "exactness" properties -- central, although rarely stated, assumptions in the theory of formal systems. For example, A.1 and A.2 say that we (or a Turing machine) can tell when we're looking at a symbol, where it begins and ends, and what it is. It is important to state these assumptions, because they need not hold in real-life pattern identification, which is imperfect and inherently fuzzy. One reason connectionism is important is that by questioning these assumptions it makes them salient. 3. Steve's (3) and (7), which require formulas to be LINEAR arrangements of tokens, are too restrictive. There is noth- ing about syntactic arrangement that requires it to be linear (think of the schemata used in long division). Indeed, the relationship between the constituent symbols need not even be spatial (e.g., they could be "arranged" in the frequency domain, e.g., a chord is a formula comprising note tokens). This is the reason my B.5 specified only "relationships" (perhaps I should have said "physical rela- tionships"). 4. Steve nowhere requires his systems to be finite (although it could be argued that this is a consequence of their being PHYSICAL systems). I think finiteness is essential. The theory of computation grew out of Hilbert's finitary approach to the foundations of mathematics, and you don't get the standard theory of computation if infinite formulas, rules, sets of rules, etc. are allowed. Hence my A.3, B.3, C.5, D.2. 5. Steve requires symbol systems to be semantically interpret- able (8), but I think this is an empty requirement. Every symbol system is interpretable -- if only as itself (essen- tially the Herbrand interpretation). Also, mathematicians routinely manipulate formulas (e.g., involving differen- tials) that have no interpretation (in standard mathematics, and ignoring "trivial" Herbrand-like interpretations). 6. Steve's (1) specifies a SET of formulas (physical tokens), but places no restrictions on that set. I'm concerned that this may permit uncountable or highly irregular sets of for- mulas (e.g., all the uncomputable real numbers). I tried to avoid this problem by requiring the formulas to be generat- able by a finite computational process. This seems to hold for all the symbol systems discussed in the literature; in fact the formation rules are usually just a context-free grammar. My B.4 says, in effect, that there is a generative grammar (not necessarily context free) for the formulas, in fact, that the set of formulas is recursively enumerable. 7. My definition does not directly require a rule itself to be expressible as a formula (nearly Steve's 3), but I believe I can derive this from my C.1, C.2, C.3, although I wouldn't want to swear to it. (Here's the idea: C.2 and C.3 imply that analysis and synthesis can be unambiguously described by formulas that are exemplars of those schemata. Hence, by C.1, every rule can be described by two examplars, which are formulas.) Let me stress that the above definition is not final. Please punch holes in it! Bruce MacLennan Department of Computer Science 107 Ayres Hall The University of Tennessee Knoxville, TN 37996-1301 (615)974-0994/5067 maclennan at cs.utk.edu From np%FRLRI61.BITNET at CUNYVM.CUNY.EDU Tue Nov 21 13:51:43 1989 From: np%FRLRI61.BITNET at CUNYVM.CUNY.EDU (np%FRLRI61.BITNET@CUNYVM.CUNY.EDU) Date: Tue, 21 Nov 89 19:51:43 +0100 Subject: No subject Message-ID: <8911211851.AA05986@sun3a.lri.fr> Subject: dynamical non-linear process control with neural networks Dear College, I am currently working on the application of Neural Networks to the control of a dynamical non-linear process. Is there anybody who has already successfully used some neural method in order to control a dynamical non-linear process ? What about the dymamical properties (and possibilities) of a feed-forward neural net (e.g. trained with the classical back-propagation algorithm)? nicolas puech, Laboratoire de Recherche Informatique, ORSAY, FRANCE From Scott.Fahlman at B.GP.CS.CMU.EDU Tue Nov 21 17:11:32 1989 From: Scott.Fahlman at B.GP.CS.CMU.EDU (Scott.Fahlman@B.GP.CS.CMU.EDU) Date: Tue, 21 Nov 89 17:11:32 EST Subject: UCLA SFINX - "neural" net simulator In-Reply-To: Your message of Mon, 20 Nov 89 13:53:15 -0800. <8911202153.AA04758@retina.cs.ucla.edu> Message-ID: The UCLA-SFINX, a "neural" network simulator is now in pub- lic domain... I'm guessing that you don't really mean "public domain" here, since later int he message you say that this code is copyrighted and you go on at some length about how to get a license. There's a big legal difference between "public domain" and "copyrighted, but available free of charge for non-commerical use". "Public domain" means that the owner of the intellectual property is relinquishing all control over it, which does not seem to be the case here. I'm not complaining, just trying to clarify the situation here. I think it's great when people make useful pieces of software available without cost, even if there are some strings attached. -- Scott From krulwich at zowie.ils.nwu.edu Tue Nov 21 16:12:04 1989 From: krulwich at zowie.ils.nwu.edu (Bruce Krulwich) Date: Tue, 21 Nov 89 15:12:04 CST Subject: Symbol systems vs AI systems Message-ID: <8911212112.AA26977@zowie.ils.nwu.edu> I'm not sure if this is a good idea, but I'm going to throw in some thoughts to the "symbol system" question. Please note that I'm responding to the CONNECTIONISTS list and do not wish my message to be forwarded to any public newsgroups. There are two implicit assumptions that I think Steve is making in his message, especially given its being sent to CONNECTIONISTS. The first is that these characteristics of a "symbol system" do not apply to connectionist nets, and the second is that the "symbol systems" that he characterizes are in fact all classical (non-connectionist) AI systems. Even if he's not making these two assumptions, I think that others do so I'm going to go ahead and discuss them. First, the assumption that Steve's characterizations of "symbol systems" do not apply to neural nets. Looking at the 8 aspects of the definition, I think that each of them apply to NN's as much as they apply to many symbol systems. In other words, they apply to symbol systems if you look at symbol systems purely syntactically and ignore the meaning and theory that goes into the system. The same is true about NN's. From an anally syntactic point of view, neural nets are simply doing transformations on sets of unit values. (I'm not restricting this to feed-forward nets like many people do. This really is the case about all nets.) They have very specific rules about combining values, in fact, all the tokens (units) in the system use the same rule on different inputs. Clearly this view of NN's is missing the forest for the trees, because the point of NN's is the semantics of the computation they engage in and of the information they encode. My claim, however, is that the same is true of all AI systems. Looking at them as syntactic "symbol systems" is missing their entire point, and is missing what differentiates them from other systems. This leads me to the assumption that all classical AI systems are "symbol systems" as defined. I think that this is less true than my claim above about connectionist nets. Let's look, for example, at a research area called "case-based reasoning." (For those unfamiliar with CBR, take a look at the book "Inside Case-Based Reasoning" by Riesbeck and Schank, published by LEA, or the proceedings of the past two case-based reasoning workshops published by Morgan Kaufman.) The basic idea in CBR is that problems are solved by making analogies to previously solved problems. The assumption is that general rules are impossible to get because there is often not enough information to generalize from the instances that an agent gets. Looking at Steve's characterizations of a "symbol system," we can see that CBR systems have (a) no explicit rules, and (b) completely semantic matching (in most cases) that is not dependant on the "shape" of the representations. Certainly there is a level at which CBR systems are "symbol systems" in the same way that all computer programs are inherently "symbol systems." The point, however, is that this is _not_ the issue in CBR systems just like its not the issue in connectionist models. Since the _theory_ embedded in CBR systems is irrespective of several of the characterizations of "symbol systems," they are only "symbol systems" in the way that all connectionist models are "symbol systems" because they are all simulated on computers. I have used CBR as an example here, but the same could be said about alot of the recent work in analogical reasoning, analytical learning (such as EBL), default reasoning, and much of the rest of the semantically oriented AI systems. My claim is that one of two things is the case: (1) Much of the current work in classical AI does not fall into what Steve has characterized as "symbol systems," or (2) Connectionist nets _do_ fall into this catagory. It doesn't really matter which of these is the case, because each of them makes the characterization useless as a characterization of AI systems. I'd like to close by apologizing to CONNECTIONISTS readers if this post starts or continues a bad trend on the mailing list. The last thing that anyone wants is for CONNECTIONISTS to mimic COMP.AI. I've tried to keep my points to ones that address the assumptions that alot of connectionist research makes in the hope of keeping this from blowing up too much. Bruce Krulwich Institute for the Learning Sciences krulwich at ils.nwu.edu From skrzypek at CS.UCLA.EDU Tue Nov 21 19:53:14 1989 From: skrzypek at CS.UCLA.EDU (Dr. Josef Skrzypek) Date: Tue, 21 Nov 89 16:53:14 PST Subject: UCLA SFINX - "neural" net simulator In-Reply-To: Scott.Fahlman@B.GP.CS.CMU.EDU's message of Tue, 21 Nov 89 17:11:32 EST <8911212212.AA28884@shemp.cs.ucla.edu> Message-ID: <8911220053.AA08590@retina.cs.ucla.edu> Date: Tue, 21 Nov 89 17:11:32 EST From: Scott.Fahlman at B.GP.CS.CMU.EDU The UCLA-SFINX, a "neural" network simulator is now in pub- lic domain... I'm guessing that you don't really mean "public domain" here, since later int he message you say that this code is copyrighted and you go on at some length about how to get a license. There's a big legal difference between "public domain" and "copyrighted, but available free of charge for non-commerical use". "Public domain" means that the owner of the intellectual property is relinquishing all control over it, which does not seem to be the case here. I'm not complaining, just trying to clarify the situation here. I think it's great when people make useful pieces of software available without cost, even if there are some strings attached. -- Scott >>>>>>>>>>>>>>>>>>>>>>>>>> You are correct that SFINX is not strictly in "public domain". Eventually, we will adapt the Free Software Foundation license agreement; the effort needed to overcome various administrative "procedures" would simply delay the release. Thanks for your comments. Josef From Dave.Touretzky at B.GP.CS.CMU.EDU Tue Nov 21 23:41:09 1989 From: Dave.Touretzky at B.GP.CS.CMU.EDU (Dave.Touretzky@B.GP.CS.CMU.EDU) Date: Tue, 21 Nov 89 23:41:09 EST Subject: neural net intro books Message-ID: <856.627712869@DST.BOLTZ.CS.CMU.EDU> A common topic of discussion when academic neural net types get together is ``What sort books are available for teaching neural nets?'' I recently got a catalog from Van Nostrand Reinhold that listed two introductory books, although they're not exactly textbooks. The details are given below. I haven't seen either of them yet, so this is not an endorsement, just an announcement of their existence. If someone has seen these books and would like to post a short review to CONNECTIONISTS, that would be helpful. -- Dave ================ NEURAL COMPUTING Theory and Practice by Philip D. Wasserman, ANZA Research, Inc. 230 pages, 100 illustrations, $36.95 The complex mathematics and algorithms of artificial neural networks are broken down into simple procedures in this welcome tutorial. Fully explored are network fundamentals, implementation of commonly-used paradigms, and how to enhance problem-solving through integration of neural net research with traditional artificial intelligence and computing methods. Real-world examples clarify applications of artificial neural networks in computer science, engineering, physiology, and psychology. CONTENTS: Introduction. Fundamentals of Artificial Neural Networks. Perceptrons. Backpropagation. Counterpropagation Networks. Statistical Methods. Hopfield Nets. Bidirectional Associative Memories. Adaptive Resonance Theory. Optical Neural Networks. The Cognitron and Neocognitron. APPENDICES. The Biological Neural Network. Vector and Matrix Operations. Training Algorithms. Index. NEURAL NETWORK ARCHITECTURES An Introduction by Judith Dayhoff, Ph.D., Editor of the Journal of Neural Network Computing 220 pages, 100 illustrations, $34.95 This down-to-earth book gives you a plain-English explanation of the relationships between biological and artificial neural networks, plus detailed assessments of important uses of network architectures today. CONTENTS: An Overview of Neural Network Technology. Neurons and Network Topologies. Early Paradigms - The Beginnings of Today's Neural Networks. The Hopfield Network: Computational Models and Result. Back-Error Propagation: Paradigms and Applications. Competitive Learning: Paradigms and Competitive Neurons from Biological Systems. Biological Neural Systems: Organization. Structural Diversity. Temporal Dynamics. Origins of Artificial Neural Systems. Brain Structure and Function. Biological Nerve Cells. Synapses - How Do Living Nerve Cells Interconnect? What Is Random and What Is Fixed in the Brain's Neural Networks? How Do Biological Systems Really Compare to Computational Neural Networks? Associative and Adaptive Networks - More Paradigms. More Applications - Emphasizing Vision, Speech, and Pattern Recognition. New Directions for Neural Networks. From elsley at jupiter.risc.com Wed Nov 22 14:08:07 1989 From: elsley at jupiter.risc.com (Dick Elsley) Date: Wed, 22 Nov 89 11:08:07 PST Subject: Research position available. Message-ID: <8911221904.AA06576@jupiter.risc.com> ***** DO NOT FORWARD TO ANY OTHER LISTS ***** The Rockwell International Science Center has a job opening for a Neural Network researcher. We are looking for someone with research experience in neural networks (including simulations) and an interest in developing the necessary conceptual bridges between neural network research and application problems. The Science Center is the corporate research lab of Rockwell International and is located in sunny Thousand Oaks, CA, just northwest of Los Angeles. We do a combination of basic and applied research in a wide variety of areas generally related to technologies that will eventually find their way into 'systems'. We also interact closely with key University groups and with the product divisions of Rockwell. Our work includes direct contracts from DARPA, ONR, etc., IR&D (discretionary), and joint work with our product divisions. Publication of results is encouraged. Very little of what we do is classified. Our neural network research team is currently active in machine vision, adaptive control, signal processing and several hardware implementation technologies. U.S. citizenship or permanent residency (green card) may be required. If you are interested, please send me a resume by E-mail or surface mail; and/or see me in Denver. Professional Staffing KS-20 Rockwell International Science Center 1049 Camino dos Rios Thousand Oaks, CA 91390 Dr. Richard K.(Dick) Elsley elsley at risc.com From jbower at smaug.cns.caltech.edu Wed Nov 22 19:29:27 1989 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Wed, 22 Nov 89 16:29:27 PST Subject: NIPS DEMOS Message-ID: <8911230029.AA27257@smaug.cns.caltech.edu> NIPS 89 MEETING ANNOUNCEMENT COMPUTER DEMOS We have finally received word that both DEC and SUN will provide computers for the computer demo room at this years NIPS meeting. The machines available include a SUN 4/110 color system with 1/4" tape and 16 Mb of memory, and a DEC workstation 3100 color system with a TK50 tape drive and 16 Mb of memory. Both machines will run the latest release of their respective operating systems. In addition, they will each run X11. Authors presenting papers at the meeting should feel free to bring software for demonstration. Software not associated with presentations will be demonstrated dependent on time and space. The demo room will be staffed throughout the conference by John Uhley. Those interested in performing demos should contact John at the meeting. No on site development will be possible. Software should be brought in the form of debugged and compiled code. From harnad at clarity.Princeton.EDU Fri Nov 24 16:31:37 1989 From: harnad at clarity.Princeton.EDU (Stevan Harnad) Date: Fri, 24 Nov 89 16:31:37 EST Subject: Connectionist Learning/Representation: BBS Call for Commentators Message-ID: <8911242131.AA00826@reason.Princeton.EDU> Below is the abstract of a forthcoming target article to appear in Behavioral and Brain Sciences (BBS), an international, interdisciplinary journal that provides Open Peer Commentary on important and controversial current research in the biobehavioral and cognitive sciences. Commentators must be current BBS Associates or nominated by a current BBS Associate. To be considered as a commentator on this article, to suggest other appropriate commentators, or for information about how to become a BBS Associate, please send email to: harnad at confidence.princeton.edu harnad at pucc.bitnet or write to: BBS, 20 Nassau Street, #240, Princeton NJ 08542 [tel: 609-921-7771] ____________________________________________________________________ WHAT CONNECTIONIST MODELS LEARN: LEARNING AND REPRESENTATION IN CONNECTIONIST NETWORKS Stephen J Hanson Learning and Knowledge Acquisition Group Siemens Research Center Princeton NJ 08540 and David J Burr Artificial Intelligence and Communications Research Group Bellcore Morristown NJ 07960 Connectionist models provide a promising alternative to the traditional computational approach that has for several decades dominated cognitive science and artificial intelligence, although the nature of connectionist models and their relation to symbol processing remains controversial. Connectionist models can be characterized by three general computational features: distinct layers of interconnected units, recursive rules for updating the strengths of the connections during learning, and "simple" homogeneous computing elements. Using just these three features one can construct surprisingly elegant and powerful models of memory, perception, motor control, categorization and reasoning. What makes the connectionist approach unique is not its variety of representational possibilities (including "distributed representations") or its departure from explicit rule-based models, or even its preoccupation with the brain metaphor. Rather, it is that connectionist models can be used to explore systematically the complex interaction between learning and representation, as we try to demonstrate through the analysis of several large networks. From edmond at CS.UCLA.EDU Mon Nov 27 20:18:03 1989 From: edmond at CS.UCLA.EDU (Edmond Mesrobian) Date: Mon, 27 Nov 89 17:18:03 PST Subject: UCLA SFINX - "neural" net simulator Message-ID: <8911280118.AA23493@retina.cs.ucla.edu> The original announcement concerning UCLA-SFINX, a neural network simulator available from the Machine Perception Lab, was ambiguous. The simulator does not have an X Window System user interface. The simulator uses a command line interpreter. However, the simulator does support a variety of graphics environments as output media for simulation results. Output graphics support is provided for the following environments: 1) X Window System 2) HP workstations using Starbase Graphics Library 3) Sun Workstations using Matrox VIP-1024 Frame Grabbers 4) IBM RTs using an Imagraph AGC-1010P color graphics card. We apologize for any confusion caused by the original announcement. If you have any questions concerning UCLA-SFINX, please send email to sfinx at retina.cs.ucla.edu or US mail to the address below. Edmond Mesrobian UCLA Machine Perception Lab 3531 Boelter Hall Los Angeles, CA 90024 From ST401843%BROWNVM.BITNET at vma.CC.CMU.EDU Tue Nov 28 13:48:29 1989 From: ST401843%BROWNVM.BITNET at vma.CC.CMU.EDU (thanasis kehagias) Date: Tue, 28 Nov 89 13:48:29 EST Subject: Probability Learning Nets Message-ID: a while ago i had asked for recurrent net references and had inserted a request for probability learninets as well. the rec-net bibliography is in the making, and the ones who have seen the preliminary version must have noticed that there was a lot of responses. not so for the probability learning nets. i had a couple of responses only. i did my own digging around and found a couple more and i enclose them in this message as a seed. but i would be really interested in any other refernces anybody has. once again, when done, i will circulate the resulting bibliography. thanasis PS: bibtex format is always welcome!!! @article{kn: title ="Cognitive and Psychological Computation with Neural Models", author ="J.A. Anderson", journal ="IEEE Transactions on Systems, Man and Cybernetics", volume ="SMC-13", year ="1983" } @article{kn: title ="Distinctive Features, Categorical Perception and Learning: some Applications of a Neural Model ", author ="J.A. Anderson et.al. ", journal ="Psychological Review", year ="1977", volume ="84", page ="413-451", } @inproceedings{kn: title ="G-Maximization: An Unsupervised Learning Procedure for discovering Regularities ", booktitle ="Neural Networks for Computing ", author ="B. Pearlmutter and G. Hinton ", editor ="J.S. Denker ", year ="1986 ", pages ="333-338", organization ="American Institute for Physics" } @techreport{kn: author ="H.J. Sussman", title ="On the Convergence of Learning Algorithms for Boltzmann Machines", number ="sycon-88-03", institution ="Rytgers Center for Systems and Control", year ="1988" } From SAYEGH%IPFWCVAX.BITNET at VMA.CC.CMU.EDU Wed Nov 1 11:05:00 1989 From: SAYEGH%IPFWCVAX.BITNET at VMA.CC.CMU.EDU (SAYEGH%IPFWCVAX.BITNET@VMA.CC.CMU.EDU) Date: Wed, 1 Nov 89 11:05 EST Subject: NN conference. Message-ID: Second Conference on Neural Networks and Parallel Distributed Processing ------------------------------------------------------------------------ Indiana-Purdue University Fort Wayne Thursday November 9, 6 to 9 pm (Neff 101) Friday November 10, 6 to 9 pm (KT 132) Saturday November 11, 9 am to 1 pm (KT 132) Schedule: Thursday: -------- "Physics and Neural Networks" Dr. Samir Sayegh, Indiana-Purdue University. "An Engineering Introduction to Back Propagation" Dr. Allen Pugh, Indiana_Purdue University and Dr. Kirk Dunkelberger, Magnavox Electronics Company. "Computer Simulation of the Motor Nervous System of a Simple Invertebrate" Dr. Ernst Niebur, California Institute of Technology. Friday: ------ "Neural Networks in Vivo" Dr. Jeff Wilson, Indiana-Purdue University. "Designing Structured Neural Networks for Speech Recognition" Dr. Alex Waibel, Carnegie Mellon University and ATR Interpreting Telephony Research Labs (Japan). "Self Organization Applied to the Design of Neural Networks Architectures" Dr. Manoel Tenorio, Purdue University. Saturday: -------- Workshop: "Applying Connectionist Models to Speech and other Real World Signals." Dr. Alex Waibel. For more information: sayegh at ipfwcvax.bitnet sayegh at ed.ecn.purdue.edu FAX: (219) 481-6880 Voice: (219) 481-6157. From KENYON at tahoma.phys.washington.edu Mon Nov 6 17:55:00 1989 From: KENYON at tahoma.phys.washington.edu (KENYON@tahoma.phys.washington.edu) Date: Mon, 6 Nov 89 14:55 PST Subject: ROOMMATES FOR NIPS 89? Message-ID: Hello, Is anyone interested in sharing the costs of a room for the upcoming NIPS conference? Please drop me line if you are. I will try to distribute any additional responses I receive to other interested connectionist. The above applies to the workshop also. Thanks, Gar Kenyon (206) 543-7334 email kenyon at phast.phys.washington.edu From gmdzi!muehlen at uunet.UU.NET Tue Nov 7 09:55:10 1989 From: gmdzi!muehlen at uunet.UU.NET (Heinz Muehlenbein) Date: Tue, 7 Nov 89 13:55:10 -0100 Subject: CFP: Parallel Computing special issue on NNs Message-ID: <8911071255.AA22543@gmdzi.UUCP> Special issue on neural networks ----------------------------------- Dear colleagues, I am editing a special issue of the journal Parallel Computing on neural networks. The following topics will be covered ---introduction to NN ---simulation of NN's ---performance of NN's --- limitations of current NN's ---the next generation I am looking for papers describing the limitations of current NN's and/or give an outline of the next generation. In my opinion, the next generation of NN's will have the following features ( to mention only some important ones). They are -modular -recurrent -asynchronous Modular neural networks are networks which are composed out of subnetworks, which can be trained independently. A major problem is to find the modular structure which fits the specific application problem. I have proposed genetic neural networks as a longterm research topic. In these networks, the genes specify the network modules and their interconnection. By simulating the evolution process, these networks adapt to the application. I believe that many researchers are going into the same direction. Why not publishing it now? Please contact me by e-mail. Deadline for abstracts is the end of November. Deadline for the finished paper (length 10-15 pages) is January, 31. The issue will appear late summer 1990. -----Heinz Muehlenbein GMD P.O 1240 5205 Sankt Augustin 1 Germany muehlen at gmdzi.uucp From Clayton.Bridges at A.GP.CS.CMU.EDU Tue Nov 7 16:23:28 1989 From: Clayton.Bridges at A.GP.CS.CMU.EDU (Clay Bridges) Date: Tue, 07 Nov 89 16:23:28 EST Subject: Genetics and Connectionism Message-ID: <5568.626477008@A.GP.CS.CMU.EDU> While I may be a bit wet behind the ears, I feel obliged to say something about the recent spate of enthusiasm about combining some sort of genetic evolution with connectionist networks. I am making the assumption that genetic evolution implies some variation on genetic algorithms. I'll admit that combining these two naturally inspired paradigms appeals to one's intuition in a powerful way (It certainly appeals to mine), but I believe that we should be wary. As far as I know, connectionist networks are still not very well understood, at least with respect to recurrent, asynchronous, or modular networks. Genetic algorithms (GAs) are even less well understood than connectionism, in general. Thus, in combining these two fields, we create the _potential_ for a subfield with little or no guiding theory, and thus one where specious projects abound, successes go unexplained, and research efforts go wasted. This is not to say that I don't think that the combination should be explored. I harbor some nascent plans to do so myself. What I am saying is that we shouldn't expect magic when we combine connectionism and GAs. We should explore the combination tentatively, with an eye toward having explanations (i.e. theory) for simple things before we attempt more complex ones. Clay Bridges clay at cs.cmu.edu From caroly at bucasb.BU.EDU Tue Nov 7 14:14:51 1989 From: caroly at bucasb.BU.EDU (Carol Yanakakis) Date: Tue, 7 Nov 89 14:14:51 EST Subject: graduate study Message-ID: <8911071914.AA11971@bucasb.bu.edu> * * * * * * * * * * * * * * * * * * * * * * * * * * * * M.A. AND Ph.D. PROGRAM in * * * * COGNITIVE AND NEURAL SYSTEMS * * * * at BOSTON UNIVERSITY * * * * Gail Carpenter and * * Stephen Grossberg, Co-Directors * * * * * * * * * * * * * * * * * * * * * * * * * * * * Boston University offers a unique M.A. and Ph.D. program in Cognitive and Neural Systems. This program presents an integrated curriculum offering the full range of psychological, neurobiological, and computational concepts, models, and methods in the broad field variously called neural networks, connectionism, parallel distributed processing, and biological information processing, in which Boston University is an acknowledged leader. Each student is required to take an equal number of carefully selected courses in one or more core departments, such as psychology, biology, computer science, mathematics, or engineering. A limited number of full-time graduate research fellowships are expected to be available. ***> For application materials, write to: Admissions Office Graduate School, Boston University 705 Commonwealth Avenue Boston, MA 02215 requesting materials for the Cognitive and Neural Systems (CNS) Program, or call: (617) 353-2697. ***> For a CNS brochure describing the curriculum and degree requirements, write to: Carol Yanakakis, Coordinator CNS Graduate Program Center for Adaptive Systems Boston University 111 Cummington Street Boston, MA 02215 or reply to: caroly at bucasb.bu.edu NOTE: You must contact BOTH the University Admissions Office and the CNS Program Coordinator in order to receive all materials necessary for applying. From eric at mcc.com Tue Nov 7 17:17:23 1989 From: eric at mcc.com (Eric Hartman) Date: Tue, 7 Nov 89 16:17:23 CST Subject: TR available Message-ID: <8911072217.AA05101@hobbes.aca.mcc.com> The following technical report is available. Requests may be sent to eric at mcc.com or via physical mail to the MCC address below. MCC Technical Report Number: ACT-ST-272-89 Layered Neural Networks With Gaussian Hidden Units as Universal Approximators Eric Hartman, James D. Keeler, and Jacek M Kowalski Microelectronics and Computer Technology Corporation 3500 W. Balcones Center Dr. Austin, TX 78759-6509 Abstract: A neural network with a single layer of hidden units of gaussian type (radial basis functions) is proved to be a universal approximator for real-valued maps defined on convex, compact sets set of R^n. (Submitted to Neural Computation) From kolen-j at cis.ohio-state.edu Wed Nov 8 09:44:07 1989 From: kolen-j at cis.ohio-state.edu (john kolen) Date: Wed, 8 Nov 89 09:44:07 EST Subject: Universal Approximators Message-ID: <8911081444.AA01671@toto.cis.ohio-state.edu> Question: How important are "universal approximator" results? Hornik, Stinchcombe, and White [1] demonstrate that a single hidden layer that uses an arbitrary squashing function can appoximate any Borel measurable function (i.e. has a countable number of discontinuities). They do this by showing the functions computable by this class of networks is dense in the set of Borel measurable functions. Great, but so are polynomials, or any sigma-algebra over the input space for that matter. [1] K. Hornik, M. Stinchcombe, H. White. "Multi-Layer Feedforward Networks are Universal Approximators". in press, Neural Networks. ---------------------------------------------------------------------- John Kolen (kolen-j at cis.ohio-state.edu)|computer science - n. A field of study Computer & Info. Sci. Dept. |somewhere between numerology and The Ohio State Univeristy |astrology, lacking the formalism of the Columbus, Ohio 43210 (USA) |former and the popularity of the later. From risto at CS.UCLA.EDU Wed Nov 8 19:24:39 1989 From: risto at CS.UCLA.EDU (Risto Miikkulainen) Date: Wed, 8 Nov 89 16:24:39 -0800 Subject: Tech report: Script Recognition with Hierarchical Feature Maps Message-ID: <8911090024.AA06401@oahu.cs.ucla.edu> **********DO NOT FORWARD TO OTHER BBOARDS************** **********DO NOT FORWARD TO OTHER BBOARDS************** **********DO NOT FORWARD TO OTHER BBOARDS************** The following tech report is available by anonymous ftp from the connectionist tech report database at Ohio State: SCRIPT RECOGNITION WITH HIERARCHICAL FEATURE MAPS Risto Miikkulainen Artificial Intelligence Laboratory Computer Science Department University of California, Los Angeles, CA 90024 risto at cs.ucla.edu Technical Report UCLA-AI-89-10 The hierarchical feature map system recognizes an input story as an instance of a particular script by classifying it at three levels: scripts, tracks and role bindings. The recognition taxonomy, i.e. the breakdown of each script into the relevant tracks and roles, is extracted automatically and independently for each script from story examples in an unsupervised learning process. The process resembles human learning in that the differentiation of the most frequently encountered scripts become gradually the most detailed. The resulting structure is a hierachical pyramid of feature maps. The number of input lines and the self-organization time required are considerably reduced compared to ordinary single-level feature mapping. The system is capable of recognizing incomplete stories and recovering the missing events. --------------- Here's how to obtain a copy: unix> ftp cheops.cis.ohio-state.edu # (or ftp 128.146.8.62) Name (cheops.cis.ohio-state.edu:): anonymous Password (cheops.cis.ohio-state.edu:anonymous): neuron ftp> cd pub/neuroprose ftp> type binary ftp> get (remote-file) miikkulainen.hierarchical.ps.Z (local-file) foo.ps.Z 119599 bytes received in 7.37 seconds (16 Kbytes/sec) ftp> quit unix> uncompress foo.ps unix> lpr -P(your_local_postscript_printer) foo.ps From postech.postech.ac.kr!jhkim at RELAY.CS.NET Thu Nov 9 01:51:24 1989 From: postech.postech.ac.kr!jhkim at RELAY.CS.NET (JooHeon Kim) Date: Thu, 9 Nov 89 15:51:24+0900 Subject: First meeting Message-ID: <8911090651.AA18730@postech.postech.ac.kr> Hello. My name is JooHeon Kim, a graduate student and a researcher at POSTECH, (Pohang Institute of Science and Technology) in KOREA. My interested area is Connectionist Modeling. > Boltzmann Machine Learning > Learning with Distributed Representation My address is JooHeon Kim, Artificial Intelligence Lab., Dept. of Computer Science, POSTECH, Pohang City, P.O.Box 125, KOREA. e-mail : Thanks. From jhkim at postech.postech.ac.kr Thu Nov 9 06:47:05 1989 From: jhkim at postech.postech.ac.kr (jhkim@postech.postech.ac.kr) Date: Thu, 9 Nov 89 13:47:05 +0200 Subject: First meeting Message-ID: <8911091147.AA06250@ariadne.csi.forth.gr> Hello. My name is JooHeon Kim, a graduate student and a researcher at POSTECH, (Pohang Institute of Science and Technology) in KOREA. My interested area is Connectionist Modeling. > Boltzmann Machine Learning > Learning with Distributed Representation My address is JooHeon Kim, Artificial Intelligence Lab., Dept. of Computer Science, POSTECH, Pohang City, P.O.Box 125, KOREA. e-mail : Thanks. From ST401843%BROWNVM.BITNET at VMA.CC.CMU.EDU Thu Nov 9 10:54:56 1989 From: ST401843%BROWNVM.BITNET at VMA.CC.CMU.EDU (thanasis kehagias) Date: Thu, 09 Nov 89 10:54:56 EST Subject: recurrent net bibliography Message-ID: a while ago i posted asking for information on recurrent nets. i collected a lot of responses and compiled a bibliography, which i sent to the list manager of connectionists. here is his response: ------------------------------------------------------------------ (List manager) I've placed a copy of your message and bibliography (recurrent.bib) in the "bibliographies" subdirectory of the connectionists directory that is accessible via anonymous FTP (instructions included below). I would suggest you post a description of the bibliography to the list, telling people that it is accessible via FTP and offering to mail it to people who cannot access it. ------------------------------------------------------------------------------- How to FTP Files from the CONNECTIONISTS Archive ------------------------------------------------ 1. Open an FTP connection to host B.GP.CS.CMU.EDU (Internet address 128.2.242.8). 2. Login as user anonymous with password your username. 3. 'cd' directly to one of the following directories: /usr/connect/connectionists/archives /usr/connect/connectionists/bibliographies 4. The archives and bibliographies directories are the ONLY ones you can access. You can't even find out whether any other directories exist. If you are using the 'cd' command you must cd DIRECTLY into one of these two directories. Access will be denied to any others, including their parent directory. 5. The archives subdirectory contains back issues of the mailing list. Some bibliographies are in the bibliographies subdirectory. Problems? - contact us at "Connectionists-Request at cs.cmu.edu". ------------------------------------------------------- (this is thanasis again): so now you know there is a bibliography and how to access it. in the rest of the message i include a small description of what is in the bibliography ... ****** THIS IS NOT A COMPLETE BIBLIOGRAPHY !!! ******** a while ago i posted a request for information on recurrent nets. i got a lot of responses and i am posting back, as i had promised. the bibliography follows at the end of the message. a few words of explanation: i was not one hundred percent sure of what i was looking for. having seen the responses, i think a fair description of the list i am sending would be: "a personal study guide for the time-space problem in connectionist networks" . what i mean is the following: i would classify neural nets in two broad categories: (1) static nets, where the output of nodes at time t is not fed back to other nodes at time t+1, and (2) dynamic nets where the output of nodes at time t is fed back to other nodes at time t+1. however, some of the static nets attempt to capture temporal relationships, and they usually do this by including time delayed inputs, this is quite commomn for the speech researchers (and i have included some references to such STATIC work), as well as for the people doing the chaotic time series problem (no references included here). for people with signal processing background, this kind of static nets is similar to FIR filters. almost by definition, all the dynamic nets exhibit temporal behavior. however, many of these dynamic nets are used in a static way. e.g. Hopfield type nets are dynamic but they are most often used to store static patterns. they are designed explicitly so they will settle down to a steady state (no oscillations either). (but i must say there are some few exceptions to this). Boltzmann machines are similar: they are in constant thermal motion, but we are basically interested in their steady state. another clarification is important: the training phase involves dynamic behavior in many cases even for static nets. as we vary the weights, the behavior of the system changes in time. but again the basic qualification about this "dynamic" behavior is that we are usually interested in the equilibrium. after all these qualifications, there are some truly dynamic nets, (meaning they are designed so as to exhibit interesting non-equlibria) analogous to the IIR filters in the signal processing paradigm , and my impression is that they are becoming more and more popular. in my opinion they are the most interesting ones and maybe the future is in these truly dynamic nets. having said all of the above, let me add that i include in the following short bibliography almost everything that was sent to me. needless to say, i have not read everything that i include in the list. there is no attempt for completeness here, and omission of some work should not be taken to mean that i consider this work inferior or unimportant. in particular i need to make the following very clear: i included very little ART type work, even if ART architectures is an example of truly dynamic networks. the reason i did this is simply that i am not familiar with this work. the bibliography is very provisional. if you find it useful use it, if not useful, do not flame me. i have a rather arbitrary grouping, with sparse comments as to what each category has. maybe a more annotated version will come later. all the more reason to send me suggestions about additions or changes. and many many thanks to all the people who send me contributions. thanasis From josh at flash.bellcore.com Thu Nov 9 14:54:04 1989 From: josh at flash.bellcore.com (Joshua Alspector) Date: Thu, 9 Nov 89 14:54:04 EST Subject: NIPS VLSI workshop Message-ID: <8911091954.AA03643@flash.bellcore.com> The following is an announcement for one of the workshops to be held December 1-2, 1989 at the Keystone resort after the NIPS-89 conference. -------------------------------------------------- VLSI NEURAL NETWORKS: CURRENT MILESTONES AND FUTURE HORIZONS Moderators: Joshua Alspector and Daniel B. Schwartz Bell Communications Research GTE Laboratories, Inc. 445 South Street 40 Sylvan Road Morristown, NJ 07960-19910 Waltham, MA 02254 (201) 829-4342 (617) 466-2414 e-mail: josh at bellcore.com e-mail: dbs%gte.com at relay.cs.net This workshop will explore the potential and problems of VLSI implementations of neural network. Several speakers will discuss their implementation strategies and speculate about where their work may lead. Workshop attendees will then be encouraged to organize working groups to address specific issues raised in connection with the presentations. An example of a topic that has lead to contentious discussion in the past is the relative virtue of analog vs. digital implementations of neural networks. Some other possible topics include: o Architectural issues - synchronous or asynchronous - full time or multiplexed interconnect - local or global connectivity o Technological issues - neural network specific VLSI technololgies - design tools and methodologies - robustness/fault tolerance o Theoretical issues - model of analog computation - complexity As part of the working groups, we also expect to make contact with deeper issues such as the limits to VLSI complexity for neural networks, the nature of VLSI compatible neural network algorithms, and which neural network applications demand special purpose hardware. Speakers (besides the moderators) include: Jay Sage - MIT-Lincoln Lab Rod Goodman - Caltech Bernhard Boser - AT&T/Bell Labs Alan Kramer - UC Berkeley Jim Burr - Stanford Nelson Morgan - ICSI From gc at s16.csrd.uiuc.edu Thu Nov 9 12:39:53 1989 From: gc at s16.csrd.uiuc.edu (George Cybenko) Date: Thu, 9 Nov 89 11:39:53 CST Subject: "Universal Approximators" Message-ID: <8911091739.AA06184@s16.csrd.uiuc.edu> Recently John Kolen asked why "universal approximator" results are interesting. Universal approximation results are important because they say that the technique being used (sigmoidal or radial basis function networks) has no a priori limitations if complexity (size) of the network is not a constraint. Such results are network analogues of Church's Thesis. Results about universal approximation properties for a variety of network types can be found in : G. Cybenko, "Approximation by Superpositions of a Sigmoidal Function", Mathematics of Control, Signals and Systems (Springer), August 1989, Vol. 2, pp. 303-314. Since such properties are shared by polynomials, Fourier series, splines, etc. etc., an important question to ask is "What makes network approaches better"? Parallelism and the existence of training algorithms does not count as an answer because polynomials and Fourier series have similar parallelism and training algorithms. I have a recent report that scratches the surface of this question and proposes a notion of complexity of classification problems that attempts to capture the way in which sigmoidal networks might be better on a class of problems. The report is titled "Designing Neural Networks" and is currently issued as CSRD Report #934. Send me mail or comments if you are interested in such questions. George Cybenko Center for Supercomputing Research and Development University of Illinois at Urbana-Champaign From rik%cs at ucsd.edu Thu Nov 9 15:53:09 1989 From: rik%cs at ucsd.edu (Rik Belew) Date: Thu, 9 Nov 89 12:53:09 PST Subject: Combining connectionism with the GA Message-ID: <8911092053.AA20392@roland.UCSD.EDU> There's been too much Email ink spilled lately about potential interactions between evolutionary algorithms and connectionist networks for me not to throw my two cents in. I announced a new tech report, "Evolution, learning and culture: Computational metaphors for adaptive algorithms" on this list a couple of weeks ago and I don't know whether the recent storm of messages has anything to do with that report or not. Anyway, some of the things I'll say below are better said there. Let me begin with Hammerstrom's analysis, related in [rudnick at cse.ogc.edu 26 Oct 89]. His basic point seems to be that connectionist algorithms (he uses NetTalk as his example) take a long time, and putting an evolutionary outer loop around them can only make matters worse. And if we are satisfied with the results of a single backpropagation (BP) search and the evloutionary loop is doing nothing more than randomly repeating this experiment, he may be right. But only if. First, who today is satisfied with vanilla BP in feed-forward networks? (And I'm not just BP bashing; almost all of what I say applies equally to other gradient descent connectionist learning algorithms.) A great deal of current research is concerned with both simplifying such nets to more minimal structures, and elaborating the nets (e.g., with recurrence) to solve more difficult problems. Also, BP performance is known to be highly variable under stochastic variation. Consequently most investigators do use an 'outer-loop' iteration, using multiple restarts to improve their confidence in the solution found. The Genetic Algoritms (GAs) can help with these connectionist problems. (To clarify, when I talk of 'evolutionary algorithms' I have some variant of John Holland's Genetic Algorithm in mind because I believe it to be the most powerful. But there are other evolution- like algorithms (eg, Ackley, Fogel Jr. & Sr.) and these may also prove useful to connectionists.) Second, there is a considerable body of work that shows that evolutionary search is far from random. GA's are extremely effective sampling procedures, at least on some kinds of problems. (See Goldberg's book or the most recent GA proceedings for characterizations of what makes a problem 'GA-hard'.) Further, there are reasons to believe that connectionist nets are a good problem for the GA: the GLOBAL SAMPLING performed by the GA is very compimentary to the LOCAL SEARCH performed by gradient descent procedures like BP. Bridges complains that we are compounding ignorance when we try to consider hybrids of connectionist and GA algorithms [clay at cs.cmu.edu 7 Nov 89]. But we are beginning to understand the basic features of connectionist search (as function approximators, via analysis of internal structure,etc.), and there are substantial things known about the GA, too (e.g., Holland's Schema Theorem and its progeny). These foundations do suggest deliberate research strategies and immediately eliminate others (eg, some of the naive ways in which we might encode a network onto a GA string). There are a tremendous number of ways the techniques can be combined, with the GA as simply an outer loop around a conventional BP simulation being one of the least imaginative. For example, when used in conjunction with the GA there is a very good question as to how long each BP training cycle must be in order to provide a useful fitness measure for the GA. Preliminary results of ours suggest much shorter training times are possible. Similarly, use of the GA seems to tolerate quicker search (i.e., higher learning rate and momentum values) than typical. Another rich dimension is just how a GA bit string is related to a net, from a literal encoding of every real number weight at one extreme to complex developmental programs that build nets at the other. One feature of these hybrids that should not be underestimated is that the GA offers a very natural mechanism for introducing PARALLELISM into connectionist algorithms, since each individual in a population can be evaluated independently. We have had some success exploiting this parallelism in two implementations, one in a SIMD (Connection Machine) environment and one using a heterogeneous mess of distributed computers. Finally we shouldn't let computer science drive the entire agenda. Theoretical biology and evolutionary neurophysiology have some extremely important and unresolved questions that models like these can help to address, for example concerning complex, Lamarckian-like interactions between the learning of individuals and the evolution of species. (I think the Harp et al. simulation may be particularly useful to evolutionary neurophysiologists.) One of the things that makes connectionism most exciting is that the same class of systems that interest (some) mathematicians as new statistical techniques also interest neuroscientists as a theory to coordinate their data collection. I think GA/connectionist hybrids are important for similar reasons: they make sense as algorithms AND they make sense as models of natural phenomena. This note has been long on enthusiasm and short on specifics. But some results have already been reported, by me and others. And I expect there to be many new results reported at the upcoming sessions (at IJCNN in Washington and at the NIPS workshop) devoted to this topic. So watch this space. Richard K. Belew rik%cs at ucsd.edu Assistant Professor CSE Department (C-014) UCSD San Diego, CA 92093 619 / 534-2601 or 534-5288 (messages) From mike at bucasb.BU.EDU Fri Nov 10 00:57:00 1989 From: mike at bucasb.BU.EDU (Michael Cohen) Date: Fri, 10 Nov 89 00:57:00 EST Subject: NEURAL NET SUMMER SCHOOL AT WANG INSTITUTE Message-ID: <8911100557.AA24680@bucasb.bu.edu> BOSTON UNIVERSITY, A WORLD LEADER IN NEURAL NETWORK RESEARCH AND TECHNOLOGY, PRESENTS TWO MAJOR SCIENTIFIC EVENTS: MAY 6--11, 1990 NEURAL NETWORKS: FROM FOUNDATIONS TO APPLICATIONS A self-contained systematic course by leading neural architects who know the field as only its creators can. MAY 11--13, 1990 NEURAL NETWORKS FOR AUTOMATIC TARGET RECOGNITION An international research conference presenting INVITED and CONTRIBUTED papers, herewith solicited, on one of the most active research topics in science and technology today. SPONSORED BY THE CENTER FOR ADAPTIVE SYSTEMS AND THE WANG INSTITUTE OF BOSTON UNIVERSITY WITH PARTIAL SUPPORT FROM THE AIR FORCE OFFICE OF SCIENTIFIC RESEARCH ----------------------------------------------------------------------------- CALL FOR PAPERS --------------- NEURAL NETWORKS FOR AUTOMATIC TARGET RECOGNITION MAY 11--13, 1990 This research conference at the cutting edge of neural network science and technology will bring together leading experts in academe, government, and industry to present their latest results on automatic target recognition in invited lectures and contributed posters. Automatic target recognition is a key process in systems designed for vision and image processing, speech and time series prediction, adaptive pattern recognition, and adaptive sensory-motor control and robotics. It is one of the areas emphasized by the DARPA Neural Networks Program, and has attracted intense research activity around the world. Invited lecturers include: JOE BROWN, Martin Marietta, "Multi-Sensor ATR using Neural Nets" GAIL CARPENTER, Boston University, "Target Recognition by Adaptive Resonance: ART for ATR" NABIL FARHAT, University of Pennsylvania, "Bifurcating Networks for Target Recognition" STEPHEN GROSSBERG, Boston University, "Recent Results on Self-Organizing ATR Networks" ROBERT HECHT-NIELSEN, HNC, "Spatiotemporal Attention Focusing by Expectation Feedback" KEN JOHNSON, Hughes Aircraft, "The Application of Neural Networks to the Acquisition and Tracking of Maneuvering Tactical Targets in High Clutter IR Imagery" PAUL KOLODZY, MIT Lincoln Laboratory, "A Multi-Dimensional ATR System" MICHAEL KUPERSTEIN, Neurogen, "Adaptive Sensory-Motor Coordination using the INFANT Controller" YANN LECUN, AT&T Bell Labs, "Structured Back Propagation Networks for Handwriting Recognition" CHRISTOPHER SCOFIELD, Nestor, "Neural Network Automatic Target Recognition by Active and Passive Sonar Signals" STEVEN SIMMES, Science Applications International Co., "Massively Parallel Approaches to Automatic Target Recognition" ALEX WAIBEL, Carnegie Mellon University, "Patterns, Sequences and Variability: Advances in Connectionist Speech Recognition" ALLEN WAXMAN, MIT Lincoln Laboratory, "Invariant Learning and Recognition of 3D Objects from Temporal View Sequences" FRED WEINGARD, Booz-Allen and Hamilton, "Current Status and Results of Two Major Government Programs in Neural Network-Based ATR" BARBARA YOON, DARPA, "DARPA Artificial Neural Networks Technology Program: Automatic Target Recognition" ------------------------------------------------------ CALL FOR PAPERS---ATR POSTER SESSION: A featured poster session on ATR neural network research will be held on May 12, 1990. Attendees who wish to present a poster should submit 3 copies of an extended abstract (1 single-spaced page), postmarked by March 1, 1990, for refereeing. Include with the abstract the name, address, and telephone number of the corresponding author. Mail to: ATR Poster Session, Neural Networks Conference, Wang Institute of Boston University, 72 Tyng Road, Tyngsboro, MA 01879. Authors will be informed of abstract acceptance by March 31, 1990. SITE: The Wang Institute possesses excellent conference facilities on a beautiful 220-acre rustic setting. It is easily reached from Boston's Logan Airport and Route 128. REGISTRATION FEE: Regular attendee--$90; full-time student--$70. Registration fee includes admission to all lectures and poster session, one reception, two continental breakfasts, one lunch, one dinner, daily morning and afternoon coffee service. STUDENTS: Read below about FELLOWSHIP support. REGISTRATION: To register by telephone with VISA or MasterCard call (508) 649-9731 between 9:00AM--5:00PM (EST). To register by FAX, fill out the registration form and FAX back to (508) 649-6926. To register by mail, complete the registration form and mail with your full form of payment as directed. Make check payable in U.S. dollars to "Boston University". See below for Registration Form. To register by electronic mail, use the address "rosenber at bu-tyng.bu.edu". On-site registration on a space-available basis will take place from 1:00--5:00PM on Friday, May 11. A RECEPTION will be held from 3:00--5:00PM on Friday, May 11. LECTURES begin at 5:00PM on Friday, May 11 and conclude at 1:00PM on Sunday, May 13. ------------------------------------------------------------------------------ NEURAL NETWORKS: FROM FOUNDATIONS TO APPLICATIONS MAY 6--11, 1989 This in-depth, systematic, 5-day course is based upon the world's leading graduate curriculum in the technology, computation, mathematics, and biology of neural networks. Developed at the Center for Adaptive Systems (CAS) and the Graduate Program in Cognitive and Neural Systems (CNS) of Boston University, twenty-eight hours of the course will be taught by six CAS/CNS faculty. Three distinguished guest lecturers will present eight hours of the course. COURSE OUTLINE -------------- MAY 7, 1990 ----------- MORNING SESSION (PROFESSOR GROSSBERG) HISTORICAL OVERVIEW: Introduction to the binary, linear, and continuous-nonlinear streams of neural network research: McCulloch-Pitts, Rosenblatt, von Neumann; Anderson, Kohonen, Widrow; Hodgkin-Huxley, Hartline-Ratliff, Grossberg. CONTENT ADDRESSABLE MEMORY: Classification and analysis of neural network models for absolutely stable CAM. Models include: Cohen-Grossberg, additive, shunting, Brain-State-In-A-Box, Hopfield, Boltzmann Machine, McCulloch-Pitts, masking field, bidirectional associative memory. COMPETITIVE DECISION MAKING: Analysis of asynchronous variable-load parallel processing by shunting competitive networks; solution of noise-saturation dilemma; classification of feedforward networks: automatic gain control, ratio processing, Weber law, total activity normalization, noise suppression, pattern matching, edge detection, brightness constancy and contrast, automatic compensation for variable illumination or other background energy distortions; classification of feedback networks: influence of nonlinear feedback signals, notably sigmoid signals, on pattern transformation and memory storage, winner-take-all choices, partial memory compression, tunable filtering, quantization and normalization of total activity, emergent boundary segmentation; method of jumps for classifying globally consistent and inconsistent competitive decision schemes. ASSOCIATIVE LEARNING: Derivation of associative equations for short-term memory and long-term memory. Overview and analysis of associative outstars, instars, computational maps, avalanches, counterpropagation nets, adaptive bidrectional associative memories. Analysis of unbiased associative pattern learning by asynchronous parallel sampling channels; classification of associative learning laws. AFTERNOON SESSION (PROFESSORS JORDAN AND MINGOLLA) COMBINATORIAL OPTIMIZATION PERCEPTRONS: Adeline, Madeline, delta rule, gradient descent, adaptive statistical predictor, nonlinear separability. INTRODUCTION TO BACK PROPAGATION: Supervised learning of multidimensional nonlinear maps, NETtalk, image compression, robotic control. RECENT DEVELOPMENTS OF BACK PROPAGATION: This two-hour guest tutorial lecture will provide a systematic review of recent developments of the back propagation learning network, especially focussing on recurrent back propagation variations and applications to outstanding technological problems. EVENING SESSION: DISCUSSIONS WITH TUTORS MAY 8, 1990 ----------- MORNING SESSION (PROFESSORS CARPENTER AND GROSSBERG) ADAPTIVE PATTERN RECOGNITION: Adaptive filtering; contrast enhancement; competitive learning of recognition categories; adaptive vector quantization; self-organizing computational maps; statistical properties of adaptive weights; learning stability and causes of instability. INTRODUCTION TO ADAPTIVE RESONANCE THEORY: Absolutely stable recognition learning, role of learned top-down expectations; attentional priming; matching by 2/3 Rule; adaptive search; self-controlled hypothesis testing; direct access to globally optimal recognition code; control of categorical coarseness by attentional vigilance; comparison with relevant behavioral and brain data to emphasize biological basis of ART computations. ANALYSIS OF ART 1: Computational analysis of ART 1 architecture for self-organized real-time hypothesis testing, learning, and recognition of arbitrary sequences of binary input patterns. AFTERNOON SESSION (PROFESSOR CARPENTER) ANALYSIS OF ART 2: Computational analysis of ART 2 architecture for self-organized real-time hypothesis testing, learning, and recognition for arbitrary sequences of analog or binary input patterns. ANALYSIS OF ART 3: Computational analysis of ART 3 architecture for self-organized real-time hypothesis testing, learning, and recognition within distributed network hierarchies; role of chemical transmitter dynamics in forming a memory representation distinct from short-term memory and long-term memory; relationships to brain data concerning neuromodulators and synergetic ionic and transmitter interactions. SELF-ORGANIZATION OF INVARIANT PATTERN RECOGNITION CODES: Computational analysis of self-organizing ART architectures for recognizing noisy imagery undergoing changes in position, rotation, and size. NEOCOGNITION: Recognition and completion of images by hierarchical bottom-up filtering and top-down attentive feedback. EVENING SESSION: DISCUSSIONS WITH TUTORS MAY 9, 1990 ----------- MORNING SESSION (PROFESSORS GROSSBERG & MINGOLLA) VISION AND IMAGE PROCESSING: Introduction to Boundary Contour System for emergent segmentation and Feature Contour System for filling-in after compensation for variable illumination; image compression, orthogonalization, and reconstruction; multidimensional filtering, multiplexing, and fusion; coherent boundary detection, regularization, self-scaling, and completion; compensation for variable illumination sources, including artificial sensors (infrared sensors, laser radars); filling-in of surface color and form; 3-D form from shading, texture, stereo, and motion; parallel processing of static form and moving form; motion capture and induced motion; synthesis of static form and motion form representations. AFTERNOON SESSION (PROFESSORS BULLOCK, COHEN, & GROSSBERG) ADAPTIVE SENSORY-MOTOR CONTROL AND ROBOTICS: Overview of recent progress in adaptive sensory-motor control and related robotics research. Reaching to, grasping, and transporting objects of variable mass and form under visual guidance in a cluttered environment will be used as a target behavioral competence to clarify subproblems of real-time adaptive sensory-motor control. The balance of the tutorial will be spent detailing neural network modules that solve various subproblems. Topics include: Self-organizing networks for real-time control of eye movements, arm movements, and eye-arm coordination; learning of invariant body-centered target position maps; learning of intermodal associative maps; real-time trajectory formation; adaptive vector encoders; circular reactions between action and sensory feedback; adaptive control of variable speed movements; varieties of error signals; supportive behavioral and neural data; inverse kinematics; automatic compensation for unexpected perturbations; independent adaptive control of force and position; adaptive gain control by cerebellar learning; position-dependent sampling from spatial maps; predictive motor planning and execution. SPEECH PERCEPTION AND PRODUCTION: Hidden Markov models; self-organization of speech perception and production codes; eighth nerve Average Localized Synchrony Response; phoneme recognition by back propagation, time delay networks, and vector quantization. MAY 10, 1990 ------------ MORNING SESSION (PROFESSORS COHEN, GROSSBERG, & MERRILL) SPEECH PERCEPTION AND PRODUCTION: Disambiguation of coarticulated vowels and consonants; dynamics of working memory; multiple-scale adaptive coding by masking fields; categorical perception; phonemic restoration; contextual disambiguation of speech tokens; resonant completion and grouping of noisy variable-rate speech streams. REINFORCEMENT LEARNING AND PREDICTION: Recognition learning, reinforcement learning, and recall learning are the 3 R's of neural network learning. Reinforcement learning clarifies how external events interact with internal organismic requirements to trigger learning processes capable of focussing attention upon and generating appropriate actions towards motivationally desired goals. A neural network model will be derived to show how reinforcement learning and recall learning can self-organize in response to asynchronous series of significant and irrelevant events. These mechanisms also control selective forgetting of memories that are no longer predictive, adaptive timing of behavioral responses, and self-organization of goal directed problem solvers. AFTERNOON SESSION (PROFESSORS GROSSBERG & MERRILL AND DR. HECHT-NIELSEN) REINFORCEMENT LEARNING AND PREDICTION: Analysis of drive representations, adaptive critics, conditioned reinforcers, role of motivational feedback in focusing attention on predictive data; attentional blocking and unblocking; adaptively timed problem solving; synthesis of perception, recognition, reinforcement, recall, and robotics mechanisms into a total neural architecture; relationship to data about hypothalamus, hippocampus, neocortex, and related brain regions. RECENT DEVELOPMENTS IN THE NEUROCOMPUTER INDUSTRY: This two-hour guest tutorial will provide an overview of the growth and prospects of the burgeoning neurocomputer industry by one of its most important leaders. EVENING SESSION: DISCUSSIONS WITH TUTORS MAY 11, 1990 ------------ MORNING SESSION (DR. FAGGIN) VLSI IMPLEMENTATION OF NEURAL NETWORKS: This is a four-hour self-contained tutorial on the application and development of VLSI techniques for creating compact real-time chips embodying neural network designs for applications in technology. Review of neural networks from a hardware implementation perspective; hardware requirements and alternatives; dedicated digital implementation of neural networks; neuromorphic design methodology using VLSI CMOS technology; applications and performance of neuromorphic implementations; comparison of neuromorphic and digital hardware; future prospectus. ---------------------------------------------------------------------------- COURSE FACULTY FROM BOSTON UNIVERSITY ------------------------------------- STEPHEN GROSSBERG, Wang Professor of CNS, as well as Professor of Mathematics, Psychology, and Biomedical Engineering, is one of the world's leading neural network pioneers and most versatile neural architects; Founder and 1988 President of the International Neural Network Society (INNS); Founder and Co-Editor-in-Chief of the INNS journal "Neural Networks"; an editor of the journals "Neural Computation", "Cognitive Science", and "IEEE Expert"; Founder and Director of the Center for Adaptive Systems; General Chairman of the 1987 IEEE First International Conference on Neural Networks (ICNN); Chief Scientist of Hecht-Nielsen Neurocomputer Company (HNC); and one of the four technical consultants to the national DARPA Neural Network Study. He is author of 200 articles and books about neural networks, including "Neural Networks and Natural Intelligence" (MIT Press, 1988), "Neural Dynamics of Adaptive Sensory-Motor Control" (with Michael Kuperstein, Pergamon Press, 1989), "The Adaptive Brain, Volumes I and II" (Elsevier/North-Holland, 1987), "Studies of Mind and Brain" (Reidel Press, 1982), and the forthcoming "Pattern Recognition by Self-Organizing Neural Networks" (with Gail Carpenter). GAIL CARPENTER is Professor of Mathematics and CNS; Co-Director of the CNS Graduate Program; 1989 Vice President of the International Neural Network Society (INNS); Organization Chairman of the 1988 INNS annual meeting; Session Chairman at the 1989 and 1990 IEEE/INNS International Joint Conference on Neural Networks (IJCNN); one of four technical consultants to the national DARPA Neural Network Study; editor of the journals "Neural Networks", "Neural Computation", and "Neural Network Review"; and a member of the scientific advisory board of HNC. A leading neural architect, Carpenter is especially well-known for her seminal work on developing the adaptive resonance theory architectures (ART 1, ART 2, ART 3) for adaptive pattern recognition. MICHAEL COHEN, Associate Professor of Computer Science and CNS, is a leading architect of neural networks for content addressable memory (Cohen-Grossberg model), vision (Feature Contour System), and speech (Masking Fields); editor of "Neural Networks"; Session Chairman at the 1987 ICNN, and the 1989 IJCNN; and member of the DARPA Neural Network Study panel on Simulation/Emulation Tools and Techniques. ENNIO MINGOLLA, Assistant Professor of Psychology and CNS, is holder of one of the first patented neural network architectures for vision and image processing (Boundary Contour System); Co-Organizer of the 3rd Workshop on Human and Machine Vision in 1985; editor of the journals "Neural Networks" and "Ecological Psychology"; member of the DARPA Neural Network Study panel of Adaptive Knowledge Processing; consultant to E.I. duPont de Nemours, Inc.; Session Chairman for vision and image processing at the 1987 ICNN, and the 1988 INNS meetings. DANIEL BULLOCK, Assistant Professor of Psychology and CNS, is developer of neural network models for real-time adaptive sensory-motor control of arm movements and eye-arm coordination, notably the VITE and FLETE models for adaptive control of multi-joint trajectories; editor of "Neural Networks"; Session Chairman for adaptive sensory-motor control and robotics at the 1987 ICNN and the 1988 INNS meetings; invited speaker at the 1990 IJCNN. JOHN MERRILL, Assistant Professor of Mathematics and CNS, is developing neural network models for adaptive pattern recognition, speech recognition, reinforcement learning, and adaptive timing in problem solving behavior, after having received his Ph.D. in mathematics from the University of Wisconsin at Madison, and completing postdoctoral research in computer science and linguistics at Indiana University. GUEST LECTURERS --------------- FEDERICO FAGGIN is co-founder and president of Synaptics, Inc. Dr. Faggin developed the Silicon Gate Technology at Fairchild Semiconductor. He also designed the first commercial circuit using Silicon Gate Technology: the 3708, an 8-bit analog multiplexer. At Intel Corporation he was responsible for designing what was to become the first microprocessor---the 4000 family, also called MCS-4. He and Hal Feeney designed the 8008, the first 8-bit microprocessor introduced in 1972, and later Faggin conceived the 8080 and with M. Shima designed it. The 8080 was the first high-performance 8-bit microprocessor. At Zilog Inc., Faggin conceived the Z80 microprocessor family and directed the design of the Z80 CPU. Faggin also started Cygnet Technologies, which developed a voice and data communication peripheral for the personal computer. In 1986 Faggin co-founded Synaptics Inc., a company dedicated to the creation of a new type of VLSI hardware for artificial neural networks and other machine intelligence applications. Faggin is the recipient of the 1988 Marconi Fellowship Award for his contributions to the birth of the microprocessor. ROBERT HECHT-NIELSEN is co-founder and chairman of the Board of Directors of Hecht-Nielsen Neurocomputer Corporation (HNC), a pioneer in neurocomputer technology and the application of neural networks, and a recognized leader in the field. Prior to the formation of HNC, he founded and managed the neurocomputer development and neural network applications at TRW (1983--1986) and Motorola (1979--1983). He has been active in neural network technology and neurocomputers since 1961 and earned his Ph.D. in mathematics in 1974. He is currently a visiting lecturer in the Electrical Engineering Department at the University of California at San Diego, and is the author of influential technical reports and papers on neurocomputers, neural networks, pattern recognition, signal processing algorithms, and artificial intelligence. MICHAEL JORDAN is an Assistant Professor of Brain and Cognitive Sciences at MIT. One of the key developers of the recurrent back propagation algorithms, Professor Jordan's research is concerned with learning in recurrent networks and with the use of networks as forward models in planning and control. His interest in interdisciplinary research on neural networks is founded in his training for a Bachelors degree in Psychology, a Masters degree in Mathematics, and a Ph.D. in Cognitive Science from the University of California at San Diego. He was a postdoctoral researcher in Computer Science at the University of Massachusetts at Amherst before assuming his present position at MIT. ---------------------------------------------------------- REGISTRATION FEE: Regular attendee--$950; full-time student--$250. Registration fee includes five days of tutorials, course notebooks, one reception, five continental breakfasts, five lunches, four dinners, daily morning and afternoon coffee service, evening discussion sessions with leading neural architects. REGISTRATION: To register by telephone with VISA or MasterCard call (508) 649-9731 between 9:00AM--5:00PM (EST). To register by FAX, fill out the registration form and FAX back to (508) 649-6926. To register by mail, complete the registration form and mail with you full form of payment as directed. Make check payable in U.S. dollars to "Boston University". See below for Registration Form. To register by electronic mail, use the address "rosenber at bu-tyng.bu.edu". On-site registration on a space-available basis will take place from 2:00--7:00PM on Sunday, May 6 and from 7:00--8:00AM on Monday, May 7, 1990. A RECEPTION will be held from 4:00--7:00PM on Sunday, May 6. LECTURES begin at 8:00AM on Monday, May 7 and conclude at 12:30PM on Friday, May 11. STUDENT FELLOWSHIPS supporting travel, registration, and lodging for the Course and the Research Conference are available to full-time graduate students in a PhD program. Applications must be postmarked by March 1, 1990. Send curriculum vitae, a one-page essay describing your interest in neural networks, and a letter from a faculty advisor to: Student Fellowships, Neural Networks Course, Wang Institute of Boston University, 72 Tyng Road, Tyngsboro, MA 01879. CNS FELLOWSHIP FUND: Net revenues from the course will endow fellowships for Ph.D. candidates in the CNS Graduate Program. Corporate and individual gifts to endow CNS Fellowships are also welcome. Please write: Cognitive and Neural Systems Fellowship Fund, Center for Adaptive Systems, Boston University, 111 Cummington Street, Boston, MA 02215. ------------------------------------------------------------------------------ REGISTRATION FOR COURSE AND RESEARCH CONFERENCE Course: Neural Network Foundations and Applications, May 6--11, 1990 Research Conference: Neural Networks for Automatic Target Recognition, May 11--13, 1990 NAME: _________________________________________________________________ ORGANIZATION (for badge): _____________________________________________ MAILING ADDRESS: ______________________________________________________ ______________________________________________________ CITY/STATE/COUNTRY: ___________________________________________________ POSTAL/ZIP CODE: ______________________________________________________ TELEPHONE(S): _________________________________________________________ COURSE RESEARCH CONFERENCE ------ ------------------- [ ] regular attendee $950 [ ] regular attendee $90 [ ] full-time student $250 [ ] full-time student $70 (limited number of spaces) (limited number of spaces) [ ] Gift to CNS Fellowship Fund TOTAL PAYMENT: $________ FORM OF PAYMENT: [ ] check or money order (payable in U.S. dollars to Boston University) [ ] VISA [ ] MasterCard Card Number: ______________________________________________ Expiration Date: ______________________________________________ Signature: ______________________________________________ Please complete and mail to: Neural Networks Wang Institute of Boston University 72 Tyng Road Tyngsboro, MA 01879 USA To register by telephone, call: (508) 649-9731. HOTEL RESERVATIONS: Room blocks have been reserved at 3 hotels near the Wang Institute. Hotel names, rates, and telephone numbers are listed below. A shuttle bus will take attendees to and from the hotels for the Course and Research Conference. Attendees should make their own reservations by calling the hotel. The special conference rate applies only if you mention the name and dates of the meeting when making the reservations. Sheraton Tara Red Roof Inn Stonehedge Inn Nashua, NH Nashua, NH Tyngsboro, MA (603) 888-9970 (603) 888-1893 (508) 649-4342 $70/night+tax $39.95/night+tax $89/night+tax The hotels in Nashua are located approximately 5 miles from the Wang Institute. A shuttle bus will be provided. ------------------------------------------------------------------------------- From mike at bucasb.BU.EDU Fri Nov 10 00:57:00 1989 From: mike at bucasb.BU.EDU (Michael Cohen) Date: Fri, 10 Nov 89 00:57:00 EST Subject: NEURAL NET SUMMER SCHOOL AT WANG INSTITUTE Message-ID: <8911100557.AA24680@bucasb.bu.edu> BOSTON UNIVERSITY, A WORLD LEADER IN NEURAL NETWORK RESEARCH AND TECHNOLOGY, PRESENTS TWO MAJOR SCIENTIFIC EVENTS: MAY 6--11, 1990 NEURAL NETWORKS: FROM FOUNDATIONS TO APPLICATIONS A self-contained systematic course by leading neural architects who know the field as only its creators can. MAY 11--13, 1990 NEURAL NETWORKS FOR AUTOMATIC TARGET RECOGNITION An international research conference presenting INVITED and CONTRIBUTED papers, herewith solicited, on one of the most active research topics in science and technology today. SPONSORED BY THE CENTER FOR ADAPTIVE SYSTEMS AND THE WANG INSTITUTE OF BOSTON UNIVERSITY WITH PARTIAL SUPPORT FROM THE AIR FORCE OFFICE OF SCIENTIFIC RESEARCH ----------------------------------------------------------------------------- CALL FOR PAPERS --------------- NEURAL NETWORKS FOR AUTOMATIC TARGET RECOGNITION MAY 11--13, 1990 This research conference at the cutting edge of neural network science and technology will bring together leading experts in academe, government, and industry to present their latest results on automatic target recognition in invited lectures and contributed posters. Automatic target recognition is a key process in systems designed for vision and image processing, speech and time series prediction, adaptive pattern recognition, and adaptive sensory-motor control and robotics. It is one of the areas emphasized by the DARPA Neural Networks Program, and has attracted intense research activity around the world. Invited lecturers include: JOE BROWN, Martin Marietta, "Multi-Sensor ATR using Neural Nets" GAIL CARPENTER, Boston University, "Target Recognition by Adaptive Resonance: ART for ATR" NABIL FARHAT, University of Pennsylvania, "Bifurcating Networks for Target Recognition" STEPHEN GROSSBERG, Boston University, "Recent Results on Self-Organizing ATR Networks" ROBERT HECHT-NIELSEN, HNC, "Spatiotemporal Attention Focusing by Expectation Feedback" KEN JOHNSON, Hughes Aircraft, "The Application of Neural Networks to the Acquisition and Tracking of Maneuvering Tactical Targets in High Clutter IR Imagery" PAUL KOLODZY, MIT Lincoln Laboratory, "A Multi-Dimensional ATR System" MICHAEL KUPERSTEIN, Neurogen, "Adaptive Sensory-Motor Coordination using the INFANT Controller" YANN LECUN, AT&T Bell Labs, "Structured Back Propagation Networks for Handwriting Recognition" CHRISTOPHER SCOFIELD, Nestor, "Neural Network Automatic Target Recognition by Active and Passive Sonar Signals" STEVEN SIMMES, Science Applications International Co., "Massively Parallel Approaches to Automatic Target Recognition" ALEX WAIBEL, Carnegie Mellon University, "Patterns, Sequences and Variability: Advances in Connectionist Speech Recognition" ALLEN WAXMAN, MIT Lincoln Laboratory, "Invariant Learning and Recognition of 3D Objects from Temporal View Sequences" FRED WEINGARD, Booz-Allen and Hamilton, "Current Status and Results of Two Major Government Programs in Neural Network-Based ATR" BARBARA YOON, DARPA, "DARPA Artificial Neural Networks Technology Program: Automatic Target Recognition" ------------------------------------------------------ CALL FOR PAPERS---ATR POSTER SESSION: A featured poster session on ATR neural network research will be held on May 12, 1990. Attendees who wish to present a poster should submit 3 copies of an extended abstract (1 single-spaced page), postmarked by March 1, 1990, for refereeing. Include with the abstract the name, address, and telephone number of the corresponding author. Mail to: ATR Poster Session, Neural Networks Conference, Wang Institute of Boston University, 72 Tyng Road, Tyngsboro, MA 01879. Authors will be informed of abstract acceptance by March 31, 1990. SITE: The Wang Institute possesses excellent conference facilities on a beautiful 220-acre rustic setting. It is easily reached from Boston's Logan Airport and Route 128. REGISTRATION FEE: Regular attendee--$90; full-time student--$70. Registration fee includes admission to all lectures and poster session, one reception, two continental breakfasts, one lunch, one dinner, daily morning and afternoon coffee service. STUDENTS: Read below about FELLOWSHIP support. REGISTRATION: To register by telephone with VISA or MasterCard call (508) 649-9731 between 9:00AM--5:00PM (EST). To register by FAX, fill out the registration form and FAX back to (508) 649-6926. To register by mail, complete the registration form and mail with your full form of payment as directed. Make check payable in U.S. dollars to "Boston University". See below for Registration Form. To register by electronic mail, use the address "rosenber at bu-tyng.bu.edu". On-site registration on a space-available basis will take place from 1:00--5:00PM on Friday, May 11. A RECEPTION will be held from 3:00--5:00PM on Friday, May 11. LECTURES begin at 5:00PM on Friday, May 11 and conclude at 1:00PM on Sunday, May 13. ------------------------------------------------------------------------------ NEURAL NETWORKS: FROM FOUNDATIONS TO APPLICATIONS MAY 6--11, 1989 This in-depth, systematic, 5-day course is based upon the world's leading graduate curriculum in the technology, computation, mathematics, and biology of neural networks. Developed at the Center for Adaptive Systems (CAS) and the Graduate Program in Cognitive and Neural Systems (CNS) of Boston University, twenty-eight hours of the course will be taught by six CAS/CNS faculty. Three distinguished guest lecturers will present eight hours of the course. COURSE OUTLINE -------------- MAY 7, 1990 ----------- MORNING SESSION (PROFESSOR GROSSBERG) HISTORICAL OVERVIEW: Introduction to the binary, linear, and continuous-nonlinear streams of neural network research: McCulloch-Pitts, Rosenblatt, von Neumann; Anderson, Kohonen, Widrow; Hodgkin-Huxley, Hartline-Ratliff, Grossberg. CONTENT ADDRESSABLE MEMORY: Classification and analysis of neural network models for absolutely stable CAM. Models include: Cohen-Grossberg, additive, shunting, Brain-State-In-A-Box, Hopfield, Boltzmann Machine, McCulloch-Pitts, masking field, bidirectional associative memory. COMPETITIVE DECISION MAKING: Analysis of asynchronous variable-load parallel processing by shunting competitive networks; solution of noise-saturation dilemma; classification of feedforward networks: automatic gain control, ratio processing, Weber law, total activity normalization, noise suppression, pattern matching, edge detection, brightness constancy and contrast, automatic compensation for variable illumination or other background energy distortions; classification of feedback networks: influence of nonlinear feedback signals, notably sigmoid signals, on pattern transformation and memory storage, winner-take-all choices, partial memory compression, tunable filtering, quantization and normalization of total activity, emergent boundary segmentation; method of jumps for classifying globally consistent and inconsistent competitive decision schemes. ASSOCIATIVE LEARNING: Derivation of associative equations for short-term memory and long-term memory. Overview and analysis of associative outstars, instars, computational maps, avalanches, counterpropagation nets, adaptive bidrectional associative memories. Analysis of unbiased associative pattern learning by asynchronous parallel sampling channels; classification of associative learning laws. AFTERNOON SESSION (PROFESSORS JORDAN AND MINGOLLA) COMBINATORIAL OPTIMIZATION PERCEPTRONS: Adeline, Madeline, delta rule, gradient descent, adaptive statistical predictor, nonlinear separability. INTRODUCTION TO BACK PROPAGATION: Supervised learning of multidimensional nonlinear maps, NETtalk, image compression, robotic control. RECENT DEVELOPMENTS OF BACK PROPAGATION: This two-hour guest tutorial lecture will provide a systematic review of recent developments of the back propagation learning network, especially focussing on recurrent back propagation variations and applications to outstanding technological problems. EVENING SESSION: DISCUSSIONS WITH TUTORS MAY 8, 1990 ----------- MORNING SESSION (PROFESSORS CARPENTER AND GROSSBERG) ADAPTIVE PATTERN RECOGNITION: Adaptive filtering; contrast enhancement; competitive learning of recognition categories; adaptive vector quantization; self-organizing computational maps; statistical properties of adaptive weights; learning stability and causes of instability. INTRODUCTION TO ADAPTIVE RESONANCE THEORY: Absolutely stable recognition learning, role of learned top-down expectations; attentional priming; matching by 2/3 Rule; adaptive search; self-controlled hypothesis testing; direct access to globally optimal recognition code; control of categorical coarseness by attentional vigilance; comparison with relevant behavioral and brain data to emphasize biological basis of ART computations. ANALYSIS OF ART 1: Computational analysis of ART 1 architecture for self-organized real-time hypothesis testing, learning, and recognition of arbitrary sequences of binary input patterns. AFTERNOON SESSION (PROFESSOR CARPENTER) ANALYSIS OF ART 2: Computational analysis of ART 2 architecture for self-organized real-time hypothesis testing, learning, and recognition for arbitrary sequences of analog or binary input patterns. ANALYSIS OF ART 3: Computational analysis of ART 3 architecture for self-organized real-time hypothesis testing, learning, and recognition within distributed network hierarchies; role of chemical transmitter dynamics in forming a memory representation distinct from short-term memory and long-term memory; relationships to brain data concerning neuromodulators and synergetic ionic and transmitter interactions. SELF-ORGANIZATION OF INVARIANT PATTERN RECOGNITION CODES: Computational analysis of self-organizing ART architectures for recognizing noisy imagery undergoing changes in position, rotation, and size. NEOCOGNITION: Recognition and completion of images by hierarchical bottom-up filtering and top-down attentive feedback. EVENING SESSION: DISCUSSIONS WITH TUTORS MAY 9, 1990 ----------- MORNING SESSION (PROFESSORS GROSSBERG & MINGOLLA) VISION AND IMAGE PROCESSING: Introduction to Boundary Contour System for emergent segmentation and Feature Contour System for filling-in after compensation for variable illumination; image compression, orthogonalization, and reconstruction; multidimensional filtering, multiplexing, and fusion; coherent boundary detection, regularization, self-scaling, and completion; compensation for variable illumination sources, including artificial sensors (infrared sensors, laser radars); filling-in of surface color and form; 3-D form from shading, texture, stereo, and motion; parallel processing of static form and moving form; motion capture and induced motion; synthesis of static form and motion form representations. AFTERNOON SESSION (PROFESSORS BULLOCK, COHEN, & GROSSBERG) ADAPTIVE SENSORY-MOTOR CONTROL AND ROBOTICS: Overview of recent progress in adaptive sensory-motor control and related robotics research. Reaching to, grasping, and transporting objects of variable mass and form under visual guidance in a cluttered environment will be used as a target behavioral competence to clarify subproblems of real-time adaptive sensory-motor control. The balance of the tutorial will be spent detailing neural network modules that solve various subproblems. Topics include: Self-organizing networks for real-time control of eye movements, arm movements, and eye-arm coordination; learning of invariant body-centered target position maps; learning of intermodal associative maps; real-time trajectory formation; adaptive vector encoders; circular reactions between action and sensory feedback; adaptive control of variable speed movements; varieties of error signals; supportive behavioral and neural data; inverse kinematics; automatic compensation for unexpected perturbations; independent adaptive control of force and position; adaptive gain control by cerebellar learning; position-dependent sampling from spatial maps; predictive motor planning and execution. SPEECH PERCEPTION AND PRODUCTION: Hidden Markov models; self-organization of speech perception and production codes; eighth nerve Average Localized Synchrony Response; phoneme recognition by back propagation, time delay networks, and vector quantization. MAY 10, 1990 ------------ MORNING SESSION (PROFESSORS COHEN, GROSSBERG, & MERRILL) SPEECH PERCEPTION AND PRODUCTION: Disambiguation of coarticulated vowels and consonants; dynamics of working memory; multiple-scale adaptive coding by masking fields; categorical perception; phonemic restoration; contextual disambiguation of speech tokens; resonant completion and grouping of noisy variable-rate speech streams. REINFORCEMENT LEARNING AND PREDICTION: Recognition learning, reinforcement learning, and recall learning are the 3 R's of neural network learning. Reinforcement learning clarifies how external events interact with internal organismic requirements to trigger learning processes capable of focussing attention upon and generating appropriate actions towards motivationally desired goals. A neural network model will be derived to show how reinforcement learning and recall learning can self-organize in response to asynchronous series of significant and irrelevant events. These mechanisms also control selective forgetting of memories that are no longer predictive, adaptive timing of behavioral responses, and self-organization of goal directed problem solvers. AFTERNOON SESSION (PROFESSORS GROSSBERG & MERRILL AND DR. HECHT-NIELSEN) REINFORCEMENT LEARNING AND PREDICTION: Analysis of drive representations, adaptive critics, conditioned reinforcers, role of motivational feedback in focusing attention on predictive data; attentional blocking and unblocking; adaptively timed problem solving; synthesis of perception, recognition, reinforcement, recall, and robotics mechanisms into a total neural architecture; relationship to data about hypothalamus, hippocampus, neocortex, and related brain regions. RECENT DEVELOPMENTS IN THE NEUROCOMPUTER INDUSTRY: This two-hour guest tutorial will provide an overview of the growth and prospects of the burgeoning neurocomputer industry by one of its most important leaders. EVENING SESSION: DISCUSSIONS WITH TUTORS MAY 11, 1990 ------------ MORNING SESSION (DR. FAGGIN) VLSI IMPLEMENTATION OF NEURAL NETWORKS: This is a four-hour self-contained tutorial on the application and development of VLSI techniques for creating compact real-time chips embodying neural network designs for applications in technology. Review of neural networks from a hardware implementation perspective; hardware requirements and alternatives; dedicated digital implementation of neural networks; neuromorphic design methodology using VLSI CMOS technology; applications and performance of neuromorphic implementations; comparison of neuromorphic and digital hardware; future prospectus. ---------------------------------------------------------------------------- COURSE FACULTY FROM BOSTON UNIVERSITY ------------------------------------- STEPHEN GROSSBERG, Wang Professor of CNS, as well as Professor of Mathematics, Psychology, and Biomedical Engineering, is one of the world's leading neural network pioneers and most versatile neural architects; Founder and 1988 President of the International Neural Network Society (INNS); Founder and Co-Editor-in-Chief of the INNS journal "Neural Networks"; an editor of the journals "Neural Computation", "Cognitive Science", and "IEEE Expert"; Founder and Director of the Center for Adaptive Systems; General Chairman of the 1987 IEEE First International Conference on Neural Networks (ICNN); Chief Scientist of Hecht-Nielsen Neurocomputer Company (HNC); and one of the four technical consultants to the national DARPA Neural Network Study. He is author of 200 articles and books about neural networks, including "Neural Networks and Natural Intelligence" (MIT Press, 1988), "Neural Dynamics of Adaptive Sensory-Motor Control" (with Michael Kuperstein, Pergamon Press, 1989), "The Adaptive Brain, Volumes I and II" (Elsevier/North-Holland, 1987), "Studies of Mind and Brain" (Reidel Press, 1982), and the forthcoming "Pattern Recognition by Self-Organizing Neural Networks" (with Gail Carpenter). GAIL CARPENTER is Professor of Mathematics and CNS; Co-Director of the CNS Graduate Program; 1989 Vice President of the International Neural Network Society (INNS); Organization Chairman of the 1988 INNS annual meeting; Session Chairman at the 1989 and 1990 IEEE/INNS International Joint Conference on Neural Networks (IJCNN); one of four technical consultants to the national DARPA Neural Network Study; editor of the journals "Neural Networks", "Neural Computation", and "Neural Network Review"; and a member of the scientific advisory board of HNC. A leading neural architect, Carpenter is especially well-known for her seminal work on developing the adaptive resonance theory architectures (ART 1, ART 2, ART 3) for adaptive pattern recognition. MICHAEL COHEN, Associate Professor of Computer Science and CNS, is a leading architect of neural networks for content addressable memory (Cohen-Grossberg model), vision (Feature Contour System), and speech (Masking Fields); editor of "Neural Networks"; Session Chairman at the 1987 ICNN, and the 1989 IJCNN; and member of the DARPA Neural Network Study panel on Simulation/Emulation Tools and Techniques. ENNIO MINGOLLA, Assistant Professor of Psychology and CNS, is holder of one of the first patented neural network architectures for vision and image processing (Boundary Contour System); Co-Organizer of the 3rd Workshop on Human and Machine Vision in 1985; editor of the journals "Neural Networks" and "Ecological Psychology"; member of the DARPA Neural Network Study panel of Adaptive Knowledge Processing; consultant to E.I. duPont de Nemours, Inc.; Session Chairman for vision and image processing at the 1987 ICNN, and the 1988 INNS meetings. DANIEL BULLOCK, Assistant Professor of Psychology and CNS, is developer of neural network models for real-time adaptive sensory-motor control of arm movements and eye-arm coordination, notably the VITE and FLETE models for adaptive control of multi-joint trajectories; editor of "Neural Networks"; Session Chairman for adaptive sensory-motor control and robotics at the 1987 ICNN and the 1988 INNS meetings; invited speaker at the 1990 IJCNN. JOHN MERRILL, Assistant Professor of Mathematics and CNS, is developing neural network models for adaptive pattern recognition, speech recognition, reinforcement learning, and adaptive timing in problem solving behavior, after having received his Ph.D. in mathematics from the University of Wisconsin at Madison, and completing postdoctoral research in computer science and linguistics at Indiana University. GUEST LECTURERS --------------- FEDERICO FAGGIN is co-founder and president of Synaptics, Inc. Dr. Faggin developed the Silicon Gate Technology at Fairchild Semiconductor. He also designed the first commercial circuit using Silicon Gate Technology: the 3708, an 8-bit analog multiplexer. At Intel Corporation he was responsible for designing what was to become the first microprocessor---the 4000 family, also called MCS-4. He and Hal Feeney designed the 8008, the first 8-bit microprocessor introduced in 1972, and later Faggin conceived the 8080 and with M. Shima designed it. The 8080 was the first high-performance 8-bit microprocessor. At Zilog Inc., Faggin conceived the Z80 microprocessor family and directed the design of the Z80 CPU. Faggin also started Cygnet Technologies, which developed a voice and data communication peripheral for the personal computer. In 1986 Faggin co-founded Synaptics Inc., a company dedicated to the creation of a new type of VLSI hardware for artificial neural networks and other machine intelligence applications. Faggin is the recipient of the 1988 Marconi Fellowship Award for his contributions to the birth of the microprocessor. ROBERT HECHT-NIELSEN is co-founder and chairman of the Board of Directors of Hecht-Nielsen Neurocomputer Corporation (HNC), a pioneer in neurocomputer technology and the application of neural networks, and a recognized leader in the field. Prior to the formation of HNC, he founded and managed the neurocomputer development and neural network applications at TRW (1983--1986) and Motorola (1979--1983). He has been active in neural network technology and neurocomputers since 1961 and earned his Ph.D. in mathematics in 1974. He is currently a visiting lecturer in the Electrical Engineering Department at the University of California at San Diego, and is the author of influential technical reports and papers on neurocomputers, neural networks, pattern recognition, signal processing algorithms, and artificial intelligence. MICHAEL JORDAN is an Assistant Professor of Brain and Cognitive Sciences at MIT. One of the key developers of the recurrent back propagation algorithms, Professor Jordan's research is concerned with learning in recurrent networks and with the use of networks as forward models in planning and control. His interest in interdisciplinary research on neural networks is founded in his training for a Bachelors degree in Psychology, a Masters degree in Mathematics, and a Ph.D. in Cognitive Science from the University of California at San Diego. He was a postdoctoral researcher in Computer Science at the University of Massachusetts at Amherst before assuming his present position at MIT. ---------------------------------------------------------- REGISTRATION FEE: Regular attendee--$950; full-time student--$250. Registration fee includes five days of tutorials, course notebooks, one reception, five continental breakfasts, five lunches, four dinners, daily morning and afternoon coffee service, evening discussion sessions with leading neural architects. REGISTRATION: To register by telephone with VISA or MasterCard call (508) 649-9731 between 9:00AM--5:00PM (EST). To register by FAX, fill out the registration form and FAX back to (508) 649-6926. To register by mail, complete the registration form and mail with you full form of payment as directed. Make check payable in U.S. dollars to "Boston University". See below for Registration Form. To register by electronic mail, use the address "rosenber at bu-tyng.bu.edu". On-site registration on a space-available basis will take place from 2:00--7:00PM on Sunday, May 6 and from 7:00--8:00AM on Monday, May 7, 1990. A RECEPTION will be held from 4:00--7:00PM on Sunday, May 6. LECTURES begin at 8:00AM on Monday, May 7 and conclude at 12:30PM on Friday, May 11. STUDENT FELLOWSHIPS supporting travel, registration, and lodging for the Course and the Research Conference are available to full-time graduate students in a PhD program. Applications must be postmarked by March 1, 1990. Send curriculum vitae, a one-page essay describing your interest in neural networks, and a letter from a faculty advisor to: Student Fellowships, Neural Networks Course, Wang Institute of Boston University, 72 Tyng Road, Tyngsboro, MA 01879. CNS FELLOWSHIP FUND: Net revenues from the course will endow fellowships for Ph.D. candidates in the CNS Graduate Program. Corporate and individual gifts to endow CNS Fellowships are also welcome. Please write: Cognitive and Neural Systems Fellowship Fund, Center for Adaptive Systems, Boston University, 111 Cummington Street, Boston, MA 02215. ------------------------------------------------------------------------------ REGISTRATION FOR COURSE AND RESEARCH CONFERENCE Course: Neural Network Foundations and Applications, May 6--11, 1990 Research Conference: Neural Networks for Automatic Target Recognition, May 11--13, 1990 NAME: _________________________________________________________________ ORGANIZATION (for badge): _____________________________________________ MAILING ADDRESS: ______________________________________________________ ______________________________________________________ CITY/STATE/COUNTRY: ___________________________________________________ POSTAL/ZIP CODE: ______________________________________________________ TELEPHONE(S): _________________________________________________________ COURSE RESEARCH CONFERENCE ------ ------------------- [ ] regular attendee $950 [ ] regular attendee $90 [ ] full-time student $250 [ ] full-time student $70 (limited number of spaces) (limited number of spaces) [ ] Gift to CNS Fellowship Fund TOTAL PAYMENT: $________ FORM OF PAYMENT: [ ] check or money order (payable in U.S. dollars to Boston University) [ ] VISA [ ] MasterCard Card Number: ______________________________________________ Expiration Date: ______________________________________________ Signature: ______________________________________________ Please complete and mail to: Neural Networks Wang Institute of Boston University 72 Tyng Road Tyngsboro, MA 01879 USA To register by telephone, call: (508) 649-9731. HOTEL RESERVATIONS: Room blocks have been reserved at 3 hotels near the Wang Institute. Hotel names, rates, and telephone numbers are listed below. A shuttle bus will take attendees to and from the hotels for the Course and Research Conference. Attendees should make their own reservations by calling the hotel. The special conference rate applies only if you mention the name and dates of the meeting when making the reservations. Sheraton Tara Red Roof Inn Stonehedge Inn Nashua, NH Nashua, NH Tyngsboro, MA (603) 888-9970 (603) 888-1893 (508) 649-4342 $70/night+tax $39.95/night+tax $89/night+tax The hotels in Nashua are located approximately 5 miles from the Wang Institute. A shuttle bus will be provided. ------------------------------------------------------------------------------- From rudnick at cse.ogc.edu Fri Nov 10 13:38:27 1989 From: rudnick at cse.ogc.edu (Mike Rudnick) Date: Fri, 10 Nov 89 10:38:27 PST Subject: fault behaviour of NNs Message-ID: <8911101838.AA17796@cse.ogc.edu> I'm searching for references to work on the analysis and performance of neural net models with/under fault conditions (including VLSI models/implementations), eg, missing connections, frozen weights and activations, etc. I will post a summary of references received. As I'm doing this as a part of my search for a dissertation topic, I'm interested in making contact with anyone actively doing research in this area. Thanks, Mike From Dave.Touretzky at B.GP.CS.CMU.EDU Fri Nov 10 19:33:33 1989 From: Dave.Touretzky at B.GP.CS.CMU.EDU (Dave.Touretzky@B.GP.CS.CMU.EDU) Date: Fri, 10 Nov 89 19:33:33 EST Subject: tech report announcement Message-ID: <922.626747613@DST.BOLTZ.CS.CMU.EDU> A Connectionist Implementation of Cognitive Phonology Deirdre W. Wheeler [1] David S. Touretzky [2] [1] Department of Linguistics [2] School of Computer Science University of Pittsburgh Carnegie Mellon University Pittsburgh, PA 15260 Pittsburgh, PA 15213 Technical report number CMU-CS-89-144 ABSTRACT This paper reports on an initial implementation of Lakoff's theory of cognitive phonology in a connectionist network. Standard generative phonological theories require serial application of rules, which results in derivations with numerous intermediate states. This is incompatible with the connectionist goals of psychological and biological plausibility, and may also hinder learnability. Lakoff's theory of cognitive phonology offers a solution to some of these problems by providing an alternative way to think about derivations and ordered rules, and by eliminating the need for right-to-left iterative rule application. On the other hand, Lakoff's proposal presents certain computational difficulties due to its appeal to Harmony Theory. We present a reformulation of cognitive phonology using a novel clustering mechanism that completely eliminates iteration and permits an efficient feed-forward implementation. An earlier version of this paper was presented at the Berkeley Workshop on Constraints vs. Rules in Phonology, May 26-27, 1989. Note: the version of the system described in this report is considerably more refined than the version in the 1989 Cognitive Science Conference paper and the "Rules and Maps" tech report. The following abstract for a talk I've been giving recently at various institutions may better explain why this material should be of interest to cognitive scientists, not just linguists: A COMPUTATIONAL BASIS FOR PHONOLOGY Phonology is the study of the sound patterns of a language. It includes processes such as nasal assimilation, vowel harmony, tone shifting, and syllabification. The phonological structure of human languages is intricate, but it is also highly constrained and stunningly regular. The easy observability of phonological processes, their discrete, symbolic nature, and their rapid acquisition by very young children suggest that this may be a good domain in which to explore issues of rules and symbolic representations in the brain. In this talk I will give a brief sketch of George Lakoff's new theory of cognitive phonology, in which sequential derivations are eliminated by having all rules apply in parallel. I will then describe how our attempt to construct a connectionist implementation of the theory led us to revise it in significant ways. The architecture we developed resulted in a novel prediction of a constraint on insertion processes. Subsequent consultations with expert phonologists have so far confirmed this prediction. If correct, it represents the first step toward our long term goal of developing a computational explanation for why phonology looks the way it does. ---------------------------------------------------------------- HOW TO ORDER THIS REPORT: Copies of the report are available by writing to Ms. Catherine Copetas, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213. Email requests may be sent to copetas at cs.cmu.edu. Ask for report number CMU-CS-89-144. There is no charge for this report. From koch%HAMLET.BITNET at VMA.CC.CMU.EDU Fri Nov 10 20:40:38 1989 From: koch%HAMLET.BITNET at VMA.CC.CMU.EDU (Christof Koch) Date: Fri, 10 Nov 89 17:40:38 PST Subject: NEURAL NET SUMMER SCHOOL AT WANG INSTITUTE In-Reply-To: Your message <8911100557.AA24680@bucasb.bu.edu> dated 10-Nov-1989 Message-ID: <891110173624.2a605c2f@Hamlet.Caltech.Edu> Given that you guys have solved problems related to Brain Science, we would be infinitely grateful if you could provide us with a condensed version of the final result. Maybe ``42''? Signed: Not one of the World's Leading Expert's From dfausett at zach.fit.edu Sat Nov 11 10:47:51 1989 From: dfausett at zach.fit.edu ( Donald W. Fausett) Date: Sat, 11 Nov 89 10:47:51 EST Subject: E-mail Distribution List Message-ID: <8911111547.AA06552@zach.fit.edu> Please add me to your connectionists e-mail distribution list. Thanks. -- Don Fausett dfausett at zach.fit.edu From dfausett at zach.fit.edu Sat Nov 11 10:45:43 1989 From: dfausett at zach.fit.edu ( Donald W. Fausett) Date: Sat, 11 Nov 89 10:45:43 EST Subject: E-mail Distribution List Message-ID: <8911111545.AA06517@zach.fit.edu> Please add me to your connectionists e-mail distribution list. Thanks. -- Don Fausett dfausett at zach.fit.edu From khaines at GALILEO.ECE.CMU.EDU Sun Nov 12 16:35:26 1989 From: khaines at GALILEO.ECE.CMU.EDU (Karen Haines) Date: Sun, 12 Nov 89 16:35:26 EST Subject: IJCNN - Request for Volunteers Message-ID: <8911122135.AA02488@galileo.ece.cmu.edu> *************************************************************************** IJCNN - REQUEST FOR VOLUNTEERS *************************************************************************** This is the final call for volunteers to help at the IJCNN conference, to be held at the Omni Shorham Hotel in Washington D.C., on January 15-19, 1990. Full admittance to the conference and a copy of the proceedings is offered in exchange for your assistance throughout the conference. I would like to point out that student registration does not include proceedings. In general, each volunteer is expected to work one shift each day of the conference. Hours are approximately: AM shift - 7:00 am - Noon PM shift - Noon - 5:00 pm In addition, assistance may be required for the social events. There a re a few positions available, but I suggest that if you are interested you conatact ma as soon as possible. Below is a list of specific volunteer events. =================================== VOLUNTEER SCHEDULE OF EVENTS =================================== Sunday, January 14, 1990 ------------------------- 10am - 2pm Volunteer Shift Signup Registration time is based upon and Registration commitment date (i.e those whose commit earlier will get first choice ) 6pm - 7pm General Meeting **** Mandatory Meeting **** 7pm - 9pm Volunteer Welcome Party To sign up please contact: Karen Haines - Volunteer Coordinator 3138 Beechwood Blvd. Pittsburgh, PA 15217 office: (412) 268-3304 message: (412) 422-6026 email: khaines at galileo.ece.cmu.edu or, Nina Kowalski - Assistant Volunteer Coordinator 209 W. 29th St. FLR 2 Baltimore, MD 21211 message: (301) 889-0587 email: nina at alpha.ece.jhu.edu If you have further questions, please feel free to contact me. Thank you, Karen Haines IJCNN Volunteer Coordinator From Connectionists-Request at CS.CMU.EDU Mon Nov 13 11:29:54 1989 From: Connectionists-Request at CS.CMU.EDU (Connectionists-Request@CS.CMU.EDU) Date: Mon, 13 Nov 89 11:29:54 EST Subject: apparent FTP problems Message-ID: <846.626977794@B.GP.CS.CMU.EDU> When logging in to B.GP.CS.CMU.EDU via anonymous FTP, you may receive the following "error" message: Connected to B.GP.CS.CMU.EDU. 220 B.GP.CS.CMU.EDU FTP server (Version 4.105 of 18-Jun-89 19:22) ready. Name: anonymous 331 Guest login ok, send ident as password. Password: 230-[ AFS authentication failed: no password ] 230 Filenames can not have '/..' in them. ftp> Don't worry about it - you should still have access to the directories /usr/connect/connectionists/archives /usr/connect/connectionists/bibliographies If you still have trouble, send mail to "Connectionists-Request at cs.cmu.edu". David Plaut Connectionists-Request at cs.cmu.edu (ARPAnet) From eric at mcc.com Mon Nov 13 14:11:01 1989 From: eric at mcc.com (Eric Hartman) Date: Mon, 13 Nov 89 13:11:01 CST Subject: "Universal Approximators" Message-ID: <8911131911.AA04381@bird.aca.mcc.com> We agree with George Cybenko's response to John Kolnen and have some additional comments: The interest in sigmoidal and gaussian functions stems at least in part from their biological relevance; they are (much) more relevant than polynomials. Showing that neural networks serve as universal approximators is much like having an existence proof for a differential equation: you know the answer exists, but the theorem does not tell you how to find it. For that reason it is an important question in principle, but not necessarily in practice. Note that the answer could just as easily have been negative: there are several classes of functions that do not serve as universal approximators. (Take, e.g., functions from C_k trying to approximate functions from C_k+1.) If the answer was negative for neural networks, then we would have to think hard about why neural networks work so well. Jim Keeler and Eric Hartman From merrill at bucasb.BU.EDU Mon Nov 13 14:49:37 1989 From: merrill at bucasb.BU.EDU (John Merrill) Date: Mon, 13 Nov 89 14:49:37 EST Subject: "Universal Approximators" Message-ID: <8911131949.AA00642@bucasb.bu.edu> JK/EH> Jim Keeler and Eric Hartman JK/EH> The interest in sigmoidal and gaussian functions stems at JK/EH> least in part from their biological relevance; they are JK/EH> (much) more relevant than polynomials. That's honestly a debatable point. It may indeed be that sigmoids are more "biologically natural" than polynomials, but their use in a discrete-time system makes the difference hard to establish. The fact is that "real" neurons perform computations which are far more complicated than any kind of "take a dot product and squash" system; indeed, all the neurobiological evidence indicates that they do no such thing. JK/EH> Showing that neural networks serve as universal approximators JK/EH> is much like having an existence proof for a differential equation: JK/EH> you know the answer exists, but the theorem does not tell you JK/EH> how to find it. For that reason it is an important question JK/EH> in principle, but not necessarily in practice. By that standard, once any one universal approximator theorem had been established, no other would possess even the faintest semblance of interest. Since Borel approximation (in the sense of either l_2 or l_\infty) is easy to establish with sigmoidal networks alone, it seems to me that the results concerning (eg.) radial basis function would be hard to swallow. In fact, the radial basis function theorem gives somewhat better bounds on the *number* of intermediate nodes necessary, and, as a consequence, indicates that if you're only interested in approximation, you want to use RBF's. --- John From ST401843%BROWNVM.BITNET at VMA.CC.CMU.EDU Mon Nov 13 17:14:14 1989 From: ST401843%BROWNVM.BITNET at VMA.CC.CMU.EDU (thanasis kehagias) Date: Mon, 13 Nov 89 17:14:14 EST Subject: No subject Message-ID: ok, several people tried to ftp the recurrent nets bibliography and it did not work. i will send it out electronically. if somebody else tried and was able to ftp, like the manager's message implies, can you please let me know? thanasis From paul at NMSU.Edu Mon Nov 13 23:37:26 1989 From: paul at NMSU.Edu (paul@NMSU.Edu) Date: Mon, 13 Nov 89 21:37:26 MST Subject: No subject Message-ID: <8911140437.AA21121@NMSU.Edu> PRAGMATICS AI PRAGMATICS AI PRAGMATICS AI PRAGMATICS AI PRAGMATICS AI PRAGMATI PRAGMATICS AI PRAGMATICS AI PRAGMATICS AI PRAGMATICS AI PRAGMATICS AI PRAGMATI TO: Newsgroups: ar.ailist comp.ai.digest comp.ai.edu comp.ai.neural-nets comp.ai.nlang-know-rep comp.ai.shells comp.ai.vision news.announce.conferences news.announce.important sci.lang soc.celtic.culture Networks: AI List AI-NL NL-KR IRL-NET The Emigrant psy-net con-net nl-kr Local: NMSU CS NMSU CRL Journals: SIGART AISB Ack: SIGART RMSAI U S WEST CC: RMCAI-90 Program Committee RMCAI-90 Program Chairperson RMCAI-90 Local Arrangements Chairperson RMCAI-90 Organizing Committee RMCAI-90 Invited Speakers FROM: Organizing Committee RMCAI-90: Paul Mc Kevitt Yorick Wilks Research Scientist Director CRL CRL SUBJECT: Please post the following in your Laboratory/Department/Journal: Cut--------------------------------------------------------------------------- SUBJECT: Please post the following in your Laboratory/Department/Journal: CALL FOR PAPERS Pragmatics in Artificial Intelligence 5th Rocky Mountain Conference on Artificial Intelligence (RMCAI-90) Las Cruces, New Mexico, USA, June 28-30, 1990 PRAGMATICS PROBLEM: The problem of pragmatics in AI is one of developing theories, models, and implementations of systems that make effective use of contextual information to solve problems in changing environments. CONFERENCE GOAL: This conference will provide a forum for researchers from all subfields of AI to discuss the problem of pragmatics in AI. The implications that each area has for the others in tackling this problem are of particular interest. ACKNOWLEDGEMENTS: In cooperation with: Association for Computing Machinery (ACM) (pending approval) Special Interest Group in Artificial Intelligence (SIGART) (pending approval) U S WEST Advanced Technologies and the Rocky Mountain Society for Artificial Intelligence (RMSAI) With grants from: Association for Computing Machinery (ACM) Special Interest Group in Artificial Intelligence (SIGART) U S WEST Advanced Technologies and the Rocky Mountain Society for Artificial Intelligence (RMSAI) THE LAND OF ENCHANTMENT: Las Cruces, lies in THE LAND OF ENCHANTMENT (New Mexico), USA and is situated in the Rio Grande Corridor with the scenic Organ Mountains overlooking the city. The city is close to Mexico, Carlsbad Caverns, and White Sands National Monument. There are a number of Indian Reservations and Pueblos in the Land Of Enchantment and the cultural and scenic cities of Taos and Santa Fe lie to the north. New Mexico has an interesting mixture of Indian, Mexican and Spanish culture. There is quite a variation of Mexican and New Mexican food to be found here too. GENERAL INFORMATION: The Rocky Mountain Conference on Artificial Intelligence is a major regional forum in the USA for scientific exchange and presentation of AI research. The conference emphasizes discussion and informal interaction as well as presentations. The conference encourages the presentation of completed research, ongoing research, and preliminary investigations. Researchers from both within and outside the region are invited to participate. Some travel awards will be available for qualified applicants. FORMAT FOR PAPERS: Submitted papers should be double spaced and no more than 5 pages long. E-mail versions will not be accepted. Send 3 copies of your paper to: Paul Mc Kevitt, Program Chairperson, RMCAI-90, Computing Research Laboratory (CRL), Dept. 3CRL, Box 30001, New Mexico State University, Las Cruces, NM 88003-0001, USA. DEADLINES: Paper submission: March 1st, 1990 Pre-registration: April 1st, 1990 Notice of acceptance: May 1st, 1990 Final papers due: June 1st, 1990 LOCAL ARRANGEMENTS: Jennifer Griffiths, Local Arrangements Chairperson, RMCAI-90. (same postal address as above). INQUIRIES: Inquiries regarding conference brochure and registration form should be addressed to the Local Arrangements Chairperson. Inquiries regarding the conference program should be addressed to the program Chairperson. Local Arrangements Chairperson: E-mail: INTERNET: rmcai at nmsu.edu Phone: (+ 1 505)-646-5466 Fax: (+ 1 505)-646-6218. Program Chairperson: E-mail: INTERNET: paul at nmsu.edu Phone: (+ 1 505)-646-5109 Fax: (+ 1 505)-646-6218. TOPICS OF INTEREST: You are invited to submit a research paper addressing Pragmatics in AI , with any of the following orientations: Philosophy, Foundations and Methodology Knowledge Representation Neural Networks and Connectionism Genetic Algorithms, Emergent Computation, Nonlinear Systems Natural Language and Speech Understanding Problem Solving, Planning, Reasoning Machine Learning Vision and Robotics Applications INVITED SPEAKERS: The following researchers have agreed to speak at the conference (a number of others have been invited): Martin Casdagli, Los Alamos National Laboratory USA (Dynamical systems, Artificial neural nets, Applications) Arthur Cater, University College Dublin IRELAND (Robust Parsing) James Martin, University of Colorado at Boulder USA (Metaphor and Context) Derek Partridge, University of Exeter UK (Connectionism, Learning) Philip Stenton, Hewlett Packard UK (Natural Language Interfaces) PROGRAM COMMITTEE: John Barnden, New Mexico State University (Connectionism, Beliefs, Metaphor processing) Hans Brunner, U S WEST Advanced Technologies (Natural language interfaces, Dialogue interfaces) Martin Casdagli, Los Alamos National Laboratory (Dynamical systems, Artificial neural nets, Applications) Mike Coombs, New Mexico State University (Problem solving, Adaptive systems, Planning) Thomas Eskridge, Lockheed Missile and Space Co. (Analogy, Problem solving) Chris Fields, New Mexico State University (Neural networks, Nonlinear systems, Applications) Roger Hartley, New Mexico State University (Knowledge Representation, Planning, Problem Solving) Paul Mc Kevitt, New Mexico State University (Natural language interfaces, Dialogue modeling) Joe Pfeiffer, New Mexico State University (Computer Vision, Parallel architectures) Keith Phillips, University of Colorado at Colorado Springs (Computer vision, Mathematical modeling) Yorick Wilks, New Mexico State University (Natural language processing, Knowledge representation) Scott Wolff, U S WEST Advanced Technologies (Intelligent tutoring, User interface design, Cognitive modeling) REGISTRATION: Pre-Registration: Professionals $50.00; Students $30.00 (Pre-Registration cutoff date is April 1st 1990) Registration: Professionals $70.00; Students $50.00 (Copied proof of student status is required). Registration form (IN BLOCK CAPITALS). Enclose payment (personal checks and Eurochecks accepted). Send to the following address: Jennifer Griffiths, Local Arrangements Chairperson, RMCAI-90 Computing Research Laboratory Dept. 3CRL, Box 30001, NMSU Las Cruces, NM 88003-0001, USA. Name:_______________________________ E-mail_____________________________ Phone__________________________ Affiliation: ____________________________________________________ Fax: ____________________________________________________ Address: ____________________________________________________ ____________________________________________________ ____________________________________________________ COUNTRY__________________________________________ Organizing Committee RMCAI-90: Paul Mc Kevitt Yorick Wilks Research Scientist Director CRL CRL cut------------------------------------------------------------------------ From yann at lesun.att.com Tue Nov 14 10:10:15 1989 From: yann at lesun.att.com (Yann le Cun) Date: Tue, 14 Nov 89 10:10:15 -0500 Subject: "Universal Approximators" In-Reply-To: Your message of Mon, 13 Nov 89 14:49:37 -0500. Message-ID: <8911141510.AA08431@lesun.> John Merrill says: > In fact, the radial basis function theorem gives somewhat better > bounds on the *number* of intermediate nodes necessary, and, as a > consequence, indicates that if you're only interested in > approximation, you want to use RBF's. But you can build RBF's (or rather, multidimensional "bumps") with two layers of sigmoid units. And you just need O(n) units for n bumps. A third (linear) layer can sum up the bumps. So i guess the bound on the number of intermediate nodes is the same in both cases. You can prove the universality of 2 hidden layer structures that way (and the proof is a lot simpler than with 1 hidden layer). Yann Le Cun From merrill at bucasb.BU.EDU Tue Nov 14 12:30:26 1989 From: merrill at bucasb.BU.EDU (John Merrill) Date: Tue, 14 Nov 89 12:30:26 EST Subject: "Universal Approximators" In-Reply-To: Yann le Cun's message of Tue, 14 Nov 89 10:10:15 -0500 <8911141510.AA08431@lesun.> Message-ID: <8911141730.AA11580@bucasb.bu.edu> YlC> Yann le Cun JM> John Merrill (me) JM> In fact, the radial basis function theorem gives somewhat better JM> bounds on the *number* of intermediate nodes necessary, and, as a JM> consequence, indicates that if you're only interested in JM> approximation, you want to use RBF's. YlC> But you can build RBF's (or rather, multidimensional "bumps") YlC> with two layers of sigmoid units. And you just need O(n) units YlC> for n bumps. A third (linear) layer can sum up the bumps. YlC> So i guess the bound on the number of intermediate nodes is the YlC> same in both cases. Not quite. If your input lies in ${\bf R}^k$, it takes at least k units (in the lower hidden level) to build a single k-dimensional bump in the upper hidden level.(*) As a consequence, the network that this argument gives is wildly more computationally demanding than the original RBF network, since it's got to have $nk^2$ edges between the input layer and the first hidden layer, as well as $nk$ more edges between the two hidden layers, for a total edge count of $n (k^2 + k)$, as compared to an edge count of $nk$ for the RBF network. If $k = 1000$, this is a factor of $1000$ more costly (in terms of edges; it's actually less than that, since each RBF node needs an extra few multiplies.) YlC> You can prove the universality of 2 hidden layer structures that YlC> way (and the proof is a lot simpler than with 1 hidden layer). Absolutely, and it's the one I would actually teach if I taught one at all. --- John (*) It might seem that one would need 2k such units; in fact, by using k-dimensional simplices as the bump-shapes, k units suffice. From sontag at fermat.rutgers.edu Tue Nov 14 16:29:52 1989 From: sontag at fermat.rutgers.edu (Eduardo Sontag) Date: Tue, 14 Nov 89 16:29:52 EST Subject: number of hidden neurons Message-ID: <8911142129.AA02052@fermat.rutgers.edu> In the context of the discussions about numbers of hidden neurons, it seems relevant to point out a short note of mine to appear in Neural Computation, Number 4, entitled: SIGMOIDS DISTINGUISH MORE EFFICIENTLY THAN HEAVISIDES I prove (this is the abstract): Every dichotomy on a $2k$-point set in $\R^N$ can be implemented by a neural net with a single hidden layer containing $k$ sigmoidal neurons. If the neurons were of a hardlimiter (Heaviside) type, $2k-1$ would be in general needed. So one can save half as many neurons using sigmoids, a fact which might not be totally obvious at first sight (but which is indeed easy to prove). The first paragraph of the intro is: The main point of this note is to draw attention to the fact mentioned in the title, that sigmoids have different recognition capabilities than hard-limitting nonlinearities. One way to exhibit this difference is through a worst-case analysis in the context of binary classification, and this is done here. Results can also be obtained in terms of VC dimension, and work is in progress in that regard. (Added note: now established that in 2-d, VC dimension of k-neuron nets is 4k. This is not written up yet, though. I also have an example of a family of boolean functions that can be computed with a fixed number of sigmoidal hidden units but which --I conjecture-- needs a growing number of Heavisides.) -eduardo Eduardo D. Sontag (Phone: (201)932-3072; dept.: (201)932-2390) sontag at fermat.rutgers.edu ...!rutgers!fermat.rutgers.edu!sontag sontag at pisces.bitnet From yann at lesun.att.com Tue Nov 14 17:15:25 1989 From: yann at lesun.att.com (Yann le Cun) Date: Tue, 14 Nov 89 17:15:25 -0500 Subject: "Universal Approximators" In-Reply-To: Your message of Tue, 14 Nov 89 12:30:26 -0500. Message-ID: <8911142215.AA08568@lesun.> John Merrill says: >If your input lies in ${\bf R}^k$, it takes at least k units >(in the lower hidden level) to build a single k-dimensional bump in >the upper hidden level True, although, as you say, it is easier with 2k units. >As a consequence, the network that this >argument gives is wildly more computationally demanding than the >original RBF network, since it's got to have $nk^2$ edges between the >input layer and the first hidden layer Not true, since each of these 2k units only needs 2 incoming weights (not k) one for the bias, and one coming from one of the inputs (*). thus the total number of edges is 6nk, just 6 times bigger than regular RBF's. It can even be better than that (almost 2nk) if your bumps are regularly spaced since they can share the first level units. And you can back-propagate through the whole thing. -- Yann Le Cun (*) you might want k incoming weights if you absolutely need to have non symetric and rotated RBF's, but otherwise 2 is enough From jagota at cs.Buffalo.EDU Wed Nov 15 13:25:22 1989 From: jagota at cs.Buffalo.EDU (Arun Jagota) Date: Wed, 15 Nov 89 13:25:22 EST Subject: Universal Approximator Results Message-ID: <8911151825.AA23769@sybil.cs.Buffalo.EDU> A question was raised some time back that since universal approximator results establish no new bounds on computability (one result is as good as another in a computability sense), what then is their significance. Don't such results for k-hidden layer networks show, additionally, that the represented function can be _evaluated_ on a point in it's domain in (k+1) inner-product + k hidden-layer function eval steps on a suitable abstract machine. Doesn't that provide a strong result on the running time, as compared with Church's thesis which says, that any algorithm (effective procedure) can be programmed on a Turing m/c but doesn't put a bound on the running time. Arun Jagota jagota at cs.buffalo.edu [I don't wish to be accused of starting a fresh round of value-less discussions (if so perceived), so I prefer receiving responses by mail. I suggest using 'mail' instead of 'R' or 'r'] From tmb at ai.mit.edu Wed Nov 15 14:02:46 1989 From: tmb at ai.mit.edu (Thomas M. Breuel) Date: Wed, 15 Nov 89 14:02:46 EST Subject: Universal Approximator Results In-Reply-To: <8911151825.AA23769@sybil.cs.Buffalo.EDU> Message-ID: <8911151902.AA09129@rice-chex> Arun Jagota writes: > A question was raised some time back that since universal approximator > results establish no new bounds on computability (one result is as good > as another in a computability sense), what then is their significance. > Don't such results for k-hidden layer networks show, additionally, > that the represented function can be _evaluated_ on a point in it's > domain in > > (k+1) inner-product + k hidden-layer function eval > steps on a suitable abstract machine. > > Doesn't that provide a strong result on the running time, > as compared with Church's thesis which says, that any algorithm > (effective procedure) can be programmed on a Turing m/c but doesn't > put a bound on the running time. The claim is still true: "computability" says nothing about time complexity, space complexity, or parallel complexity. Therefore, from a computability point of view, it makes no difference whatsoever whether you prove that something is computable via networks or via a Turing machine, even if the Turing machine takes much longer. There are two other issues here: the "universal approximator" results talk about approximating a real function, not computing a real function, and, also, all the arithmetic here is real arithmetic, so a comparison between complexity on a Turing machine and the arithmetic complexity on the network is non-trivial. Thomas. From tmb at ai.mit.edu Wed Nov 15 14:06:56 1989 From: tmb at ai.mit.edu (Thomas M. Breuel) Date: Wed, 15 Nov 89 14:06:56 EST Subject: Universal Approximator Results In-Reply-To: <8911151825.AA23769@sybil.cs.Buffalo.EDU> Message-ID: <8911151906.AA09140@rice-chex> Arun Jagota writes: > [...] Oops--my reply should not have gone to the whole list. Apologies. (I would have worded it a little more carefully if I had intended to send it to the whole list). From kawahara at av-convex.ntt.jp Thu Nov 16 14:52:41 1989 From: kawahara at av-convex.ntt.jp (Hideki KAWAHARA) Date: Fri, 17 Nov 89 04:52:41+0900 Subject: News on JNNS(Japanese Neural Network Society) Message-ID: <8911161952.AA02921@av-convex.ntt.jp> -------------------------- JNNS (Japanese Neural Network Society) have delivered its first newsletter and started a mailing list -------------------------- Japanese Neural Network society, which was founded in July 1989, have delivered its first newsletter on 14 November 1989. Prof. Shiro Usui of Toyohasi University of Technology, who is in charge of editor in chief have also started a mailing list to encouredge discussions among active researchers in Japan. Prof. Usui and I would like to introduce the connectionists mailing list to JNNS's mailing list and to quit delivering to BBORD eventually. Electronic communications in Japan is still in its infancy. JUNET, the largest one, is a volunteer based (mainly UUCP) network. However, the number of researchers who are accessible to some electronic communication systems is increasing rapidly. I look forward to see some Japanese researchers to contribute this global electronic research community. Hideki Kawahara NTT Basic Research Labs. JAPAN. PS: JNNS President is Prof.Kunihiko Fukushima JNNS V.P. is Prof.Shun'ichi Amari If you need more detailes, please e-mail to: kawahara%siva.ntt.jp at RELAY.CS.NET . From munnari!extro.ucc.su.oz.au!root at uunet.UU.NET Fri Nov 17 14:00:27 1989 From: munnari!extro.ucc.su.oz.au!root at uunet.UU.NET (Admin) Date: Fri Nov 17 14:00:27 1989 Subject: This mail got misdirected ... Message-ID: <8911170303.914@munnari.oz.au> From Fri Nov 17 12:15 EST 1989 Date: Fri Nov 17 12:18:29 1989 From: mailer-daemon keeps mailer happy <> Apparently-To: MAILER-DAEMON >From ml-connectionists-request at Q.CS.CMU.EDU@murtoa.cs.mu.oz Fri Nov 17 09:24:23 1989 From: Hideki KAWAHARA Date: Fri, 17 Nov 89 04:52:41+0900 To: connectionists at CS.CMU.EDU Subject: News on JNNS(Japanese Neural Network Society) -------------------------- JNNS (Japanese Neural Network Society) have delivered its first newsletter and started a mailing list -------------------------- Japanese Neural Network society, which was founded in July 1989, have delivered its first newsletter on 14 November 1989. Prof. Shiro Usui of Toyohasi University of Technology, who is in charge of editor in chief have also started a mailing list to encouredge discussions among active researchers in Japan. Prof. Usui and I would like to introduce the connectionists mailing list to JNNS's mailing list and to quit delivering to BBORD eventually. Electronic communications in Japan is still in its infancy. JUNET, the largest one, is a volunteer based (mainly UUCP) network. However, the number of researchers who are accessible to some electronic communication systems is increasing rapidly. I look forward to see some Japanese researchers to contribute this global electronic research community. Hideki Kawahara NTT Basic Research Labs. JAPAN. PS: JNNS President is Prof.Kunihiko Fukushima JNNS V.P. is Prof.Shun'ichi Amari If you need more detailes, please e-mail to: kawahara%siva.ntt.jp at RELAY.CS.NET . From munnari!extro.ucc.su.oz.au!root at uunet.UU.NET Fri Nov 17 14:00:13 1989 From: munnari!extro.ucc.su.oz.au!root at uunet.UU.NET (Admin) Date: Fri Nov 17 14:00:13 1989 Subject: This mail got misdirected ... Message-ID: <8911170303.900@munnari.oz.au> From Fri Nov 17 12:14 EST 1989 Date: Fri Nov 17 12:18:15 1989 From: mailer-daemon keeps mailer happy <> Apparently-To: MAILER-DAEMON >From ml-connectionists-request at Q.CS.CMU.EDU@murtoa.cs.mu.oz Fri Nov 17 09:24:23 1989 From: Hideki KAWAHARA Date: Fri, 17 Nov 89 04:52:41+0900 To: connectionists at CS.CMU.EDU Subject: News on JNNS(Japanese Neural Network Society) -------------------------- JNNS (Japanese Neural Network Society) have delivered its first newsletter and started a mailing list -------------------------- Japanese Neural Network society, which was founded in July 1989, have delivered its first newsletter on 14 November 1989. Prof. Shiro Usui of Toyohasi University of Technology, who is in charge of editor in chief have also started a mailing list to encouredge discussions among active researchers in Japan. Prof. Usui and I would like to introduce the connectionists mailing list to JNNS's mailing list and to quit delivering to BBORD eventually. Electronic communications in Japan is still in its infancy. JUNET, the largest one, is a volunteer based (mainly UUCP) network. However, the number of researchers who are accessible to some electronic communication systems is increasing rapidly. I look forward to see some Japanese researchers to contribute this global electronic research community. Hideki Kawahara NTT Basic Research Labs. JAPAN. PS: JNNS President is Prof.Kunihiko Fukushima JNNS V.P. is Prof.Shun'ichi Amari If you need more detailes, please e-mail to: kawahara%siva.ntt.jp at RELAY.CS.NET . From nrandall%watdcs.UWaterloo.ca at VMA.CC.CMU.EDU Fri Nov 17 09:50:16 1989 From: nrandall%watdcs.UWaterloo.ca at VMA.CC.CMU.EDU (Neil Randall (ENGLISH)) Date: Fri, 17 Nov 89 09:50:16 EST Subject: E-Mail Distribution List Message-ID: If possible, please add me to the e-mail distribution list. I am one of the Rhetoric group in the Department of English at Waterloo. Thank you. Neil Randall From mohandes at ed.ecn.purdue.edu Fri Nov 17 10:15:04 1989 From: mohandes at ed.ecn.purdue.edu (Mohamed Mohandes) Date: Fri, 17 Nov 89 10:15:04 -0500 Subject: News on JNNS(Japanese Neural Network Society) Message-ID: <8911171515.AA03432@ed.ecn.purdue.edu> d From smk at flash.bellcore.com Fri Nov 17 11:38:14 1989 From: smk at flash.bellcore.com (Selma M Kaufman) Date: Fri, 17 Nov 89 11:38:14 EST Subject: No subject Message-ID: <8911171638.AA20706@flash.bellcore.com> Subject: Reprints Available Identifying and Discriminating Temporal Events with Connectionist Language Users Presented at: IEE Conference on Artificial Neural Networks (London, October, 1989) 284-286. Robert B. Allen and Selma M. Kaufman The "connectionist language user" paradigm is applied to several studies of the perception, processing, and description of events. In one study, a network was trained to discriminate the order with which objects appeared in a microworld. In a second study, networks were trained to recognize and describe sequences of events in the microworld using 'verbs'. In a third study 'plan recognition' was modeled. In the final study, networks answered questions that used verbs of possession. These results further strengthen the generality of the approach as a unified model of perception, action, and language. Back-Propagation as Computational Model of Gestalt Cognition: Evidence for a Halo Effect Presented at: Second International Symposium on Artificial Intelligence (Monterrey, Mexico, October 23-27, 1989). Robert B. Allen Connectionist networks show a distinctly different type of processing than ruled-based approaches. It is proposed that connectionist networks resemble the gestalt models of cognition that were popular in the 1950s. Moreover, a context effect known as the "halo effect", which is a hallmark of gestalt models of cognition, was modeled. The effect was confirmed when networks were required to generate valences assigned to objects which were presented in the context of other objects. For paper copies, contact: Selma Kaufman, 2M-356, Bellcore, 445 South St., Morristown, NJ 07960-1910. smk at flash.bellcore.com From pa1490%sdcc13 at ucsd.edu Fri Nov 17 14:08:47 1989 From: pa1490%sdcc13 at ucsd.edu (Dave Scotese) Date: Fri, 17 Nov 89 11:08:47 PST Subject: E-Mail Distribution List Message-ID: <8911171908.AA29998@sdcc13.UCSD.EDU> From THEPCAP%SELDC52.BITNET at VMA.CC.CMU.EDU Fri Nov 17 13:32:00 1989 From: THEPCAP%SELDC52.BITNET at VMA.CC.CMU.EDU (THEPCAP%SELDC52.BITNET@VMA.CC.CMU.EDU) Date: Fri, 17 Nov 89 19:32 +0100 Subject: TR available Message-ID: October 1989 LU TP 89-19 "TEACHERS AND CLASSES" WITH NEURAL NETWORKS Lars Gislen, Carsten Peterson and Bo Soderberg Department of Theoretical Physics, University of Lund Solvegatan 14A, S-22362 Lund, Sweden Submitted to International Journal of Neural Systems Abstract: A convenient mapping and an efficient algorithm for solving scheduling problems within the neural network paradigm is presented. It is based on a reduced encoding scheme and a mean field annealing prescription, which was recently successfully applied to TSP. Most scheduling problems are characterized by a set of hard and soft constraints. The prime target of this work is the hard constraints. In this domain the algorithm persistently finds legal solutions for quite difficult problems. We also make some exploratory investigations by adding soft constraints with very encouraging results. Our numerical studies cover problem sizes up to O(5*10^4) degrees of freedom with no parameter sensitivity. We stress the importance of adding certain extra terms to the energy functions which are redundant from the encoding point of view but beneficial when it comes to ignoring local minima and to stabilizing the good solutions in the annealing process. --------------------------------------- For copies of this report send requests to: THEPCAP at SELDC52. NOTICE: Those of you who requested our previous report, "A New Way of Mapping Optimization.... (LU TP 89-1), will automatically receive this one so no request is necessary. From harnad at clarity.Princeton.EDU Mon Nov 20 11:09:13 1989 From: harnad at clarity.Princeton.EDU (Stevan Harnad) Date: Mon, 20 Nov 89 11:09:13 EST Subject: What is a Symbol System? Message-ID: <8911201609.AA01142@psycho.Princeton.EDU> There has been some difference of opinion as to whether a connectionist network is or is not, or can or cannot be, a symbol system. To answer such questions, one must first settle on what a symbol system is. Here's my candidate: What is a symbol system? From Newell (1980) Pylyshyn (1984), Fodor (1987) and the classical work of Von Neumann, Turing, Goedel, Church, etc.(see Kleene 1969) on the foundations of computation, we can reconstruct the following definition: A symbol system is: (1) a set of arbitrary PHYSICAL TOKENS (scratches on paper, holes on a tape, events in a digital computer, etc.) that are (2) manipulated on the basis of EXPLICIT RULES that are (3) likewise physical tokens and STRINGS of tokens. The rule-governed symbol-token manipulation is based (4) purely on the SHAPE of the symbol tokens (not their "meaning"), i.e., it is purely SYNTACTIC, and consists of (5) RULEFULLY COMBINING and recombining symbol tokens. There are (6) primitive ATOMIC symbol tokens and (7) COMPOSITE symbol-token strings. The entire system and all its parts -- the atomic tokens, the composite tokens, the syntactic manipulations (both actual and possible) and the rules -- are all (8) SEMANTICALLY INTERPRETABLE: The syntax can be SYSTEMATICALLY assigned a meaning (e.g., as standing for objects, as describing states of affairs). According to proponents of the symbolic model of mind such as Fodor (1980) and Pylyshyn (1980, 1984), symbol-strings of this sort capture what mental phenomena such as thoughts and beliefs are. Symbolists emphasize that the symbolic level (for them, the mental level) is a natural functional level of its own, with ruleful regularities that are independent of their specific physical realizations. For symbolists, this implementation-independence is the critical difference between cognitive phenomena and ordinary physical phenomena and their respective explanations. This concept of an autonomous symbolic level also conforms to general foundational principles in the theory of computation and applies to all the work being done in symbolic AI, the branch of science that has so far been the most successful in generating (hence explaining) intelligent behavior. All eight of the properties listed above seem to be critical to this definition of symbolic. Many phenomena have some of the properties, but that does not entail that they are symbolic in this explicit, technical sense. It is not enough, for example, for a phenomenon to be INTERPRETABLE as rule-governed, for just about anything can be interpreted as rule-governed. A thermostat may be interpreted as following the rule: Turn on the furnace if the temperature goes below 70 degrees and turn it off if it goes above 70 degrees, yet nowhere in the thermostat is that rule explicitly represented. Wittgenstein (1953) emphasized the difference between EXPLICIT and IMPLICIT rules: It is not the same thing to "follow" a rule (explicitly) and merely to behave "in accordance with" a rule (implicitly). The critical difference is in the compositeness (7) and systematicity (8) criteria. The explicitly represented symbolic rule is part of a formal system, it is decomposable (unless primitive), its application and manipulation is purely formal (syntactic, shape-dependent), and the entire system must be semantically interpretable, not just the chunk in question. An isolated ("modular") chunk cannot be symbolic; being symbolic is a combinatory, systematic property. So the mere fact that a behavior is "interpretable" as ruleful does not mean that it is really governed by a symbolic rule. Semantic interpretability must be coupled with explicit representation (2), syntactic manipulability (4), and systematicity (8) in order to be symbolic. None of these criteria is arbitrary, and, as far as I can tell, if you weaken them, you lose the grip on what looks like a natural category and you sever the links with the formal theory of computation, leaving a sense of "symbolic" that is merely unexplicated metaphor (and probably differs from speaker to speaker). Any rival definitions, counterexamples, amplifications? Excerpted from: Harnad, S. (1990) The Symbol Grounding Problem. Physica D (in press) ----------------------------------------------------- References: Fodor, J. A. (1975) The language of thought. New York: Thomas Y. Crowell Fodor, J. A. (1987) Psychosemantics. Cambridge MA: MIT/Bradford. Fodor, J. A. & Pylyshyn, Z. W. (1988) Connectionism and cognitive architecture: A critical appraisal. Cognition 28: 3 - 71. Harnad, S. (1989) Minds, Machines and Searle. Journal of Theoretical and Experimental Artificial Intelligence 1: 5-25. Kleene, S. C. (1969) Formalized recursive functionals and formalized realizability. Providence, R.I.: American Mathematical Society. Newell, A. (1980) Physical Symbol Systems. Cognitive Science 4: 135-83. Pylyshyn, Z. W. (1980) Computation and cognition: Issues in the foundations of cognitive science. Behavioral and Brain Sciences 3: 111-169. Pylyshyn, Z. W. (1984) Computation and cognition. Cambridge MA: MIT/Bradford Turing, A. M. (1964) Computing machinery and intelligence. In: Minds and machines, A.R. Anderson (ed.), Engelwood Cliffs NJ: Prentice Hall. From skrzypek at CS.UCLA.EDU Mon Nov 20 16:53:15 1989 From: skrzypek at CS.UCLA.EDU (Dr. Josef Skrzypek) Date: Mon, 20 Nov 89 13:53:15 PST Subject: UCLA SFINX - "neural" net simulator Message-ID: <8911202153.AA04758@retina.cs.ucla.edu> The UCLA-SFINX, a "neural" network simulator is now in pub- lic domain. UCLA-SFINX (Structure and Function In Neural connec- tions) is an interactive neural network simulation environment designed to provide the investigative tools for studying the behavior of various neural structures. It was designed to easily express and simulate the highly regular patterns often found in large networks, but it is also general enough to model parallel systems of arbitrary interconnectivity. UCLA-SFINX is not based on any single neural network para- digm such as Backward Error Propagation (BEP) but rather enables users to simulate a wide variety of neural network models. UCLA- SFINX has been used to simulate neural networks for the segmenta- tion of images using textural cues, architectures for color and lightness constancy, script character recognition using BEP and others. It is all written in C, includes an X11 interface, and it has been ported to HP 9000 320/350 workstations running HP-UX, Sun workstations running SUNOS 3.5, IBM RT workstations running BSD 4.3, Ardent Titan workstations running Ardent UNIX Release 2.0, and VAX 8200's running Ultrix 2.2-1. To get UCLA-SFINX source code and documentation (in LaTeX format) follow the in- structions below: 1. To obtain UCLA-SFINX via the Internet: Sign and return the enclosed UCLA-SFINX License Agreement to the address below. We will send you a copy of the signed license agreement along with instructions on how to FTP a copy of UCLA-SFINX. If you have a PostScript printer, you should be able to produce your own copy of the manual. If you wish to obtain a hardcopy of the manual, return a check for $30 along with the license. 2. To obtain UCLA-SFINX on tape: Sign and return the enclosed UCLA-SFINX License Agreement to the address below. Return a check for $100 dollars along with the license, for a hardcopy of the manual and a copy of UCLA-SFINX on 1/4 inch cartridge tape (in tar format) read- able by a Sun 3 workstation. We will also send you a copy of the signed license agreement. Checks should be made payable to the Regents of the Univer- sity of California. If you have questions regarding any of the information discussed above send electronic mail to sfinx at retina.cs.ucla.edu or US mail to: UCLA Machine Perception Laboratory, Computer Science Department, 3532 Boelter Hall, Los Angeles, CA. 90024, USA. >>>>>>>>>>>>>>>>>>>>>>>>>>>>> cut here for license <<<<<<<<<<<<<<<<<<<<<<<<< THE REGENTS OF THE UNIVERSITY OF CALIFORNIA LOS ANGELES CAMPUS UCLA-SFINX LICENSE AGREEMENT This Agreement is entered into this___________of ____________________, 198__, by and between THE REGENTS OF THE UNIVERSITY OF CALIFORNIA, a California corporation, hereinafter called "University", and ________________________________________ _____________________________________, hereinafter called "Licensee." This Agreement is made with reference to the follow- ing: 1. DEFINITION "UCLA-SFINX" is a set of copyrighted, source code computer programs and any future modifications thereof delivered by University to Licensee, and any accompanying documentation provided by University. UCLA-SFINX is a general purpose software system for the development and evaluation of con- nectionist models. UCLA-SFINX is written for and operates on UNIX systems. 2. GRANT OF RIGHTS A. University grants to Licensee and Licensee accepts a non-exclusive, non-transferable license to use UCLA- SFINX solely for Licensee's non-commercial purposes. B. Such use may include the making of sufficient copies of UCLA-SFINX for the reasonable purposes of Licensee hereunder. All copies of UCLA-SFINX made by Licensee, in whole or in part, regardless of the form in which the Licensee may subsequently use it, and regardless of any modification which the Licensee may subsequently make to it are the property of University and no title to or ownership of such materials are transferred to Licensee hereunder. Licensee shall include on any such copies labels containing the name UCLA-SFINX, the University's copyright notice, and any other proprietary or restrictive notices appearing on the la- bel of the copy of UCLA-SFINX furnished to Licensee by University. 1 C. Such use may include the modification of UCLA-SFINX by Licensee. Such modified versions of UCLA-SFINX shall remain the property of University. D. Such use shall not include further distribution, or any action which may be construed as selling or licensing UCLA-SFINX to any person or entity. 3. ACKNOWLEDGMENT A. Licensee acknowledges that UCLA-SFINX has been developed for research purposes only. B. Licensee shall require its employees and students to acknowledge in writing their use of UCLA-SFINX when re- porting any research resulting from such use. The fol- lowing notice should be used: "UCLA-SFINX from UCLA MACHINE PERCEPTION LABORATORY." 4. WARRANTIES AND INDEMNIFICATION A. University warrants that is is the owner of all right, title, and interest in and to UCLA-SFINX, including all copyright pertaining thereto and subsisting therein. B. UCLA-SFINX is licensed "AS IS," and University dis- claims all warranties, express and implied, including but not limited to, the implied warranties of merchan- tability and fitness for a particular purpose. In no event will University be liable for any business ex- pense, machine down time, loss of profits, any inciden- tal, special, exemplary or consequential damages, or any claims or demands brought against Licensee. The entire risk as to the quality and performance of UCLA- SFINX is with Licensee. C. Licensee agrees to indemnify, defend, hold harmless, and defend University, its officers, employees and agents, against any and all claims, suits, losses, dam- ages, costs, fees, and expenses resulting from or aris- ing out of any use of UCLA-SFINX by Licensee. 5. TECHNICAL SUPPORT AND FEEDBACK A. University shall have no obligation to install, sup- port, maintain, or correct any defects in UCLA-SFINX. 2 B. Licensee agrees to notify University of any errors, functional problems, and any defects in performance discovered in UCLA-SFINX and of any fixes made by Licensee. Such notice will contain a full description of the problem, indicating in what circumstances it originated, and how it manifested itself. Technical matters and errors discovered in UCLA-SFINX may be com- municated as provided in Article 9 below or via elec- tronic mail to: sfinx at retina.cs.ucla.edu. 6. TERM AND TERMINATION A. The term of this Agreement is perpetual and shall be effective from the date of its signing by duly author- ized official of Licensee. B. Any failure of Licensee to comply with all terms and conditions of this Agreement shall result in its im- mediate termination. 7. SEVERABILITY If any of the provisions or portions of this Agreement are invalid under any applicable statute or rule of law, they are to the extent of such invalidity severable and shall not affect any other provision of this Agreement. 8. APPLICABLE LAW This Agreement shall be governed by the laws of the State of California. 9. NOTICE A. Any notice under this Agreement shall be in writing and mailed to the appropriate address given below: To University regarding this Agreement: The Regents of the University of California Office of Contract and Grant Administration University of California, Los Angeles 405 Hilgard Avenue Los Angeles, California 90024-1406 Attention: Dr. Enrique Riveros-Schafer 3 B. To University regarding technical matters: UCLA Machine Perception Laboratory 3532 Boelter Hall Computer Science Department Los Angeles, California 90024 Attention: Prof. Josef Skrzypek C. To Licensee: ____________________________________________________ ____________________________________________________ ____________________________________________________ ____________________________________________________ ____________________________________________________ 10. ENTIRETY This Agreement supersedes any previous communication and, when signed by both parties, constitutes the complete under- standing of the parties. No modification or waiver of any provisions hereof shall be valid unless in writing and signed by both parties. IN WITNESS THEREOF, the parties here to have caused this Agree- ment to be executed. LICENSEE THE REGENTS OF THE UNIVERSITY OF CALIFORNIA From Dave.Touretzky at B.GP.CS.CMU.EDU Tue Nov 21 03:46:47 1989 From: Dave.Touretzky at B.GP.CS.CMU.EDU (Dave.Touretzky@B.GP.CS.CMU.EDU) Date: Tue, 21 Nov 89 03:46:47 EST Subject: What is a Symbol System? In-Reply-To: Your message of Mon, 20 Nov 89 11:09:13 -0500. <8911201609.AA01142@psycho.Princeton.EDU> Message-ID: <9939.627641207@DST.BOLTZ.CS.CMU.EDU> Okay, I'll take a shot at responding to Stevan's query. Of the eight criteria he listed, I take exception to numbers 2 and 3, that rules must be EXPLICIT and expressed as STRINGS of tokens. In my recent work on phonology with Deirdre Wheeler (see "A connectionist implementation of cognitive phonology", tech report number CMU-CS-89-144), we define an architecture for manipulating sequences of phonemes. This architecture supports a small number of primitive operations like insertion, mutation, and deletion of phonemes. I claim that the rules for deciding WHEN to apply these primitives do not need to be represented explicitly or have a symbolic representation, in order for us to have a symbol system. It suffices that the rules' actions be combinations of the primitive actions our architecture provides. This is what distinguishes our phonological model from the Rumelhart and McClelland verb learning model. In their model, rules have no explicit representation, but in addition, rules operate directly on the phoneme sequence in a totally unconstrained way, mapping activation patterns to activation patterns; there are no primitive symbolic operations. Therefore their model is non-symbolic, as they themselves point out. ================ I also think the definition of symbol system Stevan describes is likely to prove so constrained that it rules out human cognition. This sort of symbol system appears to operate only by disassembling and recombining discrete structures according to explicit axioms. What about more implicit, continuous kinds of computation, like using spreading activation to access semantically related concepts in a net? How far the activation spreads depends on a number of things, like the branching factor of the semantic net, the weights on the links, and the amount of cognitive resources (what Charniak calls "zorch") available at the moment. (People reason differently when trying to do other things at the same time, as opposed to when they're relaxed and able to concentrate on a single task.) Of course, spreading activation can be SIMULATED on a symbol processing system, such as a Turing machine or digital computer, but this raises the very important issue of levels of representation. What the Physical Symbol System Hypothesis requires is that the primitive atomic symbol tokens have meaning IN THE DOMAIN of discourse we're modeling. Although a Turing machine can be made to simulate continuous computations at any level of precision desired, it can only do so by using its primitive atomic symbols in ways that have nothing to do with the semantic net it's trying to simulate. Instead its symbols are used to represent things like the individual bits in some floating point number. To play the Physical Symbol Systems game correctly in the semantic net case, you have to choose primitives corresponding to nodes and links. But in that case there doesn't seem to be room for continuous, non-compositional sorts of computations. Another problem I see with this definition of symbol system is that it doesn't say what it means in #5 to "rulefully combine" symbols. What about stochastic systems, like Boltzmann machines? They don't follow deterministic rules, but they do obey statistical ones. What about a multilayer perceptron, which could be described as one giant rule for mapping input patterns to output patterns? -- Dave From well!mitsu at apple.com Mon Nov 20 22:31:20 1989 From: well!mitsu at apple.com (Mitsuharu Hadeishi) Date: Mon, 20 Nov 89 19:31:20 pst Subject: What is a Symbol System? Message-ID: <8911210331.AA07032@well.sf.ca.us> This is an interesting question. First of all, I think it is clear that since a recurrent neural network can emulate any finite-state automaton that they are Turing equivalent goes almost without saying, so it is also clear that recurrent NNs should be capable of the symbolic-level processing of which you speak. First of all, however, I'd like to address the symbolist point of view that higher-level cognition is purely symbolic, irrespective of the implementation scheme. I submit this is patently absurd. Symbolic representations of thought are simply models of how we think, and quite crude models at that. They happen to have several redeeming qualities however, among them that they are simple, well-defined, and easy to manipulate. However, in truth, though it is clear that many operations (such as syntactic analysis of language) operate within the structure, at least in part, of symbolic processing, others go outside (such as understanding a subtle poem). In addition, there are many other forms of higher-level cognition, such as that which visual artists engage themselves in, which do not easily lend themselves to symbolic decomposition. I submit that even everyday actions and thoughts do not follow any strict symbolic decomposition, though to some degree of approximation they can be modelled *as though* they were following rules of some kind. I think the comparison between rule-based and analog systems is apt; however, in my opinion it is the analog systems which have the greater flexibility, or one might say economy of expression. That is to say, inasmuch as one can emulate one with the other they are equivalent, but given limitations on complexity and size I think it is clear the complex analog dynamical systems have the edge. The fact is that as a model for the world or how we think rule-based representations are sorely lacking. It is similar to trying to paint a landscape using polygons; one can do it, but it is not particularly well-suited for the task, except in very simple situations (or situations where the landscape happens to be man-made.) We should not confuse the map with the territory. Just because we happen to have this crude model for thinking, i.e., the symbolic model, does not mean that is *how* we think. We may even describe our decisions this way, but the intractability of AI problems except for very limited-domain applications indicates or suggests the weaknesses with our model. For example, natural language systems only work with extremely limited context. The fact that they do work at all is evidence that our symbolic models are not completely inadequate, however, that they are limited in domain suggests they are nonetheless mere approximations. Connectionist models, I believe, have much greater chance at capturing the true complexity of cognitive systems. In addition, the recent introduction of fuzzy reasoning and nonmonotonic logic are extensions of the symbolic model which certainly improve the situation, but also point out the main weaknesses with symbolic models of cognition. Symbolic models address only one aspect of the thinking process, perhaps not even the most important part. For example, a master chess player typically only considers about a hundred possible moves, yet can beat a computer program that considers tens of thousands of moves. The intractability of even more difficult problems than chess also points this out. Before the symbolic engine can be put into action, a great deal of pre-processing goes on which will likely not be best described in symbolic terms. Mitsu Hadeishi Open Mind 16110 S. Western Avenue Gardena, CA 90247 (213) 532-1654 (213) 327-4994 mitsu at well.sf.ca.us From netlist at psych.Stanford.EDU Tue Nov 21 10:43:41 1989 From: netlist at psych.Stanford.EDU (Mark Gluck) Date: Tue, 21 Nov 89 07:43:41 PST Subject: Stanford Adaptive Network Colloq: RICHARD SUTTON, Dec 4. Message-ID: Stanford University Interdisciplinary Colloquium Series: Adaptive Networks and their Applications December 4th (Monday, 3:45pm): Room 380-380C ******************************************************************************** DYNA: AN INTEGRATED ARCHITECTURE FOR LEARNING, PLANNING, AND REACTING Richard S. Sutton GTE Laboratories Incorporated ******************************************************************************** Abstract How should a robot decide what to do? The traditional answer in AI has been that it should deduce its best action in light of its current goals and world model, i.e., that it should _plan_. However, it is now widely recognized that planning's computational complexity makes it infeasible for rapid decision making and that its dependence on a complete and accurate world model also greatly limits its applicability. An alternative is to do the planning in advance and compile it into a set of rapid _reactions_, or situation-action rules, which are then used for real-time decision making. Yet a third approach is to _learn_ a good set of reactions by trial and error; this has the advantage that it eliminates all dependence on a world model. In this talk I present _Dyna_, a simple architecture integrating and permitting tradeoffs among all three approaches. Dyna is based on the old idea that planning is like trial-and-error learning from hypothetical experience. The theory of Dyna is based on the classical optimization technique of _dynamic_programming_, and on dynamic programming's relationship to reinforcement learning, to temporal-difference learning, and to AI methods for planning and search. In this talk, I summarize Dyna theory and present Dyna systems that learn from trial and error while they simultaneously learn a world model and use it to plan optimal action sequences. This work is an integration and extension of prior work by Barto, Watkins, and Whitehead. =========================================================================== GENERAL INFO: Location: Room 380-380C, which can be reached through the lower level between the Psychology and Mathematical Sciences buildings. Level: Technically oriented for persons working in related areas. Mailing lists: To be added to the network mailing list, netmail to netlist at psych.stanford.edu with "addme" as your subject header. Additional information: Contact Mark Gluck (gluck at psych.stanford.edu). From mclennan%MACLENNAN.CS.UTK.EDU at cs.utk.edu Tue Nov 21 13:29:24 1989 From: mclennan%MACLENNAN.CS.UTK.EDU at cs.utk.edu (mclennan%MACLENNAN.CS.UTK.EDU@cs.utk.edu) Date: Tue, 21 Nov 89 14:29:24 EDT Subject: What is a symbol system? Message-ID: <8911211929.AA22810@MACLENNAN.CS.UTK.EDU> Steve Harnad has invited rival definitions of the notion of a symbol system. I formulated the following (tentative) definition as a basis for discussion in a connectionism course I taught last year. After stating the definition I'll discuss some of the ways it differs from Harnad's. PROPERTIES OF DISCRETE SYMBOL SYSTEMS A. Tokens and Types 1. TOKENS can be unerringly separated from the background. 2. Tokens can be unambiguously classified as to TYPE. 3. There are a finite number of types. B. Formulas and Schemata 1. Tokens can be put into relationships with one another. 2. A FORMULA is an assemblage of interrelated tokens. 3. Formulas comprise a finite number of tokens. 4. Every formula results from a computation (see below) starting from a given token. 5. A SCHEMA is a class of relationships among tokens that depends only on the types of those tokens. 6. It can be unerringly determined whether a formula belongs to a given schema. C. Rules 1. Rules describe ANALYSIS and SYNTHESIS. 2. Analysis determines if a formula belongs to a given schema. 3. Synthesis constructs a formula belonging to a given schema. 4. It can be unerringly determined whether a rule applies to a given formula, and what schema will result from applying that rule to that formula. 5. A computational process is described by a finite set of rules. D. Computation 1. A COMPUTATION is the successive application of the rules to a given initial formula. 2. A computation comprises a finite number of rule appli- cations. COMPARISON WITH HARNAD'S DEFINITION 1. Note that my terminology is a little different from Steve's: his "atomic tokens" are my "tokens", his "composite tokens" are my "formulas". He refers to the "shape" of tokens, whereas I distinguish the "type" of an (atomic) token from the "schema" of a formula (composite token). 2. So far as I can see, Steve's definition does not include anything corresponding to my A.1, A.2, B.6 and C.4. There are all "exactness" properties -- central, although rarely stated, assumptions in the theory of formal systems. For example, A.1 and A.2 say that we (or a Turing machine) can tell when we're looking at a symbol, where it begins and ends, and what it is. It is important to state these assumptions, because they need not hold in real-life pattern identification, which is imperfect and inherently fuzzy. One reason connectionism is important is that by questioning these assumptions it makes them salient. 3. Steve's (3) and (7), which require formulas to be LINEAR arrangements of tokens, are too restrictive. There is noth- ing about syntactic arrangement that requires it to be linear (think of the schemata used in long division). Indeed, the relationship between the constituent symbols need not even be spatial (e.g., they could be "arranged" in the frequency domain, e.g., a chord is a formula comprising note tokens). This is the reason my B.5 specified only "relationships" (perhaps I should have said "physical rela- tionships"). 4. Steve nowhere requires his systems to be finite (although it could be argued that this is a consequence of their being PHYSICAL systems). I think finiteness is essential. The theory of computation grew out of Hilbert's finitary approach to the foundations of mathematics, and you don't get the standard theory of computation if infinite formulas, rules, sets of rules, etc. are allowed. Hence my A.3, B.3, C.5, D.2. 5. Steve requires symbol systems to be semantically interpret- able (8), but I think this is an empty requirement. Every symbol system is interpretable -- if only as itself (essen- tially the Herbrand interpretation). Also, mathematicians routinely manipulate formulas (e.g., involving differen- tials) that have no interpretation (in standard mathematics, and ignoring "trivial" Herbrand-like interpretations). 6. Steve's (1) specifies a SET of formulas (physical tokens), but places no restrictions on that set. I'm concerned that this may permit uncountable or highly irregular sets of for- mulas (e.g., all the uncomputable real numbers). I tried to avoid this problem by requiring the formulas to be generat- able by a finite computational process. This seems to hold for all the symbol systems discussed in the literature; in fact the formation rules are usually just a context-free grammar. My B.4 says, in effect, that there is a generative grammar (not necessarily context free) for the formulas, in fact, that the set of formulas is recursively enumerable. 7. My definition does not directly require a rule itself to be expressible as a formula (nearly Steve's 3), but I believe I can derive this from my C.1, C.2, C.3, although I wouldn't want to swear to it. (Here's the idea: C.2 and C.3 imply that analysis and synthesis can be unambiguously described by formulas that are exemplars of those schemata. Hence, by C.1, every rule can be described by two examplars, which are formulas.) Let me stress that the above definition is not final. Please punch holes in it! Bruce MacLennan Department of Computer Science 107 Ayres Hall The University of Tennessee Knoxville, TN 37996-1301 (615)974-0994/5067 maclennan at cs.utk.edu From np%FRLRI61.BITNET at CUNYVM.CUNY.EDU Tue Nov 21 13:51:43 1989 From: np%FRLRI61.BITNET at CUNYVM.CUNY.EDU (np%FRLRI61.BITNET@CUNYVM.CUNY.EDU) Date: Tue, 21 Nov 89 19:51:43 +0100 Subject: No subject Message-ID: <8911211851.AA05986@sun3a.lri.fr> Subject: dynamical non-linear process control with neural networks Dear College, I am currently working on the application of Neural Networks to the control of a dynamical non-linear process. Is there anybody who has already successfully used some neural method in order to control a dynamical non-linear process ? What about the dymamical properties (and possibilities) of a feed-forward neural net (e.g. trained with the classical back-propagation algorithm)? nicolas puech, Laboratoire de Recherche Informatique, ORSAY, FRANCE From Scott.Fahlman at B.GP.CS.CMU.EDU Tue Nov 21 17:11:32 1989 From: Scott.Fahlman at B.GP.CS.CMU.EDU (Scott.Fahlman@B.GP.CS.CMU.EDU) Date: Tue, 21 Nov 89 17:11:32 EST Subject: UCLA SFINX - "neural" net simulator In-Reply-To: Your message of Mon, 20 Nov 89 13:53:15 -0800. <8911202153.AA04758@retina.cs.ucla.edu> Message-ID: The UCLA-SFINX, a "neural" network simulator is now in pub- lic domain... I'm guessing that you don't really mean "public domain" here, since later int he message you say that this code is copyrighted and you go on at some length about how to get a license. There's a big legal difference between "public domain" and "copyrighted, but available free of charge for non-commerical use". "Public domain" means that the owner of the intellectual property is relinquishing all control over it, which does not seem to be the case here. I'm not complaining, just trying to clarify the situation here. I think it's great when people make useful pieces of software available without cost, even if there are some strings attached. -- Scott From krulwich at zowie.ils.nwu.edu Tue Nov 21 16:12:04 1989 From: krulwich at zowie.ils.nwu.edu (Bruce Krulwich) Date: Tue, 21 Nov 89 15:12:04 CST Subject: Symbol systems vs AI systems Message-ID: <8911212112.AA26977@zowie.ils.nwu.edu> I'm not sure if this is a good idea, but I'm going to throw in some thoughts to the "symbol system" question. Please note that I'm responding to the CONNECTIONISTS list and do not wish my message to be forwarded to any public newsgroups. There are two implicit assumptions that I think Steve is making in his message, especially given its being sent to CONNECTIONISTS. The first is that these characteristics of a "symbol system" do not apply to connectionist nets, and the second is that the "symbol systems" that he characterizes are in fact all classical (non-connectionist) AI systems. Even if he's not making these two assumptions, I think that others do so I'm going to go ahead and discuss them. First, the assumption that Steve's characterizations of "symbol systems" do not apply to neural nets. Looking at the 8 aspects of the definition, I think that each of them apply to NN's as much as they apply to many symbol systems. In other words, they apply to symbol systems if you look at symbol systems purely syntactically and ignore the meaning and theory that goes into the system. The same is true about NN's. From an anally syntactic point of view, neural nets are simply doing transformations on sets of unit values. (I'm not restricting this to feed-forward nets like many people do. This really is the case about all nets.) They have very specific rules about combining values, in fact, all the tokens (units) in the system use the same rule on different inputs. Clearly this view of NN's is missing the forest for the trees, because the point of NN's is the semantics of the computation they engage in and of the information they encode. My claim, however, is that the same is true of all AI systems. Looking at them as syntactic "symbol systems" is missing their entire point, and is missing what differentiates them from other systems. This leads me to the assumption that all classical AI systems are "symbol systems" as defined. I think that this is less true than my claim above about connectionist nets. Let's look, for example, at a research area called "case-based reasoning." (For those unfamiliar with CBR, take a look at the book "Inside Case-Based Reasoning" by Riesbeck and Schank, published by LEA, or the proceedings of the past two case-based reasoning workshops published by Morgan Kaufman.) The basic idea in CBR is that problems are solved by making analogies to previously solved problems. The assumption is that general rules are impossible to get because there is often not enough information to generalize from the instances that an agent gets. Looking at Steve's characterizations of a "symbol system," we can see that CBR systems have (a) no explicit rules, and (b) completely semantic matching (in most cases) that is not dependant on the "shape" of the representations. Certainly there is a level at which CBR systems are "symbol systems" in the same way that all computer programs are inherently "symbol systems." The point, however, is that this is _not_ the issue in CBR systems just like its not the issue in connectionist models. Since the _theory_ embedded in CBR systems is irrespective of several of the characterizations of "symbol systems," they are only "symbol systems" in the way that all connectionist models are "symbol systems" because they are all simulated on computers. I have used CBR as an example here, but the same could be said about alot of the recent work in analogical reasoning, analytical learning (such as EBL), default reasoning, and much of the rest of the semantically oriented AI systems. My claim is that one of two things is the case: (1) Much of the current work in classical AI does not fall into what Steve has characterized as "symbol systems," or (2) Connectionist nets _do_ fall into this catagory. It doesn't really matter which of these is the case, because each of them makes the characterization useless as a characterization of AI systems. I'd like to close by apologizing to CONNECTIONISTS readers if this post starts or continues a bad trend on the mailing list. The last thing that anyone wants is for CONNECTIONISTS to mimic COMP.AI. I've tried to keep my points to ones that address the assumptions that alot of connectionist research makes in the hope of keeping this from blowing up too much. Bruce Krulwich Institute for the Learning Sciences krulwich at ils.nwu.edu From skrzypek at CS.UCLA.EDU Tue Nov 21 19:53:14 1989 From: skrzypek at CS.UCLA.EDU (Dr. Josef Skrzypek) Date: Tue, 21 Nov 89 16:53:14 PST Subject: UCLA SFINX - "neural" net simulator In-Reply-To: Scott.Fahlman@B.GP.CS.CMU.EDU's message of Tue, 21 Nov 89 17:11:32 EST <8911212212.AA28884@shemp.cs.ucla.edu> Message-ID: <8911220053.AA08590@retina.cs.ucla.edu> Date: Tue, 21 Nov 89 17:11:32 EST From: Scott.Fahlman at B.GP.CS.CMU.EDU The UCLA-SFINX, a "neural" network simulator is now in pub- lic domain... I'm guessing that you don't really mean "public domain" here, since later int he message you say that this code is copyrighted and you go on at some length about how to get a license. There's a big legal difference between "public domain" and "copyrighted, but available free of charge for non-commerical use". "Public domain" means that the owner of the intellectual property is relinquishing all control over it, which does not seem to be the case here. I'm not complaining, just trying to clarify the situation here. I think it's great when people make useful pieces of software available without cost, even if there are some strings attached. -- Scott >>>>>>>>>>>>>>>>>>>>>>>>>> You are correct that SFINX is not strictly in "public domain". Eventually, we will adapt the Free Software Foundation license agreement; the effort needed to overcome various administrative "procedures" would simply delay the release. Thanks for your comments. Josef From Dave.Touretzky at B.GP.CS.CMU.EDU Tue Nov 21 23:41:09 1989 From: Dave.Touretzky at B.GP.CS.CMU.EDU (Dave.Touretzky@B.GP.CS.CMU.EDU) Date: Tue, 21 Nov 89 23:41:09 EST Subject: neural net intro books Message-ID: <856.627712869@DST.BOLTZ.CS.CMU.EDU> A common topic of discussion when academic neural net types get together is ``What sort books are available for teaching neural nets?'' I recently got a catalog from Van Nostrand Reinhold that listed two introductory books, although they're not exactly textbooks. The details are given below. I haven't seen either of them yet, so this is not an endorsement, just an announcement of their existence. If someone has seen these books and would like to post a short review to CONNECTIONISTS, that would be helpful. -- Dave ================ NEURAL COMPUTING Theory and Practice by Philip D. Wasserman, ANZA Research, Inc. 230 pages, 100 illustrations, $36.95 The complex mathematics and algorithms of artificial neural networks are broken down into simple procedures in this welcome tutorial. Fully explored are network fundamentals, implementation of commonly-used paradigms, and how to enhance problem-solving through integration of neural net research with traditional artificial intelligence and computing methods. Real-world examples clarify applications of artificial neural networks in computer science, engineering, physiology, and psychology. CONTENTS: Introduction. Fundamentals of Artificial Neural Networks. Perceptrons. Backpropagation. Counterpropagation Networks. Statistical Methods. Hopfield Nets. Bidirectional Associative Memories. Adaptive Resonance Theory. Optical Neural Networks. The Cognitron and Neocognitron. APPENDICES. The Biological Neural Network. Vector and Matrix Operations. Training Algorithms. Index. NEURAL NETWORK ARCHITECTURES An Introduction by Judith Dayhoff, Ph.D., Editor of the Journal of Neural Network Computing 220 pages, 100 illustrations, $34.95 This down-to-earth book gives you a plain-English explanation of the relationships between biological and artificial neural networks, plus detailed assessments of important uses of network architectures today. CONTENTS: An Overview of Neural Network Technology. Neurons and Network Topologies. Early Paradigms - The Beginnings of Today's Neural Networks. The Hopfield Network: Computational Models and Result. Back-Error Propagation: Paradigms and Applications. Competitive Learning: Paradigms and Competitive Neurons from Biological Systems. Biological Neural Systems: Organization. Structural Diversity. Temporal Dynamics. Origins of Artificial Neural Systems. Brain Structure and Function. Biological Nerve Cells. Synapses - How Do Living Nerve Cells Interconnect? What Is Random and What Is Fixed in the Brain's Neural Networks? How Do Biological Systems Really Compare to Computational Neural Networks? Associative and Adaptive Networks - More Paradigms. More Applications - Emphasizing Vision, Speech, and Pattern Recognition. New Directions for Neural Networks. From elsley at jupiter.risc.com Wed Nov 22 14:08:07 1989 From: elsley at jupiter.risc.com (Dick Elsley) Date: Wed, 22 Nov 89 11:08:07 PST Subject: Research position available. Message-ID: <8911221904.AA06576@jupiter.risc.com> ***** DO NOT FORWARD TO ANY OTHER LISTS ***** The Rockwell International Science Center has a job opening for a Neural Network researcher. We are looking for someone with research experience in neural networks (including simulations) and an interest in developing the necessary conceptual bridges between neural network research and application problems. The Science Center is the corporate research lab of Rockwell International and is located in sunny Thousand Oaks, CA, just northwest of Los Angeles. We do a combination of basic and applied research in a wide variety of areas generally related to technologies that will eventually find their way into 'systems'. We also interact closely with key University groups and with the product divisions of Rockwell. Our work includes direct contracts from DARPA, ONR, etc., IR&D (discretionary), and joint work with our product divisions. Publication of results is encouraged. Very little of what we do is classified. Our neural network research team is currently active in machine vision, adaptive control, signal processing and several hardware implementation technologies. U.S. citizenship or permanent residency (green card) may be required. If you are interested, please send me a resume by E-mail or surface mail; and/or see me in Denver. Professional Staffing KS-20 Rockwell International Science Center 1049 Camino dos Rios Thousand Oaks, CA 91390 Dr. Richard K.(Dick) Elsley elsley at risc.com From jbower at smaug.cns.caltech.edu Wed Nov 22 19:29:27 1989 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Wed, 22 Nov 89 16:29:27 PST Subject: NIPS DEMOS Message-ID: <8911230029.AA27257@smaug.cns.caltech.edu> NIPS 89 MEETING ANNOUNCEMENT COMPUTER DEMOS We have finally received word that both DEC and SUN will provide computers for the computer demo room at this years NIPS meeting. The machines available include a SUN 4/110 color system with 1/4" tape and 16 Mb of memory, and a DEC workstation 3100 color system with a TK50 tape drive and 16 Mb of memory. Both machines will run the latest release of their respective operating systems. In addition, they will each run X11. Authors presenting papers at the meeting should feel free to bring software for demonstration. Software not associated with presentations will be demonstrated dependent on time and space. The demo room will be staffed throughout the conference by John Uhley. Those interested in performing demos should contact John at the meeting. No on site development will be possible. Software should be brought in the form of debugged and compiled code. From harnad at clarity.Princeton.EDU Fri Nov 24 16:31:37 1989 From: harnad at clarity.Princeton.EDU (Stevan Harnad) Date: Fri, 24 Nov 89 16:31:37 EST Subject: Connectionist Learning/Representation: BBS Call for Commentators Message-ID: <8911242131.AA00826@reason.Princeton.EDU> Below is the abstract of a forthcoming target article to appear in Behavioral and Brain Sciences (BBS), an international, interdisciplinary journal that provides Open Peer Commentary on important and controversial current research in the biobehavioral and cognitive sciences. Commentators must be current BBS Associates or nominated by a current BBS Associate. To be considered as a commentator on this article, to suggest other appropriate commentators, or for information about how to become a BBS Associate, please send email to: harnad at confidence.princeton.edu harnad at pucc.bitnet or write to: BBS, 20 Nassau Street, #240, Princeton NJ 08542 [tel: 609-921-7771] ____________________________________________________________________ WHAT CONNECTIONIST MODELS LEARN: LEARNING AND REPRESENTATION IN CONNECTIONIST NETWORKS Stephen J Hanson Learning and Knowledge Acquisition Group Siemens Research Center Princeton NJ 08540 and David J Burr Artificial Intelligence and Communications Research Group Bellcore Morristown NJ 07960 Connectionist models provide a promising alternative to the traditional computational approach that has for several decades dominated cognitive science and artificial intelligence, although the nature of connectionist models and their relation to symbol processing remains controversial. Connectionist models can be characterized by three general computational features: distinct layers of interconnected units, recursive rules for updating the strengths of the connections during learning, and "simple" homogeneous computing elements. Using just these three features one can construct surprisingly elegant and powerful models of memory, perception, motor control, categorization and reasoning. What makes the connectionist approach unique is not its variety of representational possibilities (including "distributed representations") or its departure from explicit rule-based models, or even its preoccupation with the brain metaphor. Rather, it is that connectionist models can be used to explore systematically the complex interaction between learning and representation, as we try to demonstrate through the analysis of several large networks. From edmond at CS.UCLA.EDU Mon Nov 27 20:18:03 1989 From: edmond at CS.UCLA.EDU (Edmond Mesrobian) Date: Mon, 27 Nov 89 17:18:03 PST Subject: UCLA SFINX - "neural" net simulator Message-ID: <8911280118.AA23493@retina.cs.ucla.edu> The original announcement concerning UCLA-SFINX, a neural network simulator available from the Machine Perception Lab, was ambiguous. The simulator does not have an X Window System user interface. The simulator uses a command line interpreter. However, the simulator does support a variety of graphics environments as output media for simulation results. Output graphics support is provided for the following environments: 1) X Window System 2) HP workstations using Starbase Graphics Library 3) Sun Workstations using Matrox VIP-1024 Frame Grabbers 4) IBM RTs using an Imagraph AGC-1010P color graphics card. We apologize for any confusion caused by the original announcement. If you have any questions concerning UCLA-SFINX, please send email to sfinx at retina.cs.ucla.edu or US mail to the address below. Edmond Mesrobian UCLA Machine Perception Lab 3531 Boelter Hall Los Angeles, CA 90024 From ST401843%BROWNVM.BITNET at vma.CC.CMU.EDU Tue Nov 28 13:48:29 1989 From: ST401843%BROWNVM.BITNET at vma.CC.CMU.EDU (thanasis kehagias) Date: Tue, 28 Nov 89 13:48:29 EST Subject: Probability Learning Nets Message-ID: a while ago i had asked for recurrent net references and had inserted a request for probability learninets as well. the rec-net bibliography is in the making, and the ones who have seen the preliminary version must have noticed that there was a lot of responses. not so for the probability learning nets. i had a couple of responses only. i did my own digging around and found a couple more and i enclose them in this message as a seed. but i would be really interested in any other refernces anybody has. once again, when done, i will circulate the resulting bibliography. thanasis PS: bibtex format is always welcome!!! @article{kn: title ="Cognitive and Psychological Computation with Neural Models", author ="J.A. Anderson", journal ="IEEE Transactions on Systems, Man and Cybernetics", volume ="SMC-13", year ="1983" } @article{kn: title ="Distinctive Features, Categorical Perception and Learning: some Applications of a Neural Model ", author ="J.A. Anderson et.al. ", journal ="Psychological Review", year ="1977", volume ="84", page ="413-451", } @inproceedings{kn: title ="G-Maximization: An Unsupervised Learning Procedure for discovering Regularities ", booktitle ="Neural Networks for Computing ", author ="B. Pearlmutter and G. Hinton ", editor ="J.S. Denker ", year ="1986 ", pages ="333-338", organization ="American Institute for Physics" } @techreport{kn: author ="H.J. Sussman", title ="On the Convergence of Learning Algorithms for Boltzmann Machines", number ="sycon-88-03", institution ="Rytgers Center for Systems and Control", year ="1988" }