From dwunsch at blake.u.washington.edu Mon Dec 3 14:47:46 1990 From: dwunsch at blake.u.washington.edu (Don Wunsch) Date: Mon, 3 Dec 90 11:47:46 -0800 Subject: IJCNN-91-Seattle paper deadline is coming up! Message-ID: <9012031947.AA23852@blake.u.washington.edu> JCNN '91 Seattle Call for Papers The International Neural Networks Society (INNS) and the Institute for Electronic and Electrical Engineers (IEEE) invite all persons interested in the field of Neural Networks to submit papers for possible presentation at the Conference. Papers must be RECEIVED by February 1, 1991. Submissions received after February 1, 1991 will be returned unopened. All submissions will be acknowledged by mail. International authors should submit their work via Air Mail or Express courier so as to ensure timely arrival. Eight copies (one original and seven copies) are required for submission. Do not fold or staple the original, camera-ready copy. Do not number the pages on the front of the camera-ready copy. Papers of no more than six pages, including figures, tables and references, should be written in English, and only complete papers will be considered. Papers must be submitted camera-ready on 8 1/2" by 11" white bond paper with 1" margins on each of the top, bottom, left and right sides, and un-numbered. They should be prepared by a typewriter or letter-quality printer in one-column format, single-spaced, in Times or similar style font of 10 point or larger, and should be printed on one side of the paper only. FAX submissions are not acceptable. Centered at the top of the first page should be the complete title, author name(s), affiliation(s) and mailing address(es). This is followed by a space and then the abstract, up to 15 lines, followed by the text. In an accompanying letter, the fillowing must be included: Corresponding Author: Name, mailing address, telephone and FAX numbers Technical Area (Neurobiology, applications, electronic implementations, optical implementations, image processing, vision, speech, network dynamics, optimization, robotics and control, learning and generalization, neural network architectures, or other) Presentation Format Preferred: Oral or Poster Presenter: Name, mailing address, telephone and FAX numbers If an oral talk is requested, include any special audio/video requests. Special audio/video requests beyond 35mm slide and overhead transparency projectors will be honored only if there are sufficient requests to justify them. If you have special audio/video needs, please contact Sarah Eck at conference management for more information. Send Papers to: IJCNN '91 Seattle Conference Management, Attn. Sarah Eck, MS/GH-22, Suite 108 5001 25th Ave. NE, Seattle, WA 98195 Tel (206) 543-0888 FAX (206) 685-9359 DEADLINE FEBRUARY 1, 1991 Submissions received after this date will be returned unopened. The cost for IJCNN-91-Seattle is $195.00 for INNS and IEEE members for early registration, deadline March 1, 1991. Non-members early registration is $295.00 and students will pay $50.00. Late registration will be honored until June 1, 1991 to members at $295.00, non-members at $395.00, and students at $75.00. On-site registration will be $395.00, $495.00 and $95.00 respectively. Tutorials will be offered at an additional cost of $195.00, or $295.00 for tutorial registration on site. Exhibitors will present the latest in neural networks, including neurocomputers, VLSI neural networks, implementations, software systems and applications at IJCNN-91-SEATTLE. IJCNN-91-SEATTLE is the neural network industry's largest trade show. Hope to see you there! Don From dyer at CS.UCLA.EDU Mon Dec 3 23:23:12 1990 From: dyer at CS.UCLA.EDU (Dr Michael G Dyer) Date: Mon, 3 Dec 90 20:23:12 PST Subject: reprints available Message-ID: <901204.042312z.04326.dyer@lanai.cs.ucla.edu> reprints available: Dyer, M. G. Distributed symbol formation and processing in connectionist networks. Journal of Experimental and Theoretical Artificial Intelligence. Vol. 2, 215-239, 1990. Abstract: Distributed connectionist (DC) systems offer a set of processing features which are distinct from those provided by traditional symbol processing (SP) systems. In general, the features of DC systems are derived from the nature of their distributed representations. Such representations have a microsemantics -- i.e. symbols with similar internal representations tend to have similar processing effects. In contrast, the symbols in SP systems have no intrinsic microsemantics of their own; e.g. SP symbols are formed by concatenating ASCII codes that are static, human engineered, and arbitrary. Such symbols possess only a macrosemantics -- i.e. symbols are placed into structured relationships with other symbols, via pointers, and bindings are propagated via variables. The fact that DC and SP systems each provide a distinct set of useful features serves as a strong research motivation for seeking a synthesis. What is needed for such a synthesis is a method by which symbols can dynamically form their own microsemantics, while at the same time entering into structured, recursive relationships with other symbols, thus developing also a macrosemantics. Here, we describe a general method, called symbol recirculation, for allowing symbols to form their own microsemantics. We then discuss three techniques for implementing variables and bindings in DC systems. Finally, we describe a number of DC systems, based on these techniques, which perform a variety of high-level cognitive tasks. requests for reprints should be sent to: valerie at cs.ucla.edu From dwunsch at blake.u.washington.edu Tue Dec 4 02:29:44 1990 From: dwunsch at blake.u.washington.edu (Don Wunsch) Date: Mon, 3 Dec 90 23:29:44 -0800 Subject: left-out info: IJCNN-91-Seattle Message-ID: <9012040729.AA14159@blake.u.washington.edu> >Date: Tue, 4 Dec 90 12:15 HKT >Hi, >The date of the conference seems to be missing. Would you please post it on >the network again? Thanks. >Regards, >Dr. Dit-Yan Yeung Thanks, Dr. Yeung, for pointing out my oversight. I don't want to use up all my permitted postings, so I'll just post the few most critical lines, this time including the date. JCNN '91 Seattle, July 8-12, 1991 Papers must be RECEIVED by February 1, 1991. Send Papers to: IJCNN '91 Seattle Conference Management, Attn. Sarah Eck, MS/GH-22, Suite 108 5001 25th Ave. NE, Seattle, WA 98195 Tel (206) 543-0888 FAX (206) 685-9359 The cost for IJCNN-91-Seattle is $195.00 for INNS and IEEE members for early registration, deadline March 1, 1991. Non-members early registration is $295.00 and students will pay $50.00. Late registration will be honored until June 1, 1991 to members at $295.00, non-members at $395.00, and students at $75.00. On-site registration will be $395.00, $495.00 and $95.00 respectively. Finally, another addition: anyone interested in volunteering should contact me at dwunsch at blake.u.washington.edu as soon as you can. Don From dave at cogsci.indiana.edu Tue Dec 4 19:17:03 1990 From: dave at cogsci.indiana.edu (David Chalmers) Date: Tue, 4 Dec 90 19:17:03 EST Subject: FTP archives; Technical report available Message-ID: (1) Following many requests, the bibliography that I have compiled on the philosophy of mind/cognition/AI is now available by anonymous ftp from cogsci.indiana.edu (129.79.238.6). It is contained in 5 files chalmers.bib.* in the directory "pub". Also contained in this archive are various articles by members of the Center for Research on Concepts and Cognition. Instructions for retrieval are given below. (2) The following technical report is now available. THE EVOLUTION OF LEARNING: AN EXPERIMENT IN GENETIC CONNECTIONISM David J. Chalmers Center for Research on Concepts and Cognition Indiana University CRCC-TR-47 This paper explores how an evolutionary process can produce systems that learn. A general framework for the evolution of learning is outlined, and is applied to the task of evolving mechanisms suitable for supervised learning in single-layer neural networks. Dynamic properties of a network's information-processing capacity are encoded genetically, and these properties are subjected to selective pressure based on their success in producing adaptive behavior in diverse environments. As a result of selection and genetic recombination, various successful learning mechanisms evolve, including the well-known delta rule. The effect of environmental diversity on the evolution of learning is investigated, and the role of different kinds of emergent phenomena in genetic and connectionist systems is discussed. A version of this paper appears in _Proceedings of the 1990 Connectionist Models Summer School_ (Touretzky, Elman, Sejnowski and Hinton, eds). ----------------------------------------------------------------------------- This paper may be retrieved by anonymous ftp from cogsci.indiana.edu (129.79.238.6). The file is chalmers.evolution.ps.Z, in the directory pub. To retrieve, do the following: unix-1> ftp cogsci.indiana.edu # (or ftp 129.79.238.6) Connected to cogsci.indiana.edu Name (cogsci.indiana.edu:): anonymous 331 Guest login ok, sent ident as password. Password: [identification] 230 Guest login ok, access restrictions apply. ftp> cd pub ftp> binary ftp> get chalmers.evolution.ps.Z ftp> quit unix-2> uncompress chalmers.evolution.ps.Z unix-3> lpr -P(your_local_postscript_printer) chalmers.evolution.ps The file is also available from the Ohio State neuroprose archives by the usual methods. If you do not have access to ftp, hardcopies may be obtained by sending e-mail to dave at cogsci.indiana.edu. From worth at park.bu.edu Wed Dec 5 11:28:44 1990 From: worth at park.bu.edu (Andrew J. Worth) Date: Wed, 5 Dec 90 11:28:44 -0500 Subject: Academic Program Info Request Message-ID: <9012051628.AA15635@park.bu.edu> ISSNNet Request for Academic Program Information The International Student Society for Neural Networks (ISSNNet) is compiling a list of academic programs relating to Neural Networks from around the world. We would like your input if you are a member of a scholastic program that is in any way related to Neural Networks, Parallel Distributed Processing, Connectionism, Computational Neuroscience, Neural Modeling, Neural Computing, etc. We hope to provide this service so that (1) interested students will be able to apply to those programs that will most closely satisfy their educational goals, and (2) current students and non-students will be aware of existing academic programs. This service is intended to not only provide an overview of these programs and contact points for more information, but also a personal glimpse into what's behind the official descriptions. All information will be made publicly available and will be updated as new programs are created and as programs change. Complying with ISSNNet's goal to be absolutely unbiased, we would like this to become THE source of information on academic programs in this field. ISSNNet would like to provide the following information: - Official address to contact for more information (surface mail and email) - Official description of the program. - Names of Faculty Members and their interests - Degrees requirements (BA, BS, MA, MS, PhD, etc.) - Short description of courses offered - Computing resources (Hardware and Software Tools) - Number of Students (grad/undergrad) and related faculty - A brief *personal* description of the program, department, etc. describing motivation, emphasis, goals, and/or overall ambiance. - Student Contacts (w/ telephone numbers, email and surface addresses, degree sought, interests, and date of graduation) This information is above and beyond the academic questionnaires that were filled out at the San Diego and Paris conferences and will eventually be made available via ftp and also by other means through ISSNNet (your submission will be taken as permission to make the information public unless we are otherwise notified). Coordinated responses from each institution are encouraged and will be appreciated. Please submit descriptions of academic programs in plain text (email is preferred) following the guidelines above to: issnnet-acad-progs at bucasb.bu.edu We will also be able to re-distribute information in other emailable formats such as postscript or LaTeX. Thank you for your time and effort, Andy. ---------------------------------------------------------------------- Andrew J. Worth (617) 353-6741 ISSNNet ISSNNet Academic Program Editor P.O. Box 557 issnnet-acad-progs at park.bu.edu New Town Br. worth at park.bu.edu Boston, MA 02215 USA ---------------------------------------------------------------------- From franklins at memstvx1.memst.edu Wed Dec 5 11:08:00 1990 From: franklins at memstvx1.memst.edu (franklins@memstvx1.memst.edu) Date: 5 Dec 90 10:08:00 CST Subject: Request for references Message-ID: I'm preparing to write two survey reports and would appreciate references. The first concerns applications of cellular automata to neural networks and/or their relative computational power. The second concerns neural computability, that is the neural network version of classical computability theory. Typical questions: What can neural networks compute under the most ideal conditions? Are there problems that are provably neurally unsolvable? I would greatly appreciate any reference to papers or articles touching upon these topics. If the response warrants, and if there is interest, I'll post bibliographies. I'll surely make the reports available. Stan Franklin Math Sciences Memphis State Memphis TN 38152 franklins at msuvx1.memst.edu franklins at memstvx1.bitnet From yuhas at faline.bellcore.com Wed Dec 5 17:30:23 1990 From: yuhas at faline.bellcore.com (Ben Yuhas) Date: Wed, 5 Dec 90 17:30:23 EST Subject: Commercial applications. Message-ID: <9012052230.AA14207@faline.bellcore.com> I am trying to gather a list of neural networks applications that have, or are about to, become commercial products. At NIPS we heard about such work from FORD and Sarnoff labs, I would appreciate any other examples that any of you are aware of. Ben yuhas at bellcore.com From reggia at cs.UMD.EDU Thu Dec 6 14:33:00 1990 From: reggia at cs.UMD.EDU (James A. Reggia) Date: Thu, 6 Dec 90 14:33:00 -0500 Subject: post-doctoral position in neural nets in France Message-ID: <9012061933.AA20258@mimsy.UMD.EDU> Post-Doctorate Position in France: A two year post-doctoral position in neural modelling is available at ONERA/CERT in Toulouse, France. ONERA/CERT is a government research laboratory: Office National d'Etudes et de Recherches Aerospatiales/Centre d'Etudes et de Recherches de Toulouse. The pay is approximately $2000/month. The working language is French, but most individuals at ONERA/CERT speak English fairly well. Work in this position would include one or more of the following: development and study of learning rules in competitive systems, matching images, or development of neural modelling software on a connection machine (the latter would require spending some time in Paris too). For further information or for answers to questions, please contact Paul Bourret at bourret at tls-cs.cert.fr via email. From ang at hertz.njit.edu Fri Dec 7 00:09:36 1990 From: ang at hertz.njit.edu (nirwan ansari fac ee) Date: Fri, 7 Dec 90 00:09:36 EST Subject: Tabu Search Message-ID: <9012070509.AA06607@hertz.njit.edu> I'm interested to do comparative studies between Tabu search and annealing algorithm. Could someone kindly give me references on tabu search? Thanks. From Dave.Touretzky at DST.BOLTZ.CS.CMU.EDU Fri Dec 7 02:04:35 1990 From: Dave.Touretzky at DST.BOLTZ.CS.CMU.EDU (Dave.Touretzky@DST.BOLTZ.CS.CMU.EDU) Date: Fri, 07 Dec 90 02:04:35 EST Subject: Tabu Search, and other stuff In-Reply-To: Your message of Fri, 07 Dec 90 00:09:36 -0500. <9012070509.AA06607@hertz.njit.edu> Message-ID: <9268.660553475@DST.BOLTZ.CS.CMU.EDU> > I'm interested to do comparative studies between Tabu search and > annealing algorithm. Could someone kindly give me references on tabu > search? Thanks. I want to remind people that the CONNECTIONISTS list is not intended for those who are too lazy to do their own library work. If you're going to post a plea for basic information, please do it on comp.ai.neural-nets. This list is intended for discussions among practicing researchers. Also, to save everyone else a lot of trouble: I never heard of Tabu search either. When Nirwan Ansari finds out what it is and compiles an extensive bibliography on the subject, I invite him to post *that* to CONNECTIONISTS. Meanwhile, here's some other news for the connectionist community: - The proceedings of the 1990 Connectionist Models Summer School, held at UCSD, are now available from Morgan Kaufmann. I'll post a table of contents and ordering information next week. - Reminder: final camera-ready papers for NIPS 3 are due by January 18. Author kits will be mailed out today or early next week. The formatting macros are very similar to last year. Seven page limit on all papers. - Reminder: the submission deadline for IJCAI papers is December 10. There is a "Connectionist and PDP Models" track. The submission deadline for AAAI-91 is January 30. -- Dave Touretzky From pako at neuronstar.it.lut.fi Fri Dec 7 07:49:28 1990 From: pako at neuronstar.it.lut.fi (Pasi Koikkalainen) Date: Fri, 7 Dec 90 14:49:28 +0200 Subject: ICANN-91 Message-ID: <9012071249.AA00402@neuronstar.it.lut.fi> January 15 is approaching fast .... ... but there is still time to write a paper for ICANN-91. -== ICANN-91 ====- -== INTERNATIONAL CONFERENCE ON ARTIFICIAL NEURAL NETWORKS ========- -== Helsinki University of Technology ==- -== Espoo, Finland, June 24-28, 1991 ==- Conference Chair: Conference Committee: Teuvo Kohonen (Finland) Bernard Angeniol (France) Eduardo Caianiello (Italy) Program Chair: Rolf Eckmiller (FRG) Igor Aleksander (England) John Hertz (Denmark) Luc Steels (Belgium) -== Second Announcement and Call for Papers =========================- THE CONFERENCE: ACTIVITIES: =============== ============ This conference will be a major - Oral and poster sessions international contact forum for - Invited talks experts from academia and industry - Industrial exhibition worldwide. Around 1000 participants - Prototype demonstrations are expected. - Video presentations -=============== TUTORIALS ==================- Nine tutorals will be given on Monday 24, 1990, covering the central techniques, developments, and prospects of Artificial Neural Networks. The tutorial speakers are leading experts in the filed: 1a J. Hertz - The Physics of Neural Networks 1b E. Oja - Pattern Recognition and Neural Networks 2 B. Widrow and T. Kohonen - Introduction to Neural Networks 3a J. Taylor - Mathematical Problems in Neural Networks Dynamics 3b F. Faggin - Hardware Implementations of Neural Networks 4a H. Ritter - Self-Organizing Map and Applications 4b T. Schwartz - How to Start a Business in Neural Networks 5a P. Werbos - Generalized Backprobagation: Basic Principles and Central Applications 5b P. Treleaven - Neural Programming Environment -==== INVITED SPEAKERS ====- In the oral sessions there will be invited talks given by some of the leading experts in various fields of Neural Networks. The invited speakers include: B. Angeniol (France), G. Carpenter (USA), R. Eckmiller (Germany), F. Fogelman (France), K. Goser (Germany), S. Grossberg (USA), J. Hertz (Denmark), K. Koenderink (Holland), A. Lansner (Sweden), C. von der Malsburg (Germany), W. von Seelen (Germany), J. G. Taylor (UK), P. Treleaven (UK) -================ PLENARY SESSIONS ====================- There will be several plenary sessions on topics that are of interest to all participants. The speakers who are pioneers in neural networks are: I. Alexander - Professor at Imperial College (England) A. Amari - Professor at Tokyo University (Japan) E. Caianiello - Professor at University of Salerno (Italy) F. Faggin - President of Synaptics Inc. (USA) R. Hecht-Nielsen - Chair of the Board of HNC corporation (USA) T. Kohonen - Professor at Helsinki University of Technology (Finland) =-= NON-COMMERCIAL DEMONSTRATIONS TRACK =-= As a new feature in neural network conferences participants will have a possibility to show video presentations and demonstrate prototype programs and systems on a non-commercial basis in a separate demonstration track, running in parallel with the oral and poster sessions. There will be a video room and PC/workstation classes available with standard equipment. The time slot reservation for the demonstration can be made using the registration form on which you also have to indicate the title of your demo. Detailed information will automatically be sent to those who reserve a time slot for demonstration. Further information can be requested from: Mr. Jari Kangas Helsinki University of Technology Laboratory of Computer and Information Science SF-02150 Espoo, Finland ----------------------------------------- E-mail (internet): icann91 at hutmc.hut.fi Fax: +358-0-4513277, Telex: 125161 HTKK SF =-= INSTRUCTIONS FOR AUTHORS =-= Complete papers of at most 6 pages are invited for oral or poster presentation in one of the sessions given below: 1. Mathematical theories of networks and dynamical systems 2. Neural network architectures and algorithms (including organizations and comparative studies) 3. Artificial associative memories 4. Pattern recognition and signal processing (especially vision and speech) 5. Self-organization and vector quantization 6. Robotics and control 7. "Neural" knowledge data bases and non-rule-based decision making 8. Software development (design tools, parallel algorithms, and software packages) 9. Hardware implementations (coprocessors, VLSI, optical, and molecular) 10. Commercial and industrial applications 11. Biological and physiological connection (synaptic and cell functions, sensory and motor functions, and memory) 12. Neural models for cognitive science and high-level brain functions 13. Physics connection (thermodynamical models, spin glasses, and chaos) Papers may be submitted for oral or poster presentation. All papers must be written in English. Only complete papers of at most 6 pages will be considered for oral presentations, and for 4 pages for posters. The program committee may designate a paper intended for oral presentation to a poster presentation instead, and may also change the intended session to balance the conference program. == DEADLINE IS January 15, 1991 Deadline for submitting manuscripts is January 15, 1991. The Conference Proceedings will be published as a book by Elsevier Science Publishers B.V. Therefore, the final versions must be typed or pasted on special forms provided by the publisher for authors of accepted papers. The papers will be reproduced directly from the received forms. In order to help the authors, the conference organizers, and the publisher, we request that the submitted manuscripts already follow the final layout. Therefore, please observe carefully the instructions below. 1. The typing area is 16.7 x 25.8 cm (6.5 x 10 in.) 2. Do not use page numbers 3. Use a font (also tables and figures) large enough to withstand reduction to 70%. Do not use font smaller than 11 points. 4. The title should be written in capital letters 2 cm from the top of the first page, followed by the authors' names and addresses and the abstract left-justified, indenting everything by 2 cm. 5. In the text, do not indent headings or captions. 6. Insert all tables, figures, and figure captions in the text at their final positions. 7. For references in the text, use numbers in square brackets. Submit 6 review copies of the manuscript. FAX OR EMAIL SUBMISSIONS ARE NOT ACCEPTED. With each manuscript, please indicate - the name of the principal author - the mail address, telephone, and fax numbers - whether the paper in intended for oral or poster presentation - which session it is submitted to (see sessions above). You can also give two alternatives. You will be notified of the review result by February 20, 1991, and the authors of accepted papers will receive an authors' kit from the publisher. Deadline for the final papers typed on the special forms is March 15, 1991. NOTICE! The final camera-ready papers must be received by the Organizing Committee by that date! === SEND THE MANUSCRIPTS TO: Prof. Olli Simula ICANN-91 Organization Chairman Helsinki University of Technology SF-02150 Espoo, Finland --------------------------- Fax: +358 0 451 3277 Telex: 125161 HTKK SF Email (internet): icann91 at hutmc.hut.fi -== CONFERENCE VENUE ===- The street address of the Conference venue is Helsinki University of Technology Otakaari 1 SF-02150 Espoo Finland -== SOCIAL PROGRAM, TOURS AND EXCURSIONS =====- In addition to the scientific program, several social occasions are included in the registration fee. These include: 24 June: Get-together party and opening of the exhibition 26 June: Concert sponsored by the City of Espoo 27 June: Banquet Several tours and excursions are optional: 24 June: City Sightseeing (90 FIM) 25 June: Porvoo by bus and boat (400 FIM) 26 June: Finnish Glass Discovery (350 FIM) 27 June: Design Tour (100 FIM) Pre- and post-conference tours and excursions will also be arranged: 21-23 June: Lapland with Midnight sun (2900 FIM) 22-23 June: Cruise to Tallinn (Estonia, USSR), (850 FIM) 28-30 June: Leningrad by air (USSR), (2950 FIM) -== GENERAL INFORMATION, REGISTRATION AND ACCOMMODATION ===- There will be a special ICANN-91 reception desk at Helsinki-Vantaa airport. The desk will be open on Sunday June 23 and on Monday June 24 from noon until midnight. Registration desk is located in the Lobby of the main building at the Helsinki University of Technology, address: Otakaari 1, 02150 Espoo. For more information about registration and accommodation, please contact: ICANN-91 CMS-CONGREX P.O.Box 151 Neitsytpolku 12 A SF-00141 Helsinki, Finland Tel.: +358 0 175 355 Fax: +358 0 170 122 Telex: 123 585 cms sf -------------------------------------------------------------------- From rwp at engineering.cambridge.ac.uk Fri Dec 7 07:21:57 1990 From: rwp at engineering.cambridge.ac.uk (Richard Prager) Date: Fri, 7 Dec 1990 12:21:57 GMT Subject: Cambridge Neural Networks Course Announcement Message-ID: <17498.9012071221@dsl.eng.cam.ac.uk> Cambridge University Programme for Industry Neural Networks Theory Design & Applications 15 - 19 April 1991 Preliminary Announcement A five-day advanced short course on the theory, design and applications of artificial neural networks, presented by leading international experts in the field: Professor David RUMELHART Stanford University Professor Geoffrey HINTON University of Toronto Dr Andy BARTO University of Massachusetts Dr Herve BOURLARD Philips Research Labs. Belgium Professor Elie BIENENSTOCK ESPCI Paris Professor Frank FALLSIDE University of Cambridge Professor Horace BARLOW University of Cambridge Dr Peter RAYNER University of Cambridge Dr Lionel TARASSENKO University of Oxford This intensive short course for scientists, engineers and their managers aims to develop an understanding of the potential for neural network-based solutions, and demonstrates techniques for transforming problems to enable neural networks to solve them more efficiently. Design methodologies for a number of common neural network architectures will be described. By the end of the course delegates will be able to assess the potential usefulness of neural network technology to their own application domains. They will have an understanding of the strength and weakness of a neural network approach and will have acquired an insight into factors affecting neural network design and performance. The lectures will be complemented by discussion sessions and practical computing sessions that will demonstrate simulated applications. The lectures will cover basic theory behind neural network algorithms, together with applications in speech and language processing, signal processing, and robotic control. If you are interested please print out the form below, fill it in and return to Pam Whitfield, Cambridge Programme for Industry. University of Cambridge. Department of Engineering. Trumpington Street. Cambridge. CB2 1PZ United Kingdom. ==================================================================== | Please send me full details of the course: NEURAL NETWORKS | | to be held at Pembroke College, Cambridge, England. | | 15 - 18 April 1991. COURSE FEE 875 pounds sterling. | | Accommodation can be arranged for delegates at Pembroke College. | | | | Name _______________________ Job Title ______________________ | | | | Company ____________________ Division _______________________ | | | | Address _____________________________________________________ | | | | _____________________________________________________ | | | | Postcode __________ Phone Number __________ Fax _____________ | ==================================================================== From fritz_dg%ncsd.dnet at gte.com Fri Dec 7 09:58:02 1990 From: fritz_dg%ncsd.dnet at gte.com (fritz_dg%ncsd.dnet@gte.com) Date: Fri, 7 Dec 90 09:58:02 -0500 Subject: requesting references on the list Message-ID: <9012071458.AA29639@bunny.gte.com> >> I'm interested to do comparative studies between Tabu search and >> annealing algorithm. Could someone kindly give me references on tabu >> search? Thanks. > I want to remind people that the CONNECTIONISTS list is not intended for > those who are too lazy to do their own library work. If you're going to > post a plea for basic information, please do it on comp.ai.neural-nets. > This list is intended for discussions among practicing researchers. I disagree. Extensive library work starting from scratch is basically a learning task for students who are still busy paying their dues (or professors writing monographs). Asking around for pointers into the literature is a time honored way of getting what you need without doing everything the hard way. Practicing Researchers are usually too busy producing to hit the library stacks with nothing but a ball-point pen in their hands every time there is a need to know. Also, some of us don't have convenient access to large libraries, or grad students to run there for us, and thus asking around can be crucial. Finally, it is often the case that if one person is interested in a topic, others are too. Any feedback from unexpected sources (eg. obscure, internally-circulated technical reports) would be a valuable, and valid, use of the list. D. Fritz fritz_dg%ncsd at gte.com From fozzard at boulder.Colorado.EDU Fri Dec 7 13:12:58 1990 From: fozzard at boulder.Colorado.EDU (Richard Fozzard) Date: Fri, 7 Dec 90 11:12:58 -0700 Subject: Tabu Search, and other stuff Message-ID: <9012071812.AA20992@alumni.colorado.edu> > I'm interested to do comparative studies between Tabu search and > annealing algorithm. Could someone kindly give me references on tabu > search? Thanks. I want to remind people that the CONNECTIONISTS list is not intended for those who are too lazy to do their own library work. If you're going to post a plea for basic information, please do it on comp.ai.neural-nets. This list is intended for discussions among practicing researchers. -- Dave Touretzky I think that this may be too strong a condemnation. I have done library search and at the same time posted to CONNECTIONISTS. I got quite a bit of information on works-in-progress, and other things I couldn't find in the library. To assume library indexes (indices?) are exaustive and up-to-date is rather unrealistic. That practicing researchers are on CONNECTIONISTS (and not on comp.ai.nn) is the very reason such requests are made. Of course, a busy practicing researcher is free not to respond to such requests. IMHO, Nirwan Ansari's 3-line message was not an abuse of bandwidth. He (she?) wasn't doing what we used to see a lot of, eg: "What's a good book on back propogation?", "Where can I find some neural net software?", etc. which I agree are inappropriate. If even Dave hasn't heard of this Tabu search, I wouldn't hold out much hope that anyone on comp.ai.nn has either. ======================================================================== Richard Fozzard "Serendipity empowers" Univ of Colorado/CIRES/NOAA R/E/FS 325 Broadway, Boulder, CO 80303 fozzard at boulder.colorado.edu (303)497-6011 or 444-3168 From koch%CITIAGO.BITNET at VMA.CC.CMU.EDU Fri Dec 7 14:05:14 1990 From: koch%CITIAGO.BITNET at VMA.CC.CMU.EDU (Christof Koch) Date: Fri, 7 Dec 90 11:05:14 PST Subject: Vehicle Guidance Workshop '91 Message-ID: <901207110314.20807159@Iago.Caltech.Edu> One-day workshop on ******************************************************* NEURAL AND FUZZY SYSTEMS, AND VEHICLE APPLICATIONS '91 ******************************************************* November 8, 1991, Tokyo, Japan ******************************************************* The Roundtable Discussion on "Neural and Fuzzy Systems, and Vehicle Applications" is tentatively scheduled for November 8, 1991, in Tokyo Japan. The focus of this roundtable discussion will be applications of neural nets and fuzzy logic to vehicles including automobiles, aircraft, and trains. The relationship between neural nets and fuzzy logic technologies will be another focus. Presentations of on-going projects as well as completed projects are welcome to stimulate the discussions. --------------------------------------------------------------- Please submit a one-page abstract by May 1, 1991, to Ichiro Masaki --------------------------------------------------------------- Related conferences include: IROS (International Workshop on Intelligent Robots and Systems) Nov. 3-5, Japan. IFES (International Fuzzy Engineering Syposium) Nov. 13-15, Japan. For further information, please contact: Ichiro Masaki Computer Science Department General Motors Research Laboratories 30500 Mound Road, Warren, Michigan 48090-9055, USA Office phone: 1-313-986-1466 Fax: 1-313-986-9356 E-Mail: MASAKI at GMR.COM If you're interested, please don't send mail to me but to Ichiro Masaki at masaki at gmr.com From inesc!lba at relay.EU.net Fri Dec 7 17:59:33 1990 From: inesc!lba at relay.EU.net (Luis Borges de Almeida) Date: Fri, 7 Dec 90 17:59:33 EST Subject: test Message-ID: <9012071759.AA11831@alf.inesc.pt> teste de uma nova mail-list; deitem fora, sff. Luis From kruschke at ucs.indiana.edu Fri Dec 7 15:48:00 1990 From: kruschke at ucs.indiana.edu (KRUSCHKE,JOHN,PSY) Date: 7 Dec 90 15:48:00 EST Subject: tech report: benefits of gain Message-ID: The following paper is available via ftp from the neuroprose archive at Ohio State (instructions for retrieval follow the abstract). This paper was witten more than two years ago, but we believe the ideas are still interesting even if the details are a bit dated. Benefits of Gain: Speeded learning and minimal hidden layers in back-propagation networks. John K. Kruschke Javier R. Movellan Indiana University Carnegie-Mellon University ABSTRACT The gain of a node in a connectionist network is a multiplicative constant that amplifies or attenuates the net input to the node. The objective of this article is to explore the benefits of adaptive gains in back propagation networks. First we show that gradient descent with respect to gain greatly increases learning speed by amplifying those directions in weight space that are successfully chosen by gradient descent on weights. Adpative gains also allow normalization of weight vectors without loss of computational capacity, and we suggest a simple modification of the learning rule that automatically achieves weight normalization. Finally, we describe a method for creating small hidden layers by making hidden node gains compete according to similarities between nodes, with the goal of improved generalization performance. Simulations show that this competition method is more effective than the special case of gain decay. To get a copy of the paper, do the following: unix> ftp cheops.cis.ohio-state.edu login: anonymous password: neuron ftp> cd pub/neuroprose ftp> binary ftp> get kruschke.gain.ps.Z ftp> bye unix> uncompress kruschke.gain.ps.Z unix> lpr kruschke.gain.ps From Dave.Touretzky at DST.BOLTZ.CS.CMU.EDU Sat Dec 8 00:34:33 1990 From: Dave.Touretzky at DST.BOLTZ.CS.CMU.EDU (Dave.Touretzky@DST.BOLTZ.CS.CMU.EDU) Date: Sat, 08 Dec 90 00:34:33 EST Subject: Tabu Search, and other stuff In-Reply-To: Your message of Fri, 07 Dec 90 11:12:58 -0700. <9012071812.AA20992@alumni.colorado.edu> Message-ID: <9767.660634473@DST.BOLTZ.CS.CMU.EDU> Okay, I was a little harder on Nirwan Ansari than I should have been. Sorry, Nirwan. But I see it's time to explain AGAIN what proper and improper uses of the CONNECTIONISTS list are. At the end of this message are some answers about Tabu search. It is always improper to ask questions on CONNECTIONISTS that you can answer for yourself with a few minutes of work, such as visiting the library or picking up a telephone. It took me a grand total of 15 *seconds* to find a reference to Tabu Search in our online library index. Most university libraries have access to some kind of on-line index, so there's no excuse for not looking there first. If one has absolutely no idea what Tabu search is, then one has no business bothering the CONNECTIONISTS readership with such an elementary question. Go to your library and DO THE WORK! Or pick up the phone and call the person who mentioned the term to you in the first place. Even though Rich Fozzard is correct that *someone* on this list is likely to have the answer to any technical question, this does not give people the right to waste the time of everyone on this list just to save themselves a tiny bit of work. Rich's comment that "busy researchers are free not to respond to such requests" is out of line, and misses the key point of CONNECTIONISTS: that busy researchers *will not be bothered* by such requests. People who refuse to understand this policy will be removed from the list. Notwithstanding the above, there *is* a proper way to ask for references on CONNECTIONISTS, and I agree with Rich that you can find things here that a library search won't turn up. But you should (a) give people something back in return for bothering them, and (b) demonstrate that you have already pursued the quick, obvious routes to finding the information you desire. The easiest way to do both these things is to FIRST do the library work to find the basic references, then POST these as part of your query. If this isn't concrete enough for some of you, here's an example: WRONG WAY: "Can someone please mail me all references to cascade correlation?" RIGHT WAY: "I'm looking for references to work on cascade correlation. I've already read Fahlman's paper in NIPS 2, and his NIPS 3 abstract, and found the code in the nn-bench archive. Is anyone aware of additional work with this algorithm? I'll summarize and post results to the list." Let's please not get into a long, boring thread about what constitutes appropriate postings to this list. If you feel you *must* express your opinion on this matter, do it via email to me alone. Most readers really don't want to hear about it. -- Dave ............ results of online index search on keyword "tabu": ............ Idnum 08784922 TYPE technical DATE 900900 AUTHOR Friden, C. and Hertz, A. and De Werra, D. TITLE Tabaris: an exact algorithm based on Tabu Search for finding a maximum independent set in a graph. (technical) SOURCE Computers & Operations Research v17 n5 p437(9) 1990 Sept SUBJECT Algorithm Analysis Theoretical Research New Technique Mathematical Models Algorithms Problem solving Graph theory Algorithms--analysis Problem solving--models Graph theory--research GRAPHICS table CAPTIONS Getting a maximum independent set in a graph. A general Tabu Search method. Algorithm STABULARGE for finding a large stable set. ABSTRACT The process of finding a maximum independent set in an arbitrary graph is examined; the problem is an ingredient of many coloring algorithms. An exact algorithm for constructing a maximum independent set in a graph is developed that is an implicit enumeration algorithm using Tabu Search techniques at some steps to obtain some bounds on the independence number of a subgraph of the graph to be examined. The general procedure is formalized, and the Tabu Search metaheuristic serving as an essential part of the enumeration procedure is described. From Dave.Touretzky at DST.BOLTZ.CS.CMU.EDU Sat Dec 8 00:40:20 1990 From: Dave.Touretzky at DST.BOLTZ.CS.CMU.EDU (Dave.Touretzky@DST.BOLTZ.CS.CMU.EDU) Date: Sat, 08 Dec 90 00:40:20 EST Subject: summer school proceedings: contents and ordering info Message-ID: <9773.660634820@DST.BOLTZ.CS.CMU.EDU> CONNECTIONIST MODELS: Proceedings of the 1990 Summer School Edited by David S. Touretzky (Carnegie Mellon University), Jeffrey L. Elman (University of California, San Diego), Terrence J. Sejnowski (The Salk Institute, UC San Diego), and Geoffrey E. Hinton (University of Toronto) ISBN 1-55860-156-2 $29.95 404 pages (For bibliographic purposes, the complete table of contents and contact numbers for additional information or for use in obtaining copies of this book follow the announcement.) TABLE OF CONTENTS PART I MEAN FIELD, BOLTZMANN, AND HOPFIELD NETWORKS Deterministic Boltzmann Learning in Networks with Asymmetric Connectivity 3 C.C. Galland and G.E. Hinton Contrastive Hebbian Learning in the Continuous Hopfield Model 10 J.R. Movellan Mean Field Networks that Learn to Discriminate Temporally Distorted Strings 18 C.K.I. Williams and G.E. Hinton Energy Minimization and the Satisfiability of Propositional Logic 23 G. Pinkas PART II REINFORCEMENT LEARNING On the Computational Economics of Reinforcement Learning 35 A.G. Barto and P.M. Todd Reinforcement Comparison 45 P. Dayan Learning Algorithms for Networks with Internal and External Feedback 52 J. Schmidhuber PART III GENETIC LEARNING Exploring Adaptive Agency I: Theory and Methods for Simulating the Evolution of Learning 65 G.F. Miller and P.M. Todd The Evolution of Learning: An Experiment in Genetic Connectionism 81 D.J. Chalmers Evolving Controls for Unstable Systems 91 A.P. Wieland PART IV TEMPORAL PROCESSING Back-Propagation, Weight Elimination and Time Series Prediction 105 A.S. Weigend, D.E. Rumelhart, and B.A. Huberman Predicting the Mackey-Glass Timeseries with Cascade-Correlation Learning 117 R.S. Crowder, III Learning in Recurrent Finite Difference Networks 124 F.S. Tsung Temporal Backpropagation: An Efficient Algorithm for Finite Impulse Response Neural Networks 131 E.A. Wan PART V THEORY AND ANALYSIS Optimal Dimensionality Reduction Using Hebbian Learning 141 A. Levin Basis-Function Trees for Approximation in High-Dimensional Spaces 145 T.D. Sanger Effects of Circuit Parameters on Convergence of Trinary Update Back-Propagation 152 R.L. Shimabukuro, P.A. Shoemaker, C.C. Guest, and M.J. Carlin Equivalence Proofs for Multi-Layer Perceptron Classifiers and the Bayesian Discriminant Function 159 J.B. Hampshire, II and B. Pearlmutter A Local Approach to Optimal Queries 173 D. Cohn PART VI MODULARITY A Modularization Scheme for Feedforward Networks 183 A. Ossen A Compositional Connectionist Architecture 188 J.R. Chen PART VII COGNITIVE MODELING AND SYMBOL PROCESSING From bates at crl.ucsd.edu Sun Dec 9 01:20:27 1990 From: bates at crl.ucsd.edu (Elizabeth Bates) Date: Sat, 8 Dec 90 22:20:27 PST Subject: Tabu Search, and other stuff Message-ID: <9012090620.AA11524@crl.ucsd.edu> It is clear that the connectionist list has gotten very large, and it now includes people at very different levels of expertise (including people like me who couldn't do a serious simulation if you put a gun to our heads, but still are interested in the basic ideas and the way that they are developing over a short period of time). I understand Touretzky's frustration with amateurs, and naive questions, but I thnk the community will be better off in the long run if you (we?) can tolerate a little more naivete, for a little longer. Important things are happening, and this medium is an important event in its own right. Let's stretch the limits a little longer, at the risk of putting up with a few more irritating messages in our daily logs than we might otherwise prefer. "in groups" abound. Generous and free-spirited exercises do not. -liz bates From eric at yin.nec.com Mon Dec 10 09:07:27 1990 From: eric at yin.nec.com (Eric B. Baum) Date: Mon, 10 Dec 90 09:07:27 EST Subject: No subject Message-ID: <9012101407.AA11222@yin.nec.com> I've been holding off replying re tabu because I'm no expert, but it appears that among connectionists few are, so here goes. Tabu search was invented by Fred Glover, Center for Applied Artificial Intelligence, Graduate School of Business, University of Colorado, Boulder. A thumbnail sketch of the idea (as I recollect it) is the following: We have a set of local search moves. So, if we're solving TSP, we might use the set of one link interchanges. At each step, we use the move in our set which produces the most optimum solution (e.g. shortest tour) with one proviso. We are not allowed to use a move which inverts one of our last x moves, where x is a small, heuristically chosen, fixed integer (e.g. 7). (i.e. each time we do a move , we remove the first element from a TABU set and insert the inverse of the current move as the x-th element in the TABU set). Note that, if we're near a local minimum for our search set, this procedure may force us to make a move which increases the tour length, since we must make some move at each step. We keep in storage the best solution yet found. We proceed till either a fixed number of iterations have been performed, or a fixed number have occured since last improvement in best value, halt and report the best value found. The general idea is that the TABU set prevents cycling, and otherwise one tries to take a naive most direct path over hills. This not only seems sensible, but is claimed to be extremely effective in many tests. One reference I have is: Glover F. "Tabu Search Methods in Artificial Intelligence and Operations Research" ORSA Artificial Intelligence Newsletter V1 No 2. 6 1987. Hopefully, somebody at Boulder reading this can encourage Prof Glover to post a more recent bibliography, since all I know about the subject comes from a talk I heard three years ago, and doubtless there have been improvements in the art. -- Eric Baum NEC Research Institute 4 Independence Way Princeton NJ 08540 Inet: eric at research.nj.nec.com UUCP: princeton!nec!eric MAIL: 4 Independence Way, Princeton NJ 08540 PHONE: (609) 951-2712 FAX: (609) 951-2482 From F_SIENKO at UNHH.UNH.EDU Mon Dec 10 13:07:22 1990 From: F_SIENKO at UNHH.UNH.EDU (F_SIENKO@UNHH.UNH.EDU) Date: Mon, 10 Dec 1990 13:07:22 EST Subject: Request for recurrent net learning code. Message-ID: <901210130722.20a0aacb@UNHH.UNH.EDU> My name is Fred Sienko, and I am a graduate student in electrical engineering at the University of New Hampshire. I am looking for a C code module which does real-time recurrent learning (Williams & Zipser) to use in a project I am currently working on. Does anyone have a copy available? From jesus!penrose at esosun.css.gov Mon Dec 10 22:57:34 1990 From: jesus!penrose at esosun.css.gov (Christopher Penrose) Date: Mon, 10 Dec 90 19:57:34 PST Subject: Tabu Search, and other stuff Message-ID: <9012110357.AA07004@ jesus > I want to remind people that the CONNECTIONISTS list is not intended for those who are too lazy to do their own library work. If you're going to post a plea for basic information, please do it on comp.ai.neural-nets. This list is intended for discussions among practicing researchers. Such an attitude is counter-productive. This statement also made me quite angry--it belittles the efforts to make the internet an interactive community that facilitates collective intellectual advancement. Christopher Penrose jesus!penrose From schmidhu at informatik.tu-muenchen.dbp.de Tue Dec 11 05:37:05 1990 From: schmidhu at informatik.tu-muenchen.dbp.de (Juergen Schmidhuber) Date: 11 Dec 90 11:37:05+0100 Subject: TRs Message-ID: <9012111037.AA22331@kiss.informatik.tu-muenchen.de> The revised and extended versions of two reports from February 1990 are available. 1. Networks adjusting networks. Technical Report FKI-125-90 (revised), Institut fuer Informatik, Technische Universitaet Muenchen, November 1990. 2. Making the world differentiable: On using fully recurrent self-supervised neural networks for dynamic reinforcement learning and planning in non-stationary environments. Technical Report FKI-126-90 (revised), Institut fuer Informatik, Technische Universitaet Muenchen, November 1990. To obtain hardcopies, send email to marquard at tumult.informatik.tu-muenchen.de Please let your message look like this: subject:FKI physical address (not more than 33 characters per line) Those who requested copies at NIPS should not send additional requests. Juergen Schmidhuber From nelsonde%avlab.dnet at wrdc.af.mil Tue Dec 11 10:09:03 1990 From: nelsonde%avlab.dnet at wrdc.af.mil (nelsonde%avlab.dnet@wrdc.af.mil) Date: Tue, 11 Dec 90 10:09:03 EST Subject: Constructive/Destructive NN Algorithms Message-ID: <9012111509.AA04225@wrdc.af.mil> I N T E R O F F I C E M E M O R A N D U M Date: 11-Dec-1990 10:27am EST From: Dale E. Nelson NELSONDE Dept: AAAT-1 Tel No: 57646 TO: Remote Addressee ( _LABDDN::"CONNECTIONISTS at CS.CMU.EDU" ) TO: Dennis S. Buck ( BUCKDS ) Subject: Constructive/Destructive NN Algorithms First, let me apologize for the formal formatting of this, but my access is through All-In-One and that is the way it comes out. I know from the NIPS Workshop, that there is intense interest in networks that develop their own topology during the training. I personally have had problems when refering to this class of algorithms. If they are called evolutionary networks, pepole say that you are using genetic algorithms. Well, that *could* be true, but that is too limiting. I would like to propose, for discussion, a possible name for this class of networks. This would give all researchers a single keyword which would facilitate searches and other research efforts. The term for these networks, which I would like to propose, is Ontogenic Neural Networks. This is based on the word ontogeny which is defined as: The development or developmental history of an individual organism. Ontogenic is the adjective form of the word. I hope that we can generate a dialogue and perhaps come to agreement on a terminology for this class of networks. What do you think??? Dale E. Nelson nelsonde%avlab.dnet at wrdc.af.mil From DBEDFORD%VAX.OXFORD.AC.UK at bitnet.CC.CMU.EDU Tue Dec 11 11:43:17 1990 From: DBEDFORD%VAX.OXFORD.AC.UK at bitnet.CC.CMU.EDU (DBEDFORD%VAX.OXFORD.AC.UK@bitnet.CC.CMU.EDU) Date: Tue, 11 DEC 90 16:43:17 GMT Subject: No subject Message-ID: <73F7C753C0E00F6C@BITNET.CC.CMU.EDU> Please add my name to your mailing list. Thanks , Dhugal Bedford ,University of Oxford,UK. DBEDFORD at UK.AC.OX.VAX From Scott.Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU Tue Dec 11 20:22:49 1990 From: Scott.Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF-PMAX.SLISP.CS.CMU.EDU) Date: Tue, 11 Dec 90 20:22:49 EST Subject: Constructive/Destructive NN Algorithms In-Reply-To: Your message of Tue, 11 Dec 90 10:09:03 -0500. <9012111509.AA04225@wrdc.af.mil> Message-ID: I know from the NIPS Workshop, that there is intense interest in networks that develop their own topology during the training. I personally have had problems when refering to this class of algorithms... The term for these networks, which I would like to propose, is Ontogenic Neural Networks. Clever, but I just can't see this term catching on. What's wrong with "constructive" and "destructive" (or maybe "additive" and "subtractive")? People immediately know what you're talking about. I don't think it's a big problem that there isn't a single word for the whole class. Usually you only want to refer to one kind or the other. Only leaders of workshops and (I hope!) funding agencies have any need to come up with one term that covers the whole spectrum of such approaches. -- Scott From koch%CITIAGO.BITNET at VMA.CC.CMU.EDU Wed Dec 12 00:32:52 1990 From: koch%CITIAGO.BITNET at VMA.CC.CMU.EDU (Christof Koch) Date: Tue, 11 Dec 90 21:32:52 PST Subject: Caltech's CNS Program Message-ID: <901211213209.20807118@Iago.Caltech.Edu> This is a short description of our CNS program. Deadline for application is end of January. Christof ******************************************************* CALIFORNIA INSTITUTE OF TECHNOLOGY ******************************************************* Computation and Neural Systems Program This interdepartmental program awards a Ph.D. in Computation and Neural Systems. No Master is awarded. Current enrollment: 28 doctoral, 18 postdoctoral Financial support: Complete support for tuition and stipend from graduate research assistantships, graduate teaching assistantships, NIH training grant, and private sources. Contact: J. Hopfield, Ph.D., Program Head, 160-30 (818) 356-2808 J. Bower, Ph.D., Chairman of Admissions, Biology Div., 216-76, (818) 356-6817 jbower at smaug.cns.caltech.edu All at California Institute of Technology, Pasadena, CA 91125 Caltech's graduate program in Computation and Neural Systems presently involves 16 faculty in the Division of Biology, Engineering and Applied Science, and Physics. This interdisciplinary program is centered on computation approaches to the study of biological and artificial information processing systems. A multidisciplinary curriculum offers training in four general areas: neurobiology; computer science and collective computation; physical computational devices; and mathematics and modeling. Students need to take courses in each of these areas in addition to an experimental laboratory course in neurobiology. The breadth of training is enhanced by close interactions among students and faculty from all parts of the program. A central focus is provided by weekly seminars, informal lunch talks, and a computer simulation laboratory open to students. Students are assigned to a research laboratory upon arrival, but have the option of rotating through several laboratories before choosing a thesis advisor. Research interests of the faculty include the collective properties and computational capacities of complex artificial and biological networks, analog VLSI devices, optical devices, and highly parallel digital computers. Neurobiological simulation approaches include modeling at the systems level (e.g., olfactory cortex, cerebellar cortex, and visual and auditory cortices) and at the cellular level (e.g., biophysical and developmental mechanisms). Computational approaches to artificial systems span a wide range, from studies of associative memory and analog networks for sensory processing to graphical image representation and the theory of computation. Interested students are encouraged to combine theoretical or modeling approaches with physiological or anatomical research on biological systems. Core faculty: Yaser Abu-Mostafa, John Allman, Alan Barr, James Bower, Rodney Goodman, John Hopfield, Bela Julesz, Christof Koch, Masakazu Konishi, Gilles Laurent, Henry Lester, Carver Mead, Jerome Pine, Edward Posner, Demitri Psaltis, David van Essen. Selection of ourses: CNS 124 : Pattern Recognition (two quarters) Covers classic results from pattern recognition and discusses in this context associative memories and related neural network models of computation. Given by D. Psaltis. CNS 174 : Computer Graphics Laboratory (three quarters) The art of making pictures by computer. Given by A. H. Barr. CNS 182 : Analog Integrated Circuit Design (three quarters) Device, circuit, and system techniques for designing large-scale CMOS analog systems. Given by C. A. Mead. CNS 184 : Analog Integrated Circuit Projects Laboratory (three quarters) Design projects in large-scale analog integrated systems. Given by C. A. Mead. CNS 185 : Collective Computation (one quarter) Neural network theory and applications. Given by J. J. Hopfield. CNS 186 : Vision: From Computational Theory to Neuronal Mechanisms (one quarter) Lecture and discussion course aimed at understanding visual information processing in both biological and artificial systems. Given by C. Koch and D. C. Van Essen. CNS 221 : Computational Neurobiology (one quarter) Lecture, discussion and laboratory aimed at understanding computational aspects of information processing within the nervous system. Given by J. Bower and C. Koch. CNS 256 : Methods of Multineural Recording (one quarter) Reading and discussion course. Topics included span a range of multineural recording techniques from multielectrode recording to positron emission tomography. Given by J. Pine. Student personal description ( H. H. Suarez, fourth year graduate student; hhs at aurel.caltech.edu): According to my experience, this program's emphasis really spans a wide range, but two areas stand out especially for me: modelling biological systems in a very detailed fashion and building artificial sensory-motor systems (analog VLSI - based systems) whose design is strongly influenced by knowledge of the corresponding biological system. The overall ambiance from a student's point of view is very good, due to the personal qualities of the faculty and the students. There is a fair amount of interaction among the researchers in the program, and on the average two or three talks a week on CNS-related topics, often from researchers outside Caltech. Thus there is little chance of getting bored ... From bogner at augean.ua.oz.au Wed Dec 12 18:26:58 1990 From: bogner at augean.ua.oz.au (bogner@augean.ua.oz.au) Date: Wed, 12 Dec 90 17:26:58 CST Subject: Postdoc needed Message-ID: <9012120657.1873@munnari.oz.au> I am needing a post-doc on neural net work as soon as possible. This is not an official advertisement, but I'd like to hear from anyone suitably experienced and available in Jan 1991 for a year. Work relates to some of: invariance, data fusion, preprocess selection. Robert E. Bogner Professor of Electrical Engineering University of Adelaide South Australia From shen at iro.umontreal.ca Wed Dec 12 18:17:19 1990 From: shen at iro.umontreal.ca (Yu Shen) Date: Wed, 12 Dec 90 18:17:19 EST Subject: reference list on parallel GA/SA Message-ID: <9012122317.AA12988@chukota.IRO.UMontreal.CA> Here is the list of reference on parallel genetic algorithm and parallel simulated annealing, compiled from the kind resposes from people on the mailing list and other whatever ways I ran into. I thank all the help again! I have done a report on the impact of parallelism on genetic algorithm and simulated annealing. Here are some of the major points: Assuming constant size of neighborhood in a spatially distributed population, the parallel genectic algorithm that selects mating partner locally can usually achieve linear speedup over the sequential version. But PGA has mainly shown advantage in improving the quality of search. For parallel paradigm increases the chance of crossing-over of gens, which is believed to be the most important mechanism of optimization. Even with linear speedup, PGA application is still very slow comparing with some conventional heuristics, for example, for the case of Traveling Salesman Problem. Parallel simulated annealing is not equivalent to parallelization of Metropolis' relaxation, where the mainstream of practices concentrate on. Metropolis' is basically a sequential approach. More significant speedup is usally achieved by alternatives away from it. It is possible to devise more parallel paradigm oriented simulation techniques, because the working mechanism of simulated annealing is Boltzman distribution, instead of the particular generating method--- Metropolis' relaxation. The report is being revised. It will be available to interested parties in January. The reference consists of two parts. The first is on genectic algorithm, the second on simulated annealing. Very godd bibliography on SA of D. Greening is refered to the original source to save space. By the way, I have profited a lot by asking question which seems simple to some BIG profs B-]. I earnestly want to make up the noise I made to them. I therefore suggest to the fellow pratictioner-to-be's let's try to make the noise as little as possible. For example, we can have the subject line as concise as possible, so non-interested people can delete it beofor seeing it. As a example, when we ask such "simple" question, we can put a X: in front of the subject line. X: may mean that it might annoy some people but it might not to some others. eg. X: Relationship between the cinema and the connectionist mailing list 8-) Yu Shen PhD Student Dept. d'Informatique et Recherche Operationnelle University de Montreal C.P. 6128 Succ. A. Montreal, Que. Canada H3C 3J7 (514) 342-7089 (H) shen.iro.umontreal.ca ---------------------PGA---------------------------------------------------- @TechReport{Goldberg90:Boltzman-Tournament, author = "David E. Goldberg", title = "A Note on Boltzmann Tournament Selection for Genetic Algorithms and Population-oriented Simulated Annealing", institution = "University of Alabama", year = "1990", OPTtype = "", OPTnumber = "90003", OPTaddress = "Tuscaloosa, AL 35487", OPTmonth = "May", OPTnote = "distribution across population" } @TechReport{ga:Goldberg90:messy, author = "David E, Goldberg", title = "An Investigation of Messy Genetic Algorithms", institution = "Dept. of Engineering Mechanics, Univ. of Alabama", year = "1990", OPTtype = "", OPTnumber = "90005", OPTaddress = "", OPTmonth = "May", OPTnote = "" } @Article{ga:Goldberg89:messy, author = "David E. Goldberg", title = "Messy Genetic Algorithms: Motivation, Analysis, and First Results", journal = "Complex Systems", year = "1989", OPTvolume = "", OPTnumber = "3", OPTpages = "493-530", OPTmonth = "", OPTnote = "" } ------------ @TechReport{ga:Goldberg90:selection, author = "David E. Goldberg", title = "A Comparative Analysis of Selection Schemes Used in Genetic Algorithms", institution = "Dept. of Engineering Mechanics, Univ. of Alabama", year = "1990", OPTtype = "", OPTnumber = "90007", OPTaddress = "", OPTmonth = "July", OPTnote = "" } ------------- From rupen at cvax.cs.uwm.edu Wed Dec 12 19:48:58 1990 From: rupen at cvax.cs.uwm.edu (Rupen Sheth) Date: Wed, 12 Dec 90 19:48:58 CDT Subject: Iris plat classification using backprop Message-ID: <9012130149.AA08037@cvax.cs.uwm.edu> I am trying to solve the Iris plant classification problem using standard backprop with a 4-4-2 (input-hidden-output) configuration. I plan on trying different configurations and varying some parameters to evaluate performance. What other configurations have other people used for this problem. Any results and comments are welcome. Thank you. Please send email to rupen at cvax.cs.uwm.edu OR rupen at gemed.ge.com From jose at learning.siemens.com Thu Dec 13 08:16:13 1990 From: jose at learning.siemens.com (Steve Hanson) Date: Thu, 13 Dec 90 08:16:13 EST Subject: Tabu Search, and other stuff Message-ID: <9012131316.AA04980@learning.siemens.com.siemens.com> Hey guys, CMU runs this thing out of the goodness of their little hearts. And as far as I know recieves no remuneration for machines, administration, or headaches that arise from the disparate personalities which are attracted to this medium, I think we should attempt to minimize whining and get on with it. Steve Hanson From shen at iro.umontreal.ca Thu Dec 13 10:23:50 1990 From: shen at iro.umontreal.ca (Yu Shen) Date: Thu, 13 Dec 90 10:23:50 EST Subject: Reference list on PGA more Message-ID: <9012131523.AA13992@chukota.IRO.UMontreal.CA> @Article{Liepins89, author = "G.E. Liepins and M.R. Hilliard", title = "GENETIC ALGORITHMS: FOUNDATIONS AND APPLICATIONS", journal = "Annals of Operations Research", year = "1989", OPTvolume = "21", OPTnumber = "", OPTpages = "31-58", OPTmonth = "", OPTnote = "see more reference there" } @InProceedings{Gorges-Schleuter89, author = "{M. G\"orges-Schleuter}", title = "Asparagos: An asynchronous parallel genetic optimization strategy", booktitle = "3rd Int. Conf. on Genetic Algorithms", year = "1989", OPTeditor = "H. Schaffer", OPTpages = "", OPTorganization = "", OPTpublisher = "Morgan-Kaufmann", OPTaddress = "", OPTmonth = "", OPTnote = "" } @InProceedings{Petty87, author = "C.B. Pettey, M.R. Leuze and J.H. Grefenstette", title = "A parallel genetic algorithm", booktitle = "Genetic Algorithms and Their Applications: Proc. Second Int. Conf. on Genetic Algorithms", year = "1987", OPTeditor = "Grefenstette", OPTpages = "155-161", OPTorganization = "", OPTpublisher = "", OPTaddress = "", OPTmonth = "", OPTnote = "" } @InProceedings{Robertson88, author = "G. G. Robertson", title = "Population size in a classifier system", booktitle = "Proc. Fifth Int. Conf. on Machine Learning ", year = "1988", OPTeditor = "", OPTpages = "142-152", OPTorganization = "", OPTpublisher = "", OPTaddress = "", OPTmonth = "", OPTnote = "" } @InProceedings{Robertson87, author = "G.G. Robertson", title = "Parallel implementation of genetic algorithms in a classifier system", booktitle = "Genetic Algorithms and Simulated Annealing", year = "1987", OPTeditor = "Lawrence Davis", OPTpages = "129-140", OPTorganization = "", OPTpublisher = "", OPTaddress = "", OPTmonth = "", OPTnote = "" } @Book{Davis87, author = "Lawrence Davis", title = "Genetic Algorithms and Simulated Annealing", publisher = "Pitman, London", year = "1987", OPTeditor = "", OPTvolume = "", OPTseries = "", OPTaddress = "", OPTedition = "", OPTmonth = "", OPTnote = "the first introduction to me, covers ga most" } @InProceedings{Sannier87, author = "A.V. Sannier II and E.D. Goodman", title = "Genetic learning procedure in distributed environments", booktitle = "Genetic Algorithms and Their Applications: Proc. Second Int. Conf. on Genetic Algorithms", year = "1987", OPTeditor = "Grefenstette", OPTpages = "162-169", OPTorganization = "", OPTpublisher = "", OPTaddress = "", OPTmonth = "", OPTnote = "" } @InProceedings{Jog87, author = "P.Jog and D.Van Gucht", title = "Parallerization of probabilistic sequential search algorithms", booktitle = "Genetic Algorithms and Their Applications: Proc. Second Int. Conf. on Genetic Algorithms", year = "1987", OPTeditor = "Grefenstette", OPTpages = "170-176", OPTorganization = "", OPTpublisher = "", OPTaddress = "", OPTmonth = "", OPTnote = "" } @InCollection{Wilson87:classifier, author = "Stewart W. Wilson", title = "Hierachical Credit Allocation in a Classifier System", booktitle = "Gentic Algorithms and Simulated Annealing", publisher = "Pitman, London", year = "1987", OPTeditor = "Lawrence Davis", OPTchapter = "8", OPTpages = "104-115", OPTaddress = "", OPTmonth = "", OPTnote = "" } From frederic at cs.unc.edu Thu Dec 13 10:37:58 1990 From: frederic at cs.unc.edu (Robin Fredericksen) Date: Thu, 13 Dec 90 10:37:58 -0500 Subject: No subject Message-ID: <9012131537.AA05285@surya.cs.unc.edu> >>I know from ...in networks that >>develop .....training. I personally have had problems >>when refering to this class of algorithms... >>The term for ...would like to propose, is Ontogenic Neural Networks. >Clever, but I just can't see this term catching on. What's wrong with >"constructive" and "destructive" (or maybe "additive" and "subtractive")? >People immediately know what you're talking about. > >I don't think it's a big problem that there isn't a single word for the >whole class. Usually you only want to refer to one kind or the other. >Only leaders of workshops and (I hope!) funding agencies have any need to >come up with one term that covers the whole spectrum of such approaches. > >-- Scott It's more than just clever. It is precise, and by that I mean: pre.cise \pri-'si-s\ aj [MF precis, fr. L praecisus, pp. of praecidere to cut off, fr]. prae- + caedere to cut - more at CONCISE 1: exactly or sharply defined or stated 2: minutely exact 3: strictly conforming to rule or convention 4: distinguished from every other : VERY {at just that ~ moment} - pre.cise.ly av During ontogeny, the CNS of a an organism experiences (uses) both cell death (destruction) and synaptogenesis (construction) to arrive at its final form. IMHO, separation of the two is artificial and incorrect. They go hand in hand in biological systems (although the periods during which they occur are not exactly coincident, but overlap) so why should they not go hand in hand in our models? The situation is not one of a 'spectrum of approaches', but one of duality. Ignoring one and studying the other provides only half of the story, and what good is half of a story? For a nice overview, try: AUTHOR: Purves, Dale. TITLE : Body and brain : a trophic theory of neural connections / IMPR : Cambridge, Mass. : Harvard University Press, 1988. I don't see why we should ignore terminology created/used in another field. This is an interdisciplinary area; if everyone reinvents the wheel then what use is collaboration? --Eric From tsejnowski at UCSD.EDU Thu Dec 13 13:54:27 1990 From: tsejnowski at UCSD.EDU (Terry Sejnowski) Date: Thu, 13 Dec 90 10:54:27 PST Subject: Negative feedback loops Message-ID: <9012131854.AA07310@sdbio2.UCSD.EDU> There is an inherent instability in the list that is caused by time delays that can be over 3 days for some sites. Negative feedback loops with delays are well known to be prone to wild oscillations. A posting that elicits strong responses will cause an impulse response that can last for a week because remote sites are not aware that the posting has already been through several cycles of responses from others. Before you make an obvious reply to a posting, please look at the date of the posting. If it was posted three or more days earlier the chances are that someone has already said what you are about to say. Terry ----- From Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU Thu Dec 13 16:47:03 1990 From: Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF1.SLISP.CS.CMU.EDU) Date: Thu, 13 Dec 90 16:47:03 EST Subject: No subject In-Reply-To: Your message of Thu, 13 Dec 90 10:37:58 -0500. <9012131537.AA05285@surya.cs.unc.edu> Message-ID: Sure, there are many interesting network architectures that add and subtract network structure at the same time. Most people at the post-NIPS workshop on this topic (including me) seemed to feel that these hybrid approaches were the most promising of all. The real problem I have with "Ontogenic" is that the term is so closely associated in most people's minds with biological development. Many people will assume that an "Ontogenic Neural Network" is a serious attempt to model the embryonic development of some real biological nervous system. That may happen some day soon, and we probably want to save the term "Ontogenic" for such applications, rather than co-opting it to refer to any old net that messes around with its own topology during learning. [Beware! Attempted humor follows:] I wonder if ontogenic neural nets would, in the course of learning, recapitulate the phylogeny of neural nets. You start with a simple perceptron -- two layers of cells -- which then begins growing a hidden layer. Unfortunately, the hidden layer is not functional. Some symbolic antibodies then attack the embryo and try to kill it off by leeching off all the nutrients, but a few isolated cells remain. The cells regroup, but very loosely. One part buds off, names itself "Art", and develops an elaborate, cryptic language of its own. The rest of the blob turns into a Hopfield net, heats up and cools down a few times, and finally develops the organs necessary for back-propagation. We don't know what happens after that because the back-propagation phase is so slow that it hasn't converged yet... -- Scott From singh at envy.cs.umass.edu Thu Dec 13 18:35:38 1990 From: singh at envy.cs.umass.edu (singh@envy.cs.umass.edu) Date: Thu, 13 Dec 90 18:35:38 EST Subject: tech report: benefits of gain In-Reply-To: "KRUSCHKE,JOHN,PSY"'s message of 7 Dec 90 15:48:00 EST <9012081955.AA22023@unix1.CS.UMASS.EDU> Message-ID: <9012132335.AA00627@gluttony.cs.umass.edu> From INS_ATGE%JHUVMS.BITNET at VMA.CC.CMU.EDU Thu Dec 13 21:44:00 1990 From: INS_ATGE%JHUVMS.BITNET at VMA.CC.CMU.EDU (INS_ATGE%JHUVMS.BITNET@VMA.CC.CMU.EDU) Date: Thu, 13 Dec 90 21:44 EST Subject: "constructive" vs. "compositional" learning Message-ID: Something to keep in mind when describing "constructive", "destructive", or "ontogenic" networks is the nature of learning. In traditional gradient-descent learning of fixed networks, the learning algorithm finds the minimum (or minima) of a fixed energy landscape. In these "constructive" or "destructive" networks, learning algorithms develop an energy landscape specifically designed to allow gradient-descent methods to best solve the problem (or at least that is what _should_ be happening). "Compositional Learning" (as used by J. Schmidhuber), is a method in which useful sub-goals are developed and utilized to solve a larger goal. (By stringing together these sub-goals). In methods such as Cascade-Correlation, new hidden units are added which serve to change the error landscape so as to best allow gradient-descent methods to find energy minima. But examining Cascade-Correlation in another light, we can say it is developing feature detectors which represent useful subgoals. Proper connection of these useful subgoals together allow us to reach our final goal (error minimization). -Thomas Edwards From jagota at cs.Buffalo.EDU Thu Dec 13 19:28:20 1990 From: jagota at cs.Buffalo.EDU (Arun Jagota) Date: Thu, 13 Dec 90 19:28:20 EST Subject: Paper Announcement Message-ID: <9012140028.AA23382@sybil.cs.Buffalo.EDU> *************** DO NOT FORWARD TO OTHER BBOARDS***************** [Note : Please do not reply with 'r' or 'R' to this message] The following paper, submitted to a special issue of IJPRAI, is now available : It describes a substantial extension of work presented at IJCNN-90, San Diego Degraded Printed Word Recognition with a Hopfield-style Network Arun Jagota (jagota at cs.buffalo.edu) Department of Computer Science State University Of New York At Buffalo ABSTRACT In this paper, the Hopfield-style network, a variant of the discrete Hopfield network, is applied to (degraded) machine printed word recognition. It is seen that the representation and dynamics properties of this network map very well to this problem. Words to be recognised are stored as content- addressable memories. Word images are first processed by a hardware OCR. The network is then used to postprocess the OCR decisions. It is shown (on postal word images) that for a small stored dictionary (~500 words), the network exact recall performance is quite good, for a large (~10,500 words) dictionary, it deteriorates dramatically, but the network still performs very well at "filtering the OCR output". The benefit of using the network for "filtering" is demonstrated by showing that a specific distance based search rule, on a dictionary of 10,500 words, gives much better word recognition performance (71% TOP choice, 84% TOP 2.6) on the network (filtered) output than on the raw OCR output. It is also shown, that for such "filtering", a special case of the network with two-valued weights performs almost as well as the general case, which verifies that the essential processing capabilities of the network are captured by the graph underlying it, the specific values of the +ve weights being relatively unimportant. This might also have implications for low prec- ision implementation. The best time efficiency is found when the dictionary of ~10,500 words is stored and the network is used to "filter" OCR output for 266 images. The training + filtering, together, take only 2 watch-timed minutes on a SUN Sparc Station. ------------------------------------------------------------------------ (Raw) FootNote: This problem seems cumbersome if viewed as one of supervised function learning (feed-forward NNs, Bayesian) due to large number (~10,500) of classes (=> large training sets/times). Conventional treatment is dictionary storage + search problem but drawback of sequential search is large search time during testing. The Hopfield-style network can be viewed as particular form of (unsupervised) distributed dictionary storage + search-by-recurrent-dynamics. Search is rapid and almost independent of dictionary size. The catch is that functional performance deteriorates rapidly with dictionary size. All is not lost, however, because partial performance (filtering) remains very good. [[Comments on above welcome but please mail to (jagota at cs.buffalo.edu) directly, not to CONNECTIONISTS]] The paper is available in compressed PostScript form by anonymous ftp unix> ftp cheops.cis.ohio-state.edu (or, ftp 128.146.8.62) Name: anonymous Password: neuron ftp> cd pub/neuroprose ftp> binary ftp> get jagota.wordrec.ps.Z ftp> quit unix> uncompress jagota.wordrec.ps.Z unix> lpr jagota.wordrec.ps ------------------------------------------------------------------------ Previous Postscript incompatibility problems have, by initial assessment, been corrected. Nevertheless, the paper is also available by e-mail (LaTeX sources) or surface mail (in that prefered order). Arun Jagota Dept Of Computer Science jagota at cs.buffalo.edu 226 Bell Hall, State University Of New York At Buffalo, NY - 14260 *************** DO NOT FORWARD TO OTHER BBOARDS***************** From Dave.Touretzky at DST.BOLTZ.CS.CMU.EDU Fri Dec 14 01:18:23 1990 From: Dave.Touretzky at DST.BOLTZ.CS.CMU.EDU (Dave.Touretzky@DST.BOLTZ.CS.CMU.EDU) Date: Fri, 14 Dec 90 01:18:23 EST Subject: for NIPS authors Message-ID: I apologize for bothering the entire list with this, but it's the only way to quickly reach the 130+ NIPS authors. Morgan Kaufmann has issued some corrections to the LaTeX formatting macros for NIPS papers; a particularly critical change is the value of \textheight parameter. If you FTP'ed the macros from B.GP.CS.CMU.EDU before December 13, please FTP them again. If you retrieved the macros on or after December 13, you already have the latest version. If you haven't received your author kit by Monday, call Sharon Montooth at Morgan Kaufmann; her number is 415-578-9911. Reminder: your final camera-ready copy is due at the publisher by January 18. The volume will be out in April. -- Dave Touretzky From LAUTRUP at nbivax.nbi.dk Fri Dec 14 03:44:00 1990 From: LAUTRUP at nbivax.nbi.dk (Benny Lautrup) Date: Fri, 14 Dec 90 09:44 +0100 (NBI, Copenhagen) Subject: IJNS issue number 4 Message-ID: Begin Message: ----------------------------------------------------------------------- INTERNATIONAL JOURNAL OF NEURAL SYSTEMS The International Journal of Neural Systems is a quarterly journal which covers information processing in natural and artificial neural systems. It publishes original contributions on all aspects of this broad subject which involves physics, biology, psychology, computer science and engineering. Contributions include research papers, reviews and short communications. The journal presents a fresh undogmatic attitude towards this multidisciplinary field with the aim to be a forum for novel ideas and improved understanding of collective and cooperative phenomena with computational capabilities. ISSN: 0129-0657 (IJNS) ---------------------------------- Contents of issue number 4 (1990): 1. A. M. Gutman: Bistability of Dendrites. 2. J. J. Atick and A. N. Redlich: Prediction Ganglion and Simple Cell Receptive Field Organisations. 3. H. H. Thodberg: Improving Generalisation of Neural Networks through Pruning. 4. O. Hendin, D. Horn and M. Usher: Chaotic Behaviour of a Neural Network with Dynamical Thresholds. 5. C. Myers: Learning with Delayed Reinforcement through Attention-Driven Buffering. 6. R. Erichson and W. K. Theumann: Mixture States and Storage with correlated Patterns in Hopfield's Model. 7. H. Shouval, I. Shariv, T. Grossman, A. A. Friesem, E. Domany: An all-optical Hopfield Network: Theory and Experiment. 8. Yves Chauvin: Gradient Descent to Global Minima in a n-dimensional Landscape. ---------------------------------- Editorial board: B. Lautrup (Niels Bohr Institute, Denmark) (Editor-in-charge) S. Brunak (Technical Univ. of Denmark) (Assistant Editor-in-Charge) D. Stork (Stanford) (Book review editor) Associate editors: B. Baird (Berkeley) D. Ballard (University of Rochester) E. Baum (NEC Research Institute) S. Bjornsson (University of Iceland) J. M. Bower (CalTech) S. S. Chen (University of North Carolina) R. Eckmiller (University of Dusseldorf) J. L. Elman (University of California, San Diego) M. V. Feigelman (Landau Institute for Theoretical Physics) F. Fogelman-Soulie (Paris) K. Fukushima (Osaka University) A. Gjedde (Montreal Neurological Institute) S. Grillner (Nobel Institute for Neurophysiology, Stockholm) T. Gulliksen (University of Oslo) D. Hammerstrom (Oregon Graduate Institute) J. Hounsgaard (University of Copenhagen) B. A. Huberman (XEROX PARC) L. B. Ioffe (Landau Institute for Theoretical Physics) P. I. M. Johannesma (Katholieke Univ. Nijmegen) M. Jordan (MIT) G. Josin (Neural Systems Inc.) I. Kanter (Princeton University) J. H. Kaas (Vanderbilt University) A. Lansner (Royal Institute of Technology, Stockholm) A. Lapedes (Los Alamos) B. McWhinney (Carnegie-Mellon University) M. Mezard (Ecole Normale Superieure, Paris) J. Moody (Yale, USA) A. F. Murray (University of Edinburgh) J. P. Nadal (Ecole Normale Superieure, Paris) E. Oja (Lappeenranta University of Technology, Finland) N. Parga (Centro Atomico Bariloche, Argentina) S. Patarnello (IBM ECSEC, Italy) P. Peretto (Centre d'Etudes Nucleaires de Grenoble) C. Peterson (University of Lund) K. Plunkett (University of Aarhus) S. A. Solla (AT&T Bell Labs) M. A. Virasoro (University of Rome) D. J. Wallace (University of Edinburgh) D. Zipser (University of California, San Diego) ---------------------------------- CALL FOR PAPERS Original contributions consistent with the scope of the journal are welcome. Complete instructions as well as sample copies and subscription information are available from The Editorial Secretariat, IJNS World Scientific Publishing Co. Pte. Ltd. 73, Lynton Mead, Totteridge London N20 8DH ENGLAND Telephone: (44)81-446-2461 or World Scientific Publishing Co. Inc. 687 Hardwell St. Teaneck New Jersey 07666 USA Telephone: (1)201-837-8858 or World Scientific Publishing Co. Pte. Ltd. Farrer Road, P. O. Box 128 SINGAPORE 9128 Telephone (65)382-5663 ----------------------------------------------------------------------- End Message From nelsonde%avlab.dnet at wrdc.af.mil Fri Dec 14 08:30:19 1990 From: nelsonde%avlab.dnet at wrdc.af.mil (nelsonde%avlab.dnet@wrdc.af.mil) Date: Fri, 14 Dec 90 08:30:19 EST Subject: Constructive/Destructive Algorithms (Ontogenic Networks) Message-ID: <9012141330.AA07533@wrdc.af.mil> I N T E R O F F I C E M E M O R A N D U M Date: 14-Dec-1990 08:47am EST From: Dale E. Nelson NELSONDE Dept: AAAT-1 Tel No: 57646 TO: Remote Addressee ( _LABDDN::"CONNECTIONISTS at CS.CMU.EDU" ) Subject: Constructive/Destructive Algorithms (Ontogenic Networks) > Clever, but I just can't see this term catching on. What's wrong with > "constructive" and "destructive" (or maybe "additive" and "subtractive")? > People immediately know what you're talking about. > I don't think it's a big problem that there isn't a single word for the > whole class. Usually you only want to refer to one kind or the other. If this is a valid argument, then why in the medical community do they call a class of diseases Cancer? Why not just refer to it as rapid, uncontrolled cell division? Why do we call them "neural networks"? Why not just say "inputs that are multiplied by weights, summed up, put through a squashing function and the result passed as input to another summer/ squasher" ???? It is because the vocabulary, as agreed to by researchers, makes the interchange of ideas easier. I believe that we need to develop the terminology and vocabulary for our research area to facilitate literature searches, and just free discussion. I have a deaf man that works for me. Our main problem is that there is *no* sign language vocabulary associated with neural networks. We have to develop it in order to effectively exchange ideas. I am open to any other suggestions. --Dale From gluck%psych at Forsythe.Stanford.EDU Fri Dec 14 12:02:01 1990 From: gluck%psych at Forsythe.Stanford.EDU (Mark Gluck) Date: Fri, 14 Dec 90 09:02:01 PST Subject: Postdoc: Cognitive Science / Neural Modeling Message-ID: <9012141702.AA19121@psych> Postdoctoral Positions in: -------------------------- COGNITIVE & NEURAL BASES OF LEARNING at Rutgers University Center for Molecular & Behavioral Neuroscience 195 University Avenue Newark, NJ 07102 Postdoctoral positions are available for recent Ph.D's in all areas of Cognitive Science (e.g., Neuroscience, Psychology, Computer Science) interested in pursuing research in the following areas of learning theory: 1. COGNITIVE SCIENCE/ADAPTIVE "CONNECTIONIST" NETWORKS: Experimental and theoretical (computational) studies of human learning and memory. 2. COMPUTATIONAL NEUROSCIENCE / COGNITIVE NEUROSCIENCE: Models of the neural bases of learning in animals and humans. Candidates with any (or all) of the following skills are particular encouraged to apply: (1) familiarity with neural network algorithms and models, (2) strong computational/analytic skills, and (3) experience with experimental methods, experimental design, and data analysis in cognitive psychology. ---------------------------------------------------------------------------- Other Information: FACILITIES: The Center is a new state-funded research center for the integrated studies of cognitive, behavioral, and molecular neuroscience. The Center has good computational resources and experimental laboratories for behavioral and neural studies. LOCATION: The Center is located in Newark, NJ, approximately 20 minutes outside of Manhattan, New York (with easy train and subway access to midtown and downtown NYC) and close to rural New Jersey countryside Numerous other research centers in the cognitive and neural sciences are located nearby including: Cognitive Science Center, Rutgers/New Brunswick; Centers for Cognitive & Neural Science, New York University; Cognitive Science Center, Princeton Univ.; Columbia Univ. & Medical School; Siemens Corporate Research, Princeton, NJ; NEC Research Labs, Princeton, NJ; AT&T Labs; Bellcore; IBM T. J. Watson Research Labs. CURRENT FACULTY: E. Abercrombie, G. Buzsaki, I. Creese, M. Gluck, H. Poizner, R. Siegel, P. Tallal, J. Tepper. Six additional faculty will be hired. The Center has a total of ten state-funded postdoctoral positions and will direct, in collaboration with the Institute for Animal Behavior, a graduate program in Behavioral and Neural Sciences. ---------------------------------------------------------------------------- For more information on learning research at the CMBN/Rutgers or to apply for these post-doctoral positions, please send a cover letter with a statement of your research interests, a CV, copies of relevant preprints, and the the names & phone numbers of references to: Dr. Mark A. Gluck Phone: (415) 725-2434 Dept. of Psychology <-[Current address to 4/91] FAX: (415) 725-5699 Jordan Hall; Bldg. 420 Stanford University email: gluck at psych.stanford.edu Stanford, CA 94305-2130 From gmk%idacrd at Princeton.EDU Fri Dec 14 12:03:54 1990 From: gmk%idacrd at Princeton.EDU (Gary M. Kuhn) Date: Fri, 14 Dec 90 12:03:54 EST Subject: 1st IEEE-SP Workshop on NN's for SP Message-ID: <9012141712.AA19624@Princeton.EDU> First IEEE-SP Workshop on Neural Networks for Signal Processing Sponsored by the IEEE Signal Processing Society in cooperation with the IEEE Neural Networks Council September 29 - October 2, 1991 Nassau Inn, Princeton, New Jersey, USA Call for Papers The first Workshop on Neural Networks for Signal Processing, sponsored by the IEEE Signal Processing Society, will be held in the fall of 1991 in Princeton, New Jersey. The beautiful Princeton area is easily accessible by train, bus or car from airports in and around New York city. Papers are solicited for technical sessions on the following topics: + Application-driven Neural Models + Neural Architecture for Signal Processing + System Identification & Spectral Estimation by Neural Networks + Neural Networks for Image Processing & Pattern Recognition + Applications of Neural Networks to Speech Processing + Nonlinear Signal and Pattern Learning Algorithms Prospective authors are invited to submit 4 copies of extended summaries of no more than 4 pages to Candace Kamm for review (address below). The top of the first page of the summary should include a title, authors' names, affiliations, addresses and telephone numbers. Photo-ready full papers of accepted proposals will be published in book form and distributed at the workshop. Due to conference facility constraints, attendance will be limited with priority given to those who submit written technical contributions. For more information, please contact Gary Kuhn, Publicity Chair, at (609) 924-4600. Schedule Submission of extended summary April 1, 1991 Notification of acceptance May 15, 1991 Submission of photo-ready paper July 1, 1991 Advanced registration, before August 31, 1991 Workshop Committee General Chair B.H. Juang S.Y. Kung Rm. 2D-534 Dept. of EE AT&T Bell Labs Princeton Univ. Murray Hill, NJ 07974 Princeton, NJ 08540 Local Arrangements John Vlontzos Siemens Corp. Research Princeton, NJ 08540 Proceedings Candace Kamm Box 1910 Bellcore 445 South St., Rm.2E-256 Morristown, NJ 07960-1910 Publicity Gary Kuhn Center for Communications Research-IDA Thanet Road Princeton, NJ 08540 Finance/Registration Bastiaan Kleijn Rm 2D-554 AT&T Bell Labs 600 Mountain Ave. Murray Hill, NJ 07974 Program Committee Rama Chellappa Lee Giles John Moody Bradley Dickinson Esther Levin Erkki Oja Tariq Durrani R. Lippmann W. Przytula F. Fallside John Makhoul Y. Tohkura K. Fukushima Y. Matsuyama C.J. Wellekens From jfeldman at ICSI.Berkeley.EDU Fri Dec 14 12:33:02 1990 From: jfeldman at ICSI.Berkeley.EDU (Jerry Feldman) Date: Fri, 14 Dec 90 09:33:02 PST Subject: Scientific American Message-ID: <9012141733.AA02407@icsib2.berkeley.edu.Berkeley.EDU> I have been asked by Scientific American magazine to write an article on connectionism and artifical intelligence. My current approach to the project can be seen from the working subtitle: "Often viewed as competing, these two approaches to understanding intelligent behavior can be combined to yield scientific and practical advances". I am looking for suggestions, particularly on success stories involving connectionist systems. For this purpose we need devices that are actually in daily use, scientific results that have had a major impact, etc. Things that are merely promising or have just provoked controversy aren't nearly as effective. Of course, I welcome any other suggestions. Please use e-mail unless you really want to address the whole group. Jerry F. jfeldman at icsi.berkeley.edu From frederic at cs.unc.edu Fri Dec 14 13:38:50 1990 From: frederic at cs.unc.edu (Robin Fredericksen) Date: Fri, 14 Dec 90 13:38:50 -0500 Subject: No subject Message-ID: <9012141838.AA07837@sargent.cs.unc.edu> I apologize if I came across too sharply. It's true that there needs to be some well thought out terminological classification. My response in part is because I *AM* interested in the ontogeny of biological systems and what it can teach us. My personal research interests includes how the biological system does what it does and how we can use that to our advantage. It is true that there are many researchers (as well as engineers) who are looking at neural networks as a new and useful tool and could care less how that tool relates to the systems that originally inspired the technique (or at least invented it first :-) ). We must find some way to differentiate between models of (CNS) ontogeny for the study of (CNS) ontogeny, and the use of ontogenically derived methods for finding the correct network (CNS) structure to solve some specific computational problem. Yes, the word ontogeny is too general: on.tog.e.ny \a:n-'ta:j-*-ne-\ n [ISV] : the development or course of development of an individual organism (don't you just love the on-line Websters?) We need to find a term that refers to *CNS* development. *But*, I think that use of the words 'destructive' and 'constructive' can be just as misleading. If I understand correctly (I was not at NIPS), 'destructive' is used to describe a network that begins with surfeit of nodes and removes nodes during learning. 'Constructive' is then used to describe networks that add nodes during learning. A more accurate phrase might be 'structurally self modifying' (or must we find a single word? should we then say it in German? :-) ) Are there such strongly qualitative differences in networks resulting from the use of either strictly 'destructive' or 'constructive' rules that we need separate terminology? We have available lots of verbs: alter, modify, transform, mutate, change, vary (ad infinitem, almost; I don't have an on-line thesaurus). tmb at ai.mit.edu says: > ... Ontogeny is a very specializec (sic) >process with a number of strict biological constraints operating; it is >far from clear that processes that operate during ontogeny are similar >or analogous to unit creation/destruction in an artificial neural >network that learns. > >The thought of equating a mathematical and engineering technique with >this biological process makes me cringe. The judgement is still out to >what degree artificial "neural networks" are related to the real thing, >so lets not aggravate the problem by introducing more biological >terminology into the field. Again, I agree that ontogeny is not the correct word, and I apologize for defending it without suggesting a better one. But is it not true that we put lots of constraints on neural networks in order to coerce them into 'learning' the correct answer? It may not be clear that there is any similarity between the learning rules that use creation/destruction of units in a network and 'rules' in the process of CNS development (not ontogeny, which is too general a term), but I would also object to having an artificial (ideological) wall put up between the concepts. In the end, the real problem in selecting terminology lies in whether you view a nn as an engineering tool that should be designed in a void, completely derived without hints or clues from nature, or whether you view it as a tool modeled after organisms (many of which have been very successful in terms of evolution) from which we might learn. My personal bias is the latter. INS_ATGE%jhuvms.hcf.jhu.edu at BITNET.CC.CMU.EDU writes: >The other problem is that there might be learning in brain which involves >recruitment of neurons or exclusion of neurons from the network performing a >cognitive function (i.e. "software routing") which occurs much later than >ontogeny (I do not know of results which show this, but constructive/ >destructive learning is so useful that it would seem useful if it was >performed by "soft" changes in neural nets as opposed to actual >synaptogenesis/cell death)(Do you know of any?). >Anyway, I understand your point about reinventing wheels, but it seems >that "ontogenic nets" seems too limiting to apply to "architecturally >changeable" learning methods, with the exception of physiological >learning which occurs during ontogeny. First, Dale Purves feels that synaptogenesis continues throughout the organisms life span (and thus CNS ontogeny). Cell death occurs over a much more limited space of time that depends on the complexity (size) of the animal but is generally restricted to pre and a small postnatal period. (Small in comparison with the animal's expected lifespan.) By software routing, do you mean adding connections to new units? That is just synaptogenesis and its counterpart (synaptic loss). Ok, so now I have wandered far away from defending the use of 'ontogenic' and delved into my true reaction. I was actually responding to what appeared to be sarcasm (sorry Scott) but was intended to be humor. My real objection is to the building of walls within what should be an interdisciplinary field through the use of imprecise or restricted terminology. The problem then is deciding what is precise, and that can come down to your personal view of what nn research is for: building tools, understanding biological systems, or both at the same time. So, any good ideas on terminology that fits the bill? [More attempted humor...] I wonder if constructive and destructive neural network learning rules will, in the course of research, recapitulate the phylogeny of biological neural networks? We could separate out (on different continents, of course) groups of researchers trying to develop a nn to solve the same class of problems, say vision (email not allowed, of course). Then after a suitable amount of time, perhaps one group will have evolved rules for building crustacean visual systems, another group will have evolved rules for the building of arthropod visual systems, and yet another will be able to build mammalian visual systems? But, will it take us hundreds of millions of years? :-) Eric Fredericksen From chuck at cs.utk.edu Fri Dec 14 22:23:35 1990 From: chuck at cs.utk.edu (chuck@cs.utk.edu) Date: Fri, 14 Dec 90 22:23:35 -0500 Subject: Negative feedback loops Message-ID: <9012150323.AA14803@alphard.cs.utk.edu> Agreed, But since I call long distance to hear all this amusing drivel some one migh bring it to Dave's attention. He uses us enough! Chuck Joyce chuck at cs.utk.edu From fellous%pipiens.usc.edu at usc.edu Fri Dec 14 20:31:13 1990 From: fellous%pipiens.usc.edu at usc.edu (Jean-Marc Fellous) Date: Fri, 14 Dec 90 17:31:13 PST Subject: No subject Message-ID: <9012150131.AA00490@pipiens.usc.edu> Subject: USC Workshop on Emotions (please forward on relevant mailing lists) __________________________________________________________________________ / U.S.C \ | | | C N E Student Workshop on Emotions | | | | CALL FOR PAPERS | | ***************** | \__________________________________________________________________________/ The Center For Neural Engineering of the university of Southern California invites all students interested in Emotions to submit a paper to be eventually presented during a one-day Workshop (of a date t.b.a. at the End of February 1991). The Workshop is opened to Graduate students (MA,MS,PhD) and College Seniors irrespective to their major (faculty will only be considered for publication), having pursued (or pursuing) research activities on such aspects of Emotions as: - The nature of Emotion - The physiology of Emotion - The perception of Emotions - The relations between Emotion and Cognition - Developemental aspects of Emotion - Artificial Intelligence models of Emotions - Neural network models of Emotions - Philosophical issues of Emotion and reductionism - ... Applicants should send a 2 page summary of the proposed paper and a letter of motivation in which they state their status, major, interests, name, address and telephone number (for reply). Materials should be submitted by January 31st to: Jean-Marc Fellous Center for Neural Engineering University of Southern California Los Angeles CA 90089-2520 Telephone: (213) 740-3506 email: fellous at rana.usc.edu ps: Travel expenses will not be covered by the CNE, but lunch will be provided. pps: Authors of the chosen papers will receive a copy of the presented papers (by mail if they could not attend the Workshop). ***************************************************************************** ppps: Please forward to relevant departments, mailing lists ... ***************************************************************************** .. From markt at umd5.umd.EDU Sat Dec 15 13:02:47 1990 From: markt at umd5.umd.EDU (Mark Turner) Date: Sat, 15 Dec 90 13:02:47 EST Subject: neural image? Message-ID: <9012151802.AA26260@umd5.UMD.EDU> I am seeking a suitable image suggesting neuronal group patterns and neural connectivity to be used as the background of a book cover. The book, titled POETIC THOUGHT: THE STUDY OF ENGLISH IN THE AGE OF COGNITIVE SCIENCE, will be published by Princeton University Press in the fall of 1991. POETIC THOUGHT is, to an extent, neurally-inspired in its themes and approaches; it focuses on conceptual connections and neural connections. I am not having any luck in this search, and would be indebted for any help. I need not only a high-quality image to be used in reproduction but also permission to use it, with the appropriate credit given of course. Mark Turner markt at umd5.umd.edu From pigmalio at batman.fi.upm.es Thu Dec 13 14:20:00 1990 From: pigmalio at batman.fi.upm.es (pigmalio@batman.fi.upm.es) Date: 13 Dec 90 20:20 +0100 Subject: No subject Message-ID: <9012131920.AA11890@batman.fi.upm.es> Subject: Request information about C compilers. We should be very pleased if you could send us information about transputer boards and intelligent C-compilers that paralelize automatically and are able to use these boards. Thanks a lot. E-mail : pigmalio at batman.fi.upm.es From worth at park.bu.EDU Sat Dec 15 13:59:52 1990 From: worth at park.bu.EDU (Andrew J. Worth) Date: Sat, 15 Dec 90 13:59:52 -0500 Subject: Connectionism vs AI Message-ID: <9012151859.AA15760@park.bu.edu> Jerry Feldman's posting ("Scientific American") foments a long standing dispute within my mind; are there fundamental differences between connectionism and AI that make them incompatible in an ideal sense? Examining each as an approach to the same goal, if one of connectionism's fundamental tenets is to mimic biological computation, and AI, on the other hand, holds sacred the extracting of the essence of "intelligence" while ignoring implementation details, (i.e. a bottom up vs. top down dichotomy) then is it not a bastardization of both to combine them? If each approach has fundamentally different and opposing assumptions, then wouldn't one or both of them have to be weakened in their combination? Andrew J. Worth worth at park.bu.edu Cognitive & Neural Systems Prog. (617) 353-6741 Boston University (617) 353-7857 (CAS Office) 111 Cummington St. Room 244 (617) 353-5235/6742 (CNS Grad Offices) Boston, MA 02215 From frederic at cs.unc.edu Sun Dec 16 13:16:33 1990 From: frederic at cs.unc.edu (Robin Fredericksen) Date: Sun, 16 Dec 90 13:16:33 -0500 Subject: apology Message-ID: <9012161816.AA03295@pooh.cs.unc.edu> I would like to apologize for the use of a quote in my previous email posting. The text from tmb at ai.mit.edu was from a personal communication, and I did not realize that it was such. So, how do I tell that a piece of email is from the connectionists mailing group? Will it have @SEF1.SLISP.CS.CMU.EDU as part of the address? Eric Fredericksen From mrj at cs.su.oz.au Sun Dec 16 06:29:54 1990 From: mrj at cs.su.oz.au (Mark James) Date: Sun, 16 Dec 90 22:29:54 +1100 Subject: Synchronization of Cortical Oscillations Message-ID: <14001.661346994@mango.cs.su.oz> I would be interested to hear from anyone who attended the NIPS workshop on cortical oscillations if anything was concluded regarding the mechanism for long distance synchronization of the oscillations. That is, is the excitation direct, via a change in effective synaptic efficacy (e.g. Eckhorn and others) or via disinhibition mediated by cells such as the spiny double bouquet cell. Thank you, Mark James | EMAIL : mrj at cs.su.oz.au | From INS_ATGE%JHUVMS.BITNET at VMA.CC.CMU.EDU Sun Dec 16 19:27:00 1990 From: INS_ATGE%JHUVMS.BITNET at VMA.CC.CMU.EDU (INS_ATGE%JHUVMS.BITNET@VMA.CC.CMU.EDU) Date: Sun, 16 Dec 90 19:27 EST Subject: Sci Am Article Message-ID: A. Worth: Jerry Feldman's posting ("Scientific American") foments a long standing dispute within my mind; are there fundamental differences between connectionism and AI that make them incompatible in an ideal sense? Examining each as an approach to the same goal, if one of connectionism's fundamental tenets is to mimic biological computation, and AI, on the other hand, holds sacred the extracting of the essence of "intelligence" while ignoring implementation details, (i.e. a bottom up vs. top down dichotomy) then is it not a bastardization of both to combine them? If each approach has fundamentally different and opposing assumptions, then wouldn't one or both of them have to be weakened in their combination? I see neural networks as a sub-field of artificial intelligence. Both are trying to develop "intelligent" artifacts. There is of course a dichotomy between symbolic AI and neural net AI. Symbolic AI has proven itself in many tasks such as symbolic calculus, theorem provers, expert systems, and other machine learning tasks. The difference between symbolic AI and neural AI is more one of computational substrate (although researchers may have artificially distanced neural nets from symbolic AI in the past). Alot of neural networks for the last few years has been applying a hill-climbing heuristic (well known to the symbolic AI community) to our nets. They learn, but not well. There is still a great deal of symbolic AI machine learning theory which could be used to set up really interesting neural networks, but there are difficulties in translating between a symbolic computational substrate and the neural network substrate. The constructive/destructive (or "ontogenic") networks which are comming down the line, such as Cascade-Correlation, are showing that hill-climbing in a fixed energy landscape is not the only way to do learning. There is also "compositional" learning (someone asked for the Schmidhuber ref. a while back...it's J. Schmidhuber. Towards compositional learning with dynamic neural networks. Report FKI-129-90, Technische Universitat Munchen, April 1990.) utilizing combining sub-goal networks together to achieve larger goals. Anyway, I think the lesson is that no matter how much connectionist researchers think their networks are capable of better inductive learning than symbolic AI systems, in order to do allow for deductive learning we are going to have to couch alot of existing symbolic AI heuristics and machine learning paradigms in a network architecture. -Thomas Edwards From Scott.Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU Sun Dec 16 16:00:36 1990 From: Scott.Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF-PMAX.SLISP.CS.CMU.EDU) Date: Sun, 16 Dec 90 16:00:36 EST Subject: apology In-Reply-To: Your message of Sun, 16 Dec 90 13:16:33 -0500. <9012161816.AA03295@pooh.cs.unc.edu> Message-ID: So, how do I tell that a piece of email is from the connectionists mailing group? Will it have @SEF1.SLISP.CS.CMU.EDU as part of the address? No, but mail that was sent to "connectionists" will generally have "connectionists at cs.cmu.edu" in either the "To:" or the "Cc:" field. If you don't see that, assume that the mail was sent just to you. Of course, mail-reading programs vary a lot, and some may hide some of this information. -- Scott From geb at dsl.pitt.edu Sun Dec 16 15:58:57 1990 From: geb at dsl.pitt.edu (Gordon E. Banks) Date: Sun, 16 Dec 90 15:58:57 -0500 Subject: Connectionism vs AI Message-ID: <9012162058.AA08797@cadre.dsl.pitt.edu> Traditional AI, rather than ignoring the details, started out by studying human behavior and cognition (see Simon & Newell, for example) in the realm of problem solving. It has always had a strong empirical element, in my opinion. Thus connectionism isn't alien in methodology from the original spirit of the pursuit of intelligent behavior. From epreston at aisun2.ai.uga.edu Sun Dec 16 14:41:02 1990 From: epreston at aisun2.ai.uga.edu (Elizabeth Preston) Date: Sun, 16 Dec 90 14:41:02 EST Subject: Connectionism vs AI In-Reply-To: "Andrew J. Worth"'s message of Sat, 15 Dec 90 13:59:52 -0500 <9012151859.AA15760@park.bu.edu> Message-ID: <9012161941.AA03794@aisun2.ai.uga.edu> Date: Sat, 15 Dec 90 13:59:52 -0500 From: "Andrew J. Worth" Jerry Feldman's posting ("Scientific American") foments a long standing dispute within my mind; are there fundamental differences between connectionism and AI that make them incompatible in an ideal sense? I'm afraid this dispute is not all in your mind; a great deal of ink and hot air has been expended on it in various public forums over the last few years. Perhaps the most familiar version of this dispute is the proposal by some that connectionism represents a paradigm shift (in the Kuhnian sense) in our understanding of cognition. This would make connectionism and AI not merely incompatible, but incommensurable; e.g., it would follow that connectionism and AI are in some sense not even working on the same problem(s), and that connectionists and AI-ists are in some sense not even understanding each other when they talk. Pretty radical stuff. Of course there are those who respond to this talk of a paradigm shift with incredulous stares and phrases on the order of "oh, pooh" and "nonsense". People on this end of the spectrum have been known to argue that connectionism is merely a way of implementing AI; in which case, far from being incompatible, they are basically the same thing. Although I think there are a lot of interesting and helpful things to be said about the similarities and differences between AI and connectionism, I doubt very much whether the question of FUNDAMENTAL compatibility/incompatibility is going to be settled on the conceptual level anytime soon. I say this partly because I don't think it has ever really been settled for the relationship between empiricism and rationalism in philosophy, or for the relationship between behaviorism and cognitivism in psychology, and these divisions in philosophy and psychology are pretty clearly the historical antecedents of the division in computational circles between connectionism and AI. And partly I say this because I don't see that the question is answerable at this stage in the development and theoretical understanding of the two fields. In fact, I'm not sure it's even askable. If each approach has fundamentally different and opposing assumptions, then wouldn't one or both of them have to be weakened in their combination? As a philosopher with a great deal of experience in armchair theorizing, I must say I wouldn't touch this one with a stick. Why don't you try it out and see what happens? Beth Preston From jlm+ at ANDREW.CMU.EDU Mon Dec 17 10:21:37 1990 From: jlm+ at ANDREW.CMU.EDU (James L. McClelland) Date: Mon, 17 Dec 90 10:21:37 -0500 (EST) Subject: Connectionism vs AI In-Reply-To: <9012151859.AA15760@park.bu.edu> References: <9012151859.AA15760@park.bu.edu> Message-ID: <0bPC_1m00jWD83q14n@andrew.cmu.edu> Regarding Worth's query about a possible fundamental opposition between connectionist and other approaches to AI: I do not think there need be such an opposition. That is, I think one can be a connectionist without imagining that any such opposition exists. Connectionist models are used for a variety of different purposes and with a variety of different goals in mind: Here are three: 1) To find better methods for solving AI problems, particularly those that have proven difficult to solve using conventional AI approaches. 2) To model actual mechanisms of neural computation. There's lots of data on such things as the stimulus conditions under which particular neurons will fire, but there is little understanding of the circuitry that leads to the patterns of firing that are seen or the role the neurons play in overall system function. Connectionist models can help in the exploration of these questions. 3) To explore mechanisms of human information processing. Here the idea is that there is a set of putative principles of human information processing that are more easily captured in connectionist models than in other formalisms. The effort to determine whether these principles are the right ones or not requires the use of models, since it is difficult to assess the adequacy of sets of principles without formalization, leading to analysis and/or simulation. There are others. The point is that the models can be viewed as tools for exploring questions. There need be no such religion as 'connectionism'; all it takes to be a connectionist is to find connectionist models useful. -- Jay McClelland From block at psyche.mit.edu Mon Dec 17 11:42:04 1990 From: block at psyche.mit.edu (Ned Block) Date: Mon, 17 Dec 90 11:42:04 EST Subject: Connectionism vs AI Message-ID: <9012171642.AA04247@psyche.mit.edu> Worth: Connectionism and AI are incompatible because connectionism is biological and AI ignores implementation. False, false, false. Connectionism IS a tyhpe of AI, albeit biologically inspired AI. Connectionist networks could easily be made more brain- like, eg, by not allowing weights to change between positive and negative. But this is not a popular idea, and ONLY becasue it would make the networks much less useful. Ned Block From bates at crl.ucsd.edu Mon Dec 17 12:57:51 1990 From: bates at crl.ucsd.edu (Elizabeth Bates) Date: Mon, 17 Dec 90 09:57:51 PST Subject: ontogenesis and synaptogenesis Message-ID: <9012171757.AA25001@crl.ucsd.edu> Just a small empirical correction to the discussion on ontogeny in neural nets. Synaptogenesis does NOT continue across the (human/primate) lifespan, at least not on any kind of a large or interesting scale. Research by Rakic, Huttenlocher and others suggests that there is a huge burst in synaptogenesis between (roughly) human postnatal months 6 - 24. There is some controversy about whether this "burst" takes place across the whole brain at once (the Rakic position) or whether different regions "burst" at different times (the Huttenlocher position), but it is fairly clear from research on both sides that the "burst" is over by the time the child is 2 years old. And of course, as already noted on the net, cell proliferation and migration is over with well before that, at least a month or so prior to birth (with the exception of a couple of areas like the olfactory bulb). The bottom line for "neurally inspired" connectionist models seems to be that the block of marble is delivered to the studio for carving by age 2. This of course does not exclude small-scale, very local changes that do occur across the lifetime (c.f. research by Merzenich and others demonstrating reorganization of somatosensory maps in adult primates -- but reorganization that appears to be restricted to within a millemeter distance). However, most developmental neurobiologists that I have read recently argue that subtractive events (i.e. axon retraction, cell death and above all synaptic degeneration) are the most interesting candidates for a brain basis of behavioral change after the infant years; there is (or so it seems right now) little likelihood that the major behavioral changes we observe across the human lifetime can be explained by recourse to additive events (cell formation, synaptogenesis, and not even the peripatetic but poorly understood event of myelination). My colleagues and I have written a chapter on the relevance of these neural events for early language development, which I would be happy to send out to anyone that is interested in this "consumer's perspective". the reference is: Bates, E., Thal, D. & Janowsky, J. (in press0. Early language development and its neural correlates. In I. Rapin & S. Segalowitz (Eds.), Handbook of Neuropsychology, Vol. 7. Holland:Elsevier. I would be interested in hearing from anyone who has tried to model higher cognitive processes in neural nets that are "exploding" (in a fairly uncontrolled way) in number of connections. Should be an interesting problem! It does appear to be the case that the first stages of language learning (from first words through the early stages of grammar) take place under precisely those circumstances. Which must be balanced against a second fact: a child who suffers a massive left hemisphere lesion up to at least age 2 - 3 can apparently acquire language at normal or near-normal levels after that point, presumably in the undamaged hemisphere. So whatever "investments" in neural tissue are being made during this rapid phase of development apparently can be "undone" and/or "redone" elsewhere. -liz bates From jbower at smaug.cns.caltech.edu Mon Dec 17 13:46:07 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Mon, 17 Dec 90 10:46:07 PST Subject: Oscillations Message-ID: <9012171846.AA03643@smaug.cns.caltech.edu> Concerning the inquiry by Mark James on the recent NIPS workshop on cortical oscillations. Matthew Wilson and I published a paper in last years NIPS proceedings that proposed a mechanism for cortical oscillations in visual cortex that involves both horizontal connections and inhibitory neurons. This work is also the subject of an upcoming paper in Neural Computation (probably the second 1991 issue). This work is derived from our efforts over the last five years to model 40 Hz oscillations in the olfactory system (c.f. Bower, J.M. 1990, Reverse engineering the nervous system: An anatomical, physiological, and computer based approach. In: An Introduction to Neural and Electronic Networks. S. Zornetzer, J. Davis, and C. Lau, editors. Academic Press pp. 3-24.) . These olfactory oscillations appear to be far more robust than those in visual cortex. I should mention that the important conclusion of the visual cortex work is that the zero phase oscillations occur only on average over a number of stimulus trials and therefore are unlikely to serve as the basis for "binding", attention, or awareness. Jim Bower jbower at smaug.cns.caltech.edu From djy at inel.gov Mon Dec 17 14:58:44 1990 From: djy at inel.gov (Daniel J Yurman) Date: Mon, 17 Dec 90 12:58:44 -0700 Subject: Multiples Message-ID: <9012171958.AA28107@gemstone.inel.gov> I have receicved over a dozen copies of this announcement. Although the announced workshop is about emotions, dry humor does not convey well via the net. I suggest the originator, whomever you are, be advised that you are scaring the fish in Idaho and we would like you to try elsewhere. Thanks for your kindness in advance. * Standard disclaimer included by reference * ------------------------------------------------------------ * Dan Yurman Idaho National Engineering Laboratory * djy at inel.gov PO Box 1625, Idaho Falls, ID 83415-3900 * phone: (208) 526-8591 fax: (208)-526-6852 * ------------------------------------------------------------ * 43N 112W -7GMT Red Shift is not a brand of chewing tobacco! From chan%unb.ca at UNBMVS1.csd.unb.ca Mon Dec 17 21:27:07 1990 From: chan%unb.ca at UNBMVS1.csd.unb.ca (Tony Chan) Date: Mon, 17 Dec 90 22:27:07 AST Subject: Connectionism vs AI? Message-ID: Jerry Feldman was asked by Scientific American magazine to write an article on connectionism and artifical intelligence. His assumption or working assumption as he embarked on the subject was the following: "Often viewed as competing, these two approaches to understanding intelligent behavior can be combined to yield scientific and practical advances." [Fri, 14 Dec 90 12:33:02 EST] A day later, Andrew J. Worth asked [Sat, 15 Dec 90 13:59:52 EST] "[A]re there fundamental differences between connectionism and AI that make them incompatible in an ideal sense?" Concerning this and related questions, Elizabeth Preston [Sun, 16 Dec 90 14:41:02 EST] mentioned, "a great deal of ink and hot air has been expended on it in various public forums over the last few years. ... I don't see that the question is answerable at this stage in the development and theoretical understanding of the two fields. In fact, I'm not sure it's even askable." I would not ask that question; instead, I would ask a larger question which, if satifactorily answered, would make the previous question obsolete: Should there exist one formal (mathematical) unifying paradigm within which all investigations concerning the artificial re-creation of intelligence are to be carried out? If the answer is yes, as I believe, an immediate question that should be asked is the legitimacy of the current ad hoc separation of these related areas: artificial intelligence, connectionism, pattern recognition, cognitive science, cybernetics, machine learning, etc.? In fact, not only there should be one---there is one! So the answer is affirmative in existential sense and in constructive sense. I refer interested readers to the communication of Lev Goldfarb [Thu, 27 Sep 90 From penrose at edda.css.gov Tue Dec 18 11:50:11 1990 From: penrose at edda.css.gov (Christopher Penrose) Date: Tue, 18 Dec 90 08:50:11 PST Subject: change of address Message-ID: <9012181650.AA00748@edda.css.gov> Please forgive me for my ever selfish use of network bandwidth! If the moderator of this list has a moment in their busy life to spare, I'd appreciate if my mailing address could change. from: {...}!esosun!jesus!penrose to: penrose at esosun.css.gov Thank you very much! I apologize to all the busy researchers whose precious moments were wasted by the callousness of this message. Christopher Penrose jesus!penrose From birnbaum at fido.ils.nwu.edu Tue Dec 18 13:02:01 1990 From: birnbaum at fido.ils.nwu.edu (Lawrence Birnbaum) Date: Tue, 18 Dec 90 12:02:01 CST Subject: ML91 Final Call for Papers Message-ID: <9012181802.AA05320@fido.ils.nwu.edu> THE EIGHTH INTERNATIONAL WORKSHOP ON MACHINE LEARNING CALL FOR PAPERS On behalf of the organizing committee, and the individual workshop committees, we are pleased to announce submission details for the eight workshop tracks that will constitute ML91, the Eighth International Workshop on Machine Learning, to be held at Northwestern University, Evanston, Illinois, USA, June 27-29, 1991. The eight workshops are: o Automated Knowledge Acquisition o Computational Models of Human Learning o Constructive Induction o Learning from Theory and Data o Learning in Intelligent Information Retrieval o Learning Reaction Strategies o Learning Relations o Machine Learning in Engineering Automation Please note that submissions must be made to the workshops individually, at the addresses given below, by March 1, 1991. The Proceedings of ML91 will be published by Morgan Kaufmann. Questions concerning individual workshops should be directed to members of the workshop committees. All other questions should be directed to the program co-chairs at ml91 at ils.nwu.edu. Details concerning the individual workshops follow. Larry Birnbaum Gregg Collins Northwestern University The Institute for the Learning Sciences 1890 Maple Avenue Evanston, IL 60201 phone (708) 491-3500 ---------------------------------------------------------------------------- AUTOMATED KNOWLEDGE ACQUISITION Research in automated knowledge acquisition shares the primary objective of machine learning research: building effective knowledge bases. However, while machine learning focuses on autonomous "knowledge discovery," automated knowledge acquisition focuses on interactive knowledge elicitation and formulation. Consequently, research in automated knowledge acquisition typically stresses different issues, including how to ask good questions, how to learn from problem-solving episodes, and how to represent the knowledge that experts can provide. In addition to the task of classification, which is widely studied in machine learning, automated knowledge acquisition studies a variety of performance tasks such as diagnosis, monitoring, configuration, and design. In doing so, research in automated knowledge acquisition is exploring a rich space of task-specific knowledge representations and problem solving methods. Recently, the automated knowledge acquisition community has proposed hybrid systems that combine machine learning techniques with interactive tools for developing knowledge-based systems. Induction tools in expert system shells are being used increasingly as knowledge acquisition front ends, to seed knowledge engineering activities and to facilitate maintenance. The possibilities of synergistic human-machine learning systems are only beginning to be explored. This workshop will examine topics that span autonomous and interactive knowledge acquisition approaches, with the aim of productive cross- fertilization of the automated knowledge acquisition and machine learning communities. Submissions to the automated knowledge acquisition track should address basic problems relevant to the construction of knowledge-based systems using automated techniques that take advantage of human input or human- generated knowledge sources and provide computational leverage in producing operational knowledge. Possible topics include: o Integrating autonomous learning and focused interaction with an expert. o Learning by asking good questions and integrating an expert's responses into a growing knowledge base. o Using existing knowledge to assist in further knowledge acquisition. o Acquiring, representing, and using generic task knowledge. o Analyzing knowledge bases for validity, consistency, completeness, and efficiency then providing recommendations and support for revision. o Automated assistance for theory / model formation and discovery. o Novel techniques for knowledge acquisition, such as explanation, analogy, reduction, case-based reasoning, model-based reasoning, and natural language understanding. o Principles for designing human-machine systems that integrate the complimentary computational and cognitive abilities of programs and users. Submissions on other topics relating automated knowledge acquisition and autonomous learning are also welcome. Each submission should specify the basic problem addressed, the application task, and the technique for addressing the problem. WORKSHOP COMMITTEE Ray Bareiss (Northwestern Univ.) Bruce Buchanan (Univ. of Pittsburg) Tom Gruber (Stanford Univ.) Sandy Marcus (Boeing) Bruce Porter (Univ. of Texas) David Wilkins (Univ. of Illinois) SUBMISSION DETAILS Papers should be approximately 4000 words in length. Authors should submit six copies, by March 1, 1991, to: Ray Bareiss Northwestern University The Institute for the Learning Sciences 1890 Maple Avenue Evanston, IL 60201 phone (708) 491-3500 Formats and deadlines for camera-ready copy will be communicated upon acceptance. ---------------------------------------------------------------------------- COMPUTATIONAL MODELS OF HUMAN LEARNING Details concerning this workshop will be forthcoming as soon as possible. ---------------------------------------------------------------------------- CONSTRUCTIVE INDUCTION Selection of an appropriate representation is critical to the success of most learning systems. In difficult learning problems (e.g., protein folding, word pronunciation, relation learning), considerable human effort is often required to identify the basic terms of the representation language. Constructive induction offers a partial solution to this problem by automatically introducing new terms into the representation as needed. Automatically constructing new terms is difficult because the environment or teacher usually provides only indirect feedback, thus raising the issue of credit assignment. However, as learning systems face tasks of greater autonomy and complexity, effective methods for constructive induction are becoming increasingly important. The objective of this workshop is to provide a forum for the interchange of ideas among researchers actively working on constructive induction issues. It is intended to identify commonalities and differences among various existing and emerging approaches such as knowledge-based term construction, relation learning, theory revision in analytic systems, learning of hidden- units in multi-layer neural networks, rule-creation in classifier systems, inverse resolution, and qualitative-law discovery. Submissions are encouraged in the following topic areas: o Empirical approaches and the use of inductive biases o Use of domain knowledge in the construction and evaluation of new terms o Construction of or from relational predicates o Theory revision in analytic-learning systems o Unsupervised learning and credit assignment in constructive induction o Interpreting hidden units as constructed features o Constructive induction in human learning o Techniques for handling noise and uncertainty o Experimental studies of constructive induction systems o Theoretical proofs, frameworks, and comparative analyses o Comparison of techniques from empirical learning, analytical learning, classifier systems, and neural networks WORKSHOP COMMITTEE Organizing Committee: Program Committee: Christopher Matheus (GTE Laboratories) Chuck Anderson (Colorado State) George Drastal (Siemens Corp.) Gunar Liepins (Oak Ridge National Lab) Larry Rendell (Univ. of Illinois) Douglas Medin (Univ. of Michigan) Paul Utgoff (Univ. of Massachusetts) SUBMISSION DETAILS Papers should be a maximum of 4000 words in length. Authors should include a cover page with authors' names, addresses, phone numbers, electronic mail addresses, paper title, and a 300 (maximum) word abstract. Do not indicate or allude to authorship anywhere within the paper. Send six copies of paper submissions, by March 1, 1991, to: Christopher Matheus GTE Laboratories 40 Sylvan Road, MS-45 Waltham MA 02254 (matheus at gte.com) Formats and deadlines for camera-ready copy will be communicated upon acceptance. ---------------------------------------------------------------------------- LEARNING FROM THEORY AND DATA Research in machine learning has primarily focused on either (1) inductively generalizing a large collection of training data (empirical learning) or (2) using a few examples to guide transformation of existing knowledge into a more usable form (explanation-based learning). Recently there has been growing interest in combining these two approaches to learning in order to overcome their individual weaknesses. Preexisting knowledge can be used to focus inductive learning and to reduce the amount of training data needed. Conversely, inductive learning techniques can be used to correct imperfections in a system's theory of the task at hand (commonly called "domain theories"). This workshop will discuss techniques for reconciling imperfect domain theories with collected data. Most systems that learn from theory and data can be viewed from the perspective of both data-driven learning (how preexisting knowledge biases empirical learning) and theory-driven learning (how empirical data can compensate for imperfect theories). A primary goal of the workshop will be to explore the relationship between these two complementary viewpoints. Papers are solicited on the following (and related) topics: o Techniques for inductively refining domain theories and knowledge bases. o Approaches that use domain theories to initialize an incremental, inductive-learning algorithm. o Theory-driven design and analysis of scientific experiments. o Systems that tightly couple data-driven and theory-driven learning as complementary techniques. o Empirical studies, on real-world problems, of approaches to learning from theory and data. o Theoretical analyses of the value of preexisting knowledge in inductive learning. o Psychological experiments that investigate the relative roles of prior knowledge and direct experience. WORKSHOP COMMITTEE Haym Hirsh (Rutgers Univ.), hirsh at cs.rutgers.edu Ray Mooney (Univ. of Texas), mooney at cs.utexas.edu Jude Shavlik (Univ. of Wisconsin), shavlik at cs.wisc.edu SUBMISSION DETAILS Papers should be single-spaced and printed using 12-point type. Authors must restrict their papers to 4000 words. Papers accepted for general presentation will be allocated 25 minutes during the workshop and four pages in the proceedings published by Morgan Kaufmann. There will also be a posters session; due to the small number of proceedings pages allocated to each workshop, poster papers will not appear in the Morgan Kaufmann proceedings. Instead, they will be allotted five pages in an informal proceedings distributed at this particular workshop only. Please indicate your preference for general or poster presentation. Also include your mailing and e-mail addresses, as well as a short list of keywords. People wishing to discuss their research at the workshop should submit four (4) copies of a paper, by March 1, 1991, to: Jude Shavlik Computer Sciences Department University of Wisconsin 1210 W. Dayton Street Madison, WI 53706 Formats and deadlines for camera-ready copy will be communicated upon acceptance. ---------------------------------------------------------------------------- LEARNING IN INTELLIGENT INFORMATION RETRIEVAL The intent of this workshop is to bring together researchers from the Information Retrieval (IR) and Machine Learning (ML) communities to explore areas of common interest. Interested researchers are encouraged to submit papers and proposals for panel discussions. The main focus will be on issues relating learning to the intelligent retrieval of textual data. Such issues include, for example: o Descriptive features, clustering, category formation, and indexing vocabularies in the domain of queries and documents. + Problems of very large, sparse feature sets. + Large, structured indexing vocabularies. + Clustering for supervised learning. + Connectionist cluster learning. + Content theories of indexing, similarity, and relevance. o Learning from failures and explanations: + Dealing with high proportions of negative examples. + Explaining failures and successes. + Incremental query formulation, incremental concept learning. + Exploiting feedback. + Dealing with near-misses. o Learning from and about humans: + Intelligent apprentice systems. + Acquiring and using knowledge about user needs and goals. + Learning new search strategies for differing user needs. + Learning to classify via user interaction. o Information Retrieval as a testbed for Machine Learning. o Particularities of linguistically-derived features. WORKSHOP COMMITTEE Christopher Owens (Univ. of Chicago), owens at gargoyle.uchicago.edu David D. Lewis (Univ. of Chicago), lewis at cs.umass.edu Nicholas Belkin (Rutgers Univ.) W. Bruce Croft (Univ. of Massachusetts) Lawrence Hunter (National Library of Medicine) David Waltz (Thinking Machines Corporation) SUBMISSION DETAILS Authors should submit 6 copies of their papers. Preference will be given to papers that sharply focus on a single issue at the intersection of Information Retrieval and Machine Learning, and that support specific claims with concrete examples and/or experimental data. To be printed in the proceedings, papers must not exceed 4 double-column pages (approximately 4000 words). Researchers who wish to propose a panel discussion should submit 6 copies of a proposal consisting of a brief (one page) description of the proposed topic, followed by a list of the proposed participants and a brief (one to two paragraph) summary of each participant's relevant work. Both papers and panel proposals should be received by March 1, 1991, at the following address: Christopher Owens Department of Computer Science The University of Chicago 1100 East 58th Street Chicago, IL 60637 Phone: (312) 702-2505 Formats and deadlines for camera-ready copy will be communicated upon acceptance. ---------------------------------------------------------------------------- LEARNING REACTION STRATEGIES The computational complexity of classical planning and the need for real-time response in many applications has led many in AI to focus on reactive systems, that is, systems that can quickly map situations to actions without extensive deliberation. Efforts to hand code such systems have made it clear that when agents must interact with complex environments the reactive mapping cannot be fully specified in advance, but must be adaptable to the agent's particular environment. Systems that learn reaction strategies from external input in a complex domain have become an important new focus within the machine learning community. Techniques used to learn strategies include (but are not limited to): o reinforcement learning o using advice and instructions during execution o genetic algorithms, including classifier systems o compilation learning driven by interaction with the world o sensorimotor learning o learning world models suitable for conversion into reactions o learning appropriate perceptual strategies WORKSHOP COMMITTEE Leslie Kaelbling (Teleos), leslie at teleos.com Charles Martin (Univ. of Chicago), martin at cs.uchicago.edu Rich Sutton (GTE), rich at gte.com Jim Firby (Univ. of Chicago), firby at cs.uchicago.edu Reid Simmons (CMU), reid.simmons at cs.cmu.edu Steve Whitehead (Univ. of Rochester), white at cs.rochester.edu SUBMISSION DETAILS Papers must be kept to four two-column pages (approximately 4000 words) for inclusion in the proceedings. Preference will be given to submissions with a single, sharp focus. Papers must be received by March 1, 1990. Send 3 copies of the paper to: Charles Martin Department of Computer Science University of Chicago 1100 East 58th Street Chicago, IL 60637 Formats and deadlines for camera-ready copy will be communicated upon acceptance. --------------------------------------------------------------------------- LEARNING RELATIONS In the past few years, there have been a number of developments in empirical learning systems that learn from relational data. Many applications (e.g. planning, design, programming languages, molecular structures, database systems, qualitative physical systems) are naturally represented in this format. Relations have also been the common language of many advanced learning styles such as analogy, learning plans and problem solving. This workshop is intended as a forum for those researchers doing relational learning to address common issues such as: Representation: Is the choice of representation a relational language, a grammar, a plan or explanation, an uncertain or probabilistic variant, or second order logic? How is the choice extended or restricted for the purposes of expressiveness or efficiency? How are relational structure mapped into neural architectures? Principles: What are the underlying principles guiding the system? For instance: similarity measures to find analogies between relational structures such as plans, "minimum encoding" and other approaches to hypothesis evaluation, the employment of additional knowledge used to constrain hypothesis generation, mechanisms for retrieval or adapation of prior plans or explanations. Theory: What theories have supported the development of the system? For instance, computational complexity theory, algebraic semantics, Bayesian and decision theory, psychological learning theories, etc. Implementation: What indexing, hashing, or programming methodologies have been used to improve performance and why? For instance, optimizing the performance for commonly encountered problems (ala CYC). The committee is soliciting papers that fall into one of three categories: Theoretical papers are encouraged that define a new theoretical framework, prove results concerning programs which carry our constructive or relational learning, or compare theoretical issues in various frameworks. Implementation papers are encouraged that provide sufficient details to allow reimplementation of learning algorithms, and discuss the key time/space complexity details motivating the design. Experimentation papers are encouraged that compare methods or address hard learning problems, with appropriate results and supporting statistics. WORKSHOP COMMITTEE Wray Buntine (RIACS and NASA Ames Research Center), wray at ptolemy.arc.nasa.gov Stephen Muggleton (Turing Institute), steve at turing.ac.uk Michael Pazzani (Univ. of California, Irvine), pazzani at ics.uci.edu Ross Quinlan (Univ. of Sydney), quinlan at cs.su.oz.au SUBMISSION DETAILS Those wishing to present papers at the workshop should submit a paper or an extended abstract, single-spaced on US letter or A4 paper, with a maximum length of 4000 words. Those wishing to attend but not present papers should send a 1 page description of their prior work and current research interests. Three copies should be sent to arrive by March 1, 1991 to: Michael Pazzani ICS Department University of California Irvine, CA 92717 USA Formats and deadlines for camera-ready copy will be communicated upon acceptance. --------------------------------------------------------------------------- MACHINE LEARNING IN ENGINEERING AUTOMATION Engineering domains present unique challenges to learning systems, such as handling continuous quantities, mathematical formulas, large problem spaces, incorporating engineering knowledge, and the need for user-system interaction. This session concerns using empirical, explanation-based, case-based, analogical, and connectionist learning techniques to solve engineering problems such as design, planning, monitoring, control, diagnosis, and analysis. Papers should describe new or modified machine learning systems that are demonstrated with real engineering problems and overcome limitations of previous systems. Papers should satisfy one or more of the following criteria: o Present new learning techniques for engineering problems. o Present a detailed case study which illustrates shortcomings preventing application of current machine learning technology to engineering problems. o Present a novel application of existing machine learning techniques to an engineering problem indicating promising areas for applying machine learning techniques to engineering problems. Machine learning programs being used by engineers must meet complex requirements. Engineers are accustomed to working with statistical programs and expect learning systems to handle noise and imprecision in a reasonable fashion. Engineers often prefer rules and classifications of events that are more general than characteristic descriptions and more specific than discriminant descriptions. Engineers have considerable domain expertise and want systems that enable application of this knowledge to the learning task. This session is intended to bring together machine learning researchers interested in real-world engineering problems and engineering researchers interested in solving problems using machine learning technology. We welcome submissions including but not limited to discussions of machine learning applied to the following areas: o manufacturing automation o design automation o automated process planning o production management o robotic and vision applications o automated monitoring, diagnosis, and control o engineering analysis WORKSHOP COMMITTEE Bradley Whitehall (Univ. of Illinois) Steve Chien (JPL) Tom Dietterich (Oregon State Univ.) Richard Doyle (JPL) Brian Falkenhainer (Xerox PARC) James Garrett (CMU) Stephen Lu (Univ. of Illinois) SUBMISSION DETAILS Submission format will be similar to AAAI-91: 12 point font, single-spaced, text and figure area 5.5" x 7.5" per page, and a maximum length of 4000 words. The cover page should include the title of the paper, names and addresses of all the authors, a list of keywords describing the paper, and a short (less than 200 words) abstract. Only hard-copy submissions will be accepted (i.e., no fax or email submissions). Four (4) copies of submitted papers should be sent to: Dr. Bradley Whitehall Knowledge-Based Engineering Systems Research Laboratory Department of Mechanical and Industrial Engineering University of Illinois at Urbana-Champaign 1206 West Green Street Urbana, IL 61801 ml-eng at kbesrl.me.uiuc.edu Formats and deadlines for camera-ready copy will be communicated upon acceptance. From hinton at ai.toronto.edu Tue Dec 18 13:17:30 1990 From: hinton at ai.toronto.edu (Geoffrey Hinton) Date: Tue, 18 Dec 1990 13:17:30 -0500 Subject: Research Associate job Message-ID: <90Dec18.131735edt.1033@neuron.ai.toronto.edu> **** DO NOT FORWARD TO OTHER BBOARDS OR MAILING LISTS **** RESEARCH ASSOCIATE POSITION AT THE UNIVERSITY OF TORONTO SALARY: $40,000 - $50,000 per annum The University of Toronto is an equal opportunity employer. THE JOB: The research associate will collaborate with graduate students on two large neural network projects, and will be expected to do a considerable amount of programming in C. The first project involves hand-printed character recognition and the second involves using a data-glove to produce speech. The funding is specifically for these two projects so the job is not suitable for someone who wishes to pursue research on other topics. The job will initially be for one year with the possibility of renewal for at least one more year. Since this is a non-permanent research position, it can be given to a non-Canadian. We are looking for someone who can start work in the spring of 1991. THE GROUP: The connectionist research group in the Department of Computer Science is directed by Geoffrey Hinton and consists of 10 graduate students (Sue Becker, Michelle Craig, Sidney Fels, Conrad Galland, Radford Neal, Steve Nowlan, Tony Plate, Evan Steeg, Chris Williams and Rich Zemel), a research associate (usually), a research programmer (Drew van Camp) and an administrator (Carol Plathan). The research focusses on developing new learning procedures and new applications for neural networks. We have our own four-processor silicon graphics machine (about 90MIPS) plus 10 sun 3/50 workstations that are used as graphics terminals. PREREQUISITES: Applicants must have a completed (or very nearly completed) PhD that involved simulations of learning in artificial neural networks. Since there are already several strong candidates for this job, we are not interested in applicants who do not already have extensive practical experience of neural networks. Experience with character recognition, speech production, or data-gloves would be valuable. HOW TO APPLY: The deadline is Jan 14 1991. Send your Curriculm Vitae (including a summary of your thesis research), copies of your one or two best papers, the names, phone numbers and email addresses of two or three references, and the date you would be available to start to: Carol Plathan, Department of Computer Science, University of Toronto, 10 Kings College Road, Toronto, Ontario M5S 1A4 Canada You could also send the information by email to carol at ai.toronto.edu. **** DO NOT FORWARD TO OTHER BBOARDS OR MAILING LISTS **** From tenorio at ecn.purdue.edu Tue Dec 18 14:46:43 1990 From: tenorio at ecn.purdue.edu (Manoel Fernando Tenorio) Date: Tue, 18 Dec 90 14:46:43 EST Subject: Connectionism vs AI? In-Reply-To: Your message of Mon, 17 Dec 90 22:27:07 D. Message-ID: <9012181946.AA19745@dynamo.ecn.purdue.edu> Bcc: -------- From: Tony Chan Concerning this and related questions, Elizabeth Preston [Sun, 16 Dec 90 14:41:02 EST] mentioned, "a great deal of ink and hot air has been expended on it in various public forums over the last few years. ... I don't see that the question is answerable at this stage in the development and theoretical understanding of the two fields. In fact, I'm not sure it's even askable." I would not ask that question; instead, I would ask a larger question which, if satifactorily answered, would make the previous question obsolete: Should there exist one formal (mathematical) unifying paradigm within which all investigations concerning the artificial re-creation of intelligence are to be carried out? If the answer is yes, as I believe, an immediate question that should be asked is the legitimacy of the current ad hoc separation of these related areas: artificial intelligence, connectionism, pattern recognition, cognitive science, cybernetics, machine learning, etc.? In fact, not only there should be one---there is one! So the answer is affirmative in existential sense and in constructive sense. I refer interested readers to the communication of Lev Goldfarb [Thu, 27 Sep 90 16:28:09 EDT]. I am not sure about a mathematical paradigm for all investigation concerning the art. intelligence. I am not really sure we can define what these are. If we restrict ourselves to inferences, in a paper soon to come out, we will shown that current NN models are of the equivalent class of the AI inference models. They can both be reduce to a graph grammar ( using cathegory theory) to a simple rewriting system of equivalent power. Neither will give much better results than a currently available for inference. We need a radical departure from these to accomplish "intelligent behavior" on this narrower sense. M. F. Tenorio. From honavar at iastate.edu Tue Dec 18 16:08:02 1990 From: honavar at iastate.edu (honavar@iastate.edu) Date: Tue, 18 Dec 90 15:08:02 CST Subject: Tech report available by ftp Message-ID: <9012182108.AA10148@iastate.edu> The following technical report is available in postscript form by anonymous ftp (courtesy Jordan Pollack of Ohio State Univ). Comments on the paper are welcome (please direct them to honavar at iastate.edu) ---------------------------------------------------------------------- Symbol Processing Systems, Connectionist Networks, and Generalized Connectionist Networks Vasant Honavar Leonard Uhr Department of Computer Science Computer Sciences Department Iowa State University University of Wisconsin-Madison Technical Report #90-24, December 1990 Department of Computer Science Iowa State University, Ames, IA 50011 Abstract Many authors have suggested that SP (symbol processing) and CN (connectionist network) models offer radically, or even fundamentally, different paradigms for modeling intelligent behavior (see Schneider, 1987) and the design of intelligent systems. Others have argued that CN models have little to contribute to our efforts to understand intelligence (Fodor & Pylyshyn, 1988). A critical examination of the popular characterizations of SP and CN models suggests that neither of these extreme positions is justified. There are many advantages to be gained by a synthesis of the best of both SP and CN approaches in the design of intelligent systems. The Generalized connectionist networks (GCN) (alternately called generalized neuromorphic systems (GNS)) introduced in this paper provide a framework for such a synthesis. ______________________________________________________________________________ You will need a POSTSCRIPT printer to print the file. To obtain a copy of the report, use anonymous ftp from cheops.cis.ohio-state.edu (here is what the transaction looks like): % ftp ftp> open cheops.cis.ohio-state.edu Connected to cheops.cis.ohio-state.edu. 220 cheops.cis.ohio-state.edu FTP server (Version blah blah) ready. Name (cheops.cis.ohio-state.edu:yourname): anonymous 331 Guest login ok, send ident as password. Password: anything 230 Guest login ok, access restrictions apply. ftp> cd pub/neuroprose 250 CWD command successful. ftp> bin 200 Type set to I. ftp> get honavar.symbolic.ps.Z 200 PORT command successful. 150 Opening BINARY mode data connection for honavar.symbolic.ps.Z (55121 bytes). 226 Transfer complete. local: honavar.symbolic.ps.Z remote: honavar.symbolic.ps.Z 55121 bytes received in 1.8 seconds (30 Kbytes/s) ftp> quit 221 Goodbye. % uncompress honavar.symbolic.ps.Z % lpr honavar.symbolic.ps From enorris at gmuvax2.gmu.edu Tue Dec 18 16:32:09 1990 From: enorris at gmuvax2.gmu.edu (Gene Norris) Date: Tue, 18 Dec 90 16:32:09 -0500 Subject: Accretional networks Message-ID: <9012182132.AA10355@gmuvax2.gmu.edu> A modest terminological proposal -- networks that change their connectivity during learning usually do so by adding weights between existing units or by adding new units (and therefore, new weights). Such networks can be called **accretional**; those that add only weights between existing units are weight-accretional, and those that add units are unit-accretional. (A network that changes its learning algorithm while learning might be called intelligence-accretional, or just plain intelligent (:-) ). If one has devised a learning algorithm that prunes networks of connections or units, these would, by extension, be called deaccretional nets (if one wanted to make a distinction). Anyway, it's a fresh, descriptive term with no (for me) connotations: good qualities for a technical term to have. --Gene Norris CS Dept George Mason University Fairfax, VA 22032 (703)323-2713 enorris at gmuvax2.gmu.edu FAX: 703 323 2630 From jbower at smaug.cns.caltech.edu Tue Dec 18 16:39:24 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Tue, 18 Dec 90 13:39:24 PST Subject: AI, NN, and CNS Message-ID: <9012182139.AA04526@smaug.cns.caltech.edu> Concerning the AI and connectionism debate. I think that this debate should take place without reference to biology. While AI by conviction has had very little to do with the structure of the nervous system, it is not at all clear that connectionism is much different. Even the "neural nets" subfield, if that is what it is, is at best vaguely connected to real biology. In my view, the modern interest in either connectionism or neural nets was not even particularly biologically inspired. As a biologist it seems to me that both have emerged from their own antecedents. Beyond origins, however, all one has to do is attend any of the annual neural net meetings to see that biology is currently at best a handmaiden to the overall effort. At worst, it is used as a justification for preconceived notions. This is not to say that there are not some people in the field who are committed to understanding what little is now known about biology, but they are few and far between. Instead, the evolution of neural networks and connectionism appears to be taking their own directions under their own priorities with real biological constraints having little effect on either. More globally, it has been pointed out before that historical attempts to understand human intelligence have always been cast in terms of the most sophisticated technology of the day. The Greeks borrowed from the technology of aqueducts in ascribing mental processes to the flow of bodily fluids. Descartes thought he thought using machines and mechanical forces. Sherrington was inspired by telephone switchboards, while theorists of the 60's and 70's considered the brain to be a digital computer. Today we would discount each of these claims believing that the brain is a parallel distributed processing device. It is important to realize, however, that these earlier speculators did not think the mechanism of mental function was similar to their favorite machine, they, like we, thought it was actually just a more complicated version of that machine. In conclusion, in my view, AI and connectionism (neural nets) should work out their own definitions on their own merits without reference to biology. Then, if there is a difference, both should fight it out based on real world performance. At that point, some biologist will probably compare the results to the abilities of some invertebrate somewhere making it clear, yet again, that we have missed the mark. Jim Bower jbower at smaug.cns.caltech.edu From goldfarb%unb.ca at UNBMVS1.csd.unb.ca Tue Dec 18 22:10:55 1990 From: goldfarb%unb.ca at UNBMVS1.csd.unb.ca (om Lev Goldfarb) Date: Tue, 18 Dec 90 23:10:55 AST Subject: Which connectionism vs which AI? Message-ID: It appears that, when discussing the relationship between NN and AI, an undue legitimacy is often granted to these two quite tentative and inadequate *formal paradigms*. The NN lacks adequate "self-programmability", while the propositional model cannot practically facilitate learning from the environment, i.e. it cannot facilitate the discovery of new useful features (new symbols) or even the recognition of "primitive" patterns. After "a great deal of ink and hot air has been expanded" (Beth Preston, Connectionism vs AI), it should be quite clear that the two formal models *in their present form* cannot naturally, or directly (in mathematical sense), be integrated into one model, in spite of the attempts by some to conveniently ignore this fact. On the other hand, if, when talking about the two "paradigms", one is not referring to the underlying *mathematical* models, then the necessity of integrating the two paradigms should be apparent and the above "great deal of ink and hot air" can easily be understood. --Lev Goldfarb From Scott.Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU Tue Dec 18 21:23:13 1990 From: Scott.Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF-PMAX.SLISP.CS.CMU.EDU) Date: Tue, 18 Dec 90 21:23:13 EST Subject: ontogenesis and synaptogenesis In-Reply-To: Your message of Mon, 17 Dec 90 09:57:51 -0800. <9012171757.AA25001@crl.ucsd.edu> Message-ID: Liz, Thanks for posting that information on synaptogenesis. It is extremely valuable to have some knowledgeable people out there helping to filter and interpret the neural-development literature for the rest of us. I'm not sure to what extent your post was inspired by some of the recent discussion of constructive/destructive algorithms (or "ontogenic", if you prefer). You're probably way ahead of me on this, but I just wanted to mention that the obvious mapping of computational theories into neuron-stuff isn't necessarily the only mapping or the best one. For example, some people automatically assume that what we call a "unit" must be implemented as a neuron, and what we call a "weight" must be implemented as an adjustable synapse. But a "unit" might instead correspond to some piece of the dendritic tree that happens to behave in a nonlinear way; on the other hand, what we call a "unit" might be implemented as a whole cluster of cooperating neurons. The situation with constructive/destructive algorithms is similar. The obvious implementation of constructive algorithms would involve growing (or recruiting) new neurons and adding them to the active network through the creation of new synapses; destructive algorithms would presumably involve elimination of existing synapses and neurons. But that isn't the only possible way of mapping these ideas to real neural systems. For example, I usually describe the Cascade-Correlation architecture as selecting new units out of a pool of candidates and "adding" them to the active network. But it is probably better to think of this event as a sort of phase transition, which I jokingly call "tenure". Before tenure, the candidate units receive a full complement of inputs, and the input weights are adjusted freely to maximize some measure of the candidate's potential usefulness (currently, the degree of correlation with the remaining error). Before tenure, these candidate units are either silent or their outputs are ignored. After tenure, the inputs are frozen (or at least much less plastic), but now the rest of the net pays attention to that unit's output. This can be a purely functional change; there doesn't really have to be any visible change in the physical topology of the network. It would be very interesting to know whether anything like this phase transition actually occurs in individual neurons, but I have no idea how one would go about looking for such a thing. -- Scott Fahlman From jose at learning.siemens.com Wed Dec 19 10:26:17 1990 From: jose at learning.siemens.com (Steve Hanson) Date: Wed, 19 Dec 90 10:26:17 EST Subject: AI, NN, and CNS Message-ID: <9012191526.AA21107@learning.siemens.com.siemens.com> Jim: It is true that connectionism and AI have tended to compete in terms modeling. Both however require something to model. I think your view on neuroscience's relation to connectionist modeling is somewhat narrow and a bit shrill. AI researchers have tended to focus on a vauge notion of "intelligence". Connectionists have tended to be less opportunistic about the modeling piece parts they have chosen but have modeled just about anything standing still. Is it all biologically motivated?--heavens no! Why should it be? Is any of it?--of course (there are several good volumes around documenting this: "Neural Modeling", Koch & Segev, 1989; "Conectionist Modeling and Brain Function", Hanson & Olson, 1990; "Connectionism and Neuroscience", Gluck & Rumelhart, 1990 and several others I forget their titles..sorry). But computational modeling can occur at any level of interest and be productive and valid. That it does not model certain cells and circuits or jim`s favorite cells or circuits does not invalidate the oppportunity for such modeling to happen and have continuity with other relevant system level modeling. It's a two-way street, both modelers and experimentalists have a responsibility to constrain and enlighten each other--if they don't then complaints like yours have a self-perpetuating flavor to them. The brain's a big place and as far as I can tell has plenty of room for lots of system level speculation. Connectionist Modelers are studying all sorts of (yes even wooly ones) system level interactions and possible mechanims for various kinds of function. Sejnowski and Churchland had a nice Science article where they attempt to lay out possible relations between "simplified" and "realistic" neural models. I wrote a paper that appeared recently in Behavioral and Brain Science on what I thought was the relation between AI and connectionism--many people responded in kind. I refer you to those papers for more detail in order to keep this conversation short. I think it is important at this point not to polarize this issue by either assuming there is only one unique way to characterize neural computation or worse that the details of the brain don't matter. Steve From gluck%psych at Forsythe.Stanford.EDU Wed Dec 19 10:30:08 1990 From: gluck%psych at Forsythe.Stanford.EDU (Mark Gluck) Date: Wed, 19 Dec 90 07:30:08 PST Subject: Preprint: Stimulus Sampling & Distributed Representations Message-ID: <9012191530.AA11533@psych> PRE-PRINT AVAILABLE: Stimulus Sampling and Distributed Representations in Adaptive Network Theories of Learning Mark A. Gluck Department of Psychology Stanford University [To appear in: A. Healy, S. Kosslyn, & R. Shiffrin (Eds.), Festschrift for W. K. Estes. NJ: Erlbaum, 1991/in press] ABSTRACT: Current adaptive network, or "connectionist", theories of human learning are reminiscent of statistical learning theories of the 1950's and early 1960's, the most influential of which was Stimulus Sampling Theory, developed by W. K. Estes and colleagues (Estes, 1959; Atkinson & Estes, 1963). This chapter reviews Stimulus Sampling Theory, noting some of its strengths and weaknesses, and compares it to a recent network model of human learning (Gluck & Bower, 1986, 1988a,b). The network model's LMS learning rule for updating associative weights represents a significant advance over Stimulus Sampling Theory's more rudimentary learning procedure. In contrast, Stimulus Sampling Theory's stochastic scheme for representing stimuli as distributed patterns of activity can overcome some limitations of network theories which identify stimulus cues with single active input nodes. This leads us to consider a distributed network model which embodies the processing assumptions of our earlier network model but employs stimulus-representation assumptions adopted from Stimulus Sampling Theory. In this distributed network, stimulus cues are represented by the stochastic activation of overlapping populations of stimulus elements (input nodes). Rather than replacing the two previous learning theories, this distributed network combines the best established concepts of the earlier theories and reduces to each of them as special cases in those training situations where the previous models have been most successful. _________________________________________________________________ To request copies, send email to: gluck at psych.stanford.edu with your hard-copy mailing address. Or mail to: Mark A. Gluck, Department of Psychology, Jordan Hall, Bldg. 420, Stanford Univ., Stanford, CA 94305-2130 From GOLDFARB%UNB.CA at UNBMVS1.csd.unb.ca Wed Dec 19 20:24:06 1990 From: GOLDFARB%UNB.CA at UNBMVS1.csd.unb.ca (Lev Goldfarb) Date: Wed, 19 Dec 90 21:24:06 AST Subject: On blurring the gap between NN and AI Message-ID: Jay McClelland: Regarding Worth's query about a possible fundamental opposition between connectionist and other approaches to AI: I do not think there need be such an opposition. That is, I think one can be a connectionist without imagining that any such opposition exists. I do not understand how one can decide or "imagine" whether there need or need not be such an opposition. In fact, as I have mentioned in my last correspondence, such "opposition" between the corresponding mathematical models *simply exists*. Wishful thinking apart, if we do not want to expand some more "hot air", we should keep in mind what John von Neumann said half a century ago: The sciences do not try to explain, they hardly even try to interpret, they mainly make models. By a model is meant a mathematical constract which, with the addition of certain verbal interpretations, describes observed phenomena. The justification of such a mathematical construct is solely and precisely that it is expected to work. Blurring the existing tentions between the two models (and I mean formal models) helps to create scientifically very unproductive state of euphoria, especially at this very critical initial stage in the development of intelligent systems. I hope, that the Scientific American article will not cater to the very popular "quest for euphoria". --Lev Goldfarb From 35C4UB2 at CMUVM.BITNET Wed Dec 19 14:18:26 1990 From: 35C4UB2 at CMUVM.BITNET (Ken Aizawa) Date: Wed, 19 Dec 90 14:18:26 EST Subject: Which connectionism vs which AI? In-Reply-To: Your message of Tue, 18 Dec 90 23:10:55 AST Message-ID: Lev Goldfarb writes (Goldfarb,Tue,18 Dec 90 23:10:55 AST): >the propositional model cannot >practically facilitate learning from the environment, i.e. it >cannot facilitate the discovery of new useful features (new symbols) >or even the recognition of "primitive" patterns. Don't the Bacon programs by Langley, Simon, Bradshaw, et. al. count as propositional models ? And don't they postulate new variables or constants (new symbols) that enter into laws of nature ? So, isn't this a counterexample to your characterization ? Ken Aizawa From worth at park.bu.edu Wed Dec 19 17:44:09 1990 From: worth at park.bu.edu (Andrew J. Worth) Date: Wed, 19 Dec 90 17:44:09 -0500 Subject: AI, NN, and CNS Message-ID: <9012192244.AA20600@park.bu.edu> The main reason I posted the original "Connectionism vs. AI" question was to imply that one should not gloss over the differences between the two philosophies when combining them. Many of the subsequent postings illustrated the problem with the definitions used in the question. I believe that the question I was thinking of could have been asked more succinctly by comparing the assumptions that lead to "the explicitly symbolic nature of many AI endeavors" with a connectionist approach that does not hold such assumptions. But this might unleash the symbol grounding debate. In lieu of that, I would like to respond to Jim Bower's posting; do AI and connectionism have anything to do with biology? Again, it depends on your definitions. "Computational Neuroscience", as described by Sejnowski, Koch, and Churchland [1] is an attempt to bring together not just biology, but also psychology and other fields to explain information processing in the brain using computational models. The results are "connectionist" models where emergent properties (not explicit symbols and rules) become important. Bower's assertion that many neural networks have little to do with biology expresses a regrettable fact. But as Bower mentioned and as Sejnowski et al. show, not all of connectionism ignores biology. Other bodies of work that do not ignore biology are those by Grossberg, et al. on vision and motor control [2,3]. Perhaps the most painless introduction to some of these ideas can be found in [4]. A practical demonstration of some of the vision work can be seen in [5]. It seems to me that the lure of connectionism is a haunting whispered promise to go where no "AI" has gone before. I consider attention to biology in general, and Grossberg et al's techniques in particular, as steps in the right direction. [1] T.J. Sejnowski, C. Koch, and P.S. Churchland, Computational Neuroscience, Science, v.241, pp.1299-1306, 9 September 1988. [2] S. Grossberg and M. Mingolla, Neural Dynamics of Perceptual Grouping: Textures, Boundaries and Emergent Segmentations, Perception & Psychophysics, v.38(2), 141-171, 1985. [3] S. Grossberg & M. Kuperstein, Neural Dynamics of Adaptive Sensory- Motor Control, New York, Pergamon Press, 1989. [4] S. Grossberg, Why do cells compete? UMAP Unit 484. The UMAP Journal, V.III, No.1 (Educational Development Center, 0197-3622/82/010101.) 1982. [5] S.M. Lehar, A.J. Worth, & D.N. Kennedy, Application of the Boundary Contour/Feature Contour System to Magnetic Resonance Brain Scan Imagery, Proc. of the International Joint Conf. on Neural Networks, v.I, p.435-440, 1990. Andrew J. Worth worth at park.bu.edu Cognitive & Neural Systems Prog. (617) 353-6741 Boston University (617) 353-7857 (CAS Office) 111 Cummington St. Room 244 (617) 353-5235/6742 (CNS Grad Offices) Boston, MA 02215 From jbower at smaug.cns.caltech.edu Thu Dec 20 02:59:29 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Wed, 19 Dec 90 23:59:29 PST Subject: AI, NN, CNS (central nervous system) Message-ID: <9012200759.AA06211@smaug.cns.caltech.edu> In response to Steve Hanson's comment on my comment. I think that Steve's remarks largely miss my primary point which is simply that an assumed relationship between biology and connectionism/neural networks should not be used as a distinguishing feature with respect to AI. There is simply no strong recent historical evidence for this relationship and the field itself is evolving independently of any association with real biological networks. While this may be unfortunate, it is not necessarily bad, and it is certainly pretty predictable. The points Steve does raise are potentially interesting, although it is not clear to me that they are of general interest to readers of this network for the reasons just stated. Briefly, however, Steve seems to be confusing what I would refer to as Computational Neuroscience, with Cognitive Neuroscience. The two are quite different in their intent. The Koch and Segev book, for example, with which I am quite familiar as codirector of the course on which the book is based and having written two of its chapters, describes models intended to shed light on structure/function relationships within the nervous system. This is Computational Neuroscience. Using modeling to explore more abstract mental or cognitive functions is Cognitive Neuroscience from which connectionism largely grew and to which it mostly relates. Thus, while computational modeling can be applied at any level of interest, where it may or may not be "productive and valid", the real question is productive and valid for what. If you are interested in cognitive neuroscience, by all means make models. Remember, however, there is no necessary relationship between these models and the way the brain actually does things. Further, the largest limitation on this connection is liable to be our poor understanding of cognitive behavior to begin with. Thus, in the absence of any example to the contrary, I reject the idea that there is necessarily a continuity between all levels of modeling no matter how abstract or detailed. And this obviously has nothing to do with my favorite cells or circuits which was a silly and inappropriate remark to begin with. Briefly, I would also like to comment on Steve's tautology regarding the necessity of interactions between experimentalists and modelers. As I believe he knows, a commitment to this interaction is at the very base of my own research effort. In fact, it is my view that this interaction should be so tight that modelers are also experimentalists. Further, in order for data to really constrain models, the models should generate data obtainable with experimental techniques. Practically, this is far easier if they are structurally realistic. This is one of the reasons I doubt the usefulness of abstract models for understanding brain function. In addition, most abstract models are intent on proving that somebodies idea is at best plausible and at worst correct. In my view, the real value of a tight connection between models and experiment is the possibility that brain structure itself will provide new ideas as to functional organization. It is also exceedingly ironic that Steve warns against assuming that the details of the brain don't matter. Of course, it is precisely my point that the details matter a great deal. In fact, this is the reason I assert that connectionist modeling is not biologically relevant. The structures of the vast majority of these models bear virtually no resemblance to the actual structure of the brain. They might even prove to be no closer than complicated switchboards or aqueducts. Finally, a general comment that relates to this debate as well as the debate about AI versus connectionist approaches. There is clearly a strong tendency to deal with potential conflicts by simply declaring that we are all part of one big happy family. While this might sooth egos, relieve self doubt, and provide new funding opportunities, I think it is important to resist the temptation. In the process of amalgamation important distinctions can be washed out. Jim Bower jbower at smaug.cns.caltech.edu From hendler at cs.UMD.EDU Thu Dec 20 09:18:09 1990 From: hendler at cs.UMD.EDU (Jim Hendler) Date: Thu, 20 Dec 90 09:18:09 -0500 Subject: More on AI vs. NN Message-ID: <9012201418.AA01724@dormouse.cs.UMD.EDU> I guess I feel compelled to add my two cents to this. Here's a copy of an editorial I wrote for a special issue of the journal Connection Science (Carfax Publ.) The issue concerned models that combined connectionist and symbolic components: ---------------------------- On The Need for Hybrid Systems J. Hendler (From: Connection Science 1(3), 1989 ) It is very easy to make an argument in favor of the development of hybrid connectionist/symbolic systems from an engineering (or task-oriented) perspective. After all, there is a clear and present need for developing systems which can perform both ``perceptual'' and ``cognitive'' tasks. Some examples include: Planning applications where recognition of other agents' plans must be coupled with intelligent counteractions, Speech understanding programs where speech processing, which has been most successful as a signal processing application, needs to be coupled with syntactic and semantic processing, Automated manufacturing or testing applications where visual perception needs to be coupled with expert reasoning, and Expert image processing systems where line or range detectors, radar signal classifiers, unknown perspective projections, quantitative image processors, etc. must be coupled with top-down knowledge sources such as maps and models of objects. To provide systems which can provide both ``low-level'' perceptual functionality as well as demonstrating high-level cognitive abilities we need to capture the best features of current connectionist and symbolic techniques. This can be done in one of four ways: We can figure out a methodology for getting traditional AI systems to handle image and signal processing, to handle pattern recognition, and to reason well about perceptual primitives, We can figure out a methodology for getting connectionist systems to handle ``high-level'' symbol-processing tasks in applied domains. This might involve connectionist systems which can manipulate data structures, handle variable binding in long inference chains, deal with the control of inferencing, etc., We can work out a new ``paradigm,'' yet another competitor to enter the set of possible models for delivering so-called intelligent behavior, Or, we can take the current connectionist systems and the current generation of AI systems and produce hybrid systems exploiting the strengths of each. While the first three of these are certainly plausible approaches, and all three are currently driving many interesting research projects, they require major technological breakthroughs, and much rethinking of the current technologies. The fourth, building hybrid models, requires no major developments, but rather the linking of current technologies. This approach, therefore appears to provide the path of least resistance in the short term. From a purely applied perspective, we see a fine reason to pursue the building of hybrid models. If this were the only reason for building hybrid models, and it is a strong one, it would legitimize much research in this area. The purpose of this editorial however, and in fact the rationale behind the editing of this special issue on hybrid models, is to convince the reader that there is more to hybrid models than simply a merging of technologies for the sake of building new applications: In particular, that the development of hybrid models holds major promise for bettering our understanding of human cognition and for helping to point the way in the development of future cognitive modeling techniques. This claim is based on facing up to reality: neither the current AI nor the current connectionist paradigms appear to be sufficient for providing a basic understanding of human cognitive processing. I realize this is quite a contentious statement, and I won't try to defend it rigorously in this short article. Instead, I'll try to outline the basic intuition behind this statement. Essentially, the purely symbolic paradigm of much of AI suffers from not being grounded in perception. Many basic types of cognitive processing, particularly those related to vision and the other senses, have been formed by many generations of evolution. While it is possible that a symbolic mechanism could duplicate the abilities of these ``hard-wired'' systems, it seems unlikely. Higher level cognitive abilities, such as understanding speech or recognizing images, which do not attempt to use low-level models may be doomed to failure. Consider, for example, the evidence for categorization errors and priming confusions in humans. Is this evidence of some sort of weakness in the processing system, or is it a result of the very mechanisms by which perceptual information processing proceeds in humans? If, as many believe, the latter is true, then it would appear to be the case that the apparatus by which humans perform perceptual categorization forces categories to have certain properties. If this is the case, then ability of humans to perform very broad generalizations and to recognize commonalities between widely divergent inputs is probably integrally related to this perceptual apparatus. If so, an attempt to model human understanding which doesn't take the ``limitations'' of this perceptual categorization mechanism seriously may be doomed to failure. Further, it may even be the case that any attempt to use a more perfect scheme for categorization will miss having this critical property. Thus, understanding perceptual processing, as it is implemented in the brain, may be crucial to an understanding of cognition as performed by the human problem solver. The connectionist approach, sometimes called the subsymbolic paradigm, suffers from a related problem. While current research appears to indicate that this approach may be better for modeling the lower level cognitive processes, there seems to be something crucially different between human cognition and that of other animals. It seems unlikely that this difference can be captured by a purely ``brain-based'' explanation. Human problem solving requires abilities in representation (the often cited issue of ``compositionality'' being one) and in symbol manipulation, which are currently beyond the scope of the connectionist approach (except in very simplified cases). While it is possible that brain size itself explains the differences between human and animal cognition, many researchers seem to believe that the answer requires more than this. Explaining human thinking requires more than simply explaining the purely associative mechanisms found in most mammals understanding these mechanisms is necessary for a deeper understanding of human cognition, but it is not sufficient . Thus a connectionist science which addresses the associative learning seen in the brain, without regard for the cognitive abilities resulting from that learning, is inadequate for a full understanding of human cognition. Unfortunately understanding and modeling human cognitive processing in a way that takes both abilities and implementation into account is not an easy task. To solve this problem we will eventually need to understand both the architecture of the brain and the ``programs'' running on that architecture. This interconnection between implementational constraints on the one hand, and functional requirements on the other, puts many bounds on the set of possible models which could truly be considered as the basis of human intelligence. But how do we probe the models lying within these bounds? What would they look like? Can they be based on current connectionist techniques? Will they function in a manner similar to current AI models? We know of no ``correct'' research paradigm for studying these problems: Connectionist models clearly must be pursued for a deeper understanding of the ``firmware'' of thought; traditional AI must be pursued to give us insight into the functional requirements of thought. But, it is my contention that a third path must also be followed: To be able to gain a true insight into what implementationally correct, cognitively robust models of human cognition will look like, we need to study models which try to connect the two paradigms. Hybrid models, rather than being viewed simply as a short term engineering solution, may be crucial to our gaining an understanding of the parameters and functions of biologically plausible cognitive models. From this understanding we might hope to see the development of a new, and potentially more correct, paradigm for the studying of ``real,'' as opposed to artificial, intelligence. From INTS%MUSICB.MCGILL.CA at BITNET.CC.CMU.EDU Thu Dec 20 09:27:43 1990 From: INTS%MUSICB.MCGILL.CA at BITNET.CC.CMU.EDU (Tom Shultz, Dept. of Psychology, McGill Univ.) Date: Thu, 20 Dec 90 09:27:43 EST Subject: ontogenesis Message-ID: <20DEC90.10219129.0061.MUSIC@MUSICB.MCGILL.CA> Relating the ongoing discussion of ontogeny in neural nets to her recent review of neural correlates of language development, Liz Bates writes: Synaptogenesis does NOT continue across the (human/primate) lifespan, at least not on any kind of a large or interesting scale. At least after a huge burst in synaptogenesis between (roughly) human postnatal months 6 - 24. Apparently, there is somewhat of a consensus that, through most of human development, a dieback model accounts for many more brain changes than does a synaptogenic model. However, there are some dissenting views and evidence on this. For example, Greenough and his colleagues at U. of Illinois have evidence that rats in enriched environments add around 20% more synapses than control rats. This is a process they call synapse-on-demand and it apparently occurs throughout rat life, not only in early development. And, as Scott Fahlman points out, algorithms like Cascade- Correlation, although often described as performing recruitment of new hidden units, can alternatively be described in ways that are more compatible with a synaptic change model. (Candidate hidden units are always in the network; it is just that they are not listened to until they start to do something useful.) This is essentially a caution that we shuld not impose a premature closure on the issue of how best to characterize either the connectionist or neurological literature on these issues of changing network topology. Tom Shultz From jlm+ at ANDREW.CMU.EDU Thu Dec 20 12:12:58 1990 From: jlm+ at ANDREW.CMU.EDU (James L. McClelland) Date: Thu, 20 Dec 90 12:12:58 -0500 (EST) Subject: On blurring the gap between NN and AI In-Reply-To: References: Message-ID: Regarding Goldfarb's comments 'on blurring': I agree that the models are different. All I was trying to address was the following notion, which seems to be implicit in some of the discussion: Either you believe cognition is really symbolic or your believe it's really subsymbolic. [Feel free to replace symbolic and/or subsymbolic with your favorite labels!] Neither of these views seems particularly productive. I'm with von Neumann -- I care about models that work. [What it means for a model to work depends on your purpose; see my previous post]. For the problems that interest me, connectionist models appear to work better than others. But this is not always the case. Some of my colleagues have gotten a long way with production systems. Which approach is right? Wrong question. Which approach is better? As in physics, some phenomena are best captured [modeled!] at the microstructure level, and others not. Which phenomena are best captured by each? We don't know; by choosing to use one or the other, we place our bets. Aspects of these views are presented at somewhat greater length in a paper: McClelland, J. L. (1988). Connectionist models and psychological evidence. Journal of Memory and Language, 27, 107-123. I'm bowing out of further discussion of these issues for the time being. Merry Christmas, Everyone! -- Jay McClelland From worth at park.bu.edu Thu Dec 20 14:54:39 1990 From: worth at park.bu.edu (Andrew J. Worth) Date: Thu, 20 Dec 90 14:54:39 -0500 Subject: AI, NN, and CNS (reference completion) Message-ID: <9012201954.AA28158@park.bu.edu> Sorry to bother you all again, but the complete reference in my last posting should have been: [5] S.M. Lehar, A.J. Worth, & D.N. Kennedy, Application of the Boundary Contour/Feature Contour System to Magnetic Resonance Brain Scan Imagery, Proc. of the International Joint Conf. on Neural Networks, San Diego, CA, v.I, p.435-440, June 1990. Andy. From der%psych at Forsythe.Stanford.EDU Thu Dec 20 14:56:52 1990 From: der%psych at Forsythe.Stanford.EDU (Dave Rumelhart) Date: Thu, 20 Dec 90 11:56:52 PST Subject: On blurring the gap between NN and AI In-Reply-To: Lev Goldfarb's message of Wed, 19 Dec 90 21:24:06 AST Message-ID: <9012201956.AA22941@psych> It seems to me that a number of issues are being confused in this discussion. One has to do with what AI is and another has to do with what "connectionism" is and why we might be interested it. First, with regard to AI it seems to me that there are at least three aspects. (1) Theoretical AI, (2) Experimental AI and (3) Applied AI. I take it that Theoretical AI is the development and analysis of algorithms and computational models which draw their inspiration from considerations of biological (especially) human intelligence. This strikes me as a special kind of applied mathematics. On this count, connectionist approaches, search based approaches, logic based approaches, problem space based approaches and many others all fall squarely in the realm of theoretical AI. These various approaches simply draw their inspiration from different aspects of natural intelligence and therefore find different mathematical formalisms more useful. I take it that experimental AI is the study of the algorithms and computational models developed in theoretical AI by experimental means -- that is through the development of computer simulations and implementations of the algorithms. If our models and algorithms could be fully analytically developed there would be no need for an experimental approach. Since most algorithms and computational models seem to be too complex and to interact with a world that is itself not easy to characterize we often resort to an experimental approach. Here AI differes from most (but not all) classical applied mathematical approaches. On the whole connectionist approaches employ the experimental method to about the same degree as most other AI approaches. To the degree that different AI approaches can be applied to the same problems it is certainly possible to compare different algorithms and computational approaches in terms of efficiency (on a machine of a particular type) quality of performance and other dimensions, but the effort is primarly one of analysis -- what are the properties of the systems under study. The third activity within AI, applied AI, is simply the applications of the AI techniques mentioned above to a real world problem. In this case, many practical issues intervene. Here we ask simply how well does the algorithm and procedure do on the area of application. The measurement criteria may involve difficulty of development as well as the quality of the performance. In this case, I would be suprised if a single approach was always better for all areas of application. At this level it is an empirical question. To summarize, as far as AI is concerned, it strikes me that the connectionist approach is one among many and partakes of most of the features of the other approaches. It may turn out that the connectionist approach is particularly well suited to particular kinds of applications, may be particularly elegant and may be nice in certain other ways, but beauty is often in the eye of the beholder. The question of whether there is a great divide between the symbolic and "sub-symbolic" approaches is one that I would rather leave to the philosophers. In any case, it has nothing to do with wether or not connectionist AI is a kind of AI. I simply can't think of any reasonable definition of AI that would exclude it. The second major point concerns the nature of "connectionism" itself. It should be noted that there are connectionist approaches to fields other than AI and in this case the connectionist approach cuts across several fields. In particular, there are connectionist approaches to psychology, to neuroscience, to linguistics and to other scientific domains. In these cases, the criteria for evaluation is rather different than for AI. In these fields we are interested in the degree to which models developed within the connectionist paradigm are useful in understanding, explaining or predicting empirical phenomena. These phenomena may involve explaining the behavior of people or other animals or in explaining the observations made by a neurobiologist when studying the brain. One of the hopes for the connectionist approach is that it will be able to provide a formalism for explaining the relatioinship between neural activity and behavior someday. The evidence that this will occur is, of course, not yet in. It is the job of connectionist researchers to do the necessary research, develop the necessary ideas and make the case to the scientific community at large that this is possible. Finally, I should say that this attempt to develop AI formalisms that have applicability to scientific model building is not unique to the connectionist approach. Many AI formalisms have been proposed as useful for explaining psychological or linguistic (but rarely neurobiological) phenomena. For a good example see Newell's new book on so-called Unified Theories. Sorry for the wordiness of this message. D. E. Rumelhart From pollack at cis.ohio-state.edu Thu Dec 20 20:16:15 1990 From: pollack at cis.ohio-state.edu (Jordan B Pollack) Date: Thu, 20 Dec 90 20:16:15 -0500 Subject: The details do not matter In-Reply-To: Jim Bower's message of Tue, 18 Dec 90 13:39:24 PST <9012182139.AA04526@smaug.cns.caltech.edu> Message-ID: <9012210116.AA00557@dendrite.cis.ohio-state.edu> Neurobiologists often write grossly inefficient programs, showing that Computer Science is merely a handservant to neuroscience, providing a catalog of algorithms, data-structures, and fancy jargon.:) Jim Bower writes: >> In conclusion, in my view, AI and connectionism (neural nets) should >>work out their own definitions on their own merits without reference to >>biology. Then, if there is a difference, both should fight it out based on >>real world performance. I fully agree with Jim Bower that biological plausibility is the wrong yardstick for the AI/Connectionist debate, but I refuse to have my fields trivialized. Maybe "Neural Networks" and "expert systems" are mere engineering, but Connectionism and AI are not! The subject of study is not brain, but MIND, and the mind is not a steam engine, or a telephone switching system, or a stored program computer, and it isn't a brain either. The dispute between AI and Connectionism is not about neural plausibility, or vague notions of intelligence, but about the PRINCIPLES OF ORGANIZATION of a computational system at the upper limit of biological complexity. With limited research resources and the impetus of its own history and living legends, AI has focused primarily on one particular organization of computation which is postulated to be able to support mind-like systems: the sequential/recursive rule-interpreter. There are very good reasons for this postulate, but no convincing argument that it is true or false. But there are several compelling deficits of rule-interpretation when taken literally as a model for mind: learnability (changes in rules are unstable) scaling (no systems with more than 1000's of rules) behavioral profile (no temporal behavior which is psych. plausible) parallelization (no easy way to distribute memory) biological plausibility (not evolutionarily or neurally justified) These deficits are all being addressed in multiple ways, both within AI and more broadly in the computer and cognitive sciences. Connectionism can be viewed as a unified attack on these problems, although any single connectionist model today only addresses one or two of these at a time. But the connectionist approach is based on a very different postulate, that the organization of the brain is extremely relevant to the organization of the mind. Again, there is no convincing argument that it is true or false. However, to compete against rule-interpretation as a literal theory of mind, connectionism cannot afford to be constrained by biological detail. So, while we may be interested in the coarse organization of the brain, such as its layers and columns, population codes, diameters and densities of connections, and so on, we ignore the "details" such as the lack of bidirectional synapses or the use of calcium in some synaptic modification event. Why? Because the complexity of a theory cannot be greater than the complexity of the mental faculty it purports to explain, or the theory would fail the parsimony test of science itself. On another note, this lack of attention to detail is also a conditioned response, for whenever a connectionist has crossed the line by taking a bold stand on a detailed model of some piece of the brain, X, they face the angry voices of the biologists: "No, you are doing it all wrong! Believe me, I'm the world's expert on X!" or "No, you can't model X yet, the `basic science' data isn't in and it will take me another 10 years of slicing and dicing ratbrains to get it! So, in conclusion: The AI/connectionist debate is about science, not engineering. What is involved is mind, not brain. The brain details don't matter as do its principles of organization. Jordan Pollack Assistant Professor CIS Dept/OSU Laboratory for AI Research 2036 Neil Ave Email: pollack at cis.ohio-state.edu Columbus, OH 43210 Fax/Phone: (614) 292-4890 From tsejnowski at UCSD.EDU Fri Dec 21 01:00:53 1990 From: tsejnowski at UCSD.EDU (Terry Sejnowski) Date: Thu, 20 Dec 90 22:00:53 PST Subject: AI, NN, CNS (central nervous system) Message-ID: <9012210600.AA15367@sdbio2.UCSD.EDU> In 1957 Hartline and Ratliff published a paper on "Inhibitory interaction of receptor units in the eye of Limulus" (J. Gen Physiol, 40: 357-376). They described a set of elegant experiments on lateral interactions between neighboring ommatidia and summarized their results in a model of an effective network of lateral inhibitory interactions. Their model would today be called a connectionist model -- in fact it was linear. Not only were they able to accurately summarize a lot of data, but they were able to use the model to explore the idea of contrast enhancement through lateral inhibition, something we take for granted today. This work led to a Nobel Prize for Hartline. In the intevening years this model has been elaborated in many interesting ways, including generalizations to time-dependent patterns of light, nonlinear interactions at extreme light levels, and the biophysical properties of noise in the photoreceptors. The essence of the orignal model, however, still stands even though these more elaborate and realistic models are more accurate and more complete. Bob Barlow at Syracuse has implemented a version of the model for the whole retina on their Connection Machine and is passing movies of the real world seen by the Limulus underwater through the simulated retina. The original Hartline-Ratliff model, however, is still a useful reference landmark toward which all these elaborations point. There is value in having an abstract, simplifying model to anchor the elaborations. This single example should be enough to 1) illustrate the utility of simplifying models in studying real biological problems and 2) underline the importance of paying careful attention to biological data when attempting to apply such models to other biological systems. It should also be noted that Hartline and Ratliff would not have been able to develop their model if the mathematics of linear networks had not already been established by mathematicians, physicists, and engineers, most of whom were not interested in biological problems. Without the development of a mathematics of nonlinear dynamical systems there will be no future models for future Hartlines and Ratliffs to apply to future biological problems. I find it encouraging that so many good scientists who are confronting so many difficult problems in psychology, biology and computation are begining to at least speak the same mathematics. I do not think that anything is going to be settled by debating ideologies, except who is the better debater. Precious bandwidth is better spent discussing specific problems. Terry ----- From harnad at clarity.Princeton.EDU Thu Dec 20 22:55:40 1990 From: harnad at clarity.Princeton.EDU (Stevan Harnad) Date: Thu, 20 Dec 90 22:55:40 EST Subject: Language, Tools and Brain: BBS Call for Commentators Message-ID: <9012210355.AA08443@reason.Princeton.EDU> Below is the abstract of a forthcoming target article to appear in Behavioral and Brain Sciences (BBS), an international, interdisciplinary journal that provides Open Peer Commentary on important and controversial current research in the biobehavioral and cognitive sciences. Commentators must be current BBS Associates or nominated by a current BBS Associate. To be considered as a commentator on this article, to suggest other appropriate commentators, or for information about how to become a BBS Associate, please send email to: harnad at clarity.princeton.edu or harnad at pucc.bitnet or write to: BBS, 20 Nassau Street, #240, Princeton NJ 08542 [tel: 609-921-7771] To help us put together a balanced list of commentators, please give some indication of the aspects of the topic on which you would bring your areas of expertise to bear if you are selected as a commentator. ____________________________________________________________________ Language, Tools, and Brain: The development and evolution of hierarchically organized sequential behavior Patricia Marks Greenfield Department of Psychology University of California, UCLA Los Angeles, CA 90024-1563 electronic mail: rygreen at uclasscf.bitnet Abstract: During the first two years of life a common neural substrate (roughly, Broca's area) underlies the hierarchically organized combination of elements in the development of both speech and manual action, including tool use. The neural evidence implicates relatively specific cortical circuitry underlying a grammatical "module." Behavioral and neurodevelopmental data suggest that the modular capacities for language and manipulation are not present at birth but come into being gradually during the third and fourth years of life. An evolutionary homologue of the common neural substrate for language production and manual action during the first two years of human life is hypothesized to have provided a foundation for the evolution of language before the divergence of hominids and the great apes. Support comes from the discovery of a Broca's area analogue in contemporary primates. In addition, chimpanzees have an identical constraint on hierarchical complexity in both tool use and symbol combination. Their performance matches that of the two-year-old child who has not yet developed the differentiated neural circuits for the relatively modularized production of complex grammar and complex manual construction activity. From geb at dsl.pitt.edu Fri Dec 21 09:34:34 1990 From: geb at dsl.pitt.edu (Gordon E. Banks) Date: Fri, 21 Dec 90 09:34:34 -0500 Subject: More on AI vs. NN Message-ID: <9012211434.AA01594@cadre.dsl.pitt.edu> If cognition is symbolic, and it is clear that many forms of cognition (e.g. recognition of visual objects) is performed just as well by, say, birds, as by humans. Does this mean that birds are adept at the use of symbols? From jose at learning.siemens.com Fri Dec 21 10:31:33 1990 From: jose at learning.siemens.com (Steve Hanson) Date: Fri, 21 Dec 90 10:31:33 EST Subject: AI, NN, CNS (central nervous system) Message-ID: <9012211531.AA28800@learning.siemens.com.siemens.com> aha--jim, we finally get to your anti-reductionist roots! (deep below your brown locks) Well you're right of course, Fooling around in computational space doesn't guarantee any connection whatsoever to the brain. And probably most of the readers and NIPS goers are concerned about making nets faster, better, brighter, cleaner, bigger, etcer... and not interested in making brains. However, lets be careful about creating a junkpile term and throwing people into it. Cognitive Neuroscience as a field, for example, has interest in both brains and function. As Jordan P. wanted to document a few notes ago this might even mean we are interested in a level of computational abstraction we could, if so inclined, refer to as the-- mind. Cognitive Neuroscience is about characterizing function at the level of the mind, in fact, but in terms of neural tissue. Consequently, Cognitive neuroscience is not about the engineering, "neuro-tech" that you seem to be glumping the entire neural net community into. I think there are distinctions to be drawn other than "BRAIN" and "NON-BRAIN". I still maintain connectionism is about "system-level" theory and explanation --this is a vast computational arena that requires careful, informed, and systematic exploration (and not just from neuroscientists). I don't see neuroscientists jumping up with "theories of the brain" or even small parts of the brain everyday. And of course good experimentalists are usually suspicious of good theorists--this seems to be endemic to such interaction. You will undoubtly, complain that worrying about language, problem solving, category learning, and even much of high-level perception at this point is premature since understanding the brain means THE BRAIN! Not some cartoon version of it --not a simplified, random looking computational bric-a-brac...yes, yes, I know... but physicists (and you're surrounded by a number of them out there) seem to appreciate abstracting abit..even before we have all the details straight -- "The art of model-building is the exclusion of real but irrelevant parts of the problem, and entails hazards for the builder and the reader. The builder may leave out something genuinely relevant; the reader, armed with too sophisticated an experimental probe or too accurate a computation, may take literally a schematized model whose main aim is to be a demonstration of possibility." -P. W. Anderson (from Nobel acceptance speech, 1977) merry xmas. Steve From epreston at aisun2.ai.uga.edu Fri Dec 21 16:35:15 1990 From: epreston at aisun2.ai.uga.edu (Elizabeth Preston) Date: Fri, 21 Dec 90 16:35:15 EST Subject: On blurring the gap between NN and AI In-Reply-To: Dave Rumelhart's message of Thu, 20 Dec 90 11:56:52 PST <9012201956.AA22941@psych> Message-ID: <9012212135.AA04788@aisun2.ai.uga.edu> Date: Thu, 20 Dec 90 11:56:52 PST From: Dave Rumelhart The question of whether there is a great divide between the symbolic and "sub-symbolic" approaches is one that I would rather leave to the philosophers. This is kind of you, and of course many of us are in the position of having to take any work we can get, but... The point I was trying to make in my reply to Worth's original message is prescisely that it is doubtful whether this question of a Great Divide is PHILOSOPHICALLY interesting in the first place, and this for the simple reason that it is too early in the development of these approaches to tell. This does not mean that comparative philosophical analysis of them is impossible or unhelpful at this point, but merely that the Big Picture is not to be had at the moment, and that anyone who thinks they have it is deceiving themselves. As Hegel so charmingly put it, the owl of Minerva flies only at dusk. In any case, it has nothing to do with wether or not connectionist AI is a kind of AI. I simply can't think of any reasonable definition of AI that would exclude it. I agree completely. But could someone please tell me then why it is so common, both in the academic and the popular literature, to talk about AI and connectionism as if they were two separate fields? Beth Preston From chan%unb.ca at unbmvs1.csd.unb.ca Fri Dec 21 17:28:31 1990 From: chan%unb.ca at unbmvs1.csd.unb.ca (Tony Chan) Date: Fri, 21 Dec 90 18:28:31 AST Subject: More on AI vs. NN Message-ID: Jim Hendler [Thu, 20 Dec 90 09:18:09 EST] suggests four ways to endow a system so that it can provide both "`low-level'' perceptual functionality as well as demonstrating high-level cognitive abilities" one of which is: "we can work out a new ``paradigm,'' yet another competitor to enter the set of possible models for delivering so-called intelligent behavior." The following short paper belongs to that class. title = "Learning as optimization: An overture", booktitle= "IASTED International Symposium on Machine Learning and Neural Networks", pages = "100--103", address = "New York", month = "Oct 10--11", year = 1990, Abstract = There are two principal paradigms for the study of, pattern learning or machine learning or simply learning. The symbolic paradigm [high-level] for learning, mainly of the AI approach, is typified by Mitchell's version space model and Lenat's heuristic model. And the numeric paradigm [low-level] is represented by the pattern recognition model and the neural net model. In this paper a unified paradigm based on an extension of the Goldfarb's metric learning model is outlined. The unified learning paradigm prescribes a special type of optimization over a parametric family of pseudometric (distance) spaces in order to achieve a certain stability structure (stable configuration) in the optimal pseudometric space which is an output of the learning procedure. This special optimization procedure provides a mathematical guidance by which a system learns to organize itself. From dambrosi at kowa.CS.ORST.EDU Fri Dec 21 07:33:02 1990 From: dambrosi at kowa.CS.ORST.EDU (dambrosi@kowa.CS.ORST.EDU) Date: Fri, 21 Dec 90 12:33:02 GMT Subject: Call for Papers: Uncertainty in AI 91 Message-ID: <9012212030.AA06246@turing.CS.ORST.EDU> THE SEVENTH CONFERENCE ON UNCERTAINTY IN ARTIFICIAL INTELLIGENCE UCLA, Los Angeles July 13-15, 1991 (Preceding AAAI) The seventh annual Conference on Uncertainty in AI is concerned with the full gamut of approaches to automated and interactive reasoning and decision making under uncertainty including both quantitative and qualitative methods. We invite original contributions on fundamental theoretical issues, on the development of software tool embedding approximate reasoning theories, and on the validation of such theories and technologies on challenging applications. Topics of particular interest include: - Foundations of uncertainty - Semantics of qualitative and quantitative uncertainty representations - The role of uncertainty in automated systems - Control of reasoning; planning under uncertainty - Comparison and integration of qualitative and quantitative schemes - Knowledge engineering tools and techniques for building approximate reasoning systems - User Interface: explanation and summarization of uncertain information - Applications of approximate reasoning techniques Papers will be carefully refereed. All accepted papers will be included in the proceedings, which will be available at the conference. Papers may be accepted for presentation in plenary sessions or poster sessions. Five copies of each paper should be sent to the Program Chair by March 4, 1991. Acceptance will be sent by April 22, 1991. Final camera-ready papers, incorporating reviewers' comments, will be due by May 10, 1991. There will be an eight page limit on the camera-ready copy (with a few extra pages available for a nominal fee.) Program Co-Chair: Bruce D'Ambrosio Philippe Smets Dept. of Computer Science IRIDIA 303 Dearborn Hall Universite Libre de Bruxelles. Oregon State University 50 av. Roosevelt, CP 194-6 Corvallis, OR 97331-3202 USA 1050 Brussels, Belgium tel: 503-737-5563 tel: +322.642.27.29 fax: 503-737-3014 fax: +322.642.27.15 e-mail: dambrosio at CS.ORST.EDU e-mail: R01501 at BBRBFU01.BITNET General Chair: Piero Bonissone General Electric Corporate Research and Development 1 River Rd., Bldg. K1-5C32a, 4 Schenectady, NY 12301 tel: 518-387-5155 fax: 518-387-6845 e-mail: bonisson at crd.ge.com Program Committee: Piero Bonissone, Peter Cheeseman, Max Henrion, Henry Kyburg, John Lemmer, Tod Levitt, Ramesh Patil, Judea Pearl, Enrique Ruspini, Ross Shachter, Glenn Shafer, Lofti Zadeh. From ernst at kafka.cns.caltech.edu Fri Dec 21 15:06:52 1990 From: ernst at kafka.cns.caltech.edu (Ernst Niebur) Date: Fri, 21 Dec 90 15:06:52 -0500 Subject: Cortical oscillations Message-ID: <9012212006.AA09524@kafka.cns.caltech.edu> Date: Thu, 20 Dec 90 19:01:59 -0500 From: ernst (Ernst Niebur) Concerning Jim Bower's remark on the NIPS workshop on cortical oscillations: Jim says that "the important conclusion of the visual cortex work is that the zero phase oscillations occur only on average over a number of stimulus trials and therefore are unlikely to serve as the basis for 'binding', attention, or awareness". This statement might be misunderstood. This was NOT the conclusion of the workshop but the result of Matt's and Jim's modeling work. In fact, this was one of the most discussed points during the workshop. At present, clear experimental evidence in favor or against this hypothesis is absent. Actually, two points all participants agreed upon were that the oscillations are present and that they are important for some task. Differences persisted concerning what this task might be: Some people tend to believe that the 40 Hz oscillations somehow serve as a fundamental mechanism during information processing, like a synchronizing signal. An experiment that clearly shows that phase coherence over long distances ONLY occurs on average would support this view, but this experiment has yet to be done. Other people believe that 40 Hz cortical oscillations do have something to do with attention or awareness or "higher order tasks". There is some experimental support which shows that oscillations might be NECESSARY for this kind of task; in fact, it is known for many years that 40 Hz oscillations in the EEG are related to different "higher order tasks". One example presented at the workshop is clinical work by Dierk Schwender who showed that 40 Hz oscillations in the EEG under anesthesia are strongly correlated with the development of hallucinations, dreams and with later conscious recall of events during surgery. Obviously, this does not prove that the presence of oscillations is also SUFFICIENT for "higher order tasks", but it makes this an attractive hypothesis which seems worthwhile to be looked at in more detail. Eventually, the question will have to be decided experimentally. The danger I see is a segregation of "believers" and "non-believers" who become somehow blind to the arguments of the "opponents". I think we should avoid this. Ernst Niebur ernst at descartes.cns.caltech.edu From SAYEGH%IPFWCVAX.BITNET at vma.CC.CMU.EDU Fri Dec 21 21:55:00 1990 From: SAYEGH%IPFWCVAX.BITNET at vma.CC.CMU.EDU (SAYEGH%IPFWCVAX.BITNET@vma.CC.CMU.EDU) Date: Fri, 21 Dec 90 21:55 EST Subject: 4th NN Conference. Indiana-Purdue. Message-ID: FOURTH CONFERENCE ON NEURAL NETWORKS ------------------------------------ AND PARALLEL DISTRIBUTED PROCESSING ----------------------------------- INDIANA UNIVERSITY-PURDUE UNIVERSITY ------------------------------------ 11,12,13 APRIL 1991 ------------------- CALL FOR PAPERS --------------- The Fourth Conference on Neural Networks and Parallel Distributed Processing at Indiana University-Purdue University will be held on the Fort Wayne Campus, April 11,12, 13, 1991. Authors are invited to submit a one page abstract of current research in their area of Neural Networks Theory or Application before February 5, 1991. Notification of acceptance or rejection will be sent by February 28. The proceedings of the third conference are now in press and will be announced on the network in early January. Conference registration is $20 and students attend free. Some limited financial support might also be available to allow students to attend. Abstracts and inquiries should be addressed to: email: sayegh at ipfwcvax.bitnet ----- US mail: ------- Prof. Samir Sayegh Physics Department Indiana University-Purdue University Fort Wayne, IN 46805 From GOLDFARB%UNB.CA at unbmvs1.csd.unb.ca Fri Dec 21 23:57:29 1990 From: GOLDFARB%UNB.CA at unbmvs1.csd.unb.ca (Lev Goldfarb) Date: Sat, 22 Dec 90 00:57:29 AST Subject: AI (discrete) moodel and NN (continuous) model Message-ID: D.E Rumelhart: It seems to me that a number of issues are being confused in this discussion. One has to do what AI is and another has to do with what "connectionism" is and why we might be interested [in] it. To prevent a greater confusion, let me stress again the point that was expressed by J. von Neumann. If we want to do what so far has been called science, we must evaluate the progress not by what "we might be interested in", but by the state of development of the corresponding mathematical models. Therefore, today AI *is* what the practitioners of it are "practicing", and it is not difficult to find what they are saying about the underlying mathematical model: "artificial intelligence (AI) is primarily concerned with propositional languages for representing knowledge and with techniques for manipulating these representations" (Nils J. Nilsson, Artificial Intelligence: Engineering, Science, or Slogan, AI Magazine, Winter 1981-82, p.2). It appears that there is a greater confusion in the NN community about what the underlying mathematical model is. I believe that the underlying mathematical model,i.e. the place where all "events" are developing, is the vector space model. This is because the NN can be viewed as a "mechanism" for transforming input vectors into the subset of real numbers (the transformation, or mapping, is composed of several vector space transformations). What the NN practitioners often forget is that the vector space they want to use is the one that has some geometry in it (there are "algebraic" vector spaces without any geometry in them). The reasons why the geometry is *necessary* for the NN are many: to measure the pattern closeness, to introduce (to construct) the optimization function, etc. Let me say it again: in the present setting, one cannot talk meaningfully about the NN without the corresponding geometry. Thus, as I have alluded to in my last correspondence, we have a "classical" opposition between the basic mathematical paradigms-- the discrete and the continuous. Mathematicians have usually resolved the "opposition" by inducing the continuous structure on top of the discrete one in such a way that the corresponding algebraic operations become continuous, which results in much richer mathematical models (inner product spaces, topological groups, etc.). In order to reconcile the present "AI" and the present "connectionism" (as mathematical models), i.e. to pave the way for the "new" and the desirable AI, one has to construct essentially the same new model that would reconcile, for example, the syntactic pattern recognition (based on Chomsky's generative grammars) and the classical vector space pattern recognition. The old mathematical "solutions", however, of "simple" fusing of the two structures do not work here, since the induced geometry must not be fixed but should vary depending on the structure of the classes that has to be learned. Thus, not only does the continuous structure must be fused with the discrete, which can be accomplished by associating the weighting scheme with the set of operations, but the system must also be able to *evolve structurally* in an efficient manner, i.e. it must be able to learn *efficiently* new (macro)operations necessary for discrimination of the learning class (all present AI learning algorithms do it *very* inefficiently, since they do not use any geometry on the learning space). The outline of my answer to the above "reconciliation" problem, as I have mentioned several months ago, can be found in the June's issue of Pattern Recognition, but the progress since then has been substantial. F-i-n-a-l-l-y, one quick note on why the vector space model is not likely to be of sufficient generality for many environments: the distance functions that can be generated even by simplest insertion/ deletion operations for the string patterns cannot be reconstructed in a Euclidean vector space of any dimension. This fact is not really surprising, since the class of Euclidean space forms a very small subclass of the class of all metric spaces. Thus, it seems to me that the current NN framework must be substantially modified. (Biological implications should also be discussed.) I'll be away for a week. Best wishes for the coming year. -- Lev Goldfarb From ernst at aurel.cns.caltech.edu Sat Dec 22 05:29:17 1990 From: ernst at aurel.cns.caltech.edu (Ernst Niebur) Date: Sat, 22 Dec 90 02:29:17 PST Subject: Cortical oscillations Message-ID: <9012221029.AA01457@aurel.cns.caltech.edu> Concerning Jim Bower's remark on the NIPS workshop on cortical oscillations: Jim says that "the important conclusion of the visual cortex work is that the zero phase oscillations occur only on average over a number of stimulus trials and therefore are unlikely to serve as the basis for 'binding', attention, or awareness". This statement might be misunderstood. This was NOT the conclusion of the workshop but the result of Matt's and Jim's modeling work. In fact, this was one of the most discussed points during the workshop. At present, clear experimental evidence in favor or against this hypothesis is absent. Actually, two points all participants agreed upon were that the oscillations are present and that they are important for some task. Differences persisted concerning what this task might be: Some people tend to believe that the 40 Hz oscillations somehow serve as a fundamental mechanism during information processing, like a synchronizing signal. An experiment that clearly shows that phase coherence over long distances ONLY occurs on average would support this view, but this experiment has yet to be done. Other people believe that 40 Hz cortical oscillations do have something to do with attention or awareness or "higher order tasks". There is some experimental support which shows that oscillations might be NECESSARY for this kind of task; in fact, it is known for many years that 40 Hz oscillations in the EEG are related to different "higher order tasks". One example presented at the workshop is clinical work by Dierk Schwender who showed that 40 Hz oscillations in the EEG under anesthesia are strongly correlated with the development of hallucinations, dreams and with later conscious recall of events during surgery. Obviously, this does not prove that the presence of oscillations is also SUFFICIENT for "higher order tasks", but it makes this an attractive hypothesis which seems worthwhile to be looked at in more detail. Eventually, the question will have to be decided experimentally. The danger I see is a segregation of "believers" and "non-believers" who become somehow blind to the arguments of the "opponents". I think we should avoid this. Ernst Niebur ernst at descartes.cns.caltech.edu From tp-temp at ai.mit.edu Sun Dec 23 11:51:26 1990 From: tp-temp at ai.mit.edu (Tomaso Poggio) Date: Sun, 23 Dec 90 11:51:26 EST Subject: Connectionism vs AI In-Reply-To: Elizabeth Preston's message of Sun, 16 Dec 90 14:41:02 EST <9012161941.AA03794@aisun2.ai.uga.edu> Message-ID: <9012231651.AA03833@erice> The issue of Daedalus titled Artificial Intelligence (Winter 1988), that also appeared as a paperback by MIT Press, is essentially about AI vs. Connectionism. From jbower at smaug.cns.caltech.edu Sun Dec 23 14:06:13 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Sun, 23 Dec 90 11:06:13 PST Subject: Oscillations Message-ID: <9012231906.AA08379@smaug.cns.caltech.edu> Just a note to thank Ernst Niebur for clarifying my poorly worded original posting on the oscillations issue. It is our models prediction that the zero phase coherence is on average only. It is also true that there is no published analysis of the data that resolves this important issue. Jim Bower jbower at smaug.cns.caltech.edu From jbower at smaug.cns.caltech.edu Sun Dec 23 16:17:20 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Sun, 23 Dec 90 13:17:20 PST Subject: AI, NN, BC, CNS Message-ID: <9012232117.AA08401@smaug.cns.caltech.edu> Ideologies: several comments on several comments At the risk of redundancy, let me say again, that blurring distinctions between fields (ideologies) should be avoided and that, in my opinion, there is much less of a relationship between neurobiology and the vast majority of what is going on in AI, NN, and connectionism than one would be led to believe from reading overview books, or even tuning in to the connectionist network. Beyond that, however, there is also an essential difference between what could be called "biological connectionism" and computational neurobiology as I would like to have it defined. This involves the process by which the available computational tools are applied to particular problems. Hartline and Ratliff looked at the specific structure of the Limulus eye and developed an abstract version of that specific circuit to explore the capabilities of the circuit. They did not, as connectionists, go looking for a brain circuit to which they could apply their modeling tools. The brain came first, the tools second. In this case the fact that their 1957 model is largely indistinguishable from several modern connectionist models is interesting but irrelevant. Hartline and Ratliff were not the first connectionists (which I do not believe is what Terry was trying to say, but in this field could be taken that way anyway). As biologists, Hartline and Ratliff invented something new and important by paying attention to the detailed structure of the brain as biologists. Certainly they did this by using existing mathematical tools, what choice is there. But their approach must fall under the category of computational neurobiology as distinct from what Dave Rumelhart in his excellent summary calls the "connectionist approach" to neuroscience and everything else. Again, the critical difference is whether a model is being used to explore possibilities, or to demonstrate a preexisting idea. Even an idea about the nature of the representation of information. This is a critical distinction. Most models, prominently including those of Grossberg et al, are in the later category. It should be obvious why biologists object to such models. There are many other comments that I am tempted to make on the last few days discourse. But I will refrain in the interest of limiting debate that is not related to Touring machines and other acceptable subjects. I would like to say, however, that the Hartline/ Ratliff model is an outstanding example of why KNOWING biological details and focusing directly on biological problems does matter. Of course, if one is of the opinion that the mind is something different from the brain, or that human intelligence is something above and beyond animal intelligence, then there is little point in paying attention to brain details anyway. And the debate from here becomes more theological than ideological. "And to all a goodnight" Jim Bower jbower at smaug.cns.caltech.edu From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Mon Dec 24 04:05:38 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Mon, 24 Dec 90 01:05:38 PST Subject: Cortical oscillations In-Reply-To: Your message <9012221029.AA01457@aurel.cns.caltech.edu> dated 22-Dec-1990 Message-ID: <901224004742.2040117e@Iago.Caltech.Edu> Two interesting points re. the cortical 40 Hz oscillations. 1. At the moment, no good solid electrophysiological evidence exists for 40 Hz o scillations in monkey, either in visual or in olfactory cortex. Mainly because people havenUt tried yet or are just working on it. There is some anectodal evidence but nothing with statistics etc. Given that a number of monkey researchers have looked over the last year at their single cell dat a in light of the Singer and Gray results and didnUt see anything obvious is qui te disconcerting to me... WeUll have to await more data. It really would be wei rd, though, it catUs would hum but not monkeys... 2. However, binding as postulated by von der Malsburg and attention/ awareness as postulated by Crick and myself does not require oscillations but ph ase-locking. Oscillations is just one way to achieve phase-locked firing. Thus, all these theories could perfectly well work with no n-oscillatory phase-locking. I havenUt seen any proof that phase-locking is mor e difficult to achieve in non-oscillatory than in oscillatory systems. Thus, the crucial experiment for these theories is a multi-electrode experiment, showing phase-locked firing on individual trials between cells in different cortical are as in an awake monkey performing some task requiring focal-attention. Christof Koch From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Mon Dec 24 04:05:38 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Mon, 24 Dec 90 01:05:38 PST Subject: Cortical oscillations In-Reply-To: Your message <9012221029.AA01457@aurel.cns.caltech.edu> dated 22-Dec-1990 Message-ID: <901224004742.2040117e@Iago.Caltech.Edu> Two interesting points re. the cortical 40 Hz oscillations. 1. At the moment, no good solid electrophysiological evidence exists for 40 Hz o scillations in monkey, either in visual or in olfactory cortex. Mainly because people havenUt tried yet or are just working on it. There is some anectodal evidence but nothing with statistics etc. Given that a number of monkey researchers have looked over the last year at their single cell dat a in light of the Singer and Gray results and didnUt see anything obvious is qui te disconcerting to me... WeUll have to await more data. It really would be wei rd, though, it catUs would hum but not monkeys... 2. However, binding as postulated by von der Malsburg and attention/ awareness as postulated by Crick and myself does not require oscillations but ph ase-locking. Oscillations is just one way to achieve phase-locked firing. Thus, all these theories could perfectly well work with no n-oscillatory phase-locking. I havenUt seen any proof that phase-locking is mor e difficult to achieve in non-oscillatory than in oscillatory systems. Thus, the crucial experiment for these theories is a multi-electrode experiment, showing phase-locked firing on individual trials between cells in different cortical are as in an awake monkey performing some task requiring focal-attention. Christof Koch From pollack at cis.ohio-state.edu Mon Dec 24 12:57:33 1990 From: pollack at cis.ohio-state.edu (Jordan B Pollack) Date: Mon, 24 Dec 90 12:57:33 -0500 Subject: behaviorism in biological clothing? In-Reply-To: Jim Bower's message of Sun, 23 Dec 90 13:17:20 PST <9012232117.AA08401@smaug.cns.caltech.edu> Message-ID: <9012241757.AA01203@dendrite.cis.ohio-state.edu> In a previous message, despite its catchy title, I didn't mean to imply that specific brain details don't matter in understanding the brain or building brain models; they just don't matter for resolving the scientific difference between traditional AI and connectionism. I have long felt that work on cognition should not pose as biology. This applies both to some arguments for linguistics as a natural science, and to psychological or computational work which gets justified as "neurally plausible." On the other hand, work on biology should not pose as cognition either. Jim B. concludes his recent message by striking a Skinnerian pose on these two questions: 1) Is the mind different than the brain? Yes, although we might agree they are intimately related. One can freeze and slice the brain. One can surgically remove parts of the brain. One can even study dead brains. But one cannot do ANY of these things to the mind, unless one were a New Age NeuroGuru. 2) Is human intelligence above and beyond animal intelligence? Only a little. There are only a few "relatively small" biological differences between mammal, primate, and human BRAINS, but differences between their respective INTELLIGENCES have been exacerbated by social and cultural evolution, especially in species with any significant postnatal development. Animal neuroscience work has made profound discoveries of brain mechanisms for memory and conditioning, for representations in sensory and motor peripheral areas, and elsewhere. But many animals have more independent behavior than can be described by memorizing an input-output mapping. Which details of a monkey's brain are responsible for its personality or its cooperative social behavior? It is not reasonable to explain cognition in detailed neurobiological terms for any mammal... not even a mouse! Jordan Pollack From jbower at smaug.cns.caltech.edu Mon Dec 24 14:40:05 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Mon, 24 Dec 90 11:40:05 PST Subject: Touring Message-ID: <9012241940.AA08755@smaug.cns.caltech.edu> My appologies to the community. I see that my spell checker does not distinguish between Touring and Turing. Interprete that as you will, but I can write effecient computer programs. Guess I'll just always be nothing more than a reverse engineer. Jim Bower jbower at smaug.cns.caltech.edu From slehar at park.bu.edu Wed Dec 26 11:30:09 1990 From: slehar at park.bu.edu (Steve Lehar) Date: Wed, 26 Dec 90 11:30:09 -0500 Subject: bio-connectionist vs comp-neurosci Message-ID: <9012261630.AA24869@park.bu.edu> Jim Bower draws a sharp distinction between biological connectionism and computational neuroscience. As I understand it, he defines these as follows: BIOLOGICAL CONNECTIONISM: Using connectionist ideas to explore the functional architecture of specific biological circuits found in nature. (Paying attention to the detailed structure of the brain as biologists) COMPUTATIONAL NEUROSCIENCE: Exploring the theoretical possibilities of abstract connectionist architectures for their information representation and processing potential. I have no problem with these definitions, they represent the two pronged approach of science, theory and observation. In physics, for instance, mathematics explores the possible functional relationships between variables- linear, quadratic, exponential, periodic etc. and physics makes use of these functional relationships wherever applicable to physical observations. Where I have difficulty with Bower's view is when he says that the theoretical exploration and the physical observations have little to do with one another. He says, for instance of the Hartline and Ratliff model "The fact that [the] model is largely indistinguishable from several modern connectionist models is interesting but irrelevant". Is he saying that theoretical exploration of systems that are similar, but not identical to a specific physical system are irrelevant to that system? Is it not exactly the marriage between extensive theoretical modeling and precise physical observation that has led to the scientific revolution? This is exemplified by the way that the findings of pure mathematics always seem to find an application in the applied world sooner or later. Theoretical investigation is particularly fruitful whenever science experiences a "paradigm shift", discovering a new mathematical formalism to better discribe an old set of data. Is this not exactly what is happening in our field today? Could one not say, for instance, that the center-surround connectionist models derived through theoretical explorations in some sense "predict" the physical observations of the Hartline and Ratliff data by showing that such architectures have desirable computational properties? Bower says "It should be obvious why biologists object to such [computational neuroscience] models." Pardon my ignorance, but I fail to see this 'obvious' fact- would you care to enlighten me as to why biologists would object to theoretical explorations of computational mechanisms that are clearly much more like the brain than alternative computational models? (O)((O))(((O)))((((O))))(((((O)))))(((((O)))))((((O))))(((O)))((O))(O) (O)((O))((( slehar at park.bu.edu )))((O))(O) (O)((O))((( Steve Lehar Boston University Boston MA )))((O))(O) (O)((O))((( (617) 424-7035 (H) (617) 353-6741 (W) )))((O))(O) (O)((O))(((O)))((((O))))(((((O)))))(((((O)))))((((O))))(((O)))((O))(O) From der%psych at Forsythe.Stanford.EDU Wed Dec 26 14:32:26 1990 From: der%psych at Forsythe.Stanford.EDU (Dave Rumelhart) Date: Wed, 26 Dec 90 11:32:26 PST Subject: AI, NN, CNS (central nervous system) In-Reply-To: Jim Bower's message of Wed, 19 Dec 90 23:59:29 PST <9012200759.AA06211@smaug.cns.caltech.edu> Message-ID: <9012261932.AA16499@psych> In response to the various comments concerning the relationship (or lack thereof) between connectionist modelling and neurobiology. I wish especially to address myself to Jim Bower's many comments. I must say that I find it counter productive to press for a hard distinction between Computational Neuroscience and connectionist approaches to neuroscience. In these cases, the goals are the same -- namely to understand/explain/predict observations made on certain parts of the brains of certain animals using certain measurement techniques. When possible, the goals would also be to relate such observations to the animal's behavior. It seems that Jim (and perhaps others) wish to distinguish between what he considers "bad" attempts at doing this which he dubs connectionist and "good" ones, which he dubs "computational neuroscience". I believe that the real issue should be framed differently -- in terms of the goals of any piece of work. In any theoretical discipline there is a need for development and analysis of the formal (mathematical/ computational) tools appropriate for expressing the theories at hand (this is in the realm of applied mathematics and AI -- not biology) and there needs to be the application of these tools in the modelling of particular phenomena (this is biology or cognitive neuroscience or cognitive science). Many scientists do both of these things. Perhaps most focus only on the biological or only on the formal aspects. It is true that much of the discussion in this forum is about the technical mathematical/computational foundations of computational/connectionist modelling rather than about the biological/behavioral phenomena to which the models are to be applied. This does not mean either that many of the participants might not be interested in the eventual biological applications nor that tools developed and analysed by those of us who participate may not be of value to the neurobiologist or the psychologist. It strikes me that much of the excitement of the field comes from the interdisciplinary cross-fertalization that has taken place over the past several years. If this communication is to continue to take place fruitfully we must keep the channels of communication open, to learn what we can about the questions which occupy the minds of our colleagues and not to discount the results of one another as "irrelevant". D. E. Rumelhart From rupen at cvax.cs.uwm.edu Wed Dec 26 16:29:17 1990 From: rupen at cvax.cs.uwm.edu (Rupen Sheth) Date: Wed, 26 Dec 90 16:29:17 CDT Subject: IJCNN '90 paper by Asoh Message-ID: <9012262229.AA14090@cvax.cs.uwm.edu> Hi: o I am looking for the following paper: "An Approximation of Nonlinear Discriminant Analysis by Multilayer N.N.'s" in the Proc. of IJCNN '90 by Asoh (from Japan). Could someone mail me a copy at : 3058 N. Maryland Avenue Milwaukee, WI 53211, USA. o What does IJCNN stand for? Thank you. Rupen Sheth. From thomasp at informatik.tu-muenchen.dbp.de Thu Dec 27 07:27:38 1990 From: thomasp at informatik.tu-muenchen.dbp.de (Patrick Thomas) Date: 27 Dec 90 13:27:38+0100 Subject: TR on the Modelling of Synaptic Plasticity Message-ID: <9012271227.AA01720@gshalle1.informatik.tu-muenchen.de> The following technical report is now available: BEYOND HEBB SYNAPSES: BIOLOGICAL BUILDING BLOCKS FOR UNSUPERVISED LEARNING IN ARTIFICIAL NEURAL NETWORKS Patrick V. Thomas Report FKI-140-90 Abstract This paper briefly reviews the neurobiology of synaptic plasticity as it is related to the formulation of learning rules for unsupervised learning in artificial neural networks. Presynaptic, postsynaptic and heterocellular mechanisms are discussed and their relevance to neural modelling is assessed. These include a variety of phenomena of potentiation as well as depression with time courses of action ranging from milliseconds to weeks. The original notion put forward by Donald Hebb stating that synaptic plasticity depends on correlated pre- and postsynaptic firing is reportedly inadequate. Although postsynaptic depolarization is necessary for associative changes in synaptic strength to take place (which conforms to the spirit of the hebbian law) the association is understood as being formed between pathways converging on the same postsynaptic neuron. The latter only serves as a supporting device carrying signals between activated dendritic regions and maintaining long-term changes through molecular mechanisms. It is further proposed to restrict the interactions of synaptic inputs to distinct compartments. The hebbian idea that the state of the postsynaptic neuron as a whole governs the sign and magnitude of changes at individual synapses is dropped in favor of local mechanisms which guide the depolarization-dependent associative learning process within dendritic compartments. Finally, a framework for the modelling of associative and non-associative mechanisms of synaptic plasticity at an intermediate level of abstraction, the Patchy Model Neuron, is sketched. To obtain a copy of the technical report FKI-140-90 please send your physical mail address to either "thomasp at lan.informatik.tu-muenchen.de" or Patrick V. Thomas, Institute for Medical Psychology, Goethe-31, 8000 Munich 2, Germany. From INAM%MUSICB.MCGILL.CA at bitnet.CC.CMU.EDU Thu Dec 27 11:05:34 1990 From: INAM%MUSICB.MCGILL.CA at bitnet.CC.CMU.EDU (Tony Marley) Date: Thu, 27 Dec 90 11:05:34 EST Subject: models versus data Message-ID: <27DEC90.11980318.0058.MUSIC@MUSICB.MCGILL.CA> FROM: Tony Marley, Director, McGill Cognitive Science Centre. RE: Models versus Data. In the light of recent discussions of modeling versus data, I thought it might be worthwhile (re)drawing attention to the following paper by Yellott. One of the recent examples discussed was the Hartline-Ratliff model of the limulus eye, and Mach bands, using a LINEAR SYSTEM WITH INHIBITION - one argument being that this was excellent work as the model was driven by biological data. The interesting aspect of Yellott's work is that he obtains Mach bands from a NONLINEAR SYSTEM WITHOUT INHIBITION, and argues that it is difficult on the basis of current physiological data to decide which model is "correct". (I do not believe that he specifically discusses the limulus eye so perhaps the data is clear there). Anyway, the point is to reiterate how theory dependent interpretation of (raw) data can be. ARTICLE: YELLOTT, J. I. (1989). Constant volume operators and lateral inhibition. Journal of Mathematical Psychology, 33, 1-35. "Constant volume (CV) operators are nonlinear image processing operators in which the area covered by the pointspread function around each point in the input image varies inversely with light intensity at that point. This operation is designed to make spatial resolution increase with retinal illumination, but it proves to have unexpected side-effects that mimic other important properties of human spatial vision, including Mach bands and Weber's law. Mach bands are usually attrtibuted to lateral inhibition in the retina, and when retinal image processing is modeled by a linear operator they imply such inhibition, since they cannot be produced by a nonnegative impulse response. CV operators demonstrate that Mach bands and other high-pass filter effects can be created by purely positive pointspread functions, i. e. without inhibition. This paper shows in addition that if one attempts to combine lateral inhibition with a CV operator, the results are dramatically wrong: the edge response always contains Mach bands that bulge in the wrong direction. Thus within the nonlinear theoretical framework provided by CV operators, lateral inhibition is neither necessary or sufficient for modeling Mach bands and other high-pass filter properties of spatial vision." (Yellott is at the Cognitive Sciences Department, University of California, Irvine, CA 92717.) From jbower at smaug.cns.caltech.edu Thu Dec 27 16:01:35 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Thu, 27 Dec 90 13:01:35 PST Subject: logout Message-ID: <9012272101.AA00979@smaug.cns.caltech.edu> First, I apologize to those that are not interested in this debate for the amount of traffic it has generated (especially from me). This will be my last posting on this subject, but the assumed nature of the relationship between AI, neural networks, and connectionism on the one hand and the structure of the brain, on the other, has been increasingly troubling me. With respect to David Rumelhart's comments, it was probably an error on my part to attempt to more narrowly define computational neuroscience as modeling related to the actual structure of the brain (incidentally Steve Lehar's interpretation of my definition is off by 180 degrees). The field as a whole would not accept this definition. However, the point that I was (and have been) trying to make, is that most connectionist modeling is related to more cognitive descriptions of brain function, not to the actual structure of the nervous system. Further, as my recent interaction with S. Hanson/J. Pollack over the net should have made clear, the mapping between cognitive descriptions of brain function, and actual brain structure is not at all straight forward. Accordingly, for those of us that are interested in understanding the brain's structure, it is not clear to me how useful these connectionist models will be just as it is not clear how useful cognitive approaches or AI will be. Thus, while my previous use of the word irrelevant was with respect to an historical argument about modeling that Terry Sejnowski had made, it is really not yet clear what the relevancy of the majority of connectionism will be to neurobiology. That is not to say that this work is irrelevant to its intended objective. Clearly connectionism is already making a substantial contribution in a number of different fields. It is also possible that useful tools will be developed. It is simply to say that if one is interested in understanding how the brain works, I believe it is necessary to address ones modeling efforts to the brain's detailed structure. Understanding this very complex system will not simply fall out by applying connectionist ideas to speech recognition problems. It is true that there is a growing effort to apply connectionist modeling techniques to actual brain structures. This network is not the right place to discuss this still relatively minor component of connectionism. However, I will say that I fail to be convinced of the usefulness of these models, and furthermore, I am concerned that these efforts may actually serve to further obscure the distinctions between brain organization and the organization of connectionist models. It seems to be precisely the association of connectionist models with network implementations that has confused the question of biological plausibility to begin with. The direct applications of connectionist tools to brain modeling makes these distinctions even tricker, especially in the larger connectionist/AI/ NN field where most practitioners know very little about the structure of the brain to begin with. As a neurobiologist, however, I would assert that even a cursory look at the brain reveals a structure having very little in common with connectionist models. In my view this is not simply a question of necessary modeling abstraction, it is a question of the basic computational assumptions underlying network construction (node structure, feed forward components, information encoding, error detecting, learning, overall complexity). Further, if these things are changed substantially, then I would say one no longer has a connectionist model. Finally, I would like to point out that I have spent much of the last seven years communicating and cross-fertilizing with my connectionist, neural network, AI, engineering, physicist friends, colleagues, and students. In fact, more than two thirds of the students in my laboratory are from one or another of these disciplines and we continue to learn a great deal from each other. But for communication to be successful, and multidisciplinary efforts to be real, there has to be a serious commitment to two way communication. In my view, within connectionism, there has been too much lip service paid to biological plausibility and not enough commitment to finding out what that really means. Jim Bower jbower at smaug.cns.caltech.edu From Scott.Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU Thu Dec 27 20:15:39 1990 From: Scott.Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF-PMAX.SLISP.CS.CMU.EDU) Date: Thu, 27 Dec 90 20:15:39 EST Subject: On blurring the gap between NN and AI In-Reply-To: Your message of Fri, 21 Dec 90 16:35:15 -0500. <9012212135.AA04788@aisun2.ai.uga.edu> Message-ID: ...But could someone please tell me then why it is so common, both in the academic and the popular literature, to talk about AI and connectionism as if they were two separate fields? I'll try... AI and connectionism have the same long-term goals: to understand this complex, wonderful phenomenon that we call intelligence. We ultimately want to understand intelligence at all levels and in precise mechanistic terms -- sufficient, in principle, to allow us to emulate the phenomenon on a machine of some kind. If you define the field in terms of its goals, it is clear that there is only one field here -- call it what you like. Both the traditional AI and connectionist camps include some people who are basically psychologists: they want want to understand how human and animal intelligence works. To these people, computer simulations are useful, but only as tools that help us to explore the capabilities and limitations of various models. Both camps also contain people who are basically engineers: they want to build more intelligent widgets, and if one can derive some useful ideas of constraints by investigating the workings of the human mind, that makes the task a bit easier. Psychology, to these people, is just a fertile source of search-guiding heuristics. And, of course, there are lots of people whose interests fall between these two extremes. But the engineer/psychologist split is orthogonal to the AI/connectionist split. What separates traditional, mainstream AI from connectionism is the choice of tools, and a parallel choice of what parts of the problem to work on now and what parts to defer until later (if ever). Traditional AI has had a good deal of success building upon the central ideas of heuristic search and symbolic description. These tools seem to be the right ones for modelling high-level conscious reasoning in clean, precise problem domains (or those in which the messy bits can be safely ignored). These tools are not so good at low-level sensory/motor tasks, flashes of recognition, and the like. AI people respond to this limitation in a variety of ways: some define the problem away by saying that this low level stuff is not really a part of "intelligence"; some say that it's important, but that we'll get to it later, once the science and technology of symbolic AI has progressed sufficiently; and some admit that connectionism probably offers a better set of tools for handling the low-level and messy parts of thought. Connectionism offers a different set of tools. These tools seem to be better for fuzzy, messy problems that are hard to cast into the rigid framework of symbols and propositions; they are not so good (yet) for modeling what goes on in high-level reasoning. Connectionists respond to these evident limitions in a number of ways: some believe that high-level symbolic reasoning will more-or-less automatically fall out of connectionist models once we have the tools to build larger, more complex nets; some believe that we should get to work now building hybrid connectionist/symbolic systems; some just think we can put off the problem of high-level reasoning for now (as evolution did for 4.5 billion years). Many mainstream AI people like to invoke the "Turing barrier": they imagine that their system runs on some sort of universal computational engine, so it doesn't really matter what that engine is made of. The underlying hardware can be parallel or serial, slow or fast -- that's just a matter of speed, not a matter of fundamental capability. Of course, that's just another way of focusing on part of the problem while deferring another part -- sooner or later, whether we are engineers or psychologists, we will have to understand the speed issues as well. Some important mental operations (e.g. "flashes" of recognition) occur so fast that it is hard to come up with a serial model that does the job. One can work on these speed/parallelism issues without leaving the world of hard-edged symbolic AI; my old NETL work was a step in this direction, and there are several other examples. But in connectionism, the tradition has been to focus on the parallel implementation as an essential part of the picture, along with the representations and algorithms. Because of the Turing barrier, AI people and biologists may feel that they have little to learn from one another; no such separation exists in connectionism, though we can certainly argue about whether our current models have abstracted away all biological relevance and whether that matters. So I would say that connectionism and traditional AI are attacking the same huge problem, but beginning at opposite ends of the problems and using very different tools. The intellectual skills needed by people in the two areas are very different: continous math and statistics for connectionists; discrete algorithms and logic for the AI people. Neither approach has a single, coherent "philosophy" or "model" or "mathematical foundation" that I can see -- I'm not really sure what sort of foundation Lev Goldfarb is talking about -- but there are two loose collections of techniques that differ rather dramtically in style. AI/Connectionism can be thought of as one field or two very different ones. It depends on whether you want to emphasize the common goals or the very different tools used by the two groups. One can define AI in an all-encompassing way, or one can define it in a way that emphasizes the use of hard-edged symbols and that rules out both connectionism and fuzzy logic. I prefer the broader definition -- it makes it a bit easier for us unprincipled pragmatists to sneak back and forth between the two camps -- but it is seldom worth arguing about where to draw the boundaries of a field. -- Scott From slehar at park.bu.edu Fri Dec 28 09:41:29 1990 From: slehar at park.bu.edu (Steve Lehar) Date: Fri, 28 Dec 90 09:41:29 -0500 Subject: logout Message-ID: <9012281441.AA23640@park.bu.edu> In his final communication Jim Bower strikes at the heart of his differences with biological connectionist philosophy. While many connectionists believe that their paradigm bears both a structural and functional similarity to brain and mind, and is thus a valid theoretical tool for exploring those entities, Bower believes that connectionism is no closer to understanding the brain than conventional AI or any other paradigm. My "180-degree misunderstanding" of his former posting was (I am left to guess) in thinking that he opposed ALL theoretical modeling, whereas what he opposes is all theoretical modeling of the BRAIN. It seems that Bower is ferverently convinced that the mechanisms of the brain are a deep dark secret that will not yield to simple investigations with numerical models. This is a curious view in an age when science has successfully probed the atomic element of the brain, the neuron, sufficiently to understand its major mode of operation. Models at this level he does not oppose, so long as they restrict themselves to strictly reproducing the experimental data. But when the theoretical insights gained from such models are generalized to information processing models, that, says Bower is going too far. I agree with Bower that many of today's popular neural models are very remote from actual biology, and there exists a need to close the gap between the abstract mathematical type models like backprop, and the lower level models like the Hartline and Ratliff model. In fact, that is the major thrust of the work of people like Grossberg. What I find very curious is Bower's resistance to this kind of effort. Bower says... "It is true that there is a growing effort to apply connectionist modeling techniques to actual brain structures. This network is not the right place to discuss this still relatively minor component of connectionism." I cannot disagree more! This network is exactly the place to discuss such models, since these are the kind of models that give direction and validity to the more abstract models. If these models are only a minor component of connectionism, that is a regretable fact which needs to be corrected by more discussion of these models. Bower continues... "However, I will say that I fail to be convinced of the usefulness of these models, and furthermore, I am concerned that these efforts may actually serve to further obscure the distinctions between brain organization and the organization of connectionist models." Of course they will obscure the distinction between brain organization and connectionist models. That is exactly the purpose of such models, to show the commonality between the brain and the models. Bower firmly believes that this commonality does not exist, and therefore it is fruitless to try to find it... "As a neurobiologist, however, I would assert that even a cursory look at the brain reveals a structure having very little in common with connectionist models. it is a question of the basic computational assumptions underlying network construction (node structure, feed forward components, information encoding, error detecting, learning, overall complexity)." A cursory glance at the brain reveals multitudes of simple computing elements richly interconnected with synaptic links. You say that has LITTLE to do with connectionist models? That was the very INSPIRATION for connectionist models! Now if we have some of the details wrong- node structure, feedback etc., then let us CORRECT those deficiencies in order to more closely model the brain. In fact, those are exactly the kinds of issues addressed by the more biological connectionist models like Grossberg's, which have dynamic properties and rich feedback connections precisely for that reason. Bower objects... "if these things are changed substantially, then I would say one no longer has a connectionist model." It doesn't matter what they're CALLED, you can call them whatever you like. What's important is that emulate the functional architecture of the brain. (O)((O))(((O)))((((O))))(((((O)))))(((((O)))))((((O))))(((O)))((O))(O) (O)((O))((( slehar at park.bu.edu )))((O))(O) (O)((O))((( Steve Lehar Boston University Boston MA )))((O))(O) (O)((O))((( (617) 424-7035 (H) (617) 353-6741 (W) )))((O))(O) (O)((O))(((O)))((((O))))(((((O)))))(((((O)))))((((O))))(((O)))((O))(O) From tom at asi.com Fri Dec 28 10:14:11 1990 From: tom at asi.com (Tom Baker) Date: Fri, 28 Dec 90 07:14:11 PST Subject: Limited Precision Neural Network Simulations Message-ID: <9012281514.AA05039@asi.com> I was encouraged by the discussion of limited precision simulations during the NIPS VLSI post-conference workshop. From the feedback that I received during my presentation, it seemed as though many of the audience had tried limited precision simulations on their own. Unfortunately, I was not able to get specific details on their work, or get anybody's name and address. I have read several papers of researchers that have used sixteen bit weights for back propagation, but I have not seen many papers about using less precision. I would like to hear about the experiences of other researchers that have tried to simulate neural networks with low precision calculations, as well as get any paper references that you may have. Let me know about any tricks or hacks that you have used to get non-floating point simulators to work. I will collect a bibliography and a separate list of techniques that others have tried. I will post the results to the net, and continue to keep the bibliography as new papers are published. I would like to keep in touch with the people that are doing research in this area. Thomas Baker INTERNET: tom at asi.com Adaptive Solutions, Inc. UUCP: (uunet,ogicse)!adaptive!tom 1400 N.W. Compton Drive, Suite 340 Beaverton, Oregon 97006 From nowlan at ai.toronto.edu Fri Dec 28 11:24:18 1990 From: nowlan at ai.toronto.edu (Steven J. Nowlan) Date: Fri, 28 Dec 1990 11:24:18 -0500 Subject: logout In-Reply-To: Your message of Fri, 28 Dec 90 09:41:29 -0500. Message-ID: <90Dec28.112432edt.827@neuron.ai.toronto.edu> I usually avoid free-wheeling network discussions such as this, but I believe that Steve Lehar is doing Jim Bower an injustice in his characterization of Jim's argument: | In his final communication Jim Bower strikes at the heart of his | differences with biological connectionist philosophy. While many | connectionists believe that their paradigm bears both a structural and | functional similarity to brain and mind, and is thus a valid | theoretical tool for exploring those entities, Bower believes that | connectionism is no closer to understanding the brain than | conventional AI or any other paradigm. My "180-degree | misunderstanding" of his former posting was (I am left to guess) in | thinking that he opposed ALL theoretical modeling, whereas what he | opposes is all theoretical modeling of the BRAIN. | | It seems that Bower is ferverently convinced that the mechanisms of | the brain are a deep dark secret that will not yield to simple | investigations with numerical models. My own (admittedly limited) understanding of the crux of Jim's argument might be summarized as follows: The idea that "a neuron functions by emitting action potentials proportional to a non-linear squashing function applied to the total activity received through its synaptic connections with other neurons" is at least as far from the truth as the idea that "a neuron represents a logical proposition." This strikes me as a reasonable statement, given what little we do know about the incredible complexity of neuronal function. I think Jim's point is important to bear in mind, because it (should) keep us from attempting to justify a connectionist model of some phenomena simply (or mainly) because it is "more brain like" than some other abstract model. This sort of reasoning is tempting, and places one on very shaky scientific ground. It is all too easy to develop some pet theory of how X is done, design some network model based on this theory, simulate the model and exclaim "Aha, this model supports theory Y about X and is a network -- so theory Y must explain how the brain does X". Since the assumptions of the theory were built into the model in the first place, the simulations may in fact tell us very little. Our models and theories need to be tested in the time honored way -- by considering what predictions the theories make and attempting to design critical experiments which will support or refute these predictions. This is not to say that connectionist modelling has nothing to say to the experimental biologist. I think a very good example of what it has to say can be seen in some of Sean Lockery's work on the leech bending reflex. What this work suggests is that single cell recordings of isolated neurons, and analysis of the synaptic organization of individual neurons is not likely to be very fruitful for understanding the functional role of these neurons because real biological neurons appear to share a computational property of connectionist models -- the functional role of any unit cannot be understood in isolation but only in the context of the functioning of other computational units. Given the current state of development of connectionist models, and understanding of biological neuronal processing, it seems that cross-fertilization of ideas is likely to be most effective at this rather abstract level of computational properties. - Steve From jbower at smaug.cns.caltech.edu Fri Dec 28 16:27:19 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Fri, 28 Dec 90 13:27:19 PST Subject: postscript Message-ID: <9012282127.AA04284@smaug.cns.caltech.edu> I'm sorry, but I feel compelled to point out that I could not have better illustrated the consequences of the prevailing assumptions about connectionism and the brain than Steve Lahar has done. Briefly however: - my point is that a closer relationship between connectionism and the brain has not been PROVEN and therefore should not be ASSUMED simply because connectionists work with network structures. I do not doubt and actually have been trying to assert that "many connectionists BELIEVE that their paradigm is a valid tool for exploring...the brain" it is a different thing to PROVE IT however. - I do not oppose "all theoretical modeling of the brain", I oppose imposing abstract theoretical constructs (e.g. ART I, II, III, etc.) on brain structure and then claiming that these models are actually derived from the structure they are imposed on. This is very different from building realistic low level models and then abstracting those. This is what I actually do for a living and is decidedly not what Grossberg has done. The difference is that, in the approach I am advocated, there is some chance that the brain will actually tell you something you didn't know before you started. Given its complexity, in my view, this is the ONLY way we will figure out how it works. - Finally, there is obviously a link between thinking that "science" understands "the major mode of operation of the neuron" (whatever that could even be) and thinking that the brain is composed of "simple computational elements". Both are absolutely wrong. As a rule of thumb, if your model is simple, it is unlikely to be capturing anything deep about the way the brain works, because the brain is almost certainly the most complicated machine we know about and its complexity is very unlikely to be a result of sloppy engineering. Show me any poorly designed hack that has 10 to the 12th components, a single component of which can not be realistically modeled on even today's fastest computer, whose source of power is as energetic as glucose, that is capable of the information processing feats the brain pulls off in real time, and still doesn't generate enough free energy to keep itself within its ideal operating range. All I am really asking for is a little respect for this system and a little less arrogance from those who do not study its structure directly. Jim Bower From aarons at cogs.sussex.ac.uk Fri Dec 28 17:40:02 1990 From: aarons at cogs.sussex.ac.uk (Aaron Sloman) Date: Fri, 28 Dec 90 22:40:02 GMT Subject: AI, NN, Neurobiology, architectures and design space Message-ID: <4016.9012282240@rsuna.cogs.susx.ac.uk> I'd like to make some comments on the recent discussions from the standpoint of a philosopher who has dabbled in various aspects of AI for about 20 years and believes that in principle it should be possible (in the VERY distant future) to replicate all or most interesting features of human minds in machines of some kind, because I don't believe in magic of any kind. Also I have seen several fashions come and go. I'll start with some background comments before getting to more meaty matters (bringing together, and extending, some of the points already made by other people). 1. Despite many written and spoken words about the differences between Connectionism and AI, I think it is clear that PDP/NN/Connectionist models have FAR more in common with AI models than they have with human brains, both in terms of what they do or how they work, and in terms of what they don't (yet) do (see below). Unfortunately people who know little about AI (e.g. those who think that expert systems and automatic theorem provers exhaust AI, because they don't know about AI work on speech, vision, robotics, numeric-based learning systems, etc.) are easily misled into believing exaggerated claims about the differences. A good antidote for such exaggerations is the technical report "Symbol Processing Systems, Connectionist Networks, and Generalized Connectionist Networks," by Honavar and Uhr, recently announced in this mail forum. Another good antidote is to survey IJCAI (International Joint Conference on AI) proceedings over the years. Some of the authors of narrowly focussed text-books on AI are partly to blame for the misconceptions about the scope of AI (e.g. books that present AI as logic). (Heated and noisy but fairly vacuous, and often transient, disputes between factions or fashions are quite common in science and academe. I leave it to others to offer sociological explanations in terms of patterns of mutual excitation and inhibition.) 2. I am not saying that there are no differences between NNs and other AI models, but that the technical, scientific, and philosophical significance of the differences has been exaggerated. E.g. I think there is NO significant philosophical difference. The biological differences are more concerned with what they are trying to explain than with correctness as models. The technical differences are still barely understood. (More on this below). NNs are closer to some non-NN AI models (e.g. in speech and vision) than those AI models are to other non-NN AI models (e.g. in theorem proving, natural language processing). E.g. So-called sub-symbolic or sub-cognitive or micro- features in NNs have much in common with low level or intermediate representations in vision. Distributed representations have much in common with intermediate databases in vision programs. There's also a loose analogy between distributed representations and theorems implicit in (=distributed over?) an axiom set. Like some of the earlier discussants, I see all AI and NN work as exploring sub-regions in the space of possible explanatory designs. We understand too little of this space to know where the major discontinuities are. 3. Concerning details of brain structure, both seem miles off. Jim Bower wrote (Thu, 27 Dec 90 13:01:35 PST) | .....As a neurobiologist, however, I would assert | that even a cursory look at the brain reveals a structure having | very little in common with connectionist models. The same can be said about existing AI models and intelligence. I'll put the point differently: it is clear from even a cursory study of literature on the anatomy and physiology of the brain that far more complex and varied designs exist than anyone has yet modelled. The same conclusion can be reached on the basis of cursory reflection on the differences between the abilities of people, squirrels, nest-building birds, etc. and the abilities of current AI or NN models. (To say nothing of what neurobiologists can explain!) 4. People working "at the coalface" in a scientific or technical discipline often find it hard to see the limitations of what they are doing, and therefore exaggerate its importance. Drew McDermott once wrote a paper called "Artificial Intelligence meets natural stupidity" (reprinted in John Haugeland, ed. Mind Design, MIT press 1981), criticising (from an AI standpoint) AI researchers who, among other things, use words and phrases from ordinary language as "wishful mnemonics", e.g. "Goal", "Understand", "Planner", "General Problem Solver". Much of what he says can be applied equally to NN research where words like "Learn", "Recognise" and "Interpret" are used to describe mechanisms that do little more than map vectors from one space into another, or store vectors using content-addressible memories. Maybe a prize should be offered for the best essay by a student re-writing Drew's paper with a title something like "Artificial nets meet dumb brains"? 5. I am not attacking NN research: I am merely pointing out commonalities between NN and AI research. The limitations of both are to be expected: Because most scientific research is inevitably piecemeal, and progress depends in part on a lot of systematic exploration of detail, a tendency that is common to most researchers is that they focus on tiny sub-mechanisms without any consideration of the global architecture within which those mechanisms might function. (I suspect this is equally true of people who study real brains, but I don't know their work so well. It is certainly true of many psychologists.) By consideration of the "global architecture" I mean the study of the capabilities of the whole system, and the analysis of a system into distinct sub-systems or sub-mechanisms with different causal roles (defined by how they interact with the environment and with other sub-systems), contributing to (and explaining) the global capabilities of the whole system. (I think this is closely related to what Jim Hendler wrote: Thu, 20 Dec 90 09:18:09 -0500). This study of global architecture is a very hard thing to do, especially when the functional decomposition may not be closely related to anatomical decomposition. So much of the analysis has to be inspired from a design standpoint (how could we make something like this?). This is best done in parallel with the more detailed studies: with feedback between them. Notes: 5.1. Don't assume that the division into sub-systems has to be rigidly defined: sub-mechanisms may share functions or change functions. 5.2 Don't assume that every system has a fixed architecture: one important kind of capability may be creation, destruction or modification of sub-structures or sub-mechanisms. Some of these may be virtual machines of changing complexity implemented in a physical mechanism of fixed complexity: a common feature of sophisticated software. Perhaps the conceptual development of a child is best understood as the development of new (virtual?) sub-mechanisms that make new processes possible: e.g. percepts or thoughts of increasing complexity, more complex motivational patterns, etc. 5.3. Don't assume that there is only one level of decomposition into sub-systems. A proper understanding of how the system works, may require some of the sub-systems to be thought of as themselves implemented in lower level mechanisms with different capabilities. There may be many levels. 5.4. Don't assume there's a fixed separation of levels of implementation: some of the very high level functionality of large scale sub-mechanism may be closely coupled with some very low level mechanism. An example might be chemical processes that alter speed of processing, or turn whole sub-mechanisms on or off. (How does alcohol alter decision making?) (Compare a computer whose programs are run by a microcode interpreter that can be altered by those very programs, or an interpreter uses subroutines written in the language being interpreted.) 6. It's clear that the global architecture of human beings includes a lot of coarse-grained parallelism. I.e. there are often many concurrent processes e.g. simultaneous walking, talking, thinking, eating, hearing, seeing, scratching one's ear, feeling hungry, feeling cold, etc. to say nothing of the regulation of internal physiological processes we are not aware of, or the decomposition of processes like walking, or talking, into different concurrent sub-processes (generating motor control signals, internal monitoring, external sensory monitoring, etc. etc.) Moreover a process that might have one label in ordinary language (e.g. "seeing") can simultaneously perform several major sub-functions (e.g. providing optical flow information for posture control, providing information about the nearby 3-d structure of the environment for short term motor planning, providing 2-d alignment information for checking direction of movement, providing information for future path-finding, providing enjoyment of the scenery, etc. etc.) I suspect that it would be quite useful for a new sub-stream of AI research to address the problem of how best to think of the high-level decomposition of a typical human mind. (Re-invent faculty psychology from an engineering standpoint?) Although some AI people have always emphasised the need to think about complete systems (the motivation for some AI robot projects), it has simply not been technically possible to aim at anything but vastly oversimplified designs. Roboticists don't normally include the study of motivation, emotions, visual enjoyment, conceptual learning, social interaction, etc, etc. So, very little is known about what kind of high level architecture might be required for designing something close to human capabilities. The techniques of software engineering (e.g. requirements analysis) coupled with philosophical conceptual analysis and surveys of what is known from psychology, linguistics, anthropology, etc. might eventually lead us to some useful conjectures about the global architecture, that could then be tested by a combination of implementational experiments (using AI, NN, or any other relevant techniques) and directed neurobiological studies. (I've been trying to do this recently in connection with attitudes, motivation, emotions and attention.) 7. Top-down illumination from this kind of architectural analysis may be required to give some direction (a) to conventional AI (since the space of possible software systems is too vast to be explored only bottom up) (b) to the exploration of NNs (since the space of possible networks, with different topologies, different self-modification algorithms, different threshold functions, etc. etc. is also too vast to be explored bottom up) and (c) to the study of real brains, since without good hypotheses about what a complex and intricate mechanism might be for, and how its functions might be implemented, it is too easy to concentrate on the wrong features: e.g. details that have little relevance to how the total system works. (Like measuring the shape, density, elasticity, etc. of something because you don't know that it's primarily a resistor in an electronic circuit. How can neurobiologists tell whether they are making this sort of mistake?) 8. The formal study of global architectures is somewhat different from mathematical analysis of formalisms or algorithms in computer science, and different from the kind of numerical mathematics that has so far dominated NN research. It will require at least a considerable generalisation of the mathematics of control theory to incorporate techniques for representing mutual causal interactions between systems undergoing qualitative and structural changes that cannot be accommodated in a set of differential equations relating a fixed set of variables. It will probably require the invention (or discovery?) of a host of new organising technical concepts, roughly analogous to the earlier discovery of concepts like feedback, information (in the mathematical sense), formal grammars, etc. (I think Lev Goldfarb was saying something similar (Sat, 22 Dec 90 00:57:29 AST), but I am not sure I understood it right.) 9. Minimising the significance of the AI/NN divide: I conjecture that most of the things that interest psychologists and cognitive scientists about human beings, e.g. our ability to perceive, learn, think, plan, act, communicate, co-operate, have desires, have emotions, be self-aware, etc. etc. depend more on the global architecture (i.e. how many co-existing, functionally distinct, causally interacting, sub-mechanisms there are, what their causal relationships are, and what functions they support in the total system) than on the implementation details of sub-mechanisms. It is not obvious what difference it makes how the various sub- components are implemented. E.g. for many components the difference between an NN implementation and a more conventional AI implementation may make a difference to speed (on particular electronic technology), or flexibility, or reliability, or modifiability -- differences that are marginal compared with the common functionality that arises not from the implementation details of individual systems but from the causal relations with other parts of the whole system. (Philosophical, scientific, and engineering issues converge here.) (Compare replacing one make of capacitor or resistor in an electronic circuit with another that has approximately the same behaviour: if the circuit is well designed, the differences in detailed behaviour will not matter, except perhaps in highly abnormal conditions, eg. high temperatures, high mechanical stress, or when exposed to a particular kind of corrosive gas etc. If the gas turns up often, the difference is important. Otherwise not.) It is quite likely that different sorts of implementation techniques will be needed for different sub-functions. E.g. rapid visual-motor feedback involved in posture control in upright bipeds (who are inherently very unstable) may be best served by NNs that map input vectors into output vectors. For representing the main high level steps in a complex sequence of actions (e.g. tying a shoelace) or for working out a plan to achieve a number of goals in a richly structured environment, very different mechanisms may be more suitable, even if NN's are useful for transforming low-level plan details to signals for co-operating muscles. NNs as currently studied may not be the best mechanism for accurate storage of long sequences of items, e.g. the alphabet, a poem, a dance routine, a memorised piano sonata, etc. When we have a good theory of the global architecture we'll be in a better position to ask which sub-mechanisms are best implemented in which ways. However, using a less suitable mechanism for one of the components may, like changing a resistor in a circuit, produce only a difference in degree, not kind, of capability for the whole system. (Which is why I think the NN/AI distinction is of no philosophical significance. This agrees with Beth Preston (21 Dec 90 16:35:15 EST) but for different reasons.) 10. The conjecture that implementation details of sub-mechanisms is relatively unimportant in explaining global capabilities, will be false (or partly false) to the extent that high level functionality depends essentially on close-coupling of different implementation levels. Are the mechanisms that allow alcohol (or other drugs) to produce qualitative changes in high level processes intimately bound up with chemical control mechanisms that are essential for normal human functioning, in the sense that no other implementation would have worked, or are they side-effects of biologically inessential implementation details that result from accidents of evolutionary history? We know that natural heart valves and kidneys can be replaced by artificial ones made very differently. We don't yet know which brain sub-mechanisms could also be replaced because most of the detail is inessential. When we know more, it may turn out that in order to explain human behaviour when drugged it is necessary to look at details that are irrelevant when explaining normal capabilities, from the design standpoint. Of course, some neurobiologists will not accept that two essentially similar circuits have the same functionality for the same reason if their components are made differently: but that's just a kind of scientific myopia. (I am not sure whether Jim Bower was saying that.) 11. It may also turn out that some aspect of the global architecture, for example the nature of the required causal links between different sub-mechanisms, favours one kind of implementation over another. Is there any intrinsic difference in the kind of information flow, or control flow, that can be implemented (a) between two or more components linked only by a few high speed parseable byte-streams, (b) between components linked by shared recursive data-structures accessed via pointers in a shared name-space, and (c) between two components linked by a web of connecting fibres? (I am not saying these are the only options for communication between sub-mechanisms.) I suspect not: only differences in speed and resistance to physical damage, etc. But perhaps there are important relevant theorems I don't know about (perhaps not yet proven?). 12. There are many problems not yet addressed by either NN or AI research, and some addressed but not solved. E.g. I suspect that neither has much to say about the representation of (arbitrary) shapes in visual systems, such that the representation can both be quickly derived from sampling the optic array and can also usefully serve a multiplicity of purposes, including: recognition, finding symmetries, seeing similarity of structure despite differences of detail, fine motor control, motor planning, explaining an object's capabilities, predicting points of contact of moving objects, etc. etc. Adequate representations of spatial structure and motion may require the invention of quite new techniques. 13. Although not everyone should put all their efforts into this, I commend the interdisciplinary exploration of the space of possible global architectures to people working in AI, NN, and neurobiology. (We may need some help from philosophers, software engineers, mathematicians, linguists, psychologists, ....) I fear there are probably a lot more fads and fashions waiting to turn up. Apologies: this message grew too long. Aaron Sloman, EMAIL aarons at cogs.sussex.ac.uk aarons%uk.ac.sussex.cogs at nsfnet-relay.ac.uk aarons%uk.ac.sussex.cogs%nsfnet-relay.ac.uk at relay.cs.net From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Fri Dec 28 14:04:04 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Fri, 28 Dec 90 11:04:04 PST Subject: Computational Neuroscience Message-ID: <901228110404.20401b2b@Iago.Caltech.Edu> Re. Steve Lehars distinction between biological connectionism and computational neuroscience. ... > BIOLOGICAL CONNECTIONISM: Using connectionist ideas to explore the > functional architecture of specific biological circuits found in > nature. (Paying attention to the detailed structure of the brain as > biologists) > > COMPUTATIONAL NEUROSCIENCE: Exploring the theoreticalpossibilities > of abstract connectionist architectures for their information > representation and processing potential. ... I, for once, disagree with these definitions. The aim of computational From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Fri Dec 28 14:04:04 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Fri, 28 Dec 90 11:04:04 PST Subject: Computational Neuroscience Message-ID: <901228110404.20401b2b@Iago.Caltech.Edu> Re. Steve Lehars distinction between biological connectionism and computational neuroscience. ... > BIOLOGICAL CONNECTIONISM: Using connectionist ideas to explore the > functional architecture of specific biological circuits found in > nature. (Paying attention to the detailed structure of the brain as > biologists) > > COMPUTATIONAL NEUROSCIENCE: Exploring the theoreticalpossibilities > of abstract connectionist architectures for their information > representation and processing potential. ... I, for once, disagree with these definitions. The aim of computational From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Fri Dec 28 14:10:05 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Fri, 28 Dec 90 11:10:05 PST Subject: Sorry, garbled transmission Message-ID: <901228110911.20401b36@Iago.Caltech.Edu> Re. Steve Lehars distinction between biological connectionism and computational neuroscience. ... > BIOLOGICAL CONNECTIONISM: Using connectionist ideas to explore the > functional architecture of specific biological circuits found in > nature. (Paying attention to the detailed structure of the brain as > biologists) > > COMPUTATIONAL NEUROSCIENCE: Exploring the theoreticalpossibilities > of abstract connectionist architectures for their information > representation and processing potential. ... I, for once, disagree with these definitions. The aim of computational From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Fri Dec 28 14:10:05 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Fri, 28 Dec 90 11:10:05 PST Subject: Sorry, garbled transmission Message-ID: <901228110911.20401b36@Iago.Caltech.Edu> Re. Steve Lehars distinction between biological connectionism and computational neuroscience. ... > BIOLOGICAL CONNECTIONISM: Using connectionist ideas to explore the > functional architecture of specific biological circuits found in > nature. (Paying attention to the detailed structure of the brain as > biologists) > > COMPUTATIONAL NEUROSCIENCE: Exploring the theoreticalpossibilities > of abstract connectionist architectures for their information > representation and processing potential. ... I, for once, disagree with these definitions. The aim of computational From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Fri Dec 28 14:20:16 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Fri, 28 Dec 90 11:20:16 PST Subject: One last time Message-ID: <901228111456.2040193d@Iago.Caltech.Edu> Re. Steve Lehars distinction between biological connectionism and computational neuroscience. ... > BIOLOGICAL CONNECTIONISM: Using connectionist ideas to explore the > functional architecture of specific biological circuits found in > nature. (Paying attention to the detailed structure of the brain as > biologists) > > > COMPUTATIONAL NEUROSCIENCE: Exploring the theoreticalpossibilities > of abstract connectionist architectures for their information > representation and processing potential. ... I, for once, disagree with these definitions. The aim of computational neuroscience (CN) is to understand information processing, storage and propagation in nervous systems, from nematodes to men. Within CN, theories can exist at many different levels of organization and complexity, ranging from biophysical faithful models of propagation of action potentials in axons (e.g. Hodgkin-Huxley and non-linear cable theory) to much more abstract models of, say, the computations underlying optical flow in visual cortex to even more abstract connectionists models of visual information processing (e.g. shape- from-shading) or higher-level cognitive operations. All these models are constrained to a greater-or-lesser extent by neurobiological and psychophysical data appropriate to their level of investigation. Thus, it would not make sense to simulate the diffusion equation in single dendritic spines when considering how we compute stereo acuity (we do not have to simulate the laws governing current flowing through a transistor when trying to understand the FFT algorithm). Thus, connectionists modelsQ-if appropriately mapped onto biologyQ-are a part of CN. Another distinction that can be made is between simplified and realistic models. The Reichardt-correlation model of motion detection in insects is a beuatiful instance of this. This model describes how the steady- state optomotor response of the fly to moving stimuli at the formal mathematical level and therefore captures the essential nonlinearity in this computation. It even carries over to human short-range motion system. Yet it specifies nothing about the implementation. This is for a latter, more realistic model. On the other hand, we will never understand the brain by building a huge detailed model of it, simulating every neuron in great detail. Even if we could, this simulation would be as complex and ill-understood as the brain itself. Thus, we need both types of models. This is a point NOT always appreciated by experimentalists, whose frequent objection to a theory is ...it does not explain my favorite observation XYZ... The point is, is this observation relevan t towards understanding the specific computation considered? For more details on this see our article on Computational Neuroscience by Sejnowski, Churchland and Koch, Science, 1988 Christof koch at iago.caltech.edu , From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Fri Dec 28 14:20:16 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Fri, 28 Dec 90 11:20:16 PST Subject: One last time Message-ID: <901228111456.2040193d@Iago.Caltech.Edu> Re. Steve Lehars distinction between biological connectionism and computational neuroscience. ... > BIOLOGICAL CONNECTIONISM: Using connectionist ideas to explore the > functional architecture of specific biological circuits found in > nature. (Paying attention to the detailed structure of the brain as > biologists) > > > COMPUTATIONAL NEUROSCIENCE: Exploring the theoreticalpossibilities > of abstract connectionist architectures for their information > representation and processing potential. ... I, for once, disagree with these definitions. The aim of computational neuroscience (CN) is to understand information processing, storage and propagation in nervous systems, from nematodes to men. Within CN, theories can exist at many different levels of organization and complexity, ranging from biophysical faithful models of propagation of action potentials in axons (e.g. Hodgkin-Huxley and non-linear cable theory) to much more abstract models of, say, the computations underlying optical flow in visual cortex to even more abstract connectionists models of visual information processing (e.g. shape- from-shading) or higher-level cognitive operations. All these models are constrained to a greater-or-lesser extent by neurobiological and psychophysical data appropriate to their level of investigation. Thus, it would not make sense to simulate the diffusion equation in single dendritic spines when considering how we compute stereo acuity (we do not have to simulate the laws governing current flowing through a transistor when trying to understand the FFT algorithm). Thus, connectionists modelsQ-if appropriately mapped onto biologyQ-are a part of CN. Another distinction that can be made is between simplified and realistic models. The Reichardt-correlation model of motion detection in insects is a beuatiful instance of this. This model describes how the steady- state optomotor response of the fly to moving stimuli at the formal mathematical level and therefore captures the essential nonlinearity in this computation. It even carries over to human short-range motion system. Yet it specifies nothing about the implementation. This is for a latter, more realistic model. On the other hand, we will never understand the brain by building a huge detailed model of it, simulating every neuron in great detail. Even if we could, this simulation would be as complex and ill-understood as the brain itself. Thus, we need both types of models. This is a point NOT always appreciated by experimentalists, whose frequent objection to a theory is ...it does not explain my favorite observation XYZ... The point is, is this observation relevan t towards understanding the specific computation considered? For more details on this see our article on Computational Neuroscience by Sejnowski, Churchland and Koch, Science, 1988 Christof koch at iago.caltech.edu , From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Sat Dec 29 02:42:07 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Fri, 28 Dec 90 23:42:07 PST Subject: logout In-Reply-To: Your message <90Dec28.112432edt.827@neuron.ai.toronto.edu> dated 28-Dec-1990 Message-ID: <901228232412.20401d16@Iago.Caltech.Edu> Having carried out detailed biophysical simulation of single neurons at the single cell level all my professional life with the aim of trying to identify and understand the elementary biophysical mechanisms underlying information processing , I disagree with Steve LeharUs statement (in reply to JimUs earlier comment): This is a curious view in an age when science has successfully probed the atomic element of the brain, the neuron, sufficiently to understand its major mode of operation. . . We do NOT understand why neurons have dendritic trees and why they come in diffe rent sizes and shapes, what---if any--- nonlinear operations go on there, why the membrane at the cell body contains more than a dozen distinct ionic currents, what nonlinear functional is being computed bat the soma, etc. etc. On the other hand, we have made progress. Thus, modeling the I/O capabilities of a neuron via linear synaptic interaction plus a nonlinear squash ing function (e.g. the Hopfield model) at the soma is a more faithful rendition than the binary McCulloch and Pitts neuron. Adding sigma-pi capabilities, i.e. allowing for the possibility of synaptic products (nominals) is a further improvement. The recent realization that the detailed temporal structure of the action potential discharge might be terrible relevant (e.g. 40 Hz oscillations) to its function a further improvment in our view of the neuron. Thus, even if some of todayUs connectionists model are still crude, they point in the right direction. Christof koch at iago.caltech.edu From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Sat Dec 29 02:42:07 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Fri, 28 Dec 90 23:42:07 PST Subject: logout In-Reply-To: Your message <90Dec28.112432edt.827@neuron.ai.toronto.edu> dated 28-Dec-1990 Message-ID: <901228232412.20401d16@Iago.Caltech.Edu> Having carried out detailed biophysical simulation of single neurons at the single cell level all my professional life with the aim of trying to identify and understand the elementary biophysical mechanisms underlying information processing , I disagree with Steve LeharUs statement (in reply to JimUs earlier comment): This is a curious view in an age when science has successfully probed the atomic element of the brain, the neuron, sufficiently to understand its major mode of operation. . . We do NOT understand why neurons have dendritic trees and why they come in diffe rent sizes and shapes, what---if any--- nonlinear operations go on there, why the membrane at the cell body contains more than a dozen distinct ionic currents, what nonlinear functional is being computed bat the soma, etc. etc. On the other hand, we have made progress. Thus, modeling the I/O capabilities of a neuron via linear synaptic interaction plus a nonlinear squash ing function (e.g. the Hopfield model) at the soma is a more faithful rendition than the binary McCulloch and Pitts neuron. Adding sigma-pi capabilities, i.e. allowing for the possibility of synaptic products (nominals) is a further improvement. The recent realization that the detailed temporal structure of the action potential discharge might be terrible relevant (e.g. 40 Hz oscillations) to its function a further improvment in our view of the neuron. Thus, even if some of todayUs connectionists model are still crude, they point in the right direction. Christof koch at iago.caltech.edu From gluck%psych at Forsythe.Stanford.EDU Mon Dec 31 23:41:43 1990 From: gluck%psych at Forsythe.Stanford.EDU (Mark Gluck) Date: Mon, 31 Dec 90 20:41:43 PST Subject: Full/Part-Time Research Assistant & Programmer Positions Message-ID: <9101010441.AA19467@psych> Two Full/Part Time Research Assistant Positions in: --------------------------------------------------- COGNITIVE PSYCHOLOGY / NEURAL NETWORK MODELING at Rutgers University Center for Molecular & Behavioral Neuroscience 195 University Avenue Newark, NJ 07102 Two research positions are available for persons interested in pursuing empirical and/or theoretical research in the in cognitive and neural sciences. The positions are ideal for someone who has just graduated with an undergraduate degree and would like a year or two of "hands on" experience in research before applying to graduate school in one of the cognitive sciences (e.g., neuroscience, psychology, computer science). We are looking for two people: 1. RESEARCH PROGRAMMER: A person with strong programming skills to work in the development of computational theories of the neural & cognitive bases of learning. Familiarity with current PDP/neural-network algorithms and research would be helpful, as would experience with C/Unix and Sun computer systems. Work would either focus on the development of network models of human learning and/or biological-circuit models of the neural bases of animal learning. 2. EXPERIMENTAL RESEARCH ASSISTANT: A person with experience in running and designing human cognitive psychology experiments to work in the design, execution, and data analysis of behavioral studies of human categorization learning. __________________________________________________________________________ Other Information: FACILITIES: The Center is a new state-funded research center for the integrated studies of cognitive, behavioral, and molecular neuroscience. The Center has good computational resources and experimental laboratories for behavioral and neural studies. LOCATION: The Center is located in Newark, NJ, approximately 20 minutes outside of Manhattan, New York (with easy train and subway access to midtown and downtown NYC) and close to rural New Jersey countryside. Numerous other research centers in the cognitive and neural sciences are located nearby including: Cognitive Science Center, Rutgers/New Brunswick; Centers for Cognitive & Neural Science, New York University; Cognitive Science Center, Princeton Univ.; Columbia Univ. & Medical School; Siemens Corporate Research, Princeton, NJ; NEC Research Labs, Princeton, NJ; AT&T Labs; Bellcore; IBM T. J. Watson Research Labs. CURRENT FACULTY: E. Abercrombie, G. Buzsaki, I. Creese, M. Gluck, H. Poizner, R. Siegel, P. Tallal, J. Tepper. Six additional faculty will be hired. The center has a total of ten state-funded postdoctoral positions and will direct, in collaboration with the Institute for Animal Behavior, a graduate program in Behavioral and Neural Sciences. ---------------------------------------------------------------------------- For more information on learning research at the CMBN/Rutgers or to apply for these post-doctoral positions, please send a cover letter with a statement of your research interests, a CV, copies of relevant preprints, and the the names & phone numbers of references to: Dr. Mark A. Gluck Phone: (415) 725-2434 Dept. of Psychology <-[Current address to 4/91] FAX: (415) 725-5699 Jordan Hall; Bldg. 420 Stanford University email: gluck at psych.stanford.edu Stanford, CA 94305-2130 From thomasp at informatik.tu-muenchen.dbp.de Thu Dec 20 11:08:18 1990 From: thomasp at informatik.tu-muenchen.dbp.de (Patrick Thomas) Date: 20 Dec 90 17:08:18+0100 Subject: TR on Modelling of Synaptic Plasticity Message-ID: <9012201608.AA13167@gshalle1.informatik.tu-muenchen.de> The following technical report is now available: BEYOND HEBB SYNAPSES: BIOLOGICAL BUILDING BLOCKS FOR UNSUPERVISED LEARNING IN ARTIFICIAL NEURAL NETWORKS Patrick V. Thomas Report FKI-140-90 Abstract This paper briefly reviews the neurobiology of synaptic plasticity as it is related to the formulation of learning rules for unsupervised learning in artificial neural networks. Presynaptic, postsynaptic and heterocellular mechanisms are discussed and their relevance to neural modelling is assessed. These include a variety of phenomena of potentiation as well as depression with time courses of action ranging from milliseconds to weeks. The original notion put forward by Donald Hebb stating that synaptic plasticity depends on correlated pre- and postsynaptic firing is reportedly inadequate. Although postsynaptic depolarization is necessary for associative changes in synaptic strength to take place (which conforms to the spirit of the hebbian law) the association is understood as being formed between pathways converging on the same postsynaptic neuron. The latter only serves as a supporting device carrying signals between activated dendritic regions and maintaining long-term changes through molecular mechanisms. It is further proposed to restrict the interactions of synaptic inputs to distinct compartments. The hebbian idea that the state of the postsynaptic neuron as a whole governs the sign and magnitude of changes at individual synapses is dropped in favor of local mechanisms which guide the depolarization-dependent associative learning process within dendritic compartments. Finally, a framework for the modelling of associative and non-associative mechanisms of synaptic plasticity at an intermediate level of abstraction, the Patchy Model Neuron, is sketched. To obtain a copy of the technical report FKI-140-90 please send your physical mail address to either "thomasp at lan.informatik.tu-muenchen.de" or Patrick V. Thomas, Institute for Medical Psychology, Goethe-31, 8000 Munich 2, Germany. From dwunsch at blake.u.washington.edu Mon Dec 3 14:47:46 1990 From: dwunsch at blake.u.washington.edu (Don Wunsch) Date: Mon, 3 Dec 90 11:47:46 -0800 Subject: IJCNN-91-Seattle paper deadline is coming up! Message-ID: <9012031947.AA23852@blake.u.washington.edu> JCNN '91 Seattle Call for Papers The International Neural Networks Society (INNS) and the Institute for Electronic and Electrical Engineers (IEEE) invite all persons interested in the field of Neural Networks to submit papers for possible presentation at the Conference. Papers must be RECEIVED by February 1, 1991. Submissions received after February 1, 1991 will be returned unopened. All submissions will be acknowledged by mail. International authors should submit their work via Air Mail or Express courier so as to ensure timely arrival. Eight copies (one original and seven copies) are required for submission. Do not fold or staple the original, camera-ready copy. Do not number the pages on the front of the camera-ready copy. Papers of no more than six pages, including figures, tables and references, should be written in English, and only complete papers will be considered. Papers must be submitted camera-ready on 8 1/2" by 11" white bond paper with 1" margins on each of the top, bottom, left and right sides, and un-numbered. They should be prepared by a typewriter or letter-quality printer in one-column format, single-spaced, in Times or similar style font of 10 point or larger, and should be printed on one side of the paper only. FAX submissions are not acceptable. Centered at the top of the first page should be the complete title, author name(s), affiliation(s) and mailing address(es). This is followed by a space and then the abstract, up to 15 lines, followed by the text. In an accompanying letter, the fillowing must be included: Corresponding Author: Name, mailing address, telephone and FAX numbers Technical Area (Neurobiology, applications, electronic implementations, optical implementations, image processing, vision, speech, network dynamics, optimization, robotics and control, learning and generalization, neural network architectures, or other) Presentation Format Preferred: Oral or Poster Presenter: Name, mailing address, telephone and FAX numbers If an oral talk is requested, include any special audio/video requests. Special audio/video requests beyond 35mm slide and overhead transparency projectors will be honored only if there are sufficient requests to justify them. If you have special audio/video needs, please contact Sarah Eck at conference management for more information. Send Papers to: IJCNN '91 Seattle Conference Management, Attn. Sarah Eck, MS/GH-22, Suite 108 5001 25th Ave. NE, Seattle, WA 98195 Tel (206) 543-0888 FAX (206) 685-9359 DEADLINE FEBRUARY 1, 1991 Submissions received after this date will be returned unopened. The cost for IJCNN-91-Seattle is $195.00 for INNS and IEEE members for early registration, deadline March 1, 1991. Non-members early registration is $295.00 and students will pay $50.00. Late registration will be honored until June 1, 1991 to members at $295.00, non-members at $395.00, and students at $75.00. On-site registration will be $395.00, $495.00 and $95.00 respectively. Tutorials will be offered at an additional cost of $195.00, or $295.00 for tutorial registration on site. Exhibitors will present the latest in neural networks, including neurocomputers, VLSI neural networks, implementations, software systems and applications at IJCNN-91-SEATTLE. IJCNN-91-SEATTLE is the neural network industry's largest trade show. Hope to see you there! Don From dyer at CS.UCLA.EDU Mon Dec 3 23:23:12 1990 From: dyer at CS.UCLA.EDU (Dr Michael G Dyer) Date: Mon, 3 Dec 90 20:23:12 PST Subject: reprints available Message-ID: <901204.042312z.04326.dyer@lanai.cs.ucla.edu> reprints available: Dyer, M. G. Distributed symbol formation and processing in connectionist networks. Journal of Experimental and Theoretical Artificial Intelligence. Vol. 2, 215-239, 1990. Abstract: Distributed connectionist (DC) systems offer a set of processing features which are distinct from those provided by traditional symbol processing (SP) systems. In general, the features of DC systems are derived from the nature of their distributed representations. Such representations have a microsemantics -- i.e. symbols with similar internal representations tend to have similar processing effects. In contrast, the symbols in SP systems have no intrinsic microsemantics of their own; e.g. SP symbols are formed by concatenating ASCII codes that are static, human engineered, and arbitrary. Such symbols possess only a macrosemantics -- i.e. symbols are placed into structured relationships with other symbols, via pointers, and bindings are propagated via variables. The fact that DC and SP systems each provide a distinct set of useful features serves as a strong research motivation for seeking a synthesis. What is needed for such a synthesis is a method by which symbols can dynamically form their own microsemantics, while at the same time entering into structured, recursive relationships with other symbols, thus developing also a macrosemantics. Here, we describe a general method, called symbol recirculation, for allowing symbols to form their own microsemantics. We then discuss three techniques for implementing variables and bindings in DC systems. Finally, we describe a number of DC systems, based on these techniques, which perform a variety of high-level cognitive tasks. requests for reprints should be sent to: valerie at cs.ucla.edu From dwunsch at blake.u.washington.edu Tue Dec 4 02:29:44 1990 From: dwunsch at blake.u.washington.edu (Don Wunsch) Date: Mon, 3 Dec 90 23:29:44 -0800 Subject: left-out info: IJCNN-91-Seattle Message-ID: <9012040729.AA14159@blake.u.washington.edu> >Date: Tue, 4 Dec 90 12:15 HKT >Hi, >The date of the conference seems to be missing. Would you please post it on >the network again? Thanks. >Regards, >Dr. Dit-Yan Yeung Thanks, Dr. Yeung, for pointing out my oversight. I don't want to use up all my permitted postings, so I'll just post the few most critical lines, this time including the date. JCNN '91 Seattle, July 8-12, 1991 Papers must be RECEIVED by February 1, 1991. Send Papers to: IJCNN '91 Seattle Conference Management, Attn. Sarah Eck, MS/GH-22, Suite 108 5001 25th Ave. NE, Seattle, WA 98195 Tel (206) 543-0888 FAX (206) 685-9359 The cost for IJCNN-91-Seattle is $195.00 for INNS and IEEE members for early registration, deadline March 1, 1991. Non-members early registration is $295.00 and students will pay $50.00. Late registration will be honored until June 1, 1991 to members at $295.00, non-members at $395.00, and students at $75.00. On-site registration will be $395.00, $495.00 and $95.00 respectively. Finally, another addition: anyone interested in volunteering should contact me at dwunsch at blake.u.washington.edu as soon as you can. Don From dave at cogsci.indiana.edu Tue Dec 4 19:17:03 1990 From: dave at cogsci.indiana.edu (David Chalmers) Date: Tue, 4 Dec 90 19:17:03 EST Subject: FTP archives; Technical report available Message-ID: (1) Following many requests, the bibliography that I have compiled on the philosophy of mind/cognition/AI is now available by anonymous ftp from cogsci.indiana.edu (129.79.238.6). It is contained in 5 files chalmers.bib.* in the directory "pub". Also contained in this archive are various articles by members of the Center for Research on Concepts and Cognition. Instructions for retrieval are given below. (2) The following technical report is now available. THE EVOLUTION OF LEARNING: AN EXPERIMENT IN GENETIC CONNECTIONISM David J. Chalmers Center for Research on Concepts and Cognition Indiana University CRCC-TR-47 This paper explores how an evolutionary process can produce systems that learn. A general framework for the evolution of learning is outlined, and is applied to the task of evolving mechanisms suitable for supervised learning in single-layer neural networks. Dynamic properties of a network's information-processing capacity are encoded genetically, and these properties are subjected to selective pressure based on their success in producing adaptive behavior in diverse environments. As a result of selection and genetic recombination, various successful learning mechanisms evolve, including the well-known delta rule. The effect of environmental diversity on the evolution of learning is investigated, and the role of different kinds of emergent phenomena in genetic and connectionist systems is discussed. A version of this paper appears in _Proceedings of the 1990 Connectionist Models Summer School_ (Touretzky, Elman, Sejnowski and Hinton, eds). ----------------------------------------------------------------------------- This paper may be retrieved by anonymous ftp from cogsci.indiana.edu (129.79.238.6). The file is chalmers.evolution.ps.Z, in the directory pub. To retrieve, do the following: unix-1> ftp cogsci.indiana.edu # (or ftp 129.79.238.6) Connected to cogsci.indiana.edu Name (cogsci.indiana.edu:): anonymous 331 Guest login ok, sent ident as password. Password: [identification] 230 Guest login ok, access restrictions apply. ftp> cd pub ftp> binary ftp> get chalmers.evolution.ps.Z ftp> quit unix-2> uncompress chalmers.evolution.ps.Z unix-3> lpr -P(your_local_postscript_printer) chalmers.evolution.ps The file is also available from the Ohio State neuroprose archives by the usual methods. If you do not have access to ftp, hardcopies may be obtained by sending e-mail to dave at cogsci.indiana.edu. From worth at park.bu.edu Wed Dec 5 11:28:44 1990 From: worth at park.bu.edu (Andrew J. Worth) Date: Wed, 5 Dec 90 11:28:44 -0500 Subject: Academic Program Info Request Message-ID: <9012051628.AA15635@park.bu.edu> ISSNNet Request for Academic Program Information The International Student Society for Neural Networks (ISSNNet) is compiling a list of academic programs relating to Neural Networks from around the world. We would like your input if you are a member of a scholastic program that is in any way related to Neural Networks, Parallel Distributed Processing, Connectionism, Computational Neuroscience, Neural Modeling, Neural Computing, etc. We hope to provide this service so that (1) interested students will be able to apply to those programs that will most closely satisfy their educational goals, and (2) current students and non-students will be aware of existing academic programs. This service is intended to not only provide an overview of these programs and contact points for more information, but also a personal glimpse into what's behind the official descriptions. All information will be made publicly available and will be updated as new programs are created and as programs change. Complying with ISSNNet's goal to be absolutely unbiased, we would like this to become THE source of information on academic programs in this field. ISSNNet would like to provide the following information: - Official address to contact for more information (surface mail and email) - Official description of the program. - Names of Faculty Members and their interests - Degrees requirements (BA, BS, MA, MS, PhD, etc.) - Short description of courses offered - Computing resources (Hardware and Software Tools) - Number of Students (grad/undergrad) and related faculty - A brief *personal* description of the program, department, etc. describing motivation, emphasis, goals, and/or overall ambiance. - Student Contacts (w/ telephone numbers, email and surface addresses, degree sought, interests, and date of graduation) This information is above and beyond the academic questionnaires that were filled out at the San Diego and Paris conferences and will eventually be made available via ftp and also by other means through ISSNNet (your submission will be taken as permission to make the information public unless we are otherwise notified). Coordinated responses from each institution are encouraged and will be appreciated. Please submit descriptions of academic programs in plain text (email is preferred) following the guidelines above to: issnnet-acad-progs at bucasb.bu.edu We will also be able to re-distribute information in other emailable formats such as postscript or LaTeX. Thank you for your time and effort, Andy. ---------------------------------------------------------------------- Andrew J. Worth (617) 353-6741 ISSNNet ISSNNet Academic Program Editor P.O. Box 557 issnnet-acad-progs at park.bu.edu New Town Br. worth at park.bu.edu Boston, MA 02215 USA ---------------------------------------------------------------------- From franklins at memstvx1.memst.edu Wed Dec 5 11:08:00 1990 From: franklins at memstvx1.memst.edu (franklins@memstvx1.memst.edu) Date: 5 Dec 90 10:08:00 CST Subject: Request for references Message-ID: I'm preparing to write two survey reports and would appreciate references. The first concerns applications of cellular automata to neural networks and/or their relative computational power. The second concerns neural computability, that is the neural network version of classical computability theory. Typical questions: What can neural networks compute under the most ideal conditions? Are there problems that are provably neurally unsolvable? I would greatly appreciate any reference to papers or articles touching upon these topics. If the response warrants, and if there is interest, I'll post bibliographies. I'll surely make the reports available. Stan Franklin Math Sciences Memphis State Memphis TN 38152 franklins at msuvx1.memst.edu franklins at memstvx1.bitnet From yuhas at faline.bellcore.com Wed Dec 5 17:30:23 1990 From: yuhas at faline.bellcore.com (Ben Yuhas) Date: Wed, 5 Dec 90 17:30:23 EST Subject: Commercial applications. Message-ID: <9012052230.AA14207@faline.bellcore.com> I am trying to gather a list of neural networks applications that have, or are about to, become commercial products. At NIPS we heard about such work from FORD and Sarnoff labs, I would appreciate any other examples that any of you are aware of. Ben yuhas at bellcore.com From reggia at cs.UMD.EDU Thu Dec 6 14:33:00 1990 From: reggia at cs.UMD.EDU (James A. Reggia) Date: Thu, 6 Dec 90 14:33:00 -0500 Subject: post-doctoral position in neural nets in France Message-ID: <9012061933.AA20258@mimsy.UMD.EDU> Post-Doctorate Position in France: A two year post-doctoral position in neural modelling is available at ONERA/CERT in Toulouse, France. ONERA/CERT is a government research laboratory: Office National d'Etudes et de Recherches Aerospatiales/Centre d'Etudes et de Recherches de Toulouse. The pay is approximately $2000/month. The working language is French, but most individuals at ONERA/CERT speak English fairly well. Work in this position would include one or more of the following: development and study of learning rules in competitive systems, matching images, or development of neural modelling software on a connection machine (the latter would require spending some time in Paris too). For further information or for answers to questions, please contact Paul Bourret at bourret at tls-cs.cert.fr via email. From ang at hertz.njit.edu Fri Dec 7 00:09:36 1990 From: ang at hertz.njit.edu (nirwan ansari fac ee) Date: Fri, 7 Dec 90 00:09:36 EST Subject: Tabu Search Message-ID: <9012070509.AA06607@hertz.njit.edu> I'm interested to do comparative studies between Tabu search and annealing algorithm. Could someone kindly give me references on tabu search? Thanks. From Dave.Touretzky at DST.BOLTZ.CS.CMU.EDU Fri Dec 7 02:04:35 1990 From: Dave.Touretzky at DST.BOLTZ.CS.CMU.EDU (Dave.Touretzky@DST.BOLTZ.CS.CMU.EDU) Date: Fri, 07 Dec 90 02:04:35 EST Subject: Tabu Search, and other stuff In-Reply-To: Your message of Fri, 07 Dec 90 00:09:36 -0500. <9012070509.AA06607@hertz.njit.edu> Message-ID: <9268.660553475@DST.BOLTZ.CS.CMU.EDU> > I'm interested to do comparative studies between Tabu search and > annealing algorithm. Could someone kindly give me references on tabu > search? Thanks. I want to remind people that the CONNECTIONISTS list is not intended for those who are too lazy to do their own library work. If you're going to post a plea for basic information, please do it on comp.ai.neural-nets. This list is intended for discussions among practicing researchers. Also, to save everyone else a lot of trouble: I never heard of Tabu search either. When Nirwan Ansari finds out what it is and compiles an extensive bibliography on the subject, I invite him to post *that* to CONNECTIONISTS. Meanwhile, here's some other news for the connectionist community: - The proceedings of the 1990 Connectionist Models Summer School, held at UCSD, are now available from Morgan Kaufmann. I'll post a table of contents and ordering information next week. - Reminder: final camera-ready papers for NIPS 3 are due by January 18. Author kits will be mailed out today or early next week. The formatting macros are very similar to last year. Seven page limit on all papers. - Reminder: the submission deadline for IJCAI papers is December 10. There is a "Connectionist and PDP Models" track. The submission deadline for AAAI-91 is January 30. -- Dave Touretzky From pako at neuronstar.it.lut.fi Fri Dec 7 07:49:28 1990 From: pako at neuronstar.it.lut.fi (Pasi Koikkalainen) Date: Fri, 7 Dec 90 14:49:28 +0200 Subject: ICANN-91 Message-ID: <9012071249.AA00402@neuronstar.it.lut.fi> January 15 is approaching fast .... ... but there is still time to write a paper for ICANN-91. -== ICANN-91 ====- -== INTERNATIONAL CONFERENCE ON ARTIFICIAL NEURAL NETWORKS ========- -== Helsinki University of Technology ==- -== Espoo, Finland, June 24-28, 1991 ==- Conference Chair: Conference Committee: Teuvo Kohonen (Finland) Bernard Angeniol (France) Eduardo Caianiello (Italy) Program Chair: Rolf Eckmiller (FRG) Igor Aleksander (England) John Hertz (Denmark) Luc Steels (Belgium) -== Second Announcement and Call for Papers =========================- THE CONFERENCE: ACTIVITIES: =============== ============ This conference will be a major - Oral and poster sessions international contact forum for - Invited talks experts from academia and industry - Industrial exhibition worldwide. Around 1000 participants - Prototype demonstrations are expected. - Video presentations -=============== TUTORIALS ==================- Nine tutorals will be given on Monday 24, 1990, covering the central techniques, developments, and prospects of Artificial Neural Networks. The tutorial speakers are leading experts in the filed: 1a J. Hertz - The Physics of Neural Networks 1b E. Oja - Pattern Recognition and Neural Networks 2 B. Widrow and T. Kohonen - Introduction to Neural Networks 3a J. Taylor - Mathematical Problems in Neural Networks Dynamics 3b F. Faggin - Hardware Implementations of Neural Networks 4a H. Ritter - Self-Organizing Map and Applications 4b T. Schwartz - How to Start a Business in Neural Networks 5a P. Werbos - Generalized Backprobagation: Basic Principles and Central Applications 5b P. Treleaven - Neural Programming Environment -==== INVITED SPEAKERS ====- In the oral sessions there will be invited talks given by some of the leading experts in various fields of Neural Networks. The invited speakers include: B. Angeniol (France), G. Carpenter (USA), R. Eckmiller (Germany), F. Fogelman (France), K. Goser (Germany), S. Grossberg (USA), J. Hertz (Denmark), K. Koenderink (Holland), A. Lansner (Sweden), C. von der Malsburg (Germany), W. von Seelen (Germany), J. G. Taylor (UK), P. Treleaven (UK) -================ PLENARY SESSIONS ====================- There will be several plenary sessions on topics that are of interest to all participants. The speakers who are pioneers in neural networks are: I. Alexander - Professor at Imperial College (England) A. Amari - Professor at Tokyo University (Japan) E. Caianiello - Professor at University of Salerno (Italy) F. Faggin - President of Synaptics Inc. (USA) R. Hecht-Nielsen - Chair of the Board of HNC corporation (USA) T. Kohonen - Professor at Helsinki University of Technology (Finland) =-= NON-COMMERCIAL DEMONSTRATIONS TRACK =-= As a new feature in neural network conferences participants will have a possibility to show video presentations and demonstrate prototype programs and systems on a non-commercial basis in a separate demonstration track, running in parallel with the oral and poster sessions. There will be a video room and PC/workstation classes available with standard equipment. The time slot reservation for the demonstration can be made using the registration form on which you also have to indicate the title of your demo. Detailed information will automatically be sent to those who reserve a time slot for demonstration. Further information can be requested from: Mr. Jari Kangas Helsinki University of Technology Laboratory of Computer and Information Science SF-02150 Espoo, Finland ----------------------------------------- E-mail (internet): icann91 at hutmc.hut.fi Fax: +358-0-4513277, Telex: 125161 HTKK SF =-= INSTRUCTIONS FOR AUTHORS =-= Complete papers of at most 6 pages are invited for oral or poster presentation in one of the sessions given below: 1. Mathematical theories of networks and dynamical systems 2. Neural network architectures and algorithms (including organizations and comparative studies) 3. Artificial associative memories 4. Pattern recognition and signal processing (especially vision and speech) 5. Self-organization and vector quantization 6. Robotics and control 7. "Neural" knowledge data bases and non-rule-based decision making 8. Software development (design tools, parallel algorithms, and software packages) 9. Hardware implementations (coprocessors, VLSI, optical, and molecular) 10. Commercial and industrial applications 11. Biological and physiological connection (synaptic and cell functions, sensory and motor functions, and memory) 12. Neural models for cognitive science and high-level brain functions 13. Physics connection (thermodynamical models, spin glasses, and chaos) Papers may be submitted for oral or poster presentation. All papers must be written in English. Only complete papers of at most 6 pages will be considered for oral presentations, and for 4 pages for posters. The program committee may designate a paper intended for oral presentation to a poster presentation instead, and may also change the intended session to balance the conference program. == DEADLINE IS January 15, 1991 Deadline for submitting manuscripts is January 15, 1991. The Conference Proceedings will be published as a book by Elsevier Science Publishers B.V. Therefore, the final versions must be typed or pasted on special forms provided by the publisher for authors of accepted papers. The papers will be reproduced directly from the received forms. In order to help the authors, the conference organizers, and the publisher, we request that the submitted manuscripts already follow the final layout. Therefore, please observe carefully the instructions below. 1. The typing area is 16.7 x 25.8 cm (6.5 x 10 in.) 2. Do not use page numbers 3. Use a font (also tables and figures) large enough to withstand reduction to 70%. Do not use font smaller than 11 points. 4. The title should be written in capital letters 2 cm from the top of the first page, followed by the authors' names and addresses and the abstract left-justified, indenting everything by 2 cm. 5. In the text, do not indent headings or captions. 6. Insert all tables, figures, and figure captions in the text at their final positions. 7. For references in the text, use numbers in square brackets. Submit 6 review copies of the manuscript. FAX OR EMAIL SUBMISSIONS ARE NOT ACCEPTED. With each manuscript, please indicate - the name of the principal author - the mail address, telephone, and fax numbers - whether the paper in intended for oral or poster presentation - which session it is submitted to (see sessions above). You can also give two alternatives. You will be notified of the review result by February 20, 1991, and the authors of accepted papers will receive an authors' kit from the publisher. Deadline for the final papers typed on the special forms is March 15, 1991. NOTICE! The final camera-ready papers must be received by the Organizing Committee by that date! === SEND THE MANUSCRIPTS TO: Prof. Olli Simula ICANN-91 Organization Chairman Helsinki University of Technology SF-02150 Espoo, Finland --------------------------- Fax: +358 0 451 3277 Telex: 125161 HTKK SF Email (internet): icann91 at hutmc.hut.fi -== CONFERENCE VENUE ===- The street address of the Conference venue is Helsinki University of Technology Otakaari 1 SF-02150 Espoo Finland -== SOCIAL PROGRAM, TOURS AND EXCURSIONS =====- In addition to the scientific program, several social occasions are included in the registration fee. These include: 24 June: Get-together party and opening of the exhibition 26 June: Concert sponsored by the City of Espoo 27 June: Banquet Several tours and excursions are optional: 24 June: City Sightseeing (90 FIM) 25 June: Porvoo by bus and boat (400 FIM) 26 June: Finnish Glass Discovery (350 FIM) 27 June: Design Tour (100 FIM) Pre- and post-conference tours and excursions will also be arranged: 21-23 June: Lapland with Midnight sun (2900 FIM) 22-23 June: Cruise to Tallinn (Estonia, USSR), (850 FIM) 28-30 June: Leningrad by air (USSR), (2950 FIM) -== GENERAL INFORMATION, REGISTRATION AND ACCOMMODATION ===- There will be a special ICANN-91 reception desk at Helsinki-Vantaa airport. The desk will be open on Sunday June 23 and on Monday June 24 from noon until midnight. Registration desk is located in the Lobby of the main building at the Helsinki University of Technology, address: Otakaari 1, 02150 Espoo. For more information about registration and accommodation, please contact: ICANN-91 CMS-CONGREX P.O.Box 151 Neitsytpolku 12 A SF-00141 Helsinki, Finland Tel.: +358 0 175 355 Fax: +358 0 170 122 Telex: 123 585 cms sf -------------------------------------------------------------------- From rwp at engineering.cambridge.ac.uk Fri Dec 7 07:21:57 1990 From: rwp at engineering.cambridge.ac.uk (Richard Prager) Date: Fri, 7 Dec 1990 12:21:57 GMT Subject: Cambridge Neural Networks Course Announcement Message-ID: <17498.9012071221@dsl.eng.cam.ac.uk> Cambridge University Programme for Industry Neural Networks Theory Design & Applications 15 - 19 April 1991 Preliminary Announcement A five-day advanced short course on the theory, design and applications of artificial neural networks, presented by leading international experts in the field: Professor David RUMELHART Stanford University Professor Geoffrey HINTON University of Toronto Dr Andy BARTO University of Massachusetts Dr Herve BOURLARD Philips Research Labs. Belgium Professor Elie BIENENSTOCK ESPCI Paris Professor Frank FALLSIDE University of Cambridge Professor Horace BARLOW University of Cambridge Dr Peter RAYNER University of Cambridge Dr Lionel TARASSENKO University of Oxford This intensive short course for scientists, engineers and their managers aims to develop an understanding of the potential for neural network-based solutions, and demonstrates techniques for transforming problems to enable neural networks to solve them more efficiently. Design methodologies for a number of common neural network architectures will be described. By the end of the course delegates will be able to assess the potential usefulness of neural network technology to their own application domains. They will have an understanding of the strength and weakness of a neural network approach and will have acquired an insight into factors affecting neural network design and performance. The lectures will be complemented by discussion sessions and practical computing sessions that will demonstrate simulated applications. The lectures will cover basic theory behind neural network algorithms, together with applications in speech and language processing, signal processing, and robotic control. If you are interested please print out the form below, fill it in and return to Pam Whitfield, Cambridge Programme for Industry. University of Cambridge. Department of Engineering. Trumpington Street. Cambridge. CB2 1PZ United Kingdom. ==================================================================== | Please send me full details of the course: NEURAL NETWORKS | | to be held at Pembroke College, Cambridge, England. | | 15 - 18 April 1991. COURSE FEE 875 pounds sterling. | | Accommodation can be arranged for delegates at Pembroke College. | | | | Name _______________________ Job Title ______________________ | | | | Company ____________________ Division _______________________ | | | | Address _____________________________________________________ | | | | _____________________________________________________ | | | | Postcode __________ Phone Number __________ Fax _____________ | ==================================================================== From fritz_dg%ncsd.dnet at gte.com Fri Dec 7 09:58:02 1990 From: fritz_dg%ncsd.dnet at gte.com (fritz_dg%ncsd.dnet@gte.com) Date: Fri, 7 Dec 90 09:58:02 -0500 Subject: requesting references on the list Message-ID: <9012071458.AA29639@bunny.gte.com> >> I'm interested to do comparative studies between Tabu search and >> annealing algorithm. Could someone kindly give me references on tabu >> search? Thanks. > I want to remind people that the CONNECTIONISTS list is not intended for > those who are too lazy to do their own library work. If you're going to > post a plea for basic information, please do it on comp.ai.neural-nets. > This list is intended for discussions among practicing researchers. I disagree. Extensive library work starting from scratch is basically a learning task for students who are still busy paying their dues (or professors writing monographs). Asking around for pointers into the literature is a time honored way of getting what you need without doing everything the hard way. Practicing Researchers are usually too busy producing to hit the library stacks with nothing but a ball-point pen in their hands every time there is a need to know. Also, some of us don't have convenient access to large libraries, or grad students to run there for us, and thus asking around can be crucial. Finally, it is often the case that if one person is interested in a topic, others are too. Any feedback from unexpected sources (eg. obscure, internally-circulated technical reports) would be a valuable, and valid, use of the list. D. Fritz fritz_dg%ncsd at gte.com From fozzard at boulder.Colorado.EDU Fri Dec 7 13:12:58 1990 From: fozzard at boulder.Colorado.EDU (Richard Fozzard) Date: Fri, 7 Dec 90 11:12:58 -0700 Subject: Tabu Search, and other stuff Message-ID: <9012071812.AA20992@alumni.colorado.edu> > I'm interested to do comparative studies between Tabu search and > annealing algorithm. Could someone kindly give me references on tabu > search? Thanks. I want to remind people that the CONNECTIONISTS list is not intended for those who are too lazy to do their own library work. If you're going to post a plea for basic information, please do it on comp.ai.neural-nets. This list is intended for discussions among practicing researchers. -- Dave Touretzky I think that this may be too strong a condemnation. I have done library search and at the same time posted to CONNECTIONISTS. I got quite a bit of information on works-in-progress, and other things I couldn't find in the library. To assume library indexes (indices?) are exaustive and up-to-date is rather unrealistic. That practicing researchers are on CONNECTIONISTS (and not on comp.ai.nn) is the very reason such requests are made. Of course, a busy practicing researcher is free not to respond to such requests. IMHO, Nirwan Ansari's 3-line message was not an abuse of bandwidth. He (she?) wasn't doing what we used to see a lot of, eg: "What's a good book on back propogation?", "Where can I find some neural net software?", etc. which I agree are inappropriate. If even Dave hasn't heard of this Tabu search, I wouldn't hold out much hope that anyone on comp.ai.nn has either. ======================================================================== Richard Fozzard "Serendipity empowers" Univ of Colorado/CIRES/NOAA R/E/FS 325 Broadway, Boulder, CO 80303 fozzard at boulder.colorado.edu (303)497-6011 or 444-3168 From koch%CITIAGO.BITNET at VMA.CC.CMU.EDU Fri Dec 7 14:05:14 1990 From: koch%CITIAGO.BITNET at VMA.CC.CMU.EDU (Christof Koch) Date: Fri, 7 Dec 90 11:05:14 PST Subject: Vehicle Guidance Workshop '91 Message-ID: <901207110314.20807159@Iago.Caltech.Edu> One-day workshop on ******************************************************* NEURAL AND FUZZY SYSTEMS, AND VEHICLE APPLICATIONS '91 ******************************************************* November 8, 1991, Tokyo, Japan ******************************************************* The Roundtable Discussion on "Neural and Fuzzy Systems, and Vehicle Applications" is tentatively scheduled for November 8, 1991, in Tokyo Japan. The focus of this roundtable discussion will be applications of neural nets and fuzzy logic to vehicles including automobiles, aircraft, and trains. The relationship between neural nets and fuzzy logic technologies will be another focus. Presentations of on-going projects as well as completed projects are welcome to stimulate the discussions. --------------------------------------------------------------- Please submit a one-page abstract by May 1, 1991, to Ichiro Masaki --------------------------------------------------------------- Related conferences include: IROS (International Workshop on Intelligent Robots and Systems) Nov. 3-5, Japan. IFES (International Fuzzy Engineering Syposium) Nov. 13-15, Japan. For further information, please contact: Ichiro Masaki Computer Science Department General Motors Research Laboratories 30500 Mound Road, Warren, Michigan 48090-9055, USA Office phone: 1-313-986-1466 Fax: 1-313-986-9356 E-Mail: MASAKI at GMR.COM If you're interested, please don't send mail to me but to Ichiro Masaki at masaki at gmr.com From inesc!lba at relay.EU.net Fri Dec 7 17:59:33 1990 From: inesc!lba at relay.EU.net (Luis Borges de Almeida) Date: Fri, 7 Dec 90 17:59:33 EST Subject: test Message-ID: <9012071759.AA11831@alf.inesc.pt> teste de uma nova mail-list; deitem fora, sff. Luis From kruschke at ucs.indiana.edu Fri Dec 7 15:48:00 1990 From: kruschke at ucs.indiana.edu (KRUSCHKE,JOHN,PSY) Date: 7 Dec 90 15:48:00 EST Subject: tech report: benefits of gain Message-ID: The following paper is available via ftp from the neuroprose archive at Ohio State (instructions for retrieval follow the abstract). This paper was witten more than two years ago, but we believe the ideas are still interesting even if the details are a bit dated. Benefits of Gain: Speeded learning and minimal hidden layers in back-propagation networks. John K. Kruschke Javier R. Movellan Indiana University Carnegie-Mellon University ABSTRACT The gain of a node in a connectionist network is a multiplicative constant that amplifies or attenuates the net input to the node. The objective of this article is to explore the benefits of adaptive gains in back propagation networks. First we show that gradient descent with respect to gain greatly increases learning speed by amplifying those directions in weight space that are successfully chosen by gradient descent on weights. Adpative gains also allow normalization of weight vectors without loss of computational capacity, and we suggest a simple modification of the learning rule that automatically achieves weight normalization. Finally, we describe a method for creating small hidden layers by making hidden node gains compete according to similarities between nodes, with the goal of improved generalization performance. Simulations show that this competition method is more effective than the special case of gain decay. To get a copy of the paper, do the following: unix> ftp cheops.cis.ohio-state.edu login: anonymous password: neuron ftp> cd pub/neuroprose ftp> binary ftp> get kruschke.gain.ps.Z ftp> bye unix> uncompress kruschke.gain.ps.Z unix> lpr kruschke.gain.ps From Dave.Touretzky at DST.BOLTZ.CS.CMU.EDU Sat Dec 8 00:34:33 1990 From: Dave.Touretzky at DST.BOLTZ.CS.CMU.EDU (Dave.Touretzky@DST.BOLTZ.CS.CMU.EDU) Date: Sat, 08 Dec 90 00:34:33 EST Subject: Tabu Search, and other stuff In-Reply-To: Your message of Fri, 07 Dec 90 11:12:58 -0700. <9012071812.AA20992@alumni.colorado.edu> Message-ID: <9767.660634473@DST.BOLTZ.CS.CMU.EDU> Okay, I was a little harder on Nirwan Ansari than I should have been. Sorry, Nirwan. But I see it's time to explain AGAIN what proper and improper uses of the CONNECTIONISTS list are. At the end of this message are some answers about Tabu search. It is always improper to ask questions on CONNECTIONISTS that you can answer for yourself with a few minutes of work, such as visiting the library or picking up a telephone. It took me a grand total of 15 *seconds* to find a reference to Tabu Search in our online library index. Most university libraries have access to some kind of on-line index, so there's no excuse for not looking there first. If one has absolutely no idea what Tabu search is, then one has no business bothering the CONNECTIONISTS readership with such an elementary question. Go to your library and DO THE WORK! Or pick up the phone and call the person who mentioned the term to you in the first place. Even though Rich Fozzard is correct that *someone* on this list is likely to have the answer to any technical question, this does not give people the right to waste the time of everyone on this list just to save themselves a tiny bit of work. Rich's comment that "busy researchers are free not to respond to such requests" is out of line, and misses the key point of CONNECTIONISTS: that busy researchers *will not be bothered* by such requests. People who refuse to understand this policy will be removed from the list. Notwithstanding the above, there *is* a proper way to ask for references on CONNECTIONISTS, and I agree with Rich that you can find things here that a library search won't turn up. But you should (a) give people something back in return for bothering them, and (b) demonstrate that you have already pursued the quick, obvious routes to finding the information you desire. The easiest way to do both these things is to FIRST do the library work to find the basic references, then POST these as part of your query. If this isn't concrete enough for some of you, here's an example: WRONG WAY: "Can someone please mail me all references to cascade correlation?" RIGHT WAY: "I'm looking for references to work on cascade correlation. I've already read Fahlman's paper in NIPS 2, and his NIPS 3 abstract, and found the code in the nn-bench archive. Is anyone aware of additional work with this algorithm? I'll summarize and post results to the list." Let's please not get into a long, boring thread about what constitutes appropriate postings to this list. If you feel you *must* express your opinion on this matter, do it via email to me alone. Most readers really don't want to hear about it. -- Dave ............ results of online index search on keyword "tabu": ............ Idnum 08784922 TYPE technical DATE 900900 AUTHOR Friden, C. and Hertz, A. and De Werra, D. TITLE Tabaris: an exact algorithm based on Tabu Search for finding a maximum independent set in a graph. (technical) SOURCE Computers & Operations Research v17 n5 p437(9) 1990 Sept SUBJECT Algorithm Analysis Theoretical Research New Technique Mathematical Models Algorithms Problem solving Graph theory Algorithms--analysis Problem solving--models Graph theory--research GRAPHICS table CAPTIONS Getting a maximum independent set in a graph. A general Tabu Search method. Algorithm STABULARGE for finding a large stable set. ABSTRACT The process of finding a maximum independent set in an arbitrary graph is examined; the problem is an ingredient of many coloring algorithms. An exact algorithm for constructing a maximum independent set in a graph is developed that is an implicit enumeration algorithm using Tabu Search techniques at some steps to obtain some bounds on the independence number of a subgraph of the graph to be examined. The general procedure is formalized, and the Tabu Search metaheuristic serving as an essential part of the enumeration procedure is described. From Dave.Touretzky at DST.BOLTZ.CS.CMU.EDU Sat Dec 8 00:40:20 1990 From: Dave.Touretzky at DST.BOLTZ.CS.CMU.EDU (Dave.Touretzky@DST.BOLTZ.CS.CMU.EDU) Date: Sat, 08 Dec 90 00:40:20 EST Subject: summer school proceedings: contents and ordering info Message-ID: <9773.660634820@DST.BOLTZ.CS.CMU.EDU> CONNECTIONIST MODELS: Proceedings of the 1990 Summer School Edited by David S. Touretzky (Carnegie Mellon University), Jeffrey L. Elman (University of California, San Diego), Terrence J. Sejnowski (The Salk Institute, UC San Diego), and Geoffrey E. Hinton (University of Toronto) ISBN 1-55860-156-2 $29.95 404 pages (For bibliographic purposes, the complete table of contents and contact numbers for additional information or for use in obtaining copies of this book follow the announcement.) TABLE OF CONTENTS PART I MEAN FIELD, BOLTZMANN, AND HOPFIELD NETWORKS Deterministic Boltzmann Learning in Networks with Asymmetric Connectivity 3 C.C. Galland and G.E. Hinton Contrastive Hebbian Learning in the Continuous Hopfield Model 10 J.R. Movellan Mean Field Networks that Learn to Discriminate Temporally Distorted Strings 18 C.K.I. Williams and G.E. Hinton Energy Minimization and the Satisfiability of Propositional Logic 23 G. Pinkas PART II REINFORCEMENT LEARNING On the Computational Economics of Reinforcement Learning 35 A.G. Barto and P.M. Todd Reinforcement Comparison 45 P. Dayan Learning Algorithms for Networks with Internal and External Feedback 52 J. Schmidhuber PART III GENETIC LEARNING Exploring Adaptive Agency I: Theory and Methods for Simulating the Evolution of Learning 65 G.F. Miller and P.M. Todd The Evolution of Learning: An Experiment in Genetic Connectionism 81 D.J. Chalmers Evolving Controls for Unstable Systems 91 A.P. Wieland PART IV TEMPORAL PROCESSING Back-Propagation, Weight Elimination and Time Series Prediction 105 A.S. Weigend, D.E. Rumelhart, and B.A. Huberman Predicting the Mackey-Glass Timeseries with Cascade-Correlation Learning 117 R.S. Crowder, III Learning in Recurrent Finite Difference Networks 124 F.S. Tsung Temporal Backpropagation: An Efficient Algorithm for Finite Impulse Response Neural Networks 131 E.A. Wan PART V THEORY AND ANALYSIS Optimal Dimensionality Reduction Using Hebbian Learning 141 A. Levin Basis-Function Trees for Approximation in High-Dimensional Spaces 145 T.D. Sanger Effects of Circuit Parameters on Convergence of Trinary Update Back-Propagation 152 R.L. Shimabukuro, P.A. Shoemaker, C.C. Guest, and M.J. Carlin Equivalence Proofs for Multi-Layer Perceptron Classifiers and the Bayesian Discriminant Function 159 J.B. Hampshire, II and B. Pearlmutter A Local Approach to Optimal Queries 173 D. Cohn PART VI MODULARITY A Modularization Scheme for Feedforward Networks 183 A. Ossen A Compositional Connectionist Architecture 188 J.R. Chen PART VII COGNITIVE MODELING AND SYMBOL PROCESSING From bates at crl.ucsd.edu Sun Dec 9 01:20:27 1990 From: bates at crl.ucsd.edu (Elizabeth Bates) Date: Sat, 8 Dec 90 22:20:27 PST Subject: Tabu Search, and other stuff Message-ID: <9012090620.AA11524@crl.ucsd.edu> It is clear that the connectionist list has gotten very large, and it now includes people at very different levels of expertise (including people like me who couldn't do a serious simulation if you put a gun to our heads, but still are interested in the basic ideas and the way that they are developing over a short period of time). I understand Touretzky's frustration with amateurs, and naive questions, but I thnk the community will be better off in the long run if you (we?) can tolerate a little more naivete, for a little longer. Important things are happening, and this medium is an important event in its own right. Let's stretch the limits a little longer, at the risk of putting up with a few more irritating messages in our daily logs than we might otherwise prefer. "in groups" abound. Generous and free-spirited exercises do not. -liz bates From eric at yin.nec.com Mon Dec 10 09:07:27 1990 From: eric at yin.nec.com (Eric B. Baum) Date: Mon, 10 Dec 90 09:07:27 EST Subject: No subject Message-ID: <9012101407.AA11222@yin.nec.com> I've been holding off replying re tabu because I'm no expert, but it appears that among connectionists few are, so here goes. Tabu search was invented by Fred Glover, Center for Applied Artificial Intelligence, Graduate School of Business, University of Colorado, Boulder. A thumbnail sketch of the idea (as I recollect it) is the following: We have a set of local search moves. So, if we're solving TSP, we might use the set of one link interchanges. At each step, we use the move in our set which produces the most optimum solution (e.g. shortest tour) with one proviso. We are not allowed to use a move which inverts one of our last x moves, where x is a small, heuristically chosen, fixed integer (e.g. 7). (i.e. each time we do a move , we remove the first element from a TABU set and insert the inverse of the current move as the x-th element in the TABU set). Note that, if we're near a local minimum for our search set, this procedure may force us to make a move which increases the tour length, since we must make some move at each step. We keep in storage the best solution yet found. We proceed till either a fixed number of iterations have been performed, or a fixed number have occured since last improvement in best value, halt and report the best value found. The general idea is that the TABU set prevents cycling, and otherwise one tries to take a naive most direct path over hills. This not only seems sensible, but is claimed to be extremely effective in many tests. One reference I have is: Glover F. "Tabu Search Methods in Artificial Intelligence and Operations Research" ORSA Artificial Intelligence Newsletter V1 No 2. 6 1987. Hopefully, somebody at Boulder reading this can encourage Prof Glover to post a more recent bibliography, since all I know about the subject comes from a talk I heard three years ago, and doubtless there have been improvements in the art. -- Eric Baum NEC Research Institute 4 Independence Way Princeton NJ 08540 Inet: eric at research.nj.nec.com UUCP: princeton!nec!eric MAIL: 4 Independence Way, Princeton NJ 08540 PHONE: (609) 951-2712 FAX: (609) 951-2482 From F_SIENKO at UNHH.UNH.EDU Mon Dec 10 13:07:22 1990 From: F_SIENKO at UNHH.UNH.EDU (F_SIENKO@UNHH.UNH.EDU) Date: Mon, 10 Dec 1990 13:07:22 EST Subject: Request for recurrent net learning code. Message-ID: <901210130722.20a0aacb@UNHH.UNH.EDU> My name is Fred Sienko, and I am a graduate student in electrical engineering at the University of New Hampshire. I am looking for a C code module which does real-time recurrent learning (Williams & Zipser) to use in a project I am currently working on. Does anyone have a copy available? From jesus!penrose at esosun.css.gov Mon Dec 10 22:57:34 1990 From: jesus!penrose at esosun.css.gov (Christopher Penrose) Date: Mon, 10 Dec 90 19:57:34 PST Subject: Tabu Search, and other stuff Message-ID: <9012110357.AA07004@ jesus > I want to remind people that the CONNECTIONISTS list is not intended for those who are too lazy to do their own library work. If you're going to post a plea for basic information, please do it on comp.ai.neural-nets. This list is intended for discussions among practicing researchers. Such an attitude is counter-productive. This statement also made me quite angry--it belittles the efforts to make the internet an interactive community that facilitates collective intellectual advancement. Christopher Penrose jesus!penrose From schmidhu at informatik.tu-muenchen.dbp.de Tue Dec 11 05:37:05 1990 From: schmidhu at informatik.tu-muenchen.dbp.de (Juergen Schmidhuber) Date: 11 Dec 90 11:37:05+0100 Subject: TRs Message-ID: <9012111037.AA22331@kiss.informatik.tu-muenchen.de> The revised and extended versions of two reports from February 1990 are available. 1. Networks adjusting networks. Technical Report FKI-125-90 (revised), Institut fuer Informatik, Technische Universitaet Muenchen, November 1990. 2. Making the world differentiable: On using fully recurrent self-supervised neural networks for dynamic reinforcement learning and planning in non-stationary environments. Technical Report FKI-126-90 (revised), Institut fuer Informatik, Technische Universitaet Muenchen, November 1990. To obtain hardcopies, send email to marquard at tumult.informatik.tu-muenchen.de Please let your message look like this: subject:FKI physical address (not more than 33 characters per line) Those who requested copies at NIPS should not send additional requests. Juergen Schmidhuber From nelsonde%avlab.dnet at wrdc.af.mil Tue Dec 11 10:09:03 1990 From: nelsonde%avlab.dnet at wrdc.af.mil (nelsonde%avlab.dnet@wrdc.af.mil) Date: Tue, 11 Dec 90 10:09:03 EST Subject: Constructive/Destructive NN Algorithms Message-ID: <9012111509.AA04225@wrdc.af.mil> I N T E R O F F I C E M E M O R A N D U M Date: 11-Dec-1990 10:27am EST From: Dale E. Nelson NELSONDE Dept: AAAT-1 Tel No: 57646 TO: Remote Addressee ( _LABDDN::"CONNECTIONISTS at CS.CMU.EDU" ) TO: Dennis S. Buck ( BUCKDS ) Subject: Constructive/Destructive NN Algorithms First, let me apologize for the formal formatting of this, but my access is through All-In-One and that is the way it comes out. I know from the NIPS Workshop, that there is intense interest in networks that develop their own topology during the training. I personally have had problems when refering to this class of algorithms. If they are called evolutionary networks, pepole say that you are using genetic algorithms. Well, that *could* be true, but that is too limiting. I would like to propose, for discussion, a possible name for this class of networks. This would give all researchers a single keyword which would facilitate searches and other research efforts. The term for these networks, which I would like to propose, is Ontogenic Neural Networks. This is based on the word ontogeny which is defined as: The development or developmental history of an individual organism. Ontogenic is the adjective form of the word. I hope that we can generate a dialogue and perhaps come to agreement on a terminology for this class of networks. What do you think??? Dale E. Nelson nelsonde%avlab.dnet at wrdc.af.mil From DBEDFORD%VAX.OXFORD.AC.UK at bitnet.CC.CMU.EDU Tue Dec 11 11:43:17 1990 From: DBEDFORD%VAX.OXFORD.AC.UK at bitnet.CC.CMU.EDU (DBEDFORD%VAX.OXFORD.AC.UK@bitnet.CC.CMU.EDU) Date: Tue, 11 DEC 90 16:43:17 GMT Subject: No subject Message-ID: <73F7C753C0E00F6C@BITNET.CC.CMU.EDU> Please add my name to your mailing list. Thanks , Dhugal Bedford ,University of Oxford,UK. DBEDFORD at UK.AC.OX.VAX From Scott.Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU Tue Dec 11 20:22:49 1990 From: Scott.Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF-PMAX.SLISP.CS.CMU.EDU) Date: Tue, 11 Dec 90 20:22:49 EST Subject: Constructive/Destructive NN Algorithms In-Reply-To: Your message of Tue, 11 Dec 90 10:09:03 -0500. <9012111509.AA04225@wrdc.af.mil> Message-ID: I know from the NIPS Workshop, that there is intense interest in networks that develop their own topology during the training. I personally have had problems when refering to this class of algorithms... The term for these networks, which I would like to propose, is Ontogenic Neural Networks. Clever, but I just can't see this term catching on. What's wrong with "constructive" and "destructive" (or maybe "additive" and "subtractive")? People immediately know what you're talking about. I don't think it's a big problem that there isn't a single word for the whole class. Usually you only want to refer to one kind or the other. Only leaders of workshops and (I hope!) funding agencies have any need to come up with one term that covers the whole spectrum of such approaches. -- Scott From koch%CITIAGO.BITNET at VMA.CC.CMU.EDU Wed Dec 12 00:32:52 1990 From: koch%CITIAGO.BITNET at VMA.CC.CMU.EDU (Christof Koch) Date: Tue, 11 Dec 90 21:32:52 PST Subject: Caltech's CNS Program Message-ID: <901211213209.20807118@Iago.Caltech.Edu> This is a short description of our CNS program. Deadline for application is end of January. Christof ******************************************************* CALIFORNIA INSTITUTE OF TECHNOLOGY ******************************************************* Computation and Neural Systems Program This interdepartmental program awards a Ph.D. in Computation and Neural Systems. No Master is awarded. Current enrollment: 28 doctoral, 18 postdoctoral Financial support: Complete support for tuition and stipend from graduate research assistantships, graduate teaching assistantships, NIH training grant, and private sources. Contact: J. Hopfield, Ph.D., Program Head, 160-30 (818) 356-2808 J. Bower, Ph.D., Chairman of Admissions, Biology Div., 216-76, (818) 356-6817 jbower at smaug.cns.caltech.edu All at California Institute of Technology, Pasadena, CA 91125 Caltech's graduate program in Computation and Neural Systems presently involves 16 faculty in the Division of Biology, Engineering and Applied Science, and Physics. This interdisciplinary program is centered on computation approaches to the study of biological and artificial information processing systems. A multidisciplinary curriculum offers training in four general areas: neurobiology; computer science and collective computation; physical computational devices; and mathematics and modeling. Students need to take courses in each of these areas in addition to an experimental laboratory course in neurobiology. The breadth of training is enhanced by close interactions among students and faculty from all parts of the program. A central focus is provided by weekly seminars, informal lunch talks, and a computer simulation laboratory open to students. Students are assigned to a research laboratory upon arrival, but have the option of rotating through several laboratories before choosing a thesis advisor. Research interests of the faculty include the collective properties and computational capacities of complex artificial and biological networks, analog VLSI devices, optical devices, and highly parallel digital computers. Neurobiological simulation approaches include modeling at the systems level (e.g., olfactory cortex, cerebellar cortex, and visual and auditory cortices) and at the cellular level (e.g., biophysical and developmental mechanisms). Computational approaches to artificial systems span a wide range, from studies of associative memory and analog networks for sensory processing to graphical image representation and the theory of computation. Interested students are encouraged to combine theoretical or modeling approaches with physiological or anatomical research on biological systems. Core faculty: Yaser Abu-Mostafa, John Allman, Alan Barr, James Bower, Rodney Goodman, John Hopfield, Bela Julesz, Christof Koch, Masakazu Konishi, Gilles Laurent, Henry Lester, Carver Mead, Jerome Pine, Edward Posner, Demitri Psaltis, David van Essen. Selection of ourses: CNS 124 : Pattern Recognition (two quarters) Covers classic results from pattern recognition and discusses in this context associative memories and related neural network models of computation. Given by D. Psaltis. CNS 174 : Computer Graphics Laboratory (three quarters) The art of making pictures by computer. Given by A. H. Barr. CNS 182 : Analog Integrated Circuit Design (three quarters) Device, circuit, and system techniques for designing large-scale CMOS analog systems. Given by C. A. Mead. CNS 184 : Analog Integrated Circuit Projects Laboratory (three quarters) Design projects in large-scale analog integrated systems. Given by C. A. Mead. CNS 185 : Collective Computation (one quarter) Neural network theory and applications. Given by J. J. Hopfield. CNS 186 : Vision: From Computational Theory to Neuronal Mechanisms (one quarter) Lecture and discussion course aimed at understanding visual information processing in both biological and artificial systems. Given by C. Koch and D. C. Van Essen. CNS 221 : Computational Neurobiology (one quarter) Lecture, discussion and laboratory aimed at understanding computational aspects of information processing within the nervous system. Given by J. Bower and C. Koch. CNS 256 : Methods of Multineural Recording (one quarter) Reading and discussion course. Topics included span a range of multineural recording techniques from multielectrode recording to positron emission tomography. Given by J. Pine. Student personal description ( H. H. Suarez, fourth year graduate student; hhs at aurel.caltech.edu): According to my experience, this program's emphasis really spans a wide range, but two areas stand out especially for me: modelling biological systems in a very detailed fashion and building artificial sensory-motor systems (analog VLSI - based systems) whose design is strongly influenced by knowledge of the corresponding biological system. The overall ambiance from a student's point of view is very good, due to the personal qualities of the faculty and the students. There is a fair amount of interaction among the researchers in the program, and on the average two or three talks a week on CNS-related topics, often from researchers outside Caltech. Thus there is little chance of getting bored ... From bogner at augean.ua.oz.au Wed Dec 12 18:26:58 1990 From: bogner at augean.ua.oz.au (bogner@augean.ua.oz.au) Date: Wed, 12 Dec 90 17:26:58 CST Subject: Postdoc needed Message-ID: <9012120657.1873@munnari.oz.au> I am needing a post-doc on neural net work as soon as possible. This is not an official advertisement, but I'd like to hear from anyone suitably experienced and available in Jan 1991 for a year. Work relates to some of: invariance, data fusion, preprocess selection. Robert E. Bogner Professor of Electrical Engineering University of Adelaide South Australia From shen at iro.umontreal.ca Wed Dec 12 18:17:19 1990 From: shen at iro.umontreal.ca (Yu Shen) Date: Wed, 12 Dec 90 18:17:19 EST Subject: reference list on parallel GA/SA Message-ID: <9012122317.AA12988@chukota.IRO.UMontreal.CA> Here is the list of reference on parallel genetic algorithm and parallel simulated annealing, compiled from the kind resposes from people on the mailing list and other whatever ways I ran into. I thank all the help again! I have done a report on the impact of parallelism on genetic algorithm and simulated annealing. Here are some of the major points: Assuming constant size of neighborhood in a spatially distributed population, the parallel genectic algorithm that selects mating partner locally can usually achieve linear speedup over the sequential version. But PGA has mainly shown advantage in improving the quality of search. For parallel paradigm increases the chance of crossing-over of gens, which is believed to be the most important mechanism of optimization. Even with linear speedup, PGA application is still very slow comparing with some conventional heuristics, for example, for the case of Traveling Salesman Problem. Parallel simulated annealing is not equivalent to parallelization of Metropolis' relaxation, where the mainstream of practices concentrate on. Metropolis' is basically a sequential approach. More significant speedup is usally achieved by alternatives away from it. It is possible to devise more parallel paradigm oriented simulation techniques, because the working mechanism of simulated annealing is Boltzman distribution, instead of the particular generating method--- Metropolis' relaxation. The report is being revised. It will be available to interested parties in January. The reference consists of two parts. The first is on genectic algorithm, the second on simulated annealing. Very godd bibliography on SA of D. Greening is refered to the original source to save space. By the way, I have profited a lot by asking question which seems simple to some BIG profs B-]. I earnestly want to make up the noise I made to them. I therefore suggest to the fellow pratictioner-to-be's let's try to make the noise as little as possible. For example, we can have the subject line as concise as possible, so non-interested people can delete it beofor seeing it. As a example, when we ask such "simple" question, we can put a X: in front of the subject line. X: may mean that it might annoy some people but it might not to some others. eg. X: Relationship between the cinema and the connectionist mailing list 8-) Yu Shen PhD Student Dept. d'Informatique et Recherche Operationnelle University de Montreal C.P. 6128 Succ. A. Montreal, Que. Canada H3C 3J7 (514) 342-7089 (H) shen.iro.umontreal.ca ---------------------PGA---------------------------------------------------- @TechReport{Goldberg90:Boltzman-Tournament, author = "David E. Goldberg", title = "A Note on Boltzmann Tournament Selection for Genetic Algorithms and Population-oriented Simulated Annealing", institution = "University of Alabama", year = "1990", OPTtype = "", OPTnumber = "90003", OPTaddress = "Tuscaloosa, AL 35487", OPTmonth = "May", OPTnote = "distribution across population" } @TechReport{ga:Goldberg90:messy, author = "David E, Goldberg", title = "An Investigation of Messy Genetic Algorithms", institution = "Dept. of Engineering Mechanics, Univ. of Alabama", year = "1990", OPTtype = "", OPTnumber = "90005", OPTaddress = "", OPTmonth = "May", OPTnote = "" } @Article{ga:Goldberg89:messy, author = "David E. Goldberg", title = "Messy Genetic Algorithms: Motivation, Analysis, and First Results", journal = "Complex Systems", year = "1989", OPTvolume = "", OPTnumber = "3", OPTpages = "493-530", OPTmonth = "", OPTnote = "" } ------------ @TechReport{ga:Goldberg90:selection, author = "David E. Goldberg", title = "A Comparative Analysis of Selection Schemes Used in Genetic Algorithms", institution = "Dept. of Engineering Mechanics, Univ. of Alabama", year = "1990", OPTtype = "", OPTnumber = "90007", OPTaddress = "", OPTmonth = "July", OPTnote = "" } ------------- From rupen at cvax.cs.uwm.edu Wed Dec 12 19:48:58 1990 From: rupen at cvax.cs.uwm.edu (Rupen Sheth) Date: Wed, 12 Dec 90 19:48:58 CDT Subject: Iris plat classification using backprop Message-ID: <9012130149.AA08037@cvax.cs.uwm.edu> I am trying to solve the Iris plant classification problem using standard backprop with a 4-4-2 (input-hidden-output) configuration. I plan on trying different configurations and varying some parameters to evaluate performance. What other configurations have other people used for this problem. Any results and comments are welcome. Thank you. Please send email to rupen at cvax.cs.uwm.edu OR rupen at gemed.ge.com From jose at learning.siemens.com Thu Dec 13 08:16:13 1990 From: jose at learning.siemens.com (Steve Hanson) Date: Thu, 13 Dec 90 08:16:13 EST Subject: Tabu Search, and other stuff Message-ID: <9012131316.AA04980@learning.siemens.com.siemens.com> Hey guys, CMU runs this thing out of the goodness of their little hearts. And as far as I know recieves no remuneration for machines, administration, or headaches that arise from the disparate personalities which are attracted to this medium, I think we should attempt to minimize whining and get on with it. Steve Hanson From shen at iro.umontreal.ca Thu Dec 13 10:23:50 1990 From: shen at iro.umontreal.ca (Yu Shen) Date: Thu, 13 Dec 90 10:23:50 EST Subject: Reference list on PGA more Message-ID: <9012131523.AA13992@chukota.IRO.UMontreal.CA> @Article{Liepins89, author = "G.E. Liepins and M.R. Hilliard", title = "GENETIC ALGORITHMS: FOUNDATIONS AND APPLICATIONS", journal = "Annals of Operations Research", year = "1989", OPTvolume = "21", OPTnumber = "", OPTpages = "31-58", OPTmonth = "", OPTnote = "see more reference there" } @InProceedings{Gorges-Schleuter89, author = "{M. G\"orges-Schleuter}", title = "Asparagos: An asynchronous parallel genetic optimization strategy", booktitle = "3rd Int. Conf. on Genetic Algorithms", year = "1989", OPTeditor = "H. Schaffer", OPTpages = "", OPTorganization = "", OPTpublisher = "Morgan-Kaufmann", OPTaddress = "", OPTmonth = "", OPTnote = "" } @InProceedings{Petty87, author = "C.B. Pettey, M.R. Leuze and J.H. Grefenstette", title = "A parallel genetic algorithm", booktitle = "Genetic Algorithms and Their Applications: Proc. Second Int. Conf. on Genetic Algorithms", year = "1987", OPTeditor = "Grefenstette", OPTpages = "155-161", OPTorganization = "", OPTpublisher = "", OPTaddress = "", OPTmonth = "", OPTnote = "" } @InProceedings{Robertson88, author = "G. G. Robertson", title = "Population size in a classifier system", booktitle = "Proc. Fifth Int. Conf. on Machine Learning ", year = "1988", OPTeditor = "", OPTpages = "142-152", OPTorganization = "", OPTpublisher = "", OPTaddress = "", OPTmonth = "", OPTnote = "" } @InProceedings{Robertson87, author = "G.G. Robertson", title = "Parallel implementation of genetic algorithms in a classifier system", booktitle = "Genetic Algorithms and Simulated Annealing", year = "1987", OPTeditor = "Lawrence Davis", OPTpages = "129-140", OPTorganization = "", OPTpublisher = "", OPTaddress = "", OPTmonth = "", OPTnote = "" } @Book{Davis87, author = "Lawrence Davis", title = "Genetic Algorithms and Simulated Annealing", publisher = "Pitman, London", year = "1987", OPTeditor = "", OPTvolume = "", OPTseries = "", OPTaddress = "", OPTedition = "", OPTmonth = "", OPTnote = "the first introduction to me, covers ga most" } @InProceedings{Sannier87, author = "A.V. Sannier II and E.D. Goodman", title = "Genetic learning procedure in distributed environments", booktitle = "Genetic Algorithms and Their Applications: Proc. Second Int. Conf. on Genetic Algorithms", year = "1987", OPTeditor = "Grefenstette", OPTpages = "162-169", OPTorganization = "", OPTpublisher = "", OPTaddress = "", OPTmonth = "", OPTnote = "" } @InProceedings{Jog87, author = "P.Jog and D.Van Gucht", title = "Parallerization of probabilistic sequential search algorithms", booktitle = "Genetic Algorithms and Their Applications: Proc. Second Int. Conf. on Genetic Algorithms", year = "1987", OPTeditor = "Grefenstette", OPTpages = "170-176", OPTorganization = "", OPTpublisher = "", OPTaddress = "", OPTmonth = "", OPTnote = "" } @InCollection{Wilson87:classifier, author = "Stewart W. Wilson", title = "Hierachical Credit Allocation in a Classifier System", booktitle = "Gentic Algorithms and Simulated Annealing", publisher = "Pitman, London", year = "1987", OPTeditor = "Lawrence Davis", OPTchapter = "8", OPTpages = "104-115", OPTaddress = "", OPTmonth = "", OPTnote = "" } From frederic at cs.unc.edu Thu Dec 13 10:37:58 1990 From: frederic at cs.unc.edu (Robin Fredericksen) Date: Thu, 13 Dec 90 10:37:58 -0500 Subject: No subject Message-ID: <9012131537.AA05285@surya.cs.unc.edu> >>I know from ...in networks that >>develop .....training. I personally have had problems >>when refering to this class of algorithms... >>The term for ...would like to propose, is Ontogenic Neural Networks. >Clever, but I just can't see this term catching on. What's wrong with >"constructive" and "destructive" (or maybe "additive" and "subtractive")? >People immediately know what you're talking about. > >I don't think it's a big problem that there isn't a single word for the >whole class. Usually you only want to refer to one kind or the other. >Only leaders of workshops and (I hope!) funding agencies have any need to >come up with one term that covers the whole spectrum of such approaches. > >-- Scott It's more than just clever. It is precise, and by that I mean: pre.cise \pri-'si-s\ aj [MF precis, fr. L praecisus, pp. of praecidere to cut off, fr]. prae- + caedere to cut - more at CONCISE 1: exactly or sharply defined or stated 2: minutely exact 3: strictly conforming to rule or convention 4: distinguished from every other : VERY {at just that ~ moment} - pre.cise.ly av During ontogeny, the CNS of a an organism experiences (uses) both cell death (destruction) and synaptogenesis (construction) to arrive at its final form. IMHO, separation of the two is artificial and incorrect. They go hand in hand in biological systems (although the periods during which they occur are not exactly coincident, but overlap) so why should they not go hand in hand in our models? The situation is not one of a 'spectrum of approaches', but one of duality. Ignoring one and studying the other provides only half of the story, and what good is half of a story? For a nice overview, try: AUTHOR: Purves, Dale. TITLE : Body and brain : a trophic theory of neural connections / IMPR : Cambridge, Mass. : Harvard University Press, 1988. I don't see why we should ignore terminology created/used in another field. This is an interdisciplinary area; if everyone reinvents the wheel then what use is collaboration? --Eric From tsejnowski at UCSD.EDU Thu Dec 13 13:54:27 1990 From: tsejnowski at UCSD.EDU (Terry Sejnowski) Date: Thu, 13 Dec 90 10:54:27 PST Subject: Negative feedback loops Message-ID: <9012131854.AA07310@sdbio2.UCSD.EDU> There is an inherent instability in the list that is caused by time delays that can be over 3 days for some sites. Negative feedback loops with delays are well known to be prone to wild oscillations. A posting that elicits strong responses will cause an impulse response that can last for a week because remote sites are not aware that the posting has already been through several cycles of responses from others. Before you make an obvious reply to a posting, please look at the date of the posting. If it was posted three or more days earlier the chances are that someone has already said what you are about to say. Terry ----- From Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU Thu Dec 13 16:47:03 1990 From: Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF1.SLISP.CS.CMU.EDU) Date: Thu, 13 Dec 90 16:47:03 EST Subject: No subject In-Reply-To: Your message of Thu, 13 Dec 90 10:37:58 -0500. <9012131537.AA05285@surya.cs.unc.edu> Message-ID: Sure, there are many interesting network architectures that add and subtract network structure at the same time. Most people at the post-NIPS workshop on this topic (including me) seemed to feel that these hybrid approaches were the most promising of all. The real problem I have with "Ontogenic" is that the term is so closely associated in most people's minds with biological development. Many people will assume that an "Ontogenic Neural Network" is a serious attempt to model the embryonic development of some real biological nervous system. That may happen some day soon, and we probably want to save the term "Ontogenic" for such applications, rather than co-opting it to refer to any old net that messes around with its own topology during learning. [Beware! Attempted humor follows:] I wonder if ontogenic neural nets would, in the course of learning, recapitulate the phylogeny of neural nets. You start with a simple perceptron -- two layers of cells -- which then begins growing a hidden layer. Unfortunately, the hidden layer is not functional. Some symbolic antibodies then attack the embryo and try to kill it off by leeching off all the nutrients, but a few isolated cells remain. The cells regroup, but very loosely. One part buds off, names itself "Art", and develops an elaborate, cryptic language of its own. The rest of the blob turns into a Hopfield net, heats up and cools down a few times, and finally develops the organs necessary for back-propagation. We don't know what happens after that because the back-propagation phase is so slow that it hasn't converged yet... -- Scott From singh at envy.cs.umass.edu Thu Dec 13 18:35:38 1990 From: singh at envy.cs.umass.edu (singh@envy.cs.umass.edu) Date: Thu, 13 Dec 90 18:35:38 EST Subject: tech report: benefits of gain In-Reply-To: "KRUSCHKE,JOHN,PSY"'s message of 7 Dec 90 15:48:00 EST <9012081955.AA22023@unix1.CS.UMASS.EDU> Message-ID: <9012132335.AA00627@gluttony.cs.umass.edu> From INS_ATGE%JHUVMS.BITNET at VMA.CC.CMU.EDU Thu Dec 13 21:44:00 1990 From: INS_ATGE%JHUVMS.BITNET at VMA.CC.CMU.EDU (INS_ATGE%JHUVMS.BITNET@VMA.CC.CMU.EDU) Date: Thu, 13 Dec 90 21:44 EST Subject: "constructive" vs. "compositional" learning Message-ID: Something to keep in mind when describing "constructive", "destructive", or "ontogenic" networks is the nature of learning. In traditional gradient-descent learning of fixed networks, the learning algorithm finds the minimum (or minima) of a fixed energy landscape. In these "constructive" or "destructive" networks, learning algorithms develop an energy landscape specifically designed to allow gradient-descent methods to best solve the problem (or at least that is what _should_ be happening). "Compositional Learning" (as used by J. Schmidhuber), is a method in which useful sub-goals are developed and utilized to solve a larger goal. (By stringing together these sub-goals). In methods such as Cascade-Correlation, new hidden units are added which serve to change the error landscape so as to best allow gradient-descent methods to find energy minima. But examining Cascade-Correlation in another light, we can say it is developing feature detectors which represent useful subgoals. Proper connection of these useful subgoals together allow us to reach our final goal (error minimization). -Thomas Edwards From jagota at cs.Buffalo.EDU Thu Dec 13 19:28:20 1990 From: jagota at cs.Buffalo.EDU (Arun Jagota) Date: Thu, 13 Dec 90 19:28:20 EST Subject: Paper Announcement Message-ID: <9012140028.AA23382@sybil.cs.Buffalo.EDU> *************** DO NOT FORWARD TO OTHER BBOARDS***************** [Note : Please do not reply with 'r' or 'R' to this message] The following paper, submitted to a special issue of IJPRAI, is now available : It describes a substantial extension of work presented at IJCNN-90, San Diego Degraded Printed Word Recognition with a Hopfield-style Network Arun Jagota (jagota at cs.buffalo.edu) Department of Computer Science State University Of New York At Buffalo ABSTRACT In this paper, the Hopfield-style network, a variant of the discrete Hopfield network, is applied to (degraded) machine printed word recognition. It is seen that the representation and dynamics properties of this network map very well to this problem. Words to be recognised are stored as content- addressable memories. Word images are first processed by a hardware OCR. The network is then used to postprocess the OCR decisions. It is shown (on postal word images) that for a small stored dictionary (~500 words), the network exact recall performance is quite good, for a large (~10,500 words) dictionary, it deteriorates dramatically, but the network still performs very well at "filtering the OCR output". The benefit of using the network for "filtering" is demonstrated by showing that a specific distance based search rule, on a dictionary of 10,500 words, gives much better word recognition performance (71% TOP choice, 84% TOP 2.6) on the network (filtered) output than on the raw OCR output. It is also shown, that for such "filtering", a special case of the network with two-valued weights performs almost as well as the general case, which verifies that the essential processing capabilities of the network are captured by the graph underlying it, the specific values of the +ve weights being relatively unimportant. This might also have implications for low prec- ision implementation. The best time efficiency is found when the dictionary of ~10,500 words is stored and the network is used to "filter" OCR output for 266 images. The training + filtering, together, take only 2 watch-timed minutes on a SUN Sparc Station. ------------------------------------------------------------------------ (Raw) FootNote: This problem seems cumbersome if viewed as one of supervised function learning (feed-forward NNs, Bayesian) due to large number (~10,500) of classes (=> large training sets/times). Conventional treatment is dictionary storage + search problem but drawback of sequential search is large search time during testing. The Hopfield-style network can be viewed as particular form of (unsupervised) distributed dictionary storage + search-by-recurrent-dynamics. Search is rapid and almost independent of dictionary size. The catch is that functional performance deteriorates rapidly with dictionary size. All is not lost, however, because partial performance (filtering) remains very good. [[Comments on above welcome but please mail to (jagota at cs.buffalo.edu) directly, not to CONNECTIONISTS]] The paper is available in compressed PostScript form by anonymous ftp unix> ftp cheops.cis.ohio-state.edu (or, ftp 128.146.8.62) Name: anonymous Password: neuron ftp> cd pub/neuroprose ftp> binary ftp> get jagota.wordrec.ps.Z ftp> quit unix> uncompress jagota.wordrec.ps.Z unix> lpr jagota.wordrec.ps ------------------------------------------------------------------------ Previous Postscript incompatibility problems have, by initial assessment, been corrected. Nevertheless, the paper is also available by e-mail (LaTeX sources) or surface mail (in that prefered order). Arun Jagota Dept Of Computer Science jagota at cs.buffalo.edu 226 Bell Hall, State University Of New York At Buffalo, NY - 14260 *************** DO NOT FORWARD TO OTHER BBOARDS***************** From Dave.Touretzky at DST.BOLTZ.CS.CMU.EDU Fri Dec 14 01:18:23 1990 From: Dave.Touretzky at DST.BOLTZ.CS.CMU.EDU (Dave.Touretzky@DST.BOLTZ.CS.CMU.EDU) Date: Fri, 14 Dec 90 01:18:23 EST Subject: for NIPS authors Message-ID: I apologize for bothering the entire list with this, but it's the only way to quickly reach the 130+ NIPS authors. Morgan Kaufmann has issued some corrections to the LaTeX formatting macros for NIPS papers; a particularly critical change is the value of \textheight parameter. If you FTP'ed the macros from B.GP.CS.CMU.EDU before December 13, please FTP them again. If you retrieved the macros on or after December 13, you already have the latest version. If you haven't received your author kit by Monday, call Sharon Montooth at Morgan Kaufmann; her number is 415-578-9911. Reminder: your final camera-ready copy is due at the publisher by January 18. The volume will be out in April. -- Dave Touretzky From LAUTRUP at nbivax.nbi.dk Fri Dec 14 03:44:00 1990 From: LAUTRUP at nbivax.nbi.dk (Benny Lautrup) Date: Fri, 14 Dec 90 09:44 +0100 (NBI, Copenhagen) Subject: IJNS issue number 4 Message-ID: Begin Message: ----------------------------------------------------------------------- INTERNATIONAL JOURNAL OF NEURAL SYSTEMS The International Journal of Neural Systems is a quarterly journal which covers information processing in natural and artificial neural systems. It publishes original contributions on all aspects of this broad subject which involves physics, biology, psychology, computer science and engineering. Contributions include research papers, reviews and short communications. The journal presents a fresh undogmatic attitude towards this multidisciplinary field with the aim to be a forum for novel ideas and improved understanding of collective and cooperative phenomena with computational capabilities. ISSN: 0129-0657 (IJNS) ---------------------------------- Contents of issue number 4 (1990): 1. A. M. Gutman: Bistability of Dendrites. 2. J. J. Atick and A. N. Redlich: Prediction Ganglion and Simple Cell Receptive Field Organisations. 3. H. H. Thodberg: Improving Generalisation of Neural Networks through Pruning. 4. O. Hendin, D. Horn and M. Usher: Chaotic Behaviour of a Neural Network with Dynamical Thresholds. 5. C. Myers: Learning with Delayed Reinforcement through Attention-Driven Buffering. 6. R. Erichson and W. K. Theumann: Mixture States and Storage with correlated Patterns in Hopfield's Model. 7. H. Shouval, I. Shariv, T. Grossman, A. A. Friesem, E. Domany: An all-optical Hopfield Network: Theory and Experiment. 8. Yves Chauvin: Gradient Descent to Global Minima in a n-dimensional Landscape. ---------------------------------- Editorial board: B. Lautrup (Niels Bohr Institute, Denmark) (Editor-in-charge) S. Brunak (Technical Univ. of Denmark) (Assistant Editor-in-Charge) D. Stork (Stanford) (Book review editor) Associate editors: B. Baird (Berkeley) D. Ballard (University of Rochester) E. Baum (NEC Research Institute) S. Bjornsson (University of Iceland) J. M. Bower (CalTech) S. S. Chen (University of North Carolina) R. Eckmiller (University of Dusseldorf) J. L. Elman (University of California, San Diego) M. V. Feigelman (Landau Institute for Theoretical Physics) F. Fogelman-Soulie (Paris) K. Fukushima (Osaka University) A. Gjedde (Montreal Neurological Institute) S. Grillner (Nobel Institute for Neurophysiology, Stockholm) T. Gulliksen (University of Oslo) D. Hammerstrom (Oregon Graduate Institute) J. Hounsgaard (University of Copenhagen) B. A. Huberman (XEROX PARC) L. B. Ioffe (Landau Institute for Theoretical Physics) P. I. M. Johannesma (Katholieke Univ. Nijmegen) M. Jordan (MIT) G. Josin (Neural Systems Inc.) I. Kanter (Princeton University) J. H. Kaas (Vanderbilt University) A. Lansner (Royal Institute of Technology, Stockholm) A. Lapedes (Los Alamos) B. McWhinney (Carnegie-Mellon University) M. Mezard (Ecole Normale Superieure, Paris) J. Moody (Yale, USA) A. F. Murray (University of Edinburgh) J. P. Nadal (Ecole Normale Superieure, Paris) E. Oja (Lappeenranta University of Technology, Finland) N. Parga (Centro Atomico Bariloche, Argentina) S. Patarnello (IBM ECSEC, Italy) P. Peretto (Centre d'Etudes Nucleaires de Grenoble) C. Peterson (University of Lund) K. Plunkett (University of Aarhus) S. A. Solla (AT&T Bell Labs) M. A. Virasoro (University of Rome) D. J. Wallace (University of Edinburgh) D. Zipser (University of California, San Diego) ---------------------------------- CALL FOR PAPERS Original contributions consistent with the scope of the journal are welcome. Complete instructions as well as sample copies and subscription information are available from The Editorial Secretariat, IJNS World Scientific Publishing Co. Pte. Ltd. 73, Lynton Mead, Totteridge London N20 8DH ENGLAND Telephone: (44)81-446-2461 or World Scientific Publishing Co. Inc. 687 Hardwell St. Teaneck New Jersey 07666 USA Telephone: (1)201-837-8858 or World Scientific Publishing Co. Pte. Ltd. Farrer Road, P. O. Box 128 SINGAPORE 9128 Telephone (65)382-5663 ----------------------------------------------------------------------- End Message From nelsonde%avlab.dnet at wrdc.af.mil Fri Dec 14 08:30:19 1990 From: nelsonde%avlab.dnet at wrdc.af.mil (nelsonde%avlab.dnet@wrdc.af.mil) Date: Fri, 14 Dec 90 08:30:19 EST Subject: Constructive/Destructive Algorithms (Ontogenic Networks) Message-ID: <9012141330.AA07533@wrdc.af.mil> I N T E R O F F I C E M E M O R A N D U M Date: 14-Dec-1990 08:47am EST From: Dale E. Nelson NELSONDE Dept: AAAT-1 Tel No: 57646 TO: Remote Addressee ( _LABDDN::"CONNECTIONISTS at CS.CMU.EDU" ) Subject: Constructive/Destructive Algorithms (Ontogenic Networks) > Clever, but I just can't see this term catching on. What's wrong with > "constructive" and "destructive" (or maybe "additive" and "subtractive")? > People immediately know what you're talking about. > I don't think it's a big problem that there isn't a single word for the > whole class. Usually you only want to refer to one kind or the other. If this is a valid argument, then why in the medical community do they call a class of diseases Cancer? Why not just refer to it as rapid, uncontrolled cell division? Why do we call them "neural networks"? Why not just say "inputs that are multiplied by weights, summed up, put through a squashing function and the result passed as input to another summer/ squasher" ???? It is because the vocabulary, as agreed to by researchers, makes the interchange of ideas easier. I believe that we need to develop the terminology and vocabulary for our research area to facilitate literature searches, and just free discussion. I have a deaf man that works for me. Our main problem is that there is *no* sign language vocabulary associated with neural networks. We have to develop it in order to effectively exchange ideas. I am open to any other suggestions. --Dale From gluck%psych at Forsythe.Stanford.EDU Fri Dec 14 12:02:01 1990 From: gluck%psych at Forsythe.Stanford.EDU (Mark Gluck) Date: Fri, 14 Dec 90 09:02:01 PST Subject: Postdoc: Cognitive Science / Neural Modeling Message-ID: <9012141702.AA19121@psych> Postdoctoral Positions in: -------------------------- COGNITIVE & NEURAL BASES OF LEARNING at Rutgers University Center for Molecular & Behavioral Neuroscience 195 University Avenue Newark, NJ 07102 Postdoctoral positions are available for recent Ph.D's in all areas of Cognitive Science (e.g., Neuroscience, Psychology, Computer Science) interested in pursuing research in the following areas of learning theory: 1. COGNITIVE SCIENCE/ADAPTIVE "CONNECTIONIST" NETWORKS: Experimental and theoretical (computational) studies of human learning and memory. 2. COMPUTATIONAL NEUROSCIENCE / COGNITIVE NEUROSCIENCE: Models of the neural bases of learning in animals and humans. Candidates with any (or all) of the following skills are particular encouraged to apply: (1) familiarity with neural network algorithms and models, (2) strong computational/analytic skills, and (3) experience with experimental methods, experimental design, and data analysis in cognitive psychology. ---------------------------------------------------------------------------- Other Information: FACILITIES: The Center is a new state-funded research center for the integrated studies of cognitive, behavioral, and molecular neuroscience. The Center has good computational resources and experimental laboratories for behavioral and neural studies. LOCATION: The Center is located in Newark, NJ, approximately 20 minutes outside of Manhattan, New York (with easy train and subway access to midtown and downtown NYC) and close to rural New Jersey countryside Numerous other research centers in the cognitive and neural sciences are located nearby including: Cognitive Science Center, Rutgers/New Brunswick; Centers for Cognitive & Neural Science, New York University; Cognitive Science Center, Princeton Univ.; Columbia Univ. & Medical School; Siemens Corporate Research, Princeton, NJ; NEC Research Labs, Princeton, NJ; AT&T Labs; Bellcore; IBM T. J. Watson Research Labs. CURRENT FACULTY: E. Abercrombie, G. Buzsaki, I. Creese, M. Gluck, H. Poizner, R. Siegel, P. Tallal, J. Tepper. Six additional faculty will be hired. The Center has a total of ten state-funded postdoctoral positions and will direct, in collaboration with the Institute for Animal Behavior, a graduate program in Behavioral and Neural Sciences. ---------------------------------------------------------------------------- For more information on learning research at the CMBN/Rutgers or to apply for these post-doctoral positions, please send a cover letter with a statement of your research interests, a CV, copies of relevant preprints, and the the names & phone numbers of references to: Dr. Mark A. Gluck Phone: (415) 725-2434 Dept. of Psychology <-[Current address to 4/91] FAX: (415) 725-5699 Jordan Hall; Bldg. 420 Stanford University email: gluck at psych.stanford.edu Stanford, CA 94305-2130 From gmk%idacrd at Princeton.EDU Fri Dec 14 12:03:54 1990 From: gmk%idacrd at Princeton.EDU (Gary M. Kuhn) Date: Fri, 14 Dec 90 12:03:54 EST Subject: 1st IEEE-SP Workshop on NN's for SP Message-ID: <9012141712.AA19624@Princeton.EDU> First IEEE-SP Workshop on Neural Networks for Signal Processing Sponsored by the IEEE Signal Processing Society in cooperation with the IEEE Neural Networks Council September 29 - October 2, 1991 Nassau Inn, Princeton, New Jersey, USA Call for Papers The first Workshop on Neural Networks for Signal Processing, sponsored by the IEEE Signal Processing Society, will be held in the fall of 1991 in Princeton, New Jersey. The beautiful Princeton area is easily accessible by train, bus or car from airports in and around New York city. Papers are solicited for technical sessions on the following topics: + Application-driven Neural Models + Neural Architecture for Signal Processing + System Identification & Spectral Estimation by Neural Networks + Neural Networks for Image Processing & Pattern Recognition + Applications of Neural Networks to Speech Processing + Nonlinear Signal and Pattern Learning Algorithms Prospective authors are invited to submit 4 copies of extended summaries of no more than 4 pages to Candace Kamm for review (address below). The top of the first page of the summary should include a title, authors' names, affiliations, addresses and telephone numbers. Photo-ready full papers of accepted proposals will be published in book form and distributed at the workshop. Due to conference facility constraints, attendance will be limited with priority given to those who submit written technical contributions. For more information, please contact Gary Kuhn, Publicity Chair, at (609) 924-4600. Schedule Submission of extended summary April 1, 1991 Notification of acceptance May 15, 1991 Submission of photo-ready paper July 1, 1991 Advanced registration, before August 31, 1991 Workshop Committee General Chair B.H. Juang S.Y. Kung Rm. 2D-534 Dept. of EE AT&T Bell Labs Princeton Univ. Murray Hill, NJ 07974 Princeton, NJ 08540 Local Arrangements John Vlontzos Siemens Corp. Research Princeton, NJ 08540 Proceedings Candace Kamm Box 1910 Bellcore 445 South St., Rm.2E-256 Morristown, NJ 07960-1910 Publicity Gary Kuhn Center for Communications Research-IDA Thanet Road Princeton, NJ 08540 Finance/Registration Bastiaan Kleijn Rm 2D-554 AT&T Bell Labs 600 Mountain Ave. Murray Hill, NJ 07974 Program Committee Rama Chellappa Lee Giles John Moody Bradley Dickinson Esther Levin Erkki Oja Tariq Durrani R. Lippmann W. Przytula F. Fallside John Makhoul Y. Tohkura K. Fukushima Y. Matsuyama C.J. Wellekens From jfeldman at ICSI.Berkeley.EDU Fri Dec 14 12:33:02 1990 From: jfeldman at ICSI.Berkeley.EDU (Jerry Feldman) Date: Fri, 14 Dec 90 09:33:02 PST Subject: Scientific American Message-ID: <9012141733.AA02407@icsib2.berkeley.edu.Berkeley.EDU> I have been asked by Scientific American magazine to write an article on connectionism and artifical intelligence. My current approach to the project can be seen from the working subtitle: "Often viewed as competing, these two approaches to understanding intelligent behavior can be combined to yield scientific and practical advances". I am looking for suggestions, particularly on success stories involving connectionist systems. For this purpose we need devices that are actually in daily use, scientific results that have had a major impact, etc. Things that are merely promising or have just provoked controversy aren't nearly as effective. Of course, I welcome any other suggestions. Please use e-mail unless you really want to address the whole group. Jerry F. jfeldman at icsi.berkeley.edu From frederic at cs.unc.edu Fri Dec 14 13:38:50 1990 From: frederic at cs.unc.edu (Robin Fredericksen) Date: Fri, 14 Dec 90 13:38:50 -0500 Subject: No subject Message-ID: <9012141838.AA07837@sargent.cs.unc.edu> I apologize if I came across too sharply. It's true that there needs to be some well thought out terminological classification. My response in part is because I *AM* interested in the ontogeny of biological systems and what it can teach us. My personal research interests includes how the biological system does what it does and how we can use that to our advantage. It is true that there are many researchers (as well as engineers) who are looking at neural networks as a new and useful tool and could care less how that tool relates to the systems that originally inspired the technique (or at least invented it first :-) ). We must find some way to differentiate between models of (CNS) ontogeny for the study of (CNS) ontogeny, and the use of ontogenically derived methods for finding the correct network (CNS) structure to solve some specific computational problem. Yes, the word ontogeny is too general: on.tog.e.ny \a:n-'ta:j-*-ne-\ n [ISV] : the development or course of development of an individual organism (don't you just love the on-line Websters?) We need to find a term that refers to *CNS* development. *But*, I think that use of the words 'destructive' and 'constructive' can be just as misleading. If I understand correctly (I was not at NIPS), 'destructive' is used to describe a network that begins with surfeit of nodes and removes nodes during learning. 'Constructive' is then used to describe networks that add nodes during learning. A more accurate phrase might be 'structurally self modifying' (or must we find a single word? should we then say it in German? :-) ) Are there such strongly qualitative differences in networks resulting from the use of either strictly 'destructive' or 'constructive' rules that we need separate terminology? We have available lots of verbs: alter, modify, transform, mutate, change, vary (ad infinitem, almost; I don't have an on-line thesaurus). tmb at ai.mit.edu says: > ... Ontogeny is a very specializec (sic) >process with a number of strict biological constraints operating; it is >far from clear that processes that operate during ontogeny are similar >or analogous to unit creation/destruction in an artificial neural >network that learns. > >The thought of equating a mathematical and engineering technique with >this biological process makes me cringe. The judgement is still out to >what degree artificial "neural networks" are related to the real thing, >so lets not aggravate the problem by introducing more biological >terminology into the field. Again, I agree that ontogeny is not the correct word, and I apologize for defending it without suggesting a better one. But is it not true that we put lots of constraints on neural networks in order to coerce them into 'learning' the correct answer? It may not be clear that there is any similarity between the learning rules that use creation/destruction of units in a network and 'rules' in the process of CNS development (not ontogeny, which is too general a term), but I would also object to having an artificial (ideological) wall put up between the concepts. In the end, the real problem in selecting terminology lies in whether you view a nn as an engineering tool that should be designed in a void, completely derived without hints or clues from nature, or whether you view it as a tool modeled after organisms (many of which have been very successful in terms of evolution) from which we might learn. My personal bias is the latter. INS_ATGE%jhuvms.hcf.jhu.edu at BITNET.CC.CMU.EDU writes: >The other problem is that there might be learning in brain which involves >recruitment of neurons or exclusion of neurons from the network performing a >cognitive function (i.e. "software routing") which occurs much later than >ontogeny (I do not know of results which show this, but constructive/ >destructive learning is so useful that it would seem useful if it was >performed by "soft" changes in neural nets as opposed to actual >synaptogenesis/cell death)(Do you know of any?). >Anyway, I understand your point about reinventing wheels, but it seems >that "ontogenic nets" seems too limiting to apply to "architecturally >changeable" learning methods, with the exception of physiological >learning which occurs during ontogeny. First, Dale Purves feels that synaptogenesis continues throughout the organisms life span (and thus CNS ontogeny). Cell death occurs over a much more limited space of time that depends on the complexity (size) of the animal but is generally restricted to pre and a small postnatal period. (Small in comparison with the animal's expected lifespan.) By software routing, do you mean adding connections to new units? That is just synaptogenesis and its counterpart (synaptic loss). Ok, so now I have wandered far away from defending the use of 'ontogenic' and delved into my true reaction. I was actually responding to what appeared to be sarcasm (sorry Scott) but was intended to be humor. My real objection is to the building of walls within what should be an interdisciplinary field through the use of imprecise or restricted terminology. The problem then is deciding what is precise, and that can come down to your personal view of what nn research is for: building tools, understanding biological systems, or both at the same time. So, any good ideas on terminology that fits the bill? [More attempted humor...] I wonder if constructive and destructive neural network learning rules will, in the course of research, recapitulate the phylogeny of biological neural networks? We could separate out (on different continents, of course) groups of researchers trying to develop a nn to solve the same class of problems, say vision (email not allowed, of course). Then after a suitable amount of time, perhaps one group will have evolved rules for building crustacean visual systems, another group will have evolved rules for the building of arthropod visual systems, and yet another will be able to build mammalian visual systems? But, will it take us hundreds of millions of years? :-) Eric Fredericksen From chuck at cs.utk.edu Fri Dec 14 22:23:35 1990 From: chuck at cs.utk.edu (chuck@cs.utk.edu) Date: Fri, 14 Dec 90 22:23:35 -0500 Subject: Negative feedback loops Message-ID: <9012150323.AA14803@alphard.cs.utk.edu> Agreed, But since I call long distance to hear all this amusing drivel some one migh bring it to Dave's attention. He uses us enough! Chuck Joyce chuck at cs.utk.edu From fellous%pipiens.usc.edu at usc.edu Fri Dec 14 20:31:13 1990 From: fellous%pipiens.usc.edu at usc.edu (Jean-Marc Fellous) Date: Fri, 14 Dec 90 17:31:13 PST Subject: No subject Message-ID: <9012150131.AA00490@pipiens.usc.edu> Subject: USC Workshop on Emotions (please forward on relevant mailing lists) __________________________________________________________________________ / U.S.C \ | | | C N E Student Workshop on Emotions | | | | CALL FOR PAPERS | | ***************** | \__________________________________________________________________________/ The Center For Neural Engineering of the university of Southern California invites all students interested in Emotions to submit a paper to be eventually presented during a one-day Workshop (of a date t.b.a. at the End of February 1991). The Workshop is opened to Graduate students (MA,MS,PhD) and College Seniors irrespective to their major (faculty will only be considered for publication), having pursued (or pursuing) research activities on such aspects of Emotions as: - The nature of Emotion - The physiology of Emotion - The perception of Emotions - The relations between Emotion and Cognition - Developemental aspects of Emotion - Artificial Intelligence models of Emotions - Neural network models of Emotions - Philosophical issues of Emotion and reductionism - ... Applicants should send a 2 page summary of the proposed paper and a letter of motivation in which they state their status, major, interests, name, address and telephone number (for reply). Materials should be submitted by January 31st to: Jean-Marc Fellous Center for Neural Engineering University of Southern California Los Angeles CA 90089-2520 Telephone: (213) 740-3506 email: fellous at rana.usc.edu ps: Travel expenses will not be covered by the CNE, but lunch will be provided. pps: Authors of the chosen papers will receive a copy of the presented papers (by mail if they could not attend the Workshop). ***************************************************************************** ppps: Please forward to relevant departments, mailing lists ... ***************************************************************************** .. From markt at umd5.umd.EDU Sat Dec 15 13:02:47 1990 From: markt at umd5.umd.EDU (Mark Turner) Date: Sat, 15 Dec 90 13:02:47 EST Subject: neural image? Message-ID: <9012151802.AA26260@umd5.UMD.EDU> I am seeking a suitable image suggesting neuronal group patterns and neural connectivity to be used as the background of a book cover. The book, titled POETIC THOUGHT: THE STUDY OF ENGLISH IN THE AGE OF COGNITIVE SCIENCE, will be published by Princeton University Press in the fall of 1991. POETIC THOUGHT is, to an extent, neurally-inspired in its themes and approaches; it focuses on conceptual connections and neural connections. I am not having any luck in this search, and would be indebted for any help. I need not only a high-quality image to be used in reproduction but also permission to use it, with the appropriate credit given of course. Mark Turner markt at umd5.umd.edu From pigmalio at batman.fi.upm.es Thu Dec 13 14:20:00 1990 From: pigmalio at batman.fi.upm.es (pigmalio@batman.fi.upm.es) Date: 13 Dec 90 20:20 +0100 Subject: No subject Message-ID: <9012131920.AA11890@batman.fi.upm.es> Subject: Request information about C compilers. We should be very pleased if you could send us information about transputer boards and intelligent C-compilers that paralelize automatically and are able to use these boards. Thanks a lot. E-mail : pigmalio at batman.fi.upm.es From worth at park.bu.EDU Sat Dec 15 13:59:52 1990 From: worth at park.bu.EDU (Andrew J. Worth) Date: Sat, 15 Dec 90 13:59:52 -0500 Subject: Connectionism vs AI Message-ID: <9012151859.AA15760@park.bu.edu> Jerry Feldman's posting ("Scientific American") foments a long standing dispute within my mind; are there fundamental differences between connectionism and AI that make them incompatible in an ideal sense? Examining each as an approach to the same goal, if one of connectionism's fundamental tenets is to mimic biological computation, and AI, on the other hand, holds sacred the extracting of the essence of "intelligence" while ignoring implementation details, (i.e. a bottom up vs. top down dichotomy) then is it not a bastardization of both to combine them? If each approach has fundamentally different and opposing assumptions, then wouldn't one or both of them have to be weakened in their combination? Andrew J. Worth worth at park.bu.edu Cognitive & Neural Systems Prog. (617) 353-6741 Boston University (617) 353-7857 (CAS Office) 111 Cummington St. Room 244 (617) 353-5235/6742 (CNS Grad Offices) Boston, MA 02215 From frederic at cs.unc.edu Sun Dec 16 13:16:33 1990 From: frederic at cs.unc.edu (Robin Fredericksen) Date: Sun, 16 Dec 90 13:16:33 -0500 Subject: apology Message-ID: <9012161816.AA03295@pooh.cs.unc.edu> I would like to apologize for the use of a quote in my previous email posting. The text from tmb at ai.mit.edu was from a personal communication, and I did not realize that it was such. So, how do I tell that a piece of email is from the connectionists mailing group? Will it have @SEF1.SLISP.CS.CMU.EDU as part of the address? Eric Fredericksen From mrj at cs.su.oz.au Sun Dec 16 06:29:54 1990 From: mrj at cs.su.oz.au (Mark James) Date: Sun, 16 Dec 90 22:29:54 +1100 Subject: Synchronization of Cortical Oscillations Message-ID: <14001.661346994@mango.cs.su.oz> I would be interested to hear from anyone who attended the NIPS workshop on cortical oscillations if anything was concluded regarding the mechanism for long distance synchronization of the oscillations. That is, is the excitation direct, via a change in effective synaptic efficacy (e.g. Eckhorn and others) or via disinhibition mediated by cells such as the spiny double bouquet cell. Thank you, Mark James | EMAIL : mrj at cs.su.oz.au | From INS_ATGE%JHUVMS.BITNET at VMA.CC.CMU.EDU Sun Dec 16 19:27:00 1990 From: INS_ATGE%JHUVMS.BITNET at VMA.CC.CMU.EDU (INS_ATGE%JHUVMS.BITNET@VMA.CC.CMU.EDU) Date: Sun, 16 Dec 90 19:27 EST Subject: Sci Am Article Message-ID: A. Worth: Jerry Feldman's posting ("Scientific American") foments a long standing dispute within my mind; are there fundamental differences between connectionism and AI that make them incompatible in an ideal sense? Examining each as an approach to the same goal, if one of connectionism's fundamental tenets is to mimic biological computation, and AI, on the other hand, holds sacred the extracting of the essence of "intelligence" while ignoring implementation details, (i.e. a bottom up vs. top down dichotomy) then is it not a bastardization of both to combine them? If each approach has fundamentally different and opposing assumptions, then wouldn't one or both of them have to be weakened in their combination? I see neural networks as a sub-field of artificial intelligence. Both are trying to develop "intelligent" artifacts. There is of course a dichotomy between symbolic AI and neural net AI. Symbolic AI has proven itself in many tasks such as symbolic calculus, theorem provers, expert systems, and other machine learning tasks. The difference between symbolic AI and neural AI is more one of computational substrate (although researchers may have artificially distanced neural nets from symbolic AI in the past). Alot of neural networks for the last few years has been applying a hill-climbing heuristic (well known to the symbolic AI community) to our nets. They learn, but not well. There is still a great deal of symbolic AI machine learning theory which could be used to set up really interesting neural networks, but there are difficulties in translating between a symbolic computational substrate and the neural network substrate. The constructive/destructive (or "ontogenic") networks which are comming down the line, such as Cascade-Correlation, are showing that hill-climbing in a fixed energy landscape is not the only way to do learning. There is also "compositional" learning (someone asked for the Schmidhuber ref. a while back...it's J. Schmidhuber. Towards compositional learning with dynamic neural networks. Report FKI-129-90, Technische Universitat Munchen, April 1990.) utilizing combining sub-goal networks together to achieve larger goals. Anyway, I think the lesson is that no matter how much connectionist researchers think their networks are capable of better inductive learning than symbolic AI systems, in order to do allow for deductive learning we are going to have to couch alot of existing symbolic AI heuristics and machine learning paradigms in a network architecture. -Thomas Edwards From Scott.Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU Sun Dec 16 16:00:36 1990 From: Scott.Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF-PMAX.SLISP.CS.CMU.EDU) Date: Sun, 16 Dec 90 16:00:36 EST Subject: apology In-Reply-To: Your message of Sun, 16 Dec 90 13:16:33 -0500. <9012161816.AA03295@pooh.cs.unc.edu> Message-ID: So, how do I tell that a piece of email is from the connectionists mailing group? Will it have @SEF1.SLISP.CS.CMU.EDU as part of the address? No, but mail that was sent to "connectionists" will generally have "connectionists at cs.cmu.edu" in either the "To:" or the "Cc:" field. If you don't see that, assume that the mail was sent just to you. Of course, mail-reading programs vary a lot, and some may hide some of this information. -- Scott From geb at dsl.pitt.edu Sun Dec 16 15:58:57 1990 From: geb at dsl.pitt.edu (Gordon E. Banks) Date: Sun, 16 Dec 90 15:58:57 -0500 Subject: Connectionism vs AI Message-ID: <9012162058.AA08797@cadre.dsl.pitt.edu> Traditional AI, rather than ignoring the details, started out by studying human behavior and cognition (see Simon & Newell, for example) in the realm of problem solving. It has always had a strong empirical element, in my opinion. Thus connectionism isn't alien in methodology from the original spirit of the pursuit of intelligent behavior. From epreston at aisun2.ai.uga.edu Sun Dec 16 14:41:02 1990 From: epreston at aisun2.ai.uga.edu (Elizabeth Preston) Date: Sun, 16 Dec 90 14:41:02 EST Subject: Connectionism vs AI In-Reply-To: "Andrew J. Worth"'s message of Sat, 15 Dec 90 13:59:52 -0500 <9012151859.AA15760@park.bu.edu> Message-ID: <9012161941.AA03794@aisun2.ai.uga.edu> Date: Sat, 15 Dec 90 13:59:52 -0500 From: "Andrew J. Worth" Jerry Feldman's posting ("Scientific American") foments a long standing dispute within my mind; are there fundamental differences between connectionism and AI that make them incompatible in an ideal sense? I'm afraid this dispute is not all in your mind; a great deal of ink and hot air has been expended on it in various public forums over the last few years. Perhaps the most familiar version of this dispute is the proposal by some that connectionism represents a paradigm shift (in the Kuhnian sense) in our understanding of cognition. This would make connectionism and AI not merely incompatible, but incommensurable; e.g., it would follow that connectionism and AI are in some sense not even working on the same problem(s), and that connectionists and AI-ists are in some sense not even understanding each other when they talk. Pretty radical stuff. Of course there are those who respond to this talk of a paradigm shift with incredulous stares and phrases on the order of "oh, pooh" and "nonsense". People on this end of the spectrum have been known to argue that connectionism is merely a way of implementing AI; in which case, far from being incompatible, they are basically the same thing. Although I think there are a lot of interesting and helpful things to be said about the similarities and differences between AI and connectionism, I doubt very much whether the question of FUNDAMENTAL compatibility/incompatibility is going to be settled on the conceptual level anytime soon. I say this partly because I don't think it has ever really been settled for the relationship between empiricism and rationalism in philosophy, or for the relationship between behaviorism and cognitivism in psychology, and these divisions in philosophy and psychology are pretty clearly the historical antecedents of the division in computational circles between connectionism and AI. And partly I say this because I don't see that the question is answerable at this stage in the development and theoretical understanding of the two fields. In fact, I'm not sure it's even askable. If each approach has fundamentally different and opposing assumptions, then wouldn't one or both of them have to be weakened in their combination? As a philosopher with a great deal of experience in armchair theorizing, I must say I wouldn't touch this one with a stick. Why don't you try it out and see what happens? Beth Preston From jlm+ at ANDREW.CMU.EDU Mon Dec 17 10:21:37 1990 From: jlm+ at ANDREW.CMU.EDU (James L. McClelland) Date: Mon, 17 Dec 90 10:21:37 -0500 (EST) Subject: Connectionism vs AI In-Reply-To: <9012151859.AA15760@park.bu.edu> References: <9012151859.AA15760@park.bu.edu> Message-ID: <0bPC_1m00jWD83q14n@andrew.cmu.edu> Regarding Worth's query about a possible fundamental opposition between connectionist and other approaches to AI: I do not think there need be such an opposition. That is, I think one can be a connectionist without imagining that any such opposition exists. Connectionist models are used for a variety of different purposes and with a variety of different goals in mind: Here are three: 1) To find better methods for solving AI problems, particularly those that have proven difficult to solve using conventional AI approaches. 2) To model actual mechanisms of neural computation. There's lots of data on such things as the stimulus conditions under which particular neurons will fire, but there is little understanding of the circuitry that leads to the patterns of firing that are seen or the role the neurons play in overall system function. Connectionist models can help in the exploration of these questions. 3) To explore mechanisms of human information processing. Here the idea is that there is a set of putative principles of human information processing that are more easily captured in connectionist models than in other formalisms. The effort to determine whether these principles are the right ones or not requires the use of models, since it is difficult to assess the adequacy of sets of principles without formalization, leading to analysis and/or simulation. There are others. The point is that the models can be viewed as tools for exploring questions. There need be no such religion as 'connectionism'; all it takes to be a connectionist is to find connectionist models useful. -- Jay McClelland From block at psyche.mit.edu Mon Dec 17 11:42:04 1990 From: block at psyche.mit.edu (Ned Block) Date: Mon, 17 Dec 90 11:42:04 EST Subject: Connectionism vs AI Message-ID: <9012171642.AA04247@psyche.mit.edu> Worth: Connectionism and AI are incompatible because connectionism is biological and AI ignores implementation. False, false, false. Connectionism IS a tyhpe of AI, albeit biologically inspired AI. Connectionist networks could easily be made more brain- like, eg, by not allowing weights to change between positive and negative. But this is not a popular idea, and ONLY becasue it would make the networks much less useful. Ned Block From bates at crl.ucsd.edu Mon Dec 17 12:57:51 1990 From: bates at crl.ucsd.edu (Elizabeth Bates) Date: Mon, 17 Dec 90 09:57:51 PST Subject: ontogenesis and synaptogenesis Message-ID: <9012171757.AA25001@crl.ucsd.edu> Just a small empirical correction to the discussion on ontogeny in neural nets. Synaptogenesis does NOT continue across the (human/primate) lifespan, at least not on any kind of a large or interesting scale. Research by Rakic, Huttenlocher and others suggests that there is a huge burst in synaptogenesis between (roughly) human postnatal months 6 - 24. There is some controversy about whether this "burst" takes place across the whole brain at once (the Rakic position) or whether different regions "burst" at different times (the Huttenlocher position), but it is fairly clear from research on both sides that the "burst" is over by the time the child is 2 years old. And of course, as already noted on the net, cell proliferation and migration is over with well before that, at least a month or so prior to birth (with the exception of a couple of areas like the olfactory bulb). The bottom line for "neurally inspired" connectionist models seems to be that the block of marble is delivered to the studio for carving by age 2. This of course does not exclude small-scale, very local changes that do occur across the lifetime (c.f. research by Merzenich and others demonstrating reorganization of somatosensory maps in adult primates -- but reorganization that appears to be restricted to within a millemeter distance). However, most developmental neurobiologists that I have read recently argue that subtractive events (i.e. axon retraction, cell death and above all synaptic degeneration) are the most interesting candidates for a brain basis of behavioral change after the infant years; there is (or so it seems right now) little likelihood that the major behavioral changes we observe across the human lifetime can be explained by recourse to additive events (cell formation, synaptogenesis, and not even the peripatetic but poorly understood event of myelination). My colleagues and I have written a chapter on the relevance of these neural events for early language development, which I would be happy to send out to anyone that is interested in this "consumer's perspective". the reference is: Bates, E., Thal, D. & Janowsky, J. (in press0. Early language development and its neural correlates. In I. Rapin & S. Segalowitz (Eds.), Handbook of Neuropsychology, Vol. 7. Holland:Elsevier. I would be interested in hearing from anyone who has tried to model higher cognitive processes in neural nets that are "exploding" (in a fairly uncontrolled way) in number of connections. Should be an interesting problem! It does appear to be the case that the first stages of language learning (from first words through the early stages of grammar) take place under precisely those circumstances. Which must be balanced against a second fact: a child who suffers a massive left hemisphere lesion up to at least age 2 - 3 can apparently acquire language at normal or near-normal levels after that point, presumably in the undamaged hemisphere. So whatever "investments" in neural tissue are being made during this rapid phase of development apparently can be "undone" and/or "redone" elsewhere. -liz bates From jbower at smaug.cns.caltech.edu Mon Dec 17 13:46:07 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Mon, 17 Dec 90 10:46:07 PST Subject: Oscillations Message-ID: <9012171846.AA03643@smaug.cns.caltech.edu> Concerning the inquiry by Mark James on the recent NIPS workshop on cortical oscillations. Matthew Wilson and I published a paper in last years NIPS proceedings that proposed a mechanism for cortical oscillations in visual cortex that involves both horizontal connections and inhibitory neurons. This work is also the subject of an upcoming paper in Neural Computation (probably the second 1991 issue). This work is derived from our efforts over the last five years to model 40 Hz oscillations in the olfactory system (c.f. Bower, J.M. 1990, Reverse engineering the nervous system: An anatomical, physiological, and computer based approach. In: An Introduction to Neural and Electronic Networks. S. Zornetzer, J. Davis, and C. Lau, editors. Academic Press pp. 3-24.) . These olfactory oscillations appear to be far more robust than those in visual cortex. I should mention that the important conclusion of the visual cortex work is that the zero phase oscillations occur only on average over a number of stimulus trials and therefore are unlikely to serve as the basis for "binding", attention, or awareness. Jim Bower jbower at smaug.cns.caltech.edu From djy at inel.gov Mon Dec 17 14:58:44 1990 From: djy at inel.gov (Daniel J Yurman) Date: Mon, 17 Dec 90 12:58:44 -0700 Subject: Multiples Message-ID: <9012171958.AA28107@gemstone.inel.gov> I have receicved over a dozen copies of this announcement. Although the announced workshop is about emotions, dry humor does not convey well via the net. I suggest the originator, whomever you are, be advised that you are scaring the fish in Idaho and we would like you to try elsewhere. Thanks for your kindness in advance. * Standard disclaimer included by reference * ------------------------------------------------------------ * Dan Yurman Idaho National Engineering Laboratory * djy at inel.gov PO Box 1625, Idaho Falls, ID 83415-3900 * phone: (208) 526-8591 fax: (208)-526-6852 * ------------------------------------------------------------ * 43N 112W -7GMT Red Shift is not a brand of chewing tobacco! From chan%unb.ca at UNBMVS1.csd.unb.ca Mon Dec 17 21:27:07 1990 From: chan%unb.ca at UNBMVS1.csd.unb.ca (Tony Chan) Date: Mon, 17 Dec 90 22:27:07 AST Subject: Connectionism vs AI? Message-ID: Jerry Feldman was asked by Scientific American magazine to write an article on connectionism and artifical intelligence. His assumption or working assumption as he embarked on the subject was the following: "Often viewed as competing, these two approaches to understanding intelligent behavior can be combined to yield scientific and practical advances." [Fri, 14 Dec 90 12:33:02 EST] A day later, Andrew J. Worth asked [Sat, 15 Dec 90 13:59:52 EST] "[A]re there fundamental differences between connectionism and AI that make them incompatible in an ideal sense?" Concerning this and related questions, Elizabeth Preston [Sun, 16 Dec 90 14:41:02 EST] mentioned, "a great deal of ink and hot air has been expended on it in various public forums over the last few years. ... I don't see that the question is answerable at this stage in the development and theoretical understanding of the two fields. In fact, I'm not sure it's even askable." I would not ask that question; instead, I would ask a larger question which, if satifactorily answered, would make the previous question obsolete: Should there exist one formal (mathematical) unifying paradigm within which all investigations concerning the artificial re-creation of intelligence are to be carried out? If the answer is yes, as I believe, an immediate question that should be asked is the legitimacy of the current ad hoc separation of these related areas: artificial intelligence, connectionism, pattern recognition, cognitive science, cybernetics, machine learning, etc.? In fact, not only there should be one---there is one! So the answer is affirmative in existential sense and in constructive sense. I refer interested readers to the communication of Lev Goldfarb [Thu, 27 Sep 90 From penrose at edda.css.gov Tue Dec 18 11:50:11 1990 From: penrose at edda.css.gov (Christopher Penrose) Date: Tue, 18 Dec 90 08:50:11 PST Subject: change of address Message-ID: <9012181650.AA00748@edda.css.gov> Please forgive me for my ever selfish use of network bandwidth! If the moderator of this list has a moment in their busy life to spare, I'd appreciate if my mailing address could change. from: {...}!esosun!jesus!penrose to: penrose at esosun.css.gov Thank you very much! I apologize to all the busy researchers whose precious moments were wasted by the callousness of this message. Christopher Penrose jesus!penrose From birnbaum at fido.ils.nwu.edu Tue Dec 18 13:02:01 1990 From: birnbaum at fido.ils.nwu.edu (Lawrence Birnbaum) Date: Tue, 18 Dec 90 12:02:01 CST Subject: ML91 Final Call for Papers Message-ID: <9012181802.AA05320@fido.ils.nwu.edu> THE EIGHTH INTERNATIONAL WORKSHOP ON MACHINE LEARNING CALL FOR PAPERS On behalf of the organizing committee, and the individual workshop committees, we are pleased to announce submission details for the eight workshop tracks that will constitute ML91, the Eighth International Workshop on Machine Learning, to be held at Northwestern University, Evanston, Illinois, USA, June 27-29, 1991. The eight workshops are: o Automated Knowledge Acquisition o Computational Models of Human Learning o Constructive Induction o Learning from Theory and Data o Learning in Intelligent Information Retrieval o Learning Reaction Strategies o Learning Relations o Machine Learning in Engineering Automation Please note that submissions must be made to the workshops individually, at the addresses given below, by March 1, 1991. The Proceedings of ML91 will be published by Morgan Kaufmann. Questions concerning individual workshops should be directed to members of the workshop committees. All other questions should be directed to the program co-chairs at ml91 at ils.nwu.edu. Details concerning the individual workshops follow. Larry Birnbaum Gregg Collins Northwestern University The Institute for the Learning Sciences 1890 Maple Avenue Evanston, IL 60201 phone (708) 491-3500 ---------------------------------------------------------------------------- AUTOMATED KNOWLEDGE ACQUISITION Research in automated knowledge acquisition shares the primary objective of machine learning research: building effective knowledge bases. However, while machine learning focuses on autonomous "knowledge discovery," automated knowledge acquisition focuses on interactive knowledge elicitation and formulation. Consequently, research in automated knowledge acquisition typically stresses different issues, including how to ask good questions, how to learn from problem-solving episodes, and how to represent the knowledge that experts can provide. In addition to the task of classification, which is widely studied in machine learning, automated knowledge acquisition studies a variety of performance tasks such as diagnosis, monitoring, configuration, and design. In doing so, research in automated knowledge acquisition is exploring a rich space of task-specific knowledge representations and problem solving methods. Recently, the automated knowledge acquisition community has proposed hybrid systems that combine machine learning techniques with interactive tools for developing knowledge-based systems. Induction tools in expert system shells are being used increasingly as knowledge acquisition front ends, to seed knowledge engineering activities and to facilitate maintenance. The possibilities of synergistic human-machine learning systems are only beginning to be explored. This workshop will examine topics that span autonomous and interactive knowledge acquisition approaches, with the aim of productive cross- fertilization of the automated knowledge acquisition and machine learning communities. Submissions to the automated knowledge acquisition track should address basic problems relevant to the construction of knowledge-based systems using automated techniques that take advantage of human input or human- generated knowledge sources and provide computational leverage in producing operational knowledge. Possible topics include: o Integrating autonomous learning and focused interaction with an expert. o Learning by asking good questions and integrating an expert's responses into a growing knowledge base. o Using existing knowledge to assist in further knowledge acquisition. o Acquiring, representing, and using generic task knowledge. o Analyzing knowledge bases for validity, consistency, completeness, and efficiency then providing recommendations and support for revision. o Automated assistance for theory / model formation and discovery. o Novel techniques for knowledge acquisition, such as explanation, analogy, reduction, case-based reasoning, model-based reasoning, and natural language understanding. o Principles for designing human-machine systems that integrate the complimentary computational and cognitive abilities of programs and users. Submissions on other topics relating automated knowledge acquisition and autonomous learning are also welcome. Each submission should specify the basic problem addressed, the application task, and the technique for addressing the problem. WORKSHOP COMMITTEE Ray Bareiss (Northwestern Univ.) Bruce Buchanan (Univ. of Pittsburg) Tom Gruber (Stanford Univ.) Sandy Marcus (Boeing) Bruce Porter (Univ. of Texas) David Wilkins (Univ. of Illinois) SUBMISSION DETAILS Papers should be approximately 4000 words in length. Authors should submit six copies, by March 1, 1991, to: Ray Bareiss Northwestern University The Institute for the Learning Sciences 1890 Maple Avenue Evanston, IL 60201 phone (708) 491-3500 Formats and deadlines for camera-ready copy will be communicated upon acceptance. ---------------------------------------------------------------------------- COMPUTATIONAL MODELS OF HUMAN LEARNING Details concerning this workshop will be forthcoming as soon as possible. ---------------------------------------------------------------------------- CONSTRUCTIVE INDUCTION Selection of an appropriate representation is critical to the success of most learning systems. In difficult learning problems (e.g., protein folding, word pronunciation, relation learning), considerable human effort is often required to identify the basic terms of the representation language. Constructive induction offers a partial solution to this problem by automatically introducing new terms into the representation as needed. Automatically constructing new terms is difficult because the environment or teacher usually provides only indirect feedback, thus raising the issue of credit assignment. However, as learning systems face tasks of greater autonomy and complexity, effective methods for constructive induction are becoming increasingly important. The objective of this workshop is to provide a forum for the interchange of ideas among researchers actively working on constructive induction issues. It is intended to identify commonalities and differences among various existing and emerging approaches such as knowledge-based term construction, relation learning, theory revision in analytic systems, learning of hidden- units in multi-layer neural networks, rule-creation in classifier systems, inverse resolution, and qualitative-law discovery. Submissions are encouraged in the following topic areas: o Empirical approaches and the use of inductive biases o Use of domain knowledge in the construction and evaluation of new terms o Construction of or from relational predicates o Theory revision in analytic-learning systems o Unsupervised learning and credit assignment in constructive induction o Interpreting hidden units as constructed features o Constructive induction in human learning o Techniques for handling noise and uncertainty o Experimental studies of constructive induction systems o Theoretical proofs, frameworks, and comparative analyses o Comparison of techniques from empirical learning, analytical learning, classifier systems, and neural networks WORKSHOP COMMITTEE Organizing Committee: Program Committee: Christopher Matheus (GTE Laboratories) Chuck Anderson (Colorado State) George Drastal (Siemens Corp.) Gunar Liepins (Oak Ridge National Lab) Larry Rendell (Univ. of Illinois) Douglas Medin (Univ. of Michigan) Paul Utgoff (Univ. of Massachusetts) SUBMISSION DETAILS Papers should be a maximum of 4000 words in length. Authors should include a cover page with authors' names, addresses, phone numbers, electronic mail addresses, paper title, and a 300 (maximum) word abstract. Do not indicate or allude to authorship anywhere within the paper. Send six copies of paper submissions, by March 1, 1991, to: Christopher Matheus GTE Laboratories 40 Sylvan Road, MS-45 Waltham MA 02254 (matheus at gte.com) Formats and deadlines for camera-ready copy will be communicated upon acceptance. ---------------------------------------------------------------------------- LEARNING FROM THEORY AND DATA Research in machine learning has primarily focused on either (1) inductively generalizing a large collection of training data (empirical learning) or (2) using a few examples to guide transformation of existing knowledge into a more usable form (explanation-based learning). Recently there has been growing interest in combining these two approaches to learning in order to overcome their individual weaknesses. Preexisting knowledge can be used to focus inductive learning and to reduce the amount of training data needed. Conversely, inductive learning techniques can be used to correct imperfections in a system's theory of the task at hand (commonly called "domain theories"). This workshop will discuss techniques for reconciling imperfect domain theories with collected data. Most systems that learn from theory and data can be viewed from the perspective of both data-driven learning (how preexisting knowledge biases empirical learning) and theory-driven learning (how empirical data can compensate for imperfect theories). A primary goal of the workshop will be to explore the relationship between these two complementary viewpoints. Papers are solicited on the following (and related) topics: o Techniques for inductively refining domain theories and knowledge bases. o Approaches that use domain theories to initialize an incremental, inductive-learning algorithm. o Theory-driven design and analysis of scientific experiments. o Systems that tightly couple data-driven and theory-driven learning as complementary techniques. o Empirical studies, on real-world problems, of approaches to learning from theory and data. o Theoretical analyses of the value of preexisting knowledge in inductive learning. o Psychological experiments that investigate the relative roles of prior knowledge and direct experience. WORKSHOP COMMITTEE Haym Hirsh (Rutgers Univ.), hirsh at cs.rutgers.edu Ray Mooney (Univ. of Texas), mooney at cs.utexas.edu Jude Shavlik (Univ. of Wisconsin), shavlik at cs.wisc.edu SUBMISSION DETAILS Papers should be single-spaced and printed using 12-point type. Authors must restrict their papers to 4000 words. Papers accepted for general presentation will be allocated 25 minutes during the workshop and four pages in the proceedings published by Morgan Kaufmann. There will also be a posters session; due to the small number of proceedings pages allocated to each workshop, poster papers will not appear in the Morgan Kaufmann proceedings. Instead, they will be allotted five pages in an informal proceedings distributed at this particular workshop only. Please indicate your preference for general or poster presentation. Also include your mailing and e-mail addresses, as well as a short list of keywords. People wishing to discuss their research at the workshop should submit four (4) copies of a paper, by March 1, 1991, to: Jude Shavlik Computer Sciences Department University of Wisconsin 1210 W. Dayton Street Madison, WI 53706 Formats and deadlines for camera-ready copy will be communicated upon acceptance. ---------------------------------------------------------------------------- LEARNING IN INTELLIGENT INFORMATION RETRIEVAL The intent of this workshop is to bring together researchers from the Information Retrieval (IR) and Machine Learning (ML) communities to explore areas of common interest. Interested researchers are encouraged to submit papers and proposals for panel discussions. The main focus will be on issues relating learning to the intelligent retrieval of textual data. Such issues include, for example: o Descriptive features, clustering, category formation, and indexing vocabularies in the domain of queries and documents. + Problems of very large, sparse feature sets. + Large, structured indexing vocabularies. + Clustering for supervised learning. + Connectionist cluster learning. + Content theories of indexing, similarity, and relevance. o Learning from failures and explanations: + Dealing with high proportions of negative examples. + Explaining failures and successes. + Incremental query formulation, incremental concept learning. + Exploiting feedback. + Dealing with near-misses. o Learning from and about humans: + Intelligent apprentice systems. + Acquiring and using knowledge about user needs and goals. + Learning new search strategies for differing user needs. + Learning to classify via user interaction. o Information Retrieval as a testbed for Machine Learning. o Particularities of linguistically-derived features. WORKSHOP COMMITTEE Christopher Owens (Univ. of Chicago), owens at gargoyle.uchicago.edu David D. Lewis (Univ. of Chicago), lewis at cs.umass.edu Nicholas Belkin (Rutgers Univ.) W. Bruce Croft (Univ. of Massachusetts) Lawrence Hunter (National Library of Medicine) David Waltz (Thinking Machines Corporation) SUBMISSION DETAILS Authors should submit 6 copies of their papers. Preference will be given to papers that sharply focus on a single issue at the intersection of Information Retrieval and Machine Learning, and that support specific claims with concrete examples and/or experimental data. To be printed in the proceedings, papers must not exceed 4 double-column pages (approximately 4000 words). Researchers who wish to propose a panel discussion should submit 6 copies of a proposal consisting of a brief (one page) description of the proposed topic, followed by a list of the proposed participants and a brief (one to two paragraph) summary of each participant's relevant work. Both papers and panel proposals should be received by March 1, 1991, at the following address: Christopher Owens Department of Computer Science The University of Chicago 1100 East 58th Street Chicago, IL 60637 Phone: (312) 702-2505 Formats and deadlines for camera-ready copy will be communicated upon acceptance. ---------------------------------------------------------------------------- LEARNING REACTION STRATEGIES The computational complexity of classical planning and the need for real-time response in many applications has led many in AI to focus on reactive systems, that is, systems that can quickly map situations to actions without extensive deliberation. Efforts to hand code such systems have made it clear that when agents must interact with complex environments the reactive mapping cannot be fully specified in advance, but must be adaptable to the agent's particular environment. Systems that learn reaction strategies from external input in a complex domain have become an important new focus within the machine learning community. Techniques used to learn strategies include (but are not limited to): o reinforcement learning o using advice and instructions during execution o genetic algorithms, including classifier systems o compilation learning driven by interaction with the world o sensorimotor learning o learning world models suitable for conversion into reactions o learning appropriate perceptual strategies WORKSHOP COMMITTEE Leslie Kaelbling (Teleos), leslie at teleos.com Charles Martin (Univ. of Chicago), martin at cs.uchicago.edu Rich Sutton (GTE), rich at gte.com Jim Firby (Univ. of Chicago), firby at cs.uchicago.edu Reid Simmons (CMU), reid.simmons at cs.cmu.edu Steve Whitehead (Univ. of Rochester), white at cs.rochester.edu SUBMISSION DETAILS Papers must be kept to four two-column pages (approximately 4000 words) for inclusion in the proceedings. Preference will be given to submissions with a single, sharp focus. Papers must be received by March 1, 1990. Send 3 copies of the paper to: Charles Martin Department of Computer Science University of Chicago 1100 East 58th Street Chicago, IL 60637 Formats and deadlines for camera-ready copy will be communicated upon acceptance. --------------------------------------------------------------------------- LEARNING RELATIONS In the past few years, there have been a number of developments in empirical learning systems that learn from relational data. Many applications (e.g. planning, design, programming languages, molecular structures, database systems, qualitative physical systems) are naturally represented in this format. Relations have also been the common language of many advanced learning styles such as analogy, learning plans and problem solving. This workshop is intended as a forum for those researchers doing relational learning to address common issues such as: Representation: Is the choice of representation a relational language, a grammar, a plan or explanation, an uncertain or probabilistic variant, or second order logic? How is the choice extended or restricted for the purposes of expressiveness or efficiency? How are relational structure mapped into neural architectures? Principles: What are the underlying principles guiding the system? For instance: similarity measures to find analogies between relational structures such as plans, "minimum encoding" and other approaches to hypothesis evaluation, the employment of additional knowledge used to constrain hypothesis generation, mechanisms for retrieval or adapation of prior plans or explanations. Theory: What theories have supported the development of the system? For instance, computational complexity theory, algebraic semantics, Bayesian and decision theory, psychological learning theories, etc. Implementation: What indexing, hashing, or programming methodologies have been used to improve performance and why? For instance, optimizing the performance for commonly encountered problems (ala CYC). The committee is soliciting papers that fall into one of three categories: Theoretical papers are encouraged that define a new theoretical framework, prove results concerning programs which carry our constructive or relational learning, or compare theoretical issues in various frameworks. Implementation papers are encouraged that provide sufficient details to allow reimplementation of learning algorithms, and discuss the key time/space complexity details motivating the design. Experimentation papers are encouraged that compare methods or address hard learning problems, with appropriate results and supporting statistics. WORKSHOP COMMITTEE Wray Buntine (RIACS and NASA Ames Research Center), wray at ptolemy.arc.nasa.gov Stephen Muggleton (Turing Institute), steve at turing.ac.uk Michael Pazzani (Univ. of California, Irvine), pazzani at ics.uci.edu Ross Quinlan (Univ. of Sydney), quinlan at cs.su.oz.au SUBMISSION DETAILS Those wishing to present papers at the workshop should submit a paper or an extended abstract, single-spaced on US letter or A4 paper, with a maximum length of 4000 words. Those wishing to attend but not present papers should send a 1 page description of their prior work and current research interests. Three copies should be sent to arrive by March 1, 1991 to: Michael Pazzani ICS Department University of California Irvine, CA 92717 USA Formats and deadlines for camera-ready copy will be communicated upon acceptance. --------------------------------------------------------------------------- MACHINE LEARNING IN ENGINEERING AUTOMATION Engineering domains present unique challenges to learning systems, such as handling continuous quantities, mathematical formulas, large problem spaces, incorporating engineering knowledge, and the need for user-system interaction. This session concerns using empirical, explanation-based, case-based, analogical, and connectionist learning techniques to solve engineering problems such as design, planning, monitoring, control, diagnosis, and analysis. Papers should describe new or modified machine learning systems that are demonstrated with real engineering problems and overcome limitations of previous systems. Papers should satisfy one or more of the following criteria: o Present new learning techniques for engineering problems. o Present a detailed case study which illustrates shortcomings preventing application of current machine learning technology to engineering problems. o Present a novel application of existing machine learning techniques to an engineering problem indicating promising areas for applying machine learning techniques to engineering problems. Machine learning programs being used by engineers must meet complex requirements. Engineers are accustomed to working with statistical programs and expect learning systems to handle noise and imprecision in a reasonable fashion. Engineers often prefer rules and classifications of events that are more general than characteristic descriptions and more specific than discriminant descriptions. Engineers have considerable domain expertise and want systems that enable application of this knowledge to the learning task. This session is intended to bring together machine learning researchers interested in real-world engineering problems and engineering researchers interested in solving problems using machine learning technology. We welcome submissions including but not limited to discussions of machine learning applied to the following areas: o manufacturing automation o design automation o automated process planning o production management o robotic and vision applications o automated monitoring, diagnosis, and control o engineering analysis WORKSHOP COMMITTEE Bradley Whitehall (Univ. of Illinois) Steve Chien (JPL) Tom Dietterich (Oregon State Univ.) Richard Doyle (JPL) Brian Falkenhainer (Xerox PARC) James Garrett (CMU) Stephen Lu (Univ. of Illinois) SUBMISSION DETAILS Submission format will be similar to AAAI-91: 12 point font, single-spaced, text and figure area 5.5" x 7.5" per page, and a maximum length of 4000 words. The cover page should include the title of the paper, names and addresses of all the authors, a list of keywords describing the paper, and a short (less than 200 words) abstract. Only hard-copy submissions will be accepted (i.e., no fax or email submissions). Four (4) copies of submitted papers should be sent to: Dr. Bradley Whitehall Knowledge-Based Engineering Systems Research Laboratory Department of Mechanical and Industrial Engineering University of Illinois at Urbana-Champaign 1206 West Green Street Urbana, IL 61801 ml-eng at kbesrl.me.uiuc.edu Formats and deadlines for camera-ready copy will be communicated upon acceptance. From hinton at ai.toronto.edu Tue Dec 18 13:17:30 1990 From: hinton at ai.toronto.edu (Geoffrey Hinton) Date: Tue, 18 Dec 1990 13:17:30 -0500 Subject: Research Associate job Message-ID: <90Dec18.131735edt.1033@neuron.ai.toronto.edu> **** DO NOT FORWARD TO OTHER BBOARDS OR MAILING LISTS **** RESEARCH ASSOCIATE POSITION AT THE UNIVERSITY OF TORONTO SALARY: $40,000 - $50,000 per annum The University of Toronto is an equal opportunity employer. THE JOB: The research associate will collaborate with graduate students on two large neural network projects, and will be expected to do a considerable amount of programming in C. The first project involves hand-printed character recognition and the second involves using a data-glove to produce speech. The funding is specifically for these two projects so the job is not suitable for someone who wishes to pursue research on other topics. The job will initially be for one year with the possibility of renewal for at least one more year. Since this is a non-permanent research position, it can be given to a non-Canadian. We are looking for someone who can start work in the spring of 1991. THE GROUP: The connectionist research group in the Department of Computer Science is directed by Geoffrey Hinton and consists of 10 graduate students (Sue Becker, Michelle Craig, Sidney Fels, Conrad Galland, Radford Neal, Steve Nowlan, Tony Plate, Evan Steeg, Chris Williams and Rich Zemel), a research associate (usually), a research programmer (Drew van Camp) and an administrator (Carol Plathan). The research focusses on developing new learning procedures and new applications for neural networks. We have our own four-processor silicon graphics machine (about 90MIPS) plus 10 sun 3/50 workstations that are used as graphics terminals. PREREQUISITES: Applicants must have a completed (or very nearly completed) PhD that involved simulations of learning in artificial neural networks. Since there are already several strong candidates for this job, we are not interested in applicants who do not already have extensive practical experience of neural networks. Experience with character recognition, speech production, or data-gloves would be valuable. HOW TO APPLY: The deadline is Jan 14 1991. Send your Curriculm Vitae (including a summary of your thesis research), copies of your one or two best papers, the names, phone numbers and email addresses of two or three references, and the date you would be available to start to: Carol Plathan, Department of Computer Science, University of Toronto, 10 Kings College Road, Toronto, Ontario M5S 1A4 Canada You could also send the information by email to carol at ai.toronto.edu. **** DO NOT FORWARD TO OTHER BBOARDS OR MAILING LISTS **** From tenorio at ecn.purdue.edu Tue Dec 18 14:46:43 1990 From: tenorio at ecn.purdue.edu (Manoel Fernando Tenorio) Date: Tue, 18 Dec 90 14:46:43 EST Subject: Connectionism vs AI? In-Reply-To: Your message of Mon, 17 Dec 90 22:27:07 D. Message-ID: <9012181946.AA19745@dynamo.ecn.purdue.edu> Bcc: -------- From: Tony Chan Concerning this and related questions, Elizabeth Preston [Sun, 16 Dec 90 14:41:02 EST] mentioned, "a great deal of ink and hot air has been expended on it in various public forums over the last few years. ... I don't see that the question is answerable at this stage in the development and theoretical understanding of the two fields. In fact, I'm not sure it's even askable." I would not ask that question; instead, I would ask a larger question which, if satifactorily answered, would make the previous question obsolete: Should there exist one formal (mathematical) unifying paradigm within which all investigations concerning the artificial re-creation of intelligence are to be carried out? If the answer is yes, as I believe, an immediate question that should be asked is the legitimacy of the current ad hoc separation of these related areas: artificial intelligence, connectionism, pattern recognition, cognitive science, cybernetics, machine learning, etc.? In fact, not only there should be one---there is one! So the answer is affirmative in existential sense and in constructive sense. I refer interested readers to the communication of Lev Goldfarb [Thu, 27 Sep 90 16:28:09 EDT]. I am not sure about a mathematical paradigm for all investigation concerning the art. intelligence. I am not really sure we can define what these are. If we restrict ourselves to inferences, in a paper soon to come out, we will shown that current NN models are of the equivalent class of the AI inference models. They can both be reduce to a graph grammar ( using cathegory theory) to a simple rewriting system of equivalent power. Neither will give much better results than a currently available for inference. We need a radical departure from these to accomplish "intelligent behavior" on this narrower sense. M. F. Tenorio. From honavar at iastate.edu Tue Dec 18 16:08:02 1990 From: honavar at iastate.edu (honavar@iastate.edu) Date: Tue, 18 Dec 90 15:08:02 CST Subject: Tech report available by ftp Message-ID: <9012182108.AA10148@iastate.edu> The following technical report is available in postscript form by anonymous ftp (courtesy Jordan Pollack of Ohio State Univ). Comments on the paper are welcome (please direct them to honavar at iastate.edu) ---------------------------------------------------------------------- Symbol Processing Systems, Connectionist Networks, and Generalized Connectionist Networks Vasant Honavar Leonard Uhr Department of Computer Science Computer Sciences Department Iowa State University University of Wisconsin-Madison Technical Report #90-24, December 1990 Department of Computer Science Iowa State University, Ames, IA 50011 Abstract Many authors have suggested that SP (symbol processing) and CN (connectionist network) models offer radically, or even fundamentally, different paradigms for modeling intelligent behavior (see Schneider, 1987) and the design of intelligent systems. Others have argued that CN models have little to contribute to our efforts to understand intelligence (Fodor & Pylyshyn, 1988). A critical examination of the popular characterizations of SP and CN models suggests that neither of these extreme positions is justified. There are many advantages to be gained by a synthesis of the best of both SP and CN approaches in the design of intelligent systems. The Generalized connectionist networks (GCN) (alternately called generalized neuromorphic systems (GNS)) introduced in this paper provide a framework for such a synthesis. ______________________________________________________________________________ You will need a POSTSCRIPT printer to print the file. To obtain a copy of the report, use anonymous ftp from cheops.cis.ohio-state.edu (here is what the transaction looks like): % ftp ftp> open cheops.cis.ohio-state.edu Connected to cheops.cis.ohio-state.edu. 220 cheops.cis.ohio-state.edu FTP server (Version blah blah) ready. Name (cheops.cis.ohio-state.edu:yourname): anonymous 331 Guest login ok, send ident as password. Password: anything 230 Guest login ok, access restrictions apply. ftp> cd pub/neuroprose 250 CWD command successful. ftp> bin 200 Type set to I. ftp> get honavar.symbolic.ps.Z 200 PORT command successful. 150 Opening BINARY mode data connection for honavar.symbolic.ps.Z (55121 bytes). 226 Transfer complete. local: honavar.symbolic.ps.Z remote: honavar.symbolic.ps.Z 55121 bytes received in 1.8 seconds (30 Kbytes/s) ftp> quit 221 Goodbye. % uncompress honavar.symbolic.ps.Z % lpr honavar.symbolic.ps From enorris at gmuvax2.gmu.edu Tue Dec 18 16:32:09 1990 From: enorris at gmuvax2.gmu.edu (Gene Norris) Date: Tue, 18 Dec 90 16:32:09 -0500 Subject: Accretional networks Message-ID: <9012182132.AA10355@gmuvax2.gmu.edu> A modest terminological proposal -- networks that change their connectivity during learning usually do so by adding weights between existing units or by adding new units (and therefore, new weights). Such networks can be called **accretional**; those that add only weights between existing units are weight-accretional, and those that add units are unit-accretional. (A network that changes its learning algorithm while learning might be called intelligence-accretional, or just plain intelligent (:-) ). If one has devised a learning algorithm that prunes networks of connections or units, these would, by extension, be called deaccretional nets (if one wanted to make a distinction). Anyway, it's a fresh, descriptive term with no (for me) connotations: good qualities for a technical term to have. --Gene Norris CS Dept George Mason University Fairfax, VA 22032 (703)323-2713 enorris at gmuvax2.gmu.edu FAX: 703 323 2630 From jbower at smaug.cns.caltech.edu Tue Dec 18 16:39:24 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Tue, 18 Dec 90 13:39:24 PST Subject: AI, NN, and CNS Message-ID: <9012182139.AA04526@smaug.cns.caltech.edu> Concerning the AI and connectionism debate. I think that this debate should take place without reference to biology. While AI by conviction has had very little to do with the structure of the nervous system, it is not at all clear that connectionism is much different. Even the "neural nets" subfield, if that is what it is, is at best vaguely connected to real biology. In my view, the modern interest in either connectionism or neural nets was not even particularly biologically inspired. As a biologist it seems to me that both have emerged from their own antecedents. Beyond origins, however, all one has to do is attend any of the annual neural net meetings to see that biology is currently at best a handmaiden to the overall effort. At worst, it is used as a justification for preconceived notions. This is not to say that there are not some people in the field who are committed to understanding what little is now known about biology, but they are few and far between. Instead, the evolution of neural networks and connectionism appears to be taking their own directions under their own priorities with real biological constraints having little effect on either. More globally, it has been pointed out before that historical attempts to understand human intelligence have always been cast in terms of the most sophisticated technology of the day. The Greeks borrowed from the technology of aqueducts in ascribing mental processes to the flow of bodily fluids. Descartes thought he thought using machines and mechanical forces. Sherrington was inspired by telephone switchboards, while theorists of the 60's and 70's considered the brain to be a digital computer. Today we would discount each of these claims believing that the brain is a parallel distributed processing device. It is important to realize, however, that these earlier speculators did not think the mechanism of mental function was similar to their favorite machine, they, like we, thought it was actually just a more complicated version of that machine. In conclusion, in my view, AI and connectionism (neural nets) should work out their own definitions on their own merits without reference to biology. Then, if there is a difference, both should fight it out based on real world performance. At that point, some biologist will probably compare the results to the abilities of some invertebrate somewhere making it clear, yet again, that we have missed the mark. Jim Bower jbower at smaug.cns.caltech.edu From goldfarb%unb.ca at UNBMVS1.csd.unb.ca Tue Dec 18 22:10:55 1990 From: goldfarb%unb.ca at UNBMVS1.csd.unb.ca (om Lev Goldfarb) Date: Tue, 18 Dec 90 23:10:55 AST Subject: Which connectionism vs which AI? Message-ID: It appears that, when discussing the relationship between NN and AI, an undue legitimacy is often granted to these two quite tentative and inadequate *formal paradigms*. The NN lacks adequate "self-programmability", while the propositional model cannot practically facilitate learning from the environment, i.e. it cannot facilitate the discovery of new useful features (new symbols) or even the recognition of "primitive" patterns. After "a great deal of ink and hot air has been expanded" (Beth Preston, Connectionism vs AI), it should be quite clear that the two formal models *in their present form* cannot naturally, or directly (in mathematical sense), be integrated into one model, in spite of the attempts by some to conveniently ignore this fact. On the other hand, if, when talking about the two "paradigms", one is not referring to the underlying *mathematical* models, then the necessity of integrating the two paradigms should be apparent and the above "great deal of ink and hot air" can easily be understood. --Lev Goldfarb From Scott.Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU Tue Dec 18 21:23:13 1990 From: Scott.Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF-PMAX.SLISP.CS.CMU.EDU) Date: Tue, 18 Dec 90 21:23:13 EST Subject: ontogenesis and synaptogenesis In-Reply-To: Your message of Mon, 17 Dec 90 09:57:51 -0800. <9012171757.AA25001@crl.ucsd.edu> Message-ID: Liz, Thanks for posting that information on synaptogenesis. It is extremely valuable to have some knowledgeable people out there helping to filter and interpret the neural-development literature for the rest of us. I'm not sure to what extent your post was inspired by some of the recent discussion of constructive/destructive algorithms (or "ontogenic", if you prefer). You're probably way ahead of me on this, but I just wanted to mention that the obvious mapping of computational theories into neuron-stuff isn't necessarily the only mapping or the best one. For example, some people automatically assume that what we call a "unit" must be implemented as a neuron, and what we call a "weight" must be implemented as an adjustable synapse. But a "unit" might instead correspond to some piece of the dendritic tree that happens to behave in a nonlinear way; on the other hand, what we call a "unit" might be implemented as a whole cluster of cooperating neurons. The situation with constructive/destructive algorithms is similar. The obvious implementation of constructive algorithms would involve growing (or recruiting) new neurons and adding them to the active network through the creation of new synapses; destructive algorithms would presumably involve elimination of existing synapses and neurons. But that isn't the only possible way of mapping these ideas to real neural systems. For example, I usually describe the Cascade-Correlation architecture as selecting new units out of a pool of candidates and "adding" them to the active network. But it is probably better to think of this event as a sort of phase transition, which I jokingly call "tenure". Before tenure, the candidate units receive a full complement of inputs, and the input weights are adjusted freely to maximize some measure of the candidate's potential usefulness (currently, the degree of correlation with the remaining error). Before tenure, these candidate units are either silent or their outputs are ignored. After tenure, the inputs are frozen (or at least much less plastic), but now the rest of the net pays attention to that unit's output. This can be a purely functional change; there doesn't really have to be any visible change in the physical topology of the network. It would be very interesting to know whether anything like this phase transition actually occurs in individual neurons, but I have no idea how one would go about looking for such a thing. -- Scott Fahlman From jose at learning.siemens.com Wed Dec 19 10:26:17 1990 From: jose at learning.siemens.com (Steve Hanson) Date: Wed, 19 Dec 90 10:26:17 EST Subject: AI, NN, and CNS Message-ID: <9012191526.AA21107@learning.siemens.com.siemens.com> Jim: It is true that connectionism and AI have tended to compete in terms modeling. Both however require something to model. I think your view on neuroscience's relation to connectionist modeling is somewhat narrow and a bit shrill. AI researchers have tended to focus on a vauge notion of "intelligence". Connectionists have tended to be less opportunistic about the modeling piece parts they have chosen but have modeled just about anything standing still. Is it all biologically motivated?--heavens no! Why should it be? Is any of it?--of course (there are several good volumes around documenting this: "Neural Modeling", Koch & Segev, 1989; "Conectionist Modeling and Brain Function", Hanson & Olson, 1990; "Connectionism and Neuroscience", Gluck & Rumelhart, 1990 and several others I forget their titles..sorry). But computational modeling can occur at any level of interest and be productive and valid. That it does not model certain cells and circuits or jim`s favorite cells or circuits does not invalidate the oppportunity for such modeling to happen and have continuity with other relevant system level modeling. It's a two-way street, both modelers and experimentalists have a responsibility to constrain and enlighten each other--if they don't then complaints like yours have a self-perpetuating flavor to them. The brain's a big place and as far as I can tell has plenty of room for lots of system level speculation. Connectionist Modelers are studying all sorts of (yes even wooly ones) system level interactions and possible mechanims for various kinds of function. Sejnowski and Churchland had a nice Science article where they attempt to lay out possible relations between "simplified" and "realistic" neural models. I wrote a paper that appeared recently in Behavioral and Brain Science on what I thought was the relation between AI and connectionism--many people responded in kind. I refer you to those papers for more detail in order to keep this conversation short. I think it is important at this point not to polarize this issue by either assuming there is only one unique way to characterize neural computation or worse that the details of the brain don't matter. Steve From gluck%psych at Forsythe.Stanford.EDU Wed Dec 19 10:30:08 1990 From: gluck%psych at Forsythe.Stanford.EDU (Mark Gluck) Date: Wed, 19 Dec 90 07:30:08 PST Subject: Preprint: Stimulus Sampling & Distributed Representations Message-ID: <9012191530.AA11533@psych> PRE-PRINT AVAILABLE: Stimulus Sampling and Distributed Representations in Adaptive Network Theories of Learning Mark A. Gluck Department of Psychology Stanford University [To appear in: A. Healy, S. Kosslyn, & R. Shiffrin (Eds.), Festschrift for W. K. Estes. NJ: Erlbaum, 1991/in press] ABSTRACT: Current adaptive network, or "connectionist", theories of human learning are reminiscent of statistical learning theories of the 1950's and early 1960's, the most influential of which was Stimulus Sampling Theory, developed by W. K. Estes and colleagues (Estes, 1959; Atkinson & Estes, 1963). This chapter reviews Stimulus Sampling Theory, noting some of its strengths and weaknesses, and compares it to a recent network model of human learning (Gluck & Bower, 1986, 1988a,b). The network model's LMS learning rule for updating associative weights represents a significant advance over Stimulus Sampling Theory's more rudimentary learning procedure. In contrast, Stimulus Sampling Theory's stochastic scheme for representing stimuli as distributed patterns of activity can overcome some limitations of network theories which identify stimulus cues with single active input nodes. This leads us to consider a distributed network model which embodies the processing assumptions of our earlier network model but employs stimulus-representation assumptions adopted from Stimulus Sampling Theory. In this distributed network, stimulus cues are represented by the stochastic activation of overlapping populations of stimulus elements (input nodes). Rather than replacing the two previous learning theories, this distributed network combines the best established concepts of the earlier theories and reduces to each of them as special cases in those training situations where the previous models have been most successful. _________________________________________________________________ To request copies, send email to: gluck at psych.stanford.edu with your hard-copy mailing address. Or mail to: Mark A. Gluck, Department of Psychology, Jordan Hall, Bldg. 420, Stanford Univ., Stanford, CA 94305-2130 From GOLDFARB%UNB.CA at UNBMVS1.csd.unb.ca Wed Dec 19 20:24:06 1990 From: GOLDFARB%UNB.CA at UNBMVS1.csd.unb.ca (Lev Goldfarb) Date: Wed, 19 Dec 90 21:24:06 AST Subject: On blurring the gap between NN and AI Message-ID: Jay McClelland: Regarding Worth's query about a possible fundamental opposition between connectionist and other approaches to AI: I do not think there need be such an opposition. That is, I think one can be a connectionist without imagining that any such opposition exists. I do not understand how one can decide or "imagine" whether there need or need not be such an opposition. In fact, as I have mentioned in my last correspondence, such "opposition" between the corresponding mathematical models *simply exists*. Wishful thinking apart, if we do not want to expand some more "hot air", we should keep in mind what John von Neumann said half a century ago: The sciences do not try to explain, they hardly even try to interpret, they mainly make models. By a model is meant a mathematical constract which, with the addition of certain verbal interpretations, describes observed phenomena. The justification of such a mathematical construct is solely and precisely that it is expected to work. Blurring the existing tentions between the two models (and I mean formal models) helps to create scientifically very unproductive state of euphoria, especially at this very critical initial stage in the development of intelligent systems. I hope, that the Scientific American article will not cater to the very popular "quest for euphoria". --Lev Goldfarb From 35C4UB2 at CMUVM.BITNET Wed Dec 19 14:18:26 1990 From: 35C4UB2 at CMUVM.BITNET (Ken Aizawa) Date: Wed, 19 Dec 90 14:18:26 EST Subject: Which connectionism vs which AI? In-Reply-To: Your message of Tue, 18 Dec 90 23:10:55 AST Message-ID: Lev Goldfarb writes (Goldfarb,Tue,18 Dec 90 23:10:55 AST): >the propositional model cannot >practically facilitate learning from the environment, i.e. it >cannot facilitate the discovery of new useful features (new symbols) >or even the recognition of "primitive" patterns. Don't the Bacon programs by Langley, Simon, Bradshaw, et. al. count as propositional models ? And don't they postulate new variables or constants (new symbols) that enter into laws of nature ? So, isn't this a counterexample to your characterization ? Ken Aizawa From worth at park.bu.edu Wed Dec 19 17:44:09 1990 From: worth at park.bu.edu (Andrew J. Worth) Date: Wed, 19 Dec 90 17:44:09 -0500 Subject: AI, NN, and CNS Message-ID: <9012192244.AA20600@park.bu.edu> The main reason I posted the original "Connectionism vs. AI" question was to imply that one should not gloss over the differences between the two philosophies when combining them. Many of the subsequent postings illustrated the problem with the definitions used in the question. I believe that the question I was thinking of could have been asked more succinctly by comparing the assumptions that lead to "the explicitly symbolic nature of many AI endeavors" with a connectionist approach that does not hold such assumptions. But this might unleash the symbol grounding debate. In lieu of that, I would like to respond to Jim Bower's posting; do AI and connectionism have anything to do with biology? Again, it depends on your definitions. "Computational Neuroscience", as described by Sejnowski, Koch, and Churchland [1] is an attempt to bring together not just biology, but also psychology and other fields to explain information processing in the brain using computational models. The results are "connectionist" models where emergent properties (not explicit symbols and rules) become important. Bower's assertion that many neural networks have little to do with biology expresses a regrettable fact. But as Bower mentioned and as Sejnowski et al. show, not all of connectionism ignores biology. Other bodies of work that do not ignore biology are those by Grossberg, et al. on vision and motor control [2,3]. Perhaps the most painless introduction to some of these ideas can be found in [4]. A practical demonstration of some of the vision work can be seen in [5]. It seems to me that the lure of connectionism is a haunting whispered promise to go where no "AI" has gone before. I consider attention to biology in general, and Grossberg et al's techniques in particular, as steps in the right direction. [1] T.J. Sejnowski, C. Koch, and P.S. Churchland, Computational Neuroscience, Science, v.241, pp.1299-1306, 9 September 1988. [2] S. Grossberg and M. Mingolla, Neural Dynamics of Perceptual Grouping: Textures, Boundaries and Emergent Segmentations, Perception & Psychophysics, v.38(2), 141-171, 1985. [3] S. Grossberg & M. Kuperstein, Neural Dynamics of Adaptive Sensory- Motor Control, New York, Pergamon Press, 1989. [4] S. Grossberg, Why do cells compete? UMAP Unit 484. The UMAP Journal, V.III, No.1 (Educational Development Center, 0197-3622/82/010101.) 1982. [5] S.M. Lehar, A.J. Worth, & D.N. Kennedy, Application of the Boundary Contour/Feature Contour System to Magnetic Resonance Brain Scan Imagery, Proc. of the International Joint Conf. on Neural Networks, v.I, p.435-440, 1990. Andrew J. Worth worth at park.bu.edu Cognitive & Neural Systems Prog. (617) 353-6741 Boston University (617) 353-7857 (CAS Office) 111 Cummington St. Room 244 (617) 353-5235/6742 (CNS Grad Offices) Boston, MA 02215 From jbower at smaug.cns.caltech.edu Thu Dec 20 02:59:29 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Wed, 19 Dec 90 23:59:29 PST Subject: AI, NN, CNS (central nervous system) Message-ID: <9012200759.AA06211@smaug.cns.caltech.edu> In response to Steve Hanson's comment on my comment. I think that Steve's remarks largely miss my primary point which is simply that an assumed relationship between biology and connectionism/neural networks should not be used as a distinguishing feature with respect to AI. There is simply no strong recent historical evidence for this relationship and the field itself is evolving independently of any association with real biological networks. While this may be unfortunate, it is not necessarily bad, and it is certainly pretty predictable. The points Steve does raise are potentially interesting, although it is not clear to me that they are of general interest to readers of this network for the reasons just stated. Briefly, however, Steve seems to be confusing what I would refer to as Computational Neuroscience, with Cognitive Neuroscience. The two are quite different in their intent. The Koch and Segev book, for example, with which I am quite familiar as codirector of the course on which the book is based and having written two of its chapters, describes models intended to shed light on structure/function relationships within the nervous system. This is Computational Neuroscience. Using modeling to explore more abstract mental or cognitive functions is Cognitive Neuroscience from which connectionism largely grew and to which it mostly relates. Thus, while computational modeling can be applied at any level of interest, where it may or may not be "productive and valid", the real question is productive and valid for what. If you are interested in cognitive neuroscience, by all means make models. Remember, however, there is no necessary relationship between these models and the way the brain actually does things. Further, the largest limitation on this connection is liable to be our poor understanding of cognitive behavior to begin with. Thus, in the absence of any example to the contrary, I reject the idea that there is necessarily a continuity between all levels of modeling no matter how abstract or detailed. And this obviously has nothing to do with my favorite cells or circuits which was a silly and inappropriate remark to begin with. Briefly, I would also like to comment on Steve's tautology regarding the necessity of interactions between experimentalists and modelers. As I believe he knows, a commitment to this interaction is at the very base of my own research effort. In fact, it is my view that this interaction should be so tight that modelers are also experimentalists. Further, in order for data to really constrain models, the models should generate data obtainable with experimental techniques. Practically, this is far easier if they are structurally realistic. This is one of the reasons I doubt the usefulness of abstract models for understanding brain function. In addition, most abstract models are intent on proving that somebodies idea is at best plausible and at worst correct. In my view, the real value of a tight connection between models and experiment is the possibility that brain structure itself will provide new ideas as to functional organization. It is also exceedingly ironic that Steve warns against assuming that the details of the brain don't matter. Of course, it is precisely my point that the details matter a great deal. In fact, this is the reason I assert that connectionist modeling is not biologically relevant. The structures of the vast majority of these models bear virtually no resemblance to the actual structure of the brain. They might even prove to be no closer than complicated switchboards or aqueducts. Finally, a general comment that relates to this debate as well as the debate about AI versus connectionist approaches. There is clearly a strong tendency to deal with potential conflicts by simply declaring that we are all part of one big happy family. While this might sooth egos, relieve self doubt, and provide new funding opportunities, I think it is important to resist the temptation. In the process of amalgamation important distinctions can be washed out. Jim Bower jbower at smaug.cns.caltech.edu From hendler at cs.UMD.EDU Thu Dec 20 09:18:09 1990 From: hendler at cs.UMD.EDU (Jim Hendler) Date: Thu, 20 Dec 90 09:18:09 -0500 Subject: More on AI vs. NN Message-ID: <9012201418.AA01724@dormouse.cs.UMD.EDU> I guess I feel compelled to add my two cents to this. Here's a copy of an editorial I wrote for a special issue of the journal Connection Science (Carfax Publ.) The issue concerned models that combined connectionist and symbolic components: ---------------------------- On The Need for Hybrid Systems J. Hendler (From: Connection Science 1(3), 1989 ) It is very easy to make an argument in favor of the development of hybrid connectionist/symbolic systems from an engineering (or task-oriented) perspective. After all, there is a clear and present need for developing systems which can perform both ``perceptual'' and ``cognitive'' tasks. Some examples include: Planning applications where recognition of other agents' plans must be coupled with intelligent counteractions, Speech understanding programs where speech processing, which has been most successful as a signal processing application, needs to be coupled with syntactic and semantic processing, Automated manufacturing or testing applications where visual perception needs to be coupled with expert reasoning, and Expert image processing systems where line or range detectors, radar signal classifiers, unknown perspective projections, quantitative image processors, etc. must be coupled with top-down knowledge sources such as maps and models of objects. To provide systems which can provide both ``low-level'' perceptual functionality as well as demonstrating high-level cognitive abilities we need to capture the best features of current connectionist and symbolic techniques. This can be done in one of four ways: We can figure out a methodology for getting traditional AI systems to handle image and signal processing, to handle pattern recognition, and to reason well about perceptual primitives, We can figure out a methodology for getting connectionist systems to handle ``high-level'' symbol-processing tasks in applied domains. This might involve connectionist systems which can manipulate data structures, handle variable binding in long inference chains, deal with the control of inferencing, etc., We can work out a new ``paradigm,'' yet another competitor to enter the set of possible models for delivering so-called intelligent behavior, Or, we can take the current connectionist systems and the current generation of AI systems and produce hybrid systems exploiting the strengths of each. While the first three of these are certainly plausible approaches, and all three are currently driving many interesting research projects, they require major technological breakthroughs, and much rethinking of the current technologies. The fourth, building hybrid models, requires no major developments, but rather the linking of current technologies. This approach, therefore appears to provide the path of least resistance in the short term. From a purely applied perspective, we see a fine reason to pursue the building of hybrid models. If this were the only reason for building hybrid models, and it is a strong one, it would legitimize much research in this area. The purpose of this editorial however, and in fact the rationale behind the editing of this special issue on hybrid models, is to convince the reader that there is more to hybrid models than simply a merging of technologies for the sake of building new applications: In particular, that the development of hybrid models holds major promise for bettering our understanding of human cognition and for helping to point the way in the development of future cognitive modeling techniques. This claim is based on facing up to reality: neither the current AI nor the current connectionist paradigms appear to be sufficient for providing a basic understanding of human cognitive processing. I realize this is quite a contentious statement, and I won't try to defend it rigorously in this short article. Instead, I'll try to outline the basic intuition behind this statement. Essentially, the purely symbolic paradigm of much of AI suffers from not being grounded in perception. Many basic types of cognitive processing, particularly those related to vision and the other senses, have been formed by many generations of evolution. While it is possible that a symbolic mechanism could duplicate the abilities of these ``hard-wired'' systems, it seems unlikely. Higher level cognitive abilities, such as understanding speech or recognizing images, which do not attempt to use low-level models may be doomed to failure. Consider, for example, the evidence for categorization errors and priming confusions in humans. Is this evidence of some sort of weakness in the processing system, or is it a result of the very mechanisms by which perceptual information processing proceeds in humans? If, as many believe, the latter is true, then it would appear to be the case that the apparatus by which humans perform perceptual categorization forces categories to have certain properties. If this is the case, then ability of humans to perform very broad generalizations and to recognize commonalities between widely divergent inputs is probably integrally related to this perceptual apparatus. If so, an attempt to model human understanding which doesn't take the ``limitations'' of this perceptual categorization mechanism seriously may be doomed to failure. Further, it may even be the case that any attempt to use a more perfect scheme for categorization will miss having this critical property. Thus, understanding perceptual processing, as it is implemented in the brain, may be crucial to an understanding of cognition as performed by the human problem solver. The connectionist approach, sometimes called the subsymbolic paradigm, suffers from a related problem. While current research appears to indicate that this approach may be better for modeling the lower level cognitive processes, there seems to be something crucially different between human cognition and that of other animals. It seems unlikely that this difference can be captured by a purely ``brain-based'' explanation. Human problem solving requires abilities in representation (the often cited issue of ``compositionality'' being one) and in symbol manipulation, which are currently beyond the scope of the connectionist approach (except in very simplified cases). While it is possible that brain size itself explains the differences between human and animal cognition, many researchers seem to believe that the answer requires more than this. Explaining human thinking requires more than simply explaining the purely associative mechanisms found in most mammals understanding these mechanisms is necessary for a deeper understanding of human cognition, but it is not sufficient . Thus a connectionist science which addresses the associative learning seen in the brain, without regard for the cognitive abilities resulting from that learning, is inadequate for a full understanding of human cognition. Unfortunately understanding and modeling human cognitive processing in a way that takes both abilities and implementation into account is not an easy task. To solve this problem we will eventually need to understand both the architecture of the brain and the ``programs'' running on that architecture. This interconnection between implementational constraints on the one hand, and functional requirements on the other, puts many bounds on the set of possible models which could truly be considered as the basis of human intelligence. But how do we probe the models lying within these bounds? What would they look like? Can they be based on current connectionist techniques? Will they function in a manner similar to current AI models? We know of no ``correct'' research paradigm for studying these problems: Connectionist models clearly must be pursued for a deeper understanding of the ``firmware'' of thought; traditional AI must be pursued to give us insight into the functional requirements of thought. But, it is my contention that a third path must also be followed: To be able to gain a true insight into what implementationally correct, cognitively robust models of human cognition will look like, we need to study models which try to connect the two paradigms. Hybrid models, rather than being viewed simply as a short term engineering solution, may be crucial to our gaining an understanding of the parameters and functions of biologically plausible cognitive models. From this understanding we might hope to see the development of a new, and potentially more correct, paradigm for the studying of ``real,'' as opposed to artificial, intelligence. From INTS%MUSICB.MCGILL.CA at BITNET.CC.CMU.EDU Thu Dec 20 09:27:43 1990 From: INTS%MUSICB.MCGILL.CA at BITNET.CC.CMU.EDU (Tom Shultz, Dept. of Psychology, McGill Univ.) Date: Thu, 20 Dec 90 09:27:43 EST Subject: ontogenesis Message-ID: <20DEC90.10219129.0061.MUSIC@MUSICB.MCGILL.CA> Relating the ongoing discussion of ontogeny in neural nets to her recent review of neural correlates of language development, Liz Bates writes: Synaptogenesis does NOT continue across the (human/primate) lifespan, at least not on any kind of a large or interesting scale. At least after a huge burst in synaptogenesis between (roughly) human postnatal months 6 - 24. Apparently, there is somewhat of a consensus that, through most of human development, a dieback model accounts for many more brain changes than does a synaptogenic model. However, there are some dissenting views and evidence on this. For example, Greenough and his colleagues at U. of Illinois have evidence that rats in enriched environments add around 20% more synapses than control rats. This is a process they call synapse-on-demand and it apparently occurs throughout rat life, not only in early development. And, as Scott Fahlman points out, algorithms like Cascade- Correlation, although often described as performing recruitment of new hidden units, can alternatively be described in ways that are more compatible with a synaptic change model. (Candidate hidden units are always in the network; it is just that they are not listened to until they start to do something useful.) This is essentially a caution that we shuld not impose a premature closure on the issue of how best to characterize either the connectionist or neurological literature on these issues of changing network topology. Tom Shultz From jlm+ at ANDREW.CMU.EDU Thu Dec 20 12:12:58 1990 From: jlm+ at ANDREW.CMU.EDU (James L. McClelland) Date: Thu, 20 Dec 90 12:12:58 -0500 (EST) Subject: On blurring the gap between NN and AI In-Reply-To: References: Message-ID: Regarding Goldfarb's comments 'on blurring': I agree that the models are different. All I was trying to address was the following notion, which seems to be implicit in some of the discussion: Either you believe cognition is really symbolic or your believe it's really subsymbolic. [Feel free to replace symbolic and/or subsymbolic with your favorite labels!] Neither of these views seems particularly productive. I'm with von Neumann -- I care about models that work. [What it means for a model to work depends on your purpose; see my previous post]. For the problems that interest me, connectionist models appear to work better than others. But this is not always the case. Some of my colleagues have gotten a long way with production systems. Which approach is right? Wrong question. Which approach is better? As in physics, some phenomena are best captured [modeled!] at the microstructure level, and others not. Which phenomena are best captured by each? We don't know; by choosing to use one or the other, we place our bets. Aspects of these views are presented at somewhat greater length in a paper: McClelland, J. L. (1988). Connectionist models and psychological evidence. Journal of Memory and Language, 27, 107-123. I'm bowing out of further discussion of these issues for the time being. Merry Christmas, Everyone! -- Jay McClelland From worth at park.bu.edu Thu Dec 20 14:54:39 1990 From: worth at park.bu.edu (Andrew J. Worth) Date: Thu, 20 Dec 90 14:54:39 -0500 Subject: AI, NN, and CNS (reference completion) Message-ID: <9012201954.AA28158@park.bu.edu> Sorry to bother you all again, but the complete reference in my last posting should have been: [5] S.M. Lehar, A.J. Worth, & D.N. Kennedy, Application of the Boundary Contour/Feature Contour System to Magnetic Resonance Brain Scan Imagery, Proc. of the International Joint Conf. on Neural Networks, San Diego, CA, v.I, p.435-440, June 1990. Andy. From der%psych at Forsythe.Stanford.EDU Thu Dec 20 14:56:52 1990 From: der%psych at Forsythe.Stanford.EDU (Dave Rumelhart) Date: Thu, 20 Dec 90 11:56:52 PST Subject: On blurring the gap between NN and AI In-Reply-To: Lev Goldfarb's message of Wed, 19 Dec 90 21:24:06 AST Message-ID: <9012201956.AA22941@psych> It seems to me that a number of issues are being confused in this discussion. One has to do with what AI is and another has to do with what "connectionism" is and why we might be interested it. First, with regard to AI it seems to me that there are at least three aspects. (1) Theoretical AI, (2) Experimental AI and (3) Applied AI. I take it that Theoretical AI is the development and analysis of algorithms and computational models which draw their inspiration from considerations of biological (especially) human intelligence. This strikes me as a special kind of applied mathematics. On this count, connectionist approaches, search based approaches, logic based approaches, problem space based approaches and many others all fall squarely in the realm of theoretical AI. These various approaches simply draw their inspiration from different aspects of natural intelligence and therefore find different mathematical formalisms more useful. I take it that experimental AI is the study of the algorithms and computational models developed in theoretical AI by experimental means -- that is through the development of computer simulations and implementations of the algorithms. If our models and algorithms could be fully analytically developed there would be no need for an experimental approach. Since most algorithms and computational models seem to be too complex and to interact with a world that is itself not easy to characterize we often resort to an experimental approach. Here AI differes from most (but not all) classical applied mathematical approaches. On the whole connectionist approaches employ the experimental method to about the same degree as most other AI approaches. To the degree that different AI approaches can be applied to the same problems it is certainly possible to compare different algorithms and computational approaches in terms of efficiency (on a machine of a particular type) quality of performance and other dimensions, but the effort is primarly one of analysis -- what are the properties of the systems under study. The third activity within AI, applied AI, is simply the applications of the AI techniques mentioned above to a real world problem. In this case, many practical issues intervene. Here we ask simply how well does the algorithm and procedure do on the area of application. The measurement criteria may involve difficulty of development as well as the quality of the performance. In this case, I would be suprised if a single approach was always better for all areas of application. At this level it is an empirical question. To summarize, as far as AI is concerned, it strikes me that the connectionist approach is one among many and partakes of most of the features of the other approaches. It may turn out that the connectionist approach is particularly well suited to particular kinds of applications, may be particularly elegant and may be nice in certain other ways, but beauty is often in the eye of the beholder. The question of whether there is a great divide between the symbolic and "sub-symbolic" approaches is one that I would rather leave to the philosophers. In any case, it has nothing to do with wether or not connectionist AI is a kind of AI. I simply can't think of any reasonable definition of AI that would exclude it. The second major point concerns the nature of "connectionism" itself. It should be noted that there are connectionist approaches to fields other than AI and in this case the connectionist approach cuts across several fields. In particular, there are connectionist approaches to psychology, to neuroscience, to linguistics and to other scientific domains. In these cases, the criteria for evaluation is rather different than for AI. In these fields we are interested in the degree to which models developed within the connectionist paradigm are useful in understanding, explaining or predicting empirical phenomena. These phenomena may involve explaining the behavior of people or other animals or in explaining the observations made by a neurobiologist when studying the brain. One of the hopes for the connectionist approach is that it will be able to provide a formalism for explaining the relatioinship between neural activity and behavior someday. The evidence that this will occur is, of course, not yet in. It is the job of connectionist researchers to do the necessary research, develop the necessary ideas and make the case to the scientific community at large that this is possible. Finally, I should say that this attempt to develop AI formalisms that have applicability to scientific model building is not unique to the connectionist approach. Many AI formalisms have been proposed as useful for explaining psychological or linguistic (but rarely neurobiological) phenomena. For a good example see Newell's new book on so-called Unified Theories. Sorry for the wordiness of this message. D. E. Rumelhart From pollack at cis.ohio-state.edu Thu Dec 20 20:16:15 1990 From: pollack at cis.ohio-state.edu (Jordan B Pollack) Date: Thu, 20 Dec 90 20:16:15 -0500 Subject: The details do not matter In-Reply-To: Jim Bower's message of Tue, 18 Dec 90 13:39:24 PST <9012182139.AA04526@smaug.cns.caltech.edu> Message-ID: <9012210116.AA00557@dendrite.cis.ohio-state.edu> Neurobiologists often write grossly inefficient programs, showing that Computer Science is merely a handservant to neuroscience, providing a catalog of algorithms, data-structures, and fancy jargon.:) Jim Bower writes: >> In conclusion, in my view, AI and connectionism (neural nets) should >>work out their own definitions on their own merits without reference to >>biology. Then, if there is a difference, both should fight it out based on >>real world performance. I fully agree with Jim Bower that biological plausibility is the wrong yardstick for the AI/Connectionist debate, but I refuse to have my fields trivialized. Maybe "Neural Networks" and "expert systems" are mere engineering, but Connectionism and AI are not! The subject of study is not brain, but MIND, and the mind is not a steam engine, or a telephone switching system, or a stored program computer, and it isn't a brain either. The dispute between AI and Connectionism is not about neural plausibility, or vague notions of intelligence, but about the PRINCIPLES OF ORGANIZATION of a computational system at the upper limit of biological complexity. With limited research resources and the impetus of its own history and living legends, AI has focused primarily on one particular organization of computation which is postulated to be able to support mind-like systems: the sequential/recursive rule-interpreter. There are very good reasons for this postulate, but no convincing argument that it is true or false. But there are several compelling deficits of rule-interpretation when taken literally as a model for mind: learnability (changes in rules are unstable) scaling (no systems with more than 1000's of rules) behavioral profile (no temporal behavior which is psych. plausible) parallelization (no easy way to distribute memory) biological plausibility (not evolutionarily or neurally justified) These deficits are all being addressed in multiple ways, both within AI and more broadly in the computer and cognitive sciences. Connectionism can be viewed as a unified attack on these problems, although any single connectionist model today only addresses one or two of these at a time. But the connectionist approach is based on a very different postulate, that the organization of the brain is extremely relevant to the organization of the mind. Again, there is no convincing argument that it is true or false. However, to compete against rule-interpretation as a literal theory of mind, connectionism cannot afford to be constrained by biological detail. So, while we may be interested in the coarse organization of the brain, such as its layers and columns, population codes, diameters and densities of connections, and so on, we ignore the "details" such as the lack of bidirectional synapses or the use of calcium in some synaptic modification event. Why? Because the complexity of a theory cannot be greater than the complexity of the mental faculty it purports to explain, or the theory would fail the parsimony test of science itself. On another note, this lack of attention to detail is also a conditioned response, for whenever a connectionist has crossed the line by taking a bold stand on a detailed model of some piece of the brain, X, they face the angry voices of the biologists: "No, you are doing it all wrong! Believe me, I'm the world's expert on X!" or "No, you can't model X yet, the `basic science' data isn't in and it will take me another 10 years of slicing and dicing ratbrains to get it! So, in conclusion: The AI/connectionist debate is about science, not engineering. What is involved is mind, not brain. The brain details don't matter as do its principles of organization. Jordan Pollack Assistant Professor CIS Dept/OSU Laboratory for AI Research 2036 Neil Ave Email: pollack at cis.ohio-state.edu Columbus, OH 43210 Fax/Phone: (614) 292-4890 From tsejnowski at UCSD.EDU Fri Dec 21 01:00:53 1990 From: tsejnowski at UCSD.EDU (Terry Sejnowski) Date: Thu, 20 Dec 90 22:00:53 PST Subject: AI, NN, CNS (central nervous system) Message-ID: <9012210600.AA15367@sdbio2.UCSD.EDU> In 1957 Hartline and Ratliff published a paper on "Inhibitory interaction of receptor units in the eye of Limulus" (J. Gen Physiol, 40: 357-376). They described a set of elegant experiments on lateral interactions between neighboring ommatidia and summarized their results in a model of an effective network of lateral inhibitory interactions. Their model would today be called a connectionist model -- in fact it was linear. Not only were they able to accurately summarize a lot of data, but they were able to use the model to explore the idea of contrast enhancement through lateral inhibition, something we take for granted today. This work led to a Nobel Prize for Hartline. In the intevening years this model has been elaborated in many interesting ways, including generalizations to time-dependent patterns of light, nonlinear interactions at extreme light levels, and the biophysical properties of noise in the photoreceptors. The essence of the orignal model, however, still stands even though these more elaborate and realistic models are more accurate and more complete. Bob Barlow at Syracuse has implemented a version of the model for the whole retina on their Connection Machine and is passing movies of the real world seen by the Limulus underwater through the simulated retina. The original Hartline-Ratliff model, however, is still a useful reference landmark toward which all these elaborations point. There is value in having an abstract, simplifying model to anchor the elaborations. This single example should be enough to 1) illustrate the utility of simplifying models in studying real biological problems and 2) underline the importance of paying careful attention to biological data when attempting to apply such models to other biological systems. It should also be noted that Hartline and Ratliff would not have been able to develop their model if the mathematics of linear networks had not already been established by mathematicians, physicists, and engineers, most of whom were not interested in biological problems. Without the development of a mathematics of nonlinear dynamical systems there will be no future models for future Hartlines and Ratliffs to apply to future biological problems. I find it encouraging that so many good scientists who are confronting so many difficult problems in psychology, biology and computation are begining to at least speak the same mathematics. I do not think that anything is going to be settled by debating ideologies, except who is the better debater. Precious bandwidth is better spent discussing specific problems. Terry ----- From harnad at clarity.Princeton.EDU Thu Dec 20 22:55:40 1990 From: harnad at clarity.Princeton.EDU (Stevan Harnad) Date: Thu, 20 Dec 90 22:55:40 EST Subject: Language, Tools and Brain: BBS Call for Commentators Message-ID: <9012210355.AA08443@reason.Princeton.EDU> Below is the abstract of a forthcoming target article to appear in Behavioral and Brain Sciences (BBS), an international, interdisciplinary journal that provides Open Peer Commentary on important and controversial current research in the biobehavioral and cognitive sciences. Commentators must be current BBS Associates or nominated by a current BBS Associate. To be considered as a commentator on this article, to suggest other appropriate commentators, or for information about how to become a BBS Associate, please send email to: harnad at clarity.princeton.edu or harnad at pucc.bitnet or write to: BBS, 20 Nassau Street, #240, Princeton NJ 08542 [tel: 609-921-7771] To help us put together a balanced list of commentators, please give some indication of the aspects of the topic on which you would bring your areas of expertise to bear if you are selected as a commentator. ____________________________________________________________________ Language, Tools, and Brain: The development and evolution of hierarchically organized sequential behavior Patricia Marks Greenfield Department of Psychology University of California, UCLA Los Angeles, CA 90024-1563 electronic mail: rygreen at uclasscf.bitnet Abstract: During the first two years of life a common neural substrate (roughly, Broca's area) underlies the hierarchically organized combination of elements in the development of both speech and manual action, including tool use. The neural evidence implicates relatively specific cortical circuitry underlying a grammatical "module." Behavioral and neurodevelopmental data suggest that the modular capacities for language and manipulation are not present at birth but come into being gradually during the third and fourth years of life. An evolutionary homologue of the common neural substrate for language production and manual action during the first two years of human life is hypothesized to have provided a foundation for the evolution of language before the divergence of hominids and the great apes. Support comes from the discovery of a Broca's area analogue in contemporary primates. In addition, chimpanzees have an identical constraint on hierarchical complexity in both tool use and symbol combination. Their performance matches that of the two-year-old child who has not yet developed the differentiated neural circuits for the relatively modularized production of complex grammar and complex manual construction activity. From geb at dsl.pitt.edu Fri Dec 21 09:34:34 1990 From: geb at dsl.pitt.edu (Gordon E. Banks) Date: Fri, 21 Dec 90 09:34:34 -0500 Subject: More on AI vs. NN Message-ID: <9012211434.AA01594@cadre.dsl.pitt.edu> If cognition is symbolic, and it is clear that many forms of cognition (e.g. recognition of visual objects) is performed just as well by, say, birds, as by humans. Does this mean that birds are adept at the use of symbols? From jose at learning.siemens.com Fri Dec 21 10:31:33 1990 From: jose at learning.siemens.com (Steve Hanson) Date: Fri, 21 Dec 90 10:31:33 EST Subject: AI, NN, CNS (central nervous system) Message-ID: <9012211531.AA28800@learning.siemens.com.siemens.com> aha--jim, we finally get to your anti-reductionist roots! (deep below your brown locks) Well you're right of course, Fooling around in computational space doesn't guarantee any connection whatsoever to the brain. And probably most of the readers and NIPS goers are concerned about making nets faster, better, brighter, cleaner, bigger, etcer... and not interested in making brains. However, lets be careful about creating a junkpile term and throwing people into it. Cognitive Neuroscience as a field, for example, has interest in both brains and function. As Jordan P. wanted to document a few notes ago this might even mean we are interested in a level of computational abstraction we could, if so inclined, refer to as the-- mind. Cognitive Neuroscience is about characterizing function at the level of the mind, in fact, but in terms of neural tissue. Consequently, Cognitive neuroscience is not about the engineering, "neuro-tech" that you seem to be glumping the entire neural net community into. I think there are distinctions to be drawn other than "BRAIN" and "NON-BRAIN". I still maintain connectionism is about "system-level" theory and explanation --this is a vast computational arena that requires careful, informed, and systematic exploration (and not just from neuroscientists). I don't see neuroscientists jumping up with "theories of the brain" or even small parts of the brain everyday. And of course good experimentalists are usually suspicious of good theorists--this seems to be endemic to such interaction. You will undoubtly, complain that worrying about language, problem solving, category learning, and even much of high-level perception at this point is premature since understanding the brain means THE BRAIN! Not some cartoon version of it --not a simplified, random looking computational bric-a-brac...yes, yes, I know... but physicists (and you're surrounded by a number of them out there) seem to appreciate abstracting abit..even before we have all the details straight -- "The art of model-building is the exclusion of real but irrelevant parts of the problem, and entails hazards for the builder and the reader. The builder may leave out something genuinely relevant; the reader, armed with too sophisticated an experimental probe or too accurate a computation, may take literally a schematized model whose main aim is to be a demonstration of possibility." -P. W. Anderson (from Nobel acceptance speech, 1977) merry xmas. Steve From epreston at aisun2.ai.uga.edu Fri Dec 21 16:35:15 1990 From: epreston at aisun2.ai.uga.edu (Elizabeth Preston) Date: Fri, 21 Dec 90 16:35:15 EST Subject: On blurring the gap between NN and AI In-Reply-To: Dave Rumelhart's message of Thu, 20 Dec 90 11:56:52 PST <9012201956.AA22941@psych> Message-ID: <9012212135.AA04788@aisun2.ai.uga.edu> Date: Thu, 20 Dec 90 11:56:52 PST From: Dave Rumelhart The question of whether there is a great divide between the symbolic and "sub-symbolic" approaches is one that I would rather leave to the philosophers. This is kind of you, and of course many of us are in the position of having to take any work we can get, but... The point I was trying to make in my reply to Worth's original message is prescisely that it is doubtful whether this question of a Great Divide is PHILOSOPHICALLY interesting in the first place, and this for the simple reason that it is too early in the development of these approaches to tell. This does not mean that comparative philosophical analysis of them is impossible or unhelpful at this point, but merely that the Big Picture is not to be had at the moment, and that anyone who thinks they have it is deceiving themselves. As Hegel so charmingly put it, the owl of Minerva flies only at dusk. In any case, it has nothing to do with wether or not connectionist AI is a kind of AI. I simply can't think of any reasonable definition of AI that would exclude it. I agree completely. But could someone please tell me then why it is so common, both in the academic and the popular literature, to talk about AI and connectionism as if they were two separate fields? Beth Preston From chan%unb.ca at unbmvs1.csd.unb.ca Fri Dec 21 17:28:31 1990 From: chan%unb.ca at unbmvs1.csd.unb.ca (Tony Chan) Date: Fri, 21 Dec 90 18:28:31 AST Subject: More on AI vs. NN Message-ID: Jim Hendler [Thu, 20 Dec 90 09:18:09 EST] suggests four ways to endow a system so that it can provide both "`low-level'' perceptual functionality as well as demonstrating high-level cognitive abilities" one of which is: "we can work out a new ``paradigm,'' yet another competitor to enter the set of possible models for delivering so-called intelligent behavior." The following short paper belongs to that class. title = "Learning as optimization: An overture", booktitle= "IASTED International Symposium on Machine Learning and Neural Networks", pages = "100--103", address = "New York", month = "Oct 10--11", year = 1990, Abstract = There are two principal paradigms for the study of, pattern learning or machine learning or simply learning. The symbolic paradigm [high-level] for learning, mainly of the AI approach, is typified by Mitchell's version space model and Lenat's heuristic model. And the numeric paradigm [low-level] is represented by the pattern recognition model and the neural net model. In this paper a unified paradigm based on an extension of the Goldfarb's metric learning model is outlined. The unified learning paradigm prescribes a special type of optimization over a parametric family of pseudometric (distance) spaces in order to achieve a certain stability structure (stable configuration) in the optimal pseudometric space which is an output of the learning procedure. This special optimization procedure provides a mathematical guidance by which a system learns to organize itself. From dambrosi at kowa.CS.ORST.EDU Fri Dec 21 07:33:02 1990 From: dambrosi at kowa.CS.ORST.EDU (dambrosi@kowa.CS.ORST.EDU) Date: Fri, 21 Dec 90 12:33:02 GMT Subject: Call for Papers: Uncertainty in AI 91 Message-ID: <9012212030.AA06246@turing.CS.ORST.EDU> THE SEVENTH CONFERENCE ON UNCERTAINTY IN ARTIFICIAL INTELLIGENCE UCLA, Los Angeles July 13-15, 1991 (Preceding AAAI) The seventh annual Conference on Uncertainty in AI is concerned with the full gamut of approaches to automated and interactive reasoning and decision making under uncertainty including both quantitative and qualitative methods. We invite original contributions on fundamental theoretical issues, on the development of software tool embedding approximate reasoning theories, and on the validation of such theories and technologies on challenging applications. Topics of particular interest include: - Foundations of uncertainty - Semantics of qualitative and quantitative uncertainty representations - The role of uncertainty in automated systems - Control of reasoning; planning under uncertainty - Comparison and integration of qualitative and quantitative schemes - Knowledge engineering tools and techniques for building approximate reasoning systems - User Interface: explanation and summarization of uncertain information - Applications of approximate reasoning techniques Papers will be carefully refereed. All accepted papers will be included in the proceedings, which will be available at the conference. Papers may be accepted for presentation in plenary sessions or poster sessions. Five copies of each paper should be sent to the Program Chair by March 4, 1991. Acceptance will be sent by April 22, 1991. Final camera-ready papers, incorporating reviewers' comments, will be due by May 10, 1991. There will be an eight page limit on the camera-ready copy (with a few extra pages available for a nominal fee.) Program Co-Chair: Bruce D'Ambrosio Philippe Smets Dept. of Computer Science IRIDIA 303 Dearborn Hall Universite Libre de Bruxelles. Oregon State University 50 av. Roosevelt, CP 194-6 Corvallis, OR 97331-3202 USA 1050 Brussels, Belgium tel: 503-737-5563 tel: +322.642.27.29 fax: 503-737-3014 fax: +322.642.27.15 e-mail: dambrosio at CS.ORST.EDU e-mail: R01501 at BBRBFU01.BITNET General Chair: Piero Bonissone General Electric Corporate Research and Development 1 River Rd., Bldg. K1-5C32a, 4 Schenectady, NY 12301 tel: 518-387-5155 fax: 518-387-6845 e-mail: bonisson at crd.ge.com Program Committee: Piero Bonissone, Peter Cheeseman, Max Henrion, Henry Kyburg, John Lemmer, Tod Levitt, Ramesh Patil, Judea Pearl, Enrique Ruspini, Ross Shachter, Glenn Shafer, Lofti Zadeh. From ernst at kafka.cns.caltech.edu Fri Dec 21 15:06:52 1990 From: ernst at kafka.cns.caltech.edu (Ernst Niebur) Date: Fri, 21 Dec 90 15:06:52 -0500 Subject: Cortical oscillations Message-ID: <9012212006.AA09524@kafka.cns.caltech.edu> Date: Thu, 20 Dec 90 19:01:59 -0500 From: ernst (Ernst Niebur) Concerning Jim Bower's remark on the NIPS workshop on cortical oscillations: Jim says that "the important conclusion of the visual cortex work is that the zero phase oscillations occur only on average over a number of stimulus trials and therefore are unlikely to serve as the basis for 'binding', attention, or awareness". This statement might be misunderstood. This was NOT the conclusion of the workshop but the result of Matt's and Jim's modeling work. In fact, this was one of the most discussed points during the workshop. At present, clear experimental evidence in favor or against this hypothesis is absent. Actually, two points all participants agreed upon were that the oscillations are present and that they are important for some task. Differences persisted concerning what this task might be: Some people tend to believe that the 40 Hz oscillations somehow serve as a fundamental mechanism during information processing, like a synchronizing signal. An experiment that clearly shows that phase coherence over long distances ONLY occurs on average would support this view, but this experiment has yet to be done. Other people believe that 40 Hz cortical oscillations do have something to do with attention or awareness or "higher order tasks". There is some experimental support which shows that oscillations might be NECESSARY for this kind of task; in fact, it is known for many years that 40 Hz oscillations in the EEG are related to different "higher order tasks". One example presented at the workshop is clinical work by Dierk Schwender who showed that 40 Hz oscillations in the EEG under anesthesia are strongly correlated with the development of hallucinations, dreams and with later conscious recall of events during surgery. Obviously, this does not prove that the presence of oscillations is also SUFFICIENT for "higher order tasks", but it makes this an attractive hypothesis which seems worthwhile to be looked at in more detail. Eventually, the question will have to be decided experimentally. The danger I see is a segregation of "believers" and "non-believers" who become somehow blind to the arguments of the "opponents". I think we should avoid this. Ernst Niebur ernst at descartes.cns.caltech.edu From SAYEGH%IPFWCVAX.BITNET at vma.CC.CMU.EDU Fri Dec 21 21:55:00 1990 From: SAYEGH%IPFWCVAX.BITNET at vma.CC.CMU.EDU (SAYEGH%IPFWCVAX.BITNET@vma.CC.CMU.EDU) Date: Fri, 21 Dec 90 21:55 EST Subject: 4th NN Conference. Indiana-Purdue. Message-ID: FOURTH CONFERENCE ON NEURAL NETWORKS ------------------------------------ AND PARALLEL DISTRIBUTED PROCESSING ----------------------------------- INDIANA UNIVERSITY-PURDUE UNIVERSITY ------------------------------------ 11,12,13 APRIL 1991 ------------------- CALL FOR PAPERS --------------- The Fourth Conference on Neural Networks and Parallel Distributed Processing at Indiana University-Purdue University will be held on the Fort Wayne Campus, April 11,12, 13, 1991. Authors are invited to submit a one page abstract of current research in their area of Neural Networks Theory or Application before February 5, 1991. Notification of acceptance or rejection will be sent by February 28. The proceedings of the third conference are now in press and will be announced on the network in early January. Conference registration is $20 and students attend free. Some limited financial support might also be available to allow students to attend. Abstracts and inquiries should be addressed to: email: sayegh at ipfwcvax.bitnet ----- US mail: ------- Prof. Samir Sayegh Physics Department Indiana University-Purdue University Fort Wayne, IN 46805 From GOLDFARB%UNB.CA at unbmvs1.csd.unb.ca Fri Dec 21 23:57:29 1990 From: GOLDFARB%UNB.CA at unbmvs1.csd.unb.ca (Lev Goldfarb) Date: Sat, 22 Dec 90 00:57:29 AST Subject: AI (discrete) moodel and NN (continuous) model Message-ID: D.E Rumelhart: It seems to me that a number of issues are being confused in this discussion. One has to do what AI is and another has to do with what "connectionism" is and why we might be interested [in] it. To prevent a greater confusion, let me stress again the point that was expressed by J. von Neumann. If we want to do what so far has been called science, we must evaluate the progress not by what "we might be interested in", but by the state of development of the corresponding mathematical models. Therefore, today AI *is* what the practitioners of it are "practicing", and it is not difficult to find what they are saying about the underlying mathematical model: "artificial intelligence (AI) is primarily concerned with propositional languages for representing knowledge and with techniques for manipulating these representations" (Nils J. Nilsson, Artificial Intelligence: Engineering, Science, or Slogan, AI Magazine, Winter 1981-82, p.2). It appears that there is a greater confusion in the NN community about what the underlying mathematical model is. I believe that the underlying mathematical model,i.e. the place where all "events" are developing, is the vector space model. This is because the NN can be viewed as a "mechanism" for transforming input vectors into the subset of real numbers (the transformation, or mapping, is composed of several vector space transformations). What the NN practitioners often forget is that the vector space they want to use is the one that has some geometry in it (there are "algebraic" vector spaces without any geometry in them). The reasons why the geometry is *necessary* for the NN are many: to measure the pattern closeness, to introduce (to construct) the optimization function, etc. Let me say it again: in the present setting, one cannot talk meaningfully about the NN without the corresponding geometry. Thus, as I have alluded to in my last correspondence, we have a "classical" opposition between the basic mathematical paradigms-- the discrete and the continuous. Mathematicians have usually resolved the "opposition" by inducing the continuous structure on top of the discrete one in such a way that the corresponding algebraic operations become continuous, which results in much richer mathematical models (inner product spaces, topological groups, etc.). In order to reconcile the present "AI" and the present "connectionism" (as mathematical models), i.e. to pave the way for the "new" and the desirable AI, one has to construct essentially the same new model that would reconcile, for example, the syntactic pattern recognition (based on Chomsky's generative grammars) and the classical vector space pattern recognition. The old mathematical "solutions", however, of "simple" fusing of the two structures do not work here, since the induced geometry must not be fixed but should vary depending on the structure of the classes that has to be learned. Thus, not only does the continuous structure must be fused with the discrete, which can be accomplished by associating the weighting scheme with the set of operations, but the system must also be able to *evolve structurally* in an efficient manner, i.e. it must be able to learn *efficiently* new (macro)operations necessary for discrimination of the learning class (all present AI learning algorithms do it *very* inefficiently, since they do not use any geometry on the learning space). The outline of my answer to the above "reconciliation" problem, as I have mentioned several months ago, can be found in the June's issue of Pattern Recognition, but the progress since then has been substantial. F-i-n-a-l-l-y, one quick note on why the vector space model is not likely to be of sufficient generality for many environments: the distance functions that can be generated even by simplest insertion/ deletion operations for the string patterns cannot be reconstructed in a Euclidean vector space of any dimension. This fact is not really surprising, since the class of Euclidean space forms a very small subclass of the class of all metric spaces. Thus, it seems to me that the current NN framework must be substantially modified. (Biological implications should also be discussed.) I'll be away for a week. Best wishes for the coming year. -- Lev Goldfarb From ernst at aurel.cns.caltech.edu Sat Dec 22 05:29:17 1990 From: ernst at aurel.cns.caltech.edu (Ernst Niebur) Date: Sat, 22 Dec 90 02:29:17 PST Subject: Cortical oscillations Message-ID: <9012221029.AA01457@aurel.cns.caltech.edu> Concerning Jim Bower's remark on the NIPS workshop on cortical oscillations: Jim says that "the important conclusion of the visual cortex work is that the zero phase oscillations occur only on average over a number of stimulus trials and therefore are unlikely to serve as the basis for 'binding', attention, or awareness". This statement might be misunderstood. This was NOT the conclusion of the workshop but the result of Matt's and Jim's modeling work. In fact, this was one of the most discussed points during the workshop. At present, clear experimental evidence in favor or against this hypothesis is absent. Actually, two points all participants agreed upon were that the oscillations are present and that they are important for some task. Differences persisted concerning what this task might be: Some people tend to believe that the 40 Hz oscillations somehow serve as a fundamental mechanism during information processing, like a synchronizing signal. An experiment that clearly shows that phase coherence over long distances ONLY occurs on average would support this view, but this experiment has yet to be done. Other people believe that 40 Hz cortical oscillations do have something to do with attention or awareness or "higher order tasks". There is some experimental support which shows that oscillations might be NECESSARY for this kind of task; in fact, it is known for many years that 40 Hz oscillations in the EEG are related to different "higher order tasks". One example presented at the workshop is clinical work by Dierk Schwender who showed that 40 Hz oscillations in the EEG under anesthesia are strongly correlated with the development of hallucinations, dreams and with later conscious recall of events during surgery. Obviously, this does not prove that the presence of oscillations is also SUFFICIENT for "higher order tasks", but it makes this an attractive hypothesis which seems worthwhile to be looked at in more detail. Eventually, the question will have to be decided experimentally. The danger I see is a segregation of "believers" and "non-believers" who become somehow blind to the arguments of the "opponents". I think we should avoid this. Ernst Niebur ernst at descartes.cns.caltech.edu From tp-temp at ai.mit.edu Sun Dec 23 11:51:26 1990 From: tp-temp at ai.mit.edu (Tomaso Poggio) Date: Sun, 23 Dec 90 11:51:26 EST Subject: Connectionism vs AI In-Reply-To: Elizabeth Preston's message of Sun, 16 Dec 90 14:41:02 EST <9012161941.AA03794@aisun2.ai.uga.edu> Message-ID: <9012231651.AA03833@erice> The issue of Daedalus titled Artificial Intelligence (Winter 1988), that also appeared as a paperback by MIT Press, is essentially about AI vs. Connectionism. From jbower at smaug.cns.caltech.edu Sun Dec 23 14:06:13 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Sun, 23 Dec 90 11:06:13 PST Subject: Oscillations Message-ID: <9012231906.AA08379@smaug.cns.caltech.edu> Just a note to thank Ernst Niebur for clarifying my poorly worded original posting on the oscillations issue. It is our models prediction that the zero phase coherence is on average only. It is also true that there is no published analysis of the data that resolves this important issue. Jim Bower jbower at smaug.cns.caltech.edu From jbower at smaug.cns.caltech.edu Sun Dec 23 16:17:20 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Sun, 23 Dec 90 13:17:20 PST Subject: AI, NN, BC, CNS Message-ID: <9012232117.AA08401@smaug.cns.caltech.edu> Ideologies: several comments on several comments At the risk of redundancy, let me say again, that blurring distinctions between fields (ideologies) should be avoided and that, in my opinion, there is much less of a relationship between neurobiology and the vast majority of what is going on in AI, NN, and connectionism than one would be led to believe from reading overview books, or even tuning in to the connectionist network. Beyond that, however, there is also an essential difference between what could be called "biological connectionism" and computational neurobiology as I would like to have it defined. This involves the process by which the available computational tools are applied to particular problems. Hartline and Ratliff looked at the specific structure of the Limulus eye and developed an abstract version of that specific circuit to explore the capabilities of the circuit. They did not, as connectionists, go looking for a brain circuit to which they could apply their modeling tools. The brain came first, the tools second. In this case the fact that their 1957 model is largely indistinguishable from several modern connectionist models is interesting but irrelevant. Hartline and Ratliff were not the first connectionists (which I do not believe is what Terry was trying to say, but in this field could be taken that way anyway). As biologists, Hartline and Ratliff invented something new and important by paying attention to the detailed structure of the brain as biologists. Certainly they did this by using existing mathematical tools, what choice is there. But their approach must fall under the category of computational neurobiology as distinct from what Dave Rumelhart in his excellent summary calls the "connectionist approach" to neuroscience and everything else. Again, the critical difference is whether a model is being used to explore possibilities, or to demonstrate a preexisting idea. Even an idea about the nature of the representation of information. This is a critical distinction. Most models, prominently including those of Grossberg et al, are in the later category. It should be obvious why biologists object to such models. There are many other comments that I am tempted to make on the last few days discourse. But I will refrain in the interest of limiting debate that is not related to Touring machines and other acceptable subjects. I would like to say, however, that the Hartline/ Ratliff model is an outstanding example of why KNOWING biological details and focusing directly on biological problems does matter. Of course, if one is of the opinion that the mind is something different from the brain, or that human intelligence is something above and beyond animal intelligence, then there is little point in paying attention to brain details anyway. And the debate from here becomes more theological than ideological. "And to all a goodnight" Jim Bower jbower at smaug.cns.caltech.edu From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Mon Dec 24 04:05:38 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Mon, 24 Dec 90 01:05:38 PST Subject: Cortical oscillations In-Reply-To: Your message <9012221029.AA01457@aurel.cns.caltech.edu> dated 22-Dec-1990 Message-ID: <901224004742.2040117e@Iago.Caltech.Edu> Two interesting points re. the cortical 40 Hz oscillations. 1. At the moment, no good solid electrophysiological evidence exists for 40 Hz o scillations in monkey, either in visual or in olfactory cortex. Mainly because people havenUt tried yet or are just working on it. There is some anectodal evidence but nothing with statistics etc. Given that a number of monkey researchers have looked over the last year at their single cell dat a in light of the Singer and Gray results and didnUt see anything obvious is qui te disconcerting to me... WeUll have to await more data. It really would be wei rd, though, it catUs would hum but not monkeys... 2. However, binding as postulated by von der Malsburg and attention/ awareness as postulated by Crick and myself does not require oscillations but ph ase-locking. Oscillations is just one way to achieve phase-locked firing. Thus, all these theories could perfectly well work with no n-oscillatory phase-locking. I havenUt seen any proof that phase-locking is mor e difficult to achieve in non-oscillatory than in oscillatory systems. Thus, the crucial experiment for these theories is a multi-electrode experiment, showing phase-locked firing on individual trials between cells in different cortical are as in an awake monkey performing some task requiring focal-attention. Christof Koch From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Mon Dec 24 04:05:38 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Mon, 24 Dec 90 01:05:38 PST Subject: Cortical oscillations In-Reply-To: Your message <9012221029.AA01457@aurel.cns.caltech.edu> dated 22-Dec-1990 Message-ID: <901224004742.2040117e@Iago.Caltech.Edu> Two interesting points re. the cortical 40 Hz oscillations. 1. At the moment, no good solid electrophysiological evidence exists for 40 Hz o scillations in monkey, either in visual or in olfactory cortex. Mainly because people havenUt tried yet or are just working on it. There is some anectodal evidence but nothing with statistics etc. Given that a number of monkey researchers have looked over the last year at their single cell dat a in light of the Singer and Gray results and didnUt see anything obvious is qui te disconcerting to me... WeUll have to await more data. It really would be wei rd, though, it catUs would hum but not monkeys... 2. However, binding as postulated by von der Malsburg and attention/ awareness as postulated by Crick and myself does not require oscillations but ph ase-locking. Oscillations is just one way to achieve phase-locked firing. Thus, all these theories could perfectly well work with no n-oscillatory phase-locking. I havenUt seen any proof that phase-locking is mor e difficult to achieve in non-oscillatory than in oscillatory systems. Thus, the crucial experiment for these theories is a multi-electrode experiment, showing phase-locked firing on individual trials between cells in different cortical are as in an awake monkey performing some task requiring focal-attention. Christof Koch From pollack at cis.ohio-state.edu Mon Dec 24 12:57:33 1990 From: pollack at cis.ohio-state.edu (Jordan B Pollack) Date: Mon, 24 Dec 90 12:57:33 -0500 Subject: behaviorism in biological clothing? In-Reply-To: Jim Bower's message of Sun, 23 Dec 90 13:17:20 PST <9012232117.AA08401@smaug.cns.caltech.edu> Message-ID: <9012241757.AA01203@dendrite.cis.ohio-state.edu> In a previous message, despite its catchy title, I didn't mean to imply that specific brain details don't matter in understanding the brain or building brain models; they just don't matter for resolving the scientific difference between traditional AI and connectionism. I have long felt that work on cognition should not pose as biology. This applies both to some arguments for linguistics as a natural science, and to psychological or computational work which gets justified as "neurally plausible." On the other hand, work on biology should not pose as cognition either. Jim B. concludes his recent message by striking a Skinnerian pose on these two questions: 1) Is the mind different than the brain? Yes, although we might agree they are intimately related. One can freeze and slice the brain. One can surgically remove parts of the brain. One can even study dead brains. But one cannot do ANY of these things to the mind, unless one were a New Age NeuroGuru. 2) Is human intelligence above and beyond animal intelligence? Only a little. There are only a few "relatively small" biological differences between mammal, primate, and human BRAINS, but differences between their respective INTELLIGENCES have been exacerbated by social and cultural evolution, especially in species with any significant postnatal development. Animal neuroscience work has made profound discoveries of brain mechanisms for memory and conditioning, for representations in sensory and motor peripheral areas, and elsewhere. But many animals have more independent behavior than can be described by memorizing an input-output mapping. Which details of a monkey's brain are responsible for its personality or its cooperative social behavior? It is not reasonable to explain cognition in detailed neurobiological terms for any mammal... not even a mouse! Jordan Pollack From jbower at smaug.cns.caltech.edu Mon Dec 24 14:40:05 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Mon, 24 Dec 90 11:40:05 PST Subject: Touring Message-ID: <9012241940.AA08755@smaug.cns.caltech.edu> My appologies to the community. I see that my spell checker does not distinguish between Touring and Turing. Interprete that as you will, but I can write effecient computer programs. Guess I'll just always be nothing more than a reverse engineer. Jim Bower jbower at smaug.cns.caltech.edu From slehar at park.bu.edu Wed Dec 26 11:30:09 1990 From: slehar at park.bu.edu (Steve Lehar) Date: Wed, 26 Dec 90 11:30:09 -0500 Subject: bio-connectionist vs comp-neurosci Message-ID: <9012261630.AA24869@park.bu.edu> Jim Bower draws a sharp distinction between biological connectionism and computational neuroscience. As I understand it, he defines these as follows: BIOLOGICAL CONNECTIONISM: Using connectionist ideas to explore the functional architecture of specific biological circuits found in nature. (Paying attention to the detailed structure of the brain as biologists) COMPUTATIONAL NEUROSCIENCE: Exploring the theoretical possibilities of abstract connectionist architectures for their information representation and processing potential. I have no problem with these definitions, they represent the two pronged approach of science, theory and observation. In physics, for instance, mathematics explores the possible functional relationships between variables- linear, quadratic, exponential, periodic etc. and physics makes use of these functional relationships wherever applicable to physical observations. Where I have difficulty with Bower's view is when he says that the theoretical exploration and the physical observations have little to do with one another. He says, for instance of the Hartline and Ratliff model "The fact that [the] model is largely indistinguishable from several modern connectionist models is interesting but irrelevant". Is he saying that theoretical exploration of systems that are similar, but not identical to a specific physical system are irrelevant to that system? Is it not exactly the marriage between extensive theoretical modeling and precise physical observation that has led to the scientific revolution? This is exemplified by the way that the findings of pure mathematics always seem to find an application in the applied world sooner or later. Theoretical investigation is particularly fruitful whenever science experiences a "paradigm shift", discovering a new mathematical formalism to better discribe an old set of data. Is this not exactly what is happening in our field today? Could one not say, for instance, that the center-surround connectionist models derived through theoretical explorations in some sense "predict" the physical observations of the Hartline and Ratliff data by showing that such architectures have desirable computational properties? Bower says "It should be obvious why biologists object to such [computational neuroscience] models." Pardon my ignorance, but I fail to see this 'obvious' fact- would you care to enlighten me as to why biologists would object to theoretical explorations of computational mechanisms that are clearly much more like the brain than alternative computational models? (O)((O))(((O)))((((O))))(((((O)))))(((((O)))))((((O))))(((O)))((O))(O) (O)((O))((( slehar at park.bu.edu )))((O))(O) (O)((O))((( Steve Lehar Boston University Boston MA )))((O))(O) (O)((O))((( (617) 424-7035 (H) (617) 353-6741 (W) )))((O))(O) (O)((O))(((O)))((((O))))(((((O)))))(((((O)))))((((O))))(((O)))((O))(O) From der%psych at Forsythe.Stanford.EDU Wed Dec 26 14:32:26 1990 From: der%psych at Forsythe.Stanford.EDU (Dave Rumelhart) Date: Wed, 26 Dec 90 11:32:26 PST Subject: AI, NN, CNS (central nervous system) In-Reply-To: Jim Bower's message of Wed, 19 Dec 90 23:59:29 PST <9012200759.AA06211@smaug.cns.caltech.edu> Message-ID: <9012261932.AA16499@psych> In response to the various comments concerning the relationship (or lack thereof) between connectionist modelling and neurobiology. I wish especially to address myself to Jim Bower's many comments. I must say that I find it counter productive to press for a hard distinction between Computational Neuroscience and connectionist approaches to neuroscience. In these cases, the goals are the same -- namely to understand/explain/predict observations made on certain parts of the brains of certain animals using certain measurement techniques. When possible, the goals would also be to relate such observations to the animal's behavior. It seems that Jim (and perhaps others) wish to distinguish between what he considers "bad" attempts at doing this which he dubs connectionist and "good" ones, which he dubs "computational neuroscience". I believe that the real issue should be framed differently -- in terms of the goals of any piece of work. In any theoretical discipline there is a need for development and analysis of the formal (mathematical/ computational) tools appropriate for expressing the theories at hand (this is in the realm of applied mathematics and AI -- not biology) and there needs to be the application of these tools in the modelling of particular phenomena (this is biology or cognitive neuroscience or cognitive science). Many scientists do both of these things. Perhaps most focus only on the biological or only on the formal aspects. It is true that much of the discussion in this forum is about the technical mathematical/computational foundations of computational/connectionist modelling rather than about the biological/behavioral phenomena to which the models are to be applied. This does not mean either that many of the participants might not be interested in the eventual biological applications nor that tools developed and analysed by those of us who participate may not be of value to the neurobiologist or the psychologist. It strikes me that much of the excitement of the field comes from the interdisciplinary cross-fertalization that has taken place over the past several years. If this communication is to continue to take place fruitfully we must keep the channels of communication open, to learn what we can about the questions which occupy the minds of our colleagues and not to discount the results of one another as "irrelevant". D. E. Rumelhart From rupen at cvax.cs.uwm.edu Wed Dec 26 16:29:17 1990 From: rupen at cvax.cs.uwm.edu (Rupen Sheth) Date: Wed, 26 Dec 90 16:29:17 CDT Subject: IJCNN '90 paper by Asoh Message-ID: <9012262229.AA14090@cvax.cs.uwm.edu> Hi: o I am looking for the following paper: "An Approximation of Nonlinear Discriminant Analysis by Multilayer N.N.'s" in the Proc. of IJCNN '90 by Asoh (from Japan). Could someone mail me a copy at : 3058 N. Maryland Avenue Milwaukee, WI 53211, USA. o What does IJCNN stand for? Thank you. Rupen Sheth. From thomasp at informatik.tu-muenchen.dbp.de Thu Dec 27 07:27:38 1990 From: thomasp at informatik.tu-muenchen.dbp.de (Patrick Thomas) Date: 27 Dec 90 13:27:38+0100 Subject: TR on the Modelling of Synaptic Plasticity Message-ID: <9012271227.AA01720@gshalle1.informatik.tu-muenchen.de> The following technical report is now available: BEYOND HEBB SYNAPSES: BIOLOGICAL BUILDING BLOCKS FOR UNSUPERVISED LEARNING IN ARTIFICIAL NEURAL NETWORKS Patrick V. Thomas Report FKI-140-90 Abstract This paper briefly reviews the neurobiology of synaptic plasticity as it is related to the formulation of learning rules for unsupervised learning in artificial neural networks. Presynaptic, postsynaptic and heterocellular mechanisms are discussed and their relevance to neural modelling is assessed. These include a variety of phenomena of potentiation as well as depression with time courses of action ranging from milliseconds to weeks. The original notion put forward by Donald Hebb stating that synaptic plasticity depends on correlated pre- and postsynaptic firing is reportedly inadequate. Although postsynaptic depolarization is necessary for associative changes in synaptic strength to take place (which conforms to the spirit of the hebbian law) the association is understood as being formed between pathways converging on the same postsynaptic neuron. The latter only serves as a supporting device carrying signals between activated dendritic regions and maintaining long-term changes through molecular mechanisms. It is further proposed to restrict the interactions of synaptic inputs to distinct compartments. The hebbian idea that the state of the postsynaptic neuron as a whole governs the sign and magnitude of changes at individual synapses is dropped in favor of local mechanisms which guide the depolarization-dependent associative learning process within dendritic compartments. Finally, a framework for the modelling of associative and non-associative mechanisms of synaptic plasticity at an intermediate level of abstraction, the Patchy Model Neuron, is sketched. To obtain a copy of the technical report FKI-140-90 please send your physical mail address to either "thomasp at lan.informatik.tu-muenchen.de" or Patrick V. Thomas, Institute for Medical Psychology, Goethe-31, 8000 Munich 2, Germany. From INAM%MUSICB.MCGILL.CA at bitnet.CC.CMU.EDU Thu Dec 27 11:05:34 1990 From: INAM%MUSICB.MCGILL.CA at bitnet.CC.CMU.EDU (Tony Marley) Date: Thu, 27 Dec 90 11:05:34 EST Subject: models versus data Message-ID: <27DEC90.11980318.0058.MUSIC@MUSICB.MCGILL.CA> FROM: Tony Marley, Director, McGill Cognitive Science Centre. RE: Models versus Data. In the light of recent discussions of modeling versus data, I thought it might be worthwhile (re)drawing attention to the following paper by Yellott. One of the recent examples discussed was the Hartline-Ratliff model of the limulus eye, and Mach bands, using a LINEAR SYSTEM WITH INHIBITION - one argument being that this was excellent work as the model was driven by biological data. The interesting aspect of Yellott's work is that he obtains Mach bands from a NONLINEAR SYSTEM WITHOUT INHIBITION, and argues that it is difficult on the basis of current physiological data to decide which model is "correct". (I do not believe that he specifically discusses the limulus eye so perhaps the data is clear there). Anyway, the point is to reiterate how theory dependent interpretation of (raw) data can be. ARTICLE: YELLOTT, J. I. (1989). Constant volume operators and lateral inhibition. Journal of Mathematical Psychology, 33, 1-35. "Constant volume (CV) operators are nonlinear image processing operators in which the area covered by the pointspread function around each point in the input image varies inversely with light intensity at that point. This operation is designed to make spatial resolution increase with retinal illumination, but it proves to have unexpected side-effects that mimic other important properties of human spatial vision, including Mach bands and Weber's law. Mach bands are usually attrtibuted to lateral inhibition in the retina, and when retinal image processing is modeled by a linear operator they imply such inhibition, since they cannot be produced by a nonnegative impulse response. CV operators demonstrate that Mach bands and other high-pass filter effects can be created by purely positive pointspread functions, i. e. without inhibition. This paper shows in addition that if one attempts to combine lateral inhibition with a CV operator, the results are dramatically wrong: the edge response always contains Mach bands that bulge in the wrong direction. Thus within the nonlinear theoretical framework provided by CV operators, lateral inhibition is neither necessary or sufficient for modeling Mach bands and other high-pass filter properties of spatial vision." (Yellott is at the Cognitive Sciences Department, University of California, Irvine, CA 92717.) From jbower at smaug.cns.caltech.edu Thu Dec 27 16:01:35 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Thu, 27 Dec 90 13:01:35 PST Subject: logout Message-ID: <9012272101.AA00979@smaug.cns.caltech.edu> First, I apologize to those that are not interested in this debate for the amount of traffic it has generated (especially from me). This will be my last posting on this subject, but the assumed nature of the relationship between AI, neural networks, and connectionism on the one hand and the structure of the brain, on the other, has been increasingly troubling me. With respect to David Rumelhart's comments, it was probably an error on my part to attempt to more narrowly define computational neuroscience as modeling related to the actual structure of the brain (incidentally Steve Lehar's interpretation of my definition is off by 180 degrees). The field as a whole would not accept this definition. However, the point that I was (and have been) trying to make, is that most connectionist modeling is related to more cognitive descriptions of brain function, not to the actual structure of the nervous system. Further, as my recent interaction with S. Hanson/J. Pollack over the net should have made clear, the mapping between cognitive descriptions of brain function, and actual brain structure is not at all straight forward. Accordingly, for those of us that are interested in understanding the brain's structure, it is not clear to me how useful these connectionist models will be just as it is not clear how useful cognitive approaches or AI will be. Thus, while my previous use of the word irrelevant was with respect to an historical argument about modeling that Terry Sejnowski had made, it is really not yet clear what the relevancy of the majority of connectionism will be to neurobiology. That is not to say that this work is irrelevant to its intended objective. Clearly connectionism is already making a substantial contribution in a number of different fields. It is also possible that useful tools will be developed. It is simply to say that if one is interested in understanding how the brain works, I believe it is necessary to address ones modeling efforts to the brain's detailed structure. Understanding this very complex system will not simply fall out by applying connectionist ideas to speech recognition problems. It is true that there is a growing effort to apply connectionist modeling techniques to actual brain structures. This network is not the right place to discuss this still relatively minor component of connectionism. However, I will say that I fail to be convinced of the usefulness of these models, and furthermore, I am concerned that these efforts may actually serve to further obscure the distinctions between brain organization and the organization of connectionist models. It seems to be precisely the association of connectionist models with network implementations that has confused the question of biological plausibility to begin with. The direct applications of connectionist tools to brain modeling makes these distinctions even tricker, especially in the larger connectionist/AI/ NN field where most practitioners know very little about the structure of the brain to begin with. As a neurobiologist, however, I would assert that even a cursory look at the brain reveals a structure having very little in common with connectionist models. In my view this is not simply a question of necessary modeling abstraction, it is a question of the basic computational assumptions underlying network construction (node structure, feed forward components, information encoding, error detecting, learning, overall complexity). Further, if these things are changed substantially, then I would say one no longer has a connectionist model. Finally, I would like to point out that I have spent much of the last seven years communicating and cross-fertilizing with my connectionist, neural network, AI, engineering, physicist friends, colleagues, and students. In fact, more than two thirds of the students in my laboratory are from one or another of these disciplines and we continue to learn a great deal from each other. But for communication to be successful, and multidisciplinary efforts to be real, there has to be a serious commitment to two way communication. In my view, within connectionism, there has been too much lip service paid to biological plausibility and not enough commitment to finding out what that really means. Jim Bower jbower at smaug.cns.caltech.edu From Scott.Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU Thu Dec 27 20:15:39 1990 From: Scott.Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF-PMAX.SLISP.CS.CMU.EDU) Date: Thu, 27 Dec 90 20:15:39 EST Subject: On blurring the gap between NN and AI In-Reply-To: Your message of Fri, 21 Dec 90 16:35:15 -0500. <9012212135.AA04788@aisun2.ai.uga.edu> Message-ID: ...But could someone please tell me then why it is so common, both in the academic and the popular literature, to talk about AI and connectionism as if they were two separate fields? I'll try... AI and connectionism have the same long-term goals: to understand this complex, wonderful phenomenon that we call intelligence. We ultimately want to understand intelligence at all levels and in precise mechanistic terms -- sufficient, in principle, to allow us to emulate the phenomenon on a machine of some kind. If you define the field in terms of its goals, it is clear that there is only one field here -- call it what you like. Both the traditional AI and connectionist camps include some people who are basically psychologists: they want want to understand how human and animal intelligence works. To these people, computer simulations are useful, but only as tools that help us to explore the capabilities and limitations of various models. Both camps also contain people who are basically engineers: they want to build more intelligent widgets, and if one can derive some useful ideas of constraints by investigating the workings of the human mind, that makes the task a bit easier. Psychology, to these people, is just a fertile source of search-guiding heuristics. And, of course, there are lots of people whose interests fall between these two extremes. But the engineer/psychologist split is orthogonal to the AI/connectionist split. What separates traditional, mainstream AI from connectionism is the choice of tools, and a parallel choice of what parts of the problem to work on now and what parts to defer until later (if ever). Traditional AI has had a good deal of success building upon the central ideas of heuristic search and symbolic description. These tools seem to be the right ones for modelling high-level conscious reasoning in clean, precise problem domains (or those in which the messy bits can be safely ignored). These tools are not so good at low-level sensory/motor tasks, flashes of recognition, and the like. AI people respond to this limitation in a variety of ways: some define the problem away by saying that this low level stuff is not really a part of "intelligence"; some say that it's important, but that we'll get to it later, once the science and technology of symbolic AI has progressed sufficiently; and some admit that connectionism probably offers a better set of tools for handling the low-level and messy parts of thought. Connectionism offers a different set of tools. These tools seem to be better for fuzzy, messy problems that are hard to cast into the rigid framework of symbols and propositions; they are not so good (yet) for modeling what goes on in high-level reasoning. Connectionists respond to these evident limitions in a number of ways: some believe that high-level symbolic reasoning will more-or-less automatically fall out of connectionist models once we have the tools to build larger, more complex nets; some believe that we should get to work now building hybrid connectionist/symbolic systems; some just think we can put off the problem of high-level reasoning for now (as evolution did for 4.5 billion years). Many mainstream AI people like to invoke the "Turing barrier": they imagine that their system runs on some sort of universal computational engine, so it doesn't really matter what that engine is made of. The underlying hardware can be parallel or serial, slow or fast -- that's just a matter of speed, not a matter of fundamental capability. Of course, that's just another way of focusing on part of the problem while deferring another part -- sooner or later, whether we are engineers or psychologists, we will have to understand the speed issues as well. Some important mental operations (e.g. "flashes" of recognition) occur so fast that it is hard to come up with a serial model that does the job. One can work on these speed/parallelism issues without leaving the world of hard-edged symbolic AI; my old NETL work was a step in this direction, and there are several other examples. But in connectionism, the tradition has been to focus on the parallel implementation as an essential part of the picture, along with the representations and algorithms. Because of the Turing barrier, AI people and biologists may feel that they have little to learn from one another; no such separation exists in connectionism, though we can certainly argue about whether our current models have abstracted away all biological relevance and whether that matters. So I would say that connectionism and traditional AI are attacking the same huge problem, but beginning at opposite ends of the problems and using very different tools. The intellectual skills needed by people in the two areas are very different: continous math and statistics for connectionists; discrete algorithms and logic for the AI people. Neither approach has a single, coherent "philosophy" or "model" or "mathematical foundation" that I can see -- I'm not really sure what sort of foundation Lev Goldfarb is talking about -- but there are two loose collections of techniques that differ rather dramtically in style. AI/Connectionism can be thought of as one field or two very different ones. It depends on whether you want to emphasize the common goals or the very different tools used by the two groups. One can define AI in an all-encompassing way, or one can define it in a way that emphasizes the use of hard-edged symbols and that rules out both connectionism and fuzzy logic. I prefer the broader definition -- it makes it a bit easier for us unprincipled pragmatists to sneak back and forth between the two camps -- but it is seldom worth arguing about where to draw the boundaries of a field. -- Scott From slehar at park.bu.edu Fri Dec 28 09:41:29 1990 From: slehar at park.bu.edu (Steve Lehar) Date: Fri, 28 Dec 90 09:41:29 -0500 Subject: logout Message-ID: <9012281441.AA23640@park.bu.edu> In his final communication Jim Bower strikes at the heart of his differences with biological connectionist philosophy. While many connectionists believe that their paradigm bears both a structural and functional similarity to brain and mind, and is thus a valid theoretical tool for exploring those entities, Bower believes that connectionism is no closer to understanding the brain than conventional AI or any other paradigm. My "180-degree misunderstanding" of his former posting was (I am left to guess) in thinking that he opposed ALL theoretical modeling, whereas what he opposes is all theoretical modeling of the BRAIN. It seems that Bower is ferverently convinced that the mechanisms of the brain are a deep dark secret that will not yield to simple investigations with numerical models. This is a curious view in an age when science has successfully probed the atomic element of the brain, the neuron, sufficiently to understand its major mode of operation. Models at this level he does not oppose, so long as they restrict themselves to strictly reproducing the experimental data. But when the theoretical insights gained from such models are generalized to information processing models, that, says Bower is going too far. I agree with Bower that many of today's popular neural models are very remote from actual biology, and there exists a need to close the gap between the abstract mathematical type models like backprop, and the lower level models like the Hartline and Ratliff model. In fact, that is the major thrust of the work of people like Grossberg. What I find very curious is Bower's resistance to this kind of effort. Bower says... "It is true that there is a growing effort to apply connectionist modeling techniques to actual brain structures. This network is not the right place to discuss this still relatively minor component of connectionism." I cannot disagree more! This network is exactly the place to discuss such models, since these are the kind of models that give direction and validity to the more abstract models. If these models are only a minor component of connectionism, that is a regretable fact which needs to be corrected by more discussion of these models. Bower continues... "However, I will say that I fail to be convinced of the usefulness of these models, and furthermore, I am concerned that these efforts may actually serve to further obscure the distinctions between brain organization and the organization of connectionist models." Of course they will obscure the distinction between brain organization and connectionist models. That is exactly the purpose of such models, to show the commonality between the brain and the models. Bower firmly believes that this commonality does not exist, and therefore it is fruitless to try to find it... "As a neurobiologist, however, I would assert that even a cursory look at the brain reveals a structure having very little in common with connectionist models. it is a question of the basic computational assumptions underlying network construction (node structure, feed forward components, information encoding, error detecting, learning, overall complexity)." A cursory glance at the brain reveals multitudes of simple computing elements richly interconnected with synaptic links. You say that has LITTLE to do with connectionist models? That was the very INSPIRATION for connectionist models! Now if we have some of the details wrong- node structure, feedback etc., then let us CORRECT those deficiencies in order to more closely model the brain. In fact, those are exactly the kinds of issues addressed by the more biological connectionist models like Grossberg's, which have dynamic properties and rich feedback connections precisely for that reason. Bower objects... "if these things are changed substantially, then I would say one no longer has a connectionist model." It doesn't matter what they're CALLED, you can call them whatever you like. What's important is that emulate the functional architecture of the brain. (O)((O))(((O)))((((O))))(((((O)))))(((((O)))))((((O))))(((O)))((O))(O) (O)((O))((( slehar at park.bu.edu )))((O))(O) (O)((O))((( Steve Lehar Boston University Boston MA )))((O))(O) (O)((O))((( (617) 424-7035 (H) (617) 353-6741 (W) )))((O))(O) (O)((O))(((O)))((((O))))(((((O)))))(((((O)))))((((O))))(((O)))((O))(O) From tom at asi.com Fri Dec 28 10:14:11 1990 From: tom at asi.com (Tom Baker) Date: Fri, 28 Dec 90 07:14:11 PST Subject: Limited Precision Neural Network Simulations Message-ID: <9012281514.AA05039@asi.com> I was encouraged by the discussion of limited precision simulations during the NIPS VLSI post-conference workshop. From the feedback that I received during my presentation, it seemed as though many of the audience had tried limited precision simulations on their own. Unfortunately, I was not able to get specific details on their work, or get anybody's name and address. I have read several papers of researchers that have used sixteen bit weights for back propagation, but I have not seen many papers about using less precision. I would like to hear about the experiences of other researchers that have tried to simulate neural networks with low precision calculations, as well as get any paper references that you may have. Let me know about any tricks or hacks that you have used to get non-floating point simulators to work. I will collect a bibliography and a separate list of techniques that others have tried. I will post the results to the net, and continue to keep the bibliography as new papers are published. I would like to keep in touch with the people that are doing research in this area. Thomas Baker INTERNET: tom at asi.com Adaptive Solutions, Inc. UUCP: (uunet,ogicse)!adaptive!tom 1400 N.W. Compton Drive, Suite 340 Beaverton, Oregon 97006 From nowlan at ai.toronto.edu Fri Dec 28 11:24:18 1990 From: nowlan at ai.toronto.edu (Steven J. Nowlan) Date: Fri, 28 Dec 1990 11:24:18 -0500 Subject: logout In-Reply-To: Your message of Fri, 28 Dec 90 09:41:29 -0500. Message-ID: <90Dec28.112432edt.827@neuron.ai.toronto.edu> I usually avoid free-wheeling network discussions such as this, but I believe that Steve Lehar is doing Jim Bower an injustice in his characterization of Jim's argument: | In his final communication Jim Bower strikes at the heart of his | differences with biological connectionist philosophy. While many | connectionists believe that their paradigm bears both a structural and | functional similarity to brain and mind, and is thus a valid | theoretical tool for exploring those entities, Bower believes that | connectionism is no closer to understanding the brain than | conventional AI or any other paradigm. My "180-degree | misunderstanding" of his former posting was (I am left to guess) in | thinking that he opposed ALL theoretical modeling, whereas what he | opposes is all theoretical modeling of the BRAIN. | | It seems that Bower is ferverently convinced that the mechanisms of | the brain are a deep dark secret that will not yield to simple | investigations with numerical models. My own (admittedly limited) understanding of the crux of Jim's argument might be summarized as follows: The idea that "a neuron functions by emitting action potentials proportional to a non-linear squashing function applied to the total activity received through its synaptic connections with other neurons" is at least as far from the truth as the idea that "a neuron represents a logical proposition." This strikes me as a reasonable statement, given what little we do know about the incredible complexity of neuronal function. I think Jim's point is important to bear in mind, because it (should) keep us from attempting to justify a connectionist model of some phenomena simply (or mainly) because it is "more brain like" than some other abstract model. This sort of reasoning is tempting, and places one on very shaky scientific ground. It is all too easy to develop some pet theory of how X is done, design some network model based on this theory, simulate the model and exclaim "Aha, this model supports theory Y about X and is a network -- so theory Y must explain how the brain does X". Since the assumptions of the theory were built into the model in the first place, the simulations may in fact tell us very little. Our models and theories need to be tested in the time honored way -- by considering what predictions the theories make and attempting to design critical experiments which will support or refute these predictions. This is not to say that connectionist modelling has nothing to say to the experimental biologist. I think a very good example of what it has to say can be seen in some of Sean Lockery's work on the leech bending reflex. What this work suggests is that single cell recordings of isolated neurons, and analysis of the synaptic organization of individual neurons is not likely to be very fruitful for understanding the functional role of these neurons because real biological neurons appear to share a computational property of connectionist models -- the functional role of any unit cannot be understood in isolation but only in the context of the functioning of other computational units. Given the current state of development of connectionist models, and understanding of biological neuronal processing, it seems that cross-fertilization of ideas is likely to be most effective at this rather abstract level of computational properties. - Steve From jbower at smaug.cns.caltech.edu Fri Dec 28 16:27:19 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Fri, 28 Dec 90 13:27:19 PST Subject: postscript Message-ID: <9012282127.AA04284@smaug.cns.caltech.edu> I'm sorry, but I feel compelled to point out that I could not have better illustrated the consequences of the prevailing assumptions about connectionism and the brain than Steve Lahar has done. Briefly however: - my point is that a closer relationship between connectionism and the brain has not been PROVEN and therefore should not be ASSUMED simply because connectionists work with network structures. I do not doubt and actually have been trying to assert that "many connectionists BELIEVE that their paradigm is a valid tool for exploring...the brain" it is a different thing to PROVE IT however. - I do not oppose "all theoretical modeling of the brain", I oppose imposing abstract theoretical constructs (e.g. ART I, II, III, etc.) on brain structure and then claiming that these models are actually derived from the structure they are imposed on. This is very different from building realistic low level models and then abstracting those. This is what I actually do for a living and is decidedly not what Grossberg has done. The difference is that, in the approach I am advocated, there is some chance that the brain will actually tell you something you didn't know before you started. Given its complexity, in my view, this is the ONLY way we will figure out how it works. - Finally, there is obviously a link between thinking that "science" understands "the major mode of operation of the neuron" (whatever that could even be) and thinking that the brain is composed of "simple computational elements". Both are absolutely wrong. As a rule of thumb, if your model is simple, it is unlikely to be capturing anything deep about the way the brain works, because the brain is almost certainly the most complicated machine we know about and its complexity is very unlikely to be a result of sloppy engineering. Show me any poorly designed hack that has 10 to the 12th components, a single component of which can not be realistically modeled on even today's fastest computer, whose source of power is as energetic as glucose, that is capable of the information processing feats the brain pulls off in real time, and still doesn't generate enough free energy to keep itself within its ideal operating range. All I am really asking for is a little respect for this system and a little less arrogance from those who do not study its structure directly. Jim Bower From aarons at cogs.sussex.ac.uk Fri Dec 28 17:40:02 1990 From: aarons at cogs.sussex.ac.uk (Aaron Sloman) Date: Fri, 28 Dec 90 22:40:02 GMT Subject: AI, NN, Neurobiology, architectures and design space Message-ID: <4016.9012282240@rsuna.cogs.susx.ac.uk> I'd like to make some comments on the recent discussions from the standpoint of a philosopher who has dabbled in various aspects of AI for about 20 years and believes that in principle it should be possible (in the VERY distant future) to replicate all or most interesting features of human minds in machines of some kind, because I don't believe in magic of any kind. Also I have seen several fashions come and go. I'll start with some background comments before getting to more meaty matters (bringing together, and extending, some of the points already made by other people). 1. Despite many written and spoken words about the differences between Connectionism and AI, I think it is clear that PDP/NN/Connectionist models have FAR more in common with AI models than they have with human brains, both in terms of what they do or how they work, and in terms of what they don't (yet) do (see below). Unfortunately people who know little about AI (e.g. those who think that expert systems and automatic theorem provers exhaust AI, because they don't know about AI work on speech, vision, robotics, numeric-based learning systems, etc.) are easily misled into believing exaggerated claims about the differences. A good antidote for such exaggerations is the technical report "Symbol Processing Systems, Connectionist Networks, and Generalized Connectionist Networks," by Honavar and Uhr, recently announced in this mail forum. Another good antidote is to survey IJCAI (International Joint Conference on AI) proceedings over the years. Some of the authors of narrowly focussed text-books on AI are partly to blame for the misconceptions about the scope of AI (e.g. books that present AI as logic). (Heated and noisy but fairly vacuous, and often transient, disputes between factions or fashions are quite common in science and academe. I leave it to others to offer sociological explanations in terms of patterns of mutual excitation and inhibition.) 2. I am not saying that there are no differences between NNs and other AI models, but that the technical, scientific, and philosophical significance of the differences has been exaggerated. E.g. I think there is NO significant philosophical difference. The biological differences are more concerned with what they are trying to explain than with correctness as models. The technical differences are still barely understood. (More on this below). NNs are closer to some non-NN AI models (e.g. in speech and vision) than those AI models are to other non-NN AI models (e.g. in theorem proving, natural language processing). E.g. So-called sub-symbolic or sub-cognitive or micro- features in NNs have much in common with low level or intermediate representations in vision. Distributed representations have much in common with intermediate databases in vision programs. There's also a loose analogy between distributed representations and theorems implicit in (=distributed over?) an axiom set. Like some of the earlier discussants, I see all AI and NN work as exploring sub-regions in the space of possible explanatory designs. We understand too little of this space to know where the major discontinuities are. 3. Concerning details of brain structure, both seem miles off. Jim Bower wrote (Thu, 27 Dec 90 13:01:35 PST) | .....As a neurobiologist, however, I would assert | that even a cursory look at the brain reveals a structure having | very little in common with connectionist models. The same can be said about existing AI models and intelligence. I'll put the point differently: it is clear from even a cursory study of literature on the anatomy and physiology of the brain that far more complex and varied designs exist than anyone has yet modelled. The same conclusion can be reached on the basis of cursory reflection on the differences between the abilities of people, squirrels, nest-building birds, etc. and the abilities of current AI or NN models. (To say nothing of what neurobiologists can explain!) 4. People working "at the coalface" in a scientific or technical discipline often find it hard to see the limitations of what they are doing, and therefore exaggerate its importance. Drew McDermott once wrote a paper called "Artificial Intelligence meets natural stupidity" (reprinted in John Haugeland, ed. Mind Design, MIT press 1981), criticising (from an AI standpoint) AI researchers who, among other things, use words and phrases from ordinary language as "wishful mnemonics", e.g. "Goal", "Understand", "Planner", "General Problem Solver". Much of what he says can be applied equally to NN research where words like "Learn", "Recognise" and "Interpret" are used to describe mechanisms that do little more than map vectors from one space into another, or store vectors using content-addressible memories. Maybe a prize should be offered for the best essay by a student re-writing Drew's paper with a title something like "Artificial nets meet dumb brains"? 5. I am not attacking NN research: I am merely pointing out commonalities between NN and AI research. The limitations of both are to be expected: Because most scientific research is inevitably piecemeal, and progress depends in part on a lot of systematic exploration of detail, a tendency that is common to most researchers is that they focus on tiny sub-mechanisms without any consideration of the global architecture within which those mechanisms might function. (I suspect this is equally true of people who study real brains, but I don't know their work so well. It is certainly true of many psychologists.) By consideration of the "global architecture" I mean the study of the capabilities of the whole system, and the analysis of a system into distinct sub-systems or sub-mechanisms with different causal roles (defined by how they interact with the environment and with other sub-systems), contributing to (and explaining) the global capabilities of the whole system. (I think this is closely related to what Jim Hendler wrote: Thu, 20 Dec 90 09:18:09 -0500). This study of global architecture is a very hard thing to do, especially when the functional decomposition may not be closely related to anatomical decomposition. So much of the analysis has to be inspired from a design standpoint (how could we make something like this?). This is best done in parallel with the more detailed studies: with feedback between them. Notes: 5.1. Don't assume that the division into sub-systems has to be rigidly defined: sub-mechanisms may share functions or change functions. 5.2 Don't assume that every system has a fixed architecture: one important kind of capability may be creation, destruction or modification of sub-structures or sub-mechanisms. Some of these may be virtual machines of changing complexity implemented in a physical mechanism of fixed complexity: a common feature of sophisticated software. Perhaps the conceptual development of a child is best understood as the development of new (virtual?) sub-mechanisms that make new processes possible: e.g. percepts or thoughts of increasing complexity, more complex motivational patterns, etc. 5.3. Don't assume that there is only one level of decomposition into sub-systems. A proper understanding of how the system works, may require some of the sub-systems to be thought of as themselves implemented in lower level mechanisms with different capabilities. There may be many levels. 5.4. Don't assume there's a fixed separation of levels of implementation: some of the very high level functionality of large scale sub-mechanism may be closely coupled with some very low level mechanism. An example might be chemical processes that alter speed of processing, or turn whole sub-mechanisms on or off. (How does alcohol alter decision making?) (Compare a computer whose programs are run by a microcode interpreter that can be altered by those very programs, or an interpreter uses subroutines written in the language being interpreted.) 6. It's clear that the global architecture of human beings includes a lot of coarse-grained parallelism. I.e. there are often many concurrent processes e.g. simultaneous walking, talking, thinking, eating, hearing, seeing, scratching one's ear, feeling hungry, feeling cold, etc. to say nothing of the regulation of internal physiological processes we are not aware of, or the decomposition of processes like walking, or talking, into different concurrent sub-processes (generating motor control signals, internal monitoring, external sensory monitoring, etc. etc.) Moreover a process that might have one label in ordinary language (e.g. "seeing") can simultaneously perform several major sub-functions (e.g. providing optical flow information for posture control, providing information about the nearby 3-d structure of the environment for short term motor planning, providing 2-d alignment information for checking direction of movement, providing information for future path-finding, providing enjoyment of the scenery, etc. etc.) I suspect that it would be quite useful for a new sub-stream of AI research to address the problem of how best to think of the high-level decomposition of a typical human mind. (Re-invent faculty psychology from an engineering standpoint?) Although some AI people have always emphasised the need to think about complete systems (the motivation for some AI robot projects), it has simply not been technically possible to aim at anything but vastly oversimplified designs. Roboticists don't normally include the study of motivation, emotions, visual enjoyment, conceptual learning, social interaction, etc, etc. So, very little is known about what kind of high level architecture might be required for designing something close to human capabilities. The techniques of software engineering (e.g. requirements analysis) coupled with philosophical conceptual analysis and surveys of what is known from psychology, linguistics, anthropology, etc. might eventually lead us to some useful conjectures about the global architecture, that could then be tested by a combination of implementational experiments (using AI, NN, or any other relevant techniques) and directed neurobiological studies. (I've been trying to do this recently in connection with attitudes, motivation, emotions and attention.) 7. Top-down illumination from this kind of architectural analysis may be required to give some direction (a) to conventional AI (since the space of possible software systems is too vast to be explored only bottom up) (b) to the exploration of NNs (since the space of possible networks, with different topologies, different self-modification algorithms, different threshold functions, etc. etc. is also too vast to be explored bottom up) and (c) to the study of real brains, since without good hypotheses about what a complex and intricate mechanism might be for, and how its functions might be implemented, it is too easy to concentrate on the wrong features: e.g. details that have little relevance to how the total system works. (Like measuring the shape, density, elasticity, etc. of something because you don't know that it's primarily a resistor in an electronic circuit. How can neurobiologists tell whether they are making this sort of mistake?) 8. The formal study of global architectures is somewhat different from mathematical analysis of formalisms or algorithms in computer science, and different from the kind of numerical mathematics that has so far dominated NN research. It will require at least a considerable generalisation of the mathematics of control theory to incorporate techniques for representing mutual causal interactions between systems undergoing qualitative and structural changes that cannot be accommodated in a set of differential equations relating a fixed set of variables. It will probably require the invention (or discovery?) of a host of new organising technical concepts, roughly analogous to the earlier discovery of concepts like feedback, information (in the mathematical sense), formal grammars, etc. (I think Lev Goldfarb was saying something similar (Sat, 22 Dec 90 00:57:29 AST), but I am not sure I understood it right.) 9. Minimising the significance of the AI/NN divide: I conjecture that most of the things that interest psychologists and cognitive scientists about human beings, e.g. our ability to perceive, learn, think, plan, act, communicate, co-operate, have desires, have emotions, be self-aware, etc. etc. depend more on the global architecture (i.e. how many co-existing, functionally distinct, causally interacting, sub-mechanisms there are, what their causal relationships are, and what functions they support in the total system) than on the implementation details of sub-mechanisms. It is not obvious what difference it makes how the various sub- components are implemented. E.g. for many components the difference between an NN implementation and a more conventional AI implementation may make a difference to speed (on particular electronic technology), or flexibility, or reliability, or modifiability -- differences that are marginal compared with the common functionality that arises not from the implementation details of individual systems but from the causal relations with other parts of the whole system. (Philosophical, scientific, and engineering issues converge here.) (Compare replacing one make of capacitor or resistor in an electronic circuit with another that has approximately the same behaviour: if the circuit is well designed, the differences in detailed behaviour will not matter, except perhaps in highly abnormal conditions, eg. high temperatures, high mechanical stress, or when exposed to a particular kind of corrosive gas etc. If the gas turns up often, the difference is important. Otherwise not.) It is quite likely that different sorts of implementation techniques will be needed for different sub-functions. E.g. rapid visual-motor feedback involved in posture control in upright bipeds (who are inherently very unstable) may be best served by NNs that map input vectors into output vectors. For representing the main high level steps in a complex sequence of actions (e.g. tying a shoelace) or for working out a plan to achieve a number of goals in a richly structured environment, very different mechanisms may be more suitable, even if NN's are useful for transforming low-level plan details to signals for co-operating muscles. NNs as currently studied may not be the best mechanism for accurate storage of long sequences of items, e.g. the alphabet, a poem, a dance routine, a memorised piano sonata, etc. When we have a good theory of the global architecture we'll be in a better position to ask which sub-mechanisms are best implemented in which ways. However, using a less suitable mechanism for one of the components may, like changing a resistor in a circuit, produce only a difference in degree, not kind, of capability for the whole system. (Which is why I think the NN/AI distinction is of no philosophical significance. This agrees with Beth Preston (21 Dec 90 16:35:15 EST) but for different reasons.) 10. The conjecture that implementation details of sub-mechanisms is relatively unimportant in explaining global capabilities, will be false (or partly false) to the extent that high level functionality depends essentially on close-coupling of different implementation levels. Are the mechanisms that allow alcohol (or other drugs) to produce qualitative changes in high level processes intimately bound up with chemical control mechanisms that are essential for normal human functioning, in the sense that no other implementation would have worked, or are they side-effects of biologically inessential implementation details that result from accidents of evolutionary history? We know that natural heart valves and kidneys can be replaced by artificial ones made very differently. We don't yet know which brain sub-mechanisms could also be replaced because most of the detail is inessential. When we know more, it may turn out that in order to explain human behaviour when drugged it is necessary to look at details that are irrelevant when explaining normal capabilities, from the design standpoint. Of course, some neurobiologists will not accept that two essentially similar circuits have the same functionality for the same reason if their components are made differently: but that's just a kind of scientific myopia. (I am not sure whether Jim Bower was saying that.) 11. It may also turn out that some aspect of the global architecture, for example the nature of the required causal links between different sub-mechanisms, favours one kind of implementation over another. Is there any intrinsic difference in the kind of information flow, or control flow, that can be implemented (a) between two or more components linked only by a few high speed parseable byte-streams, (b) between components linked by shared recursive data-structures accessed via pointers in a shared name-space, and (c) between two components linked by a web of connecting fibres? (I am not saying these are the only options for communication between sub-mechanisms.) I suspect not: only differences in speed and resistance to physical damage, etc. But perhaps there are important relevant theorems I don't know about (perhaps not yet proven?). 12. There are many problems not yet addressed by either NN or AI research, and some addressed but not solved. E.g. I suspect that neither has much to say about the representation of (arbitrary) shapes in visual systems, such that the representation can both be quickly derived from sampling the optic array and can also usefully serve a multiplicity of purposes, including: recognition, finding symmetries, seeing similarity of structure despite differences of detail, fine motor control, motor planning, explaining an object's capabilities, predicting points of contact of moving objects, etc. etc. Adequate representations of spatial structure and motion may require the invention of quite new techniques. 13. Although not everyone should put all their efforts into this, I commend the interdisciplinary exploration of the space of possible global architectures to people working in AI, NN, and neurobiology. (We may need some help from philosophers, software engineers, mathematicians, linguists, psychologists, ....) I fear there are probably a lot more fads and fashions waiting to turn up. Apologies: this message grew too long. Aaron Sloman, EMAIL aarons at cogs.sussex.ac.uk aarons%uk.ac.sussex.cogs at nsfnet-relay.ac.uk aarons%uk.ac.sussex.cogs%nsfnet-relay.ac.uk at relay.cs.net From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Fri Dec 28 14:04:04 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Fri, 28 Dec 90 11:04:04 PST Subject: Computational Neuroscience Message-ID: <901228110404.20401b2b@Iago.Caltech.Edu> Re. Steve Lehars distinction between biological connectionism and computational neuroscience. ... > BIOLOGICAL CONNECTIONISM: Using connectionist ideas to explore the > functional architecture of specific biological circuits found in > nature. (Paying attention to the detailed structure of the brain as > biologists) > > COMPUTATIONAL NEUROSCIENCE: Exploring the theoreticalpossibilities > of abstract connectionist architectures for their information > representation and processing potential. ... I, for once, disagree with these definitions. The aim of computational From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Fri Dec 28 14:04:04 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Fri, 28 Dec 90 11:04:04 PST Subject: Computational Neuroscience Message-ID: <901228110404.20401b2b@Iago.Caltech.Edu> Re. Steve Lehars distinction between biological connectionism and computational neuroscience. ... > BIOLOGICAL CONNECTIONISM: Using connectionist ideas to explore the > functional architecture of specific biological circuits found in > nature. (Paying attention to the detailed structure of the brain as > biologists) > > COMPUTATIONAL NEUROSCIENCE: Exploring the theoreticalpossibilities > of abstract connectionist architectures for their information > representation and processing potential. ... I, for once, disagree with these definitions. The aim of computational From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Fri Dec 28 14:10:05 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Fri, 28 Dec 90 11:10:05 PST Subject: Sorry, garbled transmission Message-ID: <901228110911.20401b36@Iago.Caltech.Edu> Re. Steve Lehars distinction between biological connectionism and computational neuroscience. ... > BIOLOGICAL CONNECTIONISM: Using connectionist ideas to explore the > functional architecture of specific biological circuits found in > nature. (Paying attention to the detailed structure of the brain as > biologists) > > COMPUTATIONAL NEUROSCIENCE: Exploring the theoreticalpossibilities > of abstract connectionist architectures for their information > representation and processing potential. ... I, for once, disagree with these definitions. The aim of computational From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Fri Dec 28 14:10:05 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Fri, 28 Dec 90 11:10:05 PST Subject: Sorry, garbled transmission Message-ID: <901228110911.20401b36@Iago.Caltech.Edu> Re. Steve Lehars distinction between biological connectionism and computational neuroscience. ... > BIOLOGICAL CONNECTIONISM: Using connectionist ideas to explore the > functional architecture of specific biological circuits found in > nature. (Paying attention to the detailed structure of the brain as > biologists) > > COMPUTATIONAL NEUROSCIENCE: Exploring the theoreticalpossibilities > of abstract connectionist architectures for their information > representation and processing potential. ... I, for once, disagree with these definitions. The aim of computational From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Fri Dec 28 14:20:16 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Fri, 28 Dec 90 11:20:16 PST Subject: One last time Message-ID: <901228111456.2040193d@Iago.Caltech.Edu> Re. Steve Lehars distinction between biological connectionism and computational neuroscience. ... > BIOLOGICAL CONNECTIONISM: Using connectionist ideas to explore the > functional architecture of specific biological circuits found in > nature. (Paying attention to the detailed structure of the brain as > biologists) > > > COMPUTATIONAL NEUROSCIENCE: Exploring the theoreticalpossibilities > of abstract connectionist architectures for their information > representation and processing potential. ... I, for once, disagree with these definitions. The aim of computational neuroscience (CN) is to understand information processing, storage and propagation in nervous systems, from nematodes to men. Within CN, theories can exist at many different levels of organization and complexity, ranging from biophysical faithful models of propagation of action potentials in axons (e.g. Hodgkin-Huxley and non-linear cable theory) to much more abstract models of, say, the computations underlying optical flow in visual cortex to even more abstract connectionists models of visual information processing (e.g. shape- from-shading) or higher-level cognitive operations. All these models are constrained to a greater-or-lesser extent by neurobiological and psychophysical data appropriate to their level of investigation. Thus, it would not make sense to simulate the diffusion equation in single dendritic spines when considering how we compute stereo acuity (we do not have to simulate the laws governing current flowing through a transistor when trying to understand the FFT algorithm). Thus, connectionists modelsQ-if appropriately mapped onto biologyQ-are a part of CN. Another distinction that can be made is between simplified and realistic models. The Reichardt-correlation model of motion detection in insects is a beuatiful instance of this. This model describes how the steady- state optomotor response of the fly to moving stimuli at the formal mathematical level and therefore captures the essential nonlinearity in this computation. It even carries over to human short-range motion system. Yet it specifies nothing about the implementation. This is for a latter, more realistic model. On the other hand, we will never understand the brain by building a huge detailed model of it, simulating every neuron in great detail. Even if we could, this simulation would be as complex and ill-understood as the brain itself. Thus, we need both types of models. This is a point NOT always appreciated by experimentalists, whose frequent objection to a theory is ...it does not explain my favorite observation XYZ... The point is, is this observation relevan t towards understanding the specific computation considered? For more details on this see our article on Computational Neuroscience by Sejnowski, Churchland and Koch, Science, 1988 Christof koch at iago.caltech.edu , From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Fri Dec 28 14:20:16 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Fri, 28 Dec 90 11:20:16 PST Subject: One last time Message-ID: <901228111456.2040193d@Iago.Caltech.Edu> Re. Steve Lehars distinction between biological connectionism and computational neuroscience. ... > BIOLOGICAL CONNECTIONISM: Using connectionist ideas to explore the > functional architecture of specific biological circuits found in > nature. (Paying attention to the detailed structure of the brain as > biologists) > > > COMPUTATIONAL NEUROSCIENCE: Exploring the theoreticalpossibilities > of abstract connectionist architectures for their information > representation and processing potential. ... I, for once, disagree with these definitions. The aim of computational neuroscience (CN) is to understand information processing, storage and propagation in nervous systems, from nematodes to men. Within CN, theories can exist at many different levels of organization and complexity, ranging from biophysical faithful models of propagation of action potentials in axons (e.g. Hodgkin-Huxley and non-linear cable theory) to much more abstract models of, say, the computations underlying optical flow in visual cortex to even more abstract connectionists models of visual information processing (e.g. shape- from-shading) or higher-level cognitive operations. All these models are constrained to a greater-or-lesser extent by neurobiological and psychophysical data appropriate to their level of investigation. Thus, it would not make sense to simulate the diffusion equation in single dendritic spines when considering how we compute stereo acuity (we do not have to simulate the laws governing current flowing through a transistor when trying to understand the FFT algorithm). Thus, connectionists modelsQ-if appropriately mapped onto biologyQ-are a part of CN. Another distinction that can be made is between simplified and realistic models. The Reichardt-correlation model of motion detection in insects is a beuatiful instance of this. This model describes how the steady- state optomotor response of the fly to moving stimuli at the formal mathematical level and therefore captures the essential nonlinearity in this computation. It even carries over to human short-range motion system. Yet it specifies nothing about the implementation. This is for a latter, more realistic model. On the other hand, we will never understand the brain by building a huge detailed model of it, simulating every neuron in great detail. Even if we could, this simulation would be as complex and ill-understood as the brain itself. Thus, we need both types of models. This is a point NOT always appreciated by experimentalists, whose frequent objection to a theory is ...it does not explain my favorite observation XYZ... The point is, is this observation relevan t towards understanding the specific computation considered? For more details on this see our article on Computational Neuroscience by Sejnowski, Churchland and Koch, Science, 1988 Christof koch at iago.caltech.edu , From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Sat Dec 29 02:42:07 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Fri, 28 Dec 90 23:42:07 PST Subject: logout In-Reply-To: Your message <90Dec28.112432edt.827@neuron.ai.toronto.edu> dated 28-Dec-1990 Message-ID: <901228232412.20401d16@Iago.Caltech.Edu> Having carried out detailed biophysical simulation of single neurons at the single cell level all my professional life with the aim of trying to identify and understand the elementary biophysical mechanisms underlying information processing , I disagree with Steve LeharUs statement (in reply to JimUs earlier comment): This is a curious view in an age when science has successfully probed the atomic element of the brain, the neuron, sufficiently to understand its major mode of operation. . . We do NOT understand why neurons have dendritic trees and why they come in diffe rent sizes and shapes, what---if any--- nonlinear operations go on there, why the membrane at the cell body contains more than a dozen distinct ionic currents, what nonlinear functional is being computed bat the soma, etc. etc. On the other hand, we have made progress. Thus, modeling the I/O capabilities of a neuron via linear synaptic interaction plus a nonlinear squash ing function (e.g. the Hopfield model) at the soma is a more faithful rendition than the binary McCulloch and Pitts neuron. Adding sigma-pi capabilities, i.e. allowing for the possibility of synaptic products (nominals) is a further improvement. The recent realization that the detailed temporal structure of the action potential discharge might be terrible relevant (e.g. 40 Hz oscillations) to its function a further improvment in our view of the neuron. Thus, even if some of todayUs connectionists model are still crude, they point in the right direction. Christof koch at iago.caltech.edu From koch%CITIAGO.BITNET at vma.CC.CMU.EDU Sat Dec 29 02:42:07 1990 From: koch%CITIAGO.BITNET at vma.CC.CMU.EDU (Christof Koch) Date: Fri, 28 Dec 90 23:42:07 PST Subject: logout In-Reply-To: Your message <90Dec28.112432edt.827@neuron.ai.toronto.edu> dated 28-Dec-1990 Message-ID: <901228232412.20401d16@Iago.Caltech.Edu> Having carried out detailed biophysical simulation of single neurons at the single cell level all my professional life with the aim of trying to identify and understand the elementary biophysical mechanisms underlying information processing , I disagree with Steve LeharUs statement (in reply to JimUs earlier comment): This is a curious view in an age when science has successfully probed the atomic element of the brain, the neuron, sufficiently to understand its major mode of operation. . . We do NOT understand why neurons have dendritic trees and why they come in diffe rent sizes and shapes, what---if any--- nonlinear operations go on there, why the membrane at the cell body contains more than a dozen distinct ionic currents, what nonlinear functional is being computed bat the soma, etc. etc. On the other hand, we have made progress. Thus, modeling the I/O capabilities of a neuron via linear synaptic interaction plus a nonlinear squash ing function (e.g. the Hopfield model) at the soma is a more faithful rendition than the binary McCulloch and Pitts neuron. Adding sigma-pi capabilities, i.e. allowing for the possibility of synaptic products (nominals) is a further improvement. The recent realization that the detailed temporal structure of the action potential discharge might be terrible relevant (e.g. 40 Hz oscillations) to its function a further improvment in our view of the neuron. Thus, even if some of todayUs connectionists model are still crude, they point in the right direction. Christof koch at iago.caltech.edu From gluck%psych at Forsythe.Stanford.EDU Mon Dec 31 23:41:43 1990 From: gluck%psych at Forsythe.Stanford.EDU (Mark Gluck) Date: Mon, 31 Dec 90 20:41:43 PST Subject: Full/Part-Time Research Assistant & Programmer Positions Message-ID: <9101010441.AA19467@psych> Two Full/Part Time Research Assistant Positions in: --------------------------------------------------- COGNITIVE PSYCHOLOGY / NEURAL NETWORK MODELING at Rutgers University Center for Molecular & Behavioral Neuroscience 195 University Avenue Newark, NJ 07102 Two research positions are available for persons interested in pursuing empirical and/or theoretical research in the in cognitive and neural sciences. The positions are ideal for someone who has just graduated with an undergraduate degree and would like a year or two of "hands on" experience in research before applying to graduate school in one of the cognitive sciences (e.g., neuroscience, psychology, computer science). We are looking for two people: 1. RESEARCH PROGRAMMER: A person with strong programming skills to work in the development of computational theories of the neural & cognitive bases of learning. Familiarity with current PDP/neural-network algorithms and research would be helpful, as would experience with C/Unix and Sun computer systems. Work would either focus on the development of network models of human learning and/or biological-circuit models of the neural bases of animal learning. 2. EXPERIMENTAL RESEARCH ASSISTANT: A person with experience in running and designing human cognitive psychology experiments to work in the design, execution, and data analysis of behavioral studies of human categorization learning. __________________________________________________________________________ Other Information: FACILITIES: The Center is a new state-funded research center for the integrated studies of cognitive, behavioral, and molecular neuroscience. The Center has good computational resources and experimental laboratories for behavioral and neural studies. LOCATION: The Center is located in Newark, NJ, approximately 20 minutes outside of Manhattan, New York (with easy train and subway access to midtown and downtown NYC) and close to rural New Jersey countryside. Numerous other research centers in the cognitive and neural sciences are located nearby including: Cognitive Science Center, Rutgers/New Brunswick; Centers for Cognitive & Neural Science, New York University; Cognitive Science Center, Princeton Univ.; Columbia Univ. & Medical School; Siemens Corporate Research, Princeton, NJ; NEC Research Labs, Princeton, NJ; AT&T Labs; Bellcore; IBM T. J. Watson Research Labs. CURRENT FACULTY: E. Abercrombie, G. Buzsaki, I. Creese, M. Gluck, H. Poizner, R. Siegel, P. Tallal, J. Tepper. Six additional faculty will be hired. The center has a total of ten state-funded postdoctoral positions and will direct, in collaboration with the Institute for Animal Behavior, a graduate program in Behavioral and Neural Sciences. ---------------------------------------------------------------------------- For more information on learning research at the CMBN/Rutgers or to apply for these post-doctoral positions, please send a cover letter with a statement of your research interests, a CV, copies of relevant preprints, and the the names & phone numbers of references to: Dr. Mark A. Gluck Phone: (415) 725-2434 Dept. of Psychology <-[Current address to 4/91] FAX: (415) 725-5699 Jordan Hall; Bldg. 420 Stanford University email: gluck at psych.stanford.edu Stanford, CA 94305-2130 From thomasp at informatik.tu-muenchen.dbp.de Thu Dec 20 11:08:18 1990 From: thomasp at informatik.tu-muenchen.dbp.de (Patrick Thomas) Date: 20 Dec 90 17:08:18+0100 Subject: TR on Modelling of Synaptic Plasticity Message-ID: <9012201608.AA13167@gshalle1.informatik.tu-muenchen.de> The following technical report is now available: BEYOND HEBB SYNAPSES: BIOLOGICAL BUILDING BLOCKS FOR UNSUPERVISED LEARNING IN ARTIFICIAL NEURAL NETWORKS Patrick V. Thomas Report FKI-140-90 Abstract This paper briefly reviews the neurobiology of synaptic plasticity as it is related to the formulation of learning rules for unsupervised learning in artificial neural networks. Presynaptic, postsynaptic and heterocellular mechanisms are discussed and their relevance to neural modelling is assessed. These include a variety of phenomena of potentiation as well as depression with time courses of action ranging from milliseconds to weeks. The original notion put forward by Donald Hebb stating that synaptic plasticity depends on correlated pre- and postsynaptic firing is reportedly inadequate. Although postsynaptic depolarization is necessary for associative changes in synaptic strength to take place (which conforms to the spirit of the hebbian law) the association is understood as being formed between pathways converging on the same postsynaptic neuron. The latter only serves as a supporting device carrying signals between activated dendritic regions and maintaining long-term changes through molecular mechanisms. It is further proposed to restrict the interactions of synaptic inputs to distinct compartments. The hebbian idea that the state of the postsynaptic neuron as a whole governs the sign and magnitude of changes at individual synapses is dropped in favor of local mechanisms which guide the depolarization-dependent associative learning process within dendritic compartments. Finally, a framework for the modelling of associative and non-associative mechanisms of synaptic plasticity at an intermediate level of abstraction, the Patchy Model Neuron, is sketched. To obtain a copy of the technical report FKI-140-90 please send your physical mail address to either "thomasp at lan.informatik.tu-muenchen.de" or Patrick V. Thomas, Institute for Medical Psychology, Goethe-31, 8000 Munich 2, Germany.