From Phil.Hearne at newcastle.ac.uk Mon Oct 1 05:50:26 1990 From: Phil.Hearne at newcastle.ac.uk (Phil.Hearne@newcastle.ac.uk) Date: Mon, 01 Oct 90 10:50:26 +0100 Subject: turing Message-ID: Hello, I'm a PhD student at Newcastle-upon-Tyne university in England and have recently heard about this mail group. Would anyone be kind enough to tell me how I could read the mail and perhaps get a copy of some of the debate on the Turing equivilence of connectionist models? Thanks, Phil Hearne From dario%TECHUNIX.BITNET at VMA.CC.CMU.EDU Mon Oct 1 08:12:00 1990 From: dario%TECHUNIX.BITNET at VMA.CC.CMU.EDU (Dario Ringach) Date: Mon, 1 Oct 90 14:12:00 +0200 Subject: Information-Based Complexity in Vision (request) Message-ID: <9010011212.AA11506@techunix.technion.ac.il> I would be grateful for any references on information-based complexity works (in the sense of [1]) to vision and image processing. Thanks in advance. --Dario [1] J. Traub, et al "Information-Based Complexity", Academic Press, 1988. From kamil at wdl1.wdl.fac.com Mon Oct 1 13:53:48 1990 From: kamil at wdl1.wdl.fac.com (Kamil A Grajski) Date: Mon, 1 Oct 90 10:53:48 -0700 Subject: TR - MasPar Performance Estimates Message-ID: <9010011753.AA23816@wdl1.wdl.fac.com> To receive copy of following tech report send physical address to: kamil at wdl1.fac.ford.com. (TCP/IP #137.249.32.102). ------------------------------------------------------------------------ NEUROCOMPUTING USING THE MasPar MP-1 MASSIVELY PARALLEL PROCESSOR Kamil A. Grajski Ford Aerospace Advanced Development Department / MSX-22 San Jose, CA 95161-9041 (408) 473 - 4394 ABSTRACT We present an evaluation of neurocomputing using the MasPar MP-1, massively parallel processor. Performance figures are obtained on a 2K processor element (PE) machine. Scaling behavior is evaluated for certain cases on a 4K and an 8K PE machine. Extrapolated performance figures are for the full 16K PE machine. Specific neural networks evaluated are: a.) "vanilla" back-propogation, yielding approximately 10 MCUPS real-time learning, (16K machine), for a 256-128-256 network; b.) an Elman-type recurrent network (256-128-256, 1 time delay, 16K machine) yielding approximately 9.5 MCUPS real-time learning; and c.) Kohonen self-organizing feature map yielding 1335 10-dimensional patterns per second on a 2K PE machine only (2048 units), or 27.3 MCUPS. The back-prop networks are mapped as one weight per processor. The Kohonen net is mapped as one unit per PE. The resultant performance figures suggest that for back-prop networks, a single copy, many weights per processor mapping should increase performance. Last, we present basic data transfer and arithmetic benchmarks useful for a priori estimates of machine performance on problems of interest in neurocomputing. ------------------------------------------------------------------------ If you wish to receive additional information on the machine and benchmarks for other types of problems, e.g., image processing, please contact MasPar directly. Or, only if you specifically tell me, I'll pass along your name & area of interest to the right folks over there. From perham at nada.kth.se Tue Oct 2 03:24:34 1990 From: perham at nada.kth.se (Per Hammarlund) Date: Tue, 2 Oct 90 08:24:34 +0100 Subject: Large scale biologically realistic simulations. Message-ID: <9010020724.AA22487@nada.kth.se> Hello, I have a question concerning implementations of programs for biologically realistic simulations on massively parallel machines or other "super" computers. In fact, what I am most interested in is whether anyone has, or knows of, simulations software (for any computer) that is capable of simulating something like 50,000 neurons and in the order of 1,000,000 synapses. I am interested in these programs as we have implemented a program on the Connection Machine, from Thinking Machines Corp, that can do about that, on the 8k machine we have at KTH. The title and abstract of the preliminary report (TeX syntax, please ignore) is enclosed below. Pointers to reports are greatly appriciated! Thank you very much! Per Hammarlund SANS-NADA KTH, Royal Inst. of Tech. S-100 44 Stockholm Sweden ---------------- 8< ---------------- biosim A Program for Biologically Realistic Neural Network Simulations on the Connection Machine Bj\"orn Levin and Per Hammarlund Studies of Artificial Neural Systems Department of Numerical Analysis and Computer Science} TRITA-NA-P9021 This paper describes a program for biologically realistic neural network simulations on the Connection Machine. The biosim program works as a back end to another program, swim, that handles the user interaction and specification of the neural network. The swim program takes a specification of the neural network and generates a parameter file that is read into the biosim program and processed. The output has the same format as the output of the swim program. The purpose of the biosim program is to make simulations of larger networks possible. The neuron model that is used in the simulations is a compartmentalized abstraction of the neuron. One of the compartments acts as the soma, ie the cell body, and the rest act as parts of the dendrite tree. The voltage dependent ion channels are modeled using Hodgkin-Huxley-like equations. The program supports Na+, K+, Ca2+, and Calcium dependent K+ channels. Synaptic interaction includes the voltage gated NMDA receptors and conventional kainate/AMPA receptors. Hooks provided in the code make the addition of new types of channels and receptors extremely easy. The program is capable of handling some tens of thousands of compartments and about ten times that number of synapses on an 8K machine. The numerical method used in solving the differential equations is aimed at speed at the expense of some accuracy. ---------------- 8< ---------------- From cherkaue at cs.wisc.edu Tue Oct 2 09:56:35 1990 From: cherkaue at cs.wisc.edu (Kevin Cherkauer) Date: Tue, 2 Oct 90 08:56:35 -0500 Subject: Mailing list Message-ID: <9010021356.AA29435@perenica.cs.wisc.edu> Please remove me from this list. I am getting to many advertisements. From jose at learning.siemens.com Wed Oct 3 09:33:34 1990 From: jose at learning.siemens.com (Steve Hanson) Date: Wed, 3 Oct 90 09:33:34 EDT Subject: MLP classifiers == Bayes Message-ID: <9010031333.AA18368@learning.siemens.com.siemens.com> can you guys send me a paper copy... thanks. Steve From amnon at ai.mit.edu Wed Oct 3 10:14:14 1990 From: amnon at ai.mit.edu (Amnon Shaashua) Date: Wed, 3 Oct 90 10:14:14 EDT Subject: MLP classifiers == Bayes In-Reply-To: Steve Hanson's message of Wed, 3 Oct 90 09:33:34 EDT <9010031333.AA18368@learning.siemens.com.siemens.com> Message-ID: <9010031414.AA28379@wheat-chex> didn't you get my e-mail yesterday night? I suggested there that instead of using E=(e2,e3,...,e_n,e_1) which brings the set of edges J={(1,2),(2,3),...,(n-1,n),(n,1)} to the main diagonal to use another set J' that corresponds to another predetrmined tour (the choice of J was arbitrary) find J' so that its corresponding E' has distinct roots. For instance if J'={(1,5),(5,3),(3,2),(2,4),(4,1)} then E'=(e_5,e_3,e_2,e_4,e_1) what are the roots of V=(E^t+E)/2 ? -Amnon From koch%CITIAGO.BITNET at VMA.CC.CMU.EDU Wed Oct 3 02:46:51 1990 From: koch%CITIAGO.BITNET at VMA.CC.CMU.EDU (Christof Koch) Date: Tue, 2 Oct 90 23:46:51 PDT Subject: Large scale biologically realistic simulations. In-Reply-To: Your message <9010020724.AA22487@nada.kth.se> dated 2-Oct-1990 Message-ID: <901002234651.204004af@Iago.Caltech.Edu> Have a look at Chapter 12 of the "Methods in Neuronal Modeling" book (C. Koch and I. Segev, eds., MIT Press, 1989). It's entitled "Simulating neurons and networks on parallel computers" by Nelson, Furmanski and Bower. Christof Koch From tsejnowski at ucsd.edu Wed Oct 3 23:19:01 1990 From: tsejnowski at ucsd.edu (Terry Sejnowski) Date: Wed, 3 Oct 90 20:19:01 PDT Subject: Large scale biologically realistic simulations. Message-ID: <9010040319.AA26955@sdbio2.UCSD.EDU> I visited the Royal Institute of Technology in Stockholm recently and saw a demonstration of the lamprey spinal cord model running on the 8-K CM-2 there. The model was developed in collaboration with Sten Grillner at the Karoliska Institute and is one of the best existing models of coupled nonlinear central pattern generators based on realistic models of neurons. The CM-2 demo was impressive because they have solved the problems of mapping a highly nonhomogenious network model with voltage dependent channels onto a bit sliced SIMD machine. The simulation I saw had 100 spinal segments merrily swimming along at about 10% real time. One of the realities of running the CM, however, is that the time required to load a big problem into memory is often much longer than it takes to run the problem. One strategy is to load 100 copies of the same network and run 100 different sets of parameters in parallel. Terry ----- From jls at computer-science.manchester.ac.uk Thu Oct 4 09:46:09 1990 From: jls at computer-science.manchester.ac.uk (jls@computer-science.manchester.ac.uk) Date: Thu, 4 Oct 90 09:46:09 BST Subject: Job announcement - postdoctoral research associate Message-ID: <20050.9010040846@m1.cs.man.ac.uk> POST DOCTORAL RESEARCH ASSOCIATE NEURAL NETWORK THEORY Department of Computer Science University of Manchester Applications are invited for a post-doctoral research position in neural network theory in the computer science department at the University of Manchester. This SERC funded position is concerned with developing methods of predicting neural network performance, including generalisation performance and functionality. The post is tenable for up to three years, starting any time between October 1, 1990 and April 1, 1991. Salary range is between 11,399 pounds and 18,165 pounds according to qualifications and experience. Postgraduate experience in any theoretical aspect of neural networks is desirable, as is demonstrated mathematical ability. Applications and/or informal inquiries should be sent to Jonathan Shapiro, Department of Computer Science, The University, Manchester M13 9PL. United Kingdom. Informal inquiries can be made by phone (061 275 6253) or email (jls at uk.ac.man.cs.m1 within the U.K. and jls%m1.cs.man.ac.uk at nsfnet-relay via internet). The University of Manchester is the oldest and one of the largest computer science department in Britain. Other neural network research at Manchester includes: ex-Static - a project to build a simulator for the design of massively parallel neural computing hardware (in collaboration with a group at Cambridge); a collaboration between theoretical physicists and psychologists to model memory experiments; work on image processing applications in medical biophysics; and secondary protein structure prediction. In addition, Manchester has been established as one of three national Novel Architecture Computing Centers, which means that a large collection of parallel hardware is or will be available. We are an Equal Opportunities Employer From watrous at demon.siemens.com Thu Oct 4 16:20:15 1990 From: watrous at demon.siemens.com (Raymond L Watrous) Date: Thu, 4 Oct 90 16:20:15 -0400 Subject: GRADSIM v2.0 Message-ID: <9010042020.AA17655@demon.siemens.com> ********************* PLEASE DO NOT FORWARD *************************** ********************* PLEASE DO NOT FORWARD *************************** An updated version of the GRADSIM connectionist network simulator is now available. (GRADSIM is general purpose simulator written in C that supports recurrent time-delay network optimization.) The updated simulator (version 2.0) supports second-order links, zero delay links, static pattern matching problems, and mixed unit types. The updated simulator also includes a conjugate gradient optimization module, and supports network link masking, for mixing fixed and variable links. A brief User's Guide that describes simulator modules and compilation options and the original tech report are included, in both .dvi and .ps form. The simulator is available via anonymous ftp from: linc.cis.upenn.edu as /pub/gradsim.v2.tar.Z (This file is about 300K bytes) ********************* PLEASE DO NOT FORWARD *************************** ********************* PLEASE DO NOT FORWARD *************************** From strom at asi.com Fri Oct 5 12:49:42 1990 From: strom at asi.com (Dan Hammerstrom) Date: Fri, 5 Oct 90 09:49:42 PDT Subject: Large scale biologically realistic simulations Message-ID: <9010051649.AA28477@asi.com> Dear Dr. Hammarlund: We noticed your recent posting on the Connectionists' network mailing list. We feel we may have some data that would interest you. We are currently developing a neurocomputer system designed for the high computational demands of neural network simulations. Our system is based around custom VLSI circuits organized in a SIMD (Single Instruction, Multiple Data) architecture. A single board has a peak capability of 12.8 billion multiply accumulate operations per second. In addition to developing a variety of neural network algorithms for this machine, we are also developing, under contract from the Office of Naval Research and in conjunction with the Oregon Graduate Institute and the Center for Learning and Memory at the University of California at Irvine, a real time simulation of a 10,000 neuron slice of olfactory pyriform cortex. The model we are using is that of Gary Lynch and Rick Granger and thier colleagues at UCI. Although it is much more abstract, and less realistic than the models you are working on, it still has a number of interesting properties including the ability to form hierarchical categorizations of the input vector space in an unsupervised mode. In our implementation, a linear array of piriform layer II neurons is mapped to a linear of array of processors. Up to 512 neurons can execute simultaneously in the neurocomputer; this ``slice'' of the network would constitute a single stage of a processor pipeline. Selectively piping the outputs of one slice as inputs to the next would emulate the feed--forward character of the Lateral Olfactory Tract, or LOT. We estimate that a piriform network consisting of approximately 10,000 pyramidal cells and a 512--element LOT could process 500 million eight--bit connections per second. At 10% sparse LOT connectivity, the entire system has roughly 750,000 synapses when lateral inhibitory connections are included. This performance is equivalent to roughly 1000 LOT presentations per second, or 200 presentations per second when the network is in learning mode. (Which is significantly faster than real time, and allows for some interesting research.) Researchers at the Oregon Graduate Institute are currently investigating the applications of the piriform model executing on such speech--processing applications. Please refer to the following for specific information concerning our network architecture and the piriform model (forgive the Latex format): @ARTICLE{piriform, AUTHOR = "Gary Lynch and Richard Granger and J{\'o}se Ambros-Ingerson", TITLE = "Derivation of encoding characteristics of layer 2 cerebral cortex", JOURNAL = "Journal of Cognitive Neuroscience", YEAR = {1989}, VOLUME = {1}, NUMBER = {1}, PAGES = {61--87} } @INCOLLECTION{memorial, AUTHOR = "Richard Granger and J{\'o}se Ambros--Ingerson and Ursula Staubli and Gary Lynch", TITLE = "Memorial Operation of Multiple, Interacting Simulated Brain Structures", BOOKTITLE = "Neuroscience and Connectionist Models", PUBLISHER = {Erlbaum Associates}, YEAR = {1989}, EDITOR = "M. Gluck and D. Rumelhart" } @INPROCEEDINGS{AdaptiveIJCNN, AUTHOR = "Dan Hammerstrom", TITLE = "A VLSI architecture for high-performance, low-cost, on-chip learning", BOOKTITLE = "Proceedings of the 1990 International Joint Conference on Neural Networks", YEAR = {1990}, MONTH = {June} } Sincerely, Eric Means, Adaptive Solutions Dan Hammerstrom, Adaptive Solutions / Oregon Graduate Institute Todd Leen, Oregon Graduate Institute From watrous at demon.siemens.com Mon Oct 8 15:08:56 1990 From: watrous at demon.siemens.com (Raymond L Watrous) Date: Mon, 8 Oct 90 15:08:56 -0400 Subject: GRADSIM v2.0 CORRECTION Message-ID: <9010081908.AA25249@demon.siemens.com> A version control error (mine) resulted in several lines of code being omitted from the recently released GRADSIM 2.0 simulator. The archive at linc.cis.upenn.edu has been updated with a corrected version. The corrected file alone is available as gradsim.patch.Z. Ray Watrous From rjwood at maxine.WPI.EDU Mon Oct 8 16:35:37 1990 From: rjwood at maxine.WPI.EDU (Richard J Wood) Date: Mon, 8 Oct 90 15:35:37 EST Subject: References needed... Message-ID: <9010082035.AA24549@maxine> Does anybody know of work done in object recognition using orientation selective cells? Specifically, I am looking for work done off of R. Linsker's series of papers in 1986 in the Proc. Natl. Acad. Sci. USA (Refs. below). Any references or ideas would be greatly appreciated. Thanks in advance....please send replies to rjwood at maxine.wpi.edu Rick @article{linsker1, Author="R. Linsker", Title="From basic network principles to neural architecture: Emergence of spatial-opponent cells", Journal="Proceedings of the National Academy of Sciences USA", Volume=83, Pages={7508-7512}, Year=1986} @article{linsker2, Author="R. Linsker", Title="From basic network principles to neural architecture: Emergence of orientation-selective cells", Journal="Proceedings of the National Academy of Sciences USA", Volume=83, Pages={8390-8394}, Year=1986} @article{read13, Author="R. Linsker", Title="From basic network principles to neural architecture: Emergence of orientation columns", Journal="Proceedings of the National Academy of Sciences USA", Volume=83, Pages={8779-8783}, Year=1986} From granger at ICS.UCI.EDU Mon Oct 8 15:42:10 1990 From: granger at ICS.UCI.EDU (granger@ICS.UCI.EDU) Date: Mon, 08 Oct 90 12:42:10 -0700 Subject: Large scale biologically realistic simulations In-Reply-To: Your message of Fri, 05 Oct 90 09:49:42 -0700. <9010051649.AA28477@asi.com> Message-ID: <1067.655414930@ics.uci.edu> Dan Hammerstrom's posting of 5 Oct 90 lists three of our recent publications dealing with computational analyses of the physiology and anatomy of the olfactory system. Allow me to add the (relatively recent) reference which contains our derivation of a novel hierarchical clustering mechanism from the combined operation of olfactory bulb and cortex, using a Hebb-like learning rule based on the physiology of synaptic long-term potentiation (LTP) in olfactory cortex: Ambros-Ingerson, J., Granger, R., and Lynch, G. (1990). Simulation of paleocortex performs hierarchical clustering. Science, 247: 1344-1348. It should be noted that none of these articles directly addresses issues of silicon implementation; rather, they provide the computational formalisms used by Hammerstrom et al. to design their VLSI circuits. However, the above Science article does describe the time and space complexity of parallel implementations of the derived network. - Richard Granger, Jose Ambros-Ingerson, Gary Lynch Center for the Neurobiology of Learning and Memory Bonney Center University of California Irvine, California 92717 From cleeremans at TANSY.PSY.CMU.EDU Tue Oct 9 10:54:31 1990 From: cleeremans at TANSY.PSY.CMU.EDU (Axel Cleeremans (Axel Cleeremans)) Date: Tuesday, 09 Oct 90 10:54:31 EDT Subject: Job openings Message-ID: The following is a job annoucement that may be of interest to the connectionist community. Please do not reply directly to this message : I am not affiliated in any way with any of the companies mentioned below. RESEARCH IN SPEECH PROCESSING - BRUSSELS (BELGIUM) Lernout & Hauspie Speech Products, one of the fastest growing companies in Speech Processing, has research and development openings (in the USA and in Europe) for several M.S.'s or Ph.D.'s in Computer Science, Electrical Engineering, or any other AI-related field. We are especially looking for specialists in Digital Signal Processing, Artificial Neural Networks, Microprocessors (Motorola 56000) and Computer Linguistics. Excellent research facilities in a stimulating environment are guaranteed. Please send your resume to : Jan Vandenhende Brains Trust International Boulevard Brand Witlock, 24 B-1200 Brussels Belgium Phone : 011 32 2 735 81 40 Fax : 011 32 2 735 20 75 From birnbaum at fido.ils.nwu.edu Tue Oct 9 14:43:52 1990 From: birnbaum at fido.ils.nwu.edu (Lawrence Birnbaum) Date: Tue, 9 Oct 90 13:43:52 CDT Subject: ML91 deadline extended Message-ID: <9010091843.AA03646@fido.ils.nwu.edu> ML91 -- THE EIGHTH INTERNATIONAL WORKSHOP ON MACHINE LEARNING DEADLINE FOR WORKSHOP PROPOSALS EXTENDED To make life a little easier, the deadline for workshop proposals for ML91 has been extended by a few days. The new deadline is MONDAY, OCTOBER 15. Please send proposals by email to: ml91 at ils.nwu.edu or by hardcopy to the following address: ML91 Northwestern University The Institute for the Learning Sciences 1890 Maple Avenue Evanston, IL 60201 USA fax (708) 491-5258 Please include the following information: 1. Workshop topic 2. Names, addresses, and positions of workshop committee members 3. Brief description of topic 4. Workshop format 5. Justification for workshop, including assessment of breadth of appeal On behalf of the organizing committee, Larry Birnbaum Gregg Collins Program co-chairs, ML91 From pollack at cis.ohio-state.edu Tue Oct 9 16:21:34 1990 From: pollack at cis.ohio-state.edu (Jordan B Pollack) Date: Tue, 9 Oct 90 16:21:34 -0400 Subject: Position Announcement Message-ID: <9010092021.AA09069@dendrite.cis.ohio-state.edu> Computational Neuroscience The Ohio State University For one of several open tenure-track faculty positions in Cognitive Science, the Ohio State University is seeking a computational neuroscientist. The successful candidate will hold a recent Ph.D and have a demonstrated record of research accomplishment in biologically realistic computational modelling of nervous systems. The Center for Cognitive Science is an interdisciplinary University- wide research center with approximately eighty members from sixteen departments. The faculty appointment will be in an academic department that is appropriate to the interests and background of the candidate. OSU is one of the largest universities in the country with significant research resources including a Cray YMP and a PET scanner. OSU's Ph.D. Program in Neuroscience involves some 75 faculty members from over a half dozen colleges. Columbus provides a high quality of life along with very affordable housing. To apply, please send your vita and a statement of research and teaching interests, and arrange for three recommendation letters to be sent to: Computational Neuroscience Search Committee Center for Cognitive Science 208 Ohio Stadium East 1961 Tuttle Park Place Columbus, OH 43210-1102 The Ohio State University encourages diversity in its faculty, and is an Equal Opportunity/Affirmative Action employer. From MURTAGH at SCIVAX.STSCI.EDU Thu Oct 11 10:30:44 1990 From: MURTAGH at SCIVAX.STSCI.EDU (MURTAGH@SCIVAX.STSCI.EDU) Date: Thu, 11 Oct 1990 10:30:44 EDT Subject: Workshop announcement: "NNs for Statistical & Economic Data" Message-ID: <901011103044.26014ad8@SCIVAX.STSCI.EDU> Workshop: "Neural Networks for Statistical and Economic Data" Place/date: Dublin, Ireland. December 10-11 1990. The workshop seeks to bring together those working towards applications of artificial neural networks, and those concerned with regularities, in statistical and ecomomic data. A number of invited speakers will also review closely-related domains such as nonlinear time series analysis, and complex questions in economics and economic statistics. On December 10 there will be a series of tutorial and review presentations. On December 11 there will be both invited and contributed working papers. Attendance at the workshop is limited, priority being given to those presenting new results. Sponsor: EUROSTAT/Statistical Office of the European Communities, Luxembourg. Organization: Munotec Systems Ltd., 35 St. Helen's Road, Booterstown, Co. Dublin, Ireland. Information can also be requested from: F. Murtagh, murtagh at dgaeso51.bitnet, or murtagh at scivax.stsci.edu. From reggia at cs.UMD.EDU Thu Oct 11 14:53:53 1990 From: reggia at cs.UMD.EDU (James A. Reggia) Date: Thu, 11 Oct 90 14:53:53 -0400 Subject: Call for papers: neural nets for diagnosis Message-ID: <9010111853.AA13403@brillig.cs.UMD.EDU> [[Note: I was asked to bring the following to the attention of anyone using neural modelling methods for diagnostic problem solving.]] *CALL FOR PAPERS* Second International Workshop on Principles of Diagnosis Milano (Italy), October 14-15-16, 1991 Organized by CISE Tecnologie Innovative and Dipartimento di Informatica of Universita` di Torino This workshop (which follows the successful one held at Stanford University in 1990) encourages intensive and high quality interaction and cooperation among researchers with a diversity of artificial intelligence approaches to diagnosis. Attendance will be limited to fifty participants with presentations spread over three days. Substantial time will be reserved for discussion. To attend, participants should submit papers (maximum 5000 words) to be reviewed by the committee. Submissions are welcomed on (but not limited to) the following topics: - Theory of diagnosis (abductive vs. deductive diagnosis, isolation vs. identification, diagnosis on non-monotonic theories, diagnosis of dynamic systems,...) - Computational issues (controlling the combinatorial explosion, focusing strategies, controlling diagnostic reasoning of complex systems, ...) - Modeling for diagnosis (multiple, approximate, probabilistic and qualitative models, integrating model-based diagnosis with heuristics ....) - Evaluation of theories on practical applications - Inductive approaches to diagnosis (Case-Based Reasoning, Neural Nets, ...) Accepted papers can be revised for inclusion in the workshop working notes. Although work published elsewhere is acceptable, new original work is preferred. Please send five copies of each submission to the chairman at the postal address below. Include several ways of contacting the principal author in addition to a postal address: electronic mail, fax and telephone numbers are preferred, in that order. Please indicate with your submission if you wish to make a presentation or only to attend. Submissions received after 3 May 1991 will not be considered. The decisions of the committee will be mailed by 1 July 1991. Chairman: Luca Console Dipartimento di Informatica - Universit` di Torino Corso Svizzera 185, 10149 Torino (Italy) E-mail: lconsole at itoinfo.bitnet Fax: (+39) 11 751603 Tel.: (+39) 11 771 2002 Committee: I. Bratko (U. Ljubljana), P. Dague (IBM), J. de Kleer (Xerox), G. Guida (U. Brescia), K. Eshghi (HP), W. Hamscher (Price Waterhouse), M. Kramer (MIT), W. Nejdl (U. Wien), J. Pearl (UCLA), D. Poole (U. British Columbia), O. Raiman (Xerox), J. Reggia (U. Maryland), J. Sticklen (Michigan State U.), P. Struss (Siemens), P. Szolovits (MIT), G. Tornielli (CISE). Organizing Committee: M. Migliavacca (CISE, chairman), M. Gallanti (CISE), A. Giordana (U. Torino), L. Lesmo (U. Torino). Secretarial Support: A. Camnasio, CISE, P.O. Box 12081, 20134 Milano, Tel (+39) 2 21672400, Fax (+39) 2 26920587. This workshop is sponsored by AI*IA. Sponsorship required to AAAI and ECCAI From jordan at psyche.mit.edu Thu Oct 11 15:24:11 1990 From: jordan at psyche.mit.edu (Michael Jordan) Date: Thu, 11 Oct 1990 15:24:11 EDT Subject: technical report Message-ID: The following technical report is available: Forward Models: Supervised Learning with a Distal Teacher Michael I. Jordan Massachusetts Institute of Technology David E. Rumelhart Stanford University MIT Center for Cognitive Science Occasional Paper #40 Abstract Internal models of the environment have an important role to play in adaptive systems in general and are of particular importance for the supervised learning paradigm. In this paper we demonstrate that certain classical problems associated with the notion of the ``teacher'' in supervised learning can be solved by judicious use of learned internal models as components of the adaptive system. In particular, we show how supervised learning algorithms can be utilized in cases in which an unknown dynamical system intervenes between actions and desired outcomes. Our approach applies to any supervised learning algorithm that is capable of learning in multi- layer networks. Copies can be obtained in one of two ways: (1) ftp a postscript copy from cheops.cis.ohio-state.edu. The file is jordan.forward-models.Z in the pub/neuroprose directory. You can either use the Getps script or follow these steps: unix:1> ftp cheops.cis.ohio-state.edu Connected to cheops.cis.ohio-state.edu. Name (cheops.cis.ohio-state.edu:): anonymous 331 Guest login ok, send ident as password. Password: neuron 230 Guest login ok, access restrictions apply. ftp> cd pub/neuroprose ftp> binary ftp> get jordan.forward-models.ps.Z ftp> quit unix:2> uncompress jordan.forward-models.ps.Z unix:3> lpr jordan.forward-models.ps (2) Order a hardcopy from bonsaint at psyche.mit.edu or hershey at psych.stanford.edu. (use a nearest-geographic-neighbor rule). Please use this option only if option (1) is not feasible. Mention the "Forward Models" technical report. --Mike Jordan From carol at ai.toronto.edu Fri Oct 12 07:22:50 1990 From: carol at ai.toronto.edu (Carol Plathan) Date: Fri, 12 Oct 1990 16:22:50 +0500 Subject: CRG-TR-90-5 request Message-ID: <90Oct12.162252edt.292@neuron.ai.toronto.edu> PLEASE DO NOT FORWARD TO OTHER NEWSGROUPS OF MAILING LISTS ********************************************************** The following technical report is now available. You can get it from carol at ai.toronto.edu. Send your real mail address (omitting all other information from your message). --------------------------------------------------------------------------- COMPETING EXPERTS: AN EXPERIMENTAL INVESTIGATION OF ASSOCIATIVE MIXTURE MODELS Steven J. Nowlan Department of Computer Science University of Toronto Toronto, Canada M5S 1A4 CRG-TR-90-5 Supervised algorithms, such as back-propagation, have proven capable of discovering clever internal representations of the information necessary for performing some task, while ignoring irrelevant detail in the input. However, such supervised algorithms suffer from problems of scale and interference between tasks when used to perform more than one task, or a complex task which is a disjunction of many simple subtasks. To address these problems, several authors have proposed modular systems consisting of multiple networks (Hampshire & Waibel 1989, Jacobs, Jordan & Barto 1990, Jacobs, Jordan, Nowlan, & Hinton, 1990). In this paper, we discuss experimental investigations of the model introduced by Jacobs, Jordan, Nowlan & Hinton, in which a number of simple expert networks compete to solve distinct pieces of a large task; each expert has the power of a supervised algorithm to allow it to discover clever task specific internal representations, while an unsupervised competitive mechanism decomposes the task into easily computable subtasks. The competitive mechanism is based on the mixture view of competition discussed in (Nowlan 1990, Nowlan & Hinton), and the entire system may be viewed as an associative extension of a model consisting of a mixture of simple probability generators. The task decomposition and training of individual experts are performed in parallel, leading to an interesting non-linear interaction between these two processes. Experiments on a number of simple tasks illustrate resistance to task interference, the ability to discover the ``appropriate'' number of subtasks, and good parallel scaling performance. The system of competing experts is also compared with an alternate formulation, suggested by the work of (Jacobs, Jordan, and Barto 1990), which allows cooperation rather than competition between a number of simple expert networks. Results are also described for a phoneme discrimination task, which reveals an ability for a system of competing experts to uncover interesting subtask structure in a complex task. References: J. Hampshire and A. Waibel, "The Meta-Pi network: Building distributed knowledge representations for robust pattern recognition." Technical Report CMU-CS-89-166, School of Computer Science, Carnegie Mellon University, 1989. R. A. Jacobs, M. I. Jordan, and A. G. Barto, "Task decomposition through competition in a modular connectionist architecture." Cognitive Science, 1990. In Press. R. A. Jacobs, M. I. Jordan, S. J. Nowlan and G. E. Hinton, "Adaptive mixtures of local experts" Neural Computation, 1990. In Press. S. J. Nowlan, 1990. "Maximum Likelihood Competitive Learning" in Neural Information Processing Systems 2, D. Touretzky (ed.), Morgan Kauffmann, 1990. S. J. Nowlan and G. E. Hinton, "The bootstrap Widrow-Hoff rule as a cluster formation algorithm" Neural Computation 2:3, 1990. ------------------------------------------------------------------------------ From lss at compsci.stirling.ac.uk Wed Oct 17 14:49:58 1990 From: lss at compsci.stirling.ac.uk (Dr L S Smith (Staff)) Date: 17 Oct 90 14:49:58 BST (Wed) Subject: No subject Message-ID: <9010171449.AA15213@uk.ac.stir.cs.tugrik> COGNITIVE SCIENCE/HCI INITIATIVE Department of Psychology University of St Andrews Scotland, UK and Centre for Cognitive and Computational Neuroscience University of Stirling Scotland, UK 2 POST-DOCTORAL RESEARCH FELLOWSHIPS investigating psychological and neurocomputational processes of visual word recognition. This project represents a major collaboration between the Department of Psychology, St Andrews (a leading centre for human perceptual research) and the CCCN, Stirling (a leading centre for neural network research) to develop a new computational model of visual word recognition. Applications are invited for the following 2 post-doctoral fellowships within the project: Post 1 (tenured for 3 years, based at Department of Psychology, St Andrews University) will involve developing fresh perspectives on the neural modelling of visual word recognition from human experimentation. The data from these experiments will form the basis for the computational modelling in the project. Applicants should have experience in human experimentation in cognitive science or perceptual research, be well acquainted with the use of computers in experimentation, and have some knowledge of neural network research. Post 2 (tenured for 2 years, based at Centre for Cognitive and Computational Neuroscience, Stirling University) will involve setting up and developing a new computational model of visual word recognition which combines the findings from St Andrews with fresh perspectives on neurocomputational processing. Applicants should have experience or interest in neural computation/connectionism and have a background in one or more of the following: computing science, psychology, mathematics, physics. Starting salary for each post will be on the 1A scale for research staff (up to \pounds 18165 pa). Both posts are scheduled to start as soon as possible in 1991. Application forms and further particulars for both posts can be obtained from The Director of Personnel Services, College Gate, St Andrews University, St Andrews, Fife, KY16 9AJ, to whom completed applications forms together with a CV should be submitted to arrive no later than November 30th 1990. Further information can be obtained informally from: (Post 1) Dr Tim Jordan at St Andrews (tel.0334 76161, ext 7234) (Post 2) Dr Leslie Smith at Stirling (tel.0786 67435, direct line) Previous applicants for these posts need not re-apply. Both Universities operate an Equal Opportunities Policy. From sontag at hilbert.rutgers.edu Thu Oct 18 15:43:19 1990 From: sontag at hilbert.rutgers.edu (Eduardo Sontag) Date: Thu, 18 Oct 90 15:43:19 EDT Subject: WORKSHOP ON THEORETICAL ISSUES IN NEURAL NETS, May 20-23, 1991 Message-ID: <9010181943.AA01637@hilbert.rutgers.edu> WORKSHOP ON THEORETICAL ISSUES IN NEURAL NETS Announcement and Call for Contributions The Center for Discrete Mathematics and Theoretical Computer Science (DIMACS) will host a workshop on "Theoretical Issues in Neural Nets" at Rutgers University, for four days, May 20-23, 1991. This will be a mathematically oriented meeting, where technical issues can be discussed in depth. The objective is to have a Workshop that brings together people interested in a serious study of foundations -- plus a few people who will give expository lectures on applied problems and biological nets. The area is of course very diverse, and precisely because of this it might be worth trying to search for conceptual unity in the context of the Workshop. A preliminary list of main speakers is as follows (with tentative topics listed, when available): Dave Ackley, Bellcore (Genetic algorithms: Evolution and learning) Andrew Barron, U. Illinois (Statistical selection of neural net architectures) Andy Barto, U. Mass. (Expository talk: Learning & incrmntl dynamic programming) Eric Baum, NEC Institute (Expository talk: Sample complexity) Ed Blum, USC (Feed-forward networks and approximation in various norms) Roger Brockett, Harvard (Combinatorial optimization via steepest descent) George Cybenko, U. Illinois Merrick Furst, CMU (Circuit complexity & harmonic analysis of Boolean functs) Herbert Gish, BBN (Maximum likelihood training of neural networks) Stephen Grossberg, Boston U. (Expository talk) Steve Hanson, Siemens (Expository talk: Human learning and categorization) Moe Hirsch, Berkeley (Expository talk: Network dynamics) Wolfgang Maass, U. Ill./Chicago (Boltzmann machines for classification) John Moody, Yale Sara Solla, Bell Labs (Supervised learning and statistical physics) Santosh S. Venkatesh, Penn Hal White, UCSD The organizing committee consists of Bradley W. Dickinson (Princeton), Gary M. Kuhn (Institute for Defense Analyses), and Eduardo D. Sontag and Hector J. Sussmann (Rutgers). DIMACS is a National Science Foundation Science and Technology Center, established as a cooperative project between Rutgers University, Princeton University, AT&T Bell Laboratories, and Bellcore. Its objectives are to carry out basic and applied research in discrete mathematics and theoretical computer science. The center provides excellent facilities for workshop participants, including offices and computer support. If you are interested in participating in this workshop, please send a message to Eduardo at sontag at hilbert.rutgers.edu. If you would like to give a talk, please e-mail a title and abstract to the above address by January 15th, 1991. Please keep the abstract short, but give references to published work if appropriate. (Use plain TeX, LaTeX, or a text file; please do not use snailmail.) There is a possibility of proceedings being published, but nothing has been decided in that regard. If you are interested in attending but not talking, send a note explaining your interest in the area. The committee will try to accommodate as many participants and as many talks as possible, but the numbers may have to be limited in order to achieve a relaxed workshop atmosphere conducive to interactions among participants. Notification of people concerning attendance is expected about the middle of February. From codelab at psych.purdue.edu Thu Oct 18 16:58:56 1990 From: codelab at psych.purdue.edu (Coding Lab Wasserman) Date: Thu, 18 Oct 90 15:58:56 EST Subject: room Message-ID: <9010182058.AA15575@psych.purdue.edu> I am looking for a place to stay during the Society for Neuroscience meeting in St. Louis. I am a graduate student on a limited travel budget and I wish to share any sort of inexpensive accommodations with someone else. I need a place from Sunday, October 28th to Wednesday, October 31st. Please reply to: Zixi Cheng codelab at psych.purdue.edu From rob at galab2.mh.ua.edu Fri Oct 19 23:43:54 1990 From: rob at galab2.mh.ua.edu (Robert Elliott Smith) Date: Fri, 19 Oct 90 22:43:54 CDT Subject: text availability? Message-ID: <9010200343.AA08001@galab2.mh.ua.edu> Hi, I have acquired the responsibility of teaching a introductory neural nets course to an audience of graduate students from a variety of engineering disciplines. This course was previously taught from short-course notes with the PDP books as additional material. I would prefer to teach this course from a text. Several publishers have told me that they will have such books soon4{, but the currently available texts I've seen aren't very satisfactory. Does anyone know of a good text that is available (or will be before spring)? Please respond to me directly. All comments appreciated. Rob. ------------------------------------------- Robert Elliott Smith Department of Engineering of Mechanics The University of Alabama P. O. Box 870278 Tuscaloosa, Alabama 35487 <> rob at galab2.mh.ua.edu <> (205) 348-4661 ------------------------------------------- From ngr at cs.exeter.ac.uk Thu Oct 18 12:26:59 1990 From: ngr at cs.exeter.ac.uk (Niall Griffith) Date: Thu, 18 Oct 90 17:26:59 +0100 Subject: MUSIC Message-ID: <3121.9010181626@expya.cs.exeter.ac.uk> I am working at the Connection Science lab at Exeter, and I am writing a review of connectionist research on music. It would be really useful if you could send me as many references as you have on the subject. I will of course make these publicly available. Niall Griffith Centre for Connection Science JANET: ngr at uk.ac.exeter.cs Dept. Computer Science University of Exeter UUCP: ngr at expya.uucp Exeter EX4 4PT DEVON BITNET: ngr at cs.exeter.ac.uk@UKACRL UK Niall Griffith Centre for Connection Science JANET: ngr at uk.ac.exeter.cs Dept. Computer Science University of Exeter UUCP: ngr at expya.uucp Exeter EX4 4PT DEVON BITNET: ngr at cs.exeter.ac.uk@UKACRL UK From gluck%psych at Forsythe.Stanford.EDU Sat Oct 20 14:27:57 1990 From: gluck%psych at Forsythe.Stanford.EDU (Mark Gluck) Date: Sat, 20 Oct 90 11:27:57 PDT Subject: Two Preprints: Generalization & Representation, Sensorimotor Learning Message-ID: <9010201827.AA28023@psych> TWO PRE-PRINTS AVAILABLE: 1) Stimulus Generalization and Representation in Adaptive Network Models of Category Learning 2) Sensorimotor Learning and the Cerebellum. _________________________________________________________________ Gluck, M. A. (1991, in press). Stimulus generalization and representation in adaptive network models of category learning To appear in : Psychological Science. Abstract An exponential-decay relationship relationship between the proba- bility of generalization and psychological distance has received considerable support from studies of stimulus generalization (Shepard, 1958) and categorization (Nosofsky, 1984). It is shown here how an approximate exponential generalization gradient em- erges in a "configural-cue" network model of human learning that represents stimulus patterns in terms of elementary features and pair-wise conjunctions of features (Gluck & Bower, 1988b; Gluck, Bower, & Hee, 1989) from stimulus representation assumptions iso- morphic to a special case of Shepard's (1987) theory of stimulus generalization. The network model can be viewed as a combination of Shepard's theory and an associative learning rule derived from Rescorla and Wagner's (1972) theory of classical conditioning. _________________________________________________________________ Bartha, G. T., Thompson, R. F., & Gluck, M. A. (1991, in press) Sensorimotor learning and the cerebellum. In M. A. Arbib and J.-P. Ewert (Eds.), Visual Structures and Integrated Functions, Springer Research Notes in Neural Computing, Berlin: Springer-Verlag. Abstract This paper describes our current work on integrating experimental and theoretical studies of a simple form of sensorimotor learn- ing: the classically conditioned rabbit eyelid closure response. We first review experimental efforts to determine the neural basis of the conditioned eyelid closure response and these sup- port the role of the cerebellum as the site of the memory trace. Then our current work to bring the modeling in closer contact with the biology is described. In particular, we extend our ear- lier model of response topography to be more physiological in the circuit connectivity, the learning algorithm, and the conditioned stimulus representation. The results of these extensions include a more realistic conditioned response topography and reinforce- ment learning which accounts for an experimentally established negative feedback loop. _________________________________________________________________ To request copies, send email to: gluck at psych.stanford.edu with your hard-copy mailing address. Or mail to: Mark A. Gluck, Department of Psychology, Jordan Hall, Bldg. 420, Stanford Univ., Stanford, CA 94305-2130 From jenkins%SIPI.USC.EDU at VMA.CC.CMU.EDU Fri Oct 19 16:38:44 1990 From: jenkins%SIPI.USC.EDU at VMA.CC.CMU.EDU (Keith Jenkins) Date: Fri, 19 Oct 1990 13:38:44 PDT Subject: OC91 announcement on the other newsletter Message-ID: MEETING ANNOUNCEMENT FOURTH TOPICAL MEETING ON OPTICAL COMPUTING Salt Lake City Marriott Salt Lake City, Utah March 4-6, 1991 Optical Society of America ABSTRACT AND SUMMARY DEADLINE: NOVEMBER 14, 1990 -- -- -- -- -- -- -- SCOPE The Optical Computing topical meeting will consist of invited papers and both oral and poster contributed papers. Contributions are solicited in all areas of research in materials, devices, architectures and algorithms relevant to optical computing. Topics of interest include: 1. Optical interconnections and buses for computing. 2. Optical and photonic computing systems, architectures, models and algorithms, digital or analog. 3. Hybrid optical/electronic processors. 4. Optical "neural" processors, including optical associative memories. 5. Optical memory. 6. Massively parallel architectures for optical implementation. 7. Spatial light modulators and other devices for optical computing. 8. Nonlinear optical phenomena of potential use in computing. 9. Nonlinear optical, electro-optical, and opto-electronic components. 10. Areas of application of the above components and processors. 11. Fundamental physical and computational properties relating to the capabilities and limitations of optical computers. -- -- -- -- -- -- -- -- -- -- - CONFERENCE COCHAIRS: C. Lee Giles NEC Research Institute Sing H. Lee University of California, San Diego PROGRAM CHAIR: B. Keith Jenkins University of Southern California -- -- -- -- -- -- -- -- -- -- - INVITED TALKS (Partial List) P. Bruce Berra Syracuse University "Optical Database Machines" K. Kasahara NEC Corporation "Progress in Arrays of Opto-electronic Bistable Devices and Sources" Bart Kosko University of Southern California "Adaptive Fuzzy Systems" Demetri Psaltis California Institute of Technology "Learning in Optical Neural Networks". Wilfrid B. Veldkamp MIT Lincoln Laboratory "Binary Optics and Applications" -- -- -- -- -- -- -- -- -- -- - RELATED MEETINGS Four topical meetings are being held in Salt Lake City during the two- week period of March 4-15, 1991. The meetings are: Optical Computing, Photonic Switching, Picosecond Electronics and Optoelectronics, and Quantum Wells for Optics and Optoelectronics. -- -- -- -- -- -- -- -- -- -- -- FOR MORE INFORMATION: Contact Optical Society of America Meetings Department 2010 Massachusetts Ave., NW Washington, DC 20036 USA Tel. (202) 223-0920 Fax (202) 416-6100 From amini at tcville.hac.com Sun Oct 21 22:46:03 1990 From: amini at tcville.hac.com (Afshin Amini) Date: Sun, 21 Oct 90 19:46:03 PDT Subject: job in signal processing and neural net Message-ID: <9010220246.AA05941@ai.spl> Hi there I received a job posting on the net, about two weeks ago. The job advertised opennings in signal processing, neural net and related AI fields. The main company was in Belgium. Unfortunately I did not save that message and do not have the name of the company nor the email. If anybody has saved the info, please email it to me. thanks, -aa -- ---------------------------------------------------------------------------- Afshin Amini Hughes Aircraft Co. voice: (213) 616-6558 Electro-Optical and Data Systems Group Signal Processing Lab fax: (213) 607-0918 P.O. Box 902, EO/E1/B108 email: El Segundo, CA 90245 smart: amini at tcville.hac.com Bldg. E1 Room b2316f dumb: amini%tcville at hac2arpa.hac.com uucp: hacgate!tcville!dave ---------------------------------------------------------------------------- From paul at axon.Colorado.EDU Tue Oct 23 12:04:58 1990 From: paul at axon.Colorado.EDU (Paul Smolensky) Date: Tue, 23 Oct 90 10:04:58 -0600 Subject: Connectionist Faculty Position at Boulder Message-ID: <9010231604.AA06346@axon.Colorado.EDU> The Institute for Cognitive Science at the University of Colorado, Boulder has an opening for which connectionists are invited to apply. As you can see from the official ad below, applications in another field are also being invited. However, should this year's position go to a non-connectionist, we expect another position next year and a search will be held specifically for a connectionist. We would be more than happy to answer any questions you may have... Paul Smolensky & Mike Mozer ------------------------------------- Faculty Position in Cognitive Science The Institute of Cognitive Science at the University of Colorado at Boulder invites applications for a tenured/tenure-track position, either in the area of connectionism or in the area of knowledge-based systems or cooperative problem solving. The position is open as to rank. An important selection criterion will be the candidate's potential to contribute to the Institute's interdisciplinary missions in research, teaching, and service. Candidates in the connectionist area should have demonstrated ability to contribute to connectionist theory as well as connectionist approaches to cognitive science. Candidates in the knowledge based systems or cooperative problem solving area should have an interest in large scale system building efforts and software technologies and tools. The position will be housed in an appropriate academic department associated with the Institute of Cognitive Science (e.g., Computer Science, Linguistics, Philosophy, or Psychology). A resume and three letters of reference should be sent to: Dr. Martha Polson, Assistant Director, Institute of Cognitive Science, University of Colorado, Boulder, Colorado, 80309-0345 by January 18, 1990. The University of Colorado at Boulder has a strong commitment to the principle of diversity in all areas. In that spirit, we are particularly interested in receiving applications from a broad spectrum of people, including women, members of ethnic minorities and disabled individuals. From pablo at cs.washington.edu Wed Oct 24 16:55:02 1990 From: pablo at cs.washington.edu (David Cohn) Date: Wed, 24 Oct 90 13:55:02 -0700 Subject: Shared Accommodations at NIPS Message-ID: <9010242055.AA28215@june.cs.washington.edu> I'm interested in matching up with other starving graduate students (or starving faculty, etc.) who are going to NIPS (Neural Information Processing Systems) in Denver at the end of November and would be interested in cutting costs by sharing a room. If you're interested, send e-mail with preferences (such as no smoking, quiet, late riser, etc.) to pablo at cs.washington.edu; I will assemble a list and send out a copy to everyone who writes to me. Hopefully, we can self-organize :-) and keep this as much out of the way of the connectionist mailing list as possible so that we limit noise (to people who *aren't* going to the conference. Thanks, -David "Pablo" Cohn e-mail: pablo at cs.washington.edu Dept. of Computer Science, FR-35 phone: (206) 543-7798 University of Washington Seattle, WA 98195 From jose at learning.siemens.com Thu Oct 25 08:38:10 1990 From: jose at learning.siemens.com (Steve Hanson) Date: Thu, 25 Oct 90 08:38:10 EDT Subject: CNS RFPs Message-ID: <9010251238.AA19808@learning.siemens.com.siemens.com> McDonnell-Pew Program in Cognitive Neuroscience October 1990 Individual Grants-in-Aid for Research and Training Supported jointly by the James S. McDonnell Foundation and The Pew Charitable Trusts INTRODUCTION The McDonnell-Pew Program in Cognitive Neuroscience has been created jointly by the James S. McDonnell Foundation and The Pew Charitable Trusts to promote the development of cognitive neuroscience. The foundations have allocated $12 million over an initial three-year period for this program. Cognitive neuroscience attempts to understand human mental events by specifying how neural tissue carries out computations. Work in cognitive neuroscience is interdisciplinary in character, drawing on developments in clinical and basic neuroscience, computer science, psychology, linguistics, and philosophy. Cognitive neuroscience excludes descriptions of psychological function that do not address the underlying brain mechanisms and neuroscientific descriptions that do not speak to psychological function. The program has three components. (1) Institutional grants have been awarded for the purpose of creating centers where cognitive scientists and neuroscientists can work together. (2) To encourage Ph.D. and M.D. investigators in cognitive neuroscience, small grants-in-aid will be awarded for individual research projects. (3) To encourage Ph.D. and M.D. investigators to acquire skills for interdisciplinary research, small training grants will be awarded. During the program's initial three-year period, approximately $4 million will be available for the latter two components -- individual grants-in-aid for research and training -- which this brochure describes. RESEARCH GRANTS The McDonnell-Pew Program in Cognitive Neuroscience will issue a limited number of awards to support collaborative work by cognitive neuroscientists. Applications are sought for projects of exceptional merit that are not currently fundable through other channels and from investigators who are not at institutions already funded by an institutional grant from the cognitive neuroscience program. Preference will be given to projects requiring collaboration or interaction between at least two subfields of cognitive neuroscience. The goals are to encourage broad national participation in the development of the field and to facilitate the participation of investigators outside the major centers of cognitive neuroscience. Submissions will be reviewed by the program's advisory board. Grant support under this component is limited to $30,000 per year for two years. Indirect costs are to be included in the $30,000 maximum and may not exceed 10 percent of salaries and fringe benefits. Grants are not renewable after two years. The program is looking for innovative proposals that would, for example: * combine experimental data from cognitive psychology and neuroscience; * explore the implications of neurobiological methods for the study of the higher cognitive processes; * bring formal modeling techniques to bear on cognition; * use sensing or imaging techniques to observe the brain during conscious activity; * make imaginative use of patient populations to analyze cognition; * develop new theories of the human mind/brain system. This list of examples is necessarily incomplete but should suggest the general kind of proposals desired. Ideally, a small grant-in-aid for research should facilitate the initial exploration of a novel or risky idea, with success leading to more extensive funding from other sources. TRAINING GRANTS A limited number of grants will also be awarded to support training investigators in cognitive neuroscience. Here again, the objective is to support proposals of exceptional merit that are underfunded or unlikely to be funded from other sources. Training grants to support Ph.D. thesis research of graduate students will not be funded. Some postdoctoral awards for exceptional young scientists will be available; postdoctoral stipends will be funded for up to three years at prevailing rates at the host institution. Highest priority will be given to candidates seeking postdoctoral training outside the field of their previous training. Innovative programs for training young scientists, or broadening the experience of senior scientists, are also encouraged. Some examples of appropriate proposals follow. * Collaboration between a junior scientist in a relevant discipline and a senior scientist in a different discipline has been suggested as an effective method for developing the field. * Two senior scientists might wish to learn each other's discipline through a collaborative project. * An applicant might wish to visit several laboratories in order to acquire new research techniques. * Senior researchers might wish to investigate new methods or technologies in their own fields that are unavailable at their home institutions. Here again, examples can only suggest the kind of training experience that might be considered appropriate. APPLICATIONS Applicants should submit five copies of a proposal that does not exceed 5,000 words. Proposals for research grants should include: * a description of the work to be done and where it might lead; * an account of the investigator's professional qualifications to do the work. Proposals for training grants should include: * a description of the training sought and its relationship to the applicant's work and previous training; * a statement from the mentor as well as the applicant concerning the acceptability of the training plan. Proposals for both research grants and training grants should include: * an account of any plans to collaborate with other cognitive neuroscientists; * a brief description of the available research facilities; The proposal must be accompanied by the following separate information: * a brief, itemized budget and budget justification for the proposed work, including direct and indirect costs (indirect costs may not exceed 10 percent of salaries and fringe benefits); * curriculum(a) vitae of the participating investigator(s); * evidence that the sponsoring organization is a nonprofit, tax-exempt institution; * an authorized form indicating clearance for the use of human and animal subjects; * an endorsement letter from the officer of the sponsoring institution who will be responsible for administering the grant. No other appended documents will be accepted for evaluation, and any incomplete applications will be returned to the applicant. The advisory board reviews proposals twice a year. Applications must be postmarked by the deadlines of February 1 and August 1 to be considered for review. INFORMATION For more information contact: McDonnell-Pew Program in Cognitive Neuroscience Green Hall 1-N-6 Princeton University Princeton, New Jersey 08544-1010 Telephone: 609-258-5014 Facsimile: 609-258-3031 Email: cns at confidence.princeton.edu ADVISORY BOARD Emilio Bizzi, M.D. Eugene McDermott Professor in the Brain Sciences and Human Behavior Chairman, Department of Brain and Cognitive Sciences Whitaker College Massachusetts Institute of Technology, E25-526 Cambridge, Massachusetts 02139 Sheila Blumstein, Ph.D. Professor of Cognitive and Linguistic Sciences Dean of the College Brown University University Hall, Room 218 Providence, Rhode Island 02912 Stephen J. Hanson, Ph.D. Group Leader Learning and Knowledge Acquisition Research Group Siemens Research Center 755 College Road East Princeton, New Jersey 08540 Jon Kaas, Ph.D. Centennial Professor Department of Psychology Vanderbilt University Nashville, Tennessee 37240 George A. Miller, Ph.D. James S. McDonnell Distinguished University Professor of Psychology Department of Psychology Princeton University Princeton, New Jersey 08544-1010 Mortimer Mishkin, Ph.D. Laboratory of Neuropsychology National Institute of Mental Health 9000 Rockville Pike Building 9, Room 1N107 Bethesda, Maryland 20892 Marcus Raichle, M.D. Professor of Neurology and Radiology Division of Radiation Sciences Mallinckrodt Institute of Radiology at Washington University Medical Center 510 S. Kingshighway Blvd., Campus Box 8131 St. Louis, Missouri 63110 Endel Tulving, Ph.D. Department of Psychology University of Toronto Toronto, Ontario M5S 1A1 Canada From rob at galab2.mh.ua.edu Thu Oct 25 17:43:20 1990 From: rob at galab2.mh.ua.edu (Robert Elliott Smith) Date: Thu, 25 Oct 90 16:43:20 CDT Subject: NN texts: a summary. Message-ID: <9010252143.AA14326@galab2.mh.ua.edu> Dear Connectionists, A week ago I posted a message requesting recomendations for possible texts for a introductory, graduate level engineering course neural nets. I received some interesting responses (thanks), so I decided to summarize to the net. I'll keep my editorializing to a minimum, since I have not seen any of these texts yet. If you want to comparison shop, you'll have to do like me and call the publishers. The following texts were recomended: Neurocomputing by Robert Hecht-Nielsen Addison-Wesley Publishing Company 1990 (this received the most recomendations by far) (a solution manual is rumoured to be available soon.) Neural Networks in Artificial Intelligence by Matthew Zeidenberg Ellis Horwood Ltd., distributed in the US by Simon and Schuster (sounded awfully interesting) Introduction to Neural and Cognitive Modeling by Daniel S. Levine Lawrence Erlbaum Associates (not available yet) Adaptive Pattern Recognition and Neural Networks by Y. Pao (no detailed bib entry, sorry) Neurocomputing (??) by Wasserman (no detailed bib entry, sorry) Artificial Neural Systems by Patrick Simpson Pergamon Press That's it. I hope some of you find this helpful. Sincerely, Rob ------------------------------------------- Robert Elliott Smith Department of Engineering of Mechanics The University of Alabama P. O. Box 870278 Tuscaloosa, Alabama 35487 <> rob at galab2.mh.ua.edu <> (205) 348-4661 ------------------------------------------- From sims at starbase.MITRE.ORG Fri Oct 26 09:34:04 1990 From: sims at starbase.MITRE.ORG (Jim Sims) Date: Fri, 26 Oct 90 09:34:04 EDT Subject: NN, training & noise fitting REF request Message-ID: <9010261334.AA01599@starbase> I seem to recall a recent reference in the literature or on this list to detecting when your training (via back-prop) has begun to start fitting the noise in the data rather than the features of the data space. Can someone provide a reference? thanks, jim (sims at starbase.mitre.org) From ad1n+ at ANDREW.CMU.EDU Fri Oct 26 11:38:09 1990 From: ad1n+ at ANDREW.CMU.EDU (Alexander Joshua Douglas) Date: Fri, 26 Oct 90 11:38:09 -0400 (EDT) Subject: Robotic controllers Message-ID: I am thinking of doing some reasearch in the field of robotic controllers using nueral nets. I recently saw a post about two technical reports by Jogensen. I am very interested in obtaining them, but our library does not have them, nor can they seem to get them. Perhaps somone could tell me how to get them. The reports are: Jorgensen, Charles C. (1990). "Development of a Sensor Coordinated Kinematic Model for Neural Network Controller training". RIACS Tecnical Report 90.28 Jorgensen, Charles C. (1990). "Distributed Memory Approaches for Robotic Neural Controllers". RIACS Tecnical Report 90.29 thank you, Alexander Douglas (ad1n+ at andrew.cmu.edu) From INAM%MUSICB.MCGILL.CA at BITNET.CC.CMU.EDU Fri Oct 26 15:31:16 1990 From: INAM%MUSICB.MCGILL.CA at BITNET.CC.CMU.EDU (Tony Marley) Date: Fri, 26 Oct 90 14:31:16 EST Subject: POSITION IN COGNITIVE PSYCHOLOGY, MCGILL UNIVERSITY Message-ID: <26OCT90.15683034.0069.MUSIC@MUSICB.MCGILL.CA> Although Prof. Bregman is in charge of the search for someone to fill a position in COGNITIVE PSYCHOLOGY in the Department of Psychology at McGill University, I encourage mathematically and/or computationally oriented researchers to keep me informed of their interest. Although it is unlikelty that we will hire a "straight" mathematical or computational person for this position, I will certainly push for someone with mathematical and computational skills. In particular, I would very much like to see applicants in the general area of neural modeling. Please let me know if you apply, and feel free to contact me for further information. Tony Marley Professor, Department of Psychology Director, McGill Cognitive Science Centre email: INAM at MUSICB.MCGILL.CA Tel: 514-398-6128 (office) 514-488-2067 (home) ------------------------------------------------------------------ October 4, 1990 The Department of Psychology at McGill University plans to make a tenure-track appointment of an assistant or associate professor in COGNITIVE PSYCHOLOGY. The appointment will begin in September 1991, subject to the availability of funding. The department has a strong tradition in cognitive psychology and is affiliated with the Cognitive Science Centre at the university. It is strongly supportive of younger staff and tends to promote from within the department. We are looking for an outstanding researcher. Nevertheless, we place a great stress on our teaching program and are looking for a candidate that could make a special contribution to it. The applicant's research could be concerned with any aspect of cognitive psychology, broadly interpreted. The major criterion will be the excellence of the applicant. Please bring this letter to the attention of any individuals you think might be qualified to apply or to persons who might know of such individuals. Selection will begin in mid-January, 1991. Applicants should arrange for at least three confidential letters of support to be sent to the address below. They should also send a curriculum vitae, copies of research publications and a brief statement describing their teaching and research to: A.S. Bregman, Telephone: (514) 398-6103 Cognitive Search Committee FAX: (514) 398-4896 Department of Psychology, McGill University, E-mail: in09 at musicb.mcgill.ca 1205 Dr. Penfield Avenue, or: in09 at mcgillb.bitnet Montreal, Quebec, CANADA H3A lBl ------------------------------------------------------------------ From shen at iro.umontreal.ca Mon Oct 29 09:59:43 1990 From: shen at iro.umontreal.ca (Yu Shen) Date: Mon, 29 Oct 90 09:59:43 EST Subject: Parallel Implementation of Genectic Algorithm and Simulated Message-ID: <9010291459.AA23899@kovic.IRO.UMontreal.CA> Annealing I want to do a survey on the Parallel Implementation of Genectic Algorithm and Simulated Annealing. Any pointer to the currents in the area is very much appriciated. I will compile the results of inquiry to the list, if required. Yu Shen Dept. d'Informatique et Recherche Operationnelle University de Montreal C.P. 6128 Succ. A. Montreal, Que. Canada H3S 3J7 (514) 342-7089 (H) shen.iro.umontreal.ca From P.Refenes at cs.ucl.ac.uk Mon Oct 29 12:38:38 1990 From: P.Refenes at cs.ucl.ac.uk (P.Refenes@cs.ucl.ac.uk) Date: Mon, 29 Oct 90 17:38:38 +0000 Subject: PRE-PRINT AVAILABILITY. Message-ID: The following pre-print (SPIE-90, Boston, Nov. 5-9 1990) is available. (write or e-mail to A. N. Refenes at UCL) AN INTEGRATED NEURAL NETWORK SYSTEM for HISTOLOGICAL IMAGE UNDERSTANDING A. N. REFENES, N. JAIN & M. M. ALSULAIMAN Department of Computer Science, University College London, Gower Street, WC1, 6BT, London, UK. ABSTRACT This paper describes a neural network system whose architecture was designed so that it enables the integration of heterogeneous sub-networks for performing specialised tasks. Two types of networks are integrated: a) a low-level feature extraction network for sub-symbolic computation, and b) a high-level network for decision support. The paper describes a non trivial application from histopathology, and its implementation using the Integrated Neural Network System. We show that with careful network design, the backpropagation learning procedure is an effective way of training neural networks for histological image understanding. We evaluate the use of symmetric and asymmetric squashing functions in the learning procedure and show that symmetric functions yield faster convergence and 100% generalisation performance. From IP%IRMKANT.BITNET at VMA.CC.CMU.EDU Mon Oct 29 11:19:36 1990 From: IP%IRMKANT.BITNET at VMA.CC.CMU.EDU (stefano nolfi) Date: Mon, 29 Oct 90 12:19:36 EDT Subject: paper available Message-ID: The following technical report is now available. You can get it from: STIVA AT IRMKANT.BITNET. Send your real adress. RECALL OF SEQUENCES OF ITEMS BY A NEURAL NETWORK Stefano Nolfi* Domenico Parisi* Giuseppe Vallar** Cristina Burani* *Inst. of Psychology - C.N.R. - Rome **University of Milan - Italy ABSTRACT A network architecture of the forward type but with additional 'memory' units that store the hidden units activation at time 1 and re-input this activation to the hidden units at time 2 (Jordan, 1986; Elman, 1990) is used to train a network to free recall sequences of items. The network's performance exhibits some features that are also observed in humans, such as decreasing recall with increasing sequence length and better recall of the first and the last items compared with middle items. An analysis of the network's behavior during sequence presentation can ex- plain these results. INTRODUCTION Human beings possess the ability to recall a set of items that are presented to them in a sequence. The overall capacity of the memory systems used in this task is limited and the probability of recall decreases with increasing sequence length. A second relevant feature of human performance in this task is that the last (recency effect) and the initial (primacy effect) items of the sequence tend to be recalled better than the middle items. These serial position effects have been observed both in a free recall condition, in which subjects may recall the stimuli in any order they wish, and in a serial recall condition, in which subjects must preserve the presentation order. (See reviews concerning free and serial recall of sequences and the recency ef- fect in: Glanzer, 1972; Crowder, 1976; Baddeley and Hitch, 1977; Shallice and Vallar, 1990). In this paper we report the results of a simulation experiment in which we trained neural networks to recall sequences of items. Our purpose was to explore if a particular network architecture could function as a memory store for generating free recall of sequences of items. Furthermore, we wanted to determine if the recall performances of our networks exhibited the two features of human free recall that we have mentioned, that is, decreasing probability of recall with increasing sequence length and an U-shaped recall curve (for related works see: Schneider and Detweiler, 1987; Schreter and Pfeifer, 1989; Schweickert, Guentert and Hersberger, 1989). To appear in: In D.S.Touretzky, J.L. Elman, T.J. Sejnowski and G.E. Hinton (eds.), Proceedings of the 1990 Connectionist Models Summer School. San Matteo, CA: Morgan Kaufmann. REFERENCES Baddeley A.D., Hitch G.J. (1974). Recency re-examined. In S. Dornic (Ed.). Attention and performance (Vol. 6). Hillsdale, NJ:Erlbaum, pp. 647-667. Crowder R.G. (1976). Principles of learning and memory. Hillsdale, NJ: Erlbaum. Glanzer M. (1972). Storage mechanisms in recall. In G.H. Bower (Ed.). The Psychology of learning and motivation. Advances in research and theory. (Vol. 5). New York: Academic Press, pp. 129-193. Elman, J.L. Finding structure in time. (1990). Cognitive Science, 14, 179-211. Jordan, M.I. (1986). Serial order: A parallel distributed processing approach. Institute for Cognitive Science. Report 8604. University of California, San Diego. Shallice T., Vallar G. (1990). The impairment of auditory-verbal short-term storage. In: G. Vallar and T. Shallice (Eds.). Neuropsychological impairments of short-term memory. New York: Cambridge University Press, pp.11-53. Schneider, W., & Detweiler, M. (1987). A connectionist control architecture for working memory. In G.H. Bower (Ed.) The Psychology of learning and motivation vol 21. New York: Academic Press. Schreter, Z., & Pfeirer, R. (1989). Short term memory and long term memory interactions in connectionist simulations of psychological experiments on list learning. In L. Personnaz and G. Dreyfus (Eds.), Neural Network: From models to applications. Paris: I.D.S.E.T. Schweickert, R., Guentert, L., & Hersberger, L. (1989). Neural Network Models of Memory Span. Preceedings of the Eleventh Annual Conference of the Cognitive Science Society. Ann Arbor, Michigan. From tgd at turing.CS.ORST.EDU Tue Oct 30 12:27:36 1990 From: tgd at turing.CS.ORST.EDU (Tom Dietterich) Date: Tue, 30 Oct 90 09:27:36 PST Subject: Local receptive fields Message-ID: <9010301727.AA03804@turing.CS.ORST.EDU> I am confused by what appear to be two different usages of the term "local receptive fields", and I wonder if anyone can un-confuse me. In papers about radial basis functions (e.g., Moody and Darken, Poggio and Girosi, etc.) the (single) layer of hidden units are described as having local receptive fields. However, these hidden units receive input from EVERY input unit, which strikes me as being more global than local. It is true, however, that each hidden unit will respond to only a few of the possible input vectors, and that this behavior can be described in terms of the "distance" between the input vector and the weight vector of the hidden unit. So in this sense, the hidden unit is responsive to a particular locality in the Euclidean n-space containing the input vectors. On the other hand, in papers such as those by Waibel et al on phoneme recognition or by LeCun et al on handwritten digit recognition, the hidden units have connections to only a few of the input units. These hidden units are also described as having local receptive fields. From Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU Tue Oct 30 17:46:47 1990 From: Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF1.SLISP.CS.CMU.EDU) Date: Tue, 30 Oct 90 17:46:47 EST Subject: Local receptive fields In-Reply-To: Your message of Tue, 30 Oct 90 09:27:36 -0800. <9010301727.AA03804@turing.CS.ORST.EDU> Message-ID: I am confused by what appear to be two different usages of the term "local receptive fields", and I wonder if anyone can un-confuse me. I think that the two usages of the term "local receptive field" are more or less the same idea, but different input encodings change the implementation. In both cases, you've got input units encoding some sort of N-dimensional space, and you've got hidden (or output) units that respond only to a localized patch of that hyperspace. That's the basic idea. If you think of each input as being a distinct, continous dimension, then you end up with something like the Moody and Darken units, which respond to some hyper-sphere or hyper-ellipsoid around an N-dimensional center point. On the other hand, if you think the individual units as encoding intervals or patches in this space (as in the speech networks -- each input is a little patch of time/frequency space or something like that), then you end up with hidden units that have inputs from a set of these units. Of course, there are a few more degrees of freedom in the latter case: within the "receptive field", the individual weights can encode something more complex than a simple Gaussian. So I think that the term "local receptive field" can cover both cases, but we need to specify what space we are working in and how the inputs map into that space: continuous orthogonal dimensions, mosaic encoding, or some hybrid scheme. -- Scott From jose at learning.siemens.com Tue Oct 30 17:23:55 1990 From: jose at learning.siemens.com (Steve Hanson) Date: Tue, 30 Oct 90 17:23:55 EST Subject: Local receptive fields Message-ID: <9010302223.AA03681@learning.siemens.com.siemens.com> splitting hairs: tom, I don't think we need new terms to describe what is a continuum... clearly in a limiting sense a fan-in function that responds to a specific point in input space (although measures input on each line to determine this) is similar in effect to a net that has a subset of lines from the input space. Really the important issue is spatial locality... does a hidden unit have a preference for a location or is it a global in its fan-in function.. this will have affects on learning, recognition and approximation. steve From mike at park.bu.edu Tue Oct 30 14:43:52 1990 From: mike at park.bu.edu (mike@park.bu.edu) Date: Tue, 30 Oct 90 14:43:52 -0500 Subject: CNS Program at Boston University Hiring 2 Assistant Professors Message-ID: <9010301943.AA01462@BUCASB.BU.EDU > Boston University seeks two tenure track assistant or associate professors starting in Fall, 1991 for its M.A. and Ph.D. Program in Cognitive and Neural Systems. This program offers an intergrated curriculum offering the full range of psychological, neurobiological, and computational concepts, models, and methods in the broad field variously called neural networks, connectionism, parallel distributed processing, and biological information processing, in which Boston University is a leader. Candidates should have extensive analytic or computational research experience in modelling a broad range of nonlinear neural networks, especially in one or more of the areas: vision and image processing, speech and language processing, adaptive pattern recognition, cognitive information processing, and adaptive sensory-motor control Candidates for associate professor should have an international reputation in neural network modelling. Send a complete curriculum vitae and three letters of recommendation to Search Committee, Cognitive and Neural Systems Program, Room 240, 111 Cummington Street, Boston University, Boston, MA 02215, preferably by November 15, 1990 but no later than January 1, 1991. Boston University is an Equal Opportunity/Affirmative Action employer. Boston University (617-353-7857) Email: mike at bucasb.bu.edu Smail: Michael Cohen 111 Cummington Street, RM 242 Center for Adaptive Systems Boston, Mass 02215 Boston University From jose at learning.siemens.com Tue Oct 30 20:14:47 1990 From: jose at learning.siemens.com (Steve Hanson) Date: Tue, 30 Oct 90 20:14:47 EST Subject: Local receptive fields Message-ID: <9010310114.AA03746@learning.siemens.com.siemens.com> Tom, I wasn't meaning to be obfuscate, I just believe such questions do lead to splitting hairs and in any case is a very complex issue. Which (sense) is truer to its physiological origins? who knows? (anyone who does please comment) but receptive field properties can be complex--they can be "simple". Clearly not all neurons are connected to all other neurons --so in that trivial sense they are "local". Do they have preference for other cells? No doubt. But the data are complex and difficult to map onto specific computational functions. This I think is one of the advantages to computational models in that modelers can quickly explore the computational consequences of hypotheses that neuroscientists must explore in more tedious and perhaps information poorer ways. Steve From kruschke at ucs.indiana.edu Tue Oct 30 16:36:00 1990 From: kruschke at ucs.indiana.edu (KRUSCHKE,JOHN,PSY) Date: 30 Oct 90 16:36:00 EST Subject: local receptive fields Message-ID: > Date: Tue, 30 Oct 90 09:27:36 PST > From: Tom Dietterich > Subject: Local receptive fields > > I am confused by what appear to be two different usages of the term > "local receptive fields", and I wonder if anyone can un-confuse me. > > In papers about radial basis functions (e.g., Moody and Darken, Poggio > and Girosi, etc.) the (single) layer of hidden units are described as > having local receptive fields. ... > > On the other hand, in papers such as those by Waibel et al on phoneme > recognition or by LeCun et al on handwritten digit recognition, the > hidden units have connections to only a few of the input units. These > hidden units are also described as having local receptive fields. > > From a nervous system point of view, it seems to me that the second > usage is more correct than the first. I think we need a better term for > describing the kind of locality exhibited by RBF networks. There is an important difference between the intrinsic topologies of the input layers in the two situations Dietterich describes. In RBF networks, the input nodes are usually assumed to each represent entire dimensions of variation. So if there are N input nodes, then the input space is N-dimensional. The RBF nodes at the hidden layer are responsive to only a local region of the N-dimensional input space. On the other hand, in the situations that Dietterich describes as more "neural", the input nodes are not usually assumed to each individually represent entire dimensions of variation. For example, each node might represent a small region of a 2-D image, so that the entire ensemble of N input nodes actually represents just a 2-D input space. In such a space there is an intrinsic topology so that some nodes are closer to each other than others, whereas in the input space of the RBF network, no input nodes are closer to each other than any others. So, both types of "localization in space" make good sense, but in their own representations of space. --John Kruschke From tgd at turing.CS.ORST.EDU Tue Oct 30 17:54:11 1990 From: tgd at turing.CS.ORST.EDU (Tom Dietterich) Date: Tue, 30 Oct 90 14:54:11 PST Subject: Local receptive fields In-Reply-To: Steve Hanson's message of Tue, 30 Oct 90 17:23:55 EST <9010302223.AA03681@learning.siemens.com.siemens.com> Message-ID: <9010302254.AA04488@turing.CS.ORST.EDU> I disagree that this is splitting hairs. It can be very confusing for someone entering this research area to have the same term used in two different (albeit related) ways. --Tom P.S. Which use of the term is truer to its neurophysiological origins? From dario%TECHUNIX.BITNET at VMA.CC.CMU.EDU Wed Oct 31 04:15:30 1990 From: dario%TECHUNIX.BITNET at VMA.CC.CMU.EDU (Dario Ringach) Date: Wed, 31 Oct 90 11:15:30 +0200 Subject: local receptive fields In-Reply-To: "KRUSCHKE,JOHN,PSY" "local receptive fields" (Oct 30, 4:36pm) Message-ID: <9010310915.AA11853@techunix.technion.ac.il> A good mathematical definition of "locality in space", I think, is to require the unit to have a compact support in the input space. In this sense, RBF-networks have not local units. The study of non-orthogonal and orthogonal complete systems in L^2(R) having compact support, is of importance here. See for example the work of Daubechies, Meyer, Mallat, and others on Wavelet theory. Of course, other space-frequency representation/analysis of signals, such as the Gabor transform, are also related to simultaneous localization in space and frequency. - Dario Ringach From Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU Wed Oct 31 10:02:07 1990 From: Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF1.SLISP.CS.CMU.EDU) Date: Wed, 31 Oct 90 10:02:07 EST Subject: Local receptive fields In-Reply-To: Your message of Tue, 30 Oct 90 20:14:47 -0500. <9010310114.AA03746@learning.siemens.com.siemens.com> Message-ID: Which (sense) is truer to its physiological origins? who knows? (anyone who does please comment) but receptive field properties can be complex--they can be "simple". Clearly not all neurons are connected to all other neurons --so in that trivial sense they are "local". Do they have preference for other cells? No doubt. It's clear that in the big visual areas that have been mapped, some sort of mosaic encoding is in use. That is, the most obvious dimensions of the space -- the X/Y diemensions of the 2-D image on the retina -- are not represented by varying levels of activation, but rather are mapped across sets of cells in a regular (though distorted and interleaved) 2-D mosaic. Other major systems seem to use the same kind of encoding: hearing, tactile senses, motor control, etc. There may or may not be some continuous, analog encoding (levels or pulse-frequency) at other levels or among some of the low-level control systems. So clearly, for most parts of the neural system in mammals, the idea of "local receptive field" for cells near the input layer maps cleanly into some sort of range limitation on the map in question. I think what you are talking about above, however, is what I would call "local connectivity" rather than "local receptive fields". If you allow signals to take a few hops, a net whose immediate physical connections are mostly local can give you a very large receptive field. The wire on my telephone runs to an office a few blocks away, but my telephone's receptive field includes just about any place on earth. And if a net makes significant use of recurrent links and delays, the receptive field extends back in time in a complicated way. The term "receptive field" has mainly been used in describing combinational logic (with no memory, maybe just some adaptation and gain control) and should probably be reserved for that context. -- Scott "No, no Igor! The OTHER brain..." From jbower at smaug.cns.caltech.edu Wed Oct 31 11:13:52 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Wed, 31 Oct 90 08:13:52 PST Subject: NIPS demo machines Message-ID: <9010311613.AA19176@smaug.cns.caltech.edu> Announcement NIPS Demo Machines Two machines will likely be available for demos at this year's NIPS meeting: 1 DECstation 5000/200PX with TK50 tape drive running Ultrix 4.0 1 Sparcstation 1+ or better RISC box from Sun with a 1/4" tape drive. running Sun OS 4.1c Both machines will be 8 bit depth color, have between 8 and 16Mb of memory. Participants should feel free to bring software demos. The machines are not, however, intended to be used to sell software. Machines will be schedualed on a first come first serve basis. Anyone with questions can contact John Uhley at uhley.smaug.cns.caltech.edu. John will also be available at the meeting. Jim Bower From moody-john at CS.YALE.EDU Wed Oct 31 10:56:06 1990 From: moody-john at CS.YALE.EDU (john moody) Date: Wed, 31 Oct 90 10:56:06 EST Subject: locality, RBF's, receptive fields, and all that Message-ID: <9010311556.AA29442@SUNNY.SYSTEMSX.CS.YALE.EDU> Concerning the questions of terminology raised by Tom Dietterich: After we wrote "Learning with localized receptive fields" (Proceedings of the 1988 Connectionist Models Summer School, Touretzky, Hinton, and Sejnowski, eds., Morgan Kaufmann), we found that our choice of terminology was confusing to some. We tried to remedy this confusion when we wrote "Fast learning in networks of locally-tuned processing units" (Neural Computation 1(2)281-294, 1989). The point of confusion is whether "local receptive field" should refer to the network connectivity pattern (or the afferent connectivity of a single unit) or the form of the response function of a processing unit. We agree with Dietterich that the term "local receptive field" should be reserved for internal units whose afferent connections come primarily from a local neighborhood of units in the preceding layer. We believe that this term is best NOT used to describe RBF type units, which typically (but not always) have global connectivity to their input layer. To describe localized unit response functions, we favor the term "locally- tuned processing unit" (LTPU). These include RBFs, but are not limited to units whose response functions are radially symmetric { R(x,y) = R(|x-y|) }. [Indeed, we have found that non-radial LTPUs often work better than the standard RBFs.] The distinction between "local receptive field" and "locally tuned processing unit" can become blurred when one considers "effective variables". These variables are *not* the activations of individual input units, but are rather implicitly encoded for by the population of input units. For example, the orientation selective cells of V1 have "local receptive fields" as determined by their afferent connection patterns. However, they also "respond locally" in the *effective variables* angular orientation and retinal position. The locality of response in angular orientation depends on the *values* of the afferent connections, while the locality of response to retinal position is due to the locality of the receptive fields. [The localities of response to the effective variables could be modeled by a network of LTPUs with three input variables. Of course, such a network would not be responsive to more that one object in the scene or to different levels of illumination.] Provided that one is clear about which variables one is referring to, whether input unit activations or effective variables, confusion between local connectivity patterns ("local receptive fields") and localized response functions ("LTPUs") can be avoided. --John Moody and Christian Darken ------- From gary%cs at ucsd.edu Wed Oct 31 11:49:16 1990 From: gary%cs at ucsd.edu (Gary Cottrell) Date: Wed, 31 Oct 90 08:49:16 PST Subject: splitting hairs Message-ID: <9010311649.AA18964@desi.ucsd.edu> Since Steve brought up hair splitting, it seemed like a good time to send out my latest: SEMINAR Approaches to the Inverse Dogmatics Problem: Time for a return to localist networks? Garrison W. Cottrell Department of Dog Science Condominium Community College of Southern California The innovative use of neural networks in the field of Dognitive Science has spurred the intense interest of the philosophers of Dognitive Science, the Dogmatists. The field of Dogmatics is devoted to making sense of the effect of neural networks on the conceptual underpinnings of Dognitive Science. Unfortunately, this flurry of effort has caused researchers in the rest of the fields of Dognitive Science to spend an inordinate amount of time attempting to make sense of the philosophers, otherwise known as the Inverse Dogmatics problem (Jordan, 1990). The problem seems to be that the philosophers have allowed themselves an excess of degrees of freedom in conceptual space, as it were, leaving the rest of us with an underconstrained optimization problem: Should we bother listening to these folks, who may be somewhat more interesting than old Star Trek reruns, or should we try and get our work done? The inverse dogmatics problem has become so prevalent that many philosophers are having to explain themselves daily, much to the dismay of the rest of the field. For example Gonad[1] (1990a, 1990b, 1990c, 1990d, 1990e, well, you get the idea...) has repeatedly stated that no connectionist network can pass his usually Fatal Furring Fest, where the model is picked apart, hair by hair[2], until the researchers making counterarguments have long since died[3]. One approach to this problem is to generate a connectionist network that is so hairy (e.g., Pollack's RAMS, 1990), that it will outlast Gonad's attempt to pick it apart. This is done by making a model that is at the sub-fur level, that recursively splits hairs, RAMming more and more into each hair, which generates a fractal representation that is not susceptible to linear hair splitting arguments. Another approach is to take Gonad head-on, and try to answer his fundamental question, that is, the problem of how external discrete nuggets get mapped into internal mush. This is known as the *grinding problem*. In our approach to the grinding problem, we extend our previous work on the Dog Tomatogastric Ganglion (TGG). The TGG is an oscillating circuit in the dog's motor cortex that controls muscles in the dog's stomach that expel tomatoes and other non-dogfood items from the dog's stomach. In our grinding network, we will have a similar set up, using recurrent bark propagation to train the network to oscillate in such a way that muscles in the dog's mouth will grind the nuggets ____________________ [1]Some suspect that Gonad may in fact be an agent of reactionary forces whose mission is to destroy Dognitive Science by filibuster. [2]Thus by a simple morphophonological process of reduplication, ex- haustive arguments have been replaced by exhausting arguments. [3]In this respect, Gonad's approach resembles that of Pinky and Prince, whose exhausting treatment of the Past Fence Model, Rumblephart and McNugget's connectionist model of dog escapism, has generated a sub- field of Dognitive Science composed of people trying to answer their ar- guments. into the appropriate internal representation. This representation is completely distributed. This is then transferred directly into the dog's head, or Mush Room. Thus the thinking done by this representation, like most modern distributed representations, is not Bayesian, but Hazyian. If Gonad is not satisfied by this model, we have an alternative approach to this problem. We have come up with a connectionist model that has a *finite* number of things that can be said about it. In order to do this we had to revert to a localist model, suggesting there may be some use for them after all. We will propose that all connectionist researchers boycott distributed models until the wave of interest by the philosophers passes. Then we may get back to doing science. Thus we must bring out some strong arguments in favor of localist models. The first is that they are much more biologically plausible than distributed models, since *just like real neurons*, the units themselves are much more complicated than those used in simple PDP nets. Second, just like the neuroscientists do with horseradish peroxidase, we can label the units in our network, a major advantage being that we have many more labels than the neuroscientists have, so we can keep ahead of them. Third, we don't have to learn any more than we did in AI 101, because we can use all of the same representations. As an example of the kind of model we think researchers should turn their attention to, we are proposing the logical successor to Anderson & Bower's HAM model, SPAM, for SPreading Activation Memory model. In this model, nodes represent language of thought propositions. Because we are doing Dog Modeling, we can restrict ourselves to at most 5 primitive ACTS: eat, sleep, fight, play, make whoopee. The dog's sequence of daily activities can then be simply modeled by connectivity that sequences through these units, with habituation causing sequence transitions. A fundamental problem here is, if the dog's brain can be modeled by 5 units, *what is the rest of the dog's brain doing?* Some have posited that localist networks need multiple copies of every neuron for reliability purposes, since if the make whoopee unit was traumatized, the dog would no longer be able to make whoopee. Thus these researchers would posit that the rest of the dog's brain is simply made up of copies of these five neurons. However, we believe we have a more esthetically pleasing solution to this problem that simultaneously solves the size mismatch problem. The problem is that distributed connectionists, when discussing the reliability problem of localist networks, have in mind the wimpy little neurons that distributed models use. We predict that Dognitive neuroscientists, when they actually look, will find only five neurons in the dog's brain - but they will be *really big* neurons. From jbower at smaug.cns.caltech.edu Wed Oct 31 12:07:03 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Wed, 31 Oct 90 09:07:03 PST Subject: RFs Message-ID: <9010311707.AA19206@smaug.cns.caltech.edu> Just a brief note on the question of receptive fields from a biological perspective. Classically, receptive fields are defined within neurobiology as those regions of the stimulus space that, when activated, obviously alter the firing pattern of the neuron in question (either excite or inhibit, or both). Traditionally, the issue of the actual anatomical connectivity underlying the formation of the receptive field has not been considered in much detail (often it is not known). However, physiologists have recently discovered that neuronal receptive fields can be far more complicated than previously assumed. The most famous current examples are the so- called nonclassical receptive fields of neurons in visual regions of cerebral cortex. In this case, it has been discovered that peripheral stimuli that do not by themselves activate a neuron are capable of significantly modifying neuronal responses to stimulation of classical receptive fields. These effects are probably a result of the understudied and underemphasized horizontal connectivity of cerebral cortical networks. The neglect of this important network feature is, in fact, largely due to the emphasis on the restricted "locality" of classical receptive fields and the resulting (faulty in my view) notion of cortical columnar organization. The point with respect to the current discussion is that it IS important to carefully define what is meant by a receptive field and especially to take into account the anatomical organization of the network involved. Operational definitions related to the simple response properties of the neuron can obscure important network properties. It could be that this attention to detail is less important in the case of the simple connectionist networks currently being constructed. However, as connectionist models inevitably become more complex in the pursuit of real usefulness, it will become increasingly important to be careful about what is meant by a receptive field. Jim Bower From at neural.att.com Wed Oct 31 10:20:23 1990 From: at neural.att.com (@neural.att.com) Date: Wed, 31 Oct 90 10:20:23 -0500 Subject: Local receptive fields In-Reply-To: Your message of Tue, 30 Oct 90 09:27:36 -0800. Message-ID: <9010311520.AA11642@lamoon> > I am confused by what appear to be two different usages of the term > "local receptive fields",.... I also think these two meanings of "local receptive field" is very confusing. These are two very different concepts. I guess the notion of local receptive field was introduced by neurobiologists. They meant "local" in real space, not in feature space. The region of feature space in which radial basis functions are activated, Should be called differently to avoid confusion. I think the correct word to designate the set on which a function takes non-zero values is "support". Unfortunately, most radial basis functions (such as gaussians) are non-zero everywhere. How about "activation window", or "activation region". -- Yann Le Cun From at neural.att.com Wed Oct 31 11:14:52 1990 From: at neural.att.com (@neural.att.com) Date: Wed, 31 Oct 90 11:14:52 -0500 Subject: Local receptive fields In-Reply-To: Your message of Tue, 30 Oct 90 17:23:55 -0500. Message-ID: <9010311614.AA11671@lamoon> Steve says > splitting hairs: tom, I don't think we need new terms to describe > what is a continuum... I disagree. An RBF unit in a network with 1024 inputs will have 1024 input lines (and therefore at least 1024 parameters). Now assume these 1024 inputs are actually the pixels of a 32x32 image....what is local there? certainly not the connections. Now here is a really confusing situation: imagine an image-recognition network with 32x32 inputs (pixels). Suppose that the units in the first layer are connected to, say, local (2D) 5x5 patches on the input (receptive field concept #1). Now imagine that these units are gaussian RBF units. Their activation region (receptive field concept #2) is a hypersphere (or something similar) in the 25-dimensional space of their input (5x5=25). If these units were sigmoid units, their activation region (receptive field concept #2) would be a half space in the 25-D space. As you see concept #1 and concept #2 are completely orthogonal, and can be used independently, or even combined. They, therefore, should have different names. -- Yann From mehra at aquinas.csl.uiuc.edu Wed Oct 31 15:00:27 1990 From: mehra at aquinas.csl.uiuc.edu (Pankaj Mehra) Date: Wed, 31 Oct 90 14:00:27 CST Subject: local receptive fields Message-ID: <9010312000.AA15613@rhea> In response to Tom Dietterich : > I am confused by what appear to be two different usages of the term > "local receptive fields" .... and Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU: > the two usages of the term "local receptive field" are more or > less the same idea, but different input encodings change the > implementation. I think that the question of locality starts well before the data are given to the learning program. If we measure variables (dimensions) using one sensor for each dimension, then transforming intervals (overlapping or otherwise) of that dimension into local receptive fields is a matter of representational convenience (different encodings, as Scott says). Real receptive fields (i.e. units responding to limited regions of an abstract feature space) perform oversampling of the input space, thus providing real redundancy in observed data. There are multiple sensors for each dimension. Artificial receptive fields (a good example is the BOXES representation used in Chuck Anderson's thesis, U. Mass. Amherst, 1986) merely recode the irreedundant data via a value-coded representation. Artificial overlapping receptive fields merely oversample the data which is collected non-redundantly. In natural systems, real receptive fields can therefore alleviate sensor errors (even dead cells) with a possible loss of resolution. Thus, they are in some sense ``robust'' to noisy sensors. Personally, I believe that multiple independent sensors should have statistical significance as well, although I have not seen any discussion of that in the literature I am aware of. - Pankaj Mehra University of Illinois From lyle at ai.mit.edu Wed Oct 31 16:48:31 1990 From: lyle at ai.mit.edu (Lyle J. Borg-Graham) Date: Wed, 31 Oct 90 16:48:31 EST Subject: Local receptive fields Message-ID: <9010312148.AA04635@peduncle> >But the data are complex and difficult to map onto specific >computational functions. This I think is one of the advantages >to computational models in that modelers can quickly explore >the computational consequences of hypotheses that >neuroscientists must explore in more tedious and perhaps >information poorer ways. Of course, an ever-present risk is that modellers can over-generalize from seductively-simple models (or just simply-seductive ones), evoking an Occamian motivation which helps the neuroscientists and their interpretation of the data not a whit. From jose at learning.siemens.com Wed Oct 31 17:02:51 1990 From: jose at learning.siemens.com (Steve Hanson) Date: Wed, 31 Oct 90 17:02:51 EST Subject: Local receptive fields Message-ID: <9010312202.AA05510@learning.siemens.com.siemens.com> Indeed! I agree completely... a similar point (perhaps more balanced) was made by Phil Anderson a while back "The art of model-building is the exclusion of real but irrelevant parts of the problem, and entails hazards for the builder and the reader. The builder may leave out something genuinely relevant; the reader, armed with too sophisticated an experimental probe or too accurate a computation, may take literally a schematized model whose main aim is to be a demonstration of possibility." -P. W. Anderson (from Nobel acceptance speech, 1977) From at neural.att.com Wed Oct 31 11:14:52 1990 From: at neural.att.com (@neural.att.com) Date: Wed, 31 Oct 90 11:14:52 -0500 Subject: Local receptive fields In-Reply-To: Your message of Tue, 30 Oct 90 17:23:55 -0500. Message-ID: <9010311614.AA11671@lamoon> Steve says > splitting hairs: tom, I don't think we need new terms to describe > what is a continuum... I disagree. An RBF unit in a network with 1024 inputs will have 1024 input lines (and therefore at least 1024 parameters). Now assume these 1024 inputs are actually the pixels of a 32x32 image....what is local there? certainly not the connections. Now here is a really confusing situation: imagine an image-recognition network with 32x32 inputs (pixels). Suppose that the units in the first layer are connected to, say, local (2D) 5x5 patches on the input (receptive field concept #1). Now imagine that these units are gaussian RBF units. Their activation region (receptive field concept #2) is a hypersphere (or something similar) in the 25-dimensional space of their input (5x5=25). If these units were sigmoid units, their activation region (receptive field concept #2) would be a half space in the 25-D space. As you see concept #1 and concept #2 are completely orthogonal, and can be used independently, or even combined. They, therefore, should have different names. -- Yann %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% End of returned mail From gary%cs at ucsd.edu Wed Oct 31 11:49:16 1990 From: gary%cs at ucsd.edu (Gary Cottrell) Date: Wed, 31 Oct 90 08:49:16 PST Subject: splitting hairs Message-ID: <9010311649.AA18964@desi.ucsd.edu> Since Steve brought up hair splitting, it seemed like a good time to send out my latest: SEMINAR Approaches to the Inverse Dogmatics Problem: Time for a return to localist networks? Garrison W. Cottrell Department of Dog Science Condominium Community College of Southern California The innovative use of neural networks in the field of Dognitive Science has spurred the intense interest of the philosophers of Dognitive Science, the Dogmatists. The field of Dogmatics is devoted to making sense of the effect of neural networks on the conceptual underpinnings of Dognitive Science. Unfortunately, this flurry of effort has caused researchers in the rest of the fields of Dognitive Science to spend an inordinate amount of time attempting to make sense of the philosophers, otherwise known as the Inverse Dogmatics problem (Jordan, 1990). The problem seems to be that the philosophers have allowed themselves an excess of degrees of freedom in conceptual space, as it were, leaving the rest of us with an underconstrained optimization problem: Should we bother listening to these folks, who may be somewhat more interesting than old Star Trek reruns, or should we try and get our work done? The inverse dogmatics problem has become so prevalent that many philosophers are having to explain themselves daily, much to the dismay of the rest of the field. For example Gonad[1] (1990a, 1990b, 1990c, 1990d, 1990e, well, you get the idea...) has repeatedly stated that no connectionist network can pass his usually Fatal Furring Fest, where the model is picked apart, hair by hair[2], until the researchers making counterarguments have long since died[3]. One approach to this problem is to generate a connectionist network that is so hairy (e.g., Pollack's RAMS, 1990), that it will outlast Gonad's attempt to pick it apart. This is done by making a model that is at the sub-fur level, that recursively splits hairs, RAMming more and more into each hair, which generates a fractal representation that is not susceptible to linear hair splitting arguments. Another approach is to take Gonad head-on, and try to answer his fundamental question, that is, the problem of how external discrete nuggets get mapped into internal mush. This is known as the *grinding problem*. In our approach to the grinding problem, we extend our previous work on the Dog Tomatogastric Ganglion (TGG). The TGG is an oscillating circuit in the dog's motor cortex that controls muscles in the dog's stomach that expel tomatoes and other non-dogfood items from the dog's stomach. In our grinding network, we will have a similar set up, using recurrent bark propagation to train the network to oscillate in such a way that muscles in the dog's mouth will grind the nuggets ____________________ [1]Some suspect that Gonad may in fact be an agent of reactionary forces whose mission is to destroy Dognitive Science by filibuster. [2]Thus by a simple morphophonological process of reduplication, ex- haustive arguments have been replaced by exhausting arguments. [3]In this respect, Gonad's approach resembles that of Pinky and Prince, whose exhausting treatment of the Past Fence Model, Rumblephart and McNugget's connectionist model of dog escapism, has generated a sub- field of Dognitive Science composed of people trying to answer their ar- guments. into the appropriate internal representation. This representation is completely distributed. This is then transferred directly into the dog's head, or Mush Room. Thus the thinking done by this representation, like most modern distributed representations, is not Bayesian, but Hazyian. If Gonad is not satisfied by this model, we have an alternative approach to this problem. We have come up with a connectionist model that has a *finite* number of things that can be said about it. In order to do this we had to revert to a localist model, suggesting there may be some use for them after all. We will propose that all connectionist researchers boycott distributed models until the wave of interest by the philosophers passes. Then we may get back to doing science. Thus we must bring out some strong arguments in favor of localist models. The first is that they are much more biologically plausible than distributed models, since *just like real neurons*, the units themselves are much more complicated than those used in simple PDP nets. Second, just like the neuroscientists do with horseradish peroxidase, we can label the units in our network, a major advantage being that we have many more labels than the neuroscientists have, so we can keep ahead of them. Third, we don't have to learn any more than we did in AI 101, because we can use all of the same representations. As an example of the kind of model we think researchers should turn their attention to, we are proposing the logical successor to Anderson & Bower's HAM model, SPAM, for SPreading Activation Memory model. In this model, nodes represent language of thought propositions. Because we are doing Dog Modeling, we can restrict ourselves to at most 5 primitive ACTS: eat, sleep, fight, play, make whoopee. The dog's sequence of daily activities can then be simply modeled by connectivity that sequences through these units, with habituation causing sequence transitions. A fundamental problem here is, if the dog's brain can be modeled by 5 units, *what is the rest of the dog's brain doing?* Some have posited that localist networks need multiple copies of every neuron for reliability purposes, since if the make whoopee unit was traumatized, the dog would no longer be able to make whoopee. Thus these researchers would posit that the rest of the dog's brain is simply made up of copies of these five neurons. However, we believe we have a more esthetically pleasing solution to this problem that simultaneously solves the size mismatch problem. The problem is that distributed connectionists, when discussing the reliability problem of localist networks, have in mind the wimpy little neurons that distributed models use. We predict that Dognitive neuroscientists, when they actually look, will find only five neurons in the dog's brain - but they will be *really big* neurons. From rudnick at uxh.cso.uiuc.edu Tue Oct 30 11:56:47 1990 From: rudnick at uxh.cso.uiuc.edu (Mike Rudnick) Date: Tue, 30 Oct 90 10:56:47 -0600 Subject: requst for parallel implementation of GA references Message-ID: FOGA-90 (Foundations of Genetic Algrithms) workshop Indiana U. in Bloomington. Summer, 90 Gregory Rawlins, a faculty member there. There were a few presentations on parallel GA algorithms. ------------------- From iro.umontreal.ca!relay.eu.net!gmdzi!muehlen at IRO.UMontreal.CA Tue Oct 30 07:57:14 1990 From: iro.umontreal.ca!relay.eu.net!gmdzi!muehlen at IRO.UMontreal.CA (Heinz Muehlenbein) Date: Tue, 30 Oct 90 11:57:14 -0100 Subject: parallel genetic algorithmn Message-ID: We have implemented starting in 1987 a parallel genetic algorithm. It is the most powerful random search method in traveling salesman quadratic assignmment m-graph partitioning autocorrelation. n-dimensional multimodal function optimization references: Evolution algorithms in combinatorial optimization Parallel computing 7 65-88 (1988) papers from Muehlenbein quadratic assignment and Gorges-schleuter in 3rd conference on genetic algorithms recent papers ( not yet published) give a survey parallel genetic algorithms and combinatorial optimization ( SIAM) Evolution in space and time - the parallel genetic algorithm ( FOGA) Heinz Muehlenbein ------------------------- ---------- From pattie at ai.mit.edu Mon Oct 29 10:42:58 1990 From: pattie at ai.mit.edu (Pattie Maes) Date: Mon, 29 Oct 90 10:42:58 EST Subject: Parallel Implementation of Genectic Algorithm and Simulated In-Reply-To: Yu Shen's message of Mon, 29 Oct 90 09:59:43 EST <9010291459.AA23899@kovic.IRO.UMontreal.CA> Message-ID: Piet Spiessens and Bernard Manderick from the University of Brussels in Belgium have done work on parallel implementations od GA's. I think this was published in one of the conferences on GA's. Contact them for the exact reference: piet at arti.vub.ac.be bernard at arti.vub.ac.be --------------- From marcap at concour.cs.concordia.ca Mon Oct 29 22:19:56 1990 From: marcap at concour.cs.concordia.ca (marcap@concour.cs.concordia.ca) Date: Mon, 29 Oct 90 22:19:56 -0500 Subject: No subject In-Reply-To: Your message of Mon, 29 Oct 90 10:05:52 -0500. <9010291505.AA24026@kovic.IRO.UMontreal.CA> Message-ID: There has been much work done in this area. Your best bet is to look at the first, second, and third conferences on genetic algorithms. McGill has the second conference. CISTI has all three. The third conferences title is proceedings of the third international conference on gentic algorithms. George mason University June 4-7, 1989 Editor: J. david Schaffer ISBN 1-55860-066-3 LIBRARY: QA 402.5.I512 1989 of Congress Hope that helps, MARC From lesher at ncifcrf.gov Mon Oct 29 17:55:02 1990 From: lesher at ncifcrf.gov (lesher@ncifcrf.gov) Date: Mon, 29 Oct 90 17:55:02 EST Subject: // impl GA Message-ID: @techreport{Robertson:87, author = "G. G. Robertson", title = "Parallel Implementation of Genetic Algorithms in a Classifier System", type = "Technical Report Series", number = "RL87-5", institution = "Thinking Machines Corporation", address = "245 First Street, Cambridge, MA 01242-1214", month = "May", year = "1987"} Sarah Lesher From Hiroaki.Kitano at a.nl.cs.cmu.edu Wed Oct 31 12:27:24 1990 Return-Path: Date: Mon, 29 Oct 1990 10:49-EST From: Hiroaki.Kitano at a.nl.cs.cmu.edu To: Yu Shen Subject: Re: Parallel Implementation of Genectic Algorithm and Simulated In-Reply-To: Yu Shen's mail message of Mon, 29 Oct 90 09:59:43 EST We are currently working on massively parallel implementation on GA and Classifier system. By teh end of this year, we will be able to (I hope) have a paper for distribution. Hiroaki Kitano Center for Machine Translation Carnegie Mellon Univeristy, Pittsburgh, PA 15213 U.S.A. ---------------------- %from the bibliography of parallel and supercomputing references, %icarus.riacs.edu (128.102.16.8) in the /pub/bib directory -------------GA %A Elizabeth J. O'Neill %A Craig G. Shaefer %T The ARGOT Strategy III: The BBN Butterfly Multiprocessor %E Joanne L. Martin %E Stephen F. Lundstrom %B Science and Applications: Supercomputing'88 %V II %I IEEE %C Orlando, FL %D 1988 %P 214-227 %K structural analysis, genetic algorithms, applications, %A Chrisila C. Pettey %A Michael R. Leuze %Z CS Dept., Vanderbilt U. %T Parallel Placement of Parallel Processes %J 3rd Conference on Hypercube Concurrent Computers and Applications %V I, Architecture, Software, Computer Systems and General Issues %I ACM %C Pasadena, CA %D January 1988 %P 232-238 %K load balancing and decomposition, parallel genetic algorithms (PGA/GA), simulated annealing, %A Chrisila C. Petty %A Michael R. Leuze %A John J. Grefenstette %T Genetic Algorithms on a Hypercube Multiprocessor %B Hypercube Multiprocessors 1987 %I SIAM %C Philadelphia %D 1987 %P 333-341 ----------classifier %A Stephanie Forrest %Z Teknowledge %T A Study of Parallelism in the Classifier System and its Application to Classification in KL-One Semantic Networks %R PhD thesis %I Logic of Computer Group, University of Michigan %C Ann Arbor, MI, USA %D 1985 %X Received from PARSYM. %A Stephanie Forrest %Z Teknowledge %T The Classifier System: A Computational Model that Supports Machine Intelligence %J Proceedings of the 1986 International Conference on Parallel Processing %I IEEE %D August 1986 %P 711-716 %K Artificial intelligence, KL-one, simulation, pattern recognition, ------------------PSA---------------------------------------- From ray at cs.su.oz.au Tue Oct 30 00:26:01 1990 From: ray at cs.su.oz.au (Raymond Lister) Date: Tue, 30 Oct 90 16:26:01 +1100 Subject: Parallel Implementation of Genectic Algorithm and Simulated Message-ID: I have published a paper on a parallel simulated annealing approach to the Travelling Salesman Problem. A second paper is to appear ... Lister, R (1990), "Segment Reversal and the Traveling Salesman Problem", International Joint Conference on Neural Networks (IJCNN-90-WASH DC), Washington, January 1990. Lister, R (1990), Response to a review by Van den Bout, D "A Parallel, Simulated Annealing Solution to the Traveling Salesman Problem", to appear, Neural Network Review. I'll send you a copy of both papers. I suggest you check the ftp repository set up by Dan Greening (see details below). You may want to talk to Dan directly, as he has written a review paper on parallel simulated annealing (which is in the ftp repository in the file "greening.physicad.ps") ... Greening, D "Parallel Simulated Annealing Techniques" Physica D: Nonlinear Phenomena, Vol. 42, pp 293-306, 1990. And, of course, I'd like to see the results of your survey. Raymond Lister Department of Computer Science, F09 University of Sydney NSW 2006 AUSTRALIA Internet: ray at cs.su.oz.AU --- >From dgreen%cs.ucla.edu at munnari.cs.mu.oz Sat Oct 13 07:50:51 1990 >Date: Fri, 12 Oct 90 14:01:19 pdt >From: Dan R. Greening >To: anneal at cs.ucla.edu, glb at ecegriff.ncsu.edu >Subject: Anonymous FTP Site Available for Annealing Papers. I have set up squid.cs.ucla.edu as an anonymous FTP site for distributing papers on simulated annealing, spin-glass theory, etc. To get the widest audience possible, I would encourage you to put your paper(s) there. Please include a full reference on the first page which indicates where the paper appeared, if it has been published already, such as Appeared in Physica D: Nonlinear Phenomena, vol. 42, pp. 293-306, 1990. If the paper has not yet been published, include some reference which would allow a reader to locate the paper, such as Submitted for publication to the Conference on Advanced Research in VLSI, 1990. Available as IBM Technical Report RC-23456. You may also wish to include electronic mail information in your paper. You may want to announce its availability to the mailing list, by sending a message to anneal at cs.ucla.edu. HOW TO PUT A PAPER ON SQUID: REASONABLE PAPER FORMATS: 1. POSTSCRIPT. Almost everyone has access to a PostScript printer, so unless you absolutely have no other choice, please supply your paper in PostScript form. Append ".ps" to the filename to indicate postscript. 2. TROFF. You should only include troff if you have set up your troff file so that it requires NO COMMAND OPTIONS. Preprocess the troff input so all pic, tbl, eqn stuff is already included. Any macro packages should be included in the file itself. In short, someone should be able to produce your paper using only the file you provide. Append ".troff" to the filename to indicate troff. 3. DVI. You should only include a dvi file if it DOES NOT INCLUDE ENCAPSULATED POSTSCRIPT files (presumably if you have such files, you can generate the whole paper in postscript). Append ".dvi" to the filename to indicate a dvi file. Let's say that you formatted your paper, and have created a postscript file called paper.ps. Furthermore, suppose the first author is "Josie Smith" and you have submitted this paper to IEEE Transactions on CAD. By convention, the paper should be stored on squid as "smith.ieeetcad.ps". You can embellish the name as you wish, however, there is a maximum of 255 characters in a filename. Here goes (from UNIX): % ftp 131.179.96.44 (or ftp squid.cs.ucla.edu) login: anonymous password: anything ftp> cd anneal ftp> binary ftp> put paper.ps smith.ieeetcad.ps ftp> quit Now your paper will be sitting around for anyone to read! You might get famous! HOW TO GET A PAPER FROM SQUID: OK, suppose someone announces the availability of a paper on squid, called "smith.stoc1989.ps". Let's get a copy. Here goes (from UNIX): % ftp 131.179.96.44 (or ftp squid.cs.ucla.edu) login: anonymous password: anything ftp> cd anneal ftp> ls -l (might as well look around while we're here...) ftp> binary ftp> get smith.stoc1989.ps ftp> quit Now, just print out smith.stoc1989.ps and you discover something new! Hooray! I put a couple of my papers on there, already, as well as a paper by Dannie Durand. If you guys are nice (i.e., if you make me feel fulfilled by putting your papers there), maybe I'll put my much-discussed simulated annealing bibliography there, too. % I happen to have a copy of the bibliogaraphy, I think it is the most complete % one. There might be some overlaps between here and there. Yu Shen Happy Annealing (or Tabu Searching or Spinning or Conducting or whatever it is you're doing). -- Dan Greening | NY 914-784-7861 | 12 Foster Court dgreen at cs.ucla.edu | CA 213-825-2266 | Croton-on-Hudson, NY 10520 ------------ From sorkin at ravel.berkeley.edu Mon Oct 29 17:33:51 1990 From: sorkin at ravel.berkeley.edu (sorkin@ravel.berkeley.edu) Date: Mon, 29 Oct 90 14:33:51 -0800 Subject: No subject Message-ID: You should get in touch with two people who have done recent work in the area: Dan Greening (dgreen at cs.ucla.edu) and Dannie Durand (durand at cs.columbia.edu). Both did at least some of their work while at IBM, collaborating with Steve White (white at ibm.com) and Frederica Darema (darema at ibm.com). -- Greg Sorkin (sorkin at ic.berkeley.edu) ---------------- Some of the work by Harold Szu on fast simulated annealing compares Cauchy and Boltzmann machines. References follow in bib form. a:~ (52) tiblook szu cauchy %A Harold H. Szu %T Fast Simulated Annealing %J |AIP151| %P 420-426 %K Cauchy Machine %T Fast Simulated Annealing %A Harold H. Szu %A Ralph Hartley %J |PHYLA| %V 122 %N 3,4 %P 157-162 %D |JUN| 8, 1987 %K Cauchy Machine %T Nonconvex Optimization by Fast Simulated Annealing %A Harold H. Szu %A Ralph L. Hartley %J |IEEPro| %V 75 %N 11 %D |NOV| 1987 %K Cauchy Machine %T Design of Parallel Distributed Cauchy Machines %A Y. Takefuji %A Harold H. Szu %J |IJCNN89| ba:~ (53) tibabb phyla D PHYLA Phys. Lett. A D PHYLA Physics Letters. A ba:~ (54) tibabb IEEPro D IEEPro IEEE Proc. D IEEPro Institute of Electrical and Electronics Engineers. Proceedings ba:~ (55) tibabb IJCNN89 D IJCNN89 International Joint Conference of Neural Networks\ ba:~ (56) ------------------------------------------------------------------------------- . . .__. The opinions expressed herein are soley |\./| !__! Michael Plonski those of the author and do not represent | | | "plonski at aero.org" those of The Aerospace Corporation. _______________________________________________________________________________ @begin(refstyle) Ackley, D. H. (1984, December). @i(Learning evaluation functions in stochastic parallel networks.) CMU Thesis Proposal. ------------- ----------psa %A Emile H. L. Aarts %A Jan H. M. Korst %T Computation in Massively Parallel Networks based on the Boltzmann Machine: A Review %J Parallel Computing %V 9 %N 2 %D January 1989 %P 129-145 %K PARLE: conference on parallel architectures and languages -- Europe, Boltzmann machines, simulated annealing, combinatorial optimization, connectionist networks, neural networks, learning, %A Prithviraj Banerjee %A Mark Jones %T A Parallel Simulated Annealing Algorithm for Standard Cell Placement on Hypercube Computer %J NASA Review of ICLASS: Illinois Computer Laboratory for Aerospace Systems and Software %I University of Illinois %C Urbana-Champaign, IL %D October 23, 1986 %A Prithviraj Banerjee %A Mark Howard Jones %A Jeff S. Sargent %T Parallel Simulated Annealing Algorithms for Cell Placement on Hypercube Multiprocessors %J IEEE Transactions on Parallel and Distributed Systems %V PDS-1 %N 1 %D January 1990 %P 91-106 %K cell placement, error control, hypercube multiprocessor, parallel processing, performance measurements, simulated annealing, VLSI layout, %A Valmir C. Barbosa %A Eli Gafni %T A Distributed Implementation of Simulated Annealing %J Journal of Parallel and Distributed Computing %V 6 %N 2 %D April 1989 %P 411-434 %K special issue: neural computing, research notes, %A S. Wayne Bollinger %A Scott F. Midkiff %Z VA Poly, Blacksburg, VA %T Processor and Link Assignment in Multicomputers using Simulated Annealing %J Proceedings of the 1988 International Conference on Parallel Processing %V I, Architecture %I Penn State %C University Park, Penn %D August 1988 %P 1-7 %K distributed systems, mapping problem, process annealing, connection annealing, hypercube traffic, %A Ken W. Bosworth %A G. S. Stiles %A Rick Pennington %T A Parallel Nonlinear Integer Programming Algorithm Based on Branch and Bound and Simulated Annealing %E Garry Rodrigue %B Parallel Processing for Scientific Computing %I SIAM %C Philadelphia, PA %D 1989 %P 126-130 %K numerical methods, %A Casotto %A et al. %T A parallel simulated annealing algorithm for the placement of macro-cells %J ICCAD, 1986 %A A. Casotto %A A. Sangiovanni-Vincentelli %T Placement of Standard Cells Using Simlated Annealing on a Connection Machine %I Thinking Machines Corp. %R TMC TP CAD86-3 %D Dec. 1986 %K computer aided design, %A F. Darema-Rogers %A S. Kirkpatrick %A V. A. Norton %Z IBM T. J. Watson Research Center, Yorktown Heights, NY %T Simulated Annealing on Shared Memory Parallel Systems %J IBM J. on Res. & Dev. %D 1987 %r RC 12195 (#54812) %d October 1986 %A Nigel Dodd %Z Royal Radar, Malvern %T Graph matching by stochastic optimisation applied to the implementation of multi-layer perceptrons on transputer networks %J Parallel Computing %V 10 %N 2 %D April 1989 %P 135-142 %K graph matching, graph isomorphism, transputer, parallel processing, optimization, stochastic optimization, simulated annealing, MLP, %A F. Ercal %A J. Ramanujam %A P. Sadayappan %Z CIS, OSU %T Task Allocation onto a Hypercube by Recursive Mincut Bipartitioning %J 3rd Conference on Hypercube Concurrent Computers and Applications %V I, Architecture, Software, Computer Systems and General Issues %I ACM %C Pasadena, CA %D January 1988 %P 210-221 %K load balancing and decomposition, mapping problem, ARM (Kernighan/Lin), simulated annealing (SA), %A R. Fiebrich %A C. Wong %Z Thinking Machines Corp. %T Simulated Annealing-Based Circuit Placement Algorithm on the Connection Machine %J IEEE Intl. Conference on Computer Design %C Rye Brook, NY %D October 1987 %r TMC TP CAD87-2 %A G. C. Fox %A W. Furmanski %Z Caltech %T Load Balancing Loosely Synchronous Problems with a Neural Network %J 3rd Conference on Hypercube Concurrent Computers and Applications %V I, Architecture, Software, Computer Systems and General Issues %I ACM %C Pasadena, CA %D January 1988 %P 241-278 %r cccp-363B %d February 1988 %K load balancing and decomposition, particle analogy, graph partitioning, simulated annealing, statistical physics, Monte Carlo, Hopfield & Tank and Bold networks, %A Grover %T A new simulated annealing algorithm for standard cell placements %J ICCAD %D Nov. 1986 %A Harold M. Hastings %A Stefan Waner %Z Hofstra %T Neural Nets on the MPP %J Frontiers of Massively Parallel Scientific Computation %R Conference Proc. 2478 %I NASA Goddard Space Flight Center %C Greenbelt, MD %D Sept. 1986 %D 1987 %P 69-73 %K computer science, neural network, annealing system, stochastic cellular automaton, %A Huang %A et al. %T An efficient general cooling algorithm for simulated annealing %J ICCAD %D 1986 %T Mapping Partitioned Program Modules Onto Multicomputer Nodes Using Simulated Annealing %P II-292--II-293 %A Kai Hwang %A Jian Xu %Z USC, LA %J Proceedings of the 1990 International Conference on Parallel Processing %V II, Software %I Penn State U. Press %C University Park, Penn %D August 1990 %K poster session, Parallel Software Management, %A Daniel G. Jablonski %T Simulated Annealing, Reversible Computation and Neural Nets %I Supercomputing Research Center, IDA %C Lanham, MD %R SRC-TR-88-15 %D July 1988 %A Jepsen %A Gelatt %T Macro placement by Monte Carlo annealing %J ICCD Conf. %D Oct. 1983 %P 495-498 %A A. Kashko %A H. Buxton %A B. F. Buxton %A D. A. Castelow %T Parallel Matching and Reconstruction Algorithms in Computer Vision %J Parallel Computing %V 8 %N 1-3 %D October 1988 %P 3-17 %K Proc. Intl. Conf. on Vector and Parallel Processors in Computational Science, III [VAPP III], Aug. 1987, Liverpool, England. [Cover typo.] computer vision, SIMD, DAP, correlation, relaxation, simulated annealing, graduated non-convexity, %T Parallel Simulated Annealing Using Speculative Computation %P III-286--III-290 %A E. E. Witte %A R. D. Charnberlain %A M. A. Franklin %J Proceedings of the 1990 International Conference on Parallel Processing %V III, Algorithms and Applications %I Penn State U. Press %C University Park, Penn %D August 1990 %K parallel algorithms, ---------------------- Sharpening From jm2z+ at andrew.cmu.edu Mon Oct 29 11:06:38 1990 From: jm2z+ at andrew.cmu.edu (Javier Movellan) Date: Mon, 29 Oct 90 11:06:38 -0500 (EST) Subject: Parallel Implementation of Genectic Algorithm and Simulated In-Reply-To: <9010291459.AA23899@kovic.IRO.UMontreal.CA> References: <9010291459.AA23899@kovic.IRO.UMontreal.CA> Message-ID: Yu, There is a third related procedure called "sharpening". My first contact with the sharpening procedure was through the work of Akiyama et al (see reference) in what they called Gaussian machines ( Continuous Hopfield Networks with Gaussian noise injected in the net inputs). Sharpening is also used in Mean Field Networks (Continuous Hopfield Model with the Contrastive Hebbian Learning Algorithm). Sharpening may be seen as a deterministic, continuous approximation to annealing in Stochastic Boltzmann Machines. It works by starting settling using logistic activations with very low gain and increasing it as settling progresses. Sharpening, contrary to "true annealing" is deterministic and thus it may be faster. A similar procedure is used with elastic networks solving the TSP problem. References: Peterson, C & Anderson J (1987): A mean field theory learning algorithm for neural networks. Complex Systems, 1, 995-1019. Akiyama Y, Yamashita A, Kajiura M, Aiso H (1989) Combinatorial Optimization with Gaussian Machines. Proceedings of the IJCNN, 1, 533-540. Hinton G E (1989) Deterministic Boltzmann Learning Performs Stepest Descent in Weight Space, Neural Computation, 1, 143-150. Galland C, & Hinton G (1989) Deterministic Boltzmann Learing in Networks with Asymetric Connectiviy. University of Toronot. Department of Computer Science Technical Report. CRG-TR-89-6. Movellan J R (1990) Contrastive Hebbian Learning in the Continuous Hopfield Model. Proceedings of the 1990 Connectionist Summer School. Javier From Phil.Hearne at newcastle.ac.uk Mon Oct 1 05:50:26 1990 From: Phil.Hearne at newcastle.ac.uk (Phil.Hearne@newcastle.ac.uk) Date: Mon, 01 Oct 90 10:50:26 +0100 Subject: turing Message-ID: Hello, I'm a PhD student at Newcastle-upon-Tyne university in England and have recently heard about this mail group. Would anyone be kind enough to tell me how I could read the mail and perhaps get a copy of some of the debate on the Turing equivilence of connectionist models? Thanks, Phil Hearne From dario%TECHUNIX.BITNET at VMA.CC.CMU.EDU Mon Oct 1 08:12:00 1990 From: dario%TECHUNIX.BITNET at VMA.CC.CMU.EDU (Dario Ringach) Date: Mon, 1 Oct 90 14:12:00 +0200 Subject: Information-Based Complexity in Vision (request) Message-ID: <9010011212.AA11506@techunix.technion.ac.il> I would be grateful for any references on information-based complexity works (in the sense of [1]) to vision and image processing. Thanks in advance. --Dario [1] J. Traub, et al "Information-Based Complexity", Academic Press, 1988. From kamil at wdl1.wdl.fac.com Mon Oct 1 13:53:48 1990 From: kamil at wdl1.wdl.fac.com (Kamil A Grajski) Date: Mon, 1 Oct 90 10:53:48 -0700 Subject: TR - MasPar Performance Estimates Message-ID: <9010011753.AA23816@wdl1.wdl.fac.com> To receive copy of following tech report send physical address to: kamil at wdl1.fac.ford.com. (TCP/IP #137.249.32.102). ------------------------------------------------------------------------ NEUROCOMPUTING USING THE MasPar MP-1 MASSIVELY PARALLEL PROCESSOR Kamil A. Grajski Ford Aerospace Advanced Development Department / MSX-22 San Jose, CA 95161-9041 (408) 473 - 4394 ABSTRACT We present an evaluation of neurocomputing using the MasPar MP-1, massively parallel processor. Performance figures are obtained on a 2K processor element (PE) machine. Scaling behavior is evaluated for certain cases on a 4K and an 8K PE machine. Extrapolated performance figures are for the full 16K PE machine. Specific neural networks evaluated are: a.) "vanilla" back-propogation, yielding approximately 10 MCUPS real-time learning, (16K machine), for a 256-128-256 network; b.) an Elman-type recurrent network (256-128-256, 1 time delay, 16K machine) yielding approximately 9.5 MCUPS real-time learning; and c.) Kohonen self-organizing feature map yielding 1335 10-dimensional patterns per second on a 2K PE machine only (2048 units), or 27.3 MCUPS. The back-prop networks are mapped as one weight per processor. The Kohonen net is mapped as one unit per PE. The resultant performance figures suggest that for back-prop networks, a single copy, many weights per processor mapping should increase performance. Last, we present basic data transfer and arithmetic benchmarks useful for a priori estimates of machine performance on problems of interest in neurocomputing. ------------------------------------------------------------------------ If you wish to receive additional information on the machine and benchmarks for other types of problems, e.g., image processing, please contact MasPar directly. Or, only if you specifically tell me, I'll pass along your name & area of interest to the right folks over there. From perham at nada.kth.se Tue Oct 2 03:24:34 1990 From: perham at nada.kth.se (Per Hammarlund) Date: Tue, 2 Oct 90 08:24:34 +0100 Subject: Large scale biologically realistic simulations. Message-ID: <9010020724.AA22487@nada.kth.se> Hello, I have a question concerning implementations of programs for biologically realistic simulations on massively parallel machines or other "super" computers. In fact, what I am most interested in is whether anyone has, or knows of, simulations software (for any computer) that is capable of simulating something like 50,000 neurons and in the order of 1,000,000 synapses. I am interested in these programs as we have implemented a program on the Connection Machine, from Thinking Machines Corp, that can do about that, on the 8k machine we have at KTH. The title and abstract of the preliminary report (TeX syntax, please ignore) is enclosed below. Pointers to reports are greatly appriciated! Thank you very much! Per Hammarlund SANS-NADA KTH, Royal Inst. of Tech. S-100 44 Stockholm Sweden ---------------- 8< ---------------- biosim A Program for Biologically Realistic Neural Network Simulations on the Connection Machine Bj\"orn Levin and Per Hammarlund Studies of Artificial Neural Systems Department of Numerical Analysis and Computer Science} TRITA-NA-P9021 This paper describes a program for biologically realistic neural network simulations on the Connection Machine. The biosim program works as a back end to another program, swim, that handles the user interaction and specification of the neural network. The swim program takes a specification of the neural network and generates a parameter file that is read into the biosim program and processed. The output has the same format as the output of the swim program. The purpose of the biosim program is to make simulations of larger networks possible. The neuron model that is used in the simulations is a compartmentalized abstraction of the neuron. One of the compartments acts as the soma, ie the cell body, and the rest act as parts of the dendrite tree. The voltage dependent ion channels are modeled using Hodgkin-Huxley-like equations. The program supports Na+, K+, Ca2+, and Calcium dependent K+ channels. Synaptic interaction includes the voltage gated NMDA receptors and conventional kainate/AMPA receptors. Hooks provided in the code make the addition of new types of channels and receptors extremely easy. The program is capable of handling some tens of thousands of compartments and about ten times that number of synapses on an 8K machine. The numerical method used in solving the differential equations is aimed at speed at the expense of some accuracy. ---------------- 8< ---------------- From cherkaue at cs.wisc.edu Tue Oct 2 09:56:35 1990 From: cherkaue at cs.wisc.edu (Kevin Cherkauer) Date: Tue, 2 Oct 90 08:56:35 -0500 Subject: Mailing list Message-ID: <9010021356.AA29435@perenica.cs.wisc.edu> Please remove me from this list. I am getting to many advertisements. From jose at learning.siemens.com Wed Oct 3 09:33:34 1990 From: jose at learning.siemens.com (Steve Hanson) Date: Wed, 3 Oct 90 09:33:34 EDT Subject: MLP classifiers == Bayes Message-ID: <9010031333.AA18368@learning.siemens.com.siemens.com> can you guys send me a paper copy... thanks. Steve From amnon at ai.mit.edu Wed Oct 3 10:14:14 1990 From: amnon at ai.mit.edu (Amnon Shaashua) Date: Wed, 3 Oct 90 10:14:14 EDT Subject: MLP classifiers == Bayes In-Reply-To: Steve Hanson's message of Wed, 3 Oct 90 09:33:34 EDT <9010031333.AA18368@learning.siemens.com.siemens.com> Message-ID: <9010031414.AA28379@wheat-chex> didn't you get my e-mail yesterday night? I suggested there that instead of using E=(e2,e3,...,e_n,e_1) which brings the set of edges J={(1,2),(2,3),...,(n-1,n),(n,1)} to the main diagonal to use another set J' that corresponds to another predetrmined tour (the choice of J was arbitrary) find J' so that its corresponding E' has distinct roots. For instance if J'={(1,5),(5,3),(3,2),(2,4),(4,1)} then E'=(e_5,e_3,e_2,e_4,e_1) what are the roots of V=(E^t+E)/2 ? -Amnon From koch%CITIAGO.BITNET at VMA.CC.CMU.EDU Wed Oct 3 02:46:51 1990 From: koch%CITIAGO.BITNET at VMA.CC.CMU.EDU (Christof Koch) Date: Tue, 2 Oct 90 23:46:51 PDT Subject: Large scale biologically realistic simulations. In-Reply-To: Your message <9010020724.AA22487@nada.kth.se> dated 2-Oct-1990 Message-ID: <901002234651.204004af@Iago.Caltech.Edu> Have a look at Chapter 12 of the "Methods in Neuronal Modeling" book (C. Koch and I. Segev, eds., MIT Press, 1989). It's entitled "Simulating neurons and networks on parallel computers" by Nelson, Furmanski and Bower. Christof Koch From tsejnowski at ucsd.edu Wed Oct 3 23:19:01 1990 From: tsejnowski at ucsd.edu (Terry Sejnowski) Date: Wed, 3 Oct 90 20:19:01 PDT Subject: Large scale biologically realistic simulations. Message-ID: <9010040319.AA26955@sdbio2.UCSD.EDU> I visited the Royal Institute of Technology in Stockholm recently and saw a demonstration of the lamprey spinal cord model running on the 8-K CM-2 there. The model was developed in collaboration with Sten Grillner at the Karoliska Institute and is one of the best existing models of coupled nonlinear central pattern generators based on realistic models of neurons. The CM-2 demo was impressive because they have solved the problems of mapping a highly nonhomogenious network model with voltage dependent channels onto a bit sliced SIMD machine. The simulation I saw had 100 spinal segments merrily swimming along at about 10% real time. One of the realities of running the CM, however, is that the time required to load a big problem into memory is often much longer than it takes to run the problem. One strategy is to load 100 copies of the same network and run 100 different sets of parameters in parallel. Terry ----- From jls at computer-science.manchester.ac.uk Thu Oct 4 09:46:09 1990 From: jls at computer-science.manchester.ac.uk (jls@computer-science.manchester.ac.uk) Date: Thu, 4 Oct 90 09:46:09 BST Subject: Job announcement - postdoctoral research associate Message-ID: <20050.9010040846@m1.cs.man.ac.uk> POST DOCTORAL RESEARCH ASSOCIATE NEURAL NETWORK THEORY Department of Computer Science University of Manchester Applications are invited for a post-doctoral research position in neural network theory in the computer science department at the University of Manchester. This SERC funded position is concerned with developing methods of predicting neural network performance, including generalisation performance and functionality. The post is tenable for up to three years, starting any time between October 1, 1990 and April 1, 1991. Salary range is between 11,399 pounds and 18,165 pounds according to qualifications and experience. Postgraduate experience in any theoretical aspect of neural networks is desirable, as is demonstrated mathematical ability. Applications and/or informal inquiries should be sent to Jonathan Shapiro, Department of Computer Science, The University, Manchester M13 9PL. United Kingdom. Informal inquiries can be made by phone (061 275 6253) or email (jls at uk.ac.man.cs.m1 within the U.K. and jls%m1.cs.man.ac.uk at nsfnet-relay via internet). The University of Manchester is the oldest and one of the largest computer science department in Britain. Other neural network research at Manchester includes: ex-Static - a project to build a simulator for the design of massively parallel neural computing hardware (in collaboration with a group at Cambridge); a collaboration between theoretical physicists and psychologists to model memory experiments; work on image processing applications in medical biophysics; and secondary protein structure prediction. In addition, Manchester has been established as one of three national Novel Architecture Computing Centers, which means that a large collection of parallel hardware is or will be available. We are an Equal Opportunities Employer From watrous at demon.siemens.com Thu Oct 4 16:20:15 1990 From: watrous at demon.siemens.com (Raymond L Watrous) Date: Thu, 4 Oct 90 16:20:15 -0400 Subject: GRADSIM v2.0 Message-ID: <9010042020.AA17655@demon.siemens.com> ********************* PLEASE DO NOT FORWARD *************************** ********************* PLEASE DO NOT FORWARD *************************** An updated version of the GRADSIM connectionist network simulator is now available. (GRADSIM is general purpose simulator written in C that supports recurrent time-delay network optimization.) The updated simulator (version 2.0) supports second-order links, zero delay links, static pattern matching problems, and mixed unit types. The updated simulator also includes a conjugate gradient optimization module, and supports network link masking, for mixing fixed and variable links. A brief User's Guide that describes simulator modules and compilation options and the original tech report are included, in both .dvi and .ps form. The simulator is available via anonymous ftp from: linc.cis.upenn.edu as /pub/gradsim.v2.tar.Z (This file is about 300K bytes) ********************* PLEASE DO NOT FORWARD *************************** ********************* PLEASE DO NOT FORWARD *************************** From strom at asi.com Fri Oct 5 12:49:42 1990 From: strom at asi.com (Dan Hammerstrom) Date: Fri, 5 Oct 90 09:49:42 PDT Subject: Large scale biologically realistic simulations Message-ID: <9010051649.AA28477@asi.com> Dear Dr. Hammarlund: We noticed your recent posting on the Connectionists' network mailing list. We feel we may have some data that would interest you. We are currently developing a neurocomputer system designed for the high computational demands of neural network simulations. Our system is based around custom VLSI circuits organized in a SIMD (Single Instruction, Multiple Data) architecture. A single board has a peak capability of 12.8 billion multiply accumulate operations per second. In addition to developing a variety of neural network algorithms for this machine, we are also developing, under contract from the Office of Naval Research and in conjunction with the Oregon Graduate Institute and the Center for Learning and Memory at the University of California at Irvine, a real time simulation of a 10,000 neuron slice of olfactory pyriform cortex. The model we are using is that of Gary Lynch and Rick Granger and thier colleagues at UCI. Although it is much more abstract, and less realistic than the models you are working on, it still has a number of interesting properties including the ability to form hierarchical categorizations of the input vector space in an unsupervised mode. In our implementation, a linear array of piriform layer II neurons is mapped to a linear of array of processors. Up to 512 neurons can execute simultaneously in the neurocomputer; this ``slice'' of the network would constitute a single stage of a processor pipeline. Selectively piping the outputs of one slice as inputs to the next would emulate the feed--forward character of the Lateral Olfactory Tract, or LOT. We estimate that a piriform network consisting of approximately 10,000 pyramidal cells and a 512--element LOT could process 500 million eight--bit connections per second. At 10% sparse LOT connectivity, the entire system has roughly 750,000 synapses when lateral inhibitory connections are included. This performance is equivalent to roughly 1000 LOT presentations per second, or 200 presentations per second when the network is in learning mode. (Which is significantly faster than real time, and allows for some interesting research.) Researchers at the Oregon Graduate Institute are currently investigating the applications of the piriform model executing on such speech--processing applications. Please refer to the following for specific information concerning our network architecture and the piriform model (forgive the Latex format): @ARTICLE{piriform, AUTHOR = "Gary Lynch and Richard Granger and J{\'o}se Ambros-Ingerson", TITLE = "Derivation of encoding characteristics of layer 2 cerebral cortex", JOURNAL = "Journal of Cognitive Neuroscience", YEAR = {1989}, VOLUME = {1}, NUMBER = {1}, PAGES = {61--87} } @INCOLLECTION{memorial, AUTHOR = "Richard Granger and J{\'o}se Ambros--Ingerson and Ursula Staubli and Gary Lynch", TITLE = "Memorial Operation of Multiple, Interacting Simulated Brain Structures", BOOKTITLE = "Neuroscience and Connectionist Models", PUBLISHER = {Erlbaum Associates}, YEAR = {1989}, EDITOR = "M. Gluck and D. Rumelhart" } @INPROCEEDINGS{AdaptiveIJCNN, AUTHOR = "Dan Hammerstrom", TITLE = "A VLSI architecture for high-performance, low-cost, on-chip learning", BOOKTITLE = "Proceedings of the 1990 International Joint Conference on Neural Networks", YEAR = {1990}, MONTH = {June} } Sincerely, Eric Means, Adaptive Solutions Dan Hammerstrom, Adaptive Solutions / Oregon Graduate Institute Todd Leen, Oregon Graduate Institute From watrous at demon.siemens.com Mon Oct 8 15:08:56 1990 From: watrous at demon.siemens.com (Raymond L Watrous) Date: Mon, 8 Oct 90 15:08:56 -0400 Subject: GRADSIM v2.0 CORRECTION Message-ID: <9010081908.AA25249@demon.siemens.com> A version control error (mine) resulted in several lines of code being omitted from the recently released GRADSIM 2.0 simulator. The archive at linc.cis.upenn.edu has been updated with a corrected version. The corrected file alone is available as gradsim.patch.Z. Ray Watrous From rjwood at maxine.WPI.EDU Mon Oct 8 16:35:37 1990 From: rjwood at maxine.WPI.EDU (Richard J Wood) Date: Mon, 8 Oct 90 15:35:37 EST Subject: References needed... Message-ID: <9010082035.AA24549@maxine> Does anybody know of work done in object recognition using orientation selective cells? Specifically, I am looking for work done off of R. Linsker's series of papers in 1986 in the Proc. Natl. Acad. Sci. USA (Refs. below). Any references or ideas would be greatly appreciated. Thanks in advance....please send replies to rjwood at maxine.wpi.edu Rick @article{linsker1, Author="R. Linsker", Title="From basic network principles to neural architecture: Emergence of spatial-opponent cells", Journal="Proceedings of the National Academy of Sciences USA", Volume=83, Pages={7508-7512}, Year=1986} @article{linsker2, Author="R. Linsker", Title="From basic network principles to neural architecture: Emergence of orientation-selective cells", Journal="Proceedings of the National Academy of Sciences USA", Volume=83, Pages={8390-8394}, Year=1986} @article{read13, Author="R. Linsker", Title="From basic network principles to neural architecture: Emergence of orientation columns", Journal="Proceedings of the National Academy of Sciences USA", Volume=83, Pages={8779-8783}, Year=1986} From granger at ICS.UCI.EDU Mon Oct 8 15:42:10 1990 From: granger at ICS.UCI.EDU (granger@ICS.UCI.EDU) Date: Mon, 08 Oct 90 12:42:10 -0700 Subject: Large scale biologically realistic simulations In-Reply-To: Your message of Fri, 05 Oct 90 09:49:42 -0700. <9010051649.AA28477@asi.com> Message-ID: <1067.655414930@ics.uci.edu> Dan Hammerstrom's posting of 5 Oct 90 lists three of our recent publications dealing with computational analyses of the physiology and anatomy of the olfactory system. Allow me to add the (relatively recent) reference which contains our derivation of a novel hierarchical clustering mechanism from the combined operation of olfactory bulb and cortex, using a Hebb-like learning rule based on the physiology of synaptic long-term potentiation (LTP) in olfactory cortex: Ambros-Ingerson, J., Granger, R., and Lynch, G. (1990). Simulation of paleocortex performs hierarchical clustering. Science, 247: 1344-1348. It should be noted that none of these articles directly addresses issues of silicon implementation; rather, they provide the computational formalisms used by Hammerstrom et al. to design their VLSI circuits. However, the above Science article does describe the time and space complexity of parallel implementations of the derived network. - Richard Granger, Jose Ambros-Ingerson, Gary Lynch Center for the Neurobiology of Learning and Memory Bonney Center University of California Irvine, California 92717 From cleeremans at TANSY.PSY.CMU.EDU Tue Oct 9 10:54:31 1990 From: cleeremans at TANSY.PSY.CMU.EDU (Axel Cleeremans (Axel Cleeremans)) Date: Tuesday, 09 Oct 90 10:54:31 EDT Subject: Job openings Message-ID: The following is a job annoucement that may be of interest to the connectionist community. Please do not reply directly to this message : I am not affiliated in any way with any of the companies mentioned below. RESEARCH IN SPEECH PROCESSING - BRUSSELS (BELGIUM) Lernout & Hauspie Speech Products, one of the fastest growing companies in Speech Processing, has research and development openings (in the USA and in Europe) for several M.S.'s or Ph.D.'s in Computer Science, Electrical Engineering, or any other AI-related field. We are especially looking for specialists in Digital Signal Processing, Artificial Neural Networks, Microprocessors (Motorola 56000) and Computer Linguistics. Excellent research facilities in a stimulating environment are guaranteed. Please send your resume to : Jan Vandenhende Brains Trust International Boulevard Brand Witlock, 24 B-1200 Brussels Belgium Phone : 011 32 2 735 81 40 Fax : 011 32 2 735 20 75 From birnbaum at fido.ils.nwu.edu Tue Oct 9 14:43:52 1990 From: birnbaum at fido.ils.nwu.edu (Lawrence Birnbaum) Date: Tue, 9 Oct 90 13:43:52 CDT Subject: ML91 deadline extended Message-ID: <9010091843.AA03646@fido.ils.nwu.edu> ML91 -- THE EIGHTH INTERNATIONAL WORKSHOP ON MACHINE LEARNING DEADLINE FOR WORKSHOP PROPOSALS EXTENDED To make life a little easier, the deadline for workshop proposals for ML91 has been extended by a few days. The new deadline is MONDAY, OCTOBER 15. Please send proposals by email to: ml91 at ils.nwu.edu or by hardcopy to the following address: ML91 Northwestern University The Institute for the Learning Sciences 1890 Maple Avenue Evanston, IL 60201 USA fax (708) 491-5258 Please include the following information: 1. Workshop topic 2. Names, addresses, and positions of workshop committee members 3. Brief description of topic 4. Workshop format 5. Justification for workshop, including assessment of breadth of appeal On behalf of the organizing committee, Larry Birnbaum Gregg Collins Program co-chairs, ML91 From pollack at cis.ohio-state.edu Tue Oct 9 16:21:34 1990 From: pollack at cis.ohio-state.edu (Jordan B Pollack) Date: Tue, 9 Oct 90 16:21:34 -0400 Subject: Position Announcement Message-ID: <9010092021.AA09069@dendrite.cis.ohio-state.edu> Computational Neuroscience The Ohio State University For one of several open tenure-track faculty positions in Cognitive Science, the Ohio State University is seeking a computational neuroscientist. The successful candidate will hold a recent Ph.D and have a demonstrated record of research accomplishment in biologically realistic computational modelling of nervous systems. The Center for Cognitive Science is an interdisciplinary University- wide research center with approximately eighty members from sixteen departments. The faculty appointment will be in an academic department that is appropriate to the interests and background of the candidate. OSU is one of the largest universities in the country with significant research resources including a Cray YMP and a PET scanner. OSU's Ph.D. Program in Neuroscience involves some 75 faculty members from over a half dozen colleges. Columbus provides a high quality of life along with very affordable housing. To apply, please send your vita and a statement of research and teaching interests, and arrange for three recommendation letters to be sent to: Computational Neuroscience Search Committee Center for Cognitive Science 208 Ohio Stadium East 1961 Tuttle Park Place Columbus, OH 43210-1102 The Ohio State University encourages diversity in its faculty, and is an Equal Opportunity/Affirmative Action employer. From MURTAGH at SCIVAX.STSCI.EDU Thu Oct 11 10:30:44 1990 From: MURTAGH at SCIVAX.STSCI.EDU (MURTAGH@SCIVAX.STSCI.EDU) Date: Thu, 11 Oct 1990 10:30:44 EDT Subject: Workshop announcement: "NNs for Statistical & Economic Data" Message-ID: <901011103044.26014ad8@SCIVAX.STSCI.EDU> Workshop: "Neural Networks for Statistical and Economic Data" Place/date: Dublin, Ireland. December 10-11 1990. The workshop seeks to bring together those working towards applications of artificial neural networks, and those concerned with regularities, in statistical and ecomomic data. A number of invited speakers will also review closely-related domains such as nonlinear time series analysis, and complex questions in economics and economic statistics. On December 10 there will be a series of tutorial and review presentations. On December 11 there will be both invited and contributed working papers. Attendance at the workshop is limited, priority being given to those presenting new results. Sponsor: EUROSTAT/Statistical Office of the European Communities, Luxembourg. Organization: Munotec Systems Ltd., 35 St. Helen's Road, Booterstown, Co. Dublin, Ireland. Information can also be requested from: F. Murtagh, murtagh at dgaeso51.bitnet, or murtagh at scivax.stsci.edu. From reggia at cs.UMD.EDU Thu Oct 11 14:53:53 1990 From: reggia at cs.UMD.EDU (James A. Reggia) Date: Thu, 11 Oct 90 14:53:53 -0400 Subject: Call for papers: neural nets for diagnosis Message-ID: <9010111853.AA13403@brillig.cs.UMD.EDU> [[Note: I was asked to bring the following to the attention of anyone using neural modelling methods for diagnostic problem solving.]] *CALL FOR PAPERS* Second International Workshop on Principles of Diagnosis Milano (Italy), October 14-15-16, 1991 Organized by CISE Tecnologie Innovative and Dipartimento di Informatica of Universita` di Torino This workshop (which follows the successful one held at Stanford University in 1990) encourages intensive and high quality interaction and cooperation among researchers with a diversity of artificial intelligence approaches to diagnosis. Attendance will be limited to fifty participants with presentations spread over three days. Substantial time will be reserved for discussion. To attend, participants should submit papers (maximum 5000 words) to be reviewed by the committee. Submissions are welcomed on (but not limited to) the following topics: - Theory of diagnosis (abductive vs. deductive diagnosis, isolation vs. identification, diagnosis on non-monotonic theories, diagnosis of dynamic systems,...) - Computational issues (controlling the combinatorial explosion, focusing strategies, controlling diagnostic reasoning of complex systems, ...) - Modeling for diagnosis (multiple, approximate, probabilistic and qualitative models, integrating model-based diagnosis with heuristics ....) - Evaluation of theories on practical applications - Inductive approaches to diagnosis (Case-Based Reasoning, Neural Nets, ...) Accepted papers can be revised for inclusion in the workshop working notes. Although work published elsewhere is acceptable, new original work is preferred. Please send five copies of each submission to the chairman at the postal address below. Include several ways of contacting the principal author in addition to a postal address: electronic mail, fax and telephone numbers are preferred, in that order. Please indicate with your submission if you wish to make a presentation or only to attend. Submissions received after 3 May 1991 will not be considered. The decisions of the committee will be mailed by 1 July 1991. Chairman: Luca Console Dipartimento di Informatica - Universit` di Torino Corso Svizzera 185, 10149 Torino (Italy) E-mail: lconsole at itoinfo.bitnet Fax: (+39) 11 751603 Tel.: (+39) 11 771 2002 Committee: I. Bratko (U. Ljubljana), P. Dague (IBM), J. de Kleer (Xerox), G. Guida (U. Brescia), K. Eshghi (HP), W. Hamscher (Price Waterhouse), M. Kramer (MIT), W. Nejdl (U. Wien), J. Pearl (UCLA), D. Poole (U. British Columbia), O. Raiman (Xerox), J. Reggia (U. Maryland), J. Sticklen (Michigan State U.), P. Struss (Siemens), P. Szolovits (MIT), G. Tornielli (CISE). Organizing Committee: M. Migliavacca (CISE, chairman), M. Gallanti (CISE), A. Giordana (U. Torino), L. Lesmo (U. Torino). Secretarial Support: A. Camnasio, CISE, P.O. Box 12081, 20134 Milano, Tel (+39) 2 21672400, Fax (+39) 2 26920587. This workshop is sponsored by AI*IA. Sponsorship required to AAAI and ECCAI From jordan at psyche.mit.edu Thu Oct 11 15:24:11 1990 From: jordan at psyche.mit.edu (Michael Jordan) Date: Thu, 11 Oct 1990 15:24:11 EDT Subject: technical report Message-ID: The following technical report is available: Forward Models: Supervised Learning with a Distal Teacher Michael I. Jordan Massachusetts Institute of Technology David E. Rumelhart Stanford University MIT Center for Cognitive Science Occasional Paper #40 Abstract Internal models of the environment have an important role to play in adaptive systems in general and are of particular importance for the supervised learning paradigm. In this paper we demonstrate that certain classical problems associated with the notion of the ``teacher'' in supervised learning can be solved by judicious use of learned internal models as components of the adaptive system. In particular, we show how supervised learning algorithms can be utilized in cases in which an unknown dynamical system intervenes between actions and desired outcomes. Our approach applies to any supervised learning algorithm that is capable of learning in multi- layer networks. Copies can be obtained in one of two ways: (1) ftp a postscript copy from cheops.cis.ohio-state.edu. The file is jordan.forward-models.Z in the pub/neuroprose directory. You can either use the Getps script or follow these steps: unix:1> ftp cheops.cis.ohio-state.edu Connected to cheops.cis.ohio-state.edu. Name (cheops.cis.ohio-state.edu:): anonymous 331 Guest login ok, send ident as password. Password: neuron 230 Guest login ok, access restrictions apply. ftp> cd pub/neuroprose ftp> binary ftp> get jordan.forward-models.ps.Z ftp> quit unix:2> uncompress jordan.forward-models.ps.Z unix:3> lpr jordan.forward-models.ps (2) Order a hardcopy from bonsaint at psyche.mit.edu or hershey at psych.stanford.edu. (use a nearest-geographic-neighbor rule). Please use this option only if option (1) is not feasible. Mention the "Forward Models" technical report. --Mike Jordan From carol at ai.toronto.edu Fri Oct 12 07:22:50 1990 From: carol at ai.toronto.edu (Carol Plathan) Date: Fri, 12 Oct 1990 16:22:50 +0500 Subject: CRG-TR-90-5 request Message-ID: <90Oct12.162252edt.292@neuron.ai.toronto.edu> PLEASE DO NOT FORWARD TO OTHER NEWSGROUPS OF MAILING LISTS ********************************************************** The following technical report is now available. You can get it from carol at ai.toronto.edu. Send your real mail address (omitting all other information from your message). --------------------------------------------------------------------------- COMPETING EXPERTS: AN EXPERIMENTAL INVESTIGATION OF ASSOCIATIVE MIXTURE MODELS Steven J. Nowlan Department of Computer Science University of Toronto Toronto, Canada M5S 1A4 CRG-TR-90-5 Supervised algorithms, such as back-propagation, have proven capable of discovering clever internal representations of the information necessary for performing some task, while ignoring irrelevant detail in the input. However, such supervised algorithms suffer from problems of scale and interference between tasks when used to perform more than one task, or a complex task which is a disjunction of many simple subtasks. To address these problems, several authors have proposed modular systems consisting of multiple networks (Hampshire & Waibel 1989, Jacobs, Jordan & Barto 1990, Jacobs, Jordan, Nowlan, & Hinton, 1990). In this paper, we discuss experimental investigations of the model introduced by Jacobs, Jordan, Nowlan & Hinton, in which a number of simple expert networks compete to solve distinct pieces of a large task; each expert has the power of a supervised algorithm to allow it to discover clever task specific internal representations, while an unsupervised competitive mechanism decomposes the task into easily computable subtasks. The competitive mechanism is based on the mixture view of competition discussed in (Nowlan 1990, Nowlan & Hinton), and the entire system may be viewed as an associative extension of a model consisting of a mixture of simple probability generators. The task decomposition and training of individual experts are performed in parallel, leading to an interesting non-linear interaction between these two processes. Experiments on a number of simple tasks illustrate resistance to task interference, the ability to discover the ``appropriate'' number of subtasks, and good parallel scaling performance. The system of competing experts is also compared with an alternate formulation, suggested by the work of (Jacobs, Jordan, and Barto 1990), which allows cooperation rather than competition between a number of simple expert networks. Results are also described for a phoneme discrimination task, which reveals an ability for a system of competing experts to uncover interesting subtask structure in a complex task. References: J. Hampshire and A. Waibel, "The Meta-Pi network: Building distributed knowledge representations for robust pattern recognition." Technical Report CMU-CS-89-166, School of Computer Science, Carnegie Mellon University, 1989. R. A. Jacobs, M. I. Jordan, and A. G. Barto, "Task decomposition through competition in a modular connectionist architecture." Cognitive Science, 1990. In Press. R. A. Jacobs, M. I. Jordan, S. J. Nowlan and G. E. Hinton, "Adaptive mixtures of local experts" Neural Computation, 1990. In Press. S. J. Nowlan, 1990. "Maximum Likelihood Competitive Learning" in Neural Information Processing Systems 2, D. Touretzky (ed.), Morgan Kauffmann, 1990. S. J. Nowlan and G. E. Hinton, "The bootstrap Widrow-Hoff rule as a cluster formation algorithm" Neural Computation 2:3, 1990. ------------------------------------------------------------------------------ From lss at compsci.stirling.ac.uk Wed Oct 17 14:49:58 1990 From: lss at compsci.stirling.ac.uk (Dr L S Smith (Staff)) Date: 17 Oct 90 14:49:58 BST (Wed) Subject: No subject Message-ID: <9010171449.AA15213@uk.ac.stir.cs.tugrik> COGNITIVE SCIENCE/HCI INITIATIVE Department of Psychology University of St Andrews Scotland, UK and Centre for Cognitive and Computational Neuroscience University of Stirling Scotland, UK 2 POST-DOCTORAL RESEARCH FELLOWSHIPS investigating psychological and neurocomputational processes of visual word recognition. This project represents a major collaboration between the Department of Psychology, St Andrews (a leading centre for human perceptual research) and the CCCN, Stirling (a leading centre for neural network research) to develop a new computational model of visual word recognition. Applications are invited for the following 2 post-doctoral fellowships within the project: Post 1 (tenured for 3 years, based at Department of Psychology, St Andrews University) will involve developing fresh perspectives on the neural modelling of visual word recognition from human experimentation. The data from these experiments will form the basis for the computational modelling in the project. Applicants should have experience in human experimentation in cognitive science or perceptual research, be well acquainted with the use of computers in experimentation, and have some knowledge of neural network research. Post 2 (tenured for 2 years, based at Centre for Cognitive and Computational Neuroscience, Stirling University) will involve setting up and developing a new computational model of visual word recognition which combines the findings from St Andrews with fresh perspectives on neurocomputational processing. Applicants should have experience or interest in neural computation/connectionism and have a background in one or more of the following: computing science, psychology, mathematics, physics. Starting salary for each post will be on the 1A scale for research staff (up to \pounds 18165 pa). Both posts are scheduled to start as soon as possible in 1991. Application forms and further particulars for both posts can be obtained from The Director of Personnel Services, College Gate, St Andrews University, St Andrews, Fife, KY16 9AJ, to whom completed applications forms together with a CV should be submitted to arrive no later than November 30th 1990. Further information can be obtained informally from: (Post 1) Dr Tim Jordan at St Andrews (tel.0334 76161, ext 7234) (Post 2) Dr Leslie Smith at Stirling (tel.0786 67435, direct line) Previous applicants for these posts need not re-apply. Both Universities operate an Equal Opportunities Policy. From sontag at hilbert.rutgers.edu Thu Oct 18 15:43:19 1990 From: sontag at hilbert.rutgers.edu (Eduardo Sontag) Date: Thu, 18 Oct 90 15:43:19 EDT Subject: WORKSHOP ON THEORETICAL ISSUES IN NEURAL NETS, May 20-23, 1991 Message-ID: <9010181943.AA01637@hilbert.rutgers.edu> WORKSHOP ON THEORETICAL ISSUES IN NEURAL NETS Announcement and Call for Contributions The Center for Discrete Mathematics and Theoretical Computer Science (DIMACS) will host a workshop on "Theoretical Issues in Neural Nets" at Rutgers University, for four days, May 20-23, 1991. This will be a mathematically oriented meeting, where technical issues can be discussed in depth. The objective is to have a Workshop that brings together people interested in a serious study of foundations -- plus a few people who will give expository lectures on applied problems and biological nets. The area is of course very diverse, and precisely because of this it might be worth trying to search for conceptual unity in the context of the Workshop. A preliminary list of main speakers is as follows (with tentative topics listed, when available): Dave Ackley, Bellcore (Genetic algorithms: Evolution and learning) Andrew Barron, U. Illinois (Statistical selection of neural net architectures) Andy Barto, U. Mass. (Expository talk: Learning & incrmntl dynamic programming) Eric Baum, NEC Institute (Expository talk: Sample complexity) Ed Blum, USC (Feed-forward networks and approximation in various norms) Roger Brockett, Harvard (Combinatorial optimization via steepest descent) George Cybenko, U. Illinois Merrick Furst, CMU (Circuit complexity & harmonic analysis of Boolean functs) Herbert Gish, BBN (Maximum likelihood training of neural networks) Stephen Grossberg, Boston U. (Expository talk) Steve Hanson, Siemens (Expository talk: Human learning and categorization) Moe Hirsch, Berkeley (Expository talk: Network dynamics) Wolfgang Maass, U. Ill./Chicago (Boltzmann machines for classification) John Moody, Yale Sara Solla, Bell Labs (Supervised learning and statistical physics) Santosh S. Venkatesh, Penn Hal White, UCSD The organizing committee consists of Bradley W. Dickinson (Princeton), Gary M. Kuhn (Institute for Defense Analyses), and Eduardo D. Sontag and Hector J. Sussmann (Rutgers). DIMACS is a National Science Foundation Science and Technology Center, established as a cooperative project between Rutgers University, Princeton University, AT&T Bell Laboratories, and Bellcore. Its objectives are to carry out basic and applied research in discrete mathematics and theoretical computer science. The center provides excellent facilities for workshop participants, including offices and computer support. If you are interested in participating in this workshop, please send a message to Eduardo at sontag at hilbert.rutgers.edu. If you would like to give a talk, please e-mail a title and abstract to the above address by January 15th, 1991. Please keep the abstract short, but give references to published work if appropriate. (Use plain TeX, LaTeX, or a text file; please do not use snailmail.) There is a possibility of proceedings being published, but nothing has been decided in that regard. If you are interested in attending but not talking, send a note explaining your interest in the area. The committee will try to accommodate as many participants and as many talks as possible, but the numbers may have to be limited in order to achieve a relaxed workshop atmosphere conducive to interactions among participants. Notification of people concerning attendance is expected about the middle of February. From codelab at psych.purdue.edu Thu Oct 18 16:58:56 1990 From: codelab at psych.purdue.edu (Coding Lab Wasserman) Date: Thu, 18 Oct 90 15:58:56 EST Subject: room Message-ID: <9010182058.AA15575@psych.purdue.edu> I am looking for a place to stay during the Society for Neuroscience meeting in St. Louis. I am a graduate student on a limited travel budget and I wish to share any sort of inexpensive accommodations with someone else. I need a place from Sunday, October 28th to Wednesday, October 31st. Please reply to: Zixi Cheng codelab at psych.purdue.edu From rob at galab2.mh.ua.edu Fri Oct 19 23:43:54 1990 From: rob at galab2.mh.ua.edu (Robert Elliott Smith) Date: Fri, 19 Oct 90 22:43:54 CDT Subject: text availability? Message-ID: <9010200343.AA08001@galab2.mh.ua.edu> Hi, I have acquired the responsibility of teaching a introductory neural nets course to an audience of graduate students from a variety of engineering disciplines. This course was previously taught from short-course notes with the PDP books as additional material. I would prefer to teach this course from a text. Several publishers have told me that they will have such books soon4{, but the currently available texts I've seen aren't very satisfactory. Does anyone know of a good text that is available (or will be before spring)? Please respond to me directly. All comments appreciated. Rob. ------------------------------------------- Robert Elliott Smith Department of Engineering of Mechanics The University of Alabama P. O. Box 870278 Tuscaloosa, Alabama 35487 <> rob at galab2.mh.ua.edu <> (205) 348-4661 ------------------------------------------- From ngr at cs.exeter.ac.uk Thu Oct 18 12:26:59 1990 From: ngr at cs.exeter.ac.uk (Niall Griffith) Date: Thu, 18 Oct 90 17:26:59 +0100 Subject: MUSIC Message-ID: <3121.9010181626@expya.cs.exeter.ac.uk> I am working at the Connection Science lab at Exeter, and I am writing a review of connectionist research on music. It would be really useful if you could send me as many references as you have on the subject. I will of course make these publicly available. Niall Griffith Centre for Connection Science JANET: ngr at uk.ac.exeter.cs Dept. Computer Science University of Exeter UUCP: ngr at expya.uucp Exeter EX4 4PT DEVON BITNET: ngr at cs.exeter.ac.uk@UKACRL UK Niall Griffith Centre for Connection Science JANET: ngr at uk.ac.exeter.cs Dept. Computer Science University of Exeter UUCP: ngr at expya.uucp Exeter EX4 4PT DEVON BITNET: ngr at cs.exeter.ac.uk@UKACRL UK From gluck%psych at Forsythe.Stanford.EDU Sat Oct 20 14:27:57 1990 From: gluck%psych at Forsythe.Stanford.EDU (Mark Gluck) Date: Sat, 20 Oct 90 11:27:57 PDT Subject: Two Preprints: Generalization & Representation, Sensorimotor Learning Message-ID: <9010201827.AA28023@psych> TWO PRE-PRINTS AVAILABLE: 1) Stimulus Generalization and Representation in Adaptive Network Models of Category Learning 2) Sensorimotor Learning and the Cerebellum. _________________________________________________________________ Gluck, M. A. (1991, in press). Stimulus generalization and representation in adaptive network models of category learning To appear in : Psychological Science. Abstract An exponential-decay relationship relationship between the proba- bility of generalization and psychological distance has received considerable support from studies of stimulus generalization (Shepard, 1958) and categorization (Nosofsky, 1984). It is shown here how an approximate exponential generalization gradient em- erges in a "configural-cue" network model of human learning that represents stimulus patterns in terms of elementary features and pair-wise conjunctions of features (Gluck & Bower, 1988b; Gluck, Bower, & Hee, 1989) from stimulus representation assumptions iso- morphic to a special case of Shepard's (1987) theory of stimulus generalization. The network model can be viewed as a combination of Shepard's theory and an associative learning rule derived from Rescorla and Wagner's (1972) theory of classical conditioning. _________________________________________________________________ Bartha, G. T., Thompson, R. F., & Gluck, M. A. (1991, in press) Sensorimotor learning and the cerebellum. In M. A. Arbib and J.-P. Ewert (Eds.), Visual Structures and Integrated Functions, Springer Research Notes in Neural Computing, Berlin: Springer-Verlag. Abstract This paper describes our current work on integrating experimental and theoretical studies of a simple form of sensorimotor learn- ing: the classically conditioned rabbit eyelid closure response. We first review experimental efforts to determine the neural basis of the conditioned eyelid closure response and these sup- port the role of the cerebellum as the site of the memory trace. Then our current work to bring the modeling in closer contact with the biology is described. In particular, we extend our ear- lier model of response topography to be more physiological in the circuit connectivity, the learning algorithm, and the conditioned stimulus representation. The results of these extensions include a more realistic conditioned response topography and reinforce- ment learning which accounts for an experimentally established negative feedback loop. _________________________________________________________________ To request copies, send email to: gluck at psych.stanford.edu with your hard-copy mailing address. Or mail to: Mark A. Gluck, Department of Psychology, Jordan Hall, Bldg. 420, Stanford Univ., Stanford, CA 94305-2130 From jenkins%SIPI.USC.EDU at VMA.CC.CMU.EDU Fri Oct 19 16:38:44 1990 From: jenkins%SIPI.USC.EDU at VMA.CC.CMU.EDU (Keith Jenkins) Date: Fri, 19 Oct 1990 13:38:44 PDT Subject: OC91 announcement on the other newsletter Message-ID: MEETING ANNOUNCEMENT FOURTH TOPICAL MEETING ON OPTICAL COMPUTING Salt Lake City Marriott Salt Lake City, Utah March 4-6, 1991 Optical Society of America ABSTRACT AND SUMMARY DEADLINE: NOVEMBER 14, 1990 -- -- -- -- -- -- -- SCOPE The Optical Computing topical meeting will consist of invited papers and both oral and poster contributed papers. Contributions are solicited in all areas of research in materials, devices, architectures and algorithms relevant to optical computing. Topics of interest include: 1. Optical interconnections and buses for computing. 2. Optical and photonic computing systems, architectures, models and algorithms, digital or analog. 3. Hybrid optical/electronic processors. 4. Optical "neural" processors, including optical associative memories. 5. Optical memory. 6. Massively parallel architectures for optical implementation. 7. Spatial light modulators and other devices for optical computing. 8. Nonlinear optical phenomena of potential use in computing. 9. Nonlinear optical, electro-optical, and opto-electronic components. 10. Areas of application of the above components and processors. 11. Fundamental physical and computational properties relating to the capabilities and limitations of optical computers. -- -- -- -- -- -- -- -- -- -- - CONFERENCE COCHAIRS: C. Lee Giles NEC Research Institute Sing H. Lee University of California, San Diego PROGRAM CHAIR: B. Keith Jenkins University of Southern California -- -- -- -- -- -- -- -- -- -- - INVITED TALKS (Partial List) P. Bruce Berra Syracuse University "Optical Database Machines" K. Kasahara NEC Corporation "Progress in Arrays of Opto-electronic Bistable Devices and Sources" Bart Kosko University of Southern California "Adaptive Fuzzy Systems" Demetri Psaltis California Institute of Technology "Learning in Optical Neural Networks". Wilfrid B. Veldkamp MIT Lincoln Laboratory "Binary Optics and Applications" -- -- -- -- -- -- -- -- -- -- - RELATED MEETINGS Four topical meetings are being held in Salt Lake City during the two- week period of March 4-15, 1991. The meetings are: Optical Computing, Photonic Switching, Picosecond Electronics and Optoelectronics, and Quantum Wells for Optics and Optoelectronics. -- -- -- -- -- -- -- -- -- -- -- FOR MORE INFORMATION: Contact Optical Society of America Meetings Department 2010 Massachusetts Ave., NW Washington, DC 20036 USA Tel. (202) 223-0920 Fax (202) 416-6100 From amini at tcville.hac.com Sun Oct 21 22:46:03 1990 From: amini at tcville.hac.com (Afshin Amini) Date: Sun, 21 Oct 90 19:46:03 PDT Subject: job in signal processing and neural net Message-ID: <9010220246.AA05941@ai.spl> Hi there I received a job posting on the net, about two weeks ago. The job advertised opennings in signal processing, neural net and related AI fields. The main company was in Belgium. Unfortunately I did not save that message and do not have the name of the company nor the email. If anybody has saved the info, please email it to me. thanks, -aa -- ---------------------------------------------------------------------------- Afshin Amini Hughes Aircraft Co. voice: (213) 616-6558 Electro-Optical and Data Systems Group Signal Processing Lab fax: (213) 607-0918 P.O. Box 902, EO/E1/B108 email: El Segundo, CA 90245 smart: amini at tcville.hac.com Bldg. E1 Room b2316f dumb: amini%tcville at hac2arpa.hac.com uucp: hacgate!tcville!dave ---------------------------------------------------------------------------- From paul at axon.Colorado.EDU Tue Oct 23 12:04:58 1990 From: paul at axon.Colorado.EDU (Paul Smolensky) Date: Tue, 23 Oct 90 10:04:58 -0600 Subject: Connectionist Faculty Position at Boulder Message-ID: <9010231604.AA06346@axon.Colorado.EDU> The Institute for Cognitive Science at the University of Colorado, Boulder has an opening for which connectionists are invited to apply. As you can see from the official ad below, applications in another field are also being invited. However, should this year's position go to a non-connectionist, we expect another position next year and a search will be held specifically for a connectionist. We would be more than happy to answer any questions you may have... Paul Smolensky & Mike Mozer ------------------------------------- Faculty Position in Cognitive Science The Institute of Cognitive Science at the University of Colorado at Boulder invites applications for a tenured/tenure-track position, either in the area of connectionism or in the area of knowledge-based systems or cooperative problem solving. The position is open as to rank. An important selection criterion will be the candidate's potential to contribute to the Institute's interdisciplinary missions in research, teaching, and service. Candidates in the connectionist area should have demonstrated ability to contribute to connectionist theory as well as connectionist approaches to cognitive science. Candidates in the knowledge based systems or cooperative problem solving area should have an interest in large scale system building efforts and software technologies and tools. The position will be housed in an appropriate academic department associated with the Institute of Cognitive Science (e.g., Computer Science, Linguistics, Philosophy, or Psychology). A resume and three letters of reference should be sent to: Dr. Martha Polson, Assistant Director, Institute of Cognitive Science, University of Colorado, Boulder, Colorado, 80309-0345 by January 18, 1990. The University of Colorado at Boulder has a strong commitment to the principle of diversity in all areas. In that spirit, we are particularly interested in receiving applications from a broad spectrum of people, including women, members of ethnic minorities and disabled individuals. From pablo at cs.washington.edu Wed Oct 24 16:55:02 1990 From: pablo at cs.washington.edu (David Cohn) Date: Wed, 24 Oct 90 13:55:02 -0700 Subject: Shared Accommodations at NIPS Message-ID: <9010242055.AA28215@june.cs.washington.edu> I'm interested in matching up with other starving graduate students (or starving faculty, etc.) who are going to NIPS (Neural Information Processing Systems) in Denver at the end of November and would be interested in cutting costs by sharing a room. If you're interested, send e-mail with preferences (such as no smoking, quiet, late riser, etc.) to pablo at cs.washington.edu; I will assemble a list and send out a copy to everyone who writes to me. Hopefully, we can self-organize :-) and keep this as much out of the way of the connectionist mailing list as possible so that we limit noise (to people who *aren't* going to the conference. Thanks, -David "Pablo" Cohn e-mail: pablo at cs.washington.edu Dept. of Computer Science, FR-35 phone: (206) 543-7798 University of Washington Seattle, WA 98195 From jose at learning.siemens.com Thu Oct 25 08:38:10 1990 From: jose at learning.siemens.com (Steve Hanson) Date: Thu, 25 Oct 90 08:38:10 EDT Subject: CNS RFPs Message-ID: <9010251238.AA19808@learning.siemens.com.siemens.com> McDonnell-Pew Program in Cognitive Neuroscience October 1990 Individual Grants-in-Aid for Research and Training Supported jointly by the James S. McDonnell Foundation and The Pew Charitable Trusts INTRODUCTION The McDonnell-Pew Program in Cognitive Neuroscience has been created jointly by the James S. McDonnell Foundation and The Pew Charitable Trusts to promote the development of cognitive neuroscience. The foundations have allocated $12 million over an initial three-year period for this program. Cognitive neuroscience attempts to understand human mental events by specifying how neural tissue carries out computations. Work in cognitive neuroscience is interdisciplinary in character, drawing on developments in clinical and basic neuroscience, computer science, psychology, linguistics, and philosophy. Cognitive neuroscience excludes descriptions of psychological function that do not address the underlying brain mechanisms and neuroscientific descriptions that do not speak to psychological function. The program has three components. (1) Institutional grants have been awarded for the purpose of creating centers where cognitive scientists and neuroscientists can work together. (2) To encourage Ph.D. and M.D. investigators in cognitive neuroscience, small grants-in-aid will be awarded for individual research projects. (3) To encourage Ph.D. and M.D. investigators to acquire skills for interdisciplinary research, small training grants will be awarded. During the program's initial three-year period, approximately $4 million will be available for the latter two components -- individual grants-in-aid for research and training -- which this brochure describes. RESEARCH GRANTS The McDonnell-Pew Program in Cognitive Neuroscience will issue a limited number of awards to support collaborative work by cognitive neuroscientists. Applications are sought for projects of exceptional merit that are not currently fundable through other channels and from investigators who are not at institutions already funded by an institutional grant from the cognitive neuroscience program. Preference will be given to projects requiring collaboration or interaction between at least two subfields of cognitive neuroscience. The goals are to encourage broad national participation in the development of the field and to facilitate the participation of investigators outside the major centers of cognitive neuroscience. Submissions will be reviewed by the program's advisory board. Grant support under this component is limited to $30,000 per year for two years. Indirect costs are to be included in the $30,000 maximum and may not exceed 10 percent of salaries and fringe benefits. Grants are not renewable after two years. The program is looking for innovative proposals that would, for example: * combine experimental data from cognitive psychology and neuroscience; * explore the implications of neurobiological methods for the study of the higher cognitive processes; * bring formal modeling techniques to bear on cognition; * use sensing or imaging techniques to observe the brain during conscious activity; * make imaginative use of patient populations to analyze cognition; * develop new theories of the human mind/brain system. This list of examples is necessarily incomplete but should suggest the general kind of proposals desired. Ideally, a small grant-in-aid for research should facilitate the initial exploration of a novel or risky idea, with success leading to more extensive funding from other sources. TRAINING GRANTS A limited number of grants will also be awarded to support training investigators in cognitive neuroscience. Here again, the objective is to support proposals of exceptional merit that are underfunded or unlikely to be funded from other sources. Training grants to support Ph.D. thesis research of graduate students will not be funded. Some postdoctoral awards for exceptional young scientists will be available; postdoctoral stipends will be funded for up to three years at prevailing rates at the host institution. Highest priority will be given to candidates seeking postdoctoral training outside the field of their previous training. Innovative programs for training young scientists, or broadening the experience of senior scientists, are also encouraged. Some examples of appropriate proposals follow. * Collaboration between a junior scientist in a relevant discipline and a senior scientist in a different discipline has been suggested as an effective method for developing the field. * Two senior scientists might wish to learn each other's discipline through a collaborative project. * An applicant might wish to visit several laboratories in order to acquire new research techniques. * Senior researchers might wish to investigate new methods or technologies in their own fields that are unavailable at their home institutions. Here again, examples can only suggest the kind of training experience that might be considered appropriate. APPLICATIONS Applicants should submit five copies of a proposal that does not exceed 5,000 words. Proposals for research grants should include: * a description of the work to be done and where it might lead; * an account of the investigator's professional qualifications to do the work. Proposals for training grants should include: * a description of the training sought and its relationship to the applicant's work and previous training; * a statement from the mentor as well as the applicant concerning the acceptability of the training plan. Proposals for both research grants and training grants should include: * an account of any plans to collaborate with other cognitive neuroscientists; * a brief description of the available research facilities; The proposal must be accompanied by the following separate information: * a brief, itemized budget and budget justification for the proposed work, including direct and indirect costs (indirect costs may not exceed 10 percent of salaries and fringe benefits); * curriculum(a) vitae of the participating investigator(s); * evidence that the sponsoring organization is a nonprofit, tax-exempt institution; * an authorized form indicating clearance for the use of human and animal subjects; * an endorsement letter from the officer of the sponsoring institution who will be responsible for administering the grant. No other appended documents will be accepted for evaluation, and any incomplete applications will be returned to the applicant. The advisory board reviews proposals twice a year. Applications must be postmarked by the deadlines of February 1 and August 1 to be considered for review. INFORMATION For more information contact: McDonnell-Pew Program in Cognitive Neuroscience Green Hall 1-N-6 Princeton University Princeton, New Jersey 08544-1010 Telephone: 609-258-5014 Facsimile: 609-258-3031 Email: cns at confidence.princeton.edu ADVISORY BOARD Emilio Bizzi, M.D. Eugene McDermott Professor in the Brain Sciences and Human Behavior Chairman, Department of Brain and Cognitive Sciences Whitaker College Massachusetts Institute of Technology, E25-526 Cambridge, Massachusetts 02139 Sheila Blumstein, Ph.D. Professor of Cognitive and Linguistic Sciences Dean of the College Brown University University Hall, Room 218 Providence, Rhode Island 02912 Stephen J. Hanson, Ph.D. Group Leader Learning and Knowledge Acquisition Research Group Siemens Research Center 755 College Road East Princeton, New Jersey 08540 Jon Kaas, Ph.D. Centennial Professor Department of Psychology Vanderbilt University Nashville, Tennessee 37240 George A. Miller, Ph.D. James S. McDonnell Distinguished University Professor of Psychology Department of Psychology Princeton University Princeton, New Jersey 08544-1010 Mortimer Mishkin, Ph.D. Laboratory of Neuropsychology National Institute of Mental Health 9000 Rockville Pike Building 9, Room 1N107 Bethesda, Maryland 20892 Marcus Raichle, M.D. Professor of Neurology and Radiology Division of Radiation Sciences Mallinckrodt Institute of Radiology at Washington University Medical Center 510 S. Kingshighway Blvd., Campus Box 8131 St. Louis, Missouri 63110 Endel Tulving, Ph.D. Department of Psychology University of Toronto Toronto, Ontario M5S 1A1 Canada From rob at galab2.mh.ua.edu Thu Oct 25 17:43:20 1990 From: rob at galab2.mh.ua.edu (Robert Elliott Smith) Date: Thu, 25 Oct 90 16:43:20 CDT Subject: NN texts: a summary. Message-ID: <9010252143.AA14326@galab2.mh.ua.edu> Dear Connectionists, A week ago I posted a message requesting recomendations for possible texts for a introductory, graduate level engineering course neural nets. I received some interesting responses (thanks), so I decided to summarize to the net. I'll keep my editorializing to a minimum, since I have not seen any of these texts yet. If you want to comparison shop, you'll have to do like me and call the publishers. The following texts were recomended: Neurocomputing by Robert Hecht-Nielsen Addison-Wesley Publishing Company 1990 (this received the most recomendations by far) (a solution manual is rumoured to be available soon.) Neural Networks in Artificial Intelligence by Matthew Zeidenberg Ellis Horwood Ltd., distributed in the US by Simon and Schuster (sounded awfully interesting) Introduction to Neural and Cognitive Modeling by Daniel S. Levine Lawrence Erlbaum Associates (not available yet) Adaptive Pattern Recognition and Neural Networks by Y. Pao (no detailed bib entry, sorry) Neurocomputing (??) by Wasserman (no detailed bib entry, sorry) Artificial Neural Systems by Patrick Simpson Pergamon Press That's it. I hope some of you find this helpful. Sincerely, Rob ------------------------------------------- Robert Elliott Smith Department of Engineering of Mechanics The University of Alabama P. O. Box 870278 Tuscaloosa, Alabama 35487 <> rob at galab2.mh.ua.edu <> (205) 348-4661 ------------------------------------------- From sims at starbase.MITRE.ORG Fri Oct 26 09:34:04 1990 From: sims at starbase.MITRE.ORG (Jim Sims) Date: Fri, 26 Oct 90 09:34:04 EDT Subject: NN, training & noise fitting REF request Message-ID: <9010261334.AA01599@starbase> I seem to recall a recent reference in the literature or on this list to detecting when your training (via back-prop) has begun to start fitting the noise in the data rather than the features of the data space. Can someone provide a reference? thanks, jim (sims at starbase.mitre.org) From ad1n+ at ANDREW.CMU.EDU Fri Oct 26 11:38:09 1990 From: ad1n+ at ANDREW.CMU.EDU (Alexander Joshua Douglas) Date: Fri, 26 Oct 90 11:38:09 -0400 (EDT) Subject: Robotic controllers Message-ID: I am thinking of doing some reasearch in the field of robotic controllers using nueral nets. I recently saw a post about two technical reports by Jogensen. I am very interested in obtaining them, but our library does not have them, nor can they seem to get them. Perhaps somone could tell me how to get them. The reports are: Jorgensen, Charles C. (1990). "Development of a Sensor Coordinated Kinematic Model for Neural Network Controller training". RIACS Tecnical Report 90.28 Jorgensen, Charles C. (1990). "Distributed Memory Approaches for Robotic Neural Controllers". RIACS Tecnical Report 90.29 thank you, Alexander Douglas (ad1n+ at andrew.cmu.edu) From INAM%MUSICB.MCGILL.CA at BITNET.CC.CMU.EDU Fri Oct 26 15:31:16 1990 From: INAM%MUSICB.MCGILL.CA at BITNET.CC.CMU.EDU (Tony Marley) Date: Fri, 26 Oct 90 14:31:16 EST Subject: POSITION IN COGNITIVE PSYCHOLOGY, MCGILL UNIVERSITY Message-ID: <26OCT90.15683034.0069.MUSIC@MUSICB.MCGILL.CA> Although Prof. Bregman is in charge of the search for someone to fill a position in COGNITIVE PSYCHOLOGY in the Department of Psychology at McGill University, I encourage mathematically and/or computationally oriented researchers to keep me informed of their interest. Although it is unlikelty that we will hire a "straight" mathematical or computational person for this position, I will certainly push for someone with mathematical and computational skills. In particular, I would very much like to see applicants in the general area of neural modeling. Please let me know if you apply, and feel free to contact me for further information. Tony Marley Professor, Department of Psychology Director, McGill Cognitive Science Centre email: INAM at MUSICB.MCGILL.CA Tel: 514-398-6128 (office) 514-488-2067 (home) ------------------------------------------------------------------ October 4, 1990 The Department of Psychology at McGill University plans to make a tenure-track appointment of an assistant or associate professor in COGNITIVE PSYCHOLOGY. The appointment will begin in September 1991, subject to the availability of funding. The department has a strong tradition in cognitive psychology and is affiliated with the Cognitive Science Centre at the university. It is strongly supportive of younger staff and tends to promote from within the department. We are looking for an outstanding researcher. Nevertheless, we place a great stress on our teaching program and are looking for a candidate that could make a special contribution to it. The applicant's research could be concerned with any aspect of cognitive psychology, broadly interpreted. The major criterion will be the excellence of the applicant. Please bring this letter to the attention of any individuals you think might be qualified to apply or to persons who might know of such individuals. Selection will begin in mid-January, 1991. Applicants should arrange for at least three confidential letters of support to be sent to the address below. They should also send a curriculum vitae, copies of research publications and a brief statement describing their teaching and research to: A.S. Bregman, Telephone: (514) 398-6103 Cognitive Search Committee FAX: (514) 398-4896 Department of Psychology, McGill University, E-mail: in09 at musicb.mcgill.ca 1205 Dr. Penfield Avenue, or: in09 at mcgillb.bitnet Montreal, Quebec, CANADA H3A lBl ------------------------------------------------------------------ From shen at iro.umontreal.ca Mon Oct 29 09:59:43 1990 From: shen at iro.umontreal.ca (Yu Shen) Date: Mon, 29 Oct 90 09:59:43 EST Subject: Parallel Implementation of Genectic Algorithm and Simulated Message-ID: <9010291459.AA23899@kovic.IRO.UMontreal.CA> Annealing I want to do a survey on the Parallel Implementation of Genectic Algorithm and Simulated Annealing. Any pointer to the currents in the area is very much appriciated. I will compile the results of inquiry to the list, if required. Yu Shen Dept. d'Informatique et Recherche Operationnelle University de Montreal C.P. 6128 Succ. A. Montreal, Que. Canada H3S 3J7 (514) 342-7089 (H) shen.iro.umontreal.ca From P.Refenes at cs.ucl.ac.uk Mon Oct 29 12:38:38 1990 From: P.Refenes at cs.ucl.ac.uk (P.Refenes@cs.ucl.ac.uk) Date: Mon, 29 Oct 90 17:38:38 +0000 Subject: PRE-PRINT AVAILABILITY. Message-ID: The following pre-print (SPIE-90, Boston, Nov. 5-9 1990) is available. (write or e-mail to A. N. Refenes at UCL) AN INTEGRATED NEURAL NETWORK SYSTEM for HISTOLOGICAL IMAGE UNDERSTANDING A. N. REFENES, N. JAIN & M. M. ALSULAIMAN Department of Computer Science, University College London, Gower Street, WC1, 6BT, London, UK. ABSTRACT This paper describes a neural network system whose architecture was designed so that it enables the integration of heterogeneous sub-networks for performing specialised tasks. Two types of networks are integrated: a) a low-level feature extraction network for sub-symbolic computation, and b) a high-level network for decision support. The paper describes a non trivial application from histopathology, and its implementation using the Integrated Neural Network System. We show that with careful network design, the backpropagation learning procedure is an effective way of training neural networks for histological image understanding. We evaluate the use of symmetric and asymmetric squashing functions in the learning procedure and show that symmetric functions yield faster convergence and 100% generalisation performance. From IP%IRMKANT.BITNET at VMA.CC.CMU.EDU Mon Oct 29 11:19:36 1990 From: IP%IRMKANT.BITNET at VMA.CC.CMU.EDU (stefano nolfi) Date: Mon, 29 Oct 90 12:19:36 EDT Subject: paper available Message-ID: The following technical report is now available. You can get it from: STIVA AT IRMKANT.BITNET. Send your real adress. RECALL OF SEQUENCES OF ITEMS BY A NEURAL NETWORK Stefano Nolfi* Domenico Parisi* Giuseppe Vallar** Cristina Burani* *Inst. of Psychology - C.N.R. - Rome **University of Milan - Italy ABSTRACT A network architecture of the forward type but with additional 'memory' units that store the hidden units activation at time 1 and re-input this activation to the hidden units at time 2 (Jordan, 1986; Elman, 1990) is used to train a network to free recall sequences of items. The network's performance exhibits some features that are also observed in humans, such as decreasing recall with increasing sequence length and better recall of the first and the last items compared with middle items. An analysis of the network's behavior during sequence presentation can ex- plain these results. INTRODUCTION Human beings possess the ability to recall a set of items that are presented to them in a sequence. The overall capacity of the memory systems used in this task is limited and the probability of recall decreases with increasing sequence length. A second relevant feature of human performance in this task is that the last (recency effect) and the initial (primacy effect) items of the sequence tend to be recalled better than the middle items. These serial position effects have been observed both in a free recall condition, in which subjects may recall the stimuli in any order they wish, and in a serial recall condition, in which subjects must preserve the presentation order. (See reviews concerning free and serial recall of sequences and the recency ef- fect in: Glanzer, 1972; Crowder, 1976; Baddeley and Hitch, 1977; Shallice and Vallar, 1990). In this paper we report the results of a simulation experiment in which we trained neural networks to recall sequences of items. Our purpose was to explore if a particular network architecture could function as a memory store for generating free recall of sequences of items. Furthermore, we wanted to determine if the recall performances of our networks exhibited the two features of human free recall that we have mentioned, that is, decreasing probability of recall with increasing sequence length and an U-shaped recall curve (for related works see: Schneider and Detweiler, 1987; Schreter and Pfeifer, 1989; Schweickert, Guentert and Hersberger, 1989). To appear in: In D.S.Touretzky, J.L. Elman, T.J. Sejnowski and G.E. Hinton (eds.), Proceedings of the 1990 Connectionist Models Summer School. San Matteo, CA: Morgan Kaufmann. REFERENCES Baddeley A.D., Hitch G.J. (1974). Recency re-examined. In S. Dornic (Ed.). Attention and performance (Vol. 6). Hillsdale, NJ:Erlbaum, pp. 647-667. Crowder R.G. (1976). Principles of learning and memory. Hillsdale, NJ: Erlbaum. Glanzer M. (1972). Storage mechanisms in recall. In G.H. Bower (Ed.). The Psychology of learning and motivation. Advances in research and theory. (Vol. 5). New York: Academic Press, pp. 129-193. Elman, J.L. Finding structure in time. (1990). Cognitive Science, 14, 179-211. Jordan, M.I. (1986). Serial order: A parallel distributed processing approach. Institute for Cognitive Science. Report 8604. University of California, San Diego. Shallice T., Vallar G. (1990). The impairment of auditory-verbal short-term storage. In: G. Vallar and T. Shallice (Eds.). Neuropsychological impairments of short-term memory. New York: Cambridge University Press, pp.11-53. Schneider, W., & Detweiler, M. (1987). A connectionist control architecture for working memory. In G.H. Bower (Ed.) The Psychology of learning and motivation vol 21. New York: Academic Press. Schreter, Z., & Pfeirer, R. (1989). Short term memory and long term memory interactions in connectionist simulations of psychological experiments on list learning. In L. Personnaz and G. Dreyfus (Eds.), Neural Network: From models to applications. Paris: I.D.S.E.T. Schweickert, R., Guentert, L., & Hersberger, L. (1989). Neural Network Models of Memory Span. Preceedings of the Eleventh Annual Conference of the Cognitive Science Society. Ann Arbor, Michigan. From tgd at turing.CS.ORST.EDU Tue Oct 30 12:27:36 1990 From: tgd at turing.CS.ORST.EDU (Tom Dietterich) Date: Tue, 30 Oct 90 09:27:36 PST Subject: Local receptive fields Message-ID: <9010301727.AA03804@turing.CS.ORST.EDU> I am confused by what appear to be two different usages of the term "local receptive fields", and I wonder if anyone can un-confuse me. In papers about radial basis functions (e.g., Moody and Darken, Poggio and Girosi, etc.) the (single) layer of hidden units are described as having local receptive fields. However, these hidden units receive input from EVERY input unit, which strikes me as being more global than local. It is true, however, that each hidden unit will respond to only a few of the possible input vectors, and that this behavior can be described in terms of the "distance" between the input vector and the weight vector of the hidden unit. So in this sense, the hidden unit is responsive to a particular locality in the Euclidean n-space containing the input vectors. On the other hand, in papers such as those by Waibel et al on phoneme recognition or by LeCun et al on handwritten digit recognition, the hidden units have connections to only a few of the input units. These hidden units are also described as having local receptive fields. From Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU Tue Oct 30 17:46:47 1990 From: Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF1.SLISP.CS.CMU.EDU) Date: Tue, 30 Oct 90 17:46:47 EST Subject: Local receptive fields In-Reply-To: Your message of Tue, 30 Oct 90 09:27:36 -0800. <9010301727.AA03804@turing.CS.ORST.EDU> Message-ID: I am confused by what appear to be two different usages of the term "local receptive fields", and I wonder if anyone can un-confuse me. I think that the two usages of the term "local receptive field" are more or less the same idea, but different input encodings change the implementation. In both cases, you've got input units encoding some sort of N-dimensional space, and you've got hidden (or output) units that respond only to a localized patch of that hyperspace. That's the basic idea. If you think of each input as being a distinct, continous dimension, then you end up with something like the Moody and Darken units, which respond to some hyper-sphere or hyper-ellipsoid around an N-dimensional center point. On the other hand, if you think the individual units as encoding intervals or patches in this space (as in the speech networks -- each input is a little patch of time/frequency space or something like that), then you end up with hidden units that have inputs from a set of these units. Of course, there are a few more degrees of freedom in the latter case: within the "receptive field", the individual weights can encode something more complex than a simple Gaussian. So I think that the term "local receptive field" can cover both cases, but we need to specify what space we are working in and how the inputs map into that space: continuous orthogonal dimensions, mosaic encoding, or some hybrid scheme. -- Scott From jose at learning.siemens.com Tue Oct 30 17:23:55 1990 From: jose at learning.siemens.com (Steve Hanson) Date: Tue, 30 Oct 90 17:23:55 EST Subject: Local receptive fields Message-ID: <9010302223.AA03681@learning.siemens.com.siemens.com> splitting hairs: tom, I don't think we need new terms to describe what is a continuum... clearly in a limiting sense a fan-in function that responds to a specific point in input space (although measures input on each line to determine this) is similar in effect to a net that has a subset of lines from the input space. Really the important issue is spatial locality... does a hidden unit have a preference for a location or is it a global in its fan-in function.. this will have affects on learning, recognition and approximation. steve From mike at park.bu.edu Tue Oct 30 14:43:52 1990 From: mike at park.bu.edu (mike@park.bu.edu) Date: Tue, 30 Oct 90 14:43:52 -0500 Subject: CNS Program at Boston University Hiring 2 Assistant Professors Message-ID: <9010301943.AA01462@BUCASB.BU.EDU > Boston University seeks two tenure track assistant or associate professors starting in Fall, 1991 for its M.A. and Ph.D. Program in Cognitive and Neural Systems. This program offers an intergrated curriculum offering the full range of psychological, neurobiological, and computational concepts, models, and methods in the broad field variously called neural networks, connectionism, parallel distributed processing, and biological information processing, in which Boston University is a leader. Candidates should have extensive analytic or computational research experience in modelling a broad range of nonlinear neural networks, especially in one or more of the areas: vision and image processing, speech and language processing, adaptive pattern recognition, cognitive information processing, and adaptive sensory-motor control Candidates for associate professor should have an international reputation in neural network modelling. Send a complete curriculum vitae and three letters of recommendation to Search Committee, Cognitive and Neural Systems Program, Room 240, 111 Cummington Street, Boston University, Boston, MA 02215, preferably by November 15, 1990 but no later than January 1, 1991. Boston University is an Equal Opportunity/Affirmative Action employer. Boston University (617-353-7857) Email: mike at bucasb.bu.edu Smail: Michael Cohen 111 Cummington Street, RM 242 Center for Adaptive Systems Boston, Mass 02215 Boston University From jose at learning.siemens.com Tue Oct 30 20:14:47 1990 From: jose at learning.siemens.com (Steve Hanson) Date: Tue, 30 Oct 90 20:14:47 EST Subject: Local receptive fields Message-ID: <9010310114.AA03746@learning.siemens.com.siemens.com> Tom, I wasn't meaning to be obfuscate, I just believe such questions do lead to splitting hairs and in any case is a very complex issue. Which (sense) is truer to its physiological origins? who knows? (anyone who does please comment) but receptive field properties can be complex--they can be "simple". Clearly not all neurons are connected to all other neurons --so in that trivial sense they are "local". Do they have preference for other cells? No doubt. But the data are complex and difficult to map onto specific computational functions. This I think is one of the advantages to computational models in that modelers can quickly explore the computational consequences of hypotheses that neuroscientists must explore in more tedious and perhaps information poorer ways. Steve From kruschke at ucs.indiana.edu Tue Oct 30 16:36:00 1990 From: kruschke at ucs.indiana.edu (KRUSCHKE,JOHN,PSY) Date: 30 Oct 90 16:36:00 EST Subject: local receptive fields Message-ID: > Date: Tue, 30 Oct 90 09:27:36 PST > From: Tom Dietterich > Subject: Local receptive fields > > I am confused by what appear to be two different usages of the term > "local receptive fields", and I wonder if anyone can un-confuse me. > > In papers about radial basis functions (e.g., Moody and Darken, Poggio > and Girosi, etc.) the (single) layer of hidden units are described as > having local receptive fields. ... > > On the other hand, in papers such as those by Waibel et al on phoneme > recognition or by LeCun et al on handwritten digit recognition, the > hidden units have connections to only a few of the input units. These > hidden units are also described as having local receptive fields. > > From a nervous system point of view, it seems to me that the second > usage is more correct than the first. I think we need a better term for > describing the kind of locality exhibited by RBF networks. There is an important difference between the intrinsic topologies of the input layers in the two situations Dietterich describes. In RBF networks, the input nodes are usually assumed to each represent entire dimensions of variation. So if there are N input nodes, then the input space is N-dimensional. The RBF nodes at the hidden layer are responsive to only a local region of the N-dimensional input space. On the other hand, in the situations that Dietterich describes as more "neural", the input nodes are not usually assumed to each individually represent entire dimensions of variation. For example, each node might represent a small region of a 2-D image, so that the entire ensemble of N input nodes actually represents just a 2-D input space. In such a space there is an intrinsic topology so that some nodes are closer to each other than others, whereas in the input space of the RBF network, no input nodes are closer to each other than any others. So, both types of "localization in space" make good sense, but in their own representations of space. --John Kruschke From tgd at turing.CS.ORST.EDU Tue Oct 30 17:54:11 1990 From: tgd at turing.CS.ORST.EDU (Tom Dietterich) Date: Tue, 30 Oct 90 14:54:11 PST Subject: Local receptive fields In-Reply-To: Steve Hanson's message of Tue, 30 Oct 90 17:23:55 EST <9010302223.AA03681@learning.siemens.com.siemens.com> Message-ID: <9010302254.AA04488@turing.CS.ORST.EDU> I disagree that this is splitting hairs. It can be very confusing for someone entering this research area to have the same term used in two different (albeit related) ways. --Tom P.S. Which use of the term is truer to its neurophysiological origins? From dario%TECHUNIX.BITNET at VMA.CC.CMU.EDU Wed Oct 31 04:15:30 1990 From: dario%TECHUNIX.BITNET at VMA.CC.CMU.EDU (Dario Ringach) Date: Wed, 31 Oct 90 11:15:30 +0200 Subject: local receptive fields In-Reply-To: "KRUSCHKE,JOHN,PSY" "local receptive fields" (Oct 30, 4:36pm) Message-ID: <9010310915.AA11853@techunix.technion.ac.il> A good mathematical definition of "locality in space", I think, is to require the unit to have a compact support in the input space. In this sense, RBF-networks have not local units. The study of non-orthogonal and orthogonal complete systems in L^2(R) having compact support, is of importance here. See for example the work of Daubechies, Meyer, Mallat, and others on Wavelet theory. Of course, other space-frequency representation/analysis of signals, such as the Gabor transform, are also related to simultaneous localization in space and frequency. - Dario Ringach From Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU Wed Oct 31 10:02:07 1990 From: Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF1.SLISP.CS.CMU.EDU) Date: Wed, 31 Oct 90 10:02:07 EST Subject: Local receptive fields In-Reply-To: Your message of Tue, 30 Oct 90 20:14:47 -0500. <9010310114.AA03746@learning.siemens.com.siemens.com> Message-ID: Which (sense) is truer to its physiological origins? who knows? (anyone who does please comment) but receptive field properties can be complex--they can be "simple". Clearly not all neurons are connected to all other neurons --so in that trivial sense they are "local". Do they have preference for other cells? No doubt. It's clear that in the big visual areas that have been mapped, some sort of mosaic encoding is in use. That is, the most obvious dimensions of the space -- the X/Y diemensions of the 2-D image on the retina -- are not represented by varying levels of activation, but rather are mapped across sets of cells in a regular (though distorted and interleaved) 2-D mosaic. Other major systems seem to use the same kind of encoding: hearing, tactile senses, motor control, etc. There may or may not be some continuous, analog encoding (levels or pulse-frequency) at other levels or among some of the low-level control systems. So clearly, for most parts of the neural system in mammals, the idea of "local receptive field" for cells near the input layer maps cleanly into some sort of range limitation on the map in question. I think what you are talking about above, however, is what I would call "local connectivity" rather than "local receptive fields". If you allow signals to take a few hops, a net whose immediate physical connections are mostly local can give you a very large receptive field. The wire on my telephone runs to an office a few blocks away, but my telephone's receptive field includes just about any place on earth. And if a net makes significant use of recurrent links and delays, the receptive field extends back in time in a complicated way. The term "receptive field" has mainly been used in describing combinational logic (with no memory, maybe just some adaptation and gain control) and should probably be reserved for that context. -- Scott "No, no Igor! The OTHER brain..." From jbower at smaug.cns.caltech.edu Wed Oct 31 11:13:52 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Wed, 31 Oct 90 08:13:52 PST Subject: NIPS demo machines Message-ID: <9010311613.AA19176@smaug.cns.caltech.edu> Announcement NIPS Demo Machines Two machines will likely be available for demos at this year's NIPS meeting: 1 DECstation 5000/200PX with TK50 tape drive running Ultrix 4.0 1 Sparcstation 1+ or better RISC box from Sun with a 1/4" tape drive. running Sun OS 4.1c Both machines will be 8 bit depth color, have between 8 and 16Mb of memory. Participants should feel free to bring software demos. The machines are not, however, intended to be used to sell software. Machines will be schedualed on a first come first serve basis. Anyone with questions can contact John Uhley at uhley.smaug.cns.caltech.edu. John will also be available at the meeting. Jim Bower From moody-john at CS.YALE.EDU Wed Oct 31 10:56:06 1990 From: moody-john at CS.YALE.EDU (john moody) Date: Wed, 31 Oct 90 10:56:06 EST Subject: locality, RBF's, receptive fields, and all that Message-ID: <9010311556.AA29442@SUNNY.SYSTEMSX.CS.YALE.EDU> Concerning the questions of terminology raised by Tom Dietterich: After we wrote "Learning with localized receptive fields" (Proceedings of the 1988 Connectionist Models Summer School, Touretzky, Hinton, and Sejnowski, eds., Morgan Kaufmann), we found that our choice of terminology was confusing to some. We tried to remedy this confusion when we wrote "Fast learning in networks of locally-tuned processing units" (Neural Computation 1(2)281-294, 1989). The point of confusion is whether "local receptive field" should refer to the network connectivity pattern (or the afferent connectivity of a single unit) or the form of the response function of a processing unit. We agree with Dietterich that the term "local receptive field" should be reserved for internal units whose afferent connections come primarily from a local neighborhood of units in the preceding layer. We believe that this term is best NOT used to describe RBF type units, which typically (but not always) have global connectivity to their input layer. To describe localized unit response functions, we favor the term "locally- tuned processing unit" (LTPU). These include RBFs, but are not limited to units whose response functions are radially symmetric { R(x,y) = R(|x-y|) }. [Indeed, we have found that non-radial LTPUs often work better than the standard RBFs.] The distinction between "local receptive field" and "locally tuned processing unit" can become blurred when one considers "effective variables". These variables are *not* the activations of individual input units, but are rather implicitly encoded for by the population of input units. For example, the orientation selective cells of V1 have "local receptive fields" as determined by their afferent connection patterns. However, they also "respond locally" in the *effective variables* angular orientation and retinal position. The locality of response in angular orientation depends on the *values* of the afferent connections, while the locality of response to retinal position is due to the locality of the receptive fields. [The localities of response to the effective variables could be modeled by a network of LTPUs with three input variables. Of course, such a network would not be responsive to more that one object in the scene or to different levels of illumination.] Provided that one is clear about which variables one is referring to, whether input unit activations or effective variables, confusion between local connectivity patterns ("local receptive fields") and localized response functions ("LTPUs") can be avoided. --John Moody and Christian Darken ------- From gary%cs at ucsd.edu Wed Oct 31 11:49:16 1990 From: gary%cs at ucsd.edu (Gary Cottrell) Date: Wed, 31 Oct 90 08:49:16 PST Subject: splitting hairs Message-ID: <9010311649.AA18964@desi.ucsd.edu> Since Steve brought up hair splitting, it seemed like a good time to send out my latest: SEMINAR Approaches to the Inverse Dogmatics Problem: Time for a return to localist networks? Garrison W. Cottrell Department of Dog Science Condominium Community College of Southern California The innovative use of neural networks in the field of Dognitive Science has spurred the intense interest of the philosophers of Dognitive Science, the Dogmatists. The field of Dogmatics is devoted to making sense of the effect of neural networks on the conceptual underpinnings of Dognitive Science. Unfortunately, this flurry of effort has caused researchers in the rest of the fields of Dognitive Science to spend an inordinate amount of time attempting to make sense of the philosophers, otherwise known as the Inverse Dogmatics problem (Jordan, 1990). The problem seems to be that the philosophers have allowed themselves an excess of degrees of freedom in conceptual space, as it were, leaving the rest of us with an underconstrained optimization problem: Should we bother listening to these folks, who may be somewhat more interesting than old Star Trek reruns, or should we try and get our work done? The inverse dogmatics problem has become so prevalent that many philosophers are having to explain themselves daily, much to the dismay of the rest of the field. For example Gonad[1] (1990a, 1990b, 1990c, 1990d, 1990e, well, you get the idea...) has repeatedly stated that no connectionist network can pass his usually Fatal Furring Fest, where the model is picked apart, hair by hair[2], until the researchers making counterarguments have long since died[3]. One approach to this problem is to generate a connectionist network that is so hairy (e.g., Pollack's RAMS, 1990), that it will outlast Gonad's attempt to pick it apart. This is done by making a model that is at the sub-fur level, that recursively splits hairs, RAMming more and more into each hair, which generates a fractal representation that is not susceptible to linear hair splitting arguments. Another approach is to take Gonad head-on, and try to answer his fundamental question, that is, the problem of how external discrete nuggets get mapped into internal mush. This is known as the *grinding problem*. In our approach to the grinding problem, we extend our previous work on the Dog Tomatogastric Ganglion (TGG). The TGG is an oscillating circuit in the dog's motor cortex that controls muscles in the dog's stomach that expel tomatoes and other non-dogfood items from the dog's stomach. In our grinding network, we will have a similar set up, using recurrent bark propagation to train the network to oscillate in such a way that muscles in the dog's mouth will grind the nuggets ____________________ [1]Some suspect that Gonad may in fact be an agent of reactionary forces whose mission is to destroy Dognitive Science by filibuster. [2]Thus by a simple morphophonological process of reduplication, ex- haustive arguments have been replaced by exhausting arguments. [3]In this respect, Gonad's approach resembles that of Pinky and Prince, whose exhausting treatment of the Past Fence Model, Rumblephart and McNugget's connectionist model of dog escapism, has generated a sub- field of Dognitive Science composed of people trying to answer their ar- guments. into the appropriate internal representation. This representation is completely distributed. This is then transferred directly into the dog's head, or Mush Room. Thus the thinking done by this representation, like most modern distributed representations, is not Bayesian, but Hazyian. If Gonad is not satisfied by this model, we have an alternative approach to this problem. We have come up with a connectionist model that has a *finite* number of things that can be said about it. In order to do this we had to revert to a localist model, suggesting there may be some use for them after all. We will propose that all connectionist researchers boycott distributed models until the wave of interest by the philosophers passes. Then we may get back to doing science. Thus we must bring out some strong arguments in favor of localist models. The first is that they are much more biologically plausible than distributed models, since *just like real neurons*, the units themselves are much more complicated than those used in simple PDP nets. Second, just like the neuroscientists do with horseradish peroxidase, we can label the units in our network, a major advantage being that we have many more labels than the neuroscientists have, so we can keep ahead of them. Third, we don't have to learn any more than we did in AI 101, because we can use all of the same representations. As an example of the kind of model we think researchers should turn their attention to, we are proposing the logical successor to Anderson & Bower's HAM model, SPAM, for SPreading Activation Memory model. In this model, nodes represent language of thought propositions. Because we are doing Dog Modeling, we can restrict ourselves to at most 5 primitive ACTS: eat, sleep, fight, play, make whoopee. The dog's sequence of daily activities can then be simply modeled by connectivity that sequences through these units, with habituation causing sequence transitions. A fundamental problem here is, if the dog's brain can be modeled by 5 units, *what is the rest of the dog's brain doing?* Some have posited that localist networks need multiple copies of every neuron for reliability purposes, since if the make whoopee unit was traumatized, the dog would no longer be able to make whoopee. Thus these researchers would posit that the rest of the dog's brain is simply made up of copies of these five neurons. However, we believe we have a more esthetically pleasing solution to this problem that simultaneously solves the size mismatch problem. The problem is that distributed connectionists, when discussing the reliability problem of localist networks, have in mind the wimpy little neurons that distributed models use. We predict that Dognitive neuroscientists, when they actually look, will find only five neurons in the dog's brain - but they will be *really big* neurons. From jbower at smaug.cns.caltech.edu Wed Oct 31 12:07:03 1990 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Wed, 31 Oct 90 09:07:03 PST Subject: RFs Message-ID: <9010311707.AA19206@smaug.cns.caltech.edu> Just a brief note on the question of receptive fields from a biological perspective. Classically, receptive fields are defined within neurobiology as those regions of the stimulus space that, when activated, obviously alter the firing pattern of the neuron in question (either excite or inhibit, or both). Traditionally, the issue of the actual anatomical connectivity underlying the formation of the receptive field has not been considered in much detail (often it is not known). However, physiologists have recently discovered that neuronal receptive fields can be far more complicated than previously assumed. The most famous current examples are the so- called nonclassical receptive fields of neurons in visual regions of cerebral cortex. In this case, it has been discovered that peripheral stimuli that do not by themselves activate a neuron are capable of significantly modifying neuronal responses to stimulation of classical receptive fields. These effects are probably a result of the understudied and underemphasized horizontal connectivity of cerebral cortical networks. The neglect of this important network feature is, in fact, largely due to the emphasis on the restricted "locality" of classical receptive fields and the resulting (faulty in my view) notion of cortical columnar organization. The point with respect to the current discussion is that it IS important to carefully define what is meant by a receptive field and especially to take into account the anatomical organization of the network involved. Operational definitions related to the simple response properties of the neuron can obscure important network properties. It could be that this attention to detail is less important in the case of the simple connectionist networks currently being constructed. However, as connectionist models inevitably become more complex in the pursuit of real usefulness, it will become increasingly important to be careful about what is meant by a receptive field. Jim Bower From at neural.att.com Wed Oct 31 10:20:23 1990 From: at neural.att.com (@neural.att.com) Date: Wed, 31 Oct 90 10:20:23 -0500 Subject: Local receptive fields In-Reply-To: Your message of Tue, 30 Oct 90 09:27:36 -0800. Message-ID: <9010311520.AA11642@lamoon> > I am confused by what appear to be two different usages of the term > "local receptive fields",.... I also think these two meanings of "local receptive field" is very confusing. These are two very different concepts. I guess the notion of local receptive field was introduced by neurobiologists. They meant "local" in real space, not in feature space. The region of feature space in which radial basis functions are activated, Should be called differently to avoid confusion. I think the correct word to designate the set on which a function takes non-zero values is "support". Unfortunately, most radial basis functions (such as gaussians) are non-zero everywhere. How about "activation window", or "activation region". -- Yann Le Cun From at neural.att.com Wed Oct 31 11:14:52 1990 From: at neural.att.com (@neural.att.com) Date: Wed, 31 Oct 90 11:14:52 -0500 Subject: Local receptive fields In-Reply-To: Your message of Tue, 30 Oct 90 17:23:55 -0500. Message-ID: <9010311614.AA11671@lamoon> Steve says > splitting hairs: tom, I don't think we need new terms to describe > what is a continuum... I disagree. An RBF unit in a network with 1024 inputs will have 1024 input lines (and therefore at least 1024 parameters). Now assume these 1024 inputs are actually the pixels of a 32x32 image....what is local there? certainly not the connections. Now here is a really confusing situation: imagine an image-recognition network with 32x32 inputs (pixels). Suppose that the units in the first layer are connected to, say, local (2D) 5x5 patches on the input (receptive field concept #1). Now imagine that these units are gaussian RBF units. Their activation region (receptive field concept #2) is a hypersphere (or something similar) in the 25-dimensional space of their input (5x5=25). If these units were sigmoid units, their activation region (receptive field concept #2) would be a half space in the 25-D space. As you see concept #1 and concept #2 are completely orthogonal, and can be used independently, or even combined. They, therefore, should have different names. -- Yann From mehra at aquinas.csl.uiuc.edu Wed Oct 31 15:00:27 1990 From: mehra at aquinas.csl.uiuc.edu (Pankaj Mehra) Date: Wed, 31 Oct 90 14:00:27 CST Subject: local receptive fields Message-ID: <9010312000.AA15613@rhea> In response to Tom Dietterich : > I am confused by what appear to be two different usages of the term > "local receptive fields" .... and Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU: > the two usages of the term "local receptive field" are more or > less the same idea, but different input encodings change the > implementation. I think that the question of locality starts well before the data are given to the learning program. If we measure variables (dimensions) using one sensor for each dimension, then transforming intervals (overlapping or otherwise) of that dimension into local receptive fields is a matter of representational convenience (different encodings, as Scott says). Real receptive fields (i.e. units responding to limited regions of an abstract feature space) perform oversampling of the input space, thus providing real redundancy in observed data. There are multiple sensors for each dimension. Artificial receptive fields (a good example is the BOXES representation used in Chuck Anderson's thesis, U. Mass. Amherst, 1986) merely recode the irreedundant data via a value-coded representation. Artificial overlapping receptive fields merely oversample the data which is collected non-redundantly. In natural systems, real receptive fields can therefore alleviate sensor errors (even dead cells) with a possible loss of resolution. Thus, they are in some sense ``robust'' to noisy sensors. Personally, I believe that multiple independent sensors should have statistical significance as well, although I have not seen any discussion of that in the literature I am aware of. - Pankaj Mehra University of Illinois From lyle at ai.mit.edu Wed Oct 31 16:48:31 1990 From: lyle at ai.mit.edu (Lyle J. Borg-Graham) Date: Wed, 31 Oct 90 16:48:31 EST Subject: Local receptive fields Message-ID: <9010312148.AA04635@peduncle> >But the data are complex and difficult to map onto specific >computational functions. This I think is one of the advantages >to computational models in that modelers can quickly explore >the computational consequences of hypotheses that >neuroscientists must explore in more tedious and perhaps >information poorer ways. Of course, an ever-present risk is that modellers can over-generalize from seductively-simple models (or just simply-seductive ones), evoking an Occamian motivation which helps the neuroscientists and their interpretation of the data not a whit. From jose at learning.siemens.com Wed Oct 31 17:02:51 1990 From: jose at learning.siemens.com (Steve Hanson) Date: Wed, 31 Oct 90 17:02:51 EST Subject: Local receptive fields Message-ID: <9010312202.AA05510@learning.siemens.com.siemens.com> Indeed! I agree completely... a similar point (perhaps more balanced) was made by Phil Anderson a while back "The art of model-building is the exclusion of real but irrelevant parts of the problem, and entails hazards for the builder and the reader. The builder may leave out something genuinely relevant; the reader, armed with too sophisticated an experimental probe or too accurate a computation, may take literally a schematized model whose main aim is to be a demonstration of possibility." -P. W. Anderson (from Nobel acceptance speech, 1977) From at neural.att.com Wed Oct 31 11:14:52 1990 From: at neural.att.com (@neural.att.com) Date: Wed, 31 Oct 90 11:14:52 -0500 Subject: Local receptive fields In-Reply-To: Your message of Tue, 30 Oct 90 17:23:55 -0500. Message-ID: <9010311614.AA11671@lamoon> Steve says > splitting hairs: tom, I don't think we need new terms to describe > what is a continuum... I disagree. An RBF unit in a network with 1024 inputs will have 1024 input lines (and therefore at least 1024 parameters). Now assume these 1024 inputs are actually the pixels of a 32x32 image....what is local there? certainly not the connections. Now here is a really confusing situation: imagine an image-recognition network with 32x32 inputs (pixels). Suppose that the units in the first layer are connected to, say, local (2D) 5x5 patches on the input (receptive field concept #1). Now imagine that these units are gaussian RBF units. Their activation region (receptive field concept #2) is a hypersphere (or something similar) in the 25-dimensional space of their input (5x5=25). If these units were sigmoid units, their activation region (receptive field concept #2) would be a half space in the 25-D space. As you see concept #1 and concept #2 are completely orthogonal, and can be used independently, or even combined. They, therefore, should have different names. -- Yann %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% End of returned mail From gary%cs at ucsd.edu Wed Oct 31 11:49:16 1990 From: gary%cs at ucsd.edu (Gary Cottrell) Date: Wed, 31 Oct 90 08:49:16 PST Subject: splitting hairs Message-ID: <9010311649.AA18964@desi.ucsd.edu> Since Steve brought up hair splitting, it seemed like a good time to send out my latest: SEMINAR Approaches to the Inverse Dogmatics Problem: Time for a return to localist networks? Garrison W. Cottrell Department of Dog Science Condominium Community College of Southern California The innovative use of neural networks in the field of Dognitive Science has spurred the intense interest of the philosophers of Dognitive Science, the Dogmatists. The field of Dogmatics is devoted to making sense of the effect of neural networks on the conceptual underpinnings of Dognitive Science. Unfortunately, this flurry of effort has caused researchers in the rest of the fields of Dognitive Science to spend an inordinate amount of time attempting to make sense of the philosophers, otherwise known as the Inverse Dogmatics problem (Jordan, 1990). The problem seems to be that the philosophers have allowed themselves an excess of degrees of freedom in conceptual space, as it were, leaving the rest of us with an underconstrained optimization problem: Should we bother listening to these folks, who may be somewhat more interesting than old Star Trek reruns, or should we try and get our work done? The inverse dogmatics problem has become so prevalent that many philosophers are having to explain themselves daily, much to the dismay of the rest of the field. For example Gonad[1] (1990a, 1990b, 1990c, 1990d, 1990e, well, you get the idea...) has repeatedly stated that no connectionist network can pass his usually Fatal Furring Fest, where the model is picked apart, hair by hair[2], until the researchers making counterarguments have long since died[3]. One approach to this problem is to generate a connectionist network that is so hairy (e.g., Pollack's RAMS, 1990), that it will outlast Gonad's attempt to pick it apart. This is done by making a model that is at the sub-fur level, that recursively splits hairs, RAMming more and more into each hair, which generates a fractal representation that is not susceptible to linear hair splitting arguments. Another approach is to take Gonad head-on, and try to answer his fundamental question, that is, the problem of how external discrete nuggets get mapped into internal mush. This is known as the *grinding problem*. In our approach to the grinding problem, we extend our previous work on the Dog Tomatogastric Ganglion (TGG). The TGG is an oscillating circuit in the dog's motor cortex that controls muscles in the dog's stomach that expel tomatoes and other non-dogfood items from the dog's stomach. In our grinding network, we will have a similar set up, using recurrent bark propagation to train the network to oscillate in such a way that muscles in the dog's mouth will grind the nuggets ____________________ [1]Some suspect that Gonad may in fact be an agent of reactionary forces whose mission is to destroy Dognitive Science by filibuster. [2]Thus by a simple morphophonological process of reduplication, ex- haustive arguments have been replaced by exhausting arguments. [3]In this respect, Gonad's approach resembles that of Pinky and Prince, whose exhausting treatment of the Past Fence Model, Rumblephart and McNugget's connectionist model of dog escapism, has generated a sub- field of Dognitive Science composed of people trying to answer their ar- guments. into the appropriate internal representation. This representation is completely distributed. This is then transferred directly into the dog's head, or Mush Room. Thus the thinking done by this representation, like most modern distributed representations, is not Bayesian, but Hazyian. If Gonad is not satisfied by this model, we have an alternative approach to this problem. We have come up with a connectionist model that has a *finite* number of things that can be said about it. In order to do this we had to revert to a localist model, suggesting there may be some use for them after all. We will propose that all connectionist researchers boycott distributed models until the wave of interest by the philosophers passes. Then we may get back to doing science. Thus we must bring out some strong arguments in favor of localist models. The first is that they are much more biologically plausible than distributed models, since *just like real neurons*, the units themselves are much more complicated than those used in simple PDP nets. Second, just like the neuroscientists do with horseradish peroxidase, we can label the units in our network, a major advantage being that we have many more labels than the neuroscientists have, so we can keep ahead of them. Third, we don't have to learn any more than we did in AI 101, because we can use all of the same representations. As an example of the kind of model we think researchers should turn their attention to, we are proposing the logical successor to Anderson & Bower's HAM model, SPAM, for SPreading Activation Memory model. In this model, nodes represent language of thought propositions. Because we are doing Dog Modeling, we can restrict ourselves to at most 5 primitive ACTS: eat, sleep, fight, play, make whoopee. The dog's sequence of daily activities can then be simply modeled by connectivity that sequences through these units, with habituation causing sequence transitions. A fundamental problem here is, if the dog's brain can be modeled by 5 units, *what is the rest of the dog's brain doing?* Some have posited that localist networks need multiple copies of every neuron for reliability purposes, since if the make whoopee unit was traumatized, the dog would no longer be able to make whoopee. Thus these researchers would posit that the rest of the dog's brain is simply made up of copies of these five neurons. However, we believe we have a more esthetically pleasing solution to this problem that simultaneously solves the size mismatch problem. The problem is that distributed connectionists, when discussing the reliability problem of localist networks, have in mind the wimpy little neurons that distributed models use. We predict that Dognitive neuroscientists, when they actually look, will find only five neurons in the dog's brain - but they will be *really big* neurons. From rudnick at uxh.cso.uiuc.edu Tue Oct 30 11:56:47 1990 From: rudnick at uxh.cso.uiuc.edu (Mike Rudnick) Date: Tue, 30 Oct 90 10:56:47 -0600 Subject: requst for parallel implementation of GA references Message-ID: FOGA-90 (Foundations of Genetic Algrithms) workshop Indiana U. in Bloomington. Summer, 90 Gregory Rawlins, a faculty member there. There were a few presentations on parallel GA algorithms. ------------------- From iro.umontreal.ca!relay.eu.net!gmdzi!muehlen at IRO.UMontreal.CA Tue Oct 30 07:57:14 1990 From: iro.umontreal.ca!relay.eu.net!gmdzi!muehlen at IRO.UMontreal.CA (Heinz Muehlenbein) Date: Tue, 30 Oct 90 11:57:14 -0100 Subject: parallel genetic algorithmn Message-ID: We have implemented starting in 1987 a parallel genetic algorithm. It is the most powerful random search method in traveling salesman quadratic assignmment m-graph partitioning autocorrelation. n-dimensional multimodal function optimization references: Evolution algorithms in combinatorial optimization Parallel computing 7 65-88 (1988) papers from Muehlenbein quadratic assignment and Gorges-schleuter in 3rd conference on genetic algorithms recent papers ( not yet published) give a survey parallel genetic algorithms and combinatorial optimization ( SIAM) Evolution in space and time - the parallel genetic algorithm ( FOGA) Heinz Muehlenbein ------------------------- ---------- From pattie at ai.mit.edu Mon Oct 29 10:42:58 1990 From: pattie at ai.mit.edu (Pattie Maes) Date: Mon, 29 Oct 90 10:42:58 EST Subject: Parallel Implementation of Genectic Algorithm and Simulated In-Reply-To: Yu Shen's message of Mon, 29 Oct 90 09:59:43 EST <9010291459.AA23899@kovic.IRO.UMontreal.CA> Message-ID: Piet Spiessens and Bernard Manderick from the University of Brussels in Belgium have done work on parallel implementations od GA's. I think this was published in one of the conferences on GA's. Contact them for the exact reference: piet at arti.vub.ac.be bernard at arti.vub.ac.be --------------- From marcap at concour.cs.concordia.ca Mon Oct 29 22:19:56 1990 From: marcap at concour.cs.concordia.ca (marcap@concour.cs.concordia.ca) Date: Mon, 29 Oct 90 22:19:56 -0500 Subject: No subject In-Reply-To: Your message of Mon, 29 Oct 90 10:05:52 -0500. <9010291505.AA24026@kovic.IRO.UMontreal.CA> Message-ID: There has been much work done in this area. Your best bet is to look at the first, second, and third conferences on genetic algorithms. McGill has the second conference. CISTI has all three. The third conferences title is proceedings of the third international conference on gentic algorithms. George mason University June 4-7, 1989 Editor: J. david Schaffer ISBN 1-55860-066-3 LIBRARY: QA 402.5.I512 1989 of Congress Hope that helps, MARC From lesher at ncifcrf.gov Mon Oct 29 17:55:02 1990 From: lesher at ncifcrf.gov (lesher@ncifcrf.gov) Date: Mon, 29 Oct 90 17:55:02 EST Subject: // impl GA Message-ID: @techreport{Robertson:87, author = "G. G. Robertson", title = "Parallel Implementation of Genetic Algorithms in a Classifier System", type = "Technical Report Series", number = "RL87-5", institution = "Thinking Machines Corporation", address = "245 First Street, Cambridge, MA 01242-1214", month = "May", year = "1987"} Sarah Lesher From Hiroaki.Kitano at a.nl.cs.cmu.edu Wed Oct 31 12:27:24 1990 Return-Path: Date: Mon, 29 Oct 1990 10:49-EST From: Hiroaki.Kitano at a.nl.cs.cmu.edu To: Yu Shen Subject: Re: Parallel Implementation of Genectic Algorithm and Simulated In-Reply-To: Yu Shen's mail message of Mon, 29 Oct 90 09:59:43 EST We are currently working on massively parallel implementation on GA and Classifier system. By teh end of this year, we will be able to (I hope) have a paper for distribution. Hiroaki Kitano Center for Machine Translation Carnegie Mellon Univeristy, Pittsburgh, PA 15213 U.S.A. ---------------------- %from the bibliography of parallel and supercomputing references, %icarus.riacs.edu (128.102.16.8) in the /pub/bib directory -------------GA %A Elizabeth J. O'Neill %A Craig G. Shaefer %T The ARGOT Strategy III: The BBN Butterfly Multiprocessor %E Joanne L. Martin %E Stephen F. Lundstrom %B Science and Applications: Supercomputing'88 %V II %I IEEE %C Orlando, FL %D 1988 %P 214-227 %K structural analysis, genetic algorithms, applications, %A Chrisila C. Pettey %A Michael R. Leuze %Z CS Dept., Vanderbilt U. %T Parallel Placement of Parallel Processes %J 3rd Conference on Hypercube Concurrent Computers and Applications %V I, Architecture, Software, Computer Systems and General Issues %I ACM %C Pasadena, CA %D January 1988 %P 232-238 %K load balancing and decomposition, parallel genetic algorithms (PGA/GA), simulated annealing, %A Chrisila C. Petty %A Michael R. Leuze %A John J. Grefenstette %T Genetic Algorithms on a Hypercube Multiprocessor %B Hypercube Multiprocessors 1987 %I SIAM %C Philadelphia %D 1987 %P 333-341 ----------classifier %A Stephanie Forrest %Z Teknowledge %T A Study of Parallelism in the Classifier System and its Application to Classification in KL-One Semantic Networks %R PhD thesis %I Logic of Computer Group, University of Michigan %C Ann Arbor, MI, USA %D 1985 %X Received from PARSYM. %A Stephanie Forrest %Z Teknowledge %T The Classifier System: A Computational Model that Supports Machine Intelligence %J Proceedings of the 1986 International Conference on Parallel Processing %I IEEE %D August 1986 %P 711-716 %K Artificial intelligence, KL-one, simulation, pattern recognition, ------------------PSA---------------------------------------- From ray at cs.su.oz.au Tue Oct 30 00:26:01 1990 From: ray at cs.su.oz.au (Raymond Lister) Date: Tue, 30 Oct 90 16:26:01 +1100 Subject: Parallel Implementation of Genectic Algorithm and Simulated Message-ID: I have published a paper on a parallel simulated annealing approach to the Travelling Salesman Problem. A second paper is to appear ... Lister, R (1990), "Segment Reversal and the Traveling Salesman Problem", International Joint Conference on Neural Networks (IJCNN-90-WASH DC), Washington, January 1990. Lister, R (1990), Response to a review by Van den Bout, D "A Parallel, Simulated Annealing Solution to the Traveling Salesman Problem", to appear, Neural Network Review. I'll send you a copy of both papers. I suggest you check the ftp repository set up by Dan Greening (see details below). You may want to talk to Dan directly, as he has written a review paper on parallel simulated annealing (which is in the ftp repository in the file "greening.physicad.ps") ... Greening, D "Parallel Simulated Annealing Techniques" Physica D: Nonlinear Phenomena, Vol. 42, pp 293-306, 1990. And, of course, I'd like to see the results of your survey. Raymond Lister Department of Computer Science, F09 University of Sydney NSW 2006 AUSTRALIA Internet: ray at cs.su.oz.AU --- >From dgreen%cs.ucla.edu at munnari.cs.mu.oz Sat Oct 13 07:50:51 1990 >Date: Fri, 12 Oct 90 14:01:19 pdt >From: Dan R. Greening >To: anneal at cs.ucla.edu, glb at ecegriff.ncsu.edu >Subject: Anonymous FTP Site Available for Annealing Papers. I have set up squid.cs.ucla.edu as an anonymous FTP site for distributing papers on simulated annealing, spin-glass theory, etc. To get the widest audience possible, I would encourage you to put your paper(s) there. Please include a full reference on the first page which indicates where the paper appeared, if it has been published already, such as Appeared in Physica D: Nonlinear Phenomena, vol. 42, pp. 293-306, 1990. If the paper has not yet been published, include some reference which would allow a reader to locate the paper, such as Submitted for publication to the Conference on Advanced Research in VLSI, 1990. Available as IBM Technical Report RC-23456. You may also wish to include electronic mail information in your paper. You may want to announce its availability to the mailing list, by sending a message to anneal at cs.ucla.edu. HOW TO PUT A PAPER ON SQUID: REASONABLE PAPER FORMATS: 1. POSTSCRIPT. Almost everyone has access to a PostScript printer, so unless you absolutely have no other choice, please supply your paper in PostScript form. Append ".ps" to the filename to indicate postscript. 2. TROFF. You should only include troff if you have set up your troff file so that it requires NO COMMAND OPTIONS. Preprocess the troff input so all pic, tbl, eqn stuff is already included. Any macro packages should be included in the file itself. In short, someone should be able to produce your paper using only the file you provide. Append ".troff" to the filename to indicate troff. 3. DVI. You should only include a dvi file if it DOES NOT INCLUDE ENCAPSULATED POSTSCRIPT files (presumably if you have such files, you can generate the whole paper in postscript). Append ".dvi" to the filename to indicate a dvi file. Let's say that you formatted your paper, and have created a postscript file called paper.ps. Furthermore, suppose the first author is "Josie Smith" and you have submitted this paper to IEEE Transactions on CAD. By convention, the paper should be stored on squid as "smith.ieeetcad.ps". You can embellish the name as you wish, however, there is a maximum of 255 characters in a filename. Here goes (from UNIX): % ftp 131.179.96.44 (or ftp squid.cs.ucla.edu) login: anonymous password: anything ftp> cd anneal ftp> binary ftp> put paper.ps smith.ieeetcad.ps ftp> quit Now your paper will be sitting around for anyone to read! You might get famous! HOW TO GET A PAPER FROM SQUID: OK, suppose someone announces the availability of a paper on squid, called "smith.stoc1989.ps". Let's get a copy. Here goes (from UNIX): % ftp 131.179.96.44 (or ftp squid.cs.ucla.edu) login: anonymous password: anything ftp> cd anneal ftp> ls -l (might as well look around while we're here...) ftp> binary ftp> get smith.stoc1989.ps ftp> quit Now, just print out smith.stoc1989.ps and you discover something new! Hooray! I put a couple of my papers on there, already, as well as a paper by Dannie Durand. If you guys are nice (i.e., if you make me feel fulfilled by putting your papers there), maybe I'll put my much-discussed simulated annealing bibliography there, too. % I happen to have a copy of the bibliogaraphy, I think it is the most complete % one. There might be some overlaps between here and there. Yu Shen Happy Annealing (or Tabu Searching or Spinning or Conducting or whatever it is you're doing). -- Dan Greening | NY 914-784-7861 | 12 Foster Court dgreen at cs.ucla.edu | CA 213-825-2266 | Croton-on-Hudson, NY 10520 ------------ From sorkin at ravel.berkeley.edu Mon Oct 29 17:33:51 1990 From: sorkin at ravel.berkeley.edu (sorkin@ravel.berkeley.edu) Date: Mon, 29 Oct 90 14:33:51 -0800 Subject: No subject Message-ID: You should get in touch with two people who have done recent work in the area: Dan Greening (dgreen at cs.ucla.edu) and Dannie Durand (durand at cs.columbia.edu). Both did at least some of their work while at IBM, collaborating with Steve White (white at ibm.com) and Frederica Darema (darema at ibm.com). -- Greg Sorkin (sorkin at ic.berkeley.edu) ---------------- Some of the work by Harold Szu on fast simulated annealing compares Cauchy and Boltzmann machines. References follow in bib form. a:~ (52) tiblook szu cauchy %A Harold H. Szu %T Fast Simulated Annealing %J |AIP151| %P 420-426 %K Cauchy Machine %T Fast Simulated Annealing %A Harold H. Szu %A Ralph Hartley %J |PHYLA| %V 122 %N 3,4 %P 157-162 %D |JUN| 8, 1987 %K Cauchy Machine %T Nonconvex Optimization by Fast Simulated Annealing %A Harold H. Szu %A Ralph L. Hartley %J |IEEPro| %V 75 %N 11 %D |NOV| 1987 %K Cauchy Machine %T Design of Parallel Distributed Cauchy Machines %A Y. Takefuji %A Harold H. Szu %J |IJCNN89| ba:~ (53) tibabb phyla D PHYLA Phys. Lett. A D PHYLA Physics Letters. A ba:~ (54) tibabb IEEPro D IEEPro IEEE Proc. D IEEPro Institute of Electrical and Electronics Engineers. Proceedings ba:~ (55) tibabb IJCNN89 D IJCNN89 International Joint Conference of Neural Networks\ ba:~ (56) ------------------------------------------------------------------------------- . . .__. The opinions expressed herein are soley |\./| !__! Michael Plonski those of the author and do not represent | | | "plonski at aero.org" those of The Aerospace Corporation. _______________________________________________________________________________ @begin(refstyle) Ackley, D. H. (1984, December). @i(Learning evaluation functions in stochastic parallel networks.) CMU Thesis Proposal. ------------- ----------psa %A Emile H. L. Aarts %A Jan H. M. Korst %T Computation in Massively Parallel Networks based on the Boltzmann Machine: A Review %J Parallel Computing %V 9 %N 2 %D January 1989 %P 129-145 %K PARLE: conference on parallel architectures and languages -- Europe, Boltzmann machines, simulated annealing, combinatorial optimization, connectionist networks, neural networks, learning, %A Prithviraj Banerjee %A Mark Jones %T A Parallel Simulated Annealing Algorithm for Standard Cell Placement on Hypercube Computer %J NASA Review of ICLASS: Illinois Computer Laboratory for Aerospace Systems and Software %I University of Illinois %C Urbana-Champaign, IL %D October 23, 1986 %A Prithviraj Banerjee %A Mark Howard Jones %A Jeff S. Sargent %T Parallel Simulated Annealing Algorithms for Cell Placement on Hypercube Multiprocessors %J IEEE Transactions on Parallel and Distributed Systems %V PDS-1 %N 1 %D January 1990 %P 91-106 %K cell placement, error control, hypercube multiprocessor, parallel processing, performance measurements, simulated annealing, VLSI layout, %A Valmir C. Barbosa %A Eli Gafni %T A Distributed Implementation of Simulated Annealing %J Journal of Parallel and Distributed Computing %V 6 %N 2 %D April 1989 %P 411-434 %K special issue: neural computing, research notes, %A S. Wayne Bollinger %A Scott F. Midkiff %Z VA Poly, Blacksburg, VA %T Processor and Link Assignment in Multicomputers using Simulated Annealing %J Proceedings of the 1988 International Conference on Parallel Processing %V I, Architecture %I Penn State %C University Park, Penn %D August 1988 %P 1-7 %K distributed systems, mapping problem, process annealing, connection annealing, hypercube traffic, %A Ken W. Bosworth %A G. S. Stiles %A Rick Pennington %T A Parallel Nonlinear Integer Programming Algorithm Based on Branch and Bound and Simulated Annealing %E Garry Rodrigue %B Parallel Processing for Scientific Computing %I SIAM %C Philadelphia, PA %D 1989 %P 126-130 %K numerical methods, %A Casotto %A et al. %T A parallel simulated annealing algorithm for the placement of macro-cells %J ICCAD, 1986 %A A. Casotto %A A. Sangiovanni-Vincentelli %T Placement of Standard Cells Using Simlated Annealing on a Connection Machine %I Thinking Machines Corp. %R TMC TP CAD86-3 %D Dec. 1986 %K computer aided design, %A F. Darema-Rogers %A S. Kirkpatrick %A V. A. Norton %Z IBM T. J. Watson Research Center, Yorktown Heights, NY %T Simulated Annealing on Shared Memory Parallel Systems %J IBM J. on Res. & Dev. %D 1987 %r RC 12195 (#54812) %d October 1986 %A Nigel Dodd %Z Royal Radar, Malvern %T Graph matching by stochastic optimisation applied to the implementation of multi-layer perceptrons on transputer networks %J Parallel Computing %V 10 %N 2 %D April 1989 %P 135-142 %K graph matching, graph isomorphism, transputer, parallel processing, optimization, stochastic optimization, simulated annealing, MLP, %A F. Ercal %A J. Ramanujam %A P. Sadayappan %Z CIS, OSU %T Task Allocation onto a Hypercube by Recursive Mincut Bipartitioning %J 3rd Conference on Hypercube Concurrent Computers and Applications %V I, Architecture, Software, Computer Systems and General Issues %I ACM %C Pasadena, CA %D January 1988 %P 210-221 %K load balancing and decomposition, mapping problem, ARM (Kernighan/Lin), simulated annealing (SA), %A R. Fiebrich %A C. Wong %Z Thinking Machines Corp. %T Simulated Annealing-Based Circuit Placement Algorithm on the Connection Machine %J IEEE Intl. Conference on Computer Design %C Rye Brook, NY %D October 1987 %r TMC TP CAD87-2 %A G. C. Fox %A W. Furmanski %Z Caltech %T Load Balancing Loosely Synchronous Problems with a Neural Network %J 3rd Conference on Hypercube Concurrent Computers and Applications %V I, Architecture, Software, Computer Systems and General Issues %I ACM %C Pasadena, CA %D January 1988 %P 241-278 %r cccp-363B %d February 1988 %K load balancing and decomposition, particle analogy, graph partitioning, simulated annealing, statistical physics, Monte Carlo, Hopfield & Tank and Bold networks, %A Grover %T A new simulated annealing algorithm for standard cell placements %J ICCAD %D Nov. 1986 %A Harold M. Hastings %A Stefan Waner %Z Hofstra %T Neural Nets on the MPP %J Frontiers of Massively Parallel Scientific Computation %R Conference Proc. 2478 %I NASA Goddard Space Flight Center %C Greenbelt, MD %D Sept. 1986 %D 1987 %P 69-73 %K computer science, neural network, annealing system, stochastic cellular automaton, %A Huang %A et al. %T An efficient general cooling algorithm for simulated annealing %J ICCAD %D 1986 %T Mapping Partitioned Program Modules Onto Multicomputer Nodes Using Simulated Annealing %P II-292--II-293 %A Kai Hwang %A Jian Xu %Z USC, LA %J Proceedings of the 1990 International Conference on Parallel Processing %V II, Software %I Penn State U. Press %C University Park, Penn %D August 1990 %K poster session, Parallel Software Management, %A Daniel G. Jablonski %T Simulated Annealing, Reversible Computation and Neural Nets %I Supercomputing Research Center, IDA %C Lanham, MD %R SRC-TR-88-15 %D July 1988 %A Jepsen %A Gelatt %T Macro placement by Monte Carlo annealing %J ICCD Conf. %D Oct. 1983 %P 495-498 %A A. Kashko %A H. Buxton %A B. F. Buxton %A D. A. Castelow %T Parallel Matching and Reconstruction Algorithms in Computer Vision %J Parallel Computing %V 8 %N 1-3 %D October 1988 %P 3-17 %K Proc. Intl. Conf. on Vector and Parallel Processors in Computational Science, III [VAPP III], Aug. 1987, Liverpool, England. [Cover typo.] computer vision, SIMD, DAP, correlation, relaxation, simulated annealing, graduated non-convexity, %T Parallel Simulated Annealing Using Speculative Computation %P III-286--III-290 %A E. E. Witte %A R. D. Charnberlain %A M. A. Franklin %J Proceedings of the 1990 International Conference on Parallel Processing %V III, Algorithms and Applications %I Penn State U. Press %C University Park, Penn %D August 1990 %K parallel algorithms, ---------------------- Sharpening From jm2z+ at andrew.cmu.edu Mon Oct 29 11:06:38 1990 From: jm2z+ at andrew.cmu.edu (Javier Movellan) Date: Mon, 29 Oct 90 11:06:38 -0500 (EST) Subject: Parallel Implementation of Genectic Algorithm and Simulated In-Reply-To: <9010291459.AA23899@kovic.IRO.UMontreal.CA> References: <9010291459.AA23899@kovic.IRO.UMontreal.CA> Message-ID: Yu, There is a third related procedure called "sharpening". My first contact with the sharpening procedure was through the work of Akiyama et al (see reference) in what they called Gaussian machines ( Continuous Hopfield Networks with Gaussian noise injected in the net inputs). Sharpening is also used in Mean Field Networks (Continuous Hopfield Model with the Contrastive Hebbian Learning Algorithm). Sharpening may be seen as a deterministic, continuous approximation to annealing in Stochastic Boltzmann Machines. It works by starting settling using logistic activations with very low gain and increasing it as settling progresses. Sharpening, contrary to "true annealing" is deterministic and thus it may be faster. A similar procedure is used with elastic networks solving the TSP problem. References: Peterson, C & Anderson J (1987): A mean field theory learning algorithm for neural networks. Complex Systems, 1, 995-1019. Akiyama Y, Yamashita A, Kajiura M, Aiso H (1989) Combinatorial Optimization with Gaussian Machines. Proceedings of the IJCNN, 1, 533-540. Hinton G E (1989) Deterministic Boltzmann Learning Performs Stepest Descent in Weight Space, Neural Computation, 1, 143-150. Galland C, & Hinton G (1989) Deterministic Boltzmann Learing in Networks with Asymetric Connectiviy. University of Toronot. Department of Computer Science Technical Report. CRG-TR-89-6. Movellan J R (1990) Contrastive Hebbian Learning in the Continuous Hopfield Model. Proceedings of the 1990 Connectionist Summer School. Javier