From Dave.Touretzky at B.GP.CS.CMU.EDU Mon Oct 2 17:00:06 1989 From: Dave.Touretzky at B.GP.CS.CMU.EDU (Dave.Touretzky@B.GP.CS.CMU.EDU) Date: Mon, 02 Oct 89 17:00:06 EDT Subject: NIPS '89 preliminary program Message-ID: <14643.623365206@DST.BOLTZ.CS.CMU.EDU> Below is the preliminary program for the upcoming IEEE Conference on Neural Information Processing Systems - Natural and Synthetic, which will be held November 27 through 30, 1989. A postconference workshop series will take place November 30 through December 2. For registration information, please contact the Local Arrangements Chair, Kathie Hibbard, by sending email to hibbard at boulder.colorado.edu, or by writing to: Kathie Hibbard NIPS '89 University of Colorado Campus Box 425 Boulder, Colorado 80309-0425 ================================================================ ____________________________________________ ! ! ! PRELIMINARY PROGRAM, NIPS '89 ! ! DENVER, COLORADO ! ! NOVEMBER 27 _ NOVEMBER 30, 1989 ! !___________________________________________! OUTLINE Monday, November 27, 1989 4:00 PM: Registration 6:30 PM: Reception and Conference Dinner 8:30 PM: After-Dinner Plenary Talk by Jack Cowan Tuesday, November 28, 1989 8:00 AM: Continental Breakfast 8:30 AM - 12:30 PM: Oral Session1 - Neuroscience 12:30 - 2:30 PM: Poster Preview Session 1A, 1B, 1C - Neuroscience, Implementation and Simulation, Applications 2:30 - 6:30 PM: Oral Session 2 - Algorithms, Architectures, and Theory I 7:30 - 10:30 PM: Refreshments and Poster Session 1A,1B, 1C - Neuroscience, Implementation and Simulation, Applications Wednesday, November 29, 1989 8:00 AM: Continental Breakfast 8:30 AM - 12:30 PM: Oral Session3 - Applications 12:30 - 2:30 PM: Poster Preview Session 2 - Algorithms, Architectures, and Theory 2:30 - 6:30 PM: Oral Session 4 - Implementationand Simulation 7:30 - 10:30 PM: Refreshments and Poster Session 2 - Algorithms, Architectures, and Theory Thursday, November 30, 1989 8:00 AM: Continental Breakfast 8:30 AM - 1:00 PM: OralSession 5 - Algorithms, Architectures, and Theory II Friday, December 1 - Saturday, December 2, 1989 Post Conference Workshops at Keystone ________________________________ ! MONDAY, NOVEMBER 27, 1989 ! !______________________________! 4:00 PM: Registration 6:30 PM: Reception and Conference Dinner 8:30 PM: After-Dinner Plenary Talk "Some NeuroHistory: Neural Networks from 1952-1967," by Jack Cowan - University of Chicago. ________________________________ ! TUESDAY, NOVEMBER 28, 1989 ! !_______________________________! ORAL SESSION 1 NEUROSCIENCE SESSION CHAIR: James Bower, California Institute of Technology Tuesday, 8:30 AM - 12:30 PM 8:30 "Acoustic-Imaging Computations by Echolocating Bats: Unification of Diversely-Represented Stimulus Features into Whole Images," by Jim Simmons - Brown University (Invited Talk). 9:10 "Rules for Neuromodulation of Small Neural Circuits," by Ronald M. Harris-Warrick - Section of Neurobiology and Behavior, Cornell University. 9:40 "Neural Network Analysis of Distributed Representations Of Sensory Information In The Leech," by S.R. Lockery, G. Wittenberg, W. B. Kristan Jr., N. Qian and T. J. Sejnowski -Department of Biology, University of California, San Diego and Computational Neurobiology Laboratory, The Salk Institute. 10:10 BREAK 11:00 "Reading a Neural Code,"by William Bialek, Fred Rieke, R. R. de Ruyter van Steveninck, and David Warland - Departments of Physics and Biophysics, University of Californiaat Berkeley. 11:30 "Neural Network Simulation of Somatosensory Representational Plasticity," by KamilA. Grajski and Michael M. Merzenich - Coleman Memorial Laboratories, University of California, San Francisco. 12:00 "Brain Maps and Parallel Computer Maps," by Mark E. Nelson and James Bower - Division of Biology, California Institute of Technology. POSTER PREVIEW SESSION 1A NEUROSCIENCE Tuesday, 12:30 - 2:30 PM A1. "Category Learning and Object Recognition in a Simple Oscillating Model of Cortex " by Bill Baird - Department of Physiology, University of California Berkeley. A2. "From Information Theory to Structure and Function in a Simplified Model of a Biological Perceptual System," by Ralph Linsker - IBM Research, T. J. Watson Research Center. A3. "Development and Regeneration of Brain Connections: A Computational Theory," by J.D. Cowan and A.E. Friedman - Mathematics Department, University of Chicago. A4. "Collective Oscillations in Neuronal Networks: Functional Architecture Drives Dynamics," by Daniel M. Kammen, Philip J. Holmes, and Christof Koch - Computation and Neural Systems Program, California Institute of Technology. A5. "Computer Simulation of Oscillatory Behavior in Cerebral Cortical Networks," by M.A. Wilson and J.M. Bower - Computation and Neural Systems Program, Division of Biology, California Institute of Technology. A6. "A Neural Network Model of Catecholamine Effects: Enhancement of Signal Detection Performance is an Emergent Property of Changes in Individual Unit Behavior," by David Servan-Schreiber, Harry Printz and Jonathan Cohen - Departments of Computer Science and Psychology, Carnegie Mellon University. A7. "Non-Boltzmann Dynamics in Networks of Spiking Neurons," by Michael Crair and William Bialek - Departments ofPhysics and Biophysics, University of California at Berkeley. A8. "A Computer Modeling Approach toUnderstanding the Inferior Olive and Its Relationship to the Cerebellar Cortexin Rats," by Maurice Lee and James M. Bower - Computation and Neural Systems Program, California Institute of Technology. A9. "An Analog VLSI Model of Adaptationin the Vestibulo-Ocular Reflex," by Stephen P. DeWeerth and Carver A. Mead - California Institute of Technology. A10. "Can Simple Cells Learn Curves? A Hebbian Model in a Structured Environment," by William R. Softky and Daniel M. Kammen - Divisions of Physics and Biology and Computation and Neural Systems Program, California Institute of Technology. A11. "Formation of Neuronal Groupsin Simple Cortical Models," by Alex Chernjavsky and John Moody - Section of Molecular Neurobiology, Howard Hughes Medical Institute,Yale University. A12. "Signal Propagation in Layered Networks," by Garrett T. Kenyon, Eberhard E. Fetz and Robert D. Puff - University of Washington, Department of Physics. A13. "A Systematic Study of the Input/OutputProperties of a Model Neuron With Active Membranes," by Paul Rhodes - University of California, San Diego. A14. "Analytic Solutions to the Formation of Feature-Analyzing Cells of a Three-Layer Feedforward Information Processing Neural Net," by D.S. Tang - Microelectronics and Computer Technology Corporation. A15. "The Computation of Sound Source Elevation in the Barn Owl" by C.D. Spence and J.C. Pearson, David Sarnoff Research Center. POSTER PREVIEW SESSION 1B IMPLEMENTATION AND SIMULATION Tuesday, 12:30 - 2:30 PM B1. "Real-Time Computer Vision and Robotics Using Analog VLSI Circuits," by Christof Koch, John G. Harris, Tim Horiuchi, Andrew Hsu, and Jin Luo - Computation and Neural Systems Program, California Institute of Technology. B2. "The Effects of Circuit Integration on a Feature Map Vector Quantizer," by Jim Mann - MIT Lincoln Laboratory. B3. "Pulse-Firing Neural Chips Implementing Hundreds of Neurons," by Alan F. Murray, Michael Brownlow, AlisterHamilton, Il Song Han, H. Martin Reekie, and Lionel Tarassenko - Department of Electrical Engineering, University of Edinburgh, Scotland. B4. "An Efficient Implementation ofthe Backpropagation Algorithm on the Connection Machine CM-2," by Xiru Zhang, Michael Mckenna, Jill P. Mesirov, and David Waltz - Thinking Machines Corporation. B5. "Performance of Connectionist Learning Algorithms on 2-D SIMD Processor Arrays," by Fernando J. Nunez and Jose A.B. Fortes - School of Electrical Engineering, Purdue University. B6. "Dataflow Architectures: Flexible Platforms for Neural Network Simulation," by I.G. Smotroff - The MITRE Corporation. B7. "Neural Network Visualization," by Jakub Wejchert and Gerald Tesauro - IBM Research, T.J. Watson Research Center. POSTER PREVIEW SESSION 1C APPLICATIONS Tuesday, 12:30 - 2:30 PM C1. "Computation and Learning in Artificial Dendritic-Type Structures: Application to Speech Recognition," by Tony Bell - Free University of Brussels, Belgium. C2. "Speaker Independent Speech Recognition with Neural Networks and Speech Knowledge," by Yoshua Bengio, Regis Cardin, and Renato De Mori - McGill University, School of Computer Science. C3. "HMM Speech Recognition with Neural Net Discrimination," by William Y. Huang and Richard P. Lippmann- MIT Lincoln Laboratory. C4. "Connectionist Architectures for Multi-Speaker Phoneme Recognition," by John B. Hampshire II and Alex H. Waibel - School of Computer Science, Carnegie Mellon University. C5. "Performance Comparisons Between Backpropagation Networks and Classification Trees on Three Real-World Applications," by Les Atlas, Ronald Cole, Yeshwant Muthusamy, James Taylor, and Etienne Barnard - Department of Electrical Engineering, University of Washington, Seattle. C6. "Combining Visual and Acoustic Speech Signals with a Neural Network Improves Intelligibility," by Ben P. Yuhas, M.H. Goldstein, Jr., and Terrence J. Sejnowski - Speech Processing Laboratory, Department of Electrical and Computer Engineering, Johns Hopkins University. C7. "A Neural Network for Real-Time Signal Processing," by Donald B. Malkoff - General Electric / Advanced Technology Laboratories. C8. "A Neural Network to Detect Homologies in Proteins," by Yoshua Bengio, Yannick Pouliot, Samy Bengio,and Patrick Agin - McGill University, School of Computer Science. C9. "Recognizing Hand-Drawn and Handwritten Symbols with Neural Nets," by Gale L. Martin and James A. Pittman - MCC,Austin. C10. "Model Based Image Compression and Adaptive Data Representation by Interacting Filter Banks," by Toshiaki Okamoto, Mitsuo Kawato, Toshio Inui, and Sei Miyake - ATR Auditory and Visual Perception Research Laboratories, Japan. C11. "A Large-Scale Network Which Recognizes Handwritten Kanji Characters," by Yoshihiro Mori and Kazuki Joe - ATR Auditory and Visual Perception Research Laboratories, Japan. C12. "Traffic: Object Recognition Using Hierarchical Reference Frame Transformations," by Richard S. Zemel, Michael C. Mozer, and Geoffrey Hinton - Department of Computer Science, University of Toronto. C13. "Comparing the Performance of Connectionist and Statistical Classifiers on an Image Segmentation Problem," by Sheri L. Gish and W.E. Blanz - IBM Knowledge Based Systems, Menlo Park. C14. "A Modular Architecture For Target Recognition Using Neural Networks," by Murali M. Menon and Eric J. Van Allen - MIT Lincoln Laboratory. C15. "Neurally Inspired Plasticity in Oculomotor Processes," by Paul Viola - Artificial Intelligence Laboratory, Massachusetts Institute of Technology. C16. "Neuronal Group Selection Theory: A Grounding in Robotics," by Jim Donnett and Tim Smithers - Department of Artificial Intelligence, University of Edinburgh, Scotland. C17. "Composite Holographic Associative Recall Model (CHARM) and Recognition Failure of Recallable Words," by Janet Metcalfe - Department of Psychology, University of California, San Diego. C18. "Using a Translation-Invariant Neural Network to Diagnose Heart Arrhythmia," by Susan Lee - Johns Hopkins Institute. C19. "Exploring Bifurcation Diagrams With Adaptive Networks," by Alan S. Lapedes and Robert M. Farber - Theoretical Division, Los Alamos National Laboratory. C20. "Generalized Hopfield Networks and Nonlinear Optimization," by Athanasios G. Tsirukis, Gintaras V. Reklaitis, and Manoel F. Tenorio - School of Chemical Engineering, Purdue University. ORAL SESSION 2 ARCHITECTURES, ALGORITHMS, AND THEORY I SESSION CHAIR: John Moody, Yale University Tuesday, 2:30 - 6:30 PM 2:30 "Statistical Properties of Polynomial Networks and Other Artificial Neural Networks: A Unifying View," by Andrew Barron - University of Illinois at Champaign-Urbana (Invited Talk). 3:10 "Supervised Learning: A Theoretical Framework," by Sara Solla, Naftali Tishby, and Esther Levin - AT&T Bell Laboratories. 3:40 "Practical Characteristics of Neural Network and Conventional Pattern Classifiers on Artificial and Speech Problems," by Yuchun Lee and Richard P. Lippmann - Digital Equipment Corporation and MIT Lincoln Laboratory. 4:10 BREAK 5:00 "The Cocktail Party Problem: Speech/Data Signal Separation Comparison Between Backprop and SONN," by Manoel F. Tenorio, John Kassebaum, and Christoph Schaefers - School of Electrical Engineering, Purdue University. 5:30 "Optimal Brain Damage," by Yann LeCun, John Denker, Sara Solla, Richard E. Howard, and Lawrence D. Jackel - AT&T Bell Laboratories. 6:00 "Sequential Decision Problems and Neural Networks," by Andrew G. Barto, Richard S. Sutton and Chris Watkins -Department of Computer and Information Science, University of Massachusetts, Amherst. POSTER SESSION 1A, 1B, 1C NEUROSCIENCE, IMPLEMENTATION AND SIMULATION, APPLICATIONS Tuesday, 7:30 - 10:30 PM (Papers are Listed Under Poster Preview Session) ___________________________________ ! WEDNESDAY, NOVEMBER 29, 1989 ! !__________________________________! ORAL SESSION 3 APPLICATIONS SESSION CHAIR: Richard Lippmann, MIT Lincoln Laboratory Wednesday, 8:30 AM - 12:30 PM 8:30 "Visual Preprocessing" by George Sperling - New York University (Invited Talk). 9:10 "Handwritten Digit Recognition with a Back-Propagation Network," by Y. LeCun, B. Boser, J.S. Denker, D. Henderson,R.E. Howard, W. Hubbard, and L.D. Jackel - AT&T BellLab oratories. 9:40 "A Self-Organizing Associative Memory System for Control Applications," by Michael Hormel - Department ofControl Theory and Robotics, Technical University of Darmstadt, Germany. 10:10 BREAK 11:00 "Variable Resolution Learning Techniques for Speech Recognition," by Kevin Lang and Geoffrey Hinton - Carnegie-Mellon University. 11:30 "Word Recognition in a Continuous Speech Recognition System Embedding MLP into HMM," by H. Bourlard andN. Morgan - International Computer Science Institute, Berkeley. 12:00 "A Computational Basis for Phonology," by David S. Touretzky and Deirdre W. Wheeler - Carnegie-Mellon University. POSTER PREVIEW SESSION 2 ARCHITECTURES, ALGORITHMS, AND THEORY Wednesday, 12:30 - 2:30 PM 1. "Using Local Networks to Control Movement," by ChristopherG. Atkeson - Department of Brain and Cognitive Sciencesand the Artificial Intelligence Laboratory, Massachusetts Institute of Technology. 2. "Computational Neural Theory for Learning Nonlinear Mappings," by Jacob Barhen and Sandeep Gulati - Jet PropulsionLab oratory, California Institute of Technology. 3. "Learning to Control an Unstable System Using Forward Modeling," by Michael I. Jordan and Robert A. Jacobs - Department of Brain and Cognitive Sciences, Massachusetts Institute ofTechnology. 4. "Discovering High Order Features With Mean Field Networks," by Conrad Galand and Geoffrey E. Hinton - Departmentof Computer Science, University of Toronto. 5. "Designing Application-Specific Neural Networks Using the Genetic Algorithm," by Steven A. Harp, Tariq Samad, and Aloke Guha - Honeywell CSDD. 6. "Two vs. Three Layers: An Empirical Study of Learning Performance and Emergent Representations," by Charles Martin and John Moody - Department of Computer Science, Yale University. 7. "Operational Fault Tolerance of CMAC Networks," by Michael J. Carter, Frank Rudolph, and Adam Nucci - IntelligentStructures Group, Dept. of Electrical and Computer Engineering, University of New Hampshire. 8. "A Model of Unification in Connectionist Networks," by Andreas Stolcke - Computer Science Division, University of California, Berkeley. 9. "Two-Dimensional Shape Recognition Using Sparse Distributed Memory: An Example of a Machine Vision System that Exploits Massive Parallelism for Both High-Level and Low-Level Processing," by Bruno Olshausen and Pentti Kanerva - Research Institute for Advanced Computer Science, NASA Ames Research Center. 10. "Predicting Weather Using a Genetic Memory: A Combination of Kanerva's Sparce Distributed Memory With Holland's Genetic Algorithms," by David Rogers - Research Institute for Advanced Computer Science, NASA Ames Research Center. 11. "Neural Network Weight Matrix Synthesis Using Optimal Control," by O. Farotimi, A. Dembo, and T. Kailath - Information Systems Laboratory, Department of Electrical Engineering, Stanford University. 12. "The CHIR Algorithm: A Generalization for Multiple Output Networks," by Tal Grossman - Department ofElectronics, Weizmann Institute of Science, Israel. 13. "Analysis of Linsker's Application of Hebbian Rules to Linear Networks," by David J. C. MacKay and Kenneth D. Miller - Department of Computation and Neural Systems, California Institute of Technology and Department of Physiology, University of California, San Francisco. 14. "A Generative Framework for Unsupervised Learning," by Steven J. Nowlan - Department of Computer Science, University of Toronto. 15. "An Adaptive Network Model of Basic-Level Learning in Hierarchically Structured Categories," by Mark A. Gluck, James E. Corter, and Gordon H. Bower - Stanford University. 16. "Generalization and Scaling in Reinforcement Learning," by David H. Ackley and Michael S. Littman - Bell Communications Research, Cognitive Science Research Group. 17. "Neural Implementation of Motivated Behavior: Feeding in an Artificial Insect," by Randall D. Beer and Hillel J.Chiel - Departments of Computer Engineering and Science and Biology and the Center for Automation and Intelligent Systems Research, Case Western Reserve University. 18. "Back Propagation in a Genetic Search Environment," by Wayne Mesard and Lawrence Davis - Bolt Beranek and Newman Systems and Technologies, Inc., Laboratories Division. 19. "A Method for the Associative Storage of Analog Vectors," by Amir F. Atiya and Yaser S. Abu-Mostafa - Department of Electrical Engineering, California Institute of Technology. 20. "Generalization and Parameter Estimation in Feedforward Nets: Some Experiments," by N. Morgan and H. Bourlard - International Computer Science Institute, Berkeley. 21. "Subgrouping Reduces Complexity and Speeds Up Learning in Recurrent Networks," by David Zipser- Department of Cognitive Science, University of California, San Diego. 22. "Sigma-Pi Learning: A Model for Associative Learning in Cerebral Cortex," by Bartlett W. Mel and Christof Koch - Computation and Neural Systems Program, California Institute of Technology. 23. "Complexity of Finite Precision Neural Network Classifier," by K. Y. Siu, A. Dembo, and T. Kailath - Information Systems Laboratory, Stanford University. 24. "Analog Neural Networks of Limited Precision I: Computing With Multilinear Threshold Functions," by Zoran Obradovic and Ian Parberry - Department of Computer Science, Pennsylvania State University. 25. "On the Distribution of the Local Minima of a Random Function of a Graph," by P. Baldi, Y. Rinott, and C. Stein - University of California, San Diego. 26. "A Neural Network For Feature Extraction," by Nathan Intrator - Center for Neural Science and Division of Applied Mathematics, Brown University. 27. "Meiosis Networks," by Stephen Jose Hanson - Cognitive Science Laboratory, Princeton University. 28. "Unsupervised Learning Using Velocity Field Approach," by Michail Zak - Jet Propulsion Laboratory,California Institute of Technology. 29. "Algorithms for Better Representation and Faster Learning in Radial Basis Function Networks," by Avijit Saha and James D. Keeler - MCC Austin, Texas. 30. "Generalization Performance of Overtrained Back-Propagation Networks: Some Experiments," by Y. Chauvin - Psychology Department, Stanford University. 31. "The 'Moving Targets' Training Method," by Richard Rohwer - Centre for Speech Technology Research, University of Edinburgh, Scotland. 32. "Optimal Learning and Inference Over MRF Models: Application To Computational Vision on Connectionist Architectures," by Kurt R. Smith, Badrinath Roysam, and Michael I. Miller - Washington University. 33. "A Cost Function for Learning Internal Representations," by J.A. Hertz, A. Krogh, and G.I. Thorbergsson - Niels Bohr Institute, Denmark. 34. "The Cascade-Correlation Learning Architecture," by Scott E. Fahlman and Christian Lebiere - School of Computer Science, Carnegie-Mellon University. 35. "Training Connectionist Networks With Queries and Selective Sampling," by D. Cohn, L. Atlas, R. Ladner, R. Marks II, M. El-ASharkawi, M. Aggoune, D. Park - Dept. of Electrical Engineering, University of Washington. 36. "Rule Representations in a Connectionist Chunker," by David S. Touretzky - School of Computer Science, Carnegie Mellon University. 37. "Unified Theory for Symmetric and Asymmetric Systems and the Relevance to the Class of Undecidable Problems," by I. Kanter - Princeton University. 38. "Synergy of Clustering Multiple Back Propagation Networks," by William P. Lincoln and Josef Skrzypek - Hughes Aircraft Company and Machine Perception Laboratory, UCLA. 39. "Training Stochastic Model Recognition Algorithms as Networks Can Lead to Maximum Mutual Information Estimation of Parameters," by John S. Bridle - Machine Intelligence Theory Section, Royal Signals and Radar Establishment, Great Britain. 40. "Self-Organizing Multiple-View Representations of 3D Objects," by D. Weinshall, S. Edelman, and H. Bulthoff - MIT Center for Biological Information Processing. 41. "A Recurrent Network that Learns Context-Free Grammars," by G.Z. Sun, H.H. Chen, C.L. Giles, Y.C. Lee, and D. Chen - Laboratory for Plasma Physics Research and Institute for Advanced Computer Studies, University of Maryland. 42. "Time Dependent Adaptive Neural Networks," by F. J. Pineda - Jet Propulsion Laboratory, California Institute of Technology. ORAL SESSION 4 IMPLEMENTATION AND SIMULATION SESSION CHAIR: Jay Sage, MIT Lincoln Laboratory Wednesday, 2:30 - 6:30 PM 2:30 "Visual Object Recognition" by Shimon Ullman - Massachusetts Institute of Technology and Weizmann Institute of Science (Invited Talk). 3:10 "A Reconfigurable Analog VLSI Neural Network Chip," by Srinagesh Satyanarayana, Yannis Tsividis, and Hans Peter Graf - Department of Electrical Engineering and Center for Telecommunications Research, Columbia University. 3:40 "Analog Circuits for Constrained Optimization," by John Platt - California Institute of Technology. 4:10 BREAK 5:00 "VLSI Implementation of a High-Capacity Neural Associative Memory," by Tzi-Dar Chiueh and Rodney M. Goodman - Department of Electrical Engineering, California Institute of Technology. 5:30 "Hybrid Analog-Digital 32x32x6-Bit Synapse Chips for Electronic Neural Networks," by A. Moopenn, T. Duong,and A. P. Thakoor - Jet Propulsion Laboratory, California Institute of Technology. 6:00 "Learning Aspect Graph Representations From View Sequences," by Michael Seibert and Allen M. Waxman - MIT Lincoln Laboratory. POSTER SESSION 2 ARCHITECTURES, ALGORITHMS, AND THEORY Wednesday, 7:30 - 10:30 PM (Papers are Listed Under Poster Preview Session) __________________________________ ! THURSDAY, NOVEMBER 30, 1989 ! !________________________________! ORAL SESSION 5 ARCHITECTURES, ALGORITHMS, AND THEORY II SESSION CHAIR: Eric Baum, NEC Research Institute Thursday, 8:30 AM - 1:00 PM 8:30 "Identification and Control of Dynamical Systems Using Neural Networks," by Bob Narendra - YaleUniversity (Invited Talk). 9:10 "Discovering the Structure of a Reactive Environment by Exploration," by Michael C. Mozer and Jonathan Bachrach - University of Colorado Boulder. 9:40 "The Perceptron Algorithm Is Fast at Modified Valiant Learning," by Eric B. Baum - Department of Physics, PrincetonUniversity. 10:10 BREAK 11:00 "Oscillations in Neural Computations," by Pierre Baldi and Amir Atiya - Jet Propulsion Laboratory and Division ofBiology, California Institute of Technology. 11:30 "Incremental Parsing by Modular Recurrent Connectionist Networks," by Ajay Jain and Alex Waibel - School of ComputerScience, Carnegie Mellon University. 12:00 "Neural Networks From Coupled Markov Random Fields via Mean Field Theory," by Davi Geiger and Federico Girosi - Artificial Intelligence Laboratory, Massachusetts Institute of Technology. 12:30 "Asymptotic Convergence of Back-Propagation," by Gerald Tesauro, Yu He, and Subatai Ahmad - IBM Thomas J. Watson Research Center. ____________________________________________________________ ! POST CONFERENCE WORKSHOPS AT KEYSTONE ! ! THURSDAY, NOVEMBER 30 - SATURDAY, DECEMBER 2, 1989 ! !____________________________________________________________! Thursday, November 30, 1989 5:00 PM: Registration and Reception at Keystone Friday, December 1, 1989 7:30 - 9:30 AM: Small Group Workshops 4:30 - 6:30 PM: Small Group Workshops 8:30 - 10:30 PM: Plenary Discussion Session Saturday, December 2, 1989 7:30 - 9:30 AM: Small Group Workshops 4:30 - 6:30 PM: Small Group Workshops 7:00 PM: Banquet From elman at amos.ucsd.edu Tue Oct 3 01:23:31 1989 From: elman at amos.ucsd.edu (Jeff Elman) Date: Mon, 2 Oct 89 22:23:31 PDT Subject: connectionist book series announcement Message-ID: <8910030523.AA07232@amos.ucsd.edu> - New Book Series Announcement - NEURAL NETWORK MODELING & CONNECTIONISM The MIT Press / Bradford Books This series will make available seminal state-of-the art research in neural network and connectionist modeling. The research in this area has grown explosively in recent years and has sparked controversy and debate in a wide variety of areas. Many researchers believe that this para- digm offers new and deep insights into the basis and nature of intelligent behavior in both biological and artificial systems. The series publishing program will include: monographs based on influential dissertations; monographs and in-depth reports of research programs based on mature work by leaders in the field; edited volumes and collections on topics of special interest; major reference works; undergraduate and graduate level textbooks. The series will be highly inter- disciplinary, spanning fields as diverse as psychology, linguistics, cognitive science, neuroscience, neurobiology and genetics, physics and biophysics, mathematics, computer science, artificial intelligence, engineering, and econom- ics. Potential authors are encouraged to contact any of the Editors or the Publisher. Editor: Jeffrey L. Elman Dept. of Cognitive Science UCSD; La Jolla, CA 92093 elman at amos.ucsd.edu Associate Editors: James Anderson (Brown) James McClelland (CMU) Andrew Barto (UMass/Amherst) Domenico Parisi (Rome) Gary Dell (Illinois) David Rumelhart (Stanford) Jerome Feldman (ICSI, Berkeley) Terrence Sejnowski (UCSD, Salk) Stephen Grossberg (BU) Paul Smolensky (Colorado) Stephen Hanson (Bellcore) Stephen Stich (Rutgers) Geoffrey Hinton (Toronto) David Touretzky (CMU) Michael Jordan (MIT) David Zipser (UCSD) Publisher: Henry B. Stanton The MIT Press / Bradford Books 55 Hayward Street; Cambridge MA 02142 From TESAURO at ibm.com Tue Oct 3 13:26:58 1989 From: TESAURO at ibm.com (Gerald Tesauro) Date: 3 Oct 89 13:26:58 EDT Subject: Neurogammon wins Computer Olympiad Message-ID: <100389.132658.tesauro@ibm.com> Neurogammon 1.0 is a backgammon program which uses multi-layer neural networks to make move decisions and doubling cube decisions. The networks were trained by back-propagation on large expert data sets. Neurogammon competed at the recently-held First Computer Olympiad in London, and won the backgammon competition with a perfect record of 5 wins and no losses. This is a victory not only for neural networks, but for the entire machine learning community, as it is apparently the first time in the history of computer games that a learning program has ever won a tournament. A short paper describing Neurogammon and the Olympiad results will appear in the next issue of Neural Computation. (This was inadver- tently omitted from Terry Sejnowski's recent announcement of the contents of the issue.) The paper may also be obtained on-line in plain text format by sending e-mail to TESAURO at ibm.com. From LEO at AUTOCTRL.RUG.AC.BE Tue Oct 3 11:20:00 1989 From: LEO at AUTOCTRL.RUG.AC.BE (LEO@AUTOCTRL.RUG.AC.BE) Date: Tue, 3 Oct 89 11:20 N Subject: Call for Papers Neural Network Applications Message-ID: *****************+--------------------------------------+**************** *****************|Second BIRA seminar on Neural Networks|**************** *****************+--------------------------------------+**************** CALL FOR PAPERS May 1990, Belgium Last year, BIRA (Belgian Institute for Automatic Control) organised a first seminar on Neural Networks. Some invited speakers (Prof. Fogelman Soulie, Prof. Bart Kosko, Dr. D. Handelman and Dr. S. Miyake) gave an introduction to the subject, and discussed some possible application fields. Because of the great interest from the industry as well as from the research world, we decided to organise a second edition on this subject. The aim of the second seminar is to show some excisting applications and possibilities of Neural Networks or Sub-Symbolic Systems with Neural Network features. So, if you have a working application or nice prototype of an industrial application based on Neural Networks, and you may and want to talk about it, please send us an abstract. Of course, the seminar will only be organised, if we receive enough interesting abstracts. This seminar will be organised by BIRA, Unicom and the AI-section of the Automatic Control Laboratory of the Ghent State University. Time schedule ------------- 01-01-1990 : Deadline for abstracts. 15-02-1990 : Confirmation of the speakers and the seminar 01-04-1990 : Deadline for full papers ..-05-1990 : Seminar Organisation contact information -------------------------------- Rob Vingerhoeds Leo Vercauteren State University of Ghent AI Section Automatic Control Laboratory Grote Steenweg Noord 2 B-9710 GENT - Zwijnaarde Belgium Fax: +32 91/22 85 91 Tel: +32 91/22 57 55 BIRA Coordinator: L. Pauwels BIRA-secretariaat Het Ingenieurshuis Desguinlei 214 2018 Antwerpen Belgium From John.Hampshire at SPEECH2.CS.CMU.EDU Tue Oct 3 10:08:17 1989 From: John.Hampshire at SPEECH2.CS.CMU.EDU (John.Hampshire@SPEECH2.CS.CMU.EDU) Date: Tue, 3 Oct 89 10:08:17 EDT Subject: TR announcement Message-ID: **************** PLEASE DO NOT FORWARD TO OTHER MAILING LISTS **************** Technical Report: CMU-CS-89-166 THE META-PI NETWORK: BUILDING DISTRIBUTED KNOWLEDGE REPRESENTATIONS FOR ROBUST PATTERN RECOGNITION J. B. Hampshire II and A. H. Waibel QUICK ABSTRACT (30 seconds, plain English, no frills) The "Meta-Pi" architecture is a multi-network connectionist backprop structure. It learns to focus attention on the output of a particular sub-network or group of sub-networks via multiplicative connections. When used to perform multi-speaker speech recognition this network yields recognition error rates as low as those for speaker DEpendent tasks (~98.5%), and about one third the rate of more traditional networks trained on the same multi-speaker task. Meta-Pi networks are trained for best output performance and *automatically* learn the best mix or selection of neural subcomponents. Here, for example, they learned about relevant speaker differences (and similarities) without being told to actually recognize the different speakers. If this sounds interesting, please read on. SUMMARY We present a multi-network connectionist architecture that forms distributed low-level knowledge representations critical to robust pattern recognition in non-stationary stochastic processes. This new network comprises a number of stimulus-specific sub-networks (i.e., networks trained to classify a particular type of stimulus) that are linked by a combinational superstructure. Our application employs Time-Delay Neural Network (TDNN) architectures for the sub-networks and the combinational superstructure of the Meta-Pi network, although one can use any form of backpropagation network as the basis for a Meta-Pi architecture. The combinational superstructure of the Meta-Pi network adapts to the stimulus being processed, optimally integrating stimulus-specific classifications based on its internally-developed model of the stimulus (or combination of stimuli) most likely to have produced the input signal. To train this combinational network we have developed a new form of multiplicative connection that we call the ``Meta-Pi'' connection. We illustrate how the Meta-Pi paradigm implements a dynamically adaptive Bayesian connectionist classifier. We demonstrate the Meta-Pi architecture's performance in the context of multi-speaker phoneme recognition. In this task the Meta-Pi superstructure integrates TDNN sub-networks to perform multi-speaker phoneme recognition at speaker-DEpendent rates. It achieves a 6-speaker (4 males, 2 females) recognition rate of 98.4% on a database of voiced-stops (/b,d,g/). This recognition performance represents a significant improvement over the 95.9% multi-speaker recognition rate obtained by a single TDNN trained in multi-speaker fashion. It also approaches the 98.7% average of the speaker-DEpendent recognition rates for the six speakers processed. We show that the Meta-Pi network can learn --- without direct supervision --- to recognize the speech of one particular speaker using a dynamic combination of internal models of *other* speakers exclusively (99.8% correct). The Meta-Pi model constitutes a viable basis for connectionist pattern recognition systems that can rapidly adapt to new stimuli by using dynamic, conditional combinations of existing stimulus-specific models. Additionally, it demonstrates a number of performance characteristics that would be desirable in autonomous connectionist pattern recognition systems that could develop and maintain their own database of stimuli models, adapting to new stimuli when possible, spawning new stimulus-specific learning processes when necessary, and eliminating redundant or obsolete stimulus-specific models when appropriate. This research has been funded by Bell Communications Research, ATR Interpreting Telephony Research Laboratories, and the National Science Foundation (NSF grant EET-8716324). REQUESTS: Please send requests for tech. report CMU-CS-89-166 to hamps at speech2.cs.cmu.edu (ARPAnet) "Allow 4 weeks for delivery..." **************** PLEASE DO NOT FORWARD TO OTHER MAILING LISTS **************** From pollack at cis.ohio-state.edu Wed Oct 4 00:02:32 1989 From: pollack at cis.ohio-state.edu (Jordan B Pollack) Date: Wed, 4 Oct 89 00:02:32 EDT Subject: Neurogammon wins Computer Olympiad In-Reply-To: Gerald Tesauro's message of 3 Oct 89 13:26:58 EDT <100389.132658.tesauro@ibm.com> Message-ID: <8910040402.AA03037@toto.cis.ohio-state.edu> Congratulations! It even beat Berliner's old SNIC(?) game? From mdtom at en.ecn.purdue.edu Wed Oct 4 17:39:34 1989 From: mdtom at en.ecn.purdue.edu (M Daniel Tom) Date: Wed, 04 Oct 89 16:39:34 EST Subject: TR-EE 89-54: Analyzing NETtalk for Speech Development Modelling Message-ID: <8910042139.AA19723@en.ecn.purdue.edu> ---------------------------------------------------------------------- Requests from within US, Canada, and Mexico: The technical report with figures, and cluster plots, have been placed in the account kindly provided by Ohio State. Here is the instructions to get the files: unix> ftp cheops.cis.ohio-state.edu (or, ftp 128.146.8.62) Name: anonymous Password: neuron ftp> cd pub/neuroprose ftp> mget tenorio.* (type y and hit return) ftp> quit unix> uncompress tenorio.*.Z unix> lpr -P(your_postscript_printer) tenorio.speech_dev.ps unix> lpr -P(your_132_column_printer) tenorio.cluster.plain ---------------------------------------------------------------------- Requests from outside North America: The technical report is available at a cost of US$8.38 per copy, postage included. Please make checks payable to Purdue University in US dollars. You may send your requests, checks, and full first class mail address to: J. L. Dixon School of Electrical Engineering Purdue University West Lafayette, Indiana 47907 USA Please mention the technical report number: TR-EE 89-54. Please also note that the hard copy of the technical report does not include cluster plots mentioned above. ---------------------------------------------------------------------- Adaptive Networks as a Model for Human Speech Development M. Fernando Tenorio M. Daniel Tom School of Electrical Engineering and Richard G. Schwartz Department of Audiology and Speech Sciences Parallel Distributed Structures Laboratory Purdue University West Lafayette, IN 47907 TR-EE 89-54 August 1989 Abstract Unrestricted English text can be converted to speech through the use of a look up table, or through a parallel feedforward network of deterministic processing units. Here, we reproduce the network structure used in NETtalk. Several experiments are carried out to determine which characteristics of the network are responsible for which learning behavior, and how closely that maps into human speech development. The network is trained with different levels of speech complexity (children and adult speech,) and with Spanish a second language. Developmental analyses are performed on networks separately trained with children speech, adult speech, and Spanish. Analyses on second mapping training are performed on a network trained with Spanish as a second language, and on another network trained with English as a second language. Cluster analyses of the hidden layer units of networks having different first and second language mappings reveal that the final mapping and the convergence process depend a lot on the training data. The results are shown to be highly dependent on statistical characteristics of the input. From mehra at aquinas.csl.uiuc.edu Thu Oct 5 00:24:11 1989 From: mehra at aquinas.csl.uiuc.edu (Pankaj Mehra) Date: Wed, 4 Oct 89 23:24:11 CDT Subject: help with addresses Message-ID: <8910050424.AA28363@elaine> I am looking for the address (e-mail or otherwise) of Chris Watkins, Ph.D. from Cambridge University, who has done some work on extensions of Sutton's temporal difference methods. I shall appreciate any pointers to his published papers as well. After his talk at IJCAI this year, Gerald Edelman mentioned that Fuster (sp?) has been doing some work on learning with delayed feedback. The literature in psychology on this subject is confusing (at least to an outsider). Whereas delays in feedback are bad for skill learning, their effect on "intelligent" tasks is just the contrary. I shall appreciate any references to the papers of Fuster and to other recent work in the area of reinforcement learning with delayed feedback. Pankaj Mehra e-mail: mehra at cs.uiuc.edu *** Please do not send replies to the entire mailing list. From srh at flash.bellcore.com Thu Oct 5 11:27:01 1989 From: srh at flash.bellcore.com (stevan r harnad) Date: Thu, 5 Oct 89 11:27:01 EDT Subject: Parallelism, Real vs. Simulated: A Query Message-ID: <8910051527.AA28565@flash.bellcore.com> I have a simple question: What capabilities of PDP systems do and do not depend on the net's actually being implemented in parallel, rather than just being serially simulated? Is it only speed and capacity parameters, or something more? Stevan Harnad Psychology Department Princeton University harnad at confidence.princeton.edu From skrzypek at CS.UCLA.EDU Thu Oct 5 15:59:44 1989 From: skrzypek at CS.UCLA.EDU (Dr. Josef Skrzypek) Date: Thu, 5 Oct 89 12:59:44 PDT Subject: Parallelism, Real vs. Simulated: A Query In-Reply-To: stevan r harnad's message of Thu, 5 Oct 89 11:27:01 EDT <8910051527.AA28565@flash.bellcore.com> Message-ID: <8910051959.AA20276@retina.cs.ucla.edu> I have a simple question: What capabilities of PDP systems do and do not depend on the net's actually being implemented in parallel, rather than just being serially simulated? Is it only speed and capacity parameters, or something more? >>>>>>>>>>>>>>>> Good (and dangerous) question. Applicable to Neural Nets in general and not only to PDP. It appears that you can simulate anything that you wish. In principle you trade computation in space for computation in time. If you can make your time-slices small enough and complete all of the necessary computation within each slice there seem to be no reason to have neural networks. In reality, simulation of synchronized, temporal events taking place in a 3D network that allows for feedback pathways is rather cumbersome. From mdtom at en.ecn.purdue.edu Thu Oct 5 17:09:49 1989 From: mdtom at en.ecn.purdue.edu (M Daniel Tom) Date: Thu, 05 Oct 89 16:09:49 EST Subject: TR-EE 89-54 replaced with uncompressed files Message-ID: <8910052109.AA10365@en.ecn.purdue.edu> I have replaced the compressed files with uncompressed ones: tenorio.speech_dev.ps tenorio.cluster.plain in cheops.cis.ohio-state.edu. Please try again. Thanks for your feedback about ftp-ing. I had suspected that the transmission was not perfect when I compared the file sizes with my own. This time the file sizes match. So good luck. Sincerely, M. Daniel Tom From oliver%FSU.BITNET at VMA.CC.CMU.EDU Thu Oct 5 15:44:15 1989 From: oliver%FSU.BITNET at VMA.CC.CMU.EDU (Bill Oliver, Psychology Dept., FSU, 32306) Date: Thu, 5 Oct 89 15:44:15 EDT Subject: mailing list Message-ID: <8910051543320E9.CWDV@RAI.CC.FSU.EDU> (UMass-Mailer 4.04) Please put me on your mailing list. Thanks, Bill Oliver From srh at flash.bellcore.com Thu Oct 5 11:27:01 1989 From: srh at flash.bellcore.com (stevan r harnad) Date: Thu, 5 Oct 89 11:27:01 EDT Subject: Parallelism, Real vs. Simulated: A Query Message-ID: <8910051527.AA28565@flash.bellcore.com> I have a simple question: What capabilities of PDP systems do and do not depend on the net's actually being implemented in parallel, rather than just being serially simulated? Is it only speed and capacity parameters, or something more? Stevan Harnad Psychology Department Princeton University harnad at confidence.princeton.edu From tenorio at ee.ecn.purdue.edu Fri Oct 6 13:48:03 1989 From: tenorio at ee.ecn.purdue.edu (Manoel Fernando Tenorio) Date: Fri, 06 Oct 89 12:48:03 EST Subject: NeuroGammon Message-ID: <8910061748.AA07003@ee.ecn.purdue.edu> Bcc: -------- We need more results like that! Shouted another NN colleague after Tesauro's message, to which I am quick to agree. Like probably most of you out there in email land, I got thrilled with the "new result" that NN claimed. Although my background includes AI, I am not aware of any other similar results in Machine Intelligence. But I have some doubts about what actually are the claims that we can intelligently make for NN with this program. Therefore I am opening it for discussion here. This is in no way to question or detract from the result, but rather to clarify and qualify future statements: NN has own a major victory, unparallel in Machine Intelligence, playing Backgammon. 1. Can we make the claim that we are doing better that AI (chess) efforts, mentioned as one of the AI conerstone results in the Oct88(?) AI magazine (AAAI), since it is a different game. I recall Tesauro mentioning in NIPS that backgammon was heavily pattern based, as opposed to chess. 2. Is anybody aware of results for NN in chess or AI in backgammon? 3. Could AI do better in a heavily pattern based game? 4. Does Tesauro plans some form of rule estimate to compare game complexity? 5. Should we say that NN are good for this game and not for others, but what matters is that what it does is better than the human counterparts (maybe that would mean a closer or better computational model to the human one?) 6. how can one better compare this apples and oranges results? 7. How about future results and past claims such as: NN are better than any other technique because it can solve the EX-OR problem and chaotic time series prediction (parafrased Neruocomputers Sept. 89) I would like to see a reasonable scientific discussion on the subject, because I would like to be prepared to answer to the question that will come after people read this on the papers (Can you imagine what some of the Media will do with this new result? "IBM unravels computer that mimics the brain, but beats any human. It will be call HAL..."). -Check'$', mate... (An aussie salute) --ft. From YEE at cs.umass.EDU Fri Oct 6 15:33:00 1989 From: YEE at cs.umass.EDU (Richard Yee) Date: Fri, 6 Oct 89 14:33 EST Subject: Searle, Harnad and understanding Chinese symbols Message-ID: <8910061834.AA04513@crash.cs.umass.edu> == More Chinese-Room Talk == I find myself in the apparently paradoxical position of agreeing with virtually all of Searle's assertions in the "Chinese Room" argument, and yet disagreeing with his conclusion. Likewise, I agree with Harnad that an intelligent (cognitive?) processing system's input symbols must be "grounded" in subsymbolic representations (in what I call internal semantic or interpretive representations), yet I disagree with his defense of Searle's counter to the "Systems-Reply". What follows is a rather long message which, I claim, demonstrates that the conclusion that Chinese is NOT being understood inside the Chinese Room has no basis. This rescues us from having to conclude that either understanding is uncomputable or the Church-Turing Thesis is wrong. The reason that the possibility remains open as to whether the inputs (Chinese characters) are being understood, is basically related to the Systems Reply with one important caveat: there is also no basis to the claim that the Chinese characters ARE being understood, and to the extent that the Systems Reply claims this I would not defend it. The question is whether the Chinese language performance (CLP) being observed externally arises from an understanding of Chinese (UC) or from some other process (not UC); the Chinese Room scenario does not present enough information to decide the question either way. Harnad describes the crux of Searle's argument against the Systems Reply as being that the person in the room is "in fact performing ALL the FUNCTIONS of the system" (possibly through having learned the rules), and yet clearly the person does not understand Chinese. Both of these statements are true, but this does not justify the conclusion that the process of understanding Chinese is not occurring. The determination of outputs is under the complete control of the rules, not the person. The person has no free will on this point (as he does in answering English inputs). All and only those transformations licensed by the rules will determine what the Chinese outputs will be. Thus, although it is clearly true that, e.g., the input symbols (Chinese characters), "HAN-BAO-BAO", have form but absolutely no content for the person, this in no way implies the the symbols' content will not be recognized and play a role in determining the Chinese output because this is in no way dependent upon the person's knowledge of the symbols. All that matters with regard to the person is his knowledge of how to correctly follow the rules. Whether or not the *content* of this symbols is recognized, is determined by the rules, and we simply have no basis for concluding either way. So, while the person hasn't the slightest idea whether it would be better to eat, run away from, or marry a "HAN-BAO-BAO", this knowledge may well be determined through the application of the rules, and, in such a case, they could dictate that an output be produced that takes account of this recognition. The output might well say, for example, that these things are found at McDonald's, but it would be surprising in the extreme to consider spending the rest of one's life with one. The person in the room is completely oblivious to this distinction, and yet the Chinese symbols were indeed correctly recognized for their CONTENT, and this happened WITHIN the room. On the other hand, it need not be the case that the meaning of the symbols is determined at any point. The same output could be produced by a very different process (it is hard to imagine, though, how the illusion could be maintained). Thus, I agree that looking at the I/O behavior outside of the room is not sufficient to determine if the input symbols are being understood (mapped to their meanings), or if, instead, they are treated as objects having form but no content (or treated some other way for that matter). The argument that there is no understanding of Chinese because the *person* never understands the input symbols appears to be based on a failure to distinguish between a generic Turing Machine (TM) and one that is programmable, a Universal Turing Machine (UTM). A UTM, U, is a TM that is given as input the program of another TM, T, and *its* input, x. The UTM computes a function which is itself the computation of a function of the input x. Thus, the UTM *does not* compute y = T(x); it computes y = U(T, x). If T, as a parameter of U, is held constant, then y = T(x) = U(x), but this still doesn't mean that U "experiences x" the same way T does. U merely acts as an intermediary that enables T to process x; when T is done, U returns T's result as if it were his own doing. The rules that the person is following are, in fact, a program for Chinese I/O (CLP). The person is acting as a UTM. He is following *his own set of rules* that tell him what to do with the rules and inputs that he receives. Thus, the person's program is to execute the rules' program on the inputs. It is little wonder that the person may not treat the input symbols in the same way that the rules treat them. The real question that should be asked is NOT whether the person, in following the rules, understands the Chinese characters, UC, (clearly he does not), but whether the person would understand the Chinese characters if HIS NEURONS were the ones implementing the rules and he were experiencing the results. In other words, the rules may or may not DESCRIBE A PROCESS sufficient for figuring out what the Chinese characters mean. ("UC, or not UC", do you see?) If Searle's and Harnad's arguments were correct, then one would be lead, as they seem to be, to the conclusion that a Turing Machine alone is not sufficient to produce understanding, in particular the understanding of Chinese. This would amount to the the claim that either, A. Understanding is not computable, i.e., it is not achievable through anything that could be considered an algorithm, a procedure, or any finitely describable method, =or= B. The Church-Turing Thesis is wrong. For what it is worth, I don't agree with either position. There is absolutely no reason to be persuaded that (B) is true, and I take my own understanding of English (and a little Chinese) as an existence proof that (A) is false. Richard Yee (yee at cs.umass.edu) From cole at cse.ogc.edu Fri Oct 6 15:41:40 1989 From: cole at cse.ogc.edu (Ron Cole) Date: Fri, 6 Oct 89 12:41:40 -0700 Subject: Parallelism, Real vs. Simulated: A Query Message-ID: <8910061941.AA07943@cse.ogc.edu> Massively parallel networks are likely to reveal emergent properties that cannot be predicted from serial simulations. Asking what properties these networks will have before they exist is like asking what we will see when we have more powerful telescopes. From osborn%rana.usc.edu at usc.edu Fri Oct 6 16:59:50 1989 From: osborn%rana.usc.edu at usc.edu (Tom Osborn) Date: Fri, 6 Oct 89 13:59:50 PDT Subject: No subject Message-ID: <8910062059.AA01270@rana.usc.edu> Steve Harnad asks: > I have a simple question: What capabilities of PDP systems do and > do not depend on the net's actually being implemented in parallel, > rather than just being serially simulated? Is it only speed and > capacity parameters, or something more? An alternative question to ask is: What differences does synchronous vs asynchronous processing make? Both may be _implemented_ in on serial or parallel machines - synch on serial by keeping old state vectors, synch on parallel by using some kind of lock-step control (with associated costs), asynch on serial by adding a stochastic model of unit/neuron processing, asynch on parallel - trivial. The _importance_ of of synch vs asynch is apparent for Hopfield/Little nets and Boltzmann machines: For Hopfield (utilising asynch processing, with random selection of one unit at a time and full connectivity), you get one Energy (Liapunov) function. BUT for Little nets (utilising synch processing - entire new state vector computed from the old one), you have a different but related Energy function. These two Energy function have the same stationary points, but the dynamics differ. [I can't comment on performance implications]. For Boltzmann machines, three different regimes may apply (if not all units are connected). The same two as above (with different dynamics) and I recall that there is no general convergence proof for the full synch case. Another parallel regime (ie, synch) updating where sets of neuron (no two directly connected) are processed together - dynamically, this corresponds exactly to asynch updating, but with linear performance scaling on parallel machines (assuming the partitioning problem was done ahead of time). Answers to the question for back-prop are more diverse: To maintain equivalence with asynch processing, Parallel implementations may synch process _layers_ at a time, or a pipeline effect may be set up, or the data may be managed to optimise some measure of performance (eg, for learning or info processing). HOWEVER, there _must_ be synchronisation between the computed and desired output values for back-prop learning to work (to compute the delta). Someone else should comment. Tom Osborn On Sabbatical Leave (till Jan '90) at: Center for Neural Engineering, University of Southern California Los Angeles CA 90089-0782 'Permanently', University of Technology, Sydney. From Scott.Fahlman at B.GP.CS.CMU.EDU Fri Oct 6 16:56:21 1989 From: Scott.Fahlman at B.GP.CS.CMU.EDU (Scott.Fahlman@B.GP.CS.CMU.EDU) Date: Fri, 06 Oct 89 16:56:21 EDT Subject: NeuroGammon In-Reply-To: Your message of Fri, 06 Oct 89 12:48:03 -0500. <8910061748.AA07003@ee.ecn.purdue.edu> Message-ID: 1. Can we make the claim that we are doing better that AI (chess) efforts, mentioned as one of the AI conerstone results in the Oct88(?) AI magazine (AAAI), since it is a different game. I recall Tesauro mentioning in NIPS that backgammon was heavily pattern based, as opposed to chess. 2. Is anybody aware of results for NN in chess or AI in backgammon? I agree that comparing neural-net backgammon to conventional AI chess programs would be pretty meaningless. But there are a number of conventional AI programs that play backgammon. The most famous of these, Hans Berliner's program, once beat the human world champion in backgammon. It had some lucky rolls, but then a good backgammon player willattempt to keep the board in a state where most rolls are "lucky" and where "unlucky" rolls can't do too much harm. Unfortunately, Berliner's program wasn't in the tournament that NeuroGammon won, but several other AI-type programs were there. Maybe Gerry can give us some estimate of whether these programs were in the same class as Berliner's program. 3. Could AI do better in a heavily pattern based game? Depends what you mean by "pattern-based". Backgammon is all patterns, but it also has the interesting feature of uncertainty; chess and Go are deterministic. Chess can be played using a lot of knowledge and a little search or vice-versa. People tend to use a lot of knowledge, but the current trend in computer chess is toward very search-intensive chess machines that have little in common with other areas of AI: no complex knowledge representations, no symbolic learning, etc. If this trend continues, it will mean that chess is no longer a good problem for driving progress in AI, though it will help to stimulate the development of parallel search engines. I think that Go is going to turn out to be the really interesting game for neural nets to tackle, since the search space is more intractable than the search space in chess, and since patterns of pieces influence the choice of move in subtle ways that master players cannot easily explain. There is still an important element of serial search, however -- I don't think even the masters claim to select every move by "feel" alone. 4. Does Tesauro plans some form of rule estimate to compare game complexity? The size of the rule set has very little to do with the strategic complexity of a game. Monopoly has a more complex rule set than Go, but is MUCH easier to play well. 6. how can one better compare this apples and oranges results? Try not to. -- Scott From Alex.Waibel at SPEECH2.CS.CMU.EDU Fri Oct 6 21:19:38 1989 From: Alex.Waibel at SPEECH2.CS.CMU.EDU (Alex.Waibel@SPEECH2.CS.CMU.EDU) Date: Fri, 6 Oct 89 21:19:38 EDT Subject: NIPS'89 Postconference Workshops Message-ID: Below are the preliminary program and brief descriptions of the workshop topics covered during this years NIPS-Postconference Workshops to be held in Keystone from November 30 through December 2 (right following the NIPS conference). Please register for both conference and Workshops using the general NIPS conference registration forms. With it, please indicate which workshop topic below you may be most interested in attending. Your preferences are in no way binding or limiting you to any particular workshop but will help us in allocating suitable meeting rooms and scheduling workshop sessions in an optimal way. For your convenience, you may simply include a copy of the form below with your registration material marking it for your three most prefered workshop choices in order of preference (1,2 and 3). For registration information (both NIPS conference as well as Postconference Workshops), please contact the Local Arrangements Chair, Kathie Hibbard, by sending email to hibbard at boulder.colorado.edu, or by writing to: Kathie Hibbard NIPS '89 University of Colorado Campus Box 425 Boulder, Colorado 80309-0425 For technical questions relating to individual conference workshops, please contact the individual workshop leaders listed below. Please feel free to contact me with any questions you may have about the workshops in general. See you in Denver/Keystone, Alex Waibel NIPS Workshop Program Chairman School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 412-268-7676, waibel at cs.cmu.edu ================================================================ ____________________________________________________________ ! POST CONFERENCE WORKSHOPS AT KEYSTONE ! ! THURSDAY, NOVEMBER 30 - SATURDAY, DECEMBER 2, 1989 ! !____________________________________________________________! Thursday, November 30, 1989 5:00 PM: Registration and Reception at Keystone Friday, December 1, 1989 7:30 - 9:30 AM: Small Group Workshops 4:30 - 6:30 PM: Small Group Workshops 7:30 - 10:30 PM: Banquet and Plenary Discussion Saturday, December 2, 1989 7:30 - 9:30 AM: Small Group Workshops 4:30 - 6:30 PM: Small Group Workshops 6:30 - 7:15 PM: Plenary Discussion, Summaries 7:30 - 11:00 PM: Fondue Dinner, MountainTop Restaurant ================================================================ PLEASE MARK YOUR PREFERENCES (1,2,3) AND ENCLOSE WITH REGISTRATION MATERIAL: ----------------------------------------------------------------------------- ______1. LEARNING THEORY: STATISTICAL ANALYSIS OR VC DIMENSION? ______2. STATISTICAL INFERENCE IN NEURAL NETWORK MODELLING ______3. NEURAL NETWORKS AND GENETIC ALGORITHMS ______4. VLSI NEURAL NETWORKS: CURRENT MILESTONES AND FUTURE HORIZONS ______5. APPLICATION OF NEURAL NETWORK PROCESSING TECHNIQUES TO REAL WORLD MACHINE VISION PROBLEMS ______6. IMPLEMENTATIONS OF NEURAL NETWORKS ON DIGITAL, MASSIVELY PARALLEL COMPUTERS ______7. LARGE, FAST, INTEGRATED SYSTEMS BASED ON ASSOCIATIVE MEMORIES ______8. NEURAL NETWORKS FOR SEQUENTIAL PROCESSING WITH APPLICATIONS IN SPEECH RECOGNITION ______9. LEARNING FROM NEURONS THAT LEARN ______10. NEURAL NETWORKS AND OPTIMIZATION PROBLEMS 11. (withdrawn) ______12. NETWORK DYNAMICS ______13. ARE REAL NEURONS HIGHER ORDER NETS? ______14. NEURAL NETWORK LEARNING: MOVING FROM BLACK ART TO A PRACTICAL TECHNOLOGY ______15. OTHERS ?? __________________________________________________ 1. LEARNING THEORY: STATISTICAL ANALYSIS OR VC DIMENSION? Sara A. Solla AT&T Bell Laboratories Crawford Corner Road Holmdel, NJ 07733-1988 Phone: (201) 949-6057 E-mail: solla at homxb.att.com Recent success at describing the process of learning in layered neural networks and the resulting generalization ability has emerged from two different approaches. Work based on the concept of VC dimension emphasizes the connection between learning and statistical inference in order to analyze questions of bias and variance. The statistical approach uses an ensemble description to focus on the prediction of network performance for a specific task. Participants interested in learning theory are invited to discuss the differences and similarities between the two approaches, the mathematical relation between them, and their respective range of applicability. Specific questions to be discussed include comparison of predictions for required training set sizes, for the distribution of generalization abilities, for the probability of obtaining good performance with a training set of fixed size, and for estimates of problem complexity applicable to the determination of learning times. 2. STATISTICAL INFERENCE IN NEURAL NETWORK MODELLING Workshop Chair: Richard Golden Stanford University Psychology Department Stanford, CA 94305 (415) 725-2456 E-mail: golden at psych.stanford.edu This workshop is designed to show how the theory of statistical inference is directly applicable to some difficult neural network modelling problems. The format will be tutorial in nature (85% informal lecture, 15% discussion). Topics to be discussed include: obtaining probability distributions for neural networks, interpretation and derivation of optimal learning cost functions, evaluating the generalization performance of networks, asymptotic sampling distributions of network weights, statistical mechanics calculation of learning curves in some simple examples, statistical tests for comparing internal representations and deciding which input units are relevant to the prediction task. Dr. Naftali Tishby (AT&T Bell Labs) and Professor Halbert White (UCSD Economics Department) are the invited experts. 3. Title: NEURAL NETWORKS AND GENETIC ALGORITHMS Organizers: Lawrence Davis (Bolt Beranek and Newman, Inc.) Michael Rudnick (Oregon Graduate Center) Description: Genetic algorithms have many interesting relationships with neural networks. Recently, a number of researchers have investigated some of these relationships. This workshop will be the first forum bringing those researchers together to discuss the current and future directions of their work. The workshop will last one day and will have three parts. First, a tutorial on genetic algorithms will be given, to ground those unfamiliar with the technology. Second, seven researchers will summarize their results. Finally there will be an open discussion on the topics raised in the workshop. We expect that anyone familiar with neural network technology will be comfortable with the content and level of discussion in this workshop. 4. VLSI NEURAL NETWORKS: CURRENT MILESTONES AND FUTURE HORIZONS Moderators: Joshua Alspector and Daniel B. Schwartz Bell Communications Research GTE Laboratories, Inc. 445 South Street 40 Sylvan Road Morristown, NJ 07960-19910 Waltham, MA 02254 (201) 829-4342 (617) 466-2414 e-mail: josh at bellcore.com e-mail: dbs%gte.com at relay.cs.net This workshop will explore the areas of applicability of neural network implementations in VLSI. Several speakers will discuss their present implementations and speculate about where their work may lead. Workshop attendees will then be encouraged to organize working groups to address several issues which will be raised in connection with the presentations. Although it is difficult to predict which issues will be selected, some examples might be: 1) Analog vs. digital implementations. 2) Limits to VLSI complexity for neural networks. 3) Algorithms suitable for VLSI architectures. The working groups will then report results which will be included in the workshop summary. 5. APPLICATION OF NEURAL NETWORK PROCESSING TECHNIQUES TO REAL WORLD MACHINE VISION PROBLEMS Paul J. Kolodzy (617) 981-3822 kolodzy at ll.ll.mit.edu Murali M. Menon (617) 981-5374 This workshop will discuss the application of neural networks to vision applications, including image restoration and pattern recognition. Participants will be asked to present their specific application for discussion to highlight the relevant issues. Examples of such issues include, but are not limited to, the use of deterministic versus stochastic search procedures for neural network processing, using networks to extract shape, scale and texture information for recognition and using network mapping techniques to increase data separability. The discussions will be driven by actual applications with an emphasis on the advantages of using neural networks at the system level in addition to the individual processing steps. The workshop will attempt to cover a wide breadth of network architectures and invites participation from researchers in machine vision, neural network modeling, pattern recognition and biological vision. 6. IMPLEMENTATIONS OF NEURAL NETWORKS ON DIGITAL, MASSIVELY PARALLEL COMPUTERS Dr. K. Wojtek Przytula and Prof. S.Y. Kung Hughes Research Laboratories, RL 69 3011 Malibu Cyn. Road Malibu, CA 90265 Phone: (213) 317-5892 E-mail: wojtek%csfvax at hac2arpa.hac.com Implementations of neural networks span a full spectrum from software realizations on general-purpose computers to strictly special-purpose hardware realizations. Implementations on programmable, parallel machines, which are to be discussed during the workshop, constitute a compromise between the two extremes. The architectures of programmable parallel machines reflect the structure of neural network models better than those of sequential machines, thus resulting in higher processing speed. The programmability provides more flexibility than is available in specialized hardware implementations and opens a way for realization of various models on a single machine. The issues to be discussed include: mapping neural network models onto existing parallel machines, design of specialized programmable parallel machines for neural networks, evaluation of performance of parallel machines for neural networks, uniform characterization of the computational requirements of various neural network models from the point of view of parallel implementations. 7. LARGE, FAST, INTEGRATED SYSTEMS BASED ON ASSOCIATIVE MEMORIES Michael R. Raugh Director of Learning Systems Division Research Institute for Advanced Computer Science (RIACS) NASA Ames Research Center, MS 230-5 Moffett Field, CA 94035 e-mail: raugh at riacs.edu Phone: (415) 694-4998 This workshop will address issues in the construction of large systems that have thousands or even millions of hidden units. It will present and discuss alternatives to backpropagation that allow large systems to learn rapidly. Examples from image analysis, weather prediction, and speech transcription will be discussed. The focus on backpropagation with its slow learning has kept researchers from considering such large systems. Sparse distributed memory and related associative-memory structures provide an alternative that can learn, interpolate, and abstract, and can do so rapidly. The workshop is open to everyone, with special encouragement to those working in learning, time-dependent networks, and generalization. 8. NEURAL NETWORKS FOR SEQUENTIAL PROCESSING WITH APPLICATIONS IN SPEECH RECOGNITION Herve Bourlard Philips Research Laboratory Brussels Av. Van Becelaere 2, Box 8 B-1170 Brussels, Belgium Phone: 011-32-2-674-22-74 e-mail address: bourlard at prlb.philips.be or: prlb2!bourlard at uunet.uu.net Speech recognition must contend with the statistical and sequential nature of the human speech production system. Hidden Markov Models (HMM) provide a powerful method to cope with both of these, and their use made a breakthrough in speech recognition. On the other hand, neural networks have recently been recognized as an alternative tool for pattern recognition problems such as speech recognition. Their main useful properties are their discriminative power and their capability to deal with non-explicit knowledge. However, the sequential aspect remains difficult to handle in connectionist models. If connections are supplied with delays, feedback loops can be added providing dynamic and implicit memory. However, in the framework of continuous speech recognition, it is still difficult to use only neural networks for the segmentation and recognition of a sentence into a sequence of speech units, which is efficiently solved in the HMM approach by the well known ``Dynamic Time Warping'' algorithm. This workshop should be the opportunity for reviewing neural network architectures which are potentially able to deal with sequential and stochastic inputs. It should also be discussed to which extent the different architectures can be useful in recognizing isolated units (phonemes, words, ...) or continuous speech. Amongst others, we should consider spatiotemporal models, time-delayed neural networks (Waibel, Sejnowsky), temporal flow models (Watrous), hidden-to-input (Elman) or output-to-input (Jordan) recurrent models, focused back-propagation networks (Mozer) or hybrid approaches mixing neural networks and standard sequence matching techniques (Sakoe, Bourlard). 9. LEARNING FROM NEURONS THAT LEARN Moderated by Thomas P. Vogl Environmental Research Institute of Michigan 1501 Wilson Blvd. Arlington, VA 22209 Phone: (703) 528-5250 E-mail: TVP%nihcu.bitnet at cunyvm.cuny.edu FAX: (703) 524-3527 In furthering our understanding of artificial and biological neural systems, the insights that can be gained from the perceptions of those trained in other disciplines can be particularly fruitful. Computer scientists, biophysicists, engineers, psychologists, physicists, and neurobiologists tend to have different perspectives and conceptions of the mechanisms and components of "neural networks" and to weigh differently their relative importance. The insights obvious to practitioners of one of these disciplines are often far from obvious to those trained in another, and therefore may be especially relevant to the solutions of ornery problems. The workshop provides a forum for the interdisciplinary discussion of biological and artificial networks and neurons and their behavior. Informal group discussion of ongoing research, novel ideas, approaches, comparisons, and the sharing of insights will be emphasized. The specific topics to be considered and the depth of the analysis/discussion devoted to any topic will be determined by the interest and enthusiasm of the participants as the discussion develops. Participants are encouraged to consider potential topics in advance, and to present them informally but succinctly (under five minutes) at the beginning of the workshop. 10. NEURAL NETWORKS AND OPTIMIZATION PROBLEMS ---------------------------------------- Prof. Carsten Peterson University of Lund Dept. of Theoretical Physics Solvegatan 14A S-223 62 Lund Sweden phone: 011-46-46-109002 bitnet: THEPCAP%SELDC52 Workshop description: The purpose of the workshop is twofold; to establish the present state of the art and to generate novel ideas. With respect to the former, firm answers to the following questions should emerge: (1). Does the Hopfield- Tank approach or variants thereof really work with respect to quality, reliability, parameter insensitivity and scalability? (2). If this is the case, how does it compare with other cellular approaches like "elastic snake" and genetic algorithms? Novel ideas should focus on new encoding schemes and new application areas (in particular, scheduling problems). Also, if time allows, optimization of neural network learning architectures will be covered. People interested in participating are encouraged to communicate their interests and expertise to the chairman via e-mail. This would facilitate the planning. 12. Title: NETWORK DYNAMICS Chair: Richard Rohwer Centre for Speech Technology Research Edinburgh University 80, South Bridge Edinburgh EH1 1HN, Scotland Phone: (44 or 0) (31) 225-8883 x280 e-mail: rr%uk.ac.ed.eusip at nsfnet-relay.ac.uk Summary: This workshop will be an attempt to gather and improve our knowledge about the time dimension of the activation patterns produced by real and model neural networks. This broad subject includes the description, interpretation and design of these temporal patterns. For example, methods from dynamical systems theory have been used to describe the dynamics of network models and real brains. The design problem is being approached using dynamical training algorithms. Perhaps the most important but least understood problems concern the cognitive and computational significance of these patterns. The workshop aims to summarize the methods and results of researchers from all relevant disciplines, and to draw on their diverse insights in order to frame incisive, approachable questions for future research into network dynamics. Richard Rohwer JANET: rr at uk.ac.ed.eusip Centre for Speech Technology Research ARPA: rr%uk.ac.ed.eusip at nsfnet-relay.ac.uk Edinburgh University BITNET: rr at eusip.ed.ac.uk, 80, South Bridge rr%eusip.ed.UKACRL Edinburgh EH1 1HN, Scotland UUCP: ...!{seismo,decvax,ihnp4} !mcvax!ukc!eusip!rr PHONE: (44 or 0) (31) 225-8883 x280 FAX: (44 or 0) (31) 226-2730 13. ARE REAL NEURONS HIGHER ORDER NETS? Most existing artificial neural networks have processing elements which are computationally much simpler than real neurons. One approach to enhancing the computational capacity of artificial neural networks is to simply scale up the number of processing elements, but there are limits to this. An alternative is to build modules or subnets and link these modules in a larger net. Several groups of investigators have begun to analyze the computational abilities of real single neurons in terms of equivalent neural nets, in particular higher order nets, in which the inputs explicitly interact (eg. sigma-pi units). This workshop would introduce participants to the results of these efforts, and examine the advantages and problems of applying these complex processors in larger networks. Dr. Thomas McKenna Office of Naval Research Div. Cognitive and Neural Sciences Code 1142 Biological Intelligence 800 N. Quincy St. Arlington, VA 22217-5000 phone:202-696-4503 email: mckenna at nprdc.arpa mckenna at nprdc.navy.mil 14. NEURAL NETWORK LEARNING: MOVING FROM BLACK ART TO A PRACTICAL TECHNOLOGY Scott E. Fahlman School of Computer Science Carnegie-Mellon University Pittsburgh, PA 15213 Internet: fahlman at cs.cmu.edu Phone: (412) 268-2575 There are a number of competing algorithms for neural network learning, all rather new and poorly understood. Where theory is lacking, a reliable technology can be built on shared experience, but it usually takes a long time for this experience to accumulate and propagate through the community. Currently, each research group has its own bag of tricks and its own body of folklore about how to attack certain kinds of learning tasks and how to diagnose the problem when things go wrong. Even when groups are willing to share their hard-won experience with others, this can be hard to accomplish. This workshop will bring together experienced users of back-propagation and other neural net learning algorithms, along with some interested novices, to compare views on questions like the following: I. Which algorithms and variations work best for various classes of problems? Can we come up with some diagnostic features that tell us what techniques to try? Can we predict how hard a given problem will be? II. Given a problem, how do we go about choosing the parameters for various algorithms? How do we choose what size and shape of network to try? If our first attempt fails, are there symptoms that can tell us what to try next? III. What can we do to bring more coherence into this body of folklore, and facilitate communication of this informal kind of knowledge? An online collection of standard benchmarks and public-domain programs is one idea, already implemented at CMU. How can we improve this, and what other ideas do we have? From rik%cs at ucsd.edu Fri Oct 6 17:18:42 1989 From: rik%cs at ucsd.edu (Rik Belew) Date: Fri, 6 Oct 89 14:18:42 PDT Subject: Neurogammon wins Computer Olympiad Message-ID: <8910062118.AA16164@roland.UCSD.EDU> From TESAURO at ibm.com Wed Oct 4 00:35:31 1989 To: connectionists at CS.CMU.EDU Subject: Neurogammon wins Computer Olympiad ... This is a victory not only for neural networks, but for the entire machine learning community, as it is apparently the first time in the history of computer games that a learning program has ever won a tournament. Congratulations Gerry! And I think your highlighting the win by a *learning* (v. special-purpose programmed) solution is appropriate. But as to being first, don't you think Arthur Samuel's checker player gets the distinction? At least when you think about the antique hardware/software 'environment' he had to use? Rik Belew From harnad at clarity.Princeton.EDU Sat Oct 7 13:07:51 1989 From: harnad at clarity.Princeton.EDU (Stevan Harnad) Date: Sat, 7 Oct 89 13:07:51 EDT Subject: Symbol Manipulation: Scope and Limits Message-ID: <8910071707.AA15771@psycho.Princeton.EDU> Since Richard Yee has let the cat out of the bag with his posting (I was hoping for more replies about whether the community considered there to be any essential difference between parallel implementations and serial simulations of neural nets before I revealed why I had posted my query): I've proposed a variant of Searle's Chinese Room Argument (which in its original form I take to be decisive in showing that you can't implement a mind with a just a pure symbol manipulating system) to which nets are vulnerable only if there is no essential difference between a serial simulation and a parallel implementation. That having been said, the variant is obvious, and I leave it to you as an exercise. Here's my reply to Yee, who wrote: > The real question that should be asked is NOT whether [Searle], in > following the rules, understands the Chinese characters, (clearly > he does not), but whether [Searle] would understand the Chinese > characters if HIS NEURONS were the ones implementing the rules and he > were experiencing the results. In other words, the rules may or may not > DESCRIBE A PROCESS sufficient for figuring out what the Chinese > characters mean. This may be the real question, but it's not the one Searle's answering in the negative. In the Chinese room there's only symbol manipulation going on. No person or "system" -- no "subject" -- is understanding. This means symbol manipulation alone is not sufficient to IMPLEMENT the process of understanding, any more than it can implement the process of flying. Now whether it can DESCRIBE rather than implement it is an entirely different question. I happen to see no reason why all features of a process that was sufficient to implement understanding (in neurons, say) or flying (in real airplane parts) couldn't be successfully described and pretested through symbolic simulation. But Searle has simply shown that pure symbol manipulation ITSELF cannot be the process that will successfully implement understanding (or flying). (Ditto now for PDP systems, if parallel implementations and serial simulations are equivalent or equipotent.) > I agree that looking at the I/O behavior outside of the room is not > sufficient... This seems to give up the Turing Test (Searle would shout "Bravo!"). But now Yee seems to do an about-face in the direction of resurrecting the strained efforts of the AI community to show that formal symbol manipulating rule systems have not just form but content after all, and CAN understand: > The determination of outputs is under the complete control of the > rules, not [Searle]. [Searle] has no free will on this point (as he > does in answering English inputs)... although it is clearly true that > (Chinese characters) have form but absolutely no content for the > person... [w]hether or not the *content* of this symbol is recognized, > is determined by the rules... the Chinese symbols were indeed correctly > recognized for their CONTENT, and this happened WITHIN the room... > the process of understanding Chinese is [indeed] occurring. NB: No longer described or simulated, as above, but actually OCCURRING. I ask only: Where/what are these putative contents (I see only formal symbols); and who/what is the subject of this putative understanding (I see only Searle), and would he/she/it care to join in this discussion? Now in my case this glibness is really a reflection of my belief that the Turing Test couldn't be successfully passed by a pure symbol manipulator in the first place (and hence that this whole sci-fi scenario is just a counterfactual fantasy) because of the symbol grounding problem. But Yee -- though skeptical about the Turing Test and seemingly acknowledging the simulation/implemetation distinction -- does not seem to be entirely of one mind on this matter... > [The problem is] a failure to distinguish between a generic Turing > Machine (TM) and one that is programmable, a Universal Turing Machine > (UTM)... If T, as a parameter of U, is held constant, then y = T(x) = > U(x), but this still doesn't mean that U "experiences x" the same way T > does. The rules that the person is following are, in fact, a program > for Chinese I/O... I take my own understanding of English (and a little > Chinese) as an existence proof that [Understanding is Computable] "Cogito Ergo Sum T"? -- Descartes would doubt it... I don't know what Yee means by a "T," but if it's just a pure symbol-cruncher, Searle has shown that it does not cogitate (or "experience"). If T's something more than a pure symbol-cruncher, all bets are off, and you've changed the subject. Stevan Harnad References: Searle, J. (1980) Minds, Brains and Programs. Behavioral and Brain Sciences 3: 417 - 457. Harnad, S. (1989) Minds, Machines and Searle. Journal of Experimental and Theoretical Artificial Intelligence 1: 5 - 25. Harnad, S. (1990) The Symbol Grounding Problem. Physica D (in press) From french at cogsci.indiana.edu Fri Oct 6 14:35:19 1989 From: french at cogsci.indiana.edu (Bob French) Date: Fri, 6 Oct 89 13:35:19 EST Subject: Tech Report "Towards a Cognitive Connectionism" available Message-ID: The following Tech Report is now available. It is scheduled to appear in the January 1990 issue of AI and Society: ACTIVE SYMBOLS AND INTERNAL MODELS: TOWARDS A COGNITIVE CONNECTIONISM by Stephen Kaplan, Mark Weaver and Robert M. French In this paper, we examine some recent criticisms of connectionist models. In the first section, we address the argument that connectionist models are fundamentally behaviorist in nature and, therefore, incapable of genuine cognition. We conclude that these criticisms are indeed valid, but apply only to the currently popular class of feed-forward connectionist networks. To have any hope of avoiding the charge of behaviorism, and ultimately to support full cognition, connectionist architectures must be capable of producing persistent internal states. We discuss the crucial notion of "active symbols" -- semi-autonomous representations -- as a basis for such a cognitive connectionist architecture. Active symbols arise, we argue, out of recurrent circuits, the connectionist implementation of Hebbian cell assemblies. Recurrent architectures have become more prominent in the past year. However, most researchers investigating recurrent architectures seem to view recurrent circuitry merely as an "improved back-prop" for handling time-sequencing. Our view, that the recurrent circuit must be the fundamental building block of any cognitive connectionist architecture, represents a philosophical departure from current thought. In the final section we speculate on the potentials and limits of an associationist architecture. In particular, we examine how the this type of architecture might be able to produce the structure that is evident in human cognitive capacities, and thus answer the criticisms of Fodor and Pylyshyn. This paper is scheduled to appear in AI and Society in January 1990. To obtain a copy of this paper, send e-mail to french at cogsci.indiana.edu or write to: Bob French Center for Research on Concepts and Cognition Indiana University 510 North Fess Bloomington, Indiana 47401 From derek at prodigal.psych.rochester.edu Sat Oct 7 18:32:15 1989 From: derek at prodigal.psych.rochester.edu (Derek Gross) Date: Sat, 7 Oct 89 18:32:15 EDT Subject: connectionist models of music? Message-ID: <8910072232.AA28088@prodigal.psych.rochester.edu> Does anyone know of any connectionist models of musical structures, perception or composition? If so, please send me e-mail. Thanks, Derek Gross University of Rochester Cognitive Science From todd at galadriel.Stanford.EDU Sun Oct 8 00:02:09 1989 From: todd at galadriel.Stanford.EDU (Peter Todd) Date: Sat, 07 Oct 89 21:02:09 PDT Subject: connectionist models of music Message-ID: In answer to Derek Gross's question about connectionist models of music, I wanted to point out that the current and next issues of the Computer Music Journal (MIT Press) are specifically devoted to this topic. In the current issue, 13(3), out now, there is a general tutorial on musical applications of networks, plus articles on network models of pitch perception, tonal analysis, quantization of time, complex musical patterns, and instrument fingering. In the next issue, 13(4), due out at the end of the year, there will be articles on my work using sequential networks for composition, modelling tonal expectancy (with Jamshed Bharucha, who has also published much work in the area of network modelling of human musical behavior), and another article on representations for pitch perception. Both issues were edited by D. Gareth Loy, of UC San Diego, and myself; the journal is available in some bookstores. Hope this helps-- peter todd stanford university psychology department From khaines at GALILEO.ECE.CMU.EDU Mon Oct 9 14:33:05 1989 From: khaines at GALILEO.ECE.CMU.EDU (Karen Haines) Date: Mon, 9 Oct 89 14:33:05 EDT Subject: IJCNN 1990 - Request for Volunteers Message-ID: <8910091833.AA05968@galileo.ece.cmu.edu> This is a first call for volunteers to help at the IJCNN conference, to be held at the Omni Shorham Hotel in Washington D.C., on January 15-19, 1990. Full admittance to the conference and a copy of the proceedings is offered in exchange for your assistance throughout the conference. In general, each volunteer is expected to work one shift, either in the morning or the afternnon, each day of the conference. Hours for morning shift are, approximately, 7:00 am until 12:00 noon, and for the afternoon, 12:00 noon to 5:00 pm. In addition, assistance will be required for the social events. If you can`t work all week long please contact Karen Haines to see what can be worked out. There will be a mandatory meeting for all volunteers on January 14. To sign up please contact: Karen Haines - Volunteer Coordinator 3138 Beechwood Blvd. Pittsburgh, PA 15217 office: (412) 268-3304 message: (412) 422-6026 email: khaines at galileo.ece.cmu.edu or, Nina Kowalski - Assistant Volunteer Coordinator 209 W. 29th St. FLR 2 Baltimore, MD 21211 message: (301) 889-0587 email: nina at alpha.ece.jhu.edu If you have further questions, please feel free to contact me. Thank you, Karen Haines From brittner at spot.Colorado.EDU Mon Oct 9 17:41:18 1989 From: brittner at spot.Colorado.EDU (BRITTNER RODNEY) Date: Mon, 9 Oct 89 15:41:18 MDT Subject: Tech Report "Towards a Cognitive Connectionism" available Message-ID: <8910092141.AA17506@spot.Colorado.EDU> Hello, Pleas send me a copy of the Tech report "towards a cog. connectionism"!! Thanks!! R. Brittner University of Colorado Department of Aerospace Engineering Bioserve Offices Boulder, CO 80309 From aboulang at WILMA.BBN.COM Mon Oct 9 19:16:28 1989 From: aboulang at WILMA.BBN.COM (aboulang@WILMA.BBN.COM) Date: Mon, 9 Oct 89 19:16:28 EDT Subject: Parallelism, Real vs. Simulated: A Query In-Reply-To: stevan r harnad's message of Thu, 5 Oct 89 11:27:01 EDT <8910051527.AA28565@flash.bellcore.com> Message-ID: Stevan Harnad (harnad at confidence.princeton.edu) writes: I have a simple question: What capabilities of PDP systems do and do not depend on the net's actually being implemented in parallel, rather than just being serially simulated? Is it only speed and capacity parameters, or something more? The more general question of the difference between parallel computation (under any means) and serial computation has interested me for a number of years. It is clear that synchronous parallelism and serial computation are the same modulo the speedup. The harder question is whether asynchronous parallelism computes the same way modulo the speedup. The computability results for asynchronous computation had their start with David Muller ("Infinite Sequences and Finite Machines", Switching Circuit Theory and Logical Design: Proc. 4th Ann. Symp. IEEE pp 3-16, 1963) who used as an example 2 asynchronous fed-back NOT circuits. The analysis led to the work in omega-languages which are languages over the power sets of all the possible states for such a combined circuit would produce. The computability results for omega-languages are that they are no more powerful than sequential Turing machines. Another line of reasoning is to ask whether parallel-asynchronous dynamical computational systems have a different kind of dynamics than sequential dynamical systems and whether the computational abilities of the system can make use of this difference in dynamics. Asynchronization or time-delays can act as a source of noise to help settle stochastic networks. See for example the discussion of time delays in "Separating Figure from Ground with a Boltzmann Machine" Terrence Sejnowski and Geoffrey Hinton, In Vison Bra in and Cooperative Computation, M.A. Arbib & A.R. Hanson eds., MIT Press 1985, and "Effects of Connection Delays in Two State Model Neural Circuits", Peter Conwell, IEEE First Conference on Neural Networks, III-95 - III-104. Note that the term "asynchronous updates" in the neural network research that comes from the spin-glass people normally means sequential. There has been some work on the convergence of neural nets with asynchronous-parallel updates. See for example '"Chaotic Relaxation" in Concurrently Asynchronous Neurodynamics', Jacob Barhen and Sandeep Gulati, IJCNN 1989, pp I-619 - I-626, and "Partially Asynchronous, Parallel Algorithms for Network Flow and Other Problems", P. Tseng, D.P. Bertsekas, and J.N. Tsitsiklis, Center for Intelligent Control Systems Report CICS-P-91, November 1988. The dynamics of such systems is related to the study of time-delay equations in the dynamical system literature. Work has been done on the routes to chaos, etc. in such equations with constant delay, but little or no work has been done with variable delay; which is the case with asynchronous parallelism. This is an area that I am actively studying. Finally, the computational abilities of general analog dynamical systems have been studied by several people including Bradley Dickenson, EE Dept., at Princton, so you may want to talk to him. His results indicate that the hardness of NP-complete problems translate into exponential scaling of analog resources. I believe that some further insight to your question can be had by using Kolmogorov-Chaitin algorithmic complexity applied to dynamical systems that compute. Algorithmic complexity has been applied to dynamical systems to help distinguish and separate the notions of determinism, and randomness is such systems. See the excellent paper, "How Random is a Coin Toss", Joseph Ford, April 1983 40-47. One way to summarize this work is to say that a pseudo-random number generator (which are only iterated nonlinear maps) could in principle generate truly random numbers if seeded with an infinite-algorithmic-complexity number. Another fact to appreciate is that most of the real line is made up of numbers that are algorithmically-incompressible - that is they have a maximal algorithmic complexity for their length as decimal or binary expansions. Irrational numbers would take an infinite amount of time and space resources to compute their expansions - this space-time notion is a variant of the algorithmic complexity measure sometimes called Q-logical depth. One would think that a computational system that could use such a powerful number could be used to compute beyond the capabilities (either in speed or in computability) than computational systems that don't have access to such numbers. (For example, the digits of Chaitin's OMEGA number solves the halting problem by its very definition. See for example "Randomness in Arithmetic", Gregory Chaitin, Scientific American, July 1988, 80-85.) In my mind the question of computability for analog and parallel-asynchronous architectures is precisely whether these constructs can implicitly or explicitly use numbers like Chaitin's OMEGA, to do computation faster, to scale better, or to compute normally incomputable questions than standard computer models. An example of explicit representation is representing a number like OMEGA as an analog value. Due to the fact that values at some level become quantal makes this unlikely. An example of an implicit representation is the possible irrational timing relationships between the self-timings of the elements of an asynchronous computing device. I have been thinking about this implicit possibility for a while, but the quantization of space-time at Plank lengths would eliminate this too. I have not turned my back on this since this just prevents us from using infinite-algorithmic complexity numbers. I still think we can do something with large algorithmic-complexity numbers - in either speed or scaling. Some indication of how hard it would be to go beyond normal "digital" computational devices in their abilities comes from a line of inquiry in dynamical systems. This started when people asked a natural question, "Is it reasonable to simulate chaos on the computer?." Remarkably, the trajectories of a dynamical system simulation on a computer can "shadow" true orbits for very long lengths of time. See for example "In the Shadows of Chaos", Ivar Peterson, Science News, Vol 134, December 3 1988, 360-361, and "Numerical Orbits of Chaotic Processes Represent True Orbits", Stephen Hammel, James Yorke, and Celso Grebogi, Bulletin of the AMS, Vol 19, No 2, October 1988, 465-469. A very similar question is being explored by researchers in quantum chaos. Here the phenomena is called "phase mixing" and quantum system (which cannot have chaos given the its mathematical form) will mix in a very similar way its analog classical system will, but is reversible - unlike its ergodic counterpart. For very long periods of time quantum systems can emulate chaotic classical systems. See for example "Classical Mechanics, Quantum Mechanics, and the Arrow of Time", T.A. Heppenheimer, Mosiac, Vol 20. No. 2, Summer 1989, 2-11. In closing, questions similar to yours have been asked about computers based on the rules of Quantum Mechanics. (Quantum Computers and computing-in-the-small was an area of investigation of Richard Feynman during his last years. Richard Feynman, John Hopfield, and Carver Mead did a course in the physics of computation which was a seed to Carver Mead's new book, "Analog VLSI and Neural Systems".) David Deutsch has claimed in his paper "Quantum Theory, the Church-Turing Principle and the Universal Quantum Computer", Proc. R. Soc. Lond., A400, 1985, 97-117 that quantum computers would be more powerful (in the computability sense) than the classical Turing machine. An example of this power is using a quantum computer to compute truly random numbers, or to prove that the Many-Worlds Interpretation of QM is the true one. Deutsch proposes a modified version of Church-Turing hypothesis to cover the case of quantum computers or what he calls "quantum parallelism". Contrasting to this is the work of T. Erber, who has been looking evidence for pseudo-random, as opposed to random, signatures in single-atom resonance experiments. See for example "Randomness in Quantum Mechanics - Nature's Ultimate Crytogram?", T. Erber & S. Putterman, Nature, Vol 318, No 7, November 1985, 41-43. From galem at mcc.com Tue Oct 10 10:37:18 1989 From: galem at mcc.com (Gale Martin) Date: Tue, 10 Oct 89 09:37:18 CDT Subject: harnad's question Message-ID: <8910101437.AA02886@sunkist.aca.mcc.com> From aboulang at WILMA.BBN.COM Thu Oct 12 11:13:33 1989 From: aboulang at WILMA.BBN.COM (aboulang@WILMA.BBN.COM) Date: Thu, 12 Oct 89 11:13:33 EDT Subject: Some unresolved pointers resolved! In-Reply-To: Paul Kube's message of Tue, 10 Oct 89 13:48:34 PDT <8910102048.AA29395@kokoro.UCSD.EDU> Message-ID: I am sorry for not giving complete references. There were two that people asked about: "How Random is a Coin Toss" Joseph Ford Physics Today, April 1983, 40-47 & A. Vergis, K. Steiglitz, and B. Dickinson, ``The complexity of analog computation,'' Mathematics and Computers in Simulation, vol. 28, 1986, pp. 91-113. In the shadow of your smiles :-)I, Albert Boulanger BBN Systems & Technologies Corporation aboulanger at bbn.com From sankar at caip.rutgers.edu Thu Oct 12 13:28:24 1989 From: sankar at caip.rutgers.edu (ananth sankar) Date: Thu, 12 Oct 89 13:28:24 EDT Subject: No subject Message-ID: <8910121728.AA05622@caip.rutgers.edu> A neural network basically classifies training and testing samples into different regions in an n dimensional space. By generating the output of the network for all possible points in the space one constructs a n + 1 dimensional surface with n independent variables. A polynomial can be generated to approximate this surface. It should be possible to construct a "polynomial neural network" that can do this job. The neurons individually may implement simple polynomials (using sigma-pi units maybe). I would really appreciate any pointers to any research on this (published or unpublished). The work that I am aware of dates to the late 60's--A.G. Ivakhnenko and Donald Specht--though they did not model their systems as nn's. I would also like feedback on what the potential use of such nets may be over typical work like back prop. Thanks in anticipation Ananth Sankar Dept. of Electrical Engg. Rutgers University NJ From sankar at caip.rutgers.edu Thu Oct 12 13:30:03 1989 From: sankar at caip.rutgers.edu (ananth sankar) Date: Thu, 12 Oct 89 13:30:03 EDT Subject: Polynomial Nets Message-ID: <8910121730.AA05674@caip.rutgers.edu> I am sorry to be reposting this but I forgot to put the subject in my last message. Sincere apologies to all. A neural network basically classifies training and testing samples into different regions in an n dimensional space. By generating the output of the network for all possible points in the space one constructs a n + 1 dimensional surface with n independent variables. A polynomial can be generated to approximate this surface. It should be possible to construct a "polynomial neural network" that can do this job. The neurons individually may implement simple polynomials (using sigma-pi units maybe). I would really appreciate any pointers to any research on this (published or unpublished). The work that I am aware of dates to the late 60's--A.G. Ivakhnenko and Donald Specht--though they did not model their systems as nn's. I would also like feedback on what the potential use of such nets may be over typical work like back prop. Thanks in anticipation Ananth Sankar Dept. of Electrical Engg. Rutgers University NJ From honavar at cs.wisc.edu Thu Oct 12 14:29:02 1989 From: honavar at cs.wisc.edu (A Buggy AI Program) Date: Thu, 12 Oct 89 13:29:02 -0500 Subject: Polynomial Nets Message-ID: <8910121829.AA20557@goat.cs.wisc.edu> Here is a list of papers that address the use of "higher order" neurons or links that maybe interpreted as computing terms of a polynomial: Giles, C. L., & Maxwell, T., Learning, invariance, and generalization in higher order neural networks, Applied Optics, vol 26, pp 4972-4978, 1987. Klassen, M. S., & Pao, Y. H., Characteristics of the functional link net: A higher order delta rule net, Proc. of the 2nd annual IEEE conference on Neural Networks, San Diego, CA, 1988. Honavar, V., and Uhr, L. A network of neuron-like units that learns to perceive by generation as well as reweighting of links, Proc. of the 1988 Connectionist models summer school, ed: Touretzky, Hinton, and Sejnowski, Morgan Kaufmann, CA. 1988. Honavar, V., and Uhr, L. Generation, Local receptive fields, and global convergence improve perceptual learning in connectionsit networks, Proc. of IJCAI-89, Morgan Kaufmann, CA. 1989. L. Uhr, Generation+Extraction gives optimal space-time learning of Boolean functions, to appear, Connection Science, 1989. Honavar, V., & Uhr, L. Brain-Structured Connectionist networks that perceive and learn, to appear, Connection Science, 1989. Durbin, R., & Rumelhart, D. E., Product unit: A computationally powerful and biologically plausible extension to backpropagation networks, Neural Computation, vol. 1, pp 133-142. There is also some work in more traditional (inductive) machine learning that falls in this category. Hope this helps. Vasant honavar at cs.wisc.edu From ff%FRLRI61.BITNET at CUNYVM.CUNY.EDU Thu Oct 12 10:30:12 1989 From: ff%FRLRI61.BITNET at CUNYVM.CUNY.EDU (ff%FRLRI61.BITNET@CUNYVM.CUNY.EDU) Date: Thu, 12 Oct 89 15:30:12 +0100 Subject: INNC-90-PARIS Message-ID: <8910121430.AA19182@sun3a.lri.fr> --------------------------------------------------------------------------- INNC 90 PARIS --------------------------------------------------------------------------- INTERNATIONAL NEURAL NETWORK CONFERENCE JULY 9-13, 1990 PALAIS DES CONGRES PARIS FRANCE --------------------------------------------------------------------------- Co-chairmen of the Conference: B. Widrow (Stanford University) B. Angeniol (Thomson-CSF) Program committee chairman: T. Kohonen (Helsinki University) members: I. Aleksander (Imperial College) S. Ichi Amari (Univ. of Tokyo) L. Cooper (Brown Univ.) R. Eckmiller (Univ. of Dusseldorf) F. Fogelman (Univ. of Paris 11) S. Grossberg (Boston Univ.) D. Rumelhart (Stanford Univ.) *: to be confirmed P. Treleaven (University College London) C. von der Malsburg (Univ.of South California) ----------------------------------------------------------------------------- Members of the international community are invited to submit original papers to the INNS-90-PARIS by january 20,1990, in english, on scientific and industrial developments in the following areas: A-APPLICATIONS B-IMPLEMENTATIONS C-THEORY D-COMMERCIAL ----------------------------------------------------------------------------- THE CONFERENCE will include one day of tutorials four days of conference poster sessions prototype demonstrations A forum with workshop sessions:specific interest groups,products sessions deal sessions. ---------------------------------------------------------------------------- For information, contact: Nina THELLIER NTC INNC-90-PARIS 19 rue de la Tour 75116 PARIS-FRANCE Tel: (33-1) 45 25 65 65 Fax: (33-1) 45 25 24 22 ----------------------------------------------------------------------------- Francoise Fogelman From poggio at ai.mit.edu Thu Oct 12 17:37:21 1989 From: poggio at ai.mit.edu (Tomaso Poggio) Date: Thu, 12 Oct 89 17:37:21 EDT Subject: Polynomial Nets In-Reply-To: A Buggy AI Program's message of Thu, 12 Oct 89 13:29:02 -0500 <8910121829.AA20557@goat.cs.wisc.edu> Message-ID: <8910122137.AA15281@wheat-chex> There are much older paper s that are relevant. From Dave.Touretzky at B.GP.CS.CMU.EDU Thu Oct 12 22:54:08 1989 From: Dave.Touretzky at B.GP.CS.CMU.EDU (Dave.Touretzky@B.GP.CS.CMU.EDU) Date: Thu, 12 Oct 89 22:54:08 EDT Subject: tech report available Message-ID: <9726.624250448@DST.BOLTZ.CS.CMU.EDU> The following tech report is now available: BoltzCONS: Dynamic Symbol Structures in a Connectionist Network David S. Touretzky School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213-3890 CMU-CS-89-182 BoltzCONS is a connectionist model that dynamically creates and manipulates composite symbol structures. These structures are implemented using a functional analog of linked lists, but BoltzCONS employs distributed representations and associative retrieval in place of a conventional memory organization. Associative retrieval leads to some interesting properties. For example, the model can instantaneously access any uniquely-named internal node of a tree. But the point of the work is not to reimplement linked lists in some peculiar new way; it is to show how neural networks can exhibit compositionality and distal access (the ability to reference a complex structure via an abbreviated tag), two properties that distinguish symbol processing from lower-level cognitive functions such as pattern recognition. Unlike certain other neural net models, BoltzCONS represents objects as a collection of superimposed activity patterns rather than as a set of weights. It can therefore create new structured objects dynamically, without reliance on iterative training procedures, without rehearsal of previously-learned patterns, and without resorting to grandmother cells. This paper will appear in a special issue of Artificial Intelligence devoted to connectionist symbol processing. Note: the BoltzCONS work first appeared in print in 1986. This paper offers a more detailed description of the model than previously available, and a much more thorough analysis of its significance and weak points. ................ To order a copy of this tech report, write to the School of Computer Science at the address above, or send email to Copetas at cs.cmu.edu. Ask for report number CMU-CS-89-182. From weili at wpi.wpi.edu Thu Oct 12 23:33:36 1989 From: weili at wpi.wpi.edu (Wei Li) Date: Thu, 12 Oct 89 23:33:36 EDT Subject: large scale networks Message-ID: <8910130333.AA01880@wpi.wpi.edu> Hi, I would like to get some references on performances of neural networks on large size problems such as applied to a power system network which has at least thousands of state variables. Are there any neural networks which have showed good convergence abilities for large scale problems? How could they learn so many correlations and arrive at a stable state? Can they be simulated in a work station type computer or minicomputer? Thanks a lot. e-mail: weili at wpi.wpi.edu Wei Li EE. Dept., WPI 100 Institute Road Worcester, MA 01609 From pollack at cis.ohio-state.edu Fri Oct 13 10:48:47 1989 From: pollack at cis.ohio-state.edu (Jordan B Pollack) Date: Fri, 13 Oct 89 10:48:47 EDT Subject: Job Posting Message-ID: <8910131448.AA02815@toto.cis.ohio-state.edu> My dept is recruiting a couple of faculty in areas which migbt be of interest to this group. The advertisement for COMPUTATIONAL MODELS of NEURAL INFO. PROCESSING, going out to press is enclosed below. Since the area is quite large and vague, we have two subareas in mind, but quality will overrule discipline. The first subarea is "Biologically Realistic Connec- tionism", and would deal with working models of neurons, organs, or small creatures. The second potentially skips over biology and goes right to math and physics. "Non- Linear Cognition", or the study of complex dynamical systems related either to brain or mind (e.g. self-organizing circu- itry, cellular automata (reversibility?) chaos and complex- ity theory, fractal patterns in speech/music, and so on. We are also recruiting on a separate billet in SPEECH PROCESSING, which could easily be in neural networks as well. Please contact me if you want to discuss it, or know of anybody good. Columbus is an especially nice place to live. Jordan pollack at cis.ohio-state.edu -------------------------------------------------------------------- Laboratory for Artificial Intelligence Research Department of Computer and Information Science and The Center for Cognitive Science at the The Ohio State University Position Announcement in Computational Neuroscience A tenure-track faculty position at the Assistant Pro- fessor level is expected to be available in the area of Com- putational Neuroscience. We are seeking outstanding appli- cants who have a strong background and research interest in developing computational models of neural information pro- cessing. A Ph.D. in computer science, or in some other appropriate area with a sufficiently strong background in computation, is required. The candidate will be a regular faculty member in the Department of Computer & Information Science, and will promote interactions among cognitive science, computer science and brain science through the Center for Cognitive Science. The LAIR has strong symbolic and connectionist projects underway, the Department has wide interests in parallel com- putation, and the University has the major facilities in place to support the computational neuroscience enterprise, including several parallel computers, a Cray Y/MP, and a full range of brain imaging systems in the medical school. Applicants should send a resume along with the names and addresses of at least three professional references to Prof. B. Chandrasekaran Department of Computer & Information Science Ohio State University 2036 Neil Ave. Columbus, OH 43210 The Ohio State University is an Equal Opportunity Affirmative Action Employer, and encourages applications from qualified women and minorities. From gluck at psych.Stanford.EDU Fri Oct 13 11:34:22 1989 From: gluck at psych.Stanford.EDU (Mark Gluck) Date: Fri, 13 Oct 89 08:34:22 PDT Subject: higher-order/polynomial units in human learning models Message-ID: The use of "higher-order" or polynomial units also has a long tradition in animal and human learning theory where they are called "configural-cues." We have found that such units, combined with the LMS algorithm, do quite well in predicting and fitting a wide range of complex human classification and recognition behaviors (often better than base-line backprop networks). This work is described in: Gluck, Bower, & Hee (1989). A configural-cue network model of animal and human associative learning. Proceedings of the Eleventh Annual Meeting of the Cognitive Science Society, Ann Arbor, MI. Lawrence Erlbaum Associates: Hillsdale, NJ Reprint requests can be sent to: gluck at psych.stanford.edu From jose at neuron.siemens.com Fri Oct 13 12:54:13 1989 From: jose at neuron.siemens.com (Steve Hanson) Date: Fri, 13 Oct 89 12:54:13 EDT Subject: new address Message-ID: <8910131654.AA12546@neuron.siemens.com.siemens.com> As of October 1 I have moved to Siemens in Princeton S. J. Hanson Siemens Research Center 755 College Road East Princeton, New Jersey 08540 jose at tractatus.siemens.com (609)734-3360 I can also be reached at Cognitive Science Laboratory 221 Nassau Street Princeton University Princeton, New Jersey 08542 jose at clarity.princeton.edu (609)258-2219 Please update your info. Thanks Steve From rsun at cs.brandeis.edu Mon Oct 16 12:41:32 1989 From: rsun at cs.brandeis.edu (Ron Sun) Date: Mon, 16 Oct 89 12:41:32 edt Subject: higher-order neurons Message-ID: There are more generalized neuronal models that exibit interesting properties. Some of them are discrete rather than continuous. Some have weights while others do not even have weights. They are higher-order, in the sense that they can compute more complicated mappings. Some references here: I. Aleksander, The Logic of connectionist system, in: Neural Computing Architectures, MIT Press, 1989 A. Barto, From Chemotaxis to Cooperativity, COINS TR 88-65, University of Massachusetts, 1988 J. Feldman and D. Ballard, Connectionist models and their properties, Cognitive Science, July, 1982 A. Klopf, The Hedonistic Neuron, Hemisphere, 1982 R. Sun, A discrete neural network model for conceptual representation and reasoning, 11th Cognitive Science Society Conference, 1989 R. Sun, E. Marder and D. Waltz, Model local neural networks in the lobster stomatogastric ganglion, IJCNN, 1989 R. Sun, Designing inference engines based on a discrete neural network model, Proc. IEA/AIE, 1989 R. Sun, The Discrete Neuron and The Probabilistic Discrete Neuron, submitted to INNC, 1989 R. Sutton and A. Barto, A Temporal-Difference Model of Classical Conditioning, Proceedings of 9th Cognitive Science Society Conference, 1987 From lange at CS.UCLA.EDU Tue Oct 17 08:45:25 1989 From: lange at CS.UCLA.EDU (Trent Lange) Date: Tue, 17 Oct 89 05:45:25 PDT Subject: tech report available Message-ID: <891017.124525z.22274.lange@lanai.cs.ucla.edu> **********DO NOT FORWARD TO OTHER BBOARDS************** **********DO NOT FORWARD TO OTHER BBOARDS************** **********DO NOT FORWARD TO OTHER BBOARDS************** The following tech report is now available: High-Level Inferencing in a Connectionist Network Trent E. Lange Michael G. Dyer Artificial Intelligence Laboratory Computer Science Department University of California, Los Angeles Los Angeles, CA 90024 UCLA-AI-89-12 Connectionist models have had problems representing and apply- ing general knowledge rules that specifically require variables. This variable binding problem has barred them from performing the high-level inferencing necessary for planning, reasoning, and natural language understanding. This paper describes ROBIN, a structured neural network model capable of high-level inferencing requiring variable bindings and rule application. Variable bind- ings are handled by signatures -- activation patterns which uniquely identify the concept bound to a role. Signatures allow multiple role-bindings to be propagated across the network in parallel for rule application and dynamic inference path instantiation. Signa- tures are integrated within a connectionist semantic network struc- ture whose constraint-relaxation process selects between those newly-instantiated inferences. This allows ROBIN to handle an area of high-level inferencing difficult even for symbolic models, that of resolving multiple constraints from context to select the best interpretation from among several alternative and possibly ambiguous inference paths. This is a pre-print of a paper that will appear in Connection Science. ................ To order a copy of this tech report, write to Trent Lange at the address above, or send e-mail to lange at cs.ucla.edu. Ask for tech report number UCLA-AI-89-12. From jose at neuron.siemens.com Thu Oct 19 07:21:40 1989 From: jose at neuron.siemens.com (Steve Hanson) Date: Thu, 19 Oct 89 07:21:40 EDT Subject: Job Announcement Message-ID: <8910191121.AA15853@neuron.siemens.com.siemens.com> Learning & Knowledge Acquisition Siemens Corporate Research, Inc, the US research branch of Siemens AG with sales in excess of 30$ Billion worldwide has research openings in the Learning and Knowledge Acquisition Group for research staff scientists. The group does basic and applied studies in the areas of Learning (Connectionist and AI), adaptive processes, and knowledge acquisition. Above and beyond Laboratory facilities, the group has a network of sun workstations (sparcs), file and compute servers, Lisp machines and a mini-supercomputer all managed by a group systems administrator/research programmer. Connections exist with our sister laboratory in Munich, Germany as well as with various leading Universities including MIT, CMU and Princeton University, in the form of joint seminars, shared postdoctoral position, and collaborative research. The susscessful candidate should have a Ph.D. in Computer Science Electrical Engineering, or any other AI-related or Cognitive Science field. Areas that we are soliciting for presently are in Neural Computation, or Connectionist Modeling especially related to Learning Algorithms, Novel Architectures, Dynamics, Biological Modeling, and including any of the following application areas Pattern Classification/Categorization, Speech Recognition, Visual Processing, Sensory Motor Control (Robotics), Problem Solving, Natural Language Understanding, Siemens is an equal opportunity employer, Please send your resume and a reference list to Stephen J. Hanson Learning and Knowledge Acquisition Group Siemens Corporate Research, Inc. 755 College Road East Princeton, NJ 08540 jose at tractatus.siemens.com jose at clarity.princeton.edu From skuo at caip.rutgers.edu Thu Oct 19 14:34:31 1989 From: skuo at caip.rutgers.edu (Shyh-shiaw Kuo) Date: Thu, 19 Oct 89 14:34:31 EDT Subject: No subject Message-ID: <8910191834.AA16589@caip.rutgers.edu> Hi, I will appreciate if you can tell me any reference which apply polynomials to pattern recognition, or neural nets. As I know, people name those methods as Group Method of Data Handling (GMDH) algorithm, or Self-Organizing method. I also need some current publications, say after 1983, which state the improvements of those algorithms. Your help will be highly appreciated. --- Shyh-shiaw Kuo From welleken at wind.bellcore.com Thu Oct 19 17:54:53 1989 From: welleken at wind.bellcore.com (Christian Wellekens) Date: Thu, 19 Oct 89 17:54:53 EDT Subject: Polynomials in PR Message-ID: <8910192154.AA16909@wind.bellcore.com> You could have a look on the recent book of Y-H Pao : Adaptive Pattern Recognition and Neural networks, Addison Wesley. 1989 Professor Pao is at the EEDpt of the CASE Western Reserve University and is president of AI Ware. Chris Wellekens Bellcore 2M336 445 South St Morristown NJ 07960-1910 From kube%cs at ucsd.edu Thu Oct 19 18:40:46 1989 From: kube%cs at ucsd.edu (Paul Kube) Date: Thu, 19 Oct 89 15:40:46 PDT Subject: polynomial classifiers In-Reply-To: Shyh-shiaw Kuo's message of Thu, 19 Oct 89 14:34:31 EDT <8910191834.AA16589@caip.rutgers.edu> Message-ID: <8910192240.AA08953@kokoro.UCSD.EDU> W. E. Blanz at IBM Almaden (San Jose) has worked on polynomial classifiers and compared their performance to maximum likelihood and connectionist classifiers. You might be interested in his IBM reports RJ 5418 "A comparison of polynomial and parametric gaussian maximum likelihood classifiers" and RJ 6891 "Comparing a connectionist trainable classifier with classical decision analysis methods." The latter reports, by the way, that a PDP classifier outperforms gaussian ML and less-than-quartic polynomial classfiers, and scales better besides. --Paul kube at ucsd.edu From chuck%henry at gte.com Fri Oct 20 10:37:00 1989 From: chuck%henry at gte.com (Chuck Anderson) Date: Fri, 20 Oct 89 10:37 EDT Subject: polynomial nets Message-ID: <19891020143736.9.CHUCK@henry.gte.com> A good source for descriptions of methods and applications of GMDH is the book: "Self-Organizing Methods in Modeling: GMDH Type Algorithms", edited by Stanley J. Farlow, published by Marcel Dekker, Inc., NY, 1984. Barron Associates, Inc., of Stanardsville, Virginia, have been investigating a number of variations of GMDH for some time. I have seen publications by Roger Barron and John Elder from there. In "Automated Design of Continuously-Adaptive Control..." by Elder and Barron, in the proceedings of the 1988 American Control Conference, a polynomial net is used to form a map from aircraft sensors to control actions. The sensors indicate errors between actual and desired response of the aircraft. The net learns actions that compensate for damaged control surfaces on the aircraft. Good actions were determined by an off-line optimization using an aircraft simulation. The net was used to interpolate between these optimized values. Another Barron, Andrew, at the Univ. of Ill., is contributing to this work by exploring the links between polynomial nets and more standard statistical methods. See Andrew Barron and Roger Barron, "Statistical Learning Networks: A Unifying View", Proc. of the 1988 Symposium on the Interface: Statistics and Computing Science, Reston, VA, April 21-23. And, at the upcoming NIPS conferece Andrew Barron is giving an invited presentation on polynomial networks. Chuck Anderson GTE Laboratories Inc. 40 Sylvan Road Waltham, MA 02254 617-466-4157 canderson%gte.com at relay.cs.net From rohit at faulty.che.utexas.edu Fri Oct 20 11:51:30 1989 From: rohit at faulty.che.utexas.edu (rohit@faulty.che.utexas.edu) Date: Fri, 20 Oct 89 10:51:30 CDT Subject: help with ANN Message-ID: <8910201551.AA00218@faulty.che.utexas.edu.che.utexas.edu> I am trying to design an ANN to model a function which takes a step as an input and shifts the step in time and changes the amplitude of the step . I am using backward propogation with sigmoid functions for the nodes in the hidden layers, can anyo ne make any suggestions. I had posted this question on the ai.neural.nets and a couple of people said that they had done it but nobody could tell me how they did it i.e architecture of the net waht inputs etc... I would appreciate it if somone would make some concrete suggestions. rohit at faulty.che.utexas.edu From John.Hampshire at SPEECH2.CS.CMU.EDU Sat Oct 21 19:49:50 1989 From: John.Hampshire at SPEECH2.CS.CMU.EDU (John.Hampshire@SPEECH2.CS.CMU.EDU) Date: Sat, 21 Oct 89 19:49:50 EDT Subject: connectionist step function manipulator Message-ID: If you want to time-shift and amplitude-scale a step function, I think it's easiest to build a simple algorithm to do this. On the assumption that this approach is undesirable for some reason, I'd offer the following: Look into classic linear systems/signal processing theory. Linear systems theory focusses heavily on the step response (and impulse response) of linear systems and how these time-domain responses relate to frequency-domain representations via Fourier transformation. One way to get your desired network would start with specifying the desired step response you want, differentiating it in the time domain to obtain the equivalent impulse response, then Fourier transforming this result for the frequency domain equivalent. If your desired step response involves a time shifting and amplitude scaling, then these manipulations constitute a pretty simple impulse response that transforms to a phase-shift and amplitude scaling in the frequency domain. There is a huge volume of information on the Fast Fourier Transform (dating back to C. F. Gauss, but generally associated with Cooley and Tukey [1965]). Probably the best overview is given by Oppenheim and Schafer in "Discrete Time Signal Processing" (Prentice Hall, 1989) or the predecessor by the same authors and publisher "Digital Signal Processing" (1975). One can view the various implementations of the FFT as neural networks despite the fact that they are ultimately linear operators. I imagine a some folks will dispute this equation, but if adaptive phasing devices for antenna arrays are viewed as neural nets, FFTs can be too. Anyway, if you must use a "neural network", you could implement your desired function in discrete time using an N-point FFT of your input, a frequency-domain multiplication of the transformed input and transformed impulse response, then an N-point inverse FFT. Oppenheim and Schafer (1989) chapter 8 covers this concept pretty well (there's more to it than this explanation). If you wanted to *learn* the desired time-shift and scaling factor (instead of deriving it a-priori), I imagine you could set up the FFT and inverse FFT structures and then use an error function with backpropagation (through the UN-CHANGING "connections" of the inverse FFT stage of the network). The error signal would backpropagate to alter the coefficients of your frequency domain representation of the impulse response. Of course, those frequency domain coefficients are *complex*, not pure real (as are a number of the "connections" in the inverse FFT structure), and I haven't really considered why such an idea might not work, but what the heck. Then there's the question of *why* you'd want to learn the frequency domain form of the impulse response if you can derive it in closed form a-priori, but I'm sure there's a good reason associated with your wanting to use a connectionist architecture. If all of this is old news to you, then sorry --- toss it in the trash. If it's new info then I hope it's of some help. Cheers, John P. S. all of this is really just cramming traditional signal processing into connectionist packaging. I don't mean to claim otherwise. From victor%FRLRI61.BITNET at CUNYVM.CUNY.EDU Mon Oct 23 06:44:26 1989 From: victor%FRLRI61.BITNET at CUNYVM.CUNY.EDU (victor%FRLRI61.BITNET@CUNYVM.CUNY.EDU) Date: Mon, 23 Oct 89 11:44:26 +0100 Subject: DB - NN Message-ID: <8910231044.AA16142@sun8.lri.fr> Dear colleges, I will appreciate if you can tell me any reference which apply neural networks to Data Base Management Systems (DBMS). Your help will be highly appreciated. Victor Cruz victor at lri.lri.fr In advance, thanks a lot From LIN2%YKTVMZ.BITNET at CUNYVM.CUNY.EDU Mon Oct 23 17:11:01 1989 From: LIN2%YKTVMZ.BITNET at CUNYVM.CUNY.EDU (LIN2%YKTVMZ.BITNET@CUNYVM.CUNY.EDU) Date: Mon, 23 Oct 89 17:11:01 EDT Subject: Preprint available Message-ID: ********* FOR CONNECTIONISTS ONLY - PLEASE DO NOT FORWARD *********** **************** TO OTHER BBOARDS/ELECTRONIC MEDIA ******************* The following preprint is available. If you would like a copy, please send a note to lin2 @ ibm.com CONTAINING *ONLY* THE INFORMATION ON THE FOLLOWING FOUR LINES (to allow semi-automated handling of your request): *IJ* Your Name Your Address (each line not beyond column 33) Designing a Sensory Processing System: What Can Be Learned from Principal Components Analysis? Ralph Linsker IBM Research, T. J. Watson Research Center, Yorktown Heights, NY 10598 Principal components analysis (PCA) is a useful tool for understanding some feature-analyzing properties of cells found in at least the first few stages of a sensory process- ing pathway. However, the relationships between the results obtained using PCA, and those obtained using a Hebbian model or an information-theoretic optimization principle, are not as direct or clear-cut as sometimes thought. These points are illustrated for the formation of center- surround and orientation-selective cells. For a model "cell" having spatially localized connections, the relevant PCA eigenfunction problem is shown to be separable in polar coordinates. As a result, the principal components have a radially sectored (or "pie-slice") geometric form, and (in the absence of additional degeneracies) do *not* resemble classic Hubel-Wiesel "simple" cells, except for the (odd- symmetry) eigenmodes that have exactly two sectors of oppo- site sign. However, for suitable input covariance functions, one can construct model "cells" of simple-cell type -- which are in general not PCA eigenfunctions -- as particular linear combinations of the first few leading principal components. A connection between PCA and a criterion for the minimiza- tion of a geometrically-weighted mean squared reconstruction error is also derived. This paper covers in greater detail one of the topics to be discussed in an invited talk at the IJCNN Winter 1990 Meet- ing (Washington, DC, Jan. 1990). It will be published in the conference proceedings. The paper itself contains no abstract; the above is a brief summary prepared for this preprint availability notice. From gary%cs at ucsd.edu Fri Oct 20 20:30:47 1989 From: gary%cs at ucsd.edu (Gary Cottrell) Date: Fri, 20 Oct 89 17:30:47 PDT Subject: seminar announcement: Modeling the Sequential Behavior of the Dog Message-ID: <8910210030.AA01374@desi.UCSD.EDU> SEMINAR Modeling the Sequential Behavior of the Dog: The Second Naive Dog Physics Manifesto Garrison W. Cottrell Department of Dog Science Condominium Community College of Southern California Most work in Dog Modeling has been content to make do with simple Stimulus-Response type models. However, the thing that separates current work in Parallel Dog Processing from the Behaviorists is the emphasis on looking inside the dog's head. So far, few dogs have consented to this procedure, hence, we have to make do with models that tell us what we might find if we looked. S-R models since Pavlov have assumed that there is not much in the head except a connection from the dog's nose to his salivary gland, that may be detached at the nose end and reconnected to his ear via a process called "conditioning"[1]. Departing from the Behaviorists, PDP modelers make the radical assumption that there is a brain inside the dog's head[2], mediating these reponses based on the current state of the dog's brain. However, rather than treating the dog's brain as analogous to a telephone switching network as the neo-Skinnerians do[3], we will treat the dog as a dynamical system, in particular, a dissipative system that takes in material from its environment, extracts energy to maintain its own structure, increasing the entropy of the material before returning it to the environment. The main problem of the dog owner, then, is to train this dynamical system to leave its entropy outside the house. In our work this sequence of desired behavior is specified by the following action grammar, a simplified version of the one used in (Cottrell, 1986a): Day -> Action Day | Sleep Action -> Eat | leavecondo Walk Eat -> Eat chomp | chomp Walk -> poop Walk | trot Walk | sniff Walk | entercondo As previously noted, these rules have the desirable property that entropy in the condo is ungrammatical. In our previous work (Cottrell, 1986a), we took a competence theory approach, i.e., no working computer program was necessary for our theory. While the advantages of the lack of real world constraints that a competence theory approach allows are clear[4], it lacks the advantage of interchange with experiment that performance theories enjoy. In this talk we will describe an approach that avoids the pitfalls of a performance theory (having to deal with data) while incorporating the exchange with experimental modeling by building a computer model of our competence theory[5]. In order to generate a sequence such as that specified by our action grammar, a recurrent network is necessary. To model the initial state of the de novo dog, we start with a randomly wired recurrent network with habituation in the weights. The behavior of this network is remarkably similar to that of the puppy, oscillating wildly, exhibiting totally undisciplined behavior, until reaching a fixed point. Habituation then determines the length of the sleep phase, the model slowly "wakes up", and the cycle starts again[6]. We then apply the WiZ algorithm (Willy & Zippy, 1988) for recurrent bark propagation to train the network to the adult behavior[7]. The training set of sequences of states was generated from the simplified grammar above. Note that the network must actually choose a branch of the grammar to take on any iteration. By simply training the network to take different branches from a nonterminal state on different occasions, the network is unstable when at a nonterminal node. Different actions are then possible attractors from that state. By using a random updating rule, different transitions naturally occur. Transitions out of a particular terminal state are due to habituation, i.e., our dog model stops doing something because of the equivalent of boredom with any particular state[8]. Thus, boredom is an emergent property of our model. ____________________ [1]The obvious implausibility of such a process notwithstanding (cf. Chompski's scathing critique, No Good Dogs, 1965), hordes of researchers have spent many years studying it. [2]However, it is often hard to explain this view to the lay public, especially most dog owners. [3]This was a major improvement on older behaviorist theories. All that is needed now is to posit that conditioning somehow accesses the "telephone operator" in the brain that pulls a plug and reinserts it somewhere else. This model is much more plausible since the mechanisms of conditioning have been fleshed out. It also explains why dogs some- times don't respond at all - they haven't kept up on the phone bill. [4]For example, in (Cottrell, 1986a) we were able to assume that one could generate a context free language with a feed-forward network. All you needed was "hidden units". This is the familiar cry of the "connec- tionist" who has never implemented a network. [5]This is known as the autoerotic approach to theory building. A danger here is that, since Reality has no reason to dampen the possible oscillations between computer simulation and competence theory forma- tion, the process may have positive Lyapunov exponents, and never con- verge on a theory. Such unstable loops can lead to strange attractors that never settle down, such as current linguistic theory. [6]Since the length of time spent in the attractor is determined by the number of units participating in it, it was found that most of the puppy's brain is actually needed for maintaining its extremely long sleep phase. This could be an entirely new explanation of the apparent lack of capacity for much else. [7]In order to get the proper behavior out of our network, teacher- forcing was necessary. This confirms our experience with actual dogs that force is often a necessary component of the training process. [8]This is obviously an inadequate model of how the dog stops eating, which is more due to external reality than any internal control on the dog's part. For this simple process, a Skinnerian feedforward See-Food -> Eat network is sufficient. From rik%cs at ucsd.edu Tue Oct 24 22:33:27 1989 From: rik%cs at ucsd.edu (Rik Belew) Date: Tue, 24 Oct 89 19:33:27 PDT Subject: TR available: Evolution, Learning and Culture too! Message-ID: <8910250233.AA02582@roland.UCSD.EDU> EVOLUTION, LEARNING AND CULTURE: Computational metaphors for adaptive algorithms Richard K. Belew Cognitive Computer Science Research Group Computer Science & Engr. Dept. (C-014) Univ. California at San Diego La Jolla, CA 92093 rik%cs at ucsd.edu CSE Technical Report #CS89-156 Potential interactions between connectionist learning systems and algorithms modeled after evolutionary adaptation are becoming of increasing interest. In a recent, short and elegant paper Hinton and Nowlan extend a version of Holland's Genetic Algorithm (GA) to consider ways in which the evolution of species and the learning of individuals might interact. Their model is valuable both because it provides insight into potential interactions between the {\em natural} processes of evolution and learning, and as a potential bridge between the {\em artificial} questions of efficient and effective machine learning using the GA and connectionist networks. This paper begins by describing the GA and Hinton and Nowlan's simulation. We then analyze their model, use this analysis to explain its non-trivial dynamical behaviors, and consider the sensitivity of the simulation to several key parameters. Our next step is to interpose a third adaptive system --- culture --- between the learning of individuals and the evolution of populations. Culture accumulates the ``wisdom'' of individuals' learning beyond the lifetime of any one individual but adapts more responsively than the pace of evolution allows. We describe a series of experiments in which the most minimal notion of culture has been added to the Hinton and Nowlan model, and use this experience to comment on the functional value of culture and similarities between and interactions among these three classes of adaptive systems. ------------------------------------------------------- Copies of this technical report are available by sending $3 (and asking for Technical Report #CS89-156) to: Ms. Kathleen Hutcheson CSE Dept. (C-014) Univ. Calif. -- San Diego La Jolla, CA 92093 From skuo at caip.rutgers.edu Wed Oct 25 09:27:52 1989 From: skuo at caip.rutgers.edu (Shyh-shiaw Kuo) Date: Wed, 25 Oct 89 09:27:52 EDT Subject: No subject Message-ID: <8910251327.AA02580@caip.rutgers.edu> I will appreciate if you can recommend me any recently published book which is related to Neural Network. From hinton at ai.toronto.edu Wed Oct 25 10:56:14 1989 From: hinton at ai.toronto.edu (Geoffrey Hinton) Date: Wed, 25 Oct 89 10:56:14 EDT Subject: references on predicting time series Message-ID: <89Oct25.105634edt.10980@ephemeral.ai.toronto.edu> We are interested in neural networks that take as input a sequence of symbols or a sequence of parameter-vectors and produce as output a prediction of the next term in the sequence. The prediction is the form of a probability distribution over possible symbols or parameter-vectors. I know that a number of researchers have considered this use of networks. I would like to compile a list of good references on this, particularly recent references. If you know of such a reference (and you are feeling generous), please email it to me (not to all connectionists). geoff From rik%cs at ucsd.edu Wed Oct 25 13:49:03 1989 From: rik%cs at ucsd.edu (Rik Belew) Date: Wed, 25 Oct 89 10:49:03 PDT Subject: $$ for TR's Message-ID: <8910251749.AA03323@roland.UCSD.EDU> From fjcp60!golds at uunet.UU.NET Wed Oct 25 04:22:14 1989 To: uunet!cs.ucsd.edu!rik Subject: tech report I think it is very unfortunate you require payment for the distribution of your tech report. This is contrary to the spirit of free exchange of information embodied in the connectionists mailing list. For individuals like myself who are not currently supported by a grant, or who don't work in a department rich in grant funds, the imposition of fees raises a real barrier to obtaining access research results. Other authors have found ways to provide copies of reports free of charge, and/or have placed a postscript version of their reports on the public archive machine for free FTP access. I would really appreciate it if you could pursue one of these alternate methods of distribution. Thanks... Dr. R. Goldschmidt golds at fjcp60.uu.net 2227 Greenwich St. Falls Church, VA 22043 I couldn't agree more, and you are not the only one to have expressed surprise/bemusement/anger at a charge for a TR. And I feel very silly mentioning it, too. But that happens to be how our department has established the TR distribution procedure and I'm not flush enough to cover all the costs. But for those of you for whom $3 is an impediment I will keep sending copies out gratis until my money runs out. Also, I too think Jordan Pollack's "TR server" of Postscript files is first rate. But it happens that my TR has lots of figures that I haven't been able to successfully fold into the TeX source, and I decided the text needed these figures. (And the department would charge $3 to send just them out!) But my goal is to have a fully Postscript file, next time. I apologize for taking up everyone's time with this silliness. Rik Belew From park at sophocles.cs.uiuc.edu Wed Oct 25 14:18:17 1989 From: park at sophocles.cs.uiuc.edu (Young-Tack Park) Date: Wed, 25 Oct 89 13:18:17 -0500 Subject: references on predicting time series Message-ID: <8910251818.AA02437@sophocles.cs.uiuc.edu> I am interested in the topic. Could you please pass me the list? Thanks in advance, Young-Tack From hinton at ai.toronto.edu Wed Oct 25 16:29:01 1989 From: hinton at ai.toronto.edu (Geoffrey Hinton) Date: Wed, 25 Oct 89 16:29:01 EDT Subject: references on time-series analysis Message-ID: <89Oct25.162910edt.11250@ephemeral.ai.toronto.edu> I have received many requests for copies of the reference list. I will compile all replies into one list and send it to everyone next week. So there is no longer any need to ask me for it . Geoff From loeb at PSYCHE.MIT.EDU Wed Oct 25 16:00:52 1989 From: loeb at PSYCHE.MIT.EDU (Eric Loeb) Date: Wed, 25 Oct 89 16:00:52 edt Subject: references on predicting time series Message-ID: <8910252002.AA22053@ATHENA.MIT.EDU> I am interested too. Could whoever compiles it publish it for all? Thanks, Eric Loeb From Dave.Touretzky at B.GP.CS.CMU.EDU Wed Oct 25 17:48:00 1989 From: Dave.Touretzky at B.GP.CS.CMU.EDU (Dave.Touretzky@B.GP.CS.CMU.EDU) Date: Wed, 25 Oct 89 17:48:00 EDT Subject: a note about etiquette Message-ID: <8290.625355280@DST.BOLTZ.CS.CMU.EDU> Okay you guys: the long-term subscribers to this list have started complaining, so it's time to remind people yet again of the rules of etiquette for using CONNECTIONISTS. Rule #1: if you're too lazy to do your own literature search, don't expect anyone on this list to do it for you. Messages of the form "Please send me a list of books on neural nets" are not appropriate for this forum. The CONNECTIONISTS list is for serious discussion of research, and for dissemination of tech reports and talk announcements. If you're a novice, you should be reading Neuron Digest, not this list. On the other hand, if you're an established neural net researcher already familiar with the literature, and you want to compile a bibliography which you will then distribute for the benefit of the entire community, then a call for references is fine. Rule #2: don't send personal follow-up messages like "Please send me a copy too" to the CONNECTIONISTS list. If you don't know how to use your mail system properly, kindly un-subscribe yourself from this list until you learn. Rule #3: never tell anyone about CONNECTIONISTS or NN-BENCH. Only tell them about CONNECTIONISTS-Request and NN-BENCH-Request. That way requests for subscriptions won't be sent to the wrong place and rebroadcast to 500+ sites all over the globe. Thank you for your cooperation. -- The Management From mdr at dspvax.mit.edu Thu Oct 26 09:16:52 1989 From: mdr at dspvax.mit.edu (Michael D. Richard) Date: Thu, 26 Oct 89 09:16:52 EDT Subject: Mailing list Message-ID: <8910261316.AA20534@dspvax.mit.edu> Please add my e-mail address to the Connectionist mailing list Thanks mdr at dspvax.mit.edu From rudnick at cse.ogc.edu Thu Oct 26 18:50:40 1989 From: rudnick at cse.ogc.edu (Mike Rudnick) Date: Thu, 26 Oct 89 15:50:40 PDT Subject: proteins, rna, dna refs Message-ID: <8910262250.AA19581@cse.ogc.edu> Below is a summary of responses to my earlier posting asking for references to work using ANNs for the recognition of proteins, rna, dna, and the like. They are in the order in which they were received, and have been edited both to remove duplicate references and for relevancy. Thanks to all those who responded. Mike ************************************************************************* From: ohsu-hcx!spackmank at cse.ogc.edu (Dr. Kent Spackman) Subject: connectionist protein structure The two articles I mentioned are: Holley, L.H.; Karplus, M. Protein structure prediction with a neural network. Proceeding of National Academy of Science, USA; 1989; 86: 152-156. Qian, Ning; Sejnowski, Terrence J. Predicting the secondary structure of globular proteins using neural network models. J Mol Biol; 1988; 202: 865-884. I have an article that will be published in the proceedings of the Symposium on Computer Applications in Medical Care, in Washington, D.C., in November, entitled: "Evaluation of Neural Network Performance by ROC analysis: Examples from the Biotechnology Domain". Authors are M.L. Meistrell and myself. Kent A. Spackman, MD PhD Biomedical Information Communication Center (BICC) Oregon Health Sciences University 3181 SW Sam Jackson Park Road Portland, OR 97201-3098 ---- From: Lambert.Wixson at MAPS.CS.CMU.EDU Subject: DNA,RNA, etc. Holley and Karplus, Proceedings of the National Academy of Science 86, 152-156 (89). ---- From: mv10801 at uc.msc.umn.edu Subject: Re: applications to DNA, RNA and proteins George Wilcox (mf12801 at sc.msc.umn.edu) does work on predicting protein tertiary structure using large backprop nets. --Jonathan Marshall Center for Research in Learning, Perception, and Cognition 205 Elliott Hall, Univ. of Minnesota, Minneapolis, MN 55455 ---- >From munnari!cluster.cs.su.OZ.AU!ray at uunet.UU.NET Fri Sep 29 23:40:55 1989 Subject: applications to DNA, RNA and proteins Borman, Stu "Neural Network Applications In Chemistry Begin to Appear", C&E News, April 24 1989, pp 24-28. Thornton, Janet "The shape of things to come?" Nature, Vol. 335 (1st September 1988), pp 10-11. You probably know about the Qian and Sejnowski paper already. The Thornton "paper" is a fast overview with a sentence or two comparing Q&S's work with other work. Borman's C&E piece is fairly superficial, but it mentions some other people who have played with this stuff, including Bryngelson and Hopfield, Holley and Karplus (who apparantly have published in Proc. Nat. Acad. Sci., 86(1), 152 (1989)) and Liebman. The 1990 Spring Symposium at Stanford (March 27-29, 1990) will have a session on "Artificial Intelligence and Molecular Biology". The CFP lists Neural Networks (very broad-minded of them!), so it might be worth a look when it comes around. Raymond Lister Basser Department of Computer Science University of Sydney NSW 2006 AUSTRALIA Internet: ray at cs.su.oz.AU CSNET: ray%cs.su.oz at RELAY.CS.NET UUCP: {uunet,hplabs,pyramid,mcvax,ukc,nttlab}!munnari!cs.su.oz.AU!ray ---- From: "Evan W. Steeg" Subject: NNets and macromolecules There is a fair amount of work on applying neural networks to questions involving DNA, RNA, and proteins. The two major types of application are: 1) Using neural networks to predict conformation (secondary structure and/or tertiary structure) of molecules from their sequence (primary structure). 2) Using nets to find regularities, patterns, etc. in the sequence itself, e.g. find coding regions, search for homologies between sequences, etc. The two areas are not disjoint -- one might look for alpha-helix "signals" in a protein sequence as part of a structure prediction method, for example. I did my M.Sc. on "Neural Network Algorithms for RNA Secondary Structure Prediction", basically using a modified Hopfield-Tank (Mean Field Theory) network to perform an energy minimization search for optimal structures. A technical report and journal paper will be out soon. I'm currently working on applications of nets to protein structure prediction. (Reference below). Qian and Sejnowski used a feed-forward net to predict local secondary structure of proteins. (Reference above). At least two other groups repeated and extended the Qian & Sejnowski experiments. One was Karplus et al (ref. above) and the other was Cotterill et al in Denmark. (Discussed in a poster at the Fourth International Symposium on Artificial Intelligence Systems, Trento, Italy Sept. 1988). Finally, a group in Minnesota used a supercomputer and back-prop to try to find regularities in the 2-d distance matrices (distances between alpha-carbon atoms in a protein structure). An interim report on this work was discussed at the IJCNN-88 (Wash. DC) conference. (Sorry, I don't recall the names, but the two researchers were at the Minnesota Supercomputer Center, I believe.) As for the numerous research efforts in finding signals and patterns in sequences, I don't have these references handy. But the work of Lapedes of Los Alamos comes to mind as an interesting bit of work. Refs: E.W. Steeg. Neural Network Algorithms for the Prediction of RNA Secondary Structure. M.Sc. Thesis, Computer Science Dept., University of Toronto, Toronto, Ontario, Canada, 1988. Evan W. Steeg (416) 978-7321 steeg at ai.toronto.edu (CSnet,UUCP,Bitnet) Dept of Computer Science steeg at ai.utoronto (other Bitnet) University of Toronto, steeg at ai.toronto.cdn (EAN X.400) Toronto, Canada M5S 1A4 {seismo,watmath}!ai.toronto.edu!steeg ----- From: pastor at PRC.Unisys.COM (Jon Pastor) Subject: Re: applications to DNA, RNA and proteins @article(nakata85a, Author="K. Nakata and M. Kanehisa and D. DeLisi", Title="Prediction of splice junctions in mRNA sequences", Journal="Nucleic Acids Research", Year="1985", Volume="13", Number="", Month="", Pages="5327--5340", Note="", Annote="") @article(stormo82a, Author="G.D. Stormo and T.D. Schneider and L.M. Gold ", Title="Characterization of translational initiation sites in E. coli", Journal="Nucleic Acids Research", Year="1982", Volume="10", Number="", Month="", Pages="2971--2996", Note="", Annote="") @article(stormo82b, Author="G.D. Stormo and T.D. Schneider and L.M. Gold and A. Ehrenfeucht", Title="Use of the `perceptron' algorithm to distinguish translational initiation sites in E. coli", Journal="Nucleic Acids Research", Year="1982", Volume="10", Number="", Month="", Pages="2997--3010", Note="", Annote="") In addition, there is going to be (I think) a paper by Alan Lapedes, from Los Alamos, in a forthcoming book published by the Santa Fe Institute; my group also has a paper in this book, which is how I know about Lapedes' submission. I am going to try to contact the editor to see if I can get a preprint; if so, I'll let you know. I didn't attend the meeting at which Lapedes presented his paper, but I'm told that he was looking for splice junctions. ---- From: ff%FRLRI61.BITNET at CUNYVM.CUNY.EDU (Francoise Fogelman) Subject: proteins We have done some work on the prediction of secondary structures of proteins. This was presented at a NATO meeting (Les Arcs, march 1989) and will be published in the proceedings. F. Fogelman LRI Bat 490 Universite de Paris Sud 91405 ORSAY cedex FRANCE Tel 33 1 69 41 63 69 e-mail: ff at lri.lri.fr ---- The book "Evolution, Learning and Cognition", the article "Learning to Predict the Secondary Structure of Globular Proteins" by N. Qian & T. J. Sejnowski. From inesc!lba%alf at relay.EU.net Thu Oct 26 12:37:33 1989 From: inesc!lba%alf at relay.EU.net (Luis Borges de Almeida) Date: Thu, 26 Oct 89 15:37:33 -0100 Subject: a note about etiquette In-Reply-To: Dave.Touretzky@B.GP.CS.CMU.EDU's message of Wed, 25 Oct 89 17:48:00 EDT <8290.625355280@DST.BOLTZ.CS.CMU.EDU> Message-ID: <8910261437.AA01425@alf.inesc.pt> Dear Dr. Touretzky, I have just read your "etiquette" message to the CONNECTIONISTS, with which I fully agree. However, I think that even people who can use e-mail reasonably well, sometimes don't notice that if they just 'reply' to a message, their reply will normally be sent both to the list and to the author of the message, causing quite a bit of undesirable traffic. Maybe it would be a good idea to explicitly discourage people from using 'reply', and to tell them to always address their response directly to the author of the message they received from the list. Regards, Luis B. Almeida From esmythe at andrew.ge.com Fri Oct 27 11:37:21 1989 From: esmythe at andrew.ge.com (Erich J Smythe) Date: Fri, 27 Oct 89 11:37:21 EDT Subject: a note about etiquette Message-ID: <8910271537.AA12133@ge-dab> If I may add one small request to the etiquette message: PLEASE put your email address somewhere in your posting. Our mailer gets confused with these messages and strips the sender's address, so If I want to respond to the author, I have to guess. Yes, I know, it's our mailer's fault (look at the path this message had to follow), but some things are too hard to change quickly ("you want us to spend money on _what_????"). Thanks -erich smythe GE Advanced Technology Labs. esmythe at atl.ge.com Moorestown, NJ From tomritch at admin.ogc.edu Fri Oct 27 12:23:25 1989 From: tomritch at admin.ogc.edu (Tom Ritch) Date: Fri, 27 Oct 89 09:23:25 PDT Subject: proteins, rna, dna refs Message-ID: <8910271623.AA22792@ogcadmin.OGC.EDU> Thanks for the info. I'll dig through it and see what I can find. Tom From Dave.Touretzky at B.GP.CS.CMU.EDU Sat Oct 28 05:23:24 1989 From: Dave.Touretzky at B.GP.CS.CMU.EDU (Dave.Touretzky@B.GP.CS.CMU.EDU) Date: Sat, 28 Oct 89 05:23:24 EDT Subject: a note about etiquette In-Reply-To: Your message of Thu, 26 Oct 89 15:37:33 -0100. <8910261437.AA01425@alf.inesc.pt> Message-ID: <12579.625569804@DST.BOLTZ.CS.CMU.EDU> I don't think people pay attention when you tell them not to use Reply. It's an ingrained habit and it's hard to make them change, although we do suggest it in the intro message sent to new subscribers. Some mailers allow users to set defaults controlling whether they reply to just the sender of a message or to the whole to/cc list. Some people set their defaults the "wrong" way. Other people are just too lazy to double check the To and Cc lines of their messages before mailing. These are the ones especially unlikely to use Mail instead of Reply. Flaming them periodically may encourage them to take more care. Cheers, -- Dave From ST401843%BROWNVM.BITNET at VMA.CC.CMU.EDU Sun Oct 29 16:23:21 1989 From: ST401843%BROWNVM.BITNET at VMA.CC.CMU.EDU (thanasis kehagias) Date: Sun, 29 Oct 89 16:23:21 EST Subject: request for recurrent nets bibliography Message-ID: I am trying to compile a bibliography of recurrent neural networks. i have traced about twenty references , which i enclose in this message . however i want to do something more complete, so i decided to go public with the following request: please send me any and all references you have on recurrent neural nets. recurrent neural nets, for my purposes, is any net of identical units that perform local computations, and where the output of units at time t is fed back to the units at time t+1. this is a fairly general definition. feel free to include references to older works of connectionist flavor. if you feel generous send me the references in the bibtex format , a sample of which is given below in my preliminary list of references. if you feel even more generous, send me a copy of the work you recommend (especially if it is by yourslef) to the surface address: Thanasis Kehagias Division of Applied Math Brown University Providence, RI 02912 if you see a work of yours in the preliminary list that is not fully or accurately listed, please send me the corrections. please send me more theoretically flavored work: if you have five application papers that use the same concepts from recurrent nets, you can send me just one that captures the essentials of the theory you use. supplemntary request: if you feel like it, send me references to work that deals with learning probability distributions, either of random variables (static) or stochastic processes (dynamic - this probably uses recurrent nets). of course, i am depending on your good will: send me as much as you can without interfering with your normal work schedule. any offers are welcome. it goes without saying that i will compile the list and make it available through connectionists for anybody who is interested. thank you very much - thansis kehagias preliminary bibliography: ========================= @ARTICLE{kn: AUTHOR= "R.E. Scneider", TITLE= "The Neuron as a Sequential Mahine", JOURNAL= "", YEAR= "", VOLUME= "" } @ARTICLE{kn: AUTHOR= "V. Rajlich", TITLE= "Dynamics of certain Discrete Systems and Self Reproduction of Patterns", JOURNAL= "", YEAR= "", VOLUME= "" } @ARTICLE{kn: AUTHOR= "R.M. Golden", TITLE= "A Unified Framework for Connectionist Systems", JOURNAL= "Biol. Cybernetics", YEAR= "1988", VOLUME= "59" } @ARTICLE{kn: AUTHOR= "L.I. Rozonoer", TITLE= "Random Logical Nets, I-III (in Russian)", JOURNAL= "Avtomatika i Telemekhanika" YEAR= "1969" VOLUME= "5" } @ARTICLE{kn: AUTHOR= "I. Parberry", TITLE= "Relating Boltzmann Machines to Conventional Models of Computation", JOURNAL= "Neural Networks", YEAR= "1989", VOLUME= "2" } @ARTICLE{kn: AUTHOR= "J.J. Hopfield", TITLE= "Neurons with Graded Response have Collective Computational Properties like those of Two-State Neurons", JOURNAL= "Proc. Nat'l Acad. Sci. USA", YEAR= "1984", VOLUME= "81" } @ARTICLE{kn: AUTHOR= "W.S. Stornetta", TITLE= "A Dynamical Approach to Temporal Pattern Processing", JOURNAL= "", YEAR= "", VOLUME= "" } @ARTICLE{kn: AUTHOR= "M.I. Jordan", TITLE= "Supervised Learning and Systems with Excess Degrees of Freedom", JOURNAL= "COINS Technical Report", YEAR= "1988", VOLUME= "88-27" } @ARTICLE{kn: AUTHOR= "F.J. Pineda", TITLE= "Generalization of Back Propagation to Recurrent Neural Nets", JOURNAL= "Physical Review Letters" , YEAR= "1987", VOLUME= "59" } @ARTICLE{kn: AUTHOR= "F.J. Pineda", TITLE= "Dynamics and Architecture for Neural Computation", JOURNAL= "Journal of Complexity", YEAR= "1988", VOLUME= "4" } @ARTICLE{kn: AUTHOR= "G.Z. Sun", TITLE= "A Recurrent Network that learns Context Free Grammars", JOURNAL= "", YEAR= "", VOLUME= "" } @ARTICLE{kn: AUTHOR= "B.A. Pearlmutter", TITLE= "Learning State Space Trajectories in Recurrent Neural Nets", JOURNAL= "", YEAR= "", VOLUME= "" } @ARTICLE{kn: AUTHOR= "J.J. Hopfield", TITLE= "Neural Nets and Physical Systems with Emergent Collective Computational Properties", JOURNAL= "Proc. Nat'l Acad. Sci. USA", YEAR= "1982", VOLUME= "" } @ARTICLE{kn: AUTHOR= "S. Amari", TITLE= "Characteristics of Random Nets of Analog JOURNAL= "IEEE SMC", YEAR= "1972", VOLUME= "SMC-2" } @ARTICLE{kn: AUTHOR= "D.H. Ackley et.al.", TITLE= "A Learning Algorithm for Boltzmann Machines", JOURNAL= "Cognitive Science", YEAR= "1985", VOLUME= "9" } @ARTICLE{kn: AUTHOR= "C. Peterson and J.R. Anderson", TITLE= "A Mean Field Theory Learning Algorithm for Neural Nets", JOURNAL= "Complex Systems", YEAR= "1987", VOLUME= "1" } @ARTICLE{kn: AUTHOR= "H. Bourlard and C.J. Wellekens", TITLE= "Links between Markov Models and Multilayer Perceptrons", JOURNAL= "Phillips Research Lab", YEAR= "1988", VOLUME= "M 263" } @ARTICLE{kn: AUTHOR= "H. Bourlard and C.J. Wellekens", TITLE= "Speech Dynamics and Recurrent Neural Nets", JOURNAL= "Proc. IEEE ICASSP", YEAR= "1989", VOLUME= "" } From bukys at cs.rochester.edu Tue Oct 31 16:22:13 1989 From: bukys at cs.rochester.edu (bukys@cs.rochester.edu) Date: Tue, 31 Oct 89 16:22:13 EST Subject: announcing: Rochester Connectionist Simulator, Version 4.2 Message-ID: <8910312122.AA23726@stork.cs.rochester.edu> The Rochester Connectionist Simulator, version 4.2, is now available by anonymous FTP from CS.Rochester.Edu, in the directory pub/simulator. (Don't forget the FTP BINARY mode when retrieving compressed files!) The simulator is too big to mail electronically, so please don't ask. The same files are available to subscribers of UUNET's UUCP service. They are stored in the directory ~uucp/pub/simulator. This new version includes an X11 interface, and it should run with little effort on Vaxen, Sun-3s, Sun-4s (but not on Sun386i machines), DECstations, and MIPS workstations. It includes various bug and documentation fixes that have been accumulating for the last 18 months. A Macintosh/MPW port of the 4.1 simulator has also been contributed for redistribution. Finally, version 4.2 adopts the licensing terms of the Free Software Foundation. If you are unable to obtain anonymous FTP or UUCP access to the simulator distribution, you can still order a copy the old-fashioned way. Send a check for US$150 (payable to the University of Rochester) to: Peg Meeker Computer Science Department University of Rochester Rochester, NY 14627 (USA) You will, in return, receive a distribution tape and a 200-page manual. PLEASE SPECIFY WHETHER YOU WANT: a 1600bpi 1/2" reel OR a QIC-24 (SUN) 1/4" cartridge. If you have a PostScript printer, you should be able to produce your own copy of the manual. If you want a paper copy of the manual anyway, send a check for $10 per manual (payable to the University of Rochester) to Peg Meeker at the above address. We do not have the facilities for generating invoices, so payment is required with any order. If you do decide to use the simulator, you should join the simulator users' mailing list, to keep up with the latest news, patches, and helpful hints. To join, drop me a note at the following address... Liudvikas Bukys From Dave.Touretzky at B.GP.CS.CMU.EDU Mon Oct 2 17:00:06 1989 From: Dave.Touretzky at B.GP.CS.CMU.EDU (Dave.Touretzky@B.GP.CS.CMU.EDU) Date: Mon, 02 Oct 89 17:00:06 EDT Subject: NIPS '89 preliminary program Message-ID: <14643.623365206@DST.BOLTZ.CS.CMU.EDU> Below is the preliminary program for the upcoming IEEE Conference on Neural Information Processing Systems - Natural and Synthetic, which will be held November 27 through 30, 1989. A postconference workshop series will take place November 30 through December 2. For registration information, please contact the Local Arrangements Chair, Kathie Hibbard, by sending email to hibbard at boulder.colorado.edu, or by writing to: Kathie Hibbard NIPS '89 University of Colorado Campus Box 425 Boulder, Colorado 80309-0425 ================================================================ ____________________________________________ ! ! ! PRELIMINARY PROGRAM, NIPS '89 ! ! DENVER, COLORADO ! ! NOVEMBER 27 _ NOVEMBER 30, 1989 ! !___________________________________________! OUTLINE Monday, November 27, 1989 4:00 PM: Registration 6:30 PM: Reception and Conference Dinner 8:30 PM: After-Dinner Plenary Talk by Jack Cowan Tuesday, November 28, 1989 8:00 AM: Continental Breakfast 8:30 AM - 12:30 PM: Oral Session1 - Neuroscience 12:30 - 2:30 PM: Poster Preview Session 1A, 1B, 1C - Neuroscience, Implementation and Simulation, Applications 2:30 - 6:30 PM: Oral Session 2 - Algorithms, Architectures, and Theory I 7:30 - 10:30 PM: Refreshments and Poster Session 1A,1B, 1C - Neuroscience, Implementation and Simulation, Applications Wednesday, November 29, 1989 8:00 AM: Continental Breakfast 8:30 AM - 12:30 PM: Oral Session3 - Applications 12:30 - 2:30 PM: Poster Preview Session 2 - Algorithms, Architectures, and Theory 2:30 - 6:30 PM: Oral Session 4 - Implementationand Simulation 7:30 - 10:30 PM: Refreshments and Poster Session 2 - Algorithms, Architectures, and Theory Thursday, November 30, 1989 8:00 AM: Continental Breakfast 8:30 AM - 1:00 PM: OralSession 5 - Algorithms, Architectures, and Theory II Friday, December 1 - Saturday, December 2, 1989 Post Conference Workshops at Keystone ________________________________ ! MONDAY, NOVEMBER 27, 1989 ! !______________________________! 4:00 PM: Registration 6:30 PM: Reception and Conference Dinner 8:30 PM: After-Dinner Plenary Talk "Some NeuroHistory: Neural Networks from 1952-1967," by Jack Cowan - University of Chicago. ________________________________ ! TUESDAY, NOVEMBER 28, 1989 ! !_______________________________! ORAL SESSION 1 NEUROSCIENCE SESSION CHAIR: James Bower, California Institute of Technology Tuesday, 8:30 AM - 12:30 PM 8:30 "Acoustic-Imaging Computations by Echolocating Bats: Unification of Diversely-Represented Stimulus Features into Whole Images," by Jim Simmons - Brown University (Invited Talk). 9:10 "Rules for Neuromodulation of Small Neural Circuits," by Ronald M. Harris-Warrick - Section of Neurobiology and Behavior, Cornell University. 9:40 "Neural Network Analysis of Distributed Representations Of Sensory Information In The Leech," by S.R. Lockery, G. Wittenberg, W. B. Kristan Jr., N. Qian and T. J. Sejnowski -Department of Biology, University of California, San Diego and Computational Neurobiology Laboratory, The Salk Institute. 10:10 BREAK 11:00 "Reading a Neural Code,"by William Bialek, Fred Rieke, R. R. de Ruyter van Steveninck, and David Warland - Departments of Physics and Biophysics, University of Californiaat Berkeley. 11:30 "Neural Network Simulation of Somatosensory Representational Plasticity," by KamilA. Grajski and Michael M. Merzenich - Coleman Memorial Laboratories, University of California, San Francisco. 12:00 "Brain Maps and Parallel Computer Maps," by Mark E. Nelson and James Bower - Division of Biology, California Institute of Technology. POSTER PREVIEW SESSION 1A NEUROSCIENCE Tuesday, 12:30 - 2:30 PM A1. "Category Learning and Object Recognition in a Simple Oscillating Model of Cortex " by Bill Baird - Department of Physiology, University of California Berkeley. A2. "From Information Theory to Structure and Function in a Simplified Model of a Biological Perceptual System," by Ralph Linsker - IBM Research, T. J. Watson Research Center. A3. "Development and Regeneration of Brain Connections: A Computational Theory," by J.D. Cowan and A.E. Friedman - Mathematics Department, University of Chicago. A4. "Collective Oscillations in Neuronal Networks: Functional Architecture Drives Dynamics," by Daniel M. Kammen, Philip J. Holmes, and Christof Koch - Computation and Neural Systems Program, California Institute of Technology. A5. "Computer Simulation of Oscillatory Behavior in Cerebral Cortical Networks," by M.A. Wilson and J.M. Bower - Computation and Neural Systems Program, Division of Biology, California Institute of Technology. A6. "A Neural Network Model of Catecholamine Effects: Enhancement of Signal Detection Performance is an Emergent Property of Changes in Individual Unit Behavior," by David Servan-Schreiber, Harry Printz and Jonathan Cohen - Departments of Computer Science and Psychology, Carnegie Mellon University. A7. "Non-Boltzmann Dynamics in Networks of Spiking Neurons," by Michael Crair and William Bialek - Departments ofPhysics and Biophysics, University of California at Berkeley. A8. "A Computer Modeling Approach toUnderstanding the Inferior Olive and Its Relationship to the Cerebellar Cortexin Rats," by Maurice Lee and James M. Bower - Computation and Neural Systems Program, California Institute of Technology. A9. "An Analog VLSI Model of Adaptationin the Vestibulo-Ocular Reflex," by Stephen P. DeWeerth and Carver A. Mead - California Institute of Technology. A10. "Can Simple Cells Learn Curves? A Hebbian Model in a Structured Environment," by William R. Softky and Daniel M. Kammen - Divisions of Physics and Biology and Computation and Neural Systems Program, California Institute of Technology. A11. "Formation of Neuronal Groupsin Simple Cortical Models," by Alex Chernjavsky and John Moody - Section of Molecular Neurobiology, Howard Hughes Medical Institute,Yale University. A12. "Signal Propagation in Layered Networks," by Garrett T. Kenyon, Eberhard E. Fetz and Robert D. Puff - University of Washington, Department of Physics. A13. "A Systematic Study of the Input/OutputProperties of a Model Neuron With Active Membranes," by Paul Rhodes - University of California, San Diego. A14. "Analytic Solutions to the Formation of Feature-Analyzing Cells of a Three-Layer Feedforward Information Processing Neural Net," by D.S. Tang - Microelectronics and Computer Technology Corporation. A15. "The Computation of Sound Source Elevation in the Barn Owl" by C.D. Spence and J.C. Pearson, David Sarnoff Research Center. POSTER PREVIEW SESSION 1B IMPLEMENTATION AND SIMULATION Tuesday, 12:30 - 2:30 PM B1. "Real-Time Computer Vision and Robotics Using Analog VLSI Circuits," by Christof Koch, John G. Harris, Tim Horiuchi, Andrew Hsu, and Jin Luo - Computation and Neural Systems Program, California Institute of Technology. B2. "The Effects of Circuit Integration on a Feature Map Vector Quantizer," by Jim Mann - MIT Lincoln Laboratory. B3. "Pulse-Firing Neural Chips Implementing Hundreds of Neurons," by Alan F. Murray, Michael Brownlow, AlisterHamilton, Il Song Han, H. Martin Reekie, and Lionel Tarassenko - Department of Electrical Engineering, University of Edinburgh, Scotland. B4. "An Efficient Implementation ofthe Backpropagation Algorithm on the Connection Machine CM-2," by Xiru Zhang, Michael Mckenna, Jill P. Mesirov, and David Waltz - Thinking Machines Corporation. B5. "Performance of Connectionist Learning Algorithms on 2-D SIMD Processor Arrays," by Fernando J. Nunez and Jose A.B. Fortes - School of Electrical Engineering, Purdue University. B6. "Dataflow Architectures: Flexible Platforms for Neural Network Simulation," by I.G. Smotroff - The MITRE Corporation. B7. "Neural Network Visualization," by Jakub Wejchert and Gerald Tesauro - IBM Research, T.J. Watson Research Center. POSTER PREVIEW SESSION 1C APPLICATIONS Tuesday, 12:30 - 2:30 PM C1. "Computation and Learning in Artificial Dendritic-Type Structures: Application to Speech Recognition," by Tony Bell - Free University of Brussels, Belgium. C2. "Speaker Independent Speech Recognition with Neural Networks and Speech Knowledge," by Yoshua Bengio, Regis Cardin, and Renato De Mori - McGill University, School of Computer Science. C3. "HMM Speech Recognition with Neural Net Discrimination," by William Y. Huang and Richard P. Lippmann- MIT Lincoln Laboratory. C4. "Connectionist Architectures for Multi-Speaker Phoneme Recognition," by John B. Hampshire II and Alex H. Waibel - School of Computer Science, Carnegie Mellon University. C5. "Performance Comparisons Between Backpropagation Networks and Classification Trees on Three Real-World Applications," by Les Atlas, Ronald Cole, Yeshwant Muthusamy, James Taylor, and Etienne Barnard - Department of Electrical Engineering, University of Washington, Seattle. C6. "Combining Visual and Acoustic Speech Signals with a Neural Network Improves Intelligibility," by Ben P. Yuhas, M.H. Goldstein, Jr., and Terrence J. Sejnowski - Speech Processing Laboratory, Department of Electrical and Computer Engineering, Johns Hopkins University. C7. "A Neural Network for Real-Time Signal Processing," by Donald B. Malkoff - General Electric / Advanced Technology Laboratories. C8. "A Neural Network to Detect Homologies in Proteins," by Yoshua Bengio, Yannick Pouliot, Samy Bengio,and Patrick Agin - McGill University, School of Computer Science. C9. "Recognizing Hand-Drawn and Handwritten Symbols with Neural Nets," by Gale L. Martin and James A. Pittman - MCC,Austin. C10. "Model Based Image Compression and Adaptive Data Representation by Interacting Filter Banks," by Toshiaki Okamoto, Mitsuo Kawato, Toshio Inui, and Sei Miyake - ATR Auditory and Visual Perception Research Laboratories, Japan. C11. "A Large-Scale Network Which Recognizes Handwritten Kanji Characters," by Yoshihiro Mori and Kazuki Joe - ATR Auditory and Visual Perception Research Laboratories, Japan. C12. "Traffic: Object Recognition Using Hierarchical Reference Frame Transformations," by Richard S. Zemel, Michael C. Mozer, and Geoffrey Hinton - Department of Computer Science, University of Toronto. C13. "Comparing the Performance of Connectionist and Statistical Classifiers on an Image Segmentation Problem," by Sheri L. Gish and W.E. Blanz - IBM Knowledge Based Systems, Menlo Park. C14. "A Modular Architecture For Target Recognition Using Neural Networks," by Murali M. Menon and Eric J. Van Allen - MIT Lincoln Laboratory. C15. "Neurally Inspired Plasticity in Oculomotor Processes," by Paul Viola - Artificial Intelligence Laboratory, Massachusetts Institute of Technology. C16. "Neuronal Group Selection Theory: A Grounding in Robotics," by Jim Donnett and Tim Smithers - Department of Artificial Intelligence, University of Edinburgh, Scotland. C17. "Composite Holographic Associative Recall Model (CHARM) and Recognition Failure of Recallable Words," by Janet Metcalfe - Department of Psychology, University of California, San Diego. C18. "Using a Translation-Invariant Neural Network to Diagnose Heart Arrhythmia," by Susan Lee - Johns Hopkins Institute. C19. "Exploring Bifurcation Diagrams With Adaptive Networks," by Alan S. Lapedes and Robert M. Farber - Theoretical Division, Los Alamos National Laboratory. C20. "Generalized Hopfield Networks and Nonlinear Optimization," by Athanasios G. Tsirukis, Gintaras V. Reklaitis, and Manoel F. Tenorio - School of Chemical Engineering, Purdue University. ORAL SESSION 2 ARCHITECTURES, ALGORITHMS, AND THEORY I SESSION CHAIR: John Moody, Yale University Tuesday, 2:30 - 6:30 PM 2:30 "Statistical Properties of Polynomial Networks and Other Artificial Neural Networks: A Unifying View," by Andrew Barron - University of Illinois at Champaign-Urbana (Invited Talk). 3:10 "Supervised Learning: A Theoretical Framework," by Sara Solla, Naftali Tishby, and Esther Levin - AT&T Bell Laboratories. 3:40 "Practical Characteristics of Neural Network and Conventional Pattern Classifiers on Artificial and Speech Problems," by Yuchun Lee and Richard P. Lippmann - Digital Equipment Corporation and MIT Lincoln Laboratory. 4:10 BREAK 5:00 "The Cocktail Party Problem: Speech/Data Signal Separation Comparison Between Backprop and SONN," by Manoel F. Tenorio, John Kassebaum, and Christoph Schaefers - School of Electrical Engineering, Purdue University. 5:30 "Optimal Brain Damage," by Yann LeCun, John Denker, Sara Solla, Richard E. Howard, and Lawrence D. Jackel - AT&T Bell Laboratories. 6:00 "Sequential Decision Problems and Neural Networks," by Andrew G. Barto, Richard S. Sutton and Chris Watkins -Department of Computer and Information Science, University of Massachusetts, Amherst. POSTER SESSION 1A, 1B, 1C NEUROSCIENCE, IMPLEMENTATION AND SIMULATION, APPLICATIONS Tuesday, 7:30 - 10:30 PM (Papers are Listed Under Poster Preview Session) ___________________________________ ! WEDNESDAY, NOVEMBER 29, 1989 ! !__________________________________! ORAL SESSION 3 APPLICATIONS SESSION CHAIR: Richard Lippmann, MIT Lincoln Laboratory Wednesday, 8:30 AM - 12:30 PM 8:30 "Visual Preprocessing" by George Sperling - New York University (Invited Talk). 9:10 "Handwritten Digit Recognition with a Back-Propagation Network," by Y. LeCun, B. Boser, J.S. Denker, D. Henderson,R.E. Howard, W. Hubbard, and L.D. Jackel - AT&T BellLab oratories. 9:40 "A Self-Organizing Associative Memory System for Control Applications," by Michael Hormel - Department ofControl Theory and Robotics, Technical University of Darmstadt, Germany. 10:10 BREAK 11:00 "Variable Resolution Learning Techniques for Speech Recognition," by Kevin Lang and Geoffrey Hinton - Carnegie-Mellon University. 11:30 "Word Recognition in a Continuous Speech Recognition System Embedding MLP into HMM," by H. Bourlard andN. Morgan - International Computer Science Institute, Berkeley. 12:00 "A Computational Basis for Phonology," by David S. Touretzky and Deirdre W. Wheeler - Carnegie-Mellon University. POSTER PREVIEW SESSION 2 ARCHITECTURES, ALGORITHMS, AND THEORY Wednesday, 12:30 - 2:30 PM 1. "Using Local Networks to Control Movement," by ChristopherG. Atkeson - Department of Brain and Cognitive Sciencesand the Artificial Intelligence Laboratory, Massachusetts Institute of Technology. 2. "Computational Neural Theory for Learning Nonlinear Mappings," by Jacob Barhen and Sandeep Gulati - Jet PropulsionLab oratory, California Institute of Technology. 3. "Learning to Control an Unstable System Using Forward Modeling," by Michael I. Jordan and Robert A. Jacobs - Department of Brain and Cognitive Sciences, Massachusetts Institute ofTechnology. 4. "Discovering High Order Features With Mean Field Networks," by Conrad Galand and Geoffrey E. Hinton - Departmentof Computer Science, University of Toronto. 5. "Designing Application-Specific Neural Networks Using the Genetic Algorithm," by Steven A. Harp, Tariq Samad, and Aloke Guha - Honeywell CSDD. 6. "Two vs. Three Layers: An Empirical Study of Learning Performance and Emergent Representations," by Charles Martin and John Moody - Department of Computer Science, Yale University. 7. "Operational Fault Tolerance of CMAC Networks," by Michael J. Carter, Frank Rudolph, and Adam Nucci - IntelligentStructures Group, Dept. of Electrical and Computer Engineering, University of New Hampshire. 8. "A Model of Unification in Connectionist Networks," by Andreas Stolcke - Computer Science Division, University of California, Berkeley. 9. "Two-Dimensional Shape Recognition Using Sparse Distributed Memory: An Example of a Machine Vision System that Exploits Massive Parallelism for Both High-Level and Low-Level Processing," by Bruno Olshausen and Pentti Kanerva - Research Institute for Advanced Computer Science, NASA Ames Research Center. 10. "Predicting Weather Using a Genetic Memory: A Combination of Kanerva's Sparce Distributed Memory With Holland's Genetic Algorithms," by David Rogers - Research Institute for Advanced Computer Science, NASA Ames Research Center. 11. "Neural Network Weight Matrix Synthesis Using Optimal Control," by O. Farotimi, A. Dembo, and T. Kailath - Information Systems Laboratory, Department of Electrical Engineering, Stanford University. 12. "The CHIR Algorithm: A Generalization for Multiple Output Networks," by Tal Grossman - Department ofElectronics, Weizmann Institute of Science, Israel. 13. "Analysis of Linsker's Application of Hebbian Rules to Linear Networks," by David J. C. MacKay and Kenneth D. Miller - Department of Computation and Neural Systems, California Institute of Technology and Department of Physiology, University of California, San Francisco. 14. "A Generative Framework for Unsupervised Learning," by Steven J. Nowlan - Department of Computer Science, University of Toronto. 15. "An Adaptive Network Model of Basic-Level Learning in Hierarchically Structured Categories," by Mark A. Gluck, James E. Corter, and Gordon H. Bower - Stanford University. 16. "Generalization and Scaling in Reinforcement Learning," by David H. Ackley and Michael S. Littman - Bell Communications Research, Cognitive Science Research Group. 17. "Neural Implementation of Motivated Behavior: Feeding in an Artificial Insect," by Randall D. Beer and Hillel J.Chiel - Departments of Computer Engineering and Science and Biology and the Center for Automation and Intelligent Systems Research, Case Western Reserve University. 18. "Back Propagation in a Genetic Search Environment," by Wayne Mesard and Lawrence Davis - Bolt Beranek and Newman Systems and Technologies, Inc., Laboratories Division. 19. "A Method for the Associative Storage of Analog Vectors," by Amir F. Atiya and Yaser S. Abu-Mostafa - Department of Electrical Engineering, California Institute of Technology. 20. "Generalization and Parameter Estimation in Feedforward Nets: Some Experiments," by N. Morgan and H. Bourlard - International Computer Science Institute, Berkeley. 21. "Subgrouping Reduces Complexity and Speeds Up Learning in Recurrent Networks," by David Zipser- Department of Cognitive Science, University of California, San Diego. 22. "Sigma-Pi Learning: A Model for Associative Learning in Cerebral Cortex," by Bartlett W. Mel and Christof Koch - Computation and Neural Systems Program, California Institute of Technology. 23. "Complexity of Finite Precision Neural Network Classifier," by K. Y. Siu, A. Dembo, and T. Kailath - Information Systems Laboratory, Stanford University. 24. "Analog Neural Networks of Limited Precision I: Computing With Multilinear Threshold Functions," by Zoran Obradovic and Ian Parberry - Department of Computer Science, Pennsylvania State University. 25. "On the Distribution of the Local Minima of a Random Function of a Graph," by P. Baldi, Y. Rinott, and C. Stein - University of California, San Diego. 26. "A Neural Network For Feature Extraction," by Nathan Intrator - Center for Neural Science and Division of Applied Mathematics, Brown University. 27. "Meiosis Networks," by Stephen Jose Hanson - Cognitive Science Laboratory, Princeton University. 28. "Unsupervised Learning Using Velocity Field Approach," by Michail Zak - Jet Propulsion Laboratory,California Institute of Technology. 29. "Algorithms for Better Representation and Faster Learning in Radial Basis Function Networks," by Avijit Saha and James D. Keeler - MCC Austin, Texas. 30. "Generalization Performance of Overtrained Back-Propagation Networks: Some Experiments," by Y. Chauvin - Psychology Department, Stanford University. 31. "The 'Moving Targets' Training Method," by Richard Rohwer - Centre for Speech Technology Research, University of Edinburgh, Scotland. 32. "Optimal Learning and Inference Over MRF Models: Application To Computational Vision on Connectionist Architectures," by Kurt R. Smith, Badrinath Roysam, and Michael I. Miller - Washington University. 33. "A Cost Function for Learning Internal Representations," by J.A. Hertz, A. Krogh, and G.I. Thorbergsson - Niels Bohr Institute, Denmark. 34. "The Cascade-Correlation Learning Architecture," by Scott E. Fahlman and Christian Lebiere - School of Computer Science, Carnegie-Mellon University. 35. "Training Connectionist Networks With Queries and Selective Sampling," by D. Cohn, L. Atlas, R. Ladner, R. Marks II, M. El-ASharkawi, M. Aggoune, D. Park - Dept. of Electrical Engineering, University of Washington. 36. "Rule Representations in a Connectionist Chunker," by David S. Touretzky - School of Computer Science, Carnegie Mellon University. 37. "Unified Theory for Symmetric and Asymmetric Systems and the Relevance to the Class of Undecidable Problems," by I. Kanter - Princeton University. 38. "Synergy of Clustering Multiple Back Propagation Networks," by William P. Lincoln and Josef Skrzypek - Hughes Aircraft Company and Machine Perception Laboratory, UCLA. 39. "Training Stochastic Model Recognition Algorithms as Networks Can Lead to Maximum Mutual Information Estimation of Parameters," by John S. Bridle - Machine Intelligence Theory Section, Royal Signals and Radar Establishment, Great Britain. 40. "Self-Organizing Multiple-View Representations of 3D Objects," by D. Weinshall, S. Edelman, and H. Bulthoff - MIT Center for Biological Information Processing. 41. "A Recurrent Network that Learns Context-Free Grammars," by G.Z. Sun, H.H. Chen, C.L. Giles, Y.C. Lee, and D. Chen - Laboratory for Plasma Physics Research and Institute for Advanced Computer Studies, University of Maryland. 42. "Time Dependent Adaptive Neural Networks," by F. J. Pineda - Jet Propulsion Laboratory, California Institute of Technology. ORAL SESSION 4 IMPLEMENTATION AND SIMULATION SESSION CHAIR: Jay Sage, MIT Lincoln Laboratory Wednesday, 2:30 - 6:30 PM 2:30 "Visual Object Recognition" by Shimon Ullman - Massachusetts Institute of Technology and Weizmann Institute of Science (Invited Talk). 3:10 "A Reconfigurable Analog VLSI Neural Network Chip," by Srinagesh Satyanarayana, Yannis Tsividis, and Hans Peter Graf - Department of Electrical Engineering and Center for Telecommunications Research, Columbia University. 3:40 "Analog Circuits for Constrained Optimization," by John Platt - California Institute of Technology. 4:10 BREAK 5:00 "VLSI Implementation of a High-Capacity Neural Associative Memory," by Tzi-Dar Chiueh and Rodney M. Goodman - Department of Electrical Engineering, California Institute of Technology. 5:30 "Hybrid Analog-Digital 32x32x6-Bit Synapse Chips for Electronic Neural Networks," by A. Moopenn, T. Duong,and A. P. Thakoor - Jet Propulsion Laboratory, California Institute of Technology. 6:00 "Learning Aspect Graph Representations From View Sequences," by Michael Seibert and Allen M. Waxman - MIT Lincoln Laboratory. POSTER SESSION 2 ARCHITECTURES, ALGORITHMS, AND THEORY Wednesday, 7:30 - 10:30 PM (Papers are Listed Under Poster Preview Session) __________________________________ ! THURSDAY, NOVEMBER 30, 1989 ! !________________________________! ORAL SESSION 5 ARCHITECTURES, ALGORITHMS, AND THEORY II SESSION CHAIR: Eric Baum, NEC Research Institute Thursday, 8:30 AM - 1:00 PM 8:30 "Identification and Control of Dynamical Systems Using Neural Networks," by Bob Narendra - YaleUniversity (Invited Talk). 9:10 "Discovering the Structure of a Reactive Environment by Exploration," by Michael C. Mozer and Jonathan Bachrach - University of Colorado Boulder. 9:40 "The Perceptron Algorithm Is Fast at Modified Valiant Learning," by Eric B. Baum - Department of Physics, PrincetonUniversity. 10:10 BREAK 11:00 "Oscillations in Neural Computations," by Pierre Baldi and Amir Atiya - Jet Propulsion Laboratory and Division ofBiology, California Institute of Technology. 11:30 "Incremental Parsing by Modular Recurrent Connectionist Networks," by Ajay Jain and Alex Waibel - School of ComputerScience, Carnegie Mellon University. 12:00 "Neural Networks From Coupled Markov Random Fields via Mean Field Theory," by Davi Geiger and Federico Girosi - Artificial Intelligence Laboratory, Massachusetts Institute of Technology. 12:30 "Asymptotic Convergence of Back-Propagation," by Gerald Tesauro, Yu He, and Subatai Ahmad - IBM Thomas J. Watson Research Center. ____________________________________________________________ ! POST CONFERENCE WORKSHOPS AT KEYSTONE ! ! THURSDAY, NOVEMBER 30 - SATURDAY, DECEMBER 2, 1989 ! !____________________________________________________________! Thursday, November 30, 1989 5:00 PM: Registration and Reception at Keystone Friday, December 1, 1989 7:30 - 9:30 AM: Small Group Workshops 4:30 - 6:30 PM: Small Group Workshops 8:30 - 10:30 PM: Plenary Discussion Session Saturday, December 2, 1989 7:30 - 9:30 AM: Small Group Workshops 4:30 - 6:30 PM: Small Group Workshops 7:00 PM: Banquet From elman at amos.ucsd.edu Tue Oct 3 01:23:31 1989 From: elman at amos.ucsd.edu (Jeff Elman) Date: Mon, 2 Oct 89 22:23:31 PDT Subject: connectionist book series announcement Message-ID: <8910030523.AA07232@amos.ucsd.edu> - New Book Series Announcement - NEURAL NETWORK MODELING & CONNECTIONISM The MIT Press / Bradford Books This series will make available seminal state-of-the art research in neural network and connectionist modeling. The research in this area has grown explosively in recent years and has sparked controversy and debate in a wide variety of areas. Many researchers believe that this para- digm offers new and deep insights into the basis and nature of intelligent behavior in both biological and artificial systems. The series publishing program will include: monographs based on influential dissertations; monographs and in-depth reports of research programs based on mature work by leaders in the field; edited volumes and collections on topics of special interest; major reference works; undergraduate and graduate level textbooks. The series will be highly inter- disciplinary, spanning fields as diverse as psychology, linguistics, cognitive science, neuroscience, neurobiology and genetics, physics and biophysics, mathematics, computer science, artificial intelligence, engineering, and econom- ics. Potential authors are encouraged to contact any of the Editors or the Publisher. Editor: Jeffrey L. Elman Dept. of Cognitive Science UCSD; La Jolla, CA 92093 elman at amos.ucsd.edu Associate Editors: James Anderson (Brown) James McClelland (CMU) Andrew Barto (UMass/Amherst) Domenico Parisi (Rome) Gary Dell (Illinois) David Rumelhart (Stanford) Jerome Feldman (ICSI, Berkeley) Terrence Sejnowski (UCSD, Salk) Stephen Grossberg (BU) Paul Smolensky (Colorado) Stephen Hanson (Bellcore) Stephen Stich (Rutgers) Geoffrey Hinton (Toronto) David Touretzky (CMU) Michael Jordan (MIT) David Zipser (UCSD) Publisher: Henry B. Stanton The MIT Press / Bradford Books 55 Hayward Street; Cambridge MA 02142 From TESAURO at ibm.com Tue Oct 3 13:26:58 1989 From: TESAURO at ibm.com (Gerald Tesauro) Date: 3 Oct 89 13:26:58 EDT Subject: Neurogammon wins Computer Olympiad Message-ID: <100389.132658.tesauro@ibm.com> Neurogammon 1.0 is a backgammon program which uses multi-layer neural networks to make move decisions and doubling cube decisions. The networks were trained by back-propagation on large expert data sets. Neurogammon competed at the recently-held First Computer Olympiad in London, and won the backgammon competition with a perfect record of 5 wins and no losses. This is a victory not only for neural networks, but for the entire machine learning community, as it is apparently the first time in the history of computer games that a learning program has ever won a tournament. A short paper describing Neurogammon and the Olympiad results will appear in the next issue of Neural Computation. (This was inadver- tently omitted from Terry Sejnowski's recent announcement of the contents of the issue.) The paper may also be obtained on-line in plain text format by sending e-mail to TESAURO at ibm.com. From LEO at AUTOCTRL.RUG.AC.BE Tue Oct 3 11:20:00 1989 From: LEO at AUTOCTRL.RUG.AC.BE (LEO@AUTOCTRL.RUG.AC.BE) Date: Tue, 3 Oct 89 11:20 N Subject: Call for Papers Neural Network Applications Message-ID: *****************+--------------------------------------+**************** *****************|Second BIRA seminar on Neural Networks|**************** *****************+--------------------------------------+**************** CALL FOR PAPERS May 1990, Belgium Last year, BIRA (Belgian Institute for Automatic Control) organised a first seminar on Neural Networks. Some invited speakers (Prof. Fogelman Soulie, Prof. Bart Kosko, Dr. D. Handelman and Dr. S. Miyake) gave an introduction to the subject, and discussed some possible application fields. Because of the great interest from the industry as well as from the research world, we decided to organise a second edition on this subject. The aim of the second seminar is to show some excisting applications and possibilities of Neural Networks or Sub-Symbolic Systems with Neural Network features. So, if you have a working application or nice prototype of an industrial application based on Neural Networks, and you may and want to talk about it, please send us an abstract. Of course, the seminar will only be organised, if we receive enough interesting abstracts. This seminar will be organised by BIRA, Unicom and the AI-section of the Automatic Control Laboratory of the Ghent State University. Time schedule ------------- 01-01-1990 : Deadline for abstracts. 15-02-1990 : Confirmation of the speakers and the seminar 01-04-1990 : Deadline for full papers ..-05-1990 : Seminar Organisation contact information -------------------------------- Rob Vingerhoeds Leo Vercauteren State University of Ghent AI Section Automatic Control Laboratory Grote Steenweg Noord 2 B-9710 GENT - Zwijnaarde Belgium Fax: +32 91/22 85 91 Tel: +32 91/22 57 55 BIRA Coordinator: L. Pauwels BIRA-secretariaat Het Ingenieurshuis Desguinlei 214 2018 Antwerpen Belgium From John.Hampshire at SPEECH2.CS.CMU.EDU Tue Oct 3 10:08:17 1989 From: John.Hampshire at SPEECH2.CS.CMU.EDU (John.Hampshire@SPEECH2.CS.CMU.EDU) Date: Tue, 3 Oct 89 10:08:17 EDT Subject: TR announcement Message-ID: **************** PLEASE DO NOT FORWARD TO OTHER MAILING LISTS **************** Technical Report: CMU-CS-89-166 THE META-PI NETWORK: BUILDING DISTRIBUTED KNOWLEDGE REPRESENTATIONS FOR ROBUST PATTERN RECOGNITION J. B. Hampshire II and A. H. Waibel QUICK ABSTRACT (30 seconds, plain English, no frills) The "Meta-Pi" architecture is a multi-network connectionist backprop structure. It learns to focus attention on the output of a particular sub-network or group of sub-networks via multiplicative connections. When used to perform multi-speaker speech recognition this network yields recognition error rates as low as those for speaker DEpendent tasks (~98.5%), and about one third the rate of more traditional networks trained on the same multi-speaker task. Meta-Pi networks are trained for best output performance and *automatically* learn the best mix or selection of neural subcomponents. Here, for example, they learned about relevant speaker differences (and similarities) without being told to actually recognize the different speakers. If this sounds interesting, please read on. SUMMARY We present a multi-network connectionist architecture that forms distributed low-level knowledge representations critical to robust pattern recognition in non-stationary stochastic processes. This new network comprises a number of stimulus-specific sub-networks (i.e., networks trained to classify a particular type of stimulus) that are linked by a combinational superstructure. Our application employs Time-Delay Neural Network (TDNN) architectures for the sub-networks and the combinational superstructure of the Meta-Pi network, although one can use any form of backpropagation network as the basis for a Meta-Pi architecture. The combinational superstructure of the Meta-Pi network adapts to the stimulus being processed, optimally integrating stimulus-specific classifications based on its internally-developed model of the stimulus (or combination of stimuli) most likely to have produced the input signal. To train this combinational network we have developed a new form of multiplicative connection that we call the ``Meta-Pi'' connection. We illustrate how the Meta-Pi paradigm implements a dynamically adaptive Bayesian connectionist classifier. We demonstrate the Meta-Pi architecture's performance in the context of multi-speaker phoneme recognition. In this task the Meta-Pi superstructure integrates TDNN sub-networks to perform multi-speaker phoneme recognition at speaker-DEpendent rates. It achieves a 6-speaker (4 males, 2 females) recognition rate of 98.4% on a database of voiced-stops (/b,d,g/). This recognition performance represents a significant improvement over the 95.9% multi-speaker recognition rate obtained by a single TDNN trained in multi-speaker fashion. It also approaches the 98.7% average of the speaker-DEpendent recognition rates for the six speakers processed. We show that the Meta-Pi network can learn --- without direct supervision --- to recognize the speech of one particular speaker using a dynamic combination of internal models of *other* speakers exclusively (99.8% correct). The Meta-Pi model constitutes a viable basis for connectionist pattern recognition systems that can rapidly adapt to new stimuli by using dynamic, conditional combinations of existing stimulus-specific models. Additionally, it demonstrates a number of performance characteristics that would be desirable in autonomous connectionist pattern recognition systems that could develop and maintain their own database of stimuli models, adapting to new stimuli when possible, spawning new stimulus-specific learning processes when necessary, and eliminating redundant or obsolete stimulus-specific models when appropriate. This research has been funded by Bell Communications Research, ATR Interpreting Telephony Research Laboratories, and the National Science Foundation (NSF grant EET-8716324). REQUESTS: Please send requests for tech. report CMU-CS-89-166 to hamps at speech2.cs.cmu.edu (ARPAnet) "Allow 4 weeks for delivery..." **************** PLEASE DO NOT FORWARD TO OTHER MAILING LISTS **************** From pollack at cis.ohio-state.edu Wed Oct 4 00:02:32 1989 From: pollack at cis.ohio-state.edu (Jordan B Pollack) Date: Wed, 4 Oct 89 00:02:32 EDT Subject: Neurogammon wins Computer Olympiad In-Reply-To: Gerald Tesauro's message of 3 Oct 89 13:26:58 EDT <100389.132658.tesauro@ibm.com> Message-ID: <8910040402.AA03037@toto.cis.ohio-state.edu> Congratulations! It even beat Berliner's old SNIC(?) game? From mdtom at en.ecn.purdue.edu Wed Oct 4 17:39:34 1989 From: mdtom at en.ecn.purdue.edu (M Daniel Tom) Date: Wed, 04 Oct 89 16:39:34 EST Subject: TR-EE 89-54: Analyzing NETtalk for Speech Development Modelling Message-ID: <8910042139.AA19723@en.ecn.purdue.edu> ---------------------------------------------------------------------- Requests from within US, Canada, and Mexico: The technical report with figures, and cluster plots, have been placed in the account kindly provided by Ohio State. Here is the instructions to get the files: unix> ftp cheops.cis.ohio-state.edu (or, ftp 128.146.8.62) Name: anonymous Password: neuron ftp> cd pub/neuroprose ftp> mget tenorio.* (type y and hit return) ftp> quit unix> uncompress tenorio.*.Z unix> lpr -P(your_postscript_printer) tenorio.speech_dev.ps unix> lpr -P(your_132_column_printer) tenorio.cluster.plain ---------------------------------------------------------------------- Requests from outside North America: The technical report is available at a cost of US$8.38 per copy, postage included. Please make checks payable to Purdue University in US dollars. You may send your requests, checks, and full first class mail address to: J. L. Dixon School of Electrical Engineering Purdue University West Lafayette, Indiana 47907 USA Please mention the technical report number: TR-EE 89-54. Please also note that the hard copy of the technical report does not include cluster plots mentioned above. ---------------------------------------------------------------------- Adaptive Networks as a Model for Human Speech Development M. Fernando Tenorio M. Daniel Tom School of Electrical Engineering and Richard G. Schwartz Department of Audiology and Speech Sciences Parallel Distributed Structures Laboratory Purdue University West Lafayette, IN 47907 TR-EE 89-54 August 1989 Abstract Unrestricted English text can be converted to speech through the use of a look up table, or through a parallel feedforward network of deterministic processing units. Here, we reproduce the network structure used in NETtalk. Several experiments are carried out to determine which characteristics of the network are responsible for which learning behavior, and how closely that maps into human speech development. The network is trained with different levels of speech complexity (children and adult speech,) and with Spanish a second language. Developmental analyses are performed on networks separately trained with children speech, adult speech, and Spanish. Analyses on second mapping training are performed on a network trained with Spanish as a second language, and on another network trained with English as a second language. Cluster analyses of the hidden layer units of networks having different first and second language mappings reveal that the final mapping and the convergence process depend a lot on the training data. The results are shown to be highly dependent on statistical characteristics of the input. From mehra at aquinas.csl.uiuc.edu Thu Oct 5 00:24:11 1989 From: mehra at aquinas.csl.uiuc.edu (Pankaj Mehra) Date: Wed, 4 Oct 89 23:24:11 CDT Subject: help with addresses Message-ID: <8910050424.AA28363@elaine> I am looking for the address (e-mail or otherwise) of Chris Watkins, Ph.D. from Cambridge University, who has done some work on extensions of Sutton's temporal difference methods. I shall appreciate any pointers to his published papers as well. After his talk at IJCAI this year, Gerald Edelman mentioned that Fuster (sp?) has been doing some work on learning with delayed feedback. The literature in psychology on this subject is confusing (at least to an outsider). Whereas delays in feedback are bad for skill learning, their effect on "intelligent" tasks is just the contrary. I shall appreciate any references to the papers of Fuster and to other recent work in the area of reinforcement learning with delayed feedback. Pankaj Mehra e-mail: mehra at cs.uiuc.edu *** Please do not send replies to the entire mailing list. From srh at flash.bellcore.com Thu Oct 5 11:27:01 1989 From: srh at flash.bellcore.com (stevan r harnad) Date: Thu, 5 Oct 89 11:27:01 EDT Subject: Parallelism, Real vs. Simulated: A Query Message-ID: <8910051527.AA28565@flash.bellcore.com> I have a simple question: What capabilities of PDP systems do and do not depend on the net's actually being implemented in parallel, rather than just being serially simulated? Is it only speed and capacity parameters, or something more? Stevan Harnad Psychology Department Princeton University harnad at confidence.princeton.edu From skrzypek at CS.UCLA.EDU Thu Oct 5 15:59:44 1989 From: skrzypek at CS.UCLA.EDU (Dr. Josef Skrzypek) Date: Thu, 5 Oct 89 12:59:44 PDT Subject: Parallelism, Real vs. Simulated: A Query In-Reply-To: stevan r harnad's message of Thu, 5 Oct 89 11:27:01 EDT <8910051527.AA28565@flash.bellcore.com> Message-ID: <8910051959.AA20276@retina.cs.ucla.edu> I have a simple question: What capabilities of PDP systems do and do not depend on the net's actually being implemented in parallel, rather than just being serially simulated? Is it only speed and capacity parameters, or something more? >>>>>>>>>>>>>>>> Good (and dangerous) question. Applicable to Neural Nets in general and not only to PDP. It appears that you can simulate anything that you wish. In principle you trade computation in space for computation in time. If you can make your time-slices small enough and complete all of the necessary computation within each slice there seem to be no reason to have neural networks. In reality, simulation of synchronized, temporal events taking place in a 3D network that allows for feedback pathways is rather cumbersome. From mdtom at en.ecn.purdue.edu Thu Oct 5 17:09:49 1989 From: mdtom at en.ecn.purdue.edu (M Daniel Tom) Date: Thu, 05 Oct 89 16:09:49 EST Subject: TR-EE 89-54 replaced with uncompressed files Message-ID: <8910052109.AA10365@en.ecn.purdue.edu> I have replaced the compressed files with uncompressed ones: tenorio.speech_dev.ps tenorio.cluster.plain in cheops.cis.ohio-state.edu. Please try again. Thanks for your feedback about ftp-ing. I had suspected that the transmission was not perfect when I compared the file sizes with my own. This time the file sizes match. So good luck. Sincerely, M. Daniel Tom From oliver%FSU.BITNET at VMA.CC.CMU.EDU Thu Oct 5 15:44:15 1989 From: oliver%FSU.BITNET at VMA.CC.CMU.EDU (Bill Oliver, Psychology Dept., FSU, 32306) Date: Thu, 5 Oct 89 15:44:15 EDT Subject: mailing list Message-ID: <8910051543320E9.CWDV@RAI.CC.FSU.EDU> (UMass-Mailer 4.04) Please put me on your mailing list. Thanks, Bill Oliver From srh at flash.bellcore.com Thu Oct 5 11:27:01 1989 From: srh at flash.bellcore.com (stevan r harnad) Date: Thu, 5 Oct 89 11:27:01 EDT Subject: Parallelism, Real vs. Simulated: A Query Message-ID: <8910051527.AA28565@flash.bellcore.com> I have a simple question: What capabilities of PDP systems do and do not depend on the net's actually being implemented in parallel, rather than just being serially simulated? Is it only speed and capacity parameters, or something more? Stevan Harnad Psychology Department Princeton University harnad at confidence.princeton.edu From tenorio at ee.ecn.purdue.edu Fri Oct 6 13:48:03 1989 From: tenorio at ee.ecn.purdue.edu (Manoel Fernando Tenorio) Date: Fri, 06 Oct 89 12:48:03 EST Subject: NeuroGammon Message-ID: <8910061748.AA07003@ee.ecn.purdue.edu> Bcc: -------- We need more results like that! Shouted another NN colleague after Tesauro's message, to which I am quick to agree. Like probably most of you out there in email land, I got thrilled with the "new result" that NN claimed. Although my background includes AI, I am not aware of any other similar results in Machine Intelligence. But I have some doubts about what actually are the claims that we can intelligently make for NN with this program. Therefore I am opening it for discussion here. This is in no way to question or detract from the result, but rather to clarify and qualify future statements: NN has own a major victory, unparallel in Machine Intelligence, playing Backgammon. 1. Can we make the claim that we are doing better that AI (chess) efforts, mentioned as one of the AI conerstone results in the Oct88(?) AI magazine (AAAI), since it is a different game. I recall Tesauro mentioning in NIPS that backgammon was heavily pattern based, as opposed to chess. 2. Is anybody aware of results for NN in chess or AI in backgammon? 3. Could AI do better in a heavily pattern based game? 4. Does Tesauro plans some form of rule estimate to compare game complexity? 5. Should we say that NN are good for this game and not for others, but what matters is that what it does is better than the human counterparts (maybe that would mean a closer or better computational model to the human one?) 6. how can one better compare this apples and oranges results? 7. How about future results and past claims such as: NN are better than any other technique because it can solve the EX-OR problem and chaotic time series prediction (parafrased Neruocomputers Sept. 89) I would like to see a reasonable scientific discussion on the subject, because I would like to be prepared to answer to the question that will come after people read this on the papers (Can you imagine what some of the Media will do with this new result? "IBM unravels computer that mimics the brain, but beats any human. It will be call HAL..."). -Check'$', mate... (An aussie salute) --ft. From YEE at cs.umass.EDU Fri Oct 6 15:33:00 1989 From: YEE at cs.umass.EDU (Richard Yee) Date: Fri, 6 Oct 89 14:33 EST Subject: Searle, Harnad and understanding Chinese symbols Message-ID: <8910061834.AA04513@crash.cs.umass.edu> == More Chinese-Room Talk == I find myself in the apparently paradoxical position of agreeing with virtually all of Searle's assertions in the "Chinese Room" argument, and yet disagreeing with his conclusion. Likewise, I agree with Harnad that an intelligent (cognitive?) processing system's input symbols must be "grounded" in subsymbolic representations (in what I call internal semantic or interpretive representations), yet I disagree with his defense of Searle's counter to the "Systems-Reply". What follows is a rather long message which, I claim, demonstrates that the conclusion that Chinese is NOT being understood inside the Chinese Room has no basis. This rescues us from having to conclude that either understanding is uncomputable or the Church-Turing Thesis is wrong. The reason that the possibility remains open as to whether the inputs (Chinese characters) are being understood, is basically related to the Systems Reply with one important caveat: there is also no basis to the claim that the Chinese characters ARE being understood, and to the extent that the Systems Reply claims this I would not defend it. The question is whether the Chinese language performance (CLP) being observed externally arises from an understanding of Chinese (UC) or from some other process (not UC); the Chinese Room scenario does not present enough information to decide the question either way. Harnad describes the crux of Searle's argument against the Systems Reply as being that the person in the room is "in fact performing ALL the FUNCTIONS of the system" (possibly through having learned the rules), and yet clearly the person does not understand Chinese. Both of these statements are true, but this does not justify the conclusion that the process of understanding Chinese is not occurring. The determination of outputs is under the complete control of the rules, not the person. The person has no free will on this point (as he does in answering English inputs). All and only those transformations licensed by the rules will determine what the Chinese outputs will be. Thus, although it is clearly true that, e.g., the input symbols (Chinese characters), "HAN-BAO-BAO", have form but absolutely no content for the person, this in no way implies the the symbols' content will not be recognized and play a role in determining the Chinese output because this is in no way dependent upon the person's knowledge of the symbols. All that matters with regard to the person is his knowledge of how to correctly follow the rules. Whether or not the *content* of this symbols is recognized, is determined by the rules, and we simply have no basis for concluding either way. So, while the person hasn't the slightest idea whether it would be better to eat, run away from, or marry a "HAN-BAO-BAO", this knowledge may well be determined through the application of the rules, and, in such a case, they could dictate that an output be produced that takes account of this recognition. The output might well say, for example, that these things are found at McDonald's, but it would be surprising in the extreme to consider spending the rest of one's life with one. The person in the room is completely oblivious to this distinction, and yet the Chinese symbols were indeed correctly recognized for their CONTENT, and this happened WITHIN the room. On the other hand, it need not be the case that the meaning of the symbols is determined at any point. The same output could be produced by a very different process (it is hard to imagine, though, how the illusion could be maintained). Thus, I agree that looking at the I/O behavior outside of the room is not sufficient to determine if the input symbols are being understood (mapped to their meanings), or if, instead, they are treated as objects having form but no content (or treated some other way for that matter). The argument that there is no understanding of Chinese because the *person* never understands the input symbols appears to be based on a failure to distinguish between a generic Turing Machine (TM) and one that is programmable, a Universal Turing Machine (UTM). A UTM, U, is a TM that is given as input the program of another TM, T, and *its* input, x. The UTM computes a function which is itself the computation of a function of the input x. Thus, the UTM *does not* compute y = T(x); it computes y = U(T, x). If T, as a parameter of U, is held constant, then y = T(x) = U(x), but this still doesn't mean that U "experiences x" the same way T does. U merely acts as an intermediary that enables T to process x; when T is done, U returns T's result as if it were his own doing. The rules that the person is following are, in fact, a program for Chinese I/O (CLP). The person is acting as a UTM. He is following *his own set of rules* that tell him what to do with the rules and inputs that he receives. Thus, the person's program is to execute the rules' program on the inputs. It is little wonder that the person may not treat the input symbols in the same way that the rules treat them. The real question that should be asked is NOT whether the person, in following the rules, understands the Chinese characters, UC, (clearly he does not), but whether the person would understand the Chinese characters if HIS NEURONS were the ones implementing the rules and he were experiencing the results. In other words, the rules may or may not DESCRIBE A PROCESS sufficient for figuring out what the Chinese characters mean. ("UC, or not UC", do you see?) If Searle's and Harnad's arguments were correct, then one would be lead, as they seem to be, to the conclusion that a Turing Machine alone is not sufficient to produce understanding, in particular the understanding of Chinese. This would amount to the the claim that either, A. Understanding is not computable, i.e., it is not achievable through anything that could be considered an algorithm, a procedure, or any finitely describable method, =or= B. The Church-Turing Thesis is wrong. For what it is worth, I don't agree with either position. There is absolutely no reason to be persuaded that (B) is true, and I take my own understanding of English (and a little Chinese) as an existence proof that (A) is false. Richard Yee (yee at cs.umass.edu) From cole at cse.ogc.edu Fri Oct 6 15:41:40 1989 From: cole at cse.ogc.edu (Ron Cole) Date: Fri, 6 Oct 89 12:41:40 -0700 Subject: Parallelism, Real vs. Simulated: A Query Message-ID: <8910061941.AA07943@cse.ogc.edu> Massively parallel networks are likely to reveal emergent properties that cannot be predicted from serial simulations. Asking what properties these networks will have before they exist is like asking what we will see when we have more powerful telescopes. From osborn%rana.usc.edu at usc.edu Fri Oct 6 16:59:50 1989 From: osborn%rana.usc.edu at usc.edu (Tom Osborn) Date: Fri, 6 Oct 89 13:59:50 PDT Subject: No subject Message-ID: <8910062059.AA01270@rana.usc.edu> Steve Harnad asks: > I have a simple question: What capabilities of PDP systems do and > do not depend on the net's actually being implemented in parallel, > rather than just being serially simulated? Is it only speed and > capacity parameters, or something more? An alternative question to ask is: What differences does synchronous vs asynchronous processing make? Both may be _implemented_ in on serial or parallel machines - synch on serial by keeping old state vectors, synch on parallel by using some kind of lock-step control (with associated costs), asynch on serial by adding a stochastic model of unit/neuron processing, asynch on parallel - trivial. The _importance_ of of synch vs asynch is apparent for Hopfield/Little nets and Boltzmann machines: For Hopfield (utilising asynch processing, with random selection of one unit at a time and full connectivity), you get one Energy (Liapunov) function. BUT for Little nets (utilising synch processing - entire new state vector computed from the old one), you have a different but related Energy function. These two Energy function have the same stationary points, but the dynamics differ. [I can't comment on performance implications]. For Boltzmann machines, three different regimes may apply (if not all units are connected). The same two as above (with different dynamics) and I recall that there is no general convergence proof for the full synch case. Another parallel regime (ie, synch) updating where sets of neuron (no two directly connected) are processed together - dynamically, this corresponds exactly to asynch updating, but with linear performance scaling on parallel machines (assuming the partitioning problem was done ahead of time). Answers to the question for back-prop are more diverse: To maintain equivalence with asynch processing, Parallel implementations may synch process _layers_ at a time, or a pipeline effect may be set up, or the data may be managed to optimise some measure of performance (eg, for learning or info processing). HOWEVER, there _must_ be synchronisation between the computed and desired output values for back-prop learning to work (to compute the delta). Someone else should comment. Tom Osborn On Sabbatical Leave (till Jan '90) at: Center for Neural Engineering, University of Southern California Los Angeles CA 90089-0782 'Permanently', University of Technology, Sydney. From Scott.Fahlman at B.GP.CS.CMU.EDU Fri Oct 6 16:56:21 1989 From: Scott.Fahlman at B.GP.CS.CMU.EDU (Scott.Fahlman@B.GP.CS.CMU.EDU) Date: Fri, 06 Oct 89 16:56:21 EDT Subject: NeuroGammon In-Reply-To: Your message of Fri, 06 Oct 89 12:48:03 -0500. <8910061748.AA07003@ee.ecn.purdue.edu> Message-ID: 1. Can we make the claim that we are doing better that AI (chess) efforts, mentioned as one of the AI conerstone results in the Oct88(?) AI magazine (AAAI), since it is a different game. I recall Tesauro mentioning in NIPS that backgammon was heavily pattern based, as opposed to chess. 2. Is anybody aware of results for NN in chess or AI in backgammon? I agree that comparing neural-net backgammon to conventional AI chess programs would be pretty meaningless. But there are a number of conventional AI programs that play backgammon. The most famous of these, Hans Berliner's program, once beat the human world champion in backgammon. It had some lucky rolls, but then a good backgammon player willattempt to keep the board in a state where most rolls are "lucky" and where "unlucky" rolls can't do too much harm. Unfortunately, Berliner's program wasn't in the tournament that NeuroGammon won, but several other AI-type programs were there. Maybe Gerry can give us some estimate of whether these programs were in the same class as Berliner's program. 3. Could AI do better in a heavily pattern based game? Depends what you mean by "pattern-based". Backgammon is all patterns, but it also has the interesting feature of uncertainty; chess and Go are deterministic. Chess can be played using a lot of knowledge and a little search or vice-versa. People tend to use a lot of knowledge, but the current trend in computer chess is toward very search-intensive chess machines that have little in common with other areas of AI: no complex knowledge representations, no symbolic learning, etc. If this trend continues, it will mean that chess is no longer a good problem for driving progress in AI, though it will help to stimulate the development of parallel search engines. I think that Go is going to turn out to be the really interesting game for neural nets to tackle, since the search space is more intractable than the search space in chess, and since patterns of pieces influence the choice of move in subtle ways that master players cannot easily explain. There is still an important element of serial search, however -- I don't think even the masters claim to select every move by "feel" alone. 4. Does Tesauro plans some form of rule estimate to compare game complexity? The size of the rule set has very little to do with the strategic complexity of a game. Monopoly has a more complex rule set than Go, but is MUCH easier to play well. 6. how can one better compare this apples and oranges results? Try not to. -- Scott From Alex.Waibel at SPEECH2.CS.CMU.EDU Fri Oct 6 21:19:38 1989 From: Alex.Waibel at SPEECH2.CS.CMU.EDU (Alex.Waibel@SPEECH2.CS.CMU.EDU) Date: Fri, 6 Oct 89 21:19:38 EDT Subject: NIPS'89 Postconference Workshops Message-ID: Below are the preliminary program and brief descriptions of the workshop topics covered during this years NIPS-Postconference Workshops to be held in Keystone from November 30 through December 2 (right following the NIPS conference). Please register for both conference and Workshops using the general NIPS conference registration forms. With it, please indicate which workshop topic below you may be most interested in attending. Your preferences are in no way binding or limiting you to any particular workshop but will help us in allocating suitable meeting rooms and scheduling workshop sessions in an optimal way. For your convenience, you may simply include a copy of the form below with your registration material marking it for your three most prefered workshop choices in order of preference (1,2 and 3). For registration information (both NIPS conference as well as Postconference Workshops), please contact the Local Arrangements Chair, Kathie Hibbard, by sending email to hibbard at boulder.colorado.edu, or by writing to: Kathie Hibbard NIPS '89 University of Colorado Campus Box 425 Boulder, Colorado 80309-0425 For technical questions relating to individual conference workshops, please contact the individual workshop leaders listed below. Please feel free to contact me with any questions you may have about the workshops in general. See you in Denver/Keystone, Alex Waibel NIPS Workshop Program Chairman School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 412-268-7676, waibel at cs.cmu.edu ================================================================ ____________________________________________________________ ! POST CONFERENCE WORKSHOPS AT KEYSTONE ! ! THURSDAY, NOVEMBER 30 - SATURDAY, DECEMBER 2, 1989 ! !____________________________________________________________! Thursday, November 30, 1989 5:00 PM: Registration and Reception at Keystone Friday, December 1, 1989 7:30 - 9:30 AM: Small Group Workshops 4:30 - 6:30 PM: Small Group Workshops 7:30 - 10:30 PM: Banquet and Plenary Discussion Saturday, December 2, 1989 7:30 - 9:30 AM: Small Group Workshops 4:30 - 6:30 PM: Small Group Workshops 6:30 - 7:15 PM: Plenary Discussion, Summaries 7:30 - 11:00 PM: Fondue Dinner, MountainTop Restaurant ================================================================ PLEASE MARK YOUR PREFERENCES (1,2,3) AND ENCLOSE WITH REGISTRATION MATERIAL: ----------------------------------------------------------------------------- ______1. LEARNING THEORY: STATISTICAL ANALYSIS OR VC DIMENSION? ______2. STATISTICAL INFERENCE IN NEURAL NETWORK MODELLING ______3. NEURAL NETWORKS AND GENETIC ALGORITHMS ______4. VLSI NEURAL NETWORKS: CURRENT MILESTONES AND FUTURE HORIZONS ______5. APPLICATION OF NEURAL NETWORK PROCESSING TECHNIQUES TO REAL WORLD MACHINE VISION PROBLEMS ______6. IMPLEMENTATIONS OF NEURAL NETWORKS ON DIGITAL, MASSIVELY PARALLEL COMPUTERS ______7. LARGE, FAST, INTEGRATED SYSTEMS BASED ON ASSOCIATIVE MEMORIES ______8. NEURAL NETWORKS FOR SEQUENTIAL PROCESSING WITH APPLICATIONS IN SPEECH RECOGNITION ______9. LEARNING FROM NEURONS THAT LEARN ______10. NEURAL NETWORKS AND OPTIMIZATION PROBLEMS 11. (withdrawn) ______12. NETWORK DYNAMICS ______13. ARE REAL NEURONS HIGHER ORDER NETS? ______14. NEURAL NETWORK LEARNING: MOVING FROM BLACK ART TO A PRACTICAL TECHNOLOGY ______15. OTHERS ?? __________________________________________________ 1. LEARNING THEORY: STATISTICAL ANALYSIS OR VC DIMENSION? Sara A. Solla AT&T Bell Laboratories Crawford Corner Road Holmdel, NJ 07733-1988 Phone: (201) 949-6057 E-mail: solla at homxb.att.com Recent success at describing the process of learning in layered neural networks and the resulting generalization ability has emerged from two different approaches. Work based on the concept of VC dimension emphasizes the connection between learning and statistical inference in order to analyze questions of bias and variance. The statistical approach uses an ensemble description to focus on the prediction of network performance for a specific task. Participants interested in learning theory are invited to discuss the differences and similarities between the two approaches, the mathematical relation between them, and their respective range of applicability. Specific questions to be discussed include comparison of predictions for required training set sizes, for the distribution of generalization abilities, for the probability of obtaining good performance with a training set of fixed size, and for estimates of problem complexity applicable to the determination of learning times. 2. STATISTICAL INFERENCE IN NEURAL NETWORK MODELLING Workshop Chair: Richard Golden Stanford University Psychology Department Stanford, CA 94305 (415) 725-2456 E-mail: golden at psych.stanford.edu This workshop is designed to show how the theory of statistical inference is directly applicable to some difficult neural network modelling problems. The format will be tutorial in nature (85% informal lecture, 15% discussion). Topics to be discussed include: obtaining probability distributions for neural networks, interpretation and derivation of optimal learning cost functions, evaluating the generalization performance of networks, asymptotic sampling distributions of network weights, statistical mechanics calculation of learning curves in some simple examples, statistical tests for comparing internal representations and deciding which input units are relevant to the prediction task. Dr. Naftali Tishby (AT&T Bell Labs) and Professor Halbert White (UCSD Economics Department) are the invited experts. 3. Title: NEURAL NETWORKS AND GENETIC ALGORITHMS Organizers: Lawrence Davis (Bolt Beranek and Newman, Inc.) Michael Rudnick (Oregon Graduate Center) Description: Genetic algorithms have many interesting relationships with neural networks. Recently, a number of researchers have investigated some of these relationships. This workshop will be the first forum bringing those researchers together to discuss the current and future directions of their work. The workshop will last one day and will have three parts. First, a tutorial on genetic algorithms will be given, to ground those unfamiliar with the technology. Second, seven researchers will summarize their results. Finally there will be an open discussion on the topics raised in the workshop. We expect that anyone familiar with neural network technology will be comfortable with the content and level of discussion in this workshop. 4. VLSI NEURAL NETWORKS: CURRENT MILESTONES AND FUTURE HORIZONS Moderators: Joshua Alspector and Daniel B. Schwartz Bell Communications Research GTE Laboratories, Inc. 445 South Street 40 Sylvan Road Morristown, NJ 07960-19910 Waltham, MA 02254 (201) 829-4342 (617) 466-2414 e-mail: josh at bellcore.com e-mail: dbs%gte.com at relay.cs.net This workshop will explore the areas of applicability of neural network implementations in VLSI. Several speakers will discuss their present implementations and speculate about where their work may lead. Workshop attendees will then be encouraged to organize working groups to address several issues which will be raised in connection with the presentations. Although it is difficult to predict which issues will be selected, some examples might be: 1) Analog vs. digital implementations. 2) Limits to VLSI complexity for neural networks. 3) Algorithms suitable for VLSI architectures. The working groups will then report results which will be included in the workshop summary. 5. APPLICATION OF NEURAL NETWORK PROCESSING TECHNIQUES TO REAL WORLD MACHINE VISION PROBLEMS Paul J. Kolodzy (617) 981-3822 kolodzy at ll.ll.mit.edu Murali M. Menon (617) 981-5374 This workshop will discuss the application of neural networks to vision applications, including image restoration and pattern recognition. Participants will be asked to present their specific application for discussion to highlight the relevant issues. Examples of such issues include, but are not limited to, the use of deterministic versus stochastic search procedures for neural network processing, using networks to extract shape, scale and texture information for recognition and using network mapping techniques to increase data separability. The discussions will be driven by actual applications with an emphasis on the advantages of using neural networks at the system level in addition to the individual processing steps. The workshop will attempt to cover a wide breadth of network architectures and invites participation from researchers in machine vision, neural network modeling, pattern recognition and biological vision. 6. IMPLEMENTATIONS OF NEURAL NETWORKS ON DIGITAL, MASSIVELY PARALLEL COMPUTERS Dr. K. Wojtek Przytula and Prof. S.Y. Kung Hughes Research Laboratories, RL 69 3011 Malibu Cyn. Road Malibu, CA 90265 Phone: (213) 317-5892 E-mail: wojtek%csfvax at hac2arpa.hac.com Implementations of neural networks span a full spectrum from software realizations on general-purpose computers to strictly special-purpose hardware realizations. Implementations on programmable, parallel machines, which are to be discussed during the workshop, constitute a compromise between the two extremes. The architectures of programmable parallel machines reflect the structure of neural network models better than those of sequential machines, thus resulting in higher processing speed. The programmability provides more flexibility than is available in specialized hardware implementations and opens a way for realization of various models on a single machine. The issues to be discussed include: mapping neural network models onto existing parallel machines, design of specialized programmable parallel machines for neural networks, evaluation of performance of parallel machines for neural networks, uniform characterization of the computational requirements of various neural network models from the point of view of parallel implementations. 7. LARGE, FAST, INTEGRATED SYSTEMS BASED ON ASSOCIATIVE MEMORIES Michael R. Raugh Director of Learning Systems Division Research Institute for Advanced Computer Science (RIACS) NASA Ames Research Center, MS 230-5 Moffett Field, CA 94035 e-mail: raugh at riacs.edu Phone: (415) 694-4998 This workshop will address issues in the construction of large systems that have thousands or even millions of hidden units. It will present and discuss alternatives to backpropagation that allow large systems to learn rapidly. Examples from image analysis, weather prediction, and speech transcription will be discussed. The focus on backpropagation with its slow learning has kept researchers from considering such large systems. Sparse distributed memory and related associative-memory structures provide an alternative that can learn, interpolate, and abstract, and can do so rapidly. The workshop is open to everyone, with special encouragement to those working in learning, time-dependent networks, and generalization. 8. NEURAL NETWORKS FOR SEQUENTIAL PROCESSING WITH APPLICATIONS IN SPEECH RECOGNITION Herve Bourlard Philips Research Laboratory Brussels Av. Van Becelaere 2, Box 8 B-1170 Brussels, Belgium Phone: 011-32-2-674-22-74 e-mail address: bourlard at prlb.philips.be or: prlb2!bourlard at uunet.uu.net Speech recognition must contend with the statistical and sequential nature of the human speech production system. Hidden Markov Models (HMM) provide a powerful method to cope with both of these, and their use made a breakthrough in speech recognition. On the other hand, neural networks have recently been recognized as an alternative tool for pattern recognition problems such as speech recognition. Their main useful properties are their discriminative power and their capability to deal with non-explicit knowledge. However, the sequential aspect remains difficult to handle in connectionist models. If connections are supplied with delays, feedback loops can be added providing dynamic and implicit memory. However, in the framework of continuous speech recognition, it is still difficult to use only neural networks for the segmentation and recognition of a sentence into a sequence of speech units, which is efficiently solved in the HMM approach by the well known ``Dynamic Time Warping'' algorithm. This workshop should be the opportunity for reviewing neural network architectures which are potentially able to deal with sequential and stochastic inputs. It should also be discussed to which extent the different architectures can be useful in recognizing isolated units (phonemes, words, ...) or continuous speech. Amongst others, we should consider spatiotemporal models, time-delayed neural networks (Waibel, Sejnowsky), temporal flow models (Watrous), hidden-to-input (Elman) or output-to-input (Jordan) recurrent models, focused back-propagation networks (Mozer) or hybrid approaches mixing neural networks and standard sequence matching techniques (Sakoe, Bourlard). 9. LEARNING FROM NEURONS THAT LEARN Moderated by Thomas P. Vogl Environmental Research Institute of Michigan 1501 Wilson Blvd. Arlington, VA 22209 Phone: (703) 528-5250 E-mail: TVP%nihcu.bitnet at cunyvm.cuny.edu FAX: (703) 524-3527 In furthering our understanding of artificial and biological neural systems, the insights that can be gained from the perceptions of those trained in other disciplines can be particularly fruitful. Computer scientists, biophysicists, engineers, psychologists, physicists, and neurobiologists tend to have different perspectives and conceptions of the mechanisms and components of "neural networks" and to weigh differently their relative importance. The insights obvious to practitioners of one of these disciplines are often far from obvious to those trained in another, and therefore may be especially relevant to the solutions of ornery problems. The workshop provides a forum for the interdisciplinary discussion of biological and artificial networks and neurons and their behavior. Informal group discussion of ongoing research, novel ideas, approaches, comparisons, and the sharing of insights will be emphasized. The specific topics to be considered and the depth of the analysis/discussion devoted to any topic will be determined by the interest and enthusiasm of the participants as the discussion develops. Participants are encouraged to consider potential topics in advance, and to present them informally but succinctly (under five minutes) at the beginning of the workshop. 10. NEURAL NETWORKS AND OPTIMIZATION PROBLEMS ---------------------------------------- Prof. Carsten Peterson University of Lund Dept. of Theoretical Physics Solvegatan 14A S-223 62 Lund Sweden phone: 011-46-46-109002 bitnet: THEPCAP%SELDC52 Workshop description: The purpose of the workshop is twofold; to establish the present state of the art and to generate novel ideas. With respect to the former, firm answers to the following questions should emerge: (1). Does the Hopfield- Tank approach or variants thereof really work with respect to quality, reliability, parameter insensitivity and scalability? (2). If this is the case, how does it compare with other cellular approaches like "elastic snake" and genetic algorithms? Novel ideas should focus on new encoding schemes and new application areas (in particular, scheduling problems). Also, if time allows, optimization of neural network learning architectures will be covered. People interested in participating are encouraged to communicate their interests and expertise to the chairman via e-mail. This would facilitate the planning. 12. Title: NETWORK DYNAMICS Chair: Richard Rohwer Centre for Speech Technology Research Edinburgh University 80, South Bridge Edinburgh EH1 1HN, Scotland Phone: (44 or 0) (31) 225-8883 x280 e-mail: rr%uk.ac.ed.eusip at nsfnet-relay.ac.uk Summary: This workshop will be an attempt to gather and improve our knowledge about the time dimension of the activation patterns produced by real and model neural networks. This broad subject includes the description, interpretation and design of these temporal patterns. For example, methods from dynamical systems theory have been used to describe the dynamics of network models and real brains. The design problem is being approached using dynamical training algorithms. Perhaps the most important but least understood problems concern the cognitive and computational significance of these patterns. The workshop aims to summarize the methods and results of researchers from all relevant disciplines, and to draw on their diverse insights in order to frame incisive, approachable questions for future research into network dynamics. Richard Rohwer JANET: rr at uk.ac.ed.eusip Centre for Speech Technology Research ARPA: rr%uk.ac.ed.eusip at nsfnet-relay.ac.uk Edinburgh University BITNET: rr at eusip.ed.ac.uk, 80, South Bridge rr%eusip.ed.UKACRL Edinburgh EH1 1HN, Scotland UUCP: ...!{seismo,decvax,ihnp4} !mcvax!ukc!eusip!rr PHONE: (44 or 0) (31) 225-8883 x280 FAX: (44 or 0) (31) 226-2730 13. ARE REAL NEURONS HIGHER ORDER NETS? Most existing artificial neural networks have processing elements which are computationally much simpler than real neurons. One approach to enhancing the computational capacity of artificial neural networks is to simply scale up the number of processing elements, but there are limits to this. An alternative is to build modules or subnets and link these modules in a larger net. Several groups of investigators have begun to analyze the computational abilities of real single neurons in terms of equivalent neural nets, in particular higher order nets, in which the inputs explicitly interact (eg. sigma-pi units). This workshop would introduce participants to the results of these efforts, and examine the advantages and problems of applying these complex processors in larger networks. Dr. Thomas McKenna Office of Naval Research Div. Cognitive and Neural Sciences Code 1142 Biological Intelligence 800 N. Quincy St. Arlington, VA 22217-5000 phone:202-696-4503 email: mckenna at nprdc.arpa mckenna at nprdc.navy.mil 14. NEURAL NETWORK LEARNING: MOVING FROM BLACK ART TO A PRACTICAL TECHNOLOGY Scott E. Fahlman School of Computer Science Carnegie-Mellon University Pittsburgh, PA 15213 Internet: fahlman at cs.cmu.edu Phone: (412) 268-2575 There are a number of competing algorithms for neural network learning, all rather new and poorly understood. Where theory is lacking, a reliable technology can be built on shared experience, but it usually takes a long time for this experience to accumulate and propagate through the community. Currently, each research group has its own bag of tricks and its own body of folklore about how to attack certain kinds of learning tasks and how to diagnose the problem when things go wrong. Even when groups are willing to share their hard-won experience with others, this can be hard to accomplish. This workshop will bring together experienced users of back-propagation and other neural net learning algorithms, along with some interested novices, to compare views on questions like the following: I. Which algorithms and variations work best for various classes of problems? Can we come up with some diagnostic features that tell us what techniques to try? Can we predict how hard a given problem will be? II. Given a problem, how do we go about choosing the parameters for various algorithms? How do we choose what size and shape of network to try? If our first attempt fails, are there symptoms that can tell us what to try next? III. What can we do to bring more coherence into this body of folklore, and facilitate communication of this informal kind of knowledge? An online collection of standard benchmarks and public-domain programs is one idea, already implemented at CMU. How can we improve this, and what other ideas do we have? From rik%cs at ucsd.edu Fri Oct 6 17:18:42 1989 From: rik%cs at ucsd.edu (Rik Belew) Date: Fri, 6 Oct 89 14:18:42 PDT Subject: Neurogammon wins Computer Olympiad Message-ID: <8910062118.AA16164@roland.UCSD.EDU> From TESAURO at ibm.com Wed Oct 4 00:35:31 1989 To: connectionists at CS.CMU.EDU Subject: Neurogammon wins Computer Olympiad ... This is a victory not only for neural networks, but for the entire machine learning community, as it is apparently the first time in the history of computer games that a learning program has ever won a tournament. Congratulations Gerry! And I think your highlighting the win by a *learning* (v. special-purpose programmed) solution is appropriate. But as to being first, don't you think Arthur Samuel's checker player gets the distinction? At least when you think about the antique hardware/software 'environment' he had to use? Rik Belew From harnad at clarity.Princeton.EDU Sat Oct 7 13:07:51 1989 From: harnad at clarity.Princeton.EDU (Stevan Harnad) Date: Sat, 7 Oct 89 13:07:51 EDT Subject: Symbol Manipulation: Scope and Limits Message-ID: <8910071707.AA15771@psycho.Princeton.EDU> Since Richard Yee has let the cat out of the bag with his posting (I was hoping for more replies about whether the community considered there to be any essential difference between parallel implementations and serial simulations of neural nets before I revealed why I had posted my query): I've proposed a variant of Searle's Chinese Room Argument (which in its original form I take to be decisive in showing that you can't implement a mind with a just a pure symbol manipulating system) to which nets are vulnerable only if there is no essential difference between a serial simulation and a parallel implementation. That having been said, the variant is obvious, and I leave it to you as an exercise. Here's my reply to Yee, who wrote: > The real question that should be asked is NOT whether [Searle], in > following the rules, understands the Chinese characters, (clearly > he does not), but whether [Searle] would understand the Chinese > characters if HIS NEURONS were the ones implementing the rules and he > were experiencing the results. In other words, the rules may or may not > DESCRIBE A PROCESS sufficient for figuring out what the Chinese > characters mean. This may be the real question, but it's not the one Searle's answering in the negative. In the Chinese room there's only symbol manipulation going on. No person or "system" -- no "subject" -- is understanding. This means symbol manipulation alone is not sufficient to IMPLEMENT the process of understanding, any more than it can implement the process of flying. Now whether it can DESCRIBE rather than implement it is an entirely different question. I happen to see no reason why all features of a process that was sufficient to implement understanding (in neurons, say) or flying (in real airplane parts) couldn't be successfully described and pretested through symbolic simulation. But Searle has simply shown that pure symbol manipulation ITSELF cannot be the process that will successfully implement understanding (or flying). (Ditto now for PDP systems, if parallel implementations and serial simulations are equivalent or equipotent.) > I agree that looking at the I/O behavior outside of the room is not > sufficient... This seems to give up the Turing Test (Searle would shout "Bravo!"). But now Yee seems to do an about-face in the direction of resurrecting the strained efforts of the AI community to show that formal symbol manipulating rule systems have not just form but content after all, and CAN understand: > The determination of outputs is under the complete control of the > rules, not [Searle]. [Searle] has no free will on this point (as he > does in answering English inputs)... although it is clearly true that > (Chinese characters) have form but absolutely no content for the > person... [w]hether or not the *content* of this symbol is recognized, > is determined by the rules... the Chinese symbols were indeed correctly > recognized for their CONTENT, and this happened WITHIN the room... > the process of understanding Chinese is [indeed] occurring. NB: No longer described or simulated, as above, but actually OCCURRING. I ask only: Where/what are these putative contents (I see only formal symbols); and who/what is the subject of this putative understanding (I see only Searle), and would he/she/it care to join in this discussion? Now in my case this glibness is really a reflection of my belief that the Turing Test couldn't be successfully passed by a pure symbol manipulator in the first place (and hence that this whole sci-fi scenario is just a counterfactual fantasy) because of the symbol grounding problem. But Yee -- though skeptical about the Turing Test and seemingly acknowledging the simulation/implemetation distinction -- does not seem to be entirely of one mind on this matter... > [The problem is] a failure to distinguish between a generic Turing > Machine (TM) and one that is programmable, a Universal Turing Machine > (UTM)... If T, as a parameter of U, is held constant, then y = T(x) = > U(x), but this still doesn't mean that U "experiences x" the same way T > does. The rules that the person is following are, in fact, a program > for Chinese I/O... I take my own understanding of English (and a little > Chinese) as an existence proof that [Understanding is Computable] "Cogito Ergo Sum T"? -- Descartes would doubt it... I don't know what Yee means by a "T," but if it's just a pure symbol-cruncher, Searle has shown that it does not cogitate (or "experience"). If T's something more than a pure symbol-cruncher, all bets are off, and you've changed the subject. Stevan Harnad References: Searle, J. (1980) Minds, Brains and Programs. Behavioral and Brain Sciences 3: 417 - 457. Harnad, S. (1989) Minds, Machines and Searle. Journal of Experimental and Theoretical Artificial Intelligence 1: 5 - 25. Harnad, S. (1990) The Symbol Grounding Problem. Physica D (in press) From french at cogsci.indiana.edu Fri Oct 6 14:35:19 1989 From: french at cogsci.indiana.edu (Bob French) Date: Fri, 6 Oct 89 13:35:19 EST Subject: Tech Report "Towards a Cognitive Connectionism" available Message-ID: The following Tech Report is now available. It is scheduled to appear in the January 1990 issue of AI and Society: ACTIVE SYMBOLS AND INTERNAL MODELS: TOWARDS A COGNITIVE CONNECTIONISM by Stephen Kaplan, Mark Weaver and Robert M. French In this paper, we examine some recent criticisms of connectionist models. In the first section, we address the argument that connectionist models are fundamentally behaviorist in nature and, therefore, incapable of genuine cognition. We conclude that these criticisms are indeed valid, but apply only to the currently popular class of feed-forward connectionist networks. To have any hope of avoiding the charge of behaviorism, and ultimately to support full cognition, connectionist architectures must be capable of producing persistent internal states. We discuss the crucial notion of "active symbols" -- semi-autonomous representations -- as a basis for such a cognitive connectionist architecture. Active symbols arise, we argue, out of recurrent circuits, the connectionist implementation of Hebbian cell assemblies. Recurrent architectures have become more prominent in the past year. However, most researchers investigating recurrent architectures seem to view recurrent circuitry merely as an "improved back-prop" for handling time-sequencing. Our view, that the recurrent circuit must be the fundamental building block of any cognitive connectionist architecture, represents a philosophical departure from current thought. In the final section we speculate on the potentials and limits of an associationist architecture. In particular, we examine how the this type of architecture might be able to produce the structure that is evident in human cognitive capacities, and thus answer the criticisms of Fodor and Pylyshyn. This paper is scheduled to appear in AI and Society in January 1990. To obtain a copy of this paper, send e-mail to french at cogsci.indiana.edu or write to: Bob French Center for Research on Concepts and Cognition Indiana University 510 North Fess Bloomington, Indiana 47401 From derek at prodigal.psych.rochester.edu Sat Oct 7 18:32:15 1989 From: derek at prodigal.psych.rochester.edu (Derek Gross) Date: Sat, 7 Oct 89 18:32:15 EDT Subject: connectionist models of music? Message-ID: <8910072232.AA28088@prodigal.psych.rochester.edu> Does anyone know of any connectionist models of musical structures, perception or composition? If so, please send me e-mail. Thanks, Derek Gross University of Rochester Cognitive Science From todd at galadriel.Stanford.EDU Sun Oct 8 00:02:09 1989 From: todd at galadriel.Stanford.EDU (Peter Todd) Date: Sat, 07 Oct 89 21:02:09 PDT Subject: connectionist models of music Message-ID: In answer to Derek Gross's question about connectionist models of music, I wanted to point out that the current and next issues of the Computer Music Journal (MIT Press) are specifically devoted to this topic. In the current issue, 13(3), out now, there is a general tutorial on musical applications of networks, plus articles on network models of pitch perception, tonal analysis, quantization of time, complex musical patterns, and instrument fingering. In the next issue, 13(4), due out at the end of the year, there will be articles on my work using sequential networks for composition, modelling tonal expectancy (with Jamshed Bharucha, who has also published much work in the area of network modelling of human musical behavior), and another article on representations for pitch perception. Both issues were edited by D. Gareth Loy, of UC San Diego, and myself; the journal is available in some bookstores. Hope this helps-- peter todd stanford university psychology department From khaines at GALILEO.ECE.CMU.EDU Mon Oct 9 14:33:05 1989 From: khaines at GALILEO.ECE.CMU.EDU (Karen Haines) Date: Mon, 9 Oct 89 14:33:05 EDT Subject: IJCNN 1990 - Request for Volunteers Message-ID: <8910091833.AA05968@galileo.ece.cmu.edu> This is a first call for volunteers to help at the IJCNN conference, to be held at the Omni Shorham Hotel in Washington D.C., on January 15-19, 1990. Full admittance to the conference and a copy of the proceedings is offered in exchange for your assistance throughout the conference. In general, each volunteer is expected to work one shift, either in the morning or the afternnon, each day of the conference. Hours for morning shift are, approximately, 7:00 am until 12:00 noon, and for the afternoon, 12:00 noon to 5:00 pm. In addition, assistance will be required for the social events. If you can`t work all week long please contact Karen Haines to see what can be worked out. There will be a mandatory meeting for all volunteers on January 14. To sign up please contact: Karen Haines - Volunteer Coordinator 3138 Beechwood Blvd. Pittsburgh, PA 15217 office: (412) 268-3304 message: (412) 422-6026 email: khaines at galileo.ece.cmu.edu or, Nina Kowalski - Assistant Volunteer Coordinator 209 W. 29th St. FLR 2 Baltimore, MD 21211 message: (301) 889-0587 email: nina at alpha.ece.jhu.edu If you have further questions, please feel free to contact me. Thank you, Karen Haines From brittner at spot.Colorado.EDU Mon Oct 9 17:41:18 1989 From: brittner at spot.Colorado.EDU (BRITTNER RODNEY) Date: Mon, 9 Oct 89 15:41:18 MDT Subject: Tech Report "Towards a Cognitive Connectionism" available Message-ID: <8910092141.AA17506@spot.Colorado.EDU> Hello, Pleas send me a copy of the Tech report "towards a cog. connectionism"!! Thanks!! R. Brittner University of Colorado Department of Aerospace Engineering Bioserve Offices Boulder, CO 80309 From aboulang at WILMA.BBN.COM Mon Oct 9 19:16:28 1989 From: aboulang at WILMA.BBN.COM (aboulang@WILMA.BBN.COM) Date: Mon, 9 Oct 89 19:16:28 EDT Subject: Parallelism, Real vs. Simulated: A Query In-Reply-To: stevan r harnad's message of Thu, 5 Oct 89 11:27:01 EDT <8910051527.AA28565@flash.bellcore.com> Message-ID: Stevan Harnad (harnad at confidence.princeton.edu) writes: I have a simple question: What capabilities of PDP systems do and do not depend on the net's actually being implemented in parallel, rather than just being serially simulated? Is it only speed and capacity parameters, or something more? The more general question of the difference between parallel computation (under any means) and serial computation has interested me for a number of years. It is clear that synchronous parallelism and serial computation are the same modulo the speedup. The harder question is whether asynchronous parallelism computes the same way modulo the speedup. The computability results for asynchronous computation had their start with David Muller ("Infinite Sequences and Finite Machines", Switching Circuit Theory and Logical Design: Proc. 4th Ann. Symp. IEEE pp 3-16, 1963) who used as an example 2 asynchronous fed-back NOT circuits. The analysis led to the work in omega-languages which are languages over the power sets of all the possible states for such a combined circuit would produce. The computability results for omega-languages are that they are no more powerful than sequential Turing machines. Another line of reasoning is to ask whether parallel-asynchronous dynamical computational systems have a different kind of dynamics than sequential dynamical systems and whether the computational abilities of the system can make use of this difference in dynamics. Asynchronization or time-delays can act as a source of noise to help settle stochastic networks. See for example the discussion of time delays in "Separating Figure from Ground with a Boltzmann Machine" Terrence Sejnowski and Geoffrey Hinton, In Vison Bra in and Cooperative Computation, M.A. Arbib & A.R. Hanson eds., MIT Press 1985, and "Effects of Connection Delays in Two State Model Neural Circuits", Peter Conwell, IEEE First Conference on Neural Networks, III-95 - III-104. Note that the term "asynchronous updates" in the neural network research that comes from the spin-glass people normally means sequential. There has been some work on the convergence of neural nets with asynchronous-parallel updates. See for example '"Chaotic Relaxation" in Concurrently Asynchronous Neurodynamics', Jacob Barhen and Sandeep Gulati, IJCNN 1989, pp I-619 - I-626, and "Partially Asynchronous, Parallel Algorithms for Network Flow and Other Problems", P. Tseng, D.P. Bertsekas, and J.N. Tsitsiklis, Center for Intelligent Control Systems Report CICS-P-91, November 1988. The dynamics of such systems is related to the study of time-delay equations in the dynamical system literature. Work has been done on the routes to chaos, etc. in such equations with constant delay, but little or no work has been done with variable delay; which is the case with asynchronous parallelism. This is an area that I am actively studying. Finally, the computational abilities of general analog dynamical systems have been studied by several people including Bradley Dickenson, EE Dept., at Princton, so you may want to talk to him. His results indicate that the hardness of NP-complete problems translate into exponential scaling of analog resources. I believe that some further insight to your question can be had by using Kolmogorov-Chaitin algorithmic complexity applied to dynamical systems that compute. Algorithmic complexity has been applied to dynamical systems to help distinguish and separate the notions of determinism, and randomness is such systems. See the excellent paper, "How Random is a Coin Toss", Joseph Ford, April 1983 40-47. One way to summarize this work is to say that a pseudo-random number generator (which are only iterated nonlinear maps) could in principle generate truly random numbers if seeded with an infinite-algorithmic-complexity number. Another fact to appreciate is that most of the real line is made up of numbers that are algorithmically-incompressible - that is they have a maximal algorithmic complexity for their length as decimal or binary expansions. Irrational numbers would take an infinite amount of time and space resources to compute their expansions - this space-time notion is a variant of the algorithmic complexity measure sometimes called Q-logical depth. One would think that a computational system that could use such a powerful number could be used to compute beyond the capabilities (either in speed or in computability) than computational systems that don't have access to such numbers. (For example, the digits of Chaitin's OMEGA number solves the halting problem by its very definition. See for example "Randomness in Arithmetic", Gregory Chaitin, Scientific American, July 1988, 80-85.) In my mind the question of computability for analog and parallel-asynchronous architectures is precisely whether these constructs can implicitly or explicitly use numbers like Chaitin's OMEGA, to do computation faster, to scale better, or to compute normally incomputable questions than standard computer models. An example of explicit representation is representing a number like OMEGA as an analog value. Due to the fact that values at some level become quantal makes this unlikely. An example of an implicit representation is the possible irrational timing relationships between the self-timings of the elements of an asynchronous computing device. I have been thinking about this implicit possibility for a while, but the quantization of space-time at Plank lengths would eliminate this too. I have not turned my back on this since this just prevents us from using infinite-algorithmic complexity numbers. I still think we can do something with large algorithmic-complexity numbers - in either speed or scaling. Some indication of how hard it would be to go beyond normal "digital" computational devices in their abilities comes from a line of inquiry in dynamical systems. This started when people asked a natural question, "Is it reasonable to simulate chaos on the computer?." Remarkably, the trajectories of a dynamical system simulation on a computer can "shadow" true orbits for very long lengths of time. See for example "In the Shadows of Chaos", Ivar Peterson, Science News, Vol 134, December 3 1988, 360-361, and "Numerical Orbits of Chaotic Processes Represent True Orbits", Stephen Hammel, James Yorke, and Celso Grebogi, Bulletin of the AMS, Vol 19, No 2, October 1988, 465-469. A very similar question is being explored by researchers in quantum chaos. Here the phenomena is called "phase mixing" and quantum system (which cannot have chaos given the its mathematical form) will mix in a very similar way its analog classical system will, but is reversible - unlike its ergodic counterpart. For very long periods of time quantum systems can emulate chaotic classical systems. See for example "Classical Mechanics, Quantum Mechanics, and the Arrow of Time", T.A. Heppenheimer, Mosiac, Vol 20. No. 2, Summer 1989, 2-11. In closing, questions similar to yours have been asked about computers based on the rules of Quantum Mechanics. (Quantum Computers and computing-in-the-small was an area of investigation of Richard Feynman during his last years. Richard Feynman, John Hopfield, and Carver Mead did a course in the physics of computation which was a seed to Carver Mead's new book, "Analog VLSI and Neural Systems".) David Deutsch has claimed in his paper "Quantum Theory, the Church-Turing Principle and the Universal Quantum Computer", Proc. R. Soc. Lond., A400, 1985, 97-117 that quantum computers would be more powerful (in the computability sense) than the classical Turing machine. An example of this power is using a quantum computer to compute truly random numbers, or to prove that the Many-Worlds Interpretation of QM is the true one. Deutsch proposes a modified version of Church-Turing hypothesis to cover the case of quantum computers or what he calls "quantum parallelism". Contrasting to this is the work of T. Erber, who has been looking evidence for pseudo-random, as opposed to random, signatures in single-atom resonance experiments. See for example "Randomness in Quantum Mechanics - Nature's Ultimate Crytogram?", T. Erber & S. Putterman, Nature, Vol 318, No 7, November 1985, 41-43. From galem at mcc.com Tue Oct 10 10:37:18 1989 From: galem at mcc.com (Gale Martin) Date: Tue, 10 Oct 89 09:37:18 CDT Subject: harnad's question Message-ID: <8910101437.AA02886@sunkist.aca.mcc.com> From aboulang at WILMA.BBN.COM Thu Oct 12 11:13:33 1989 From: aboulang at WILMA.BBN.COM (aboulang@WILMA.BBN.COM) Date: Thu, 12 Oct 89 11:13:33 EDT Subject: Some unresolved pointers resolved! In-Reply-To: Paul Kube's message of Tue, 10 Oct 89 13:48:34 PDT <8910102048.AA29395@kokoro.UCSD.EDU> Message-ID: I am sorry for not giving complete references. There were two that people asked about: "How Random is a Coin Toss" Joseph Ford Physics Today, April 1983, 40-47 & A. Vergis, K. Steiglitz, and B. Dickinson, ``The complexity of analog computation,'' Mathematics and Computers in Simulation, vol. 28, 1986, pp. 91-113. In the shadow of your smiles :-)I, Albert Boulanger BBN Systems & Technologies Corporation aboulanger at bbn.com From sankar at caip.rutgers.edu Thu Oct 12 13:28:24 1989 From: sankar at caip.rutgers.edu (ananth sankar) Date: Thu, 12 Oct 89 13:28:24 EDT Subject: No subject Message-ID: <8910121728.AA05622@caip.rutgers.edu> A neural network basically classifies training and testing samples into different regions in an n dimensional space. By generating the output of the network for all possible points in the space one constructs a n + 1 dimensional surface with n independent variables. A polynomial can be generated to approximate this surface. It should be possible to construct a "polynomial neural network" that can do this job. The neurons individually may implement simple polynomials (using sigma-pi units maybe). I would really appreciate any pointers to any research on this (published or unpublished). The work that I am aware of dates to the late 60's--A.G. Ivakhnenko and Donald Specht--though they did not model their systems as nn's. I would also like feedback on what the potential use of such nets may be over typical work like back prop. Thanks in anticipation Ananth Sankar Dept. of Electrical Engg. Rutgers University NJ From sankar at caip.rutgers.edu Thu Oct 12 13:30:03 1989 From: sankar at caip.rutgers.edu (ananth sankar) Date: Thu, 12 Oct 89 13:30:03 EDT Subject: Polynomial Nets Message-ID: <8910121730.AA05674@caip.rutgers.edu> I am sorry to be reposting this but I forgot to put the subject in my last message. Sincere apologies to all. A neural network basically classifies training and testing samples into different regions in an n dimensional space. By generating the output of the network for all possible points in the space one constructs a n + 1 dimensional surface with n independent variables. A polynomial can be generated to approximate this surface. It should be possible to construct a "polynomial neural network" that can do this job. The neurons individually may implement simple polynomials (using sigma-pi units maybe). I would really appreciate any pointers to any research on this (published or unpublished). The work that I am aware of dates to the late 60's--A.G. Ivakhnenko and Donald Specht--though they did not model their systems as nn's. I would also like feedback on what the potential use of such nets may be over typical work like back prop. Thanks in anticipation Ananth Sankar Dept. of Electrical Engg. Rutgers University NJ From honavar at cs.wisc.edu Thu Oct 12 14:29:02 1989 From: honavar at cs.wisc.edu (A Buggy AI Program) Date: Thu, 12 Oct 89 13:29:02 -0500 Subject: Polynomial Nets Message-ID: <8910121829.AA20557@goat.cs.wisc.edu> Here is a list of papers that address the use of "higher order" neurons or links that maybe interpreted as computing terms of a polynomial: Giles, C. L., & Maxwell, T., Learning, invariance, and generalization in higher order neural networks, Applied Optics, vol 26, pp 4972-4978, 1987. Klassen, M. S., & Pao, Y. H., Characteristics of the functional link net: A higher order delta rule net, Proc. of the 2nd annual IEEE conference on Neural Networks, San Diego, CA, 1988. Honavar, V., and Uhr, L. A network of neuron-like units that learns to perceive by generation as well as reweighting of links, Proc. of the 1988 Connectionist models summer school, ed: Touretzky, Hinton, and Sejnowski, Morgan Kaufmann, CA. 1988. Honavar, V., and Uhr, L. Generation, Local receptive fields, and global convergence improve perceptual learning in connectionsit networks, Proc. of IJCAI-89, Morgan Kaufmann, CA. 1989. L. Uhr, Generation+Extraction gives optimal space-time learning of Boolean functions, to appear, Connection Science, 1989. Honavar, V., & Uhr, L. Brain-Structured Connectionist networks that perceive and learn, to appear, Connection Science, 1989. Durbin, R., & Rumelhart, D. E., Product unit: A computationally powerful and biologically plausible extension to backpropagation networks, Neural Computation, vol. 1, pp 133-142. There is also some work in more traditional (inductive) machine learning that falls in this category. Hope this helps. Vasant honavar at cs.wisc.edu From ff%FRLRI61.BITNET at CUNYVM.CUNY.EDU Thu Oct 12 10:30:12 1989 From: ff%FRLRI61.BITNET at CUNYVM.CUNY.EDU (ff%FRLRI61.BITNET@CUNYVM.CUNY.EDU) Date: Thu, 12 Oct 89 15:30:12 +0100 Subject: INNC-90-PARIS Message-ID: <8910121430.AA19182@sun3a.lri.fr> --------------------------------------------------------------------------- INNC 90 PARIS --------------------------------------------------------------------------- INTERNATIONAL NEURAL NETWORK CONFERENCE JULY 9-13, 1990 PALAIS DES CONGRES PARIS FRANCE --------------------------------------------------------------------------- Co-chairmen of the Conference: B. Widrow (Stanford University) B. Angeniol (Thomson-CSF) Program committee chairman: T. Kohonen (Helsinki University) members: I. Aleksander (Imperial College) S. Ichi Amari (Univ. of Tokyo) L. Cooper (Brown Univ.) R. Eckmiller (Univ. of Dusseldorf) F. Fogelman (Univ. of Paris 11) S. Grossberg (Boston Univ.) D. Rumelhart (Stanford Univ.) *: to be confirmed P. Treleaven (University College London) C. von der Malsburg (Univ.of South California) ----------------------------------------------------------------------------- Members of the international community are invited to submit original papers to the INNS-90-PARIS by january 20,1990, in english, on scientific and industrial developments in the following areas: A-APPLICATIONS B-IMPLEMENTATIONS C-THEORY D-COMMERCIAL ----------------------------------------------------------------------------- THE CONFERENCE will include one day of tutorials four days of conference poster sessions prototype demonstrations A forum with workshop sessions:specific interest groups,products sessions deal sessions. ---------------------------------------------------------------------------- For information, contact: Nina THELLIER NTC INNC-90-PARIS 19 rue de la Tour 75116 PARIS-FRANCE Tel: (33-1) 45 25 65 65 Fax: (33-1) 45 25 24 22 ----------------------------------------------------------------------------- Francoise Fogelman From poggio at ai.mit.edu Thu Oct 12 17:37:21 1989 From: poggio at ai.mit.edu (Tomaso Poggio) Date: Thu, 12 Oct 89 17:37:21 EDT Subject: Polynomial Nets In-Reply-To: A Buggy AI Program's message of Thu, 12 Oct 89 13:29:02 -0500 <8910121829.AA20557@goat.cs.wisc.edu> Message-ID: <8910122137.AA15281@wheat-chex> There are much older paper s that are relevant. From Dave.Touretzky at B.GP.CS.CMU.EDU Thu Oct 12 22:54:08 1989 From: Dave.Touretzky at B.GP.CS.CMU.EDU (Dave.Touretzky@B.GP.CS.CMU.EDU) Date: Thu, 12 Oct 89 22:54:08 EDT Subject: tech report available Message-ID: <9726.624250448@DST.BOLTZ.CS.CMU.EDU> The following tech report is now available: BoltzCONS: Dynamic Symbol Structures in a Connectionist Network David S. Touretzky School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213-3890 CMU-CS-89-182 BoltzCONS is a connectionist model that dynamically creates and manipulates composite symbol structures. These structures are implemented using a functional analog of linked lists, but BoltzCONS employs distributed representations and associative retrieval in place of a conventional memory organization. Associative retrieval leads to some interesting properties. For example, the model can instantaneously access any uniquely-named internal node of a tree. But the point of the work is not to reimplement linked lists in some peculiar new way; it is to show how neural networks can exhibit compositionality and distal access (the ability to reference a complex structure via an abbreviated tag), two properties that distinguish symbol processing from lower-level cognitive functions such as pattern recognition. Unlike certain other neural net models, BoltzCONS represents objects as a collection of superimposed activity patterns rather than as a set of weights. It can therefore create new structured objects dynamically, without reliance on iterative training procedures, without rehearsal of previously-learned patterns, and without resorting to grandmother cells. This paper will appear in a special issue of Artificial Intelligence devoted to connectionist symbol processing. Note: the BoltzCONS work first appeared in print in 1986. This paper offers a more detailed description of the model than previously available, and a much more thorough analysis of its significance and weak points. ................ To order a copy of this tech report, write to the School of Computer Science at the address above, or send email to Copetas at cs.cmu.edu. Ask for report number CMU-CS-89-182. From weili at wpi.wpi.edu Thu Oct 12 23:33:36 1989 From: weili at wpi.wpi.edu (Wei Li) Date: Thu, 12 Oct 89 23:33:36 EDT Subject: large scale networks Message-ID: <8910130333.AA01880@wpi.wpi.edu> Hi, I would like to get some references on performances of neural networks on large size problems such as applied to a power system network which has at least thousands of state variables. Are there any neural networks which have showed good convergence abilities for large scale problems? How could they learn so many correlations and arrive at a stable state? Can they be simulated in a work station type computer or minicomputer? Thanks a lot. e-mail: weili at wpi.wpi.edu Wei Li EE. Dept., WPI 100 Institute Road Worcester, MA 01609 From pollack at cis.ohio-state.edu Fri Oct 13 10:48:47 1989 From: pollack at cis.ohio-state.edu (Jordan B Pollack) Date: Fri, 13 Oct 89 10:48:47 EDT Subject: Job Posting Message-ID: <8910131448.AA02815@toto.cis.ohio-state.edu> My dept is recruiting a couple of faculty in areas which migbt be of interest to this group. The advertisement for COMPUTATIONAL MODELS of NEURAL INFO. PROCESSING, going out to press is enclosed below. Since the area is quite large and vague, we have two subareas in mind, but quality will overrule discipline. The first subarea is "Biologically Realistic Connec- tionism", and would deal with working models of neurons, organs, or small creatures. The second potentially skips over biology and goes right to math and physics. "Non- Linear Cognition", or the study of complex dynamical systems related either to brain or mind (e.g. self-organizing circu- itry, cellular automata (reversibility?) chaos and complex- ity theory, fractal patterns in speech/music, and so on. We are also recruiting on a separate billet in SPEECH PROCESSING, which could easily be in neural networks as well. Please contact me if you want to discuss it, or know of anybody good. Columbus is an especially nice place to live. Jordan pollack at cis.ohio-state.edu -------------------------------------------------------------------- Laboratory for Artificial Intelligence Research Department of Computer and Information Science and The Center for Cognitive Science at the The Ohio State University Position Announcement in Computational Neuroscience A tenure-track faculty position at the Assistant Pro- fessor level is expected to be available in the area of Com- putational Neuroscience. We are seeking outstanding appli- cants who have a strong background and research interest in developing computational models of neural information pro- cessing. A Ph.D. in computer science, or in some other appropriate area with a sufficiently strong background in computation, is required. The candidate will be a regular faculty member in the Department of Computer & Information Science, and will promote interactions among cognitive science, computer science and brain science through the Center for Cognitive Science. The LAIR has strong symbolic and connectionist projects underway, the Department has wide interests in parallel com- putation, and the University has the major facilities in place to support the computational neuroscience enterprise, including several parallel computers, a Cray Y/MP, and a full range of brain imaging systems in the medical school. Applicants should send a resume along with the names and addresses of at least three professional references to Prof. B. Chandrasekaran Department of Computer & Information Science Ohio State University 2036 Neil Ave. Columbus, OH 43210 The Ohio State University is an Equal Opportunity Affirmative Action Employer, and encourages applications from qualified women and minorities. From gluck at psych.Stanford.EDU Fri Oct 13 11:34:22 1989 From: gluck at psych.Stanford.EDU (Mark Gluck) Date: Fri, 13 Oct 89 08:34:22 PDT Subject: higher-order/polynomial units in human learning models Message-ID: The use of "higher-order" or polynomial units also has a long tradition in animal and human learning theory where they are called "configural-cues." We have found that such units, combined with the LMS algorithm, do quite well in predicting and fitting a wide range of complex human classification and recognition behaviors (often better than base-line backprop networks). This work is described in: Gluck, Bower, & Hee (1989). A configural-cue network model of animal and human associative learning. Proceedings of the Eleventh Annual Meeting of the Cognitive Science Society, Ann Arbor, MI. Lawrence Erlbaum Associates: Hillsdale, NJ Reprint requests can be sent to: gluck at psych.stanford.edu From jose at neuron.siemens.com Fri Oct 13 12:54:13 1989 From: jose at neuron.siemens.com (Steve Hanson) Date: Fri, 13 Oct 89 12:54:13 EDT Subject: new address Message-ID: <8910131654.AA12546@neuron.siemens.com.siemens.com> As of October 1 I have moved to Siemens in Princeton S. J. Hanson Siemens Research Center 755 College Road East Princeton, New Jersey 08540 jose at tractatus.siemens.com (609)734-3360 I can also be reached at Cognitive Science Laboratory 221 Nassau Street Princeton University Princeton, New Jersey 08542 jose at clarity.princeton.edu (609)258-2219 Please update your info. Thanks Steve From rsun at cs.brandeis.edu Mon Oct 16 12:41:32 1989 From: rsun at cs.brandeis.edu (Ron Sun) Date: Mon, 16 Oct 89 12:41:32 edt Subject: higher-order neurons Message-ID: There are more generalized neuronal models that exibit interesting properties. Some of them are discrete rather than continuous. Some have weights while others do not even have weights. They are higher-order, in the sense that they can compute more complicated mappings. Some references here: I. Aleksander, The Logic of connectionist system, in: Neural Computing Architectures, MIT Press, 1989 A. Barto, From Chemotaxis to Cooperativity, COINS TR 88-65, University of Massachusetts, 1988 J. Feldman and D. Ballard, Connectionist models and their properties, Cognitive Science, July, 1982 A. Klopf, The Hedonistic Neuron, Hemisphere, 1982 R. Sun, A discrete neural network model for conceptual representation and reasoning, 11th Cognitive Science Society Conference, 1989 R. Sun, E. Marder and D. Waltz, Model local neural networks in the lobster stomatogastric ganglion, IJCNN, 1989 R. Sun, Designing inference engines based on a discrete neural network model, Proc. IEA/AIE, 1989 R. Sun, The Discrete Neuron and The Probabilistic Discrete Neuron, submitted to INNC, 1989 R. Sutton and A. Barto, A Temporal-Difference Model of Classical Conditioning, Proceedings of 9th Cognitive Science Society Conference, 1987 From lange at CS.UCLA.EDU Tue Oct 17 08:45:25 1989 From: lange at CS.UCLA.EDU (Trent Lange) Date: Tue, 17 Oct 89 05:45:25 PDT Subject: tech report available Message-ID: <891017.124525z.22274.lange@lanai.cs.ucla.edu> **********DO NOT FORWARD TO OTHER BBOARDS************** **********DO NOT FORWARD TO OTHER BBOARDS************** **********DO NOT FORWARD TO OTHER BBOARDS************** The following tech report is now available: High-Level Inferencing in a Connectionist Network Trent E. Lange Michael G. Dyer Artificial Intelligence Laboratory Computer Science Department University of California, Los Angeles Los Angeles, CA 90024 UCLA-AI-89-12 Connectionist models have had problems representing and apply- ing general knowledge rules that specifically require variables. This variable binding problem has barred them from performing the high-level inferencing necessary for planning, reasoning, and natural language understanding. This paper describes ROBIN, a structured neural network model capable of high-level inferencing requiring variable bindings and rule application. Variable bind- ings are handled by signatures -- activation patterns which uniquely identify the concept bound to a role. Signatures allow multiple role-bindings to be propagated across the network in parallel for rule application and dynamic inference path instantiation. Signa- tures are integrated within a connectionist semantic network struc- ture whose constraint-relaxation process selects between those newly-instantiated inferences. This allows ROBIN to handle an area of high-level inferencing difficult even for symbolic models, that of resolving multiple constraints from context to select the best interpretation from among several alternative and possibly ambiguous inference paths. This is a pre-print of a paper that will appear in Connection Science. ................ To order a copy of this tech report, write to Trent Lange at the address above, or send e-mail to lange at cs.ucla.edu. Ask for tech report number UCLA-AI-89-12. From jose at neuron.siemens.com Thu Oct 19 07:21:40 1989 From: jose at neuron.siemens.com (Steve Hanson) Date: Thu, 19 Oct 89 07:21:40 EDT Subject: Job Announcement Message-ID: <8910191121.AA15853@neuron.siemens.com.siemens.com> Learning & Knowledge Acquisition Siemens Corporate Research, Inc, the US research branch of Siemens AG with sales in excess of 30$ Billion worldwide has research openings in the Learning and Knowledge Acquisition Group for research staff scientists. The group does basic and applied studies in the areas of Learning (Connectionist and AI), adaptive processes, and knowledge acquisition. Above and beyond Laboratory facilities, the group has a network of sun workstations (sparcs), file and compute servers, Lisp machines and a mini-supercomputer all managed by a group systems administrator/research programmer. Connections exist with our sister laboratory in Munich, Germany as well as with various leading Universities including MIT, CMU and Princeton University, in the form of joint seminars, shared postdoctoral position, and collaborative research. The susscessful candidate should have a Ph.D. in Computer Science Electrical Engineering, or any other AI-related or Cognitive Science field. Areas that we are soliciting for presently are in Neural Computation, or Connectionist Modeling especially related to Learning Algorithms, Novel Architectures, Dynamics, Biological Modeling, and including any of the following application areas Pattern Classification/Categorization, Speech Recognition, Visual Processing, Sensory Motor Control (Robotics), Problem Solving, Natural Language Understanding, Siemens is an equal opportunity employer, Please send your resume and a reference list to Stephen J. Hanson Learning and Knowledge Acquisition Group Siemens Corporate Research, Inc. 755 College Road East Princeton, NJ 08540 jose at tractatus.siemens.com jose at clarity.princeton.edu From skuo at caip.rutgers.edu Thu Oct 19 14:34:31 1989 From: skuo at caip.rutgers.edu (Shyh-shiaw Kuo) Date: Thu, 19 Oct 89 14:34:31 EDT Subject: No subject Message-ID: <8910191834.AA16589@caip.rutgers.edu> Hi, I will appreciate if you can tell me any reference which apply polynomials to pattern recognition, or neural nets. As I know, people name those methods as Group Method of Data Handling (GMDH) algorithm, or Self-Organizing method. I also need some current publications, say after 1983, which state the improvements of those algorithms. Your help will be highly appreciated. --- Shyh-shiaw Kuo From welleken at wind.bellcore.com Thu Oct 19 17:54:53 1989 From: welleken at wind.bellcore.com (Christian Wellekens) Date: Thu, 19 Oct 89 17:54:53 EDT Subject: Polynomials in PR Message-ID: <8910192154.AA16909@wind.bellcore.com> You could have a look on the recent book of Y-H Pao : Adaptive Pattern Recognition and Neural networks, Addison Wesley. 1989 Professor Pao is at the EEDpt of the CASE Western Reserve University and is president of AI Ware. Chris Wellekens Bellcore 2M336 445 South St Morristown NJ 07960-1910 From kube%cs at ucsd.edu Thu Oct 19 18:40:46 1989 From: kube%cs at ucsd.edu (Paul Kube) Date: Thu, 19 Oct 89 15:40:46 PDT Subject: polynomial classifiers In-Reply-To: Shyh-shiaw Kuo's message of Thu, 19 Oct 89 14:34:31 EDT <8910191834.AA16589@caip.rutgers.edu> Message-ID: <8910192240.AA08953@kokoro.UCSD.EDU> W. E. Blanz at IBM Almaden (San Jose) has worked on polynomial classifiers and compared their performance to maximum likelihood and connectionist classifiers. You might be interested in his IBM reports RJ 5418 "A comparison of polynomial and parametric gaussian maximum likelihood classifiers" and RJ 6891 "Comparing a connectionist trainable classifier with classical decision analysis methods." The latter reports, by the way, that a PDP classifier outperforms gaussian ML and less-than-quartic polynomial classfiers, and scales better besides. --Paul kube at ucsd.edu From chuck%henry at gte.com Fri Oct 20 10:37:00 1989 From: chuck%henry at gte.com (Chuck Anderson) Date: Fri, 20 Oct 89 10:37 EDT Subject: polynomial nets Message-ID: <19891020143736.9.CHUCK@henry.gte.com> A good source for descriptions of methods and applications of GMDH is the book: "Self-Organizing Methods in Modeling: GMDH Type Algorithms", edited by Stanley J. Farlow, published by Marcel Dekker, Inc., NY, 1984. Barron Associates, Inc., of Stanardsville, Virginia, have been investigating a number of variations of GMDH for some time. I have seen publications by Roger Barron and John Elder from there. In "Automated Design of Continuously-Adaptive Control..." by Elder and Barron, in the proceedings of the 1988 American Control Conference, a polynomial net is used to form a map from aircraft sensors to control actions. The sensors indicate errors between actual and desired response of the aircraft. The net learns actions that compensate for damaged control surfaces on the aircraft. Good actions were determined by an off-line optimization using an aircraft simulation. The net was used to interpolate between these optimized values. Another Barron, Andrew, at the Univ. of Ill., is contributing to this work by exploring the links between polynomial nets and more standard statistical methods. See Andrew Barron and Roger Barron, "Statistical Learning Networks: A Unifying View", Proc. of the 1988 Symposium on the Interface: Statistics and Computing Science, Reston, VA, April 21-23. And, at the upcoming NIPS conferece Andrew Barron is giving an invited presentation on polynomial networks. Chuck Anderson GTE Laboratories Inc. 40 Sylvan Road Waltham, MA 02254 617-466-4157 canderson%gte.com at relay.cs.net From rohit at faulty.che.utexas.edu Fri Oct 20 11:51:30 1989 From: rohit at faulty.che.utexas.edu (rohit@faulty.che.utexas.edu) Date: Fri, 20 Oct 89 10:51:30 CDT Subject: help with ANN Message-ID: <8910201551.AA00218@faulty.che.utexas.edu.che.utexas.edu> I am trying to design an ANN to model a function which takes a step as an input and shifts the step in time and changes the amplitude of the step . I am using backward propogation with sigmoid functions for the nodes in the hidden layers, can anyo ne make any suggestions. I had posted this question on the ai.neural.nets and a couple of people said that they had done it but nobody could tell me how they did it i.e architecture of the net waht inputs etc... I would appreciate it if somone would make some concrete suggestions. rohit at faulty.che.utexas.edu From John.Hampshire at SPEECH2.CS.CMU.EDU Sat Oct 21 19:49:50 1989 From: John.Hampshire at SPEECH2.CS.CMU.EDU (John.Hampshire@SPEECH2.CS.CMU.EDU) Date: Sat, 21 Oct 89 19:49:50 EDT Subject: connectionist step function manipulator Message-ID: If you want to time-shift and amplitude-scale a step function, I think it's easiest to build a simple algorithm to do this. On the assumption that this approach is undesirable for some reason, I'd offer the following: Look into classic linear systems/signal processing theory. Linear systems theory focusses heavily on the step response (and impulse response) of linear systems and how these time-domain responses relate to frequency-domain representations via Fourier transformation. One way to get your desired network would start with specifying the desired step response you want, differentiating it in the time domain to obtain the equivalent impulse response, then Fourier transforming this result for the frequency domain equivalent. If your desired step response involves a time shifting and amplitude scaling, then these manipulations constitute a pretty simple impulse response that transforms to a phase-shift and amplitude scaling in the frequency domain. There is a huge volume of information on the Fast Fourier Transform (dating back to C. F. Gauss, but generally associated with Cooley and Tukey [1965]). Probably the best overview is given by Oppenheim and Schafer in "Discrete Time Signal Processing" (Prentice Hall, 1989) or the predecessor by the same authors and publisher "Digital Signal Processing" (1975). One can view the various implementations of the FFT as neural networks despite the fact that they are ultimately linear operators. I imagine a some folks will dispute this equation, but if adaptive phasing devices for antenna arrays are viewed as neural nets, FFTs can be too. Anyway, if you must use a "neural network", you could implement your desired function in discrete time using an N-point FFT of your input, a frequency-domain multiplication of the transformed input and transformed impulse response, then an N-point inverse FFT. Oppenheim and Schafer (1989) chapter 8 covers this concept pretty well (there's more to it than this explanation). If you wanted to *learn* the desired time-shift and scaling factor (instead of deriving it a-priori), I imagine you could set up the FFT and inverse FFT structures and then use an error function with backpropagation (through the UN-CHANGING "connections" of the inverse FFT stage of the network). The error signal would backpropagate to alter the coefficients of your frequency domain representation of the impulse response. Of course, those frequency domain coefficients are *complex*, not pure real (as are a number of the "connections" in the inverse FFT structure), and I haven't really considered why such an idea might not work, but what the heck. Then there's the question of *why* you'd want to learn the frequency domain form of the impulse response if you can derive it in closed form a-priori, but I'm sure there's a good reason associated with your wanting to use a connectionist architecture. If all of this is old news to you, then sorry --- toss it in the trash. If it's new info then I hope it's of some help. Cheers, John P. S. all of this is really just cramming traditional signal processing into connectionist packaging. I don't mean to claim otherwise. From victor%FRLRI61.BITNET at CUNYVM.CUNY.EDU Mon Oct 23 06:44:26 1989 From: victor%FRLRI61.BITNET at CUNYVM.CUNY.EDU (victor%FRLRI61.BITNET@CUNYVM.CUNY.EDU) Date: Mon, 23 Oct 89 11:44:26 +0100 Subject: DB - NN Message-ID: <8910231044.AA16142@sun8.lri.fr> Dear colleges, I will appreciate if you can tell me any reference which apply neural networks to Data Base Management Systems (DBMS). Your help will be highly appreciated. Victor Cruz victor at lri.lri.fr In advance, thanks a lot From LIN2%YKTVMZ.BITNET at CUNYVM.CUNY.EDU Mon Oct 23 17:11:01 1989 From: LIN2%YKTVMZ.BITNET at CUNYVM.CUNY.EDU (LIN2%YKTVMZ.BITNET@CUNYVM.CUNY.EDU) Date: Mon, 23 Oct 89 17:11:01 EDT Subject: Preprint available Message-ID: ********* FOR CONNECTIONISTS ONLY - PLEASE DO NOT FORWARD *********** **************** TO OTHER BBOARDS/ELECTRONIC MEDIA ******************* The following preprint is available. If you would like a copy, please send a note to lin2 @ ibm.com CONTAINING *ONLY* THE INFORMATION ON THE FOLLOWING FOUR LINES (to allow semi-automated handling of your request): *IJ* Your Name Your Address (each line not beyond column 33) Designing a Sensory Processing System: What Can Be Learned from Principal Components Analysis? Ralph Linsker IBM Research, T. J. Watson Research Center, Yorktown Heights, NY 10598 Principal components analysis (PCA) is a useful tool for understanding some feature-analyzing properties of cells found in at least the first few stages of a sensory process- ing pathway. However, the relationships between the results obtained using PCA, and those obtained using a Hebbian model or an information-theoretic optimization principle, are not as direct or clear-cut as sometimes thought. These points are illustrated for the formation of center- surround and orientation-selective cells. For a model "cell" having spatially localized connections, the relevant PCA eigenfunction problem is shown to be separable in polar coordinates. As a result, the principal components have a radially sectored (or "pie-slice") geometric form, and (in the absence of additional degeneracies) do *not* resemble classic Hubel-Wiesel "simple" cells, except for the (odd- symmetry) eigenmodes that have exactly two sectors of oppo- site sign. However, for suitable input covariance functions, one can construct model "cells" of simple-cell type -- which are in general not PCA eigenfunctions -- as particular linear combinations of the first few leading principal components. A connection between PCA and a criterion for the minimiza- tion of a geometrically-weighted mean squared reconstruction error is also derived. This paper covers in greater detail one of the topics to be discussed in an invited talk at the IJCNN Winter 1990 Meet- ing (Washington, DC, Jan. 1990). It will be published in the conference proceedings. The paper itself contains no abstract; the above is a brief summary prepared for this preprint availability notice. From gary%cs at ucsd.edu Fri Oct 20 20:30:47 1989 From: gary%cs at ucsd.edu (Gary Cottrell) Date: Fri, 20 Oct 89 17:30:47 PDT Subject: seminar announcement: Modeling the Sequential Behavior of the Dog Message-ID: <8910210030.AA01374@desi.UCSD.EDU> SEMINAR Modeling the Sequential Behavior of the Dog: The Second Naive Dog Physics Manifesto Garrison W. Cottrell Department of Dog Science Condominium Community College of Southern California Most work in Dog Modeling has been content to make do with simple Stimulus-Response type models. However, the thing that separates current work in Parallel Dog Processing from the Behaviorists is the emphasis on looking inside the dog's head. So far, few dogs have consented to this procedure, hence, we have to make do with models that tell us what we might find if we looked. S-R models since Pavlov have assumed that there is not much in the head except a connection from the dog's nose to his salivary gland, that may be detached at the nose end and reconnected to his ear via a process called "conditioning"[1]. Departing from the Behaviorists, PDP modelers make the radical assumption that there is a brain inside the dog's head[2], mediating these reponses based on the current state of the dog's brain. However, rather than treating the dog's brain as analogous to a telephone switching network as the neo-Skinnerians do[3], we will treat the dog as a dynamical system, in particular, a dissipative system that takes in material from its environment, extracts energy to maintain its own structure, increasing the entropy of the material before returning it to the environment. The main problem of the dog owner, then, is to train this dynamical system to leave its entropy outside the house. In our work this sequence of desired behavior is specified by the following action grammar, a simplified version of the one used in (Cottrell, 1986a): Day -> Action Day | Sleep Action -> Eat | leavecondo Walk Eat -> Eat chomp | chomp Walk -> poop Walk | trot Walk | sniff Walk | entercondo As previously noted, these rules have the desirable property that entropy in the condo is ungrammatical. In our previous work (Cottrell, 1986a), we took a competence theory approach, i.e., no working computer program was necessary for our theory. While the advantages of the lack of real world constraints that a competence theory approach allows are clear[4], it lacks the advantage of interchange with experiment that performance theories enjoy. In this talk we will describe an approach that avoids the pitfalls of a performance theory (having to deal with data) while incorporating the exchange with experimental modeling by building a computer model of our competence theory[5]. In order to generate a sequence such as that specified by our action grammar, a recurrent network is necessary. To model the initial state of the de novo dog, we start with a randomly wired recurrent network with habituation in the weights. The behavior of this network is remarkably similar to that of the puppy, oscillating wildly, exhibiting totally undisciplined behavior, until reaching a fixed point. Habituation then determines the length of the sleep phase, the model slowly "wakes up", and the cycle starts again[6]. We then apply the WiZ algorithm (Willy & Zippy, 1988) for recurrent bark propagation to train the network to the adult behavior[7]. The training set of sequences of states was generated from the simplified grammar above. Note that the network must actually choose a branch of the grammar to take on any iteration. By simply training the network to take different branches from a nonterminal state on different occasions, the network is unstable when at a nonterminal node. Different actions are then possible attractors from that state. By using a random updating rule, different transitions naturally occur. Transitions out of a particular terminal state are due to habituation, i.e., our dog model stops doing something because of the equivalent of boredom with any particular state[8]. Thus, boredom is an emergent property of our model. ____________________ [1]The obvious implausibility of such a process notwithstanding (cf. Chompski's scathing critique, No Good Dogs, 1965), hordes of researchers have spent many years studying it. [2]However, it is often hard to explain this view to the lay public, especially most dog owners. [3]This was a major improvement on older behaviorist theories. All that is needed now is to posit that conditioning somehow accesses the "telephone operator" in the brain that pulls a plug and reinserts it somewhere else. This model is much more plausible since the mechanisms of conditioning have been fleshed out. It also explains why dogs some- times don't respond at all - they haven't kept up on the phone bill. [4]For example, in (Cottrell, 1986a) we were able to assume that one could generate a context free language with a feed-forward network. All you needed was "hidden units". This is the familiar cry of the "connec- tionist" who has never implemented a network. [5]This is known as the autoerotic approach to theory building. A danger here is that, since Reality has no reason to dampen the possible oscillations between computer simulation and competence theory forma- tion, the process may have positive Lyapunov exponents, and never con- verge on a theory. Such unstable loops can lead to strange attractors that never settle down, such as current linguistic theory. [6]Since the length of time spent in the attractor is determined by the number of units participating in it, it was found that most of the puppy's brain is actually needed for maintaining its extremely long sleep phase. This could be an entirely new explanation of the apparent lack of capacity for much else. [7]In order to get the proper behavior out of our network, teacher- forcing was necessary. This confirms our experience with actual dogs that force is often a necessary component of the training process. [8]This is obviously an inadequate model of how the dog stops eating, which is more due to external reality than any internal control on the dog's part. For this simple process, a Skinnerian feedforward See-Food -> Eat network is sufficient. From rik%cs at ucsd.edu Tue Oct 24 22:33:27 1989 From: rik%cs at ucsd.edu (Rik Belew) Date: Tue, 24 Oct 89 19:33:27 PDT Subject: TR available: Evolution, Learning and Culture too! Message-ID: <8910250233.AA02582@roland.UCSD.EDU> EVOLUTION, LEARNING AND CULTURE: Computational metaphors for adaptive algorithms Richard K. Belew Cognitive Computer Science Research Group Computer Science & Engr. Dept. (C-014) Univ. California at San Diego La Jolla, CA 92093 rik%cs at ucsd.edu CSE Technical Report #CS89-156 Potential interactions between connectionist learning systems and algorithms modeled after evolutionary adaptation are becoming of increasing interest. In a recent, short and elegant paper Hinton and Nowlan extend a version of Holland's Genetic Algorithm (GA) to consider ways in which the evolution of species and the learning of individuals might interact. Their model is valuable both because it provides insight into potential interactions between the {\em natural} processes of evolution and learning, and as a potential bridge between the {\em artificial} questions of efficient and effective machine learning using the GA and connectionist networks. This paper begins by describing the GA and Hinton and Nowlan's simulation. We then analyze their model, use this analysis to explain its non-trivial dynamical behaviors, and consider the sensitivity of the simulation to several key parameters. Our next step is to interpose a third adaptive system --- culture --- between the learning of individuals and the evolution of populations. Culture accumulates the ``wisdom'' of individuals' learning beyond the lifetime of any one individual but adapts more responsively than the pace of evolution allows. We describe a series of experiments in which the most minimal notion of culture has been added to the Hinton and Nowlan model, and use this experience to comment on the functional value of culture and similarities between and interactions among these three classes of adaptive systems. ------------------------------------------------------- Copies of this technical report are available by sending $3 (and asking for Technical Report #CS89-156) to: Ms. Kathleen Hutcheson CSE Dept. (C-014) Univ. Calif. -- San Diego La Jolla, CA 92093 From skuo at caip.rutgers.edu Wed Oct 25 09:27:52 1989 From: skuo at caip.rutgers.edu (Shyh-shiaw Kuo) Date: Wed, 25 Oct 89 09:27:52 EDT Subject: No subject Message-ID: <8910251327.AA02580@caip.rutgers.edu> I will appreciate if you can recommend me any recently published book which is related to Neural Network. From hinton at ai.toronto.edu Wed Oct 25 10:56:14 1989 From: hinton at ai.toronto.edu (Geoffrey Hinton) Date: Wed, 25 Oct 89 10:56:14 EDT Subject: references on predicting time series Message-ID: <89Oct25.105634edt.10980@ephemeral.ai.toronto.edu> We are interested in neural networks that take as input a sequence of symbols or a sequence of parameter-vectors and produce as output a prediction of the next term in the sequence. The prediction is the form of a probability distribution over possible symbols or parameter-vectors. I know that a number of researchers have considered this use of networks. I would like to compile a list of good references on this, particularly recent references. If you know of such a reference (and you are feeling generous), please email it to me (not to all connectionists). geoff From rik%cs at ucsd.edu Wed Oct 25 13:49:03 1989 From: rik%cs at ucsd.edu (Rik Belew) Date: Wed, 25 Oct 89 10:49:03 PDT Subject: $$ for TR's Message-ID: <8910251749.AA03323@roland.UCSD.EDU> From fjcp60!golds at uunet.UU.NET Wed Oct 25 04:22:14 1989 To: uunet!cs.ucsd.edu!rik Subject: tech report I think it is very unfortunate you require payment for the distribution of your tech report. This is contrary to the spirit of free exchange of information embodied in the connectionists mailing list. For individuals like myself who are not currently supported by a grant, or who don't work in a department rich in grant funds, the imposition of fees raises a real barrier to obtaining access research results. Other authors have found ways to provide copies of reports free of charge, and/or have placed a postscript version of their reports on the public archive machine for free FTP access. I would really appreciate it if you could pursue one of these alternate methods of distribution. Thanks... Dr. R. Goldschmidt golds at fjcp60.uu.net 2227 Greenwich St. Falls Church, VA 22043 I couldn't agree more, and you are not the only one to have expressed surprise/bemusement/anger at a charge for a TR. And I feel very silly mentioning it, too. But that happens to be how our department has established the TR distribution procedure and I'm not flush enough to cover all the costs. But for those of you for whom $3 is an impediment I will keep sending copies out gratis until my money runs out. Also, I too think Jordan Pollack's "TR server" of Postscript files is first rate. But it happens that my TR has lots of figures that I haven't been able to successfully fold into the TeX source, and I decided the text needed these figures. (And the department would charge $3 to send just them out!) But my goal is to have a fully Postscript file, next time. I apologize for taking up everyone's time with this silliness. Rik Belew From park at sophocles.cs.uiuc.edu Wed Oct 25 14:18:17 1989 From: park at sophocles.cs.uiuc.edu (Young-Tack Park) Date: Wed, 25 Oct 89 13:18:17 -0500 Subject: references on predicting time series Message-ID: <8910251818.AA02437@sophocles.cs.uiuc.edu> I am interested in the topic. Could you please pass me the list? Thanks in advance, Young-Tack From hinton at ai.toronto.edu Wed Oct 25 16:29:01 1989 From: hinton at ai.toronto.edu (Geoffrey Hinton) Date: Wed, 25 Oct 89 16:29:01 EDT Subject: references on time-series analysis Message-ID: <89Oct25.162910edt.11250@ephemeral.ai.toronto.edu> I have received many requests for copies of the reference list. I will compile all replies into one list and send it to everyone next week. So there is no longer any need to ask me for it . Geoff From loeb at PSYCHE.MIT.EDU Wed Oct 25 16:00:52 1989 From: loeb at PSYCHE.MIT.EDU (Eric Loeb) Date: Wed, 25 Oct 89 16:00:52 edt Subject: references on predicting time series Message-ID: <8910252002.AA22053@ATHENA.MIT.EDU> I am interested too. Could whoever compiles it publish it for all? Thanks, Eric Loeb From Dave.Touretzky at B.GP.CS.CMU.EDU Wed Oct 25 17:48:00 1989 From: Dave.Touretzky at B.GP.CS.CMU.EDU (Dave.Touretzky@B.GP.CS.CMU.EDU) Date: Wed, 25 Oct 89 17:48:00 EDT Subject: a note about etiquette Message-ID: <8290.625355280@DST.BOLTZ.CS.CMU.EDU> Okay you guys: the long-term subscribers to this list have started complaining, so it's time to remind people yet again of the rules of etiquette for using CONNECTIONISTS. Rule #1: if you're too lazy to do your own literature search, don't expect anyone on this list to do it for you. Messages of the form "Please send me a list of books on neural nets" are not appropriate for this forum. The CONNECTIONISTS list is for serious discussion of research, and for dissemination of tech reports and talk announcements. If you're a novice, you should be reading Neuron Digest, not this list. On the other hand, if you're an established neural net researcher already familiar with the literature, and you want to compile a bibliography which you will then distribute for the benefit of the entire community, then a call for references is fine. Rule #2: don't send personal follow-up messages like "Please send me a copy too" to the CONNECTIONISTS list. If you don't know how to use your mail system properly, kindly un-subscribe yourself from this list until you learn. Rule #3: never tell anyone about CONNECTIONISTS or NN-BENCH. Only tell them about CONNECTIONISTS-Request and NN-BENCH-Request. That way requests for subscriptions won't be sent to the wrong place and rebroadcast to 500+ sites all over the globe. Thank you for your cooperation. -- The Management From mdr at dspvax.mit.edu Thu Oct 26 09:16:52 1989 From: mdr at dspvax.mit.edu (Michael D. Richard) Date: Thu, 26 Oct 89 09:16:52 EDT Subject: Mailing list Message-ID: <8910261316.AA20534@dspvax.mit.edu> Please add my e-mail address to the Connectionist mailing list Thanks mdr at dspvax.mit.edu From rudnick at cse.ogc.edu Thu Oct 26 18:50:40 1989 From: rudnick at cse.ogc.edu (Mike Rudnick) Date: Thu, 26 Oct 89 15:50:40 PDT Subject: proteins, rna, dna refs Message-ID: <8910262250.AA19581@cse.ogc.edu> Below is a summary of responses to my earlier posting asking for references to work using ANNs for the recognition of proteins, rna, dna, and the like. They are in the order in which they were received, and have been edited both to remove duplicate references and for relevancy. Thanks to all those who responded. Mike ************************************************************************* From: ohsu-hcx!spackmank at cse.ogc.edu (Dr. Kent Spackman) Subject: connectionist protein structure The two articles I mentioned are: Holley, L.H.; Karplus, M. Protein structure prediction with a neural network. Proceeding of National Academy of Science, USA; 1989; 86: 152-156. Qian, Ning; Sejnowski, Terrence J. Predicting the secondary structure of globular proteins using neural network models. J Mol Biol; 1988; 202: 865-884. I have an article that will be published in the proceedings of the Symposium on Computer Applications in Medical Care, in Washington, D.C., in November, entitled: "Evaluation of Neural Network Performance by ROC analysis: Examples from the Biotechnology Domain". Authors are M.L. Meistrell and myself. Kent A. Spackman, MD PhD Biomedical Information Communication Center (BICC) Oregon Health Sciences University 3181 SW Sam Jackson Park Road Portland, OR 97201-3098 ---- From: Lambert.Wixson at MAPS.CS.CMU.EDU Subject: DNA,RNA, etc. Holley and Karplus, Proceedings of the National Academy of Science 86, 152-156 (89). ---- From: mv10801 at uc.msc.umn.edu Subject: Re: applications to DNA, RNA and proteins George Wilcox (mf12801 at sc.msc.umn.edu) does work on predicting protein tertiary structure using large backprop nets. --Jonathan Marshall Center for Research in Learning, Perception, and Cognition 205 Elliott Hall, Univ. of Minnesota, Minneapolis, MN 55455 ---- >From munnari!cluster.cs.su.OZ.AU!ray at uunet.UU.NET Fri Sep 29 23:40:55 1989 Subject: applications to DNA, RNA and proteins Borman, Stu "Neural Network Applications In Chemistry Begin to Appear", C&E News, April 24 1989, pp 24-28. Thornton, Janet "The shape of things to come?" Nature, Vol. 335 (1st September 1988), pp 10-11. You probably know about the Qian and Sejnowski paper already. The Thornton "paper" is a fast overview with a sentence or two comparing Q&S's work with other work. Borman's C&E piece is fairly superficial, but it mentions some other people who have played with this stuff, including Bryngelson and Hopfield, Holley and Karplus (who apparantly have published in Proc. Nat. Acad. Sci., 86(1), 152 (1989)) and Liebman. The 1990 Spring Symposium at Stanford (March 27-29, 1990) will have a session on "Artificial Intelligence and Molecular Biology". The CFP lists Neural Networks (very broad-minded of them!), so it might be worth a look when it comes around. Raymond Lister Basser Department of Computer Science University of Sydney NSW 2006 AUSTRALIA Internet: ray at cs.su.oz.AU CSNET: ray%cs.su.oz at RELAY.CS.NET UUCP: {uunet,hplabs,pyramid,mcvax,ukc,nttlab}!munnari!cs.su.oz.AU!ray ---- From: "Evan W. Steeg" Subject: NNets and macromolecules There is a fair amount of work on applying neural networks to questions involving DNA, RNA, and proteins. The two major types of application are: 1) Using neural networks to predict conformation (secondary structure and/or tertiary structure) of molecules from their sequence (primary structure). 2) Using nets to find regularities, patterns, etc. in the sequence itself, e.g. find coding regions, search for homologies between sequences, etc. The two areas are not disjoint -- one might look for alpha-helix "signals" in a protein sequence as part of a structure prediction method, for example. I did my M.Sc. on "Neural Network Algorithms for RNA Secondary Structure Prediction", basically using a modified Hopfield-Tank (Mean Field Theory) network to perform an energy minimization search for optimal structures. A technical report and journal paper will be out soon. I'm currently working on applications of nets to protein structure prediction. (Reference below). Qian and Sejnowski used a feed-forward net to predict local secondary structure of proteins. (Reference above). At least two other groups repeated and extended the Qian & Sejnowski experiments. One was Karplus et al (ref. above) and the other was Cotterill et al in Denmark. (Discussed in a poster at the Fourth International Symposium on Artificial Intelligence Systems, Trento, Italy Sept. 1988). Finally, a group in Minnesota used a supercomputer and back-prop to try to find regularities in the 2-d distance matrices (distances between alpha-carbon atoms in a protein structure). An interim report on this work was discussed at the IJCNN-88 (Wash. DC) conference. (Sorry, I don't recall the names, but the two researchers were at the Minnesota Supercomputer Center, I believe.) As for the numerous research efforts in finding signals and patterns in sequences, I don't have these references handy. But the work of Lapedes of Los Alamos comes to mind as an interesting bit of work. Refs: E.W. Steeg. Neural Network Algorithms for the Prediction of RNA Secondary Structure. M.Sc. Thesis, Computer Science Dept., University of Toronto, Toronto, Ontario, Canada, 1988. Evan W. Steeg (416) 978-7321 steeg at ai.toronto.edu (CSnet,UUCP,Bitnet) Dept of Computer Science steeg at ai.utoronto (other Bitnet) University of Toronto, steeg at ai.toronto.cdn (EAN X.400) Toronto, Canada M5S 1A4 {seismo,watmath}!ai.toronto.edu!steeg ----- From: pastor at PRC.Unisys.COM (Jon Pastor) Subject: Re: applications to DNA, RNA and proteins @article(nakata85a, Author="K. Nakata and M. Kanehisa and D. DeLisi", Title="Prediction of splice junctions in mRNA sequences", Journal="Nucleic Acids Research", Year="1985", Volume="13", Number="", Month="", Pages="5327--5340", Note="", Annote="") @article(stormo82a, Author="G.D. Stormo and T.D. Schneider and L.M. Gold ", Title="Characterization of translational initiation sites in E. coli", Journal="Nucleic Acids Research", Year="1982", Volume="10", Number="", Month="", Pages="2971--2996", Note="", Annote="") @article(stormo82b, Author="G.D. Stormo and T.D. Schneider and L.M. Gold and A. Ehrenfeucht", Title="Use of the `perceptron' algorithm to distinguish translational initiation sites in E. coli", Journal="Nucleic Acids Research", Year="1982", Volume="10", Number="", Month="", Pages="2997--3010", Note="", Annote="") In addition, there is going to be (I think) a paper by Alan Lapedes, from Los Alamos, in a forthcoming book published by the Santa Fe Institute; my group also has a paper in this book, which is how I know about Lapedes' submission. I am going to try to contact the editor to see if I can get a preprint; if so, I'll let you know. I didn't attend the meeting at which Lapedes presented his paper, but I'm told that he was looking for splice junctions. ---- From: ff%FRLRI61.BITNET at CUNYVM.CUNY.EDU (Francoise Fogelman) Subject: proteins We have done some work on the prediction of secondary structures of proteins. This was presented at a NATO meeting (Les Arcs, march 1989) and will be published in the proceedings. F. Fogelman LRI Bat 490 Universite de Paris Sud 91405 ORSAY cedex FRANCE Tel 33 1 69 41 63 69 e-mail: ff at lri.lri.fr ---- The book "Evolution, Learning and Cognition", the article "Learning to Predict the Secondary Structure of Globular Proteins" by N. Qian & T. J. Sejnowski. From inesc!lba%alf at relay.EU.net Thu Oct 26 12:37:33 1989 From: inesc!lba%alf at relay.EU.net (Luis Borges de Almeida) Date: Thu, 26 Oct 89 15:37:33 -0100 Subject: a note about etiquette In-Reply-To: Dave.Touretzky@B.GP.CS.CMU.EDU's message of Wed, 25 Oct 89 17:48:00 EDT <8290.625355280@DST.BOLTZ.CS.CMU.EDU> Message-ID: <8910261437.AA01425@alf.inesc.pt> Dear Dr. Touretzky, I have just read your "etiquette" message to the CONNECTIONISTS, with which I fully agree. However, I think that even people who can use e-mail reasonably well, sometimes don't notice that if they just 'reply' to a message, their reply will normally be sent both to the list and to the author of the message, causing quite a bit of undesirable traffic. Maybe it would be a good idea to explicitly discourage people from using 'reply', and to tell them to always address their response directly to the author of the message they received from the list. Regards, Luis B. Almeida From esmythe at andrew.ge.com Fri Oct 27 11:37:21 1989 From: esmythe at andrew.ge.com (Erich J Smythe) Date: Fri, 27 Oct 89 11:37:21 EDT Subject: a note about etiquette Message-ID: <8910271537.AA12133@ge-dab> If I may add one small request to the etiquette message: PLEASE put your email address somewhere in your posting. Our mailer gets confused with these messages and strips the sender's address, so If I want to respond to the author, I have to guess. Yes, I know, it's our mailer's fault (look at the path this message had to follow), but some things are too hard to change quickly ("you want us to spend money on _what_????"). Thanks -erich smythe GE Advanced Technology Labs. esmythe at atl.ge.com Moorestown, NJ From tomritch at admin.ogc.edu Fri Oct 27 12:23:25 1989 From: tomritch at admin.ogc.edu (Tom Ritch) Date: Fri, 27 Oct 89 09:23:25 PDT Subject: proteins, rna, dna refs Message-ID: <8910271623.AA22792@ogcadmin.OGC.EDU> Thanks for the info. I'll dig through it and see what I can find. Tom From Dave.Touretzky at B.GP.CS.CMU.EDU Sat Oct 28 05:23:24 1989 From: Dave.Touretzky at B.GP.CS.CMU.EDU (Dave.Touretzky@B.GP.CS.CMU.EDU) Date: Sat, 28 Oct 89 05:23:24 EDT Subject: a note about etiquette In-Reply-To: Your message of Thu, 26 Oct 89 15:37:33 -0100. <8910261437.AA01425@alf.inesc.pt> Message-ID: <12579.625569804@DST.BOLTZ.CS.CMU.EDU> I don't think people pay attention when you tell them not to use Reply. It's an ingrained habit and it's hard to make them change, although we do suggest it in the intro message sent to new subscribers. Some mailers allow users to set defaults controlling whether they reply to just the sender of a message or to the whole to/cc list. Some people set their defaults the "wrong" way. Other people are just too lazy to double check the To and Cc lines of their messages before mailing. These are the ones especially unlikely to use Mail instead of Reply. Flaming them periodically may encourage them to take more care. Cheers, -- Dave From ST401843%BROWNVM.BITNET at VMA.CC.CMU.EDU Sun Oct 29 16:23:21 1989 From: ST401843%BROWNVM.BITNET at VMA.CC.CMU.EDU (thanasis kehagias) Date: Sun, 29 Oct 89 16:23:21 EST Subject: request for recurrent nets bibliography Message-ID: I am trying to compile a bibliography of recurrent neural networks. i have traced about twenty references , which i enclose in this message . however i want to do something more complete, so i decided to go public with the following request: please send me any and all references you have on recurrent neural nets. recurrent neural nets, for my purposes, is any net of identical units that perform local computations, and where the output of units at time t is fed back to the units at time t+1. this is a fairly general definition. feel free to include references to older works of connectionist flavor. if you feel generous send me the references in the bibtex format , a sample of which is given below in my preliminary list of references. if you feel even more generous, send me a copy of the work you recommend (especially if it is by yourslef) to the surface address: Thanasis Kehagias Division of Applied Math Brown University Providence, RI 02912 if you see a work of yours in the preliminary list that is not fully or accurately listed, please send me the corrections. please send me more theoretically flavored work: if you have five application papers that use the same concepts from recurrent nets, you can send me just one that captures the essentials of the theory you use. supplemntary request: if you feel like it, send me references to work that deals with learning probability distributions, either of random variables (static) or stochastic processes (dynamic - this probably uses recurrent nets). of course, i am depending on your good will: send me as much as you can without interfering with your normal work schedule. any offers are welcome. it goes without saying that i will compile the list and make it available through connectionists for anybody who is interested. thank you very much - thansis kehagias preliminary bibliography: ========================= @ARTICLE{kn: AUTHOR= "R.E. Scneider", TITLE= "The Neuron as a Sequential Mahine", JOURNAL= "", YEAR= "", VOLUME= "" } @ARTICLE{kn: AUTHOR= "V. Rajlich", TITLE= "Dynamics of certain Discrete Systems and Self Reproduction of Patterns", JOURNAL= "", YEAR= "", VOLUME= "" } @ARTICLE{kn: AUTHOR= "R.M. Golden", TITLE= "A Unified Framework for Connectionist Systems", JOURNAL= "Biol. Cybernetics", YEAR= "1988", VOLUME= "59" } @ARTICLE{kn: AUTHOR= "L.I. Rozonoer", TITLE= "Random Logical Nets, I-III (in Russian)", JOURNAL= "Avtomatika i Telemekhanika" YEAR= "1969" VOLUME= "5" } @ARTICLE{kn: AUTHOR= "I. Parberry", TITLE= "Relating Boltzmann Machines to Conventional Models of Computation", JOURNAL= "Neural Networks", YEAR= "1989", VOLUME= "2" } @ARTICLE{kn: AUTHOR= "J.J. Hopfield", TITLE= "Neurons with Graded Response have Collective Computational Properties like those of Two-State Neurons", JOURNAL= "Proc. Nat'l Acad. Sci. USA", YEAR= "1984", VOLUME= "81" } @ARTICLE{kn: AUTHOR= "W.S. Stornetta", TITLE= "A Dynamical Approach to Temporal Pattern Processing", JOURNAL= "", YEAR= "", VOLUME= "" } @ARTICLE{kn: AUTHOR= "M.I. Jordan", TITLE= "Supervised Learning and Systems with Excess Degrees of Freedom", JOURNAL= "COINS Technical Report", YEAR= "1988", VOLUME= "88-27" } @ARTICLE{kn: AUTHOR= "F.J. Pineda", TITLE= "Generalization of Back Propagation to Recurrent Neural Nets", JOURNAL= "Physical Review Letters" , YEAR= "1987", VOLUME= "59" } @ARTICLE{kn: AUTHOR= "F.J. Pineda", TITLE= "Dynamics and Architecture for Neural Computation", JOURNAL= "Journal of Complexity", YEAR= "1988", VOLUME= "4" } @ARTICLE{kn: AUTHOR= "G.Z. Sun", TITLE= "A Recurrent Network that learns Context Free Grammars", JOURNAL= "", YEAR= "", VOLUME= "" } @ARTICLE{kn: AUTHOR= "B.A. Pearlmutter", TITLE= "Learning State Space Trajectories in Recurrent Neural Nets", JOURNAL= "", YEAR= "", VOLUME= "" } @ARTICLE{kn: AUTHOR= "J.J. Hopfield", TITLE= "Neural Nets and Physical Systems with Emergent Collective Computational Properties", JOURNAL= "Proc. Nat'l Acad. Sci. USA", YEAR= "1982", VOLUME= "" } @ARTICLE{kn: AUTHOR= "S. Amari", TITLE= "Characteristics of Random Nets of Analog JOURNAL= "IEEE SMC", YEAR= "1972", VOLUME= "SMC-2" } @ARTICLE{kn: AUTHOR= "D.H. Ackley et.al.", TITLE= "A Learning Algorithm for Boltzmann Machines", JOURNAL= "Cognitive Science", YEAR= "1985", VOLUME= "9" } @ARTICLE{kn: AUTHOR= "C. Peterson and J.R. Anderson", TITLE= "A Mean Field Theory Learning Algorithm for Neural Nets", JOURNAL= "Complex Systems", YEAR= "1987", VOLUME= "1" } @ARTICLE{kn: AUTHOR= "H. Bourlard and C.J. Wellekens", TITLE= "Links between Markov Models and Multilayer Perceptrons", JOURNAL= "Phillips Research Lab", YEAR= "1988", VOLUME= "M 263" } @ARTICLE{kn: AUTHOR= "H. Bourlard and C.J. Wellekens", TITLE= "Speech Dynamics and Recurrent Neural Nets", JOURNAL= "Proc. IEEE ICASSP", YEAR= "1989", VOLUME= "" } From bukys at cs.rochester.edu Tue Oct 31 16:22:13 1989 From: bukys at cs.rochester.edu (bukys@cs.rochester.edu) Date: Tue, 31 Oct 89 16:22:13 EST Subject: announcing: Rochester Connectionist Simulator, Version 4.2 Message-ID: <8910312122.AA23726@stork.cs.rochester.edu> The Rochester Connectionist Simulator, version 4.2, is now available by anonymous FTP from CS.Rochester.Edu, in the directory pub/simulator. (Don't forget the FTP BINARY mode when retrieving compressed files!) The simulator is too big to mail electronically, so please don't ask. The same files are available to subscribers of UUNET's UUCP service. They are stored in the directory ~uucp/pub/simulator. This new version includes an X11 interface, and it should run with little effort on Vaxen, Sun-3s, Sun-4s (but not on Sun386i machines), DECstations, and MIPS workstations. It includes various bug and documentation fixes that have been accumulating for the last 18 months. A Macintosh/MPW port of the 4.1 simulator has also been contributed for redistribution. Finally, version 4.2 adopts the licensing terms of the Free Software Foundation. If you are unable to obtain anonymous FTP or UUCP access to the simulator distribution, you can still order a copy the old-fashioned way. Send a check for US$150 (payable to the University of Rochester) to: Peg Meeker Computer Science Department University of Rochester Rochester, NY 14627 (USA) You will, in return, receive a distribution tape and a 200-page manual. PLEASE SPECIFY WHETHER YOU WANT: a 1600bpi 1/2" reel OR a QIC-24 (SUN) 1/4" cartridge. If you have a PostScript printer, you should be able to produce your own copy of the manual. If you want a paper copy of the manual anyway, send a check for $10 per manual (payable to the University of Rochester) to Peg Meeker at the above address. We do not have the facilities for generating invoices, so payment is required with any order. If you do decide to use the simulator, you should join the simulator users' mailing list, to keep up with the latest news, patches, and helpful hints. To join, drop me a note at the following address... Liudvikas Bukys