From Connectionists-Request at cs.cmu.edu Wed Nov 1 00:05:57 1995 From: Connectionists-Request at cs.cmu.edu (Connectionists-Request@cs.cmu.edu) Date: Wed, 01 Nov 95 00:05:57 EST Subject: Bi-monthly Reminder Message-ID: <25785.815202357@B.GP.CS.CMU.EDU> *** DO NOT FORWARD TO ANY OTHER LISTS *** This note was last updated September 9, 1994. This is an automatically posted bi-monthly reminder about how the CONNECTIONISTS list works and how to access various online resources. CONNECTIONISTS is a moderated forum for enlightened technical discussions and professional announcements. It is not a random free-for-all like comp.ai.neural-nets. Membership in CONNECTIONISTS is restricted to persons actively involved in neural net research. The following posting guidelines are designed to reduce the amount of irrelevant messages sent to the list. Before you post, please remember that this list is distributed to thousands of busy people who don't want their time wasted on trivia. Also, many subscribers pay cash for each kbyte; they shouldn't be forced to pay for junk mail. -- Dave Touretzky & Lisa Saksida --------------------------------------------------------------------- What to post to CONNECTIONISTS ------------------------------ - The list is primarily intended to support the discussion of technical issues relating to neural computation. - We encourage people to post the abstracts of their latest papers and tech reports. - Conferences and workshops may be announced on this list AT MOST twice: once to send out a call for papers, and once to remind non-authors about the registration deadline. A flood of repetitive announcements about the same conference is not welcome here. - Requests for ADDITIONAL references. This has been a particularly sensitive subject. Please try to (a) demonstrate that you have already pursued the quick, obvious routes to finding the information you desire, and (b) give people something back in return for bothering them. The easiest way to do both these things is to FIRST do the library work to find the basic references, then POST these as part of your query. Here's an example: WRONG WAY: "Can someone please mail me all references to cascade correlation?" RIGHT WAY: "I'm looking for references to work on cascade correlation. I've already read Fahlman's paper in NIPS 2, his NIPS 3 abstract, corresponded with him directly and retrieved the code in the nn-bench archive. Is anyone aware of additional work with this algorithm? I'll summarize and post results to the list." - Announcements of job openings related to neural computation. - Short reviews of new textbooks related to neural computation. To send mail to everyone on the list, address it to Connectionists at CS.CMU.EDU ------------------------------------------------------------------- What NOT to post to CONNECTIONISTS: ----------------------------------- - Requests for addition to the list, change of address and other administrative matters should be sent to: "Connectionists-Request at cs.cmu.edu" (note the exact spelling: many "connectionists", one "request"). If you mention our mailing list to someone who may apply to be added to it, please make sure they use the above and NOT "Connectionists at cs.cmu.edu". - Requests for e-mail addresses of people who are believed to subscribe to CONNECTIONISTS should be sent to postmaster at appropriate-site. If the site address is unknown, send your request to Connectionists-Request at cs.cmu.edu and we'll do our best to help. A phone call to the appropriate institution may sometimes be simpler and faster. - Note that in many mail programs a reply to a message is automatically "CC"-ed to all the addresses on the "To" and "CC" lines of the original message. If the mailer you use has this property, please make sure your personal response (request for a Tech Report etc.) is NOT broadcast over the net. ------------------------------------------------------------------------------- The CONNECTIONISTS Archive: --------------------------- All e-mail messages sent to "Connectionists at cs.cmu.edu" starting 27-Feb-88 are now available for public perusal. A separate file exists for each month. The files' names are: arch.yymm where yymm stand for the obvious thing. Thus the earliest available data are in the file: arch.8802 Files ending with .Z are compressed using the standard unix compress program. To browse through these files (as well as through other files, see below) you must FTP them to your local machine. The file "current" in the same directory contains the archives for the current month. ------------------------------------------------------------------------------- How to FTP Files from the CONNECTIONISTS Archive ------------------------------------------------ 1. Open an FTP connection to host B.GP.CS.CMU.EDU 2. Login as user anonymous with password your username. 3. 'cd' directly to the following directory: /afs/cs/project/connect/connect-archives The archive directory is the ONLY one you can access. You can't even find out whether any other directories exist. If you are using the 'cd' command you must cd DIRECTLY into this directory. Problems? - contact us at "Connectionists-Request at cs.cmu.edu". ------------------------------------------------------------------------------- Using Mosaic and the World Wide Web ----------------------------------- You can also access these files using the following url: http://www.cs.cmu.edu:8001/afs/cs/project/connect/connect-archives ---------------------------------------------------------------------- The NEUROPROSE Archive ---------------------- Anonymous FTP on archive.cis.ohio-state.edu (128.146.8.52) pub/neuroprose directory This directory contains technical reports as a public service to the connectionist and neural network scientific community which has an organized mailing list (for info: connectionists-request at cs.cmu.edu) Researchers may place electronic versions of their preprints in this directory, announce availability, and other interested researchers can rapidly retrieve and print the postscripts. This saves copying, postage and handling, by having the interested reader supply the paper. We strongly discourage the merger into the repository of existing bodies of work or the use of this medium as a vanity press for papers which are not of publication quality. PLACING A FILE To place a file, put it in the Inbox subdirectory, and send mail to pollack at cis.ohio-state.edu. Within a couple of days, I will move and protect it, and suggest a different name if necessary. Current naming convention is author.title.filetype.Z where title is just enough to discriminate among the files of the same author. The filetype is usually "ps" for postscript, our desired universal printing format, but may be tex, which requires more local software than a spooler. The Z indicates that the file has been compressed by the standard unix "compress" utility, which results in the .Z affix. To place or retrieve .Z files, make sure to issue the FTP command "BINARY" before transfering files. After retrieval, call the standard unix "uncompress" utility, which removes the .Z affix. An example of placing a file is in the appendix. Make sure your paper is single-spaced, so as to save paper, and include an INDEX Entry, consisting of 1) the filename, 2) the email contact for problems, 3) the number of pages and 4) a one sentence description. See the INDEX file for examples. ANNOUNCING YOUR PAPER It is the author's responsibility to invite other researchers to make copies of their paper. Before announcing, have a friend at another institution retrieve and print the file, so as to avoid easily found local postscript library errors. And let the community know how many pages to expect on their printer. Finally, information about where the paper will/might appear is appropriate inside the paper as well as in the announcement. In your subject line of your mail message, rather than "paper available via FTP," please indicate the subject or title, e.g. "paper available "Solving Towers of Hanoi with ART-4" Please add two lines to your mail header, or the top of your message, so as to facilitate the development of mailer scripts and macros which can automatically retrieve files from both NEUROPROSE and other lab-specific repositories: FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/filename.ps.Z When you announce a paper, you should consider whether (A) you want it automatically forwarded to other groups, like NEURON-DIGEST, (which gets posted to comp.ai.neural-networks) and if you want to provide (B) free or (C) prepaid hard copies for those unable to use FTP. To prevent forwarding, place a "**DO NOT FORWARD TO OTHER GROUPS**" at the top of your file. If you do offer hard copies, be prepared for a high cost. One author reported that when they allowed combination AB, the rattling around of their "free paper offer" on the worldwide data net generated over 2000 hardcopy requests! A shell script called Getps, written by Tony Plate, is in the directory, and can perform the necessary retrieval operations, given the file name. Functions for GNU Emacs RMAIL, and other mailing systems will also be posted as debugged and available. At any time, for any reason, the author may request their paper be updated or removed. For further questions contact: Jordan Pollack Associate Professor Computer Science Department Center for Complex Systems Brandeis University Phone: (617) 736-2713/* to fax Waltham, MA 02254 email: pollack at cs.brandeis.edu APPENDIX: Here is an example of naming and placing a file: unix> compress myname.title.ps unix> ftp archive.cis.ohio-state.edu Connected to archive.cis.ohio-state.edu. 220 archive.cis.ohio-state.edu FTP server ready. Name: anonymous 331 Guest login ok, send ident as password. Password:neuron 230 Guest login ok, access restrictions apply. ftp> binary 200 Type set to I. ftp> cd pub/neuroprose/Inbox 250 CWD command successful. ftp> put myname.title.ps.Z 200 PORT command successful. 150 Opening BINARY mode data connection for myname.title.ps.Z 226 Transfer complete. 100000 bytes sent in 1.414 seconds ftp> quit 221 Goodbye. unix> mail pollack at cis.ohio-state.edu Subject: file in Inbox. Jordan, I just placed the file myname.title.ps.Z in the Inbox. Here is the INDEX entry: myname.title.ps.Z mylogin at my.email.address 12 pages. A random paper which everyone will want to read Let me know when it is in place so I can announce it to Connectionists at cmu. ^D AFTER RECEIVING THE GO-AHEAD, AND HAVING A FRIEND TEST RETRIEVE THE FILE, HE DOES THE FOLLOWING: unix> mail connectionists Subject: TR announcement: Born Again Perceptrons FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/myname.title.ps.Z The file myname.title.ps.Z is now available for copying from the Neuroprose repository: Random Paper (12 pages) Somebody Somewhere Cornell University ABSTRACT: In this unpublishable paper, I generate another alternative to the back-propagation algorithm which performs 50% better on learning the exclusive-or problem. ~r.signature ^D ------------------------------------------------------------------------ How to FTP Files from the NN-Bench Collection --------------------------------------------- 1. Create an FTP connection from wherever you are to machine "ftp.cs.cmu.edu" (128.2.254.155). 2. Log in as user "anonymous" with password your username. 3. Change remote directory to "/afs/cs/project/connect/bench". Any subdirectories of this one should also be accessible. Parent directories should not be. Another valid directory is "/afs/cs/project/connect/code", where we store various supported and unsupported neural network simulators and related software. 4. At this point FTP should be able to get a listing of files in this directory and fetch the ones you want. Problems? - contact us at "neural-bench at cs.cmu.edu". From marks at u.washington.edu Wed Nov 1 19:40:55 1995 From: marks at u.washington.edu (Robert Marks) Date: Wed, 1 Nov 95 16:40:55 -0800 Subject: NN Press Release Message-ID: <9511020040.AA13754@carson.u.washington.edu> IEEE Neural Networks Council PRESS RELEASE Detroit, Michigan, November 1, 1995 Awards Committee, IEEE Neural Networks Council 1995 IEEE Neural Networks Council Pioneer Awards Professors Michael A. Arbib and Nils J. Nilsson and Dr. Paul J. Werbos have been selected to receive the 1995 IEEE Neural Networks Council Pioneer Awards. The awards will be presented at the Banquet of the 1995 IEEE International Conference on Neural Networks (ICNN '95) in Perth (at the Tumbulgum Farm), Western Australia, on Thursday November 30, 1995. The IEEE Neural Networks Council Pioneer Awards have been established to recognize and honor the vision of those people whose efforts resulted in significant contributions to the early concepts and developments in the neural networks field. 1995 marks the fifth year for this award, which is to be presented to outstanding individuals for contributions made at least fifteen years earlier. The three individuals receiving Pioneer Awards in 1995 are internationally recognized experts who have made pioneering technical contributions in the neural networks field. The following is a brief description of the awardeesU pioneering contributions that the Pioneer Award recognize and biographies which provide an overview of the distinguished careers of the awardees. Michael A. Arbib pioneered Neural Networks in Australia, writing his first paper on the subject as an undergraduate at Sydney University in 1960, and basing his first book (Brains, Machines and Mathematics, McGraw- Hill 1964) on lectures presented at the University of New South Wales. He is being honored for his pioneering work on the development of a system-theoretic approach to the brain in the early sixties. He has very actively advanced the notion that the brain is not a computer in the recent technological sense, but that we can learn much about brains from studying machines, and much about machines from studying brains. His thoughts have influenced, encouraged, and encharmed many researchers in the field of neural networks. Arbib is Professor of Computer Science, Neurobiology and Physiology, as well as of Biomedical Engineering, Electrical Engineering, and Psychology at the University of Southern California, which he joined in September of 1986. Born in England in 1940, he grew up in Australia where he had earned his B.Sc. (Hons.) from Sydney University. Dr. Arbib later moved to the U.S. where he received his Ph.D. in Mathematics from MIT in 1963, spending two years as a Research Assistant to Warren McCulloch. After five years at Stanford University (as Assistant Professor and later as Associate Professor), he became Chairman of the Department of Computer and Information Science at the University of Massachusetts at Amherst in 1970, and remained in the Department until August of 1986. Dr. Arbib currently directs a major interdisciplinary project on "Neural Plasticity: Data and Computational Structures", integrating studies of the brain with new approaches to databases, visualization, simulation, and the World Wide Web. His own research focuses on mechanisms underlying the coordination of perception and action. The author of twenty books and the editor of eleven more, Arbib has most recently edited The Handbook of Brain Theory and Neural Networks (The MIT Press, 1995). Nils J. Nilsson is being honored with the IEEE Neural Networks Council Pioneer Award for his contribution to the theory of perceptrons and learning machines. His outstanding contribution in the area of neural networks was his 1965 Pioneering book, Learning Machines: Foundations of Trainable Pattern-Classifying Systems. This was the definitive book on the subject during that decade. The book treated algorithms, learning, capacity, and multi-layer perceptrons. He did it in an accessible manner, which influenced a whole decade of research in the area. Nils J. Nilsson is Professor of Computer Science at Stanford University. Born in Saginaw, Michigan, in 1933, Nilsson's early education was in schools in Michigan and Southern California. He attended Stanford University both as an undergraduate and a graduate student and earned M.S. and Ph.D. degrees in Electrical Engineering, in 1956 and 1958, respectively. For three years after his Ph.D., he served as an Air Force lieutenant at the Rome Air Development Center in Rome, New York, where he performed research in radar signal detection. Dr. Nilsson joined SRI International (then called the Stanford Research Institute) in 1961. His early work there was on statistical and neural-network approaches to pattern recognition and led to his influential book Learning Machines: Foundations of Trainable Pattern-Classifying Systems (McGraw- Hill, 1965). Later at SRI, Dr. Nilsson became interested in broader aspects of AI which led to the publication of his two books: Problem Solving Methods in Artificial Intelligence (McGraw-Hill, 1971), and Principles of Artificial Intelligence (Morgan Kaufmann, San Francisco, CA 1980). Dr. Nilsson also led the project that developed the SRI robot RShakeyS and served as the director of the SRI Artificial Intelligence Center for the years 1980 to 1984. Professor Nilsson returned to Stanford in 1985 as the Chairman of the Department of Computer Science, a position he held until August 1990. At Stanford, he coauthored (with Michael Genesereth) the book Logical Foundations of Artificial Intelligence (Morgan Kaufmann, San Francisco, CA 1987). His most recent research is on the problem of combining deliberate (and sometimes slow) robot-reasoning processes with mechanisms for making more rapid, stero-typical responses to dynamic, time-critical situations. He is also interested in applying machine learning and adaptive computation techniques to this problem. Professor Nilsson served as the AI Area Editor for the Journal of the Association for Computing Machinery, is on the editorial board of the journal Artificial Intelligence, and is a past-president and Fellow of the American Association for Artificial Intelligence. He is also a fellow of the American Association for the Advancement of Science and has been elected as a foreign member of the Royal Swedish Academy of Engineering Sciences. He helped found and is on the board of directors of Morgan Kaufmann Publishers, Inc. Paul J. Werbos is being honored for doing much of the ground work, in the early seventies, for what has now emerged as the practical back-propagation learning algorithm in multi-layer networks; and for his continuing and sustained contributions to current advances in neurocontrol. Dr. Werbos holds four degrees from Harvard University and the London School of Economics, covering economics, mathematical physics, decision and control. His 1974 Harvard Ph.D. thesis presented the Rbackpropagation methodS for the first time, permitting the efficient calculation of derivatives and adaptation of all kinds of nonlinear sparse structures, including neural networks; it has been reprinted in its entirety in his book, The Roots of Backpropagation (Wiley, 1994) along with several related seminal and tutorial papers. In these and other more recent papers, he has described how backpropagation may be incorporated into new intelligent control designs with extensive parallels to the structure of the human brain. Dr. Werbos runs the Neuroengineering program and the SBIR Next Generation Vehicle program at the National Science Foundation. He is Past President of the International Neural Network Society, and a member of the IEEE Control and SMC Societies. Prior to NSF, he worked at the University of Maryland and the U.S. Department of Energy. He was born in 1947 near Philadelphia, Pennsylvania. His publications range from neural networks to quantum foundations, energy economics, and issues of consciousness. Mohamad H. Hassoun Professor Department of Electrical and Computer Engineering Wayne State University 5050 Anthony Wayne Drive Detroit, MI 48202 Tel. (313) 577-3920 Fax. (313) 577-1101 From malaka at ira.uka.de Thu Nov 2 05:09:47 1995 From: malaka at ira.uka.de (Rainer Malaka) Date: Thu, 02 Nov 1995 11:09:47 +0100 Subject: (p)reprints and WWW-site on olfactory modelling Message-ID: <"irafs2.ira.414:02.11.95.10.10.06"@ira.uka.de> Dear connectionists, several re- and preprints on olfactory modelling, classical conditioning and spiking neural networks are available on our WWW-server: http://i11www.ira.uka.de:80/~malaka/publications.html including the following papers: R. Malaka Dynamical odor coding in a model of the antennal lobe. In Proceedings of the International Conference on Artificial Neural Networks (ICANN`95), Paris , volume 2 Abstract: A model for the insect antennal lobe is presented. The model is embedded into a framework covering chemosensory input and associative learning of odors. The resulting dynamic representation of odors in spatio-temporal activity patterns corresponds to response patterns observed in the generalistic olfactory systems. We discuss the meaning of symmetrical and asymmetrical connections and temporal coding for classical conditioning, and demonstrate, that non-converging activity patterns can be learned and discriminated. R. Malaka, M. Hammer (1996), Real-time models of classical conditioning. Submitted to the ICNN`96 conference, Washington Abstract: Real-time models of classical conditioning simulate features of associative learning including its dependence on the timing of stimuli. We present the Sutton/Barto model, the TD model, the CP model, the drive-reinforcement model, and the SOP model in a framework of reinforcement learning rules. The role of eligibility and reinforcement is analyzed and the ability of the models to simulate time-dependent learning (e.g. inhibitory backward conditioning) and other conditioning phenomena is compared. A new model is introduced, that is mathematically simple, and overcomes weaknesses of the other models. This model combines the two antagonistic US traces of the SOP model with the reinforcement term of the TD model. R. Malaka, T. Ragg, M. Hammer (1995) A Model for Chemosensory Reception, In G. Tesauro, D.S. Touretzky, T.K. Leen (eds), Advances in Neural Information Processing Systems, Vol. 7 Abstract: A new model for chemosensory reception is presented. It models reactions between odor molecules and receptor proteins and the activation of second messenger by receptor proteins. The mathematical formulation of the reaction kinetics is transformed into an artificial neural network (ANN). The resulting feed-forward network provides a powerful means for parameter fitting by applying learning algorithms. The weights of the network corresponding to chemical parameters can be trained by presenting experimental data. We demonstrate the simulation capabilities of the model with experimental data from honey bee chemosensory neurons. It can be shown that our model is sufficient to rebuild the observed data and that simpler models are not able to do this task. R. Malaka, U. Koelsch (1994) Pattern Segmentation in Recurrent Networks of Biologically Plausible Neural Elements. In Intelligent Engineering Systems Through Artificial Neural Networks, Vol. 4 Abstract: We introduce a neural network model using spiking neurons. The neuron model is a biological neuron with Hodgkin-Huxley channels. We compare the network's ability of auto-associative pattern recognition with to that of the Hopfield network. The model recognizes patterns by converging into dynamic stable states of synchonous firing activity. This activity can last for arbitrary time or return to a resting activation after stimulus offset. If one presents overlayed patterns to the network, the network is able to separate the components. The single components are encoded by synchronous firing patterns. and some others. Yours, Rainer Malaka ------------------------------------------------------------------------------ Rainer Malaka /| phone: (+49) (721) 608-4212 Universitaet Karlsruhe | | /| fax : (+49) (721) 608-4211 Institut fuer Logik, Komplexitaet /||/ | | csnet: malaka at ira.uka.de und Deduktionssysteme | | /||/ P.O.-Box 6980 |/ | | WWW : D-76128 Karlsruhe, Germany |/ http://i11www.ira.uka.de/~malaka/ ------------------------------------------------------------------------------ From marwan at sedal.usyd.edu.AU Thu Nov 2 09:10:08 1995 From: marwan at sedal.usyd.edu.AU (Marwan A. Jabri, Sydney Univ. Elec. Eng., Tel: +61-2 692 2240) Date: Fri, 3 Nov 1995 01:10:08 +1100 Subject: Girling Watson Research Fellowship Message-ID: <199511021410.BAA08045@sedal.sedal.su.OZ.AU> GIRLING WATSON RESEARCH FELLOWSHIP (Renewable) Reference No. B42/01 Systems Engineering & Design Automation Laboratory Department of Electrical Engineering The University of Sydney Applications are invited for a Girling Watson Research Fellowship at Sydney University Electrical Engineering. The Fellow will work with SEDAL (Systems Engineering and Design Automation Laboratory). SEDAL has research projects on optical character recognition, adiabatic computing, pattern recognition for implantable devices, VLSI implementation of connectionist architectures, intra-cardiac electrogram classification, software and hardware implementations for video-telephony, pattern recognition for cochlear implants, audio localisation and time series prediction. SEDAL collaborates with Australian and multinational companies in many of these research projects. The Fellow should have a PhD and an excellent research background in one of the following areas: Pattern recognition and analysis; connectionist architectures; analog and digital microelectronics; or time series modelling. The Fellow is expected to play a leading role in research, post graduate student supervision and at providing occasional teaching support in his/her area of expertise. The appointment is available for a period of three years. There is the possibility of further offers of employment for another three years, subject to funding and need. Membership of a University approved superannuation scheme is a condition of employment for new appointees. For further information contact Marwan Jabri tel (+61-2) 351 2240, fax 660 1228, Email: marwan at sedal.su.oz.au Salary: Research Fellow $42,198 - $50,111 p.a. Closing: 7 December 1995 Applications should be sent to the Personeel Services at the address below. Applications should quote the reference number B42/01, a CV describing research achievements and the names, addresses, teplephones, fax and email of three referees who can comment on research performance, Personnel Services Academic Group B J13 The University of Sydney NSW 2006 Australia From maja at cs.brandeis.edu Thu Nov 2 10:28:44 1995 From: maja at cs.brandeis.edu (Maja Mataric) Date: Thu, 2 Nov 1995 10:28:44 -0500 Subject: CFP - Adaptive Behavior Journal Message-ID: <199511021528.KAA13997@garnet.cs.brandeis.edu> CALL FOR PAPERS (http://www.cs.brandeis.edu:80/~maja/abj-special-issue/) ADAPTIVE BEHAVIOR Journal Special Issue on COMPLETE AGENT LEARNING IN COMPLEX ENVIRONMENTS Guest editors: Maja J Mataric Submission Deadline: June 1, 1996. Adaptive Behavior is an international journal published by MIT Press; Editor-in-Chief: Jean-Arcady Meyer, Ecole Normale Superieure, Paris. In the last decade, the problems being treated in AI, Alife, and Robotics have witnessed an increase in complexity as the domains under investigation have transitioned from theoretically clean scenarios to more complex dynamic environments. Agents that must adapt in environments such as the physical world, an active ecology or economy, and the World Wide Web, challenge traditional assumptions and approaches to learning. As a consequence, novel methods for automated adaptation, action selection, and new behavior acquisition have become the focus of much research in the field. This special issue of Adaptive Behavior will focus on situated agent learning in challenging environments that feature noise, uncertainty, and complex dynamics. We are soliciting papers describing finished work on autonomous learning and adaptation during the lifetime of a complete agent situated in a dynamic environment. We encourage submissions that address several of the following topics within a whole agent learning system: * learning from ambiguous perceptual inputs * learning with noisy/uncertain action/motor outputs * learning from sparse, irregular, inconsistent, and noisy reinforcement/feedback * learning in real time * combining built-in and learned knowledge * learning in complex environments requiring generalization in state representation * learning from incremental and delayed feedback * learning in smoothly or discontinuously changing environments We invite submissions from all areas in AI, Alife, and Robotics that treat either complete synthetic systems or models of biological adaptive systems situated in complex environments. Submitted papers should be delivered by June 1, 1996. Authors intending to submit a manuscript should contact the guest editor as soon as possible to discuss paper ideas and suitability for this issue. Use maja at cs.brandeis.edu or tel: (617) 736-2708 or fax: (617) 736-2741. Manuscripts should be typed or laser-printed in English (with American spelling preferred) and double-spaced. Both paper and electronic submission are possible, as described below. Copies of the complete Adaptive Behavior Instructions to Contributors are available on request--also see the Adaptive Behavior journal's home page at: http://www.ens.fr:80/bioinfo/www/francais/AB.html. For paper submissions, send five (5) copies of submitted papers (hard-copy only) to: Maja Mataric Volen Center for Complex Systems Computer Science Department Brandeis University Waltham, MA 02254-9110, USA For electronic submissions, use Postscript format, ftp the file to ftp.cs.brandeis.edu/incoming, and send an email notification to maja at cs.brandeis.edu. For a Web page of this call, and detailed ftp directions, see: http://www.cs.brandeis.edu/~maja/abj-special-issue/ \end{document} From szepes at sol.cc.u-szeged.hu Thu Nov 2 09:02:44 1995 From: szepes at sol.cc.u-szeged.hu (Csaba Szepesvari) Date: Thu, 2 Nov 1995 14:02:44 +0000 Subject: PA: Approximate Geometry Representations and Sensory Fusion Message-ID: <9511021303.AA22334@sol.cc.u-szeged.hu> *****************Pre-print Available via FTP ******************* URL ftp:// iserv.iki.kfki.hu/pub/papers/szepes.fusion.ps.Z WWW http://iserv.iki.kfki.hu/adaptlab.html Title: Approximate Geometry Representations and Sensory Fusion Keywords: self-organizing networks, sensory fusion, geometry representation, topographical mapping, Kohonen network Csaba Szepesvari^* Andras Lorincz Adaptive Systems Laboratory, Insitute of Isotopes, Hungarian Academny of Sciences *Bolyai Institute of Mathematics, Jozsef Attila University of Szeged Knowledge of the geometry of the world external to a system is essential in cases such as navigation when for predicting the trajectories of moving objects. It may also play a role in recognition tasks, particularly when the procedures used for image segmentation and feature extraction utilize information on the geometry. A more abstract example is function approximation, where this information is used to create better interpolation. This paper summarizes the recent advances in the theory of self-organizing development of approximate geometry representations based on the use of neural networks. Part of this work is based on the theoretical approach of (Szepesvari, 1993), which is different from that of (Martinetz, 1993) and also is somewhat more general. The Martinetz approach treats signals provided by artificial neuron-like entities whereas the present work uses the entities of the external world as its starting point. The relationship between the present work and the Martinetz approach will be detailed. We approach the problem of approximate geometry representations by first examining the problem of sensory fusion, i.e., the problem of fusing information from different transductors. A straightforward solution is the simultaneous discretization of the output of all transductors, which means the discretization of a space defined as the product of the individual transductor output spaces. However, the geometry relations are defined for the **external world** only, so it is still an open question how to define the metrics on the product of output spaces. It will be shown that **simple Hebbian learning** can result in the formation of a correct geometry representation. Some mathematical considerations will be presented to help us clarify the underlying concepts and assumptions. The mathematical framework gives rise to a corollary on the "topographical mappings" realized by Kohonen networks. In fact, the present work as well as (Martinetz, 1993) may be considered as a generalization of Kohonen's topographic maps. We develop topographic maps with self-organizing interneural connections. *****************Pre-print Available via FTP ******************* ========================================================================== Csaba Szepesvari ---------------------------------------+---------------------------------- Bolyai Institute of Mathematics |e-mail: szepes at math.u-szeged.hu "Jozsef Attila" University of Szeged |http://www.inf.u-szeged.hu/~szepes Szeged 6720 | Aradi vrt tere 1. | HUNGARY | Tel.: (36-62) 311-622/3706 |phone at home (tel/fax): Tel/Fax: (36-62) 326-246 | (36-62) 494-225 ---------------------------------------+---------------------------------- From dnoelle at cs.ucsd.edu Thu Nov 2 20:00:49 1995 From: dnoelle at cs.ucsd.edu (David Noelle) Date: Thu, 2 Nov 95 17:00:49 -0800 Subject: Cognitive Science '96 Message-ID: <9511030100.AA22084@beowulf> Eighteenth Annual Conference of the COGNITIVE SCIENCE SOCIETY July 12-15, 1996 University of California, San Diego La Jolla, California CALL FOR PAPERS DUE DATE: Thursday, February 1, 1996 The Annual Cognitive Science Conference began with the La Jolla Conference on Cognitive Science in August of 1979. The organizing committee of the Eighteenth Annual Conference would like to welcome members home to La Jolla. We plan to recapture the pioneering spirit of the original conference, extending our welcome to fields on the expanding frontier of Cognitive Science, including Artificial Life, Cognitive and Computational Neuroscience, Evolutionary Psychology, as well as the core areas of Anthropology, Computer Science, Linguistics, Neuroscience, Philosophy, and Psychology. As a change this year, we follow the example of Psychonomics and the Neuroscience Conferences and invite Members of the Society to submit one-page abstracts for guaranteed poster presentation at the conference. A second change is that all papers accepted as posters will only get one page in the proceedings. The conference will feature plenary addresses by invited speakers, invited symposia by leaders in their fields, technical paper and a poster sessions, a banquet, and a Blues Party. San Diego is the home of the world-famous San Diego Zoo and Wild Animal Park, Sea World, the historic all-wooden Hotel Del Coronado, beautiful beaches, mountain areas and deserts, is a short drive from Mexico, and features a high Cappuccino Index. Bring the whole family and stay a while! GUIDELINES FOR PAPER SUBMISSIONS Novel research papers are invited on any topic related to cognition. Members of the Society may submit a one page abstract for poster presentation, which will be automatically accepted for publication in the proceedings. Submitted full-length papers will be evaluated through peer review with respect to several criteria, including originality, quality, and significance of research, relevance to a broad audience of cognitive science researchers, and clarity of presentation. Papers will either be accepted for publication in the proceedings, or (if the author is a Society member) will be accepted as a poster, and a one-page abstract will be published. Such authors will get a chance to flesh out the abstract to a page when submitting their camera ready copy. Poster abstracts from non-members will be accepted, but the presenter should join the Society prior to presenting the poster. Accepted papers will be presented at the conference as talks. Papers may present results from completed research as well as report on current research with an emphasis on novel approaches, methods, ideas, and perspectives. Posters may report on recent work to be published elsewhere that has not been previously presented at the conference. Authors should submit five (5) copies of the paper in hard copy form by Thursday, February 1, 1996, to: Dr. Garrison W. Cottrell Computer Science and Engineering 0114 FED EX ONLY: 3250 Applied Physics and Math University of California San Diego La Jolla, Ca. 92093-0114 phone for FED EX: 619-534-5948 (my secretary, Marie Kreider) If confirmation of receipt is desired, please use certified mail or enclose a self-addressed stamped envelope or postcard. DAVID MARR MEMORIAL PRIZES FOR EXCELLENT STUDENT PAPERS Papers with a student first author are eligible to compete for a David Marr Memorial Prize for excellence in research and presentation. The David Marr Prizes are accompanied by a $300.00 honorarium, and are funded by an anonymous donor. LENGTH Papers must be a maximum of eleven (11) pages long (excluding only the cover page but including figures and references), with 1 inch margins on all sides (i.e., the text should be 6.5 inches by 9 inches, including footnotes but excluding page numbers), double-spaced, and in 12-point type. Each page should be numbered (excluding the cover page). Template and style files conforming to these specifications for several text formatting programs, including LaTeX, Framemaker, Word, Word Perfect, and HTML will be available by anonymous FTP after November 15th, 1995. (Check "http://www.cse.ucsd.edu/events/cogsci96/" for details). Submitted abstracts should be one page in length, with the same margins as full papers. Style files for these will be available at the same location as above. Final versions of papers and poster abstracts will be required only after authors are notified of acceptance; accepted papers will be published in a CD-ROM version of the proceedings. Abstracts will be available before the meeting from a WWW server. Final versions must follow the HTML style guidelines referred to above. This year we will continue to publish the proceedings in two modalities, paper and a CD-ROM version. When the procedures for efficient HTML submission stabilize, we will be switching from paper to CD-ROM publication in order to control escalating costs and permit use of search software. [Comments on this change should be directed to "alan at lrdc4.lrdc.pitt.edu" (Alan Lesgold, Secretary/Treasurer).] COVER PAGE Each copy of the submitted paper must include a cover page, separate from the body of the paper, which includes: 1. Title of paper. 2. Full names, postal addresses, phone numbers, and e-mail addresses of all authors. 3. An abstract of no more than 200 words. 4. Three to five keywords in decreasing order of relevance. The keywords will be used in the index for the proceedings. 5. Preference for presentation format: Talk or poster, talk only, poster only. Poster only submissions should follow paper format, but be no more than 2 pages in this format (final poster abstracts will follow the same 2 column format as papers). Accepted papers will be presented as talks. Submitted posters by Society Members will be accepted for poster presentation, but may, at the discretion of the Program Committee, be invited for oral presentation. Non-members may join the Society at the time of submission. 6. A note stating if the paper is eligible to compete for a Marr Prize. DEADLINE Papers must be received by Thursday, February 1, 1996. Papers received after this date will be recycled. CALL FOR SYMPOSIA In addition to technical papers, posters, plenary sessions, and invited symposia, the conference will accept submitted research symposia. Proposals for symposia are invited and should indicate: 1. A brief description of the topic; 2. How the symposium would address a broad cognitive science audience, and some evidence of interest; 3. Names of symposium organizer(s); 4. List of potential speakers, their topics, and some estimate of their likelihood of participation; 5. Proposed symposium format (designed to last 90 minutes). Symposium proposals should be sent as soon as possible, but no later than January 1, 1996. Abstracts of the symposium talks will be included in the proceedings and must be made available in HTML format, as above. CONFERENCE CHAIRS Edwin Hutchins and Walter Savitch hutchins at cogsci.ucsd.edu savitch at cogsci.ucsd.edu PROGRAM CHAIR Garrison W. Cottrell gary at cs.ucsd.edu From steven.young at psy.ox.ac.uk Fri Nov 3 09:40:37 1995 From: steven.young at psy.ox.ac.uk (Steven Young) Date: Fri, 3 Nov 1995 14:40:37 +0000 (GMT) Subject: Oxford Summer School on Connectionist Modelling Message-ID: <199511031440.OAA04054@cogsci2.psych.ox.ac.uk> The call for participation for the 1996 Oxford Summer School on Connectionist Modelling follows. Please pass on this information to people you know you would be interested. -------- OXFORD SUMMER SCHOOL ON CONNECTIONIST MODELLING Department of Experimental Psychology University of Oxford 21 July - 2nd August 1996 Applications are invited for participation in a 2-week residential Summer School on techniques in connectionist modelling. The course is aimed primarily at researchers who wish to exploit neural network models in their teaching and/or research and it will provide a general introduction to connectionist modelling through lectures and exercises on Power PCs. The course is interdisciplinary in content though many of the illustrative examples are taken from cognitive and developmental psychology, and cognitive neuroscience. The instructors with primary responsibility for teaching the course are Kim Plunkett and Edmund Rolls. No prior knowledge of computational modelling will be required though simple word processing skills will be assumed. Participants will be encouraged to start work on their own modelling projects during the Summer School. The cost of participation in the Summer School is #750 to include accommodation (bed and breakfast at St. John's College) and registration. Participants will be expected to cover their own travel and meal costs. A small number of graduate student scholarships providing partial funding may be available. Applicants should indicate whether they wish to be considered for a graduate student scholarship but are advised to seek their own funding as well, since in previous years the number of graduate student applications has far exceeded the number of scholarships available. There is a Summer School World Wide Web page describing the contents of the 1995 Summer School available on: http://cogsci1.psych.ox.ac.uk/summer-school/ Further information about contents of the course can be obtained from Steven.Young at psy.ox.ac.uk If you are interested in participating in the Summer School, please contact: Mrs Sue King Department of Experimental Psychology University of Oxford South Parks Road Oxford OX1 3UD Tel: (01865) 271353 Email: susan.king at psy.oxford.ac.uk Please send a brief description of your background with an explanation of why you would like to attend the Summer School (one page maximum) no later than 31st January 1996. Regards, Steven Young. -- Facility for Computational Modelling in Cognitive Science McDonnell-Pew Centre for Cognitive Neuroscience, Oxford From Dave_Touretzky at DST.BOLTZ.CS.CMU.EDU Sat Nov 4 05:06:55 1995 From: Dave_Touretzky at DST.BOLTZ.CS.CMU.EDU (Dave_Touretzky@DST.BOLTZ.CS.CMU.EDU) Date: Sat, 04 Nov 95 05:06:55 EST Subject: Neural Processes in Cognition training program Message-ID: <9295.815479615@DST.BOLTZ.CS.CMU.EDU> Applications are being accepted for both pre- and postdoctoral training in Neural Processes in Cognition, a joint program of the Center for the Neural Basis of Cognition operated by the University of Pittsburgh, its School of Medicine, and Carnegie Mellon University. This is an interdisciplinary program investigating the neurobiology of cognition, and utilizing neuroanatomical, neurophysiological, behavioral, and computer simulation techniques. Some of the departments offering training as part of the Neural Processes in Cognition program are: CMU: Psychology, Computer Science, Biological Sciences, Robotics University of Pittsburgh: Psychology, Neuroscience, Neurobiology, Mathematics, Information Science, Radiology, Neurology Research facilities include: computerized microscopy, human and animal electrophysiological instrumentation, behavioral assessment laboratories, brain imaging, the Pittsburgh Supercomputing Center, and access to human clinical populations. The application deadline is February 1, 1996. Additional details about the Center for the Neural Basis of Cognition, the Neural Processes in Cognition program, and how to apply to the program can be found on the World-Wide Web: http://neurocog.lrdc.pitt.edu/npc/ http://www.cs.cmu.edu/Web/Groups/CNBC/CNBC.html For more information contact: Professor Walter Schneider University of Pittsburgh 3939 O'Hara Street Pittsburgh, PA 15260 tel. 412-624-7061 E-mail: NEUROCOG at PITTVMS.BITNET From b-wah at uiuc.edu Sun Nov 5 15:03:58 1995 From: b-wah at uiuc.edu (Benjamin Wah) Date: Sun, 05 Nov 95 14:03:58 CST Subject: ICNN'96 Extension of Submission Deadline Message-ID: <199511052003.AA06874@teacher.crhc.uiuc.edu> International Conference on Neural Networks IMPORTANT EXTENSION Deadline for submission to ICNN'96 has been extended to December 29, 1995 Authors interested to submit papers must have their papers received by the Program Chair (see address to the left) by the deadline. Papers received after that date will be returned unopened. Papers submitted must be in final publishable form and will be reviewed by senior researchers in the field using the same standard as papers submitted before the October 16, 1995, deadline. However, papers submitted after the October 16, 1995, deadline will be either accepted or rejected, and authors of accepted papers will not have a chance to revise their papers. (Authors of accepted papers submitted before the October 16 deadline will be allowed to revise their papers.) Authors submitting papers late will be notified of the final decision by February 15, 1996. Six copies (one original and five copies) of the paper must be submitted. Papers must be camera-ready on 8 1/2-by-11 white paper, one-column format in Times or similar font style, 10 points or larger with one inch margins on all four sides. Do not fold or staple the original camera-ready copy. Four pages are encouraged; however, the paper must not exceed six pages, including figures, tables, and refer- ences, and should be written in English. Submissions that do not adhere to the guidelines above will be returned unre- viewed. Centered at the top of the first page should be the complete title, author name(s) and postal and electronic mailing addresses. In the accompanying letter, the following infor- mation must be included: (a) full title of paper; (b) pre- sentation preferred (oral or poster); (c) audio visual requirements (e.g. 35 mm slide, OHP, VCR); (d) corresponding author (name, postal and e-mail addresses, telephone & fax numbers); and (e) presenter (name, postal and e-mail addresses, telephone & fax numbers). For further information on ICNN'96, please consult our World Wide Web home page at http://www-ece.rice.edu/96icnn, or send electronic mail to icnn96 at manip.crhc.uiuc.edu. From pau at ac.upc.es Mon Nov 6 11:03:33 1995 From: pau at ac.upc.es (Pau Bofill) Date: Mon, 6 Nov 1995 16:03:33 +0000 (METDST) Subject: No Free Lunch? (Delayed reply) Message-ID: <9511061603.AA05736@gaudi.ac.upc.es> (I've just been added to the Connectionists list, and therefore I apologize if I'm repeating something that's been said before.) I happened to read the message from Bill Macready on Free Lunch Theorems for Search, and I was surprised by his (and D.H. Wolpert's) statement that "any two search algorithms have exactly the same expected performance, over the space of all fitness functions." which seemed to invalidate statements like "my search algorithm beats your search algorithm according to a particular performance measure for the following fitness function." Thus, I picked the papers anounced on that message (see below) and tryied to find out what did they mean. As far as I can see, the goal of a particular optimization PROBLEM, is to find a target POINT in search space whith specific properties. The role of any fitness function, then, is to assign a cost value to each point in search space in such a way that it helps the algorithm beeing used to find the target point. In particular, for optimization purposes, a necessary condition for the validity of a fitness function is that it assigns the maximum (or minimum) cost value to all valid target points, and only to target points. If I understood them properly, Wolpert and Macready define the space of all possible fitness functions as the set of ALL possible assignments of cost values to points in search space. And they use as a generalized perfomace measure the histogram of cost values, regardless of the points where they were found. Then, if what I understood is right, they are ignoring the previous necessary condition on fitness functions and their statement is equivalent to, "any two search algorithms have exactly the same expected performance, over the space of all optimization problems that can be defined whithin a search space." which stated otherwise would mean, "If the target doesn't matter, all algorithms perform alike." I don't doubt that Wolpert & Macready's "No Free Lunch Theorem" can be useful as a tool for deriving further results, but one should be carefull when considering its "physical" meaning. Thus, I believe it is very important to define carefully the meaning of, "My algorithm performs better than yours." In particular, algorithms should be compared on PROBLEMS, not on fitness functions, with properly defined performance measures. Probably, the most fair one-to-one comparision would use the best (found) fitness function for each algorithm (finding the best fitness function for a particular problem AND algorithm is, in turn, an optimization problem). Averaging over algorithms in order to measure problem hardness is "dangerous" in the same sense. Pau Bofill Universitat Politecnica de Catalunya. ************************************************************************** The papers mentioned are at ftp.santafe.edu /pub/wgm/nfl.ps /hard.ps From esann at dice.ucl.ac.be Mon Nov 6 13:52:37 1995 From: esann at dice.ucl.ac.be (esann@dice.ucl.ac.be) Date: Mon, 6 Nov 1995 19:52:37 +0100 Subject: ESANN'96 Annoucement and Call for Papers Message-ID: <199511061850.TAA10706@ns1.dice.ucl.ac.be> Dear Colleagues, You will find below the Announcement and Call for Papers of ESANN'96, the fourth European Symposium on Artificial Neural Networks, which will be held in Bruges (Belgium), on April 24-26, 1996. All information concerning this conference is also available on the following servers: - WWW: http://www.dice.ucl.ac.be/neural-nets/ESANN/ESANN'96.html - FTP: server: ftp.dice.ucl.ac.be directory: /pub/neural-nets/ESANN login: anonymous password: your e-mail address Please don't hesitate to connect to these servers, and to contact the conference secretariat (see address below) for any supplementary information. Looking forward to meeting you during ESANN'96, Sincerely yours, Michel Verleysen _____________________________________________________________________ ******************************************************** * ESANN96 * * * * 4th European Symposium on Artificial Neural Networks * * Bruges - April 24-25-26, 1996 * * * ******************************************************** First announcement and call for papers ______________________________________ Invitation and Scope of the conference ______________________________________ The fourth European Symposium on Artificial Neural Networks will be organized in Bruges, Belgium, in April 1996. The three first successful editions, organized in Brussels, each gathered between 90 and 110 scientists, coming from Western and Eastern Europe, but also from USA, Japan, Australia, New Zealand, South America... The field of Artificial Neural Networks includes a lot of different disciplines, from mathematics and statistics to robotics and electronics. For this reason, actual studies concern various aspects of the field, sometimes to the detriment of strong, well established foundations for these researches; it is obvious that a better knowledge of the basic concepts and opportunities of neurocomputing, and more effective comparisons with other computing methods are strongly needed for a profitable long-term use of neural networks in applications. The purpose of the ESANN conferences is to present the latest results in the fundamental aspects of artificial neural networks. Invited and survey talks will also present a comprehensive view of particular topics of the conference. The program committee of ESANN'96 welcomes papers covering new results or being of tutorial nature, and dealing with theoretical, biological or mathematical aspects of artificial neural networks, or with the relations between neural networks and other fields. Presentation of results from large research project (ESPRIT,...) is also encouraged. The following is a non-exhaustive list of topics which will be covered during ESANN'96 : + theory + models and architectures + mathematics + learning algorithms + biologically plausible artificial networks + formal models of biological phenomena + neurobiological systems + approximation of functions + identification of non-linear dynamic systems + adaptive behavior + adaptive control + signal processing + statistics + self-organization + evolutive learning Invited talks will cover several of the above topics; an invited talk will be given by Prof. Nicolas Franceschini (CNRS Marseilles, France). Other invited talks are to be announced. The conference will be held in Bruges (also called "Venice of the North"), one of the most beautiful towns in Europe. Bruges can be reached by train from Brussels in less than one hour (frequent trains). The town of Bruges is worldwide known, and famous both by its architectural style, its canals, and its pleasant atmosphere. Call for contributions ______________________ Prospective authors are invited to submit six originals of their contribution before December 8, 1995. Working language of the conference (including proceedings) is English. Papers should not exceed six A4 pages (including figures and references). Printing area will be 12.2 x 19.3 cm (centered on the A4 page); left, right, top and bottom margins will thus respectively be 4.4, 4.4, 5.2 and 5.2 cm. 10-point Times font will be used for the main text; headings will be in bold characters (but not underlined), and will be separated from the main text by two blank lines before and one after. Manuscripts prepared in this format will be reproduced in the same size in the book. The first page will be typed in the format indicated in the figure file ESANN_format.eps that can be found on the anonymous FTP ESANN'96 server (see below). The next pages will be similar, except the heading which will be omitted. Originals of the figures will be pasted into the manuscript and centered between the margins. The lettering of the figures should be in 10-point Times font size. Figures should be numbered. The legends also should be centered between the margins and be written in 9-point Times font size as follows: Fig. 3. Text follows ... The pages of the manuscript will not be numbered (numbering decided by the editor). A separate page (not included in the manuscript) will indicate: + the title of the manuscript + author(s) name(s) + the complete address (including phone & fax numbers and E-mail) of the corresponding author + a list of five keywords or topics On the same page, the authors will copy and sign the following paragraph: "in case of acceptation of the paper for presentation at ESANN 96: - at least one of the authors will register to the conference and will present the paper - the author(s) give their rights up over the paper to the organizers of ESANN 96, for the proceedings and any publication that could directly be generated by the conference - if the paper does not match the format requirements for the proceedings, the author(s) will send a revised version within two weeks of the notification of acceptation." Contributions must be sent to the conference secretariat. Examples of camera-ready contributions can be obtained by writing to the same address. Registration fees _________________ registration before registration after February 1st, 1996 February 1st, 1996 Universities BEF 15500 BEF 16500 Industries BEF 19500 BEF 20500 An "advanced registration form" is available by writing to the conference secretariat (see reply form below). Please ask for this form in order to benefit from the reduced registration fee before February 1st, 1996. Deadlines _________ Submission of papers December 8, 1995 Notification of acceptance January 31, 1996 Symposium April 24-261, 1996 Grants ______ We regret that no grants are available this year, because of a lack of funding from the European Community. Conference secretariat ______________________ Dr. Michel Verleysen D facto conference services 45 rue Masui B - 1210 Brussels (Belgium) phone: + 32 2 245 43 63 Fax: + 32 2 245 46 94 E-mail: esann at dice.ucl.ac.be Information is also available through WWW and anonymous FTP on the following sites: URL: http://www.dice.ucl.ac.be/neural-nets/ESANN/ESANN96.html FTP: ftp.dice.ucl.ac.be, directory /pub/neural-nets/ESANN Steering committee __________________ Francois Blayo - Univ. Paris I (F) Marie Cottrell - Univ. Paris I (F) Jeanny Herault - INPG Grenoble (F) Joos Vandewalle - KUL Leuven (B) Michel Verleysen - UCL Louvain-la-Neuve (B) Scientific committee ____________________ (to be confirmed) Agnes Babloyantz - Univ. Libre Bruxelles (Belgium) Herve Bourlard - ICSI Berkeley (USA) Joan Cabestany - Univ. Polit. de Catalunya (E) Dave Cliff - University of Sussex (UK) Pierre Comon - Thomson-Sintra Sophia Antipolis (F) Holk Cruse - Universitat Bielefeld (D) Dante Del Corso - Politecnico di Torino (I) Wlodek Duch - Nicholas Copernicus Univ. (PL) Marc Duranton - Philips / LEP (F) Jean-Claude Fort - Universite Nancy I (F) Bernd Fritzke - Ruhr-Universitat Bochum (D) Karl Goser - Universitat Dortmund (D) Manuel Grana - UPV San Sebastian (E) Martin Hasler - EPFL Lausanne (CH) Kurt Hornik - Techische Univ. Wien (A) Christian Jutten - INPG Grenoble (F) Vera Kurkova - Acad. of Science of the Czech Rep. (CZ) Petr Lansky - Acad. of Science of the Czech Rep. (CZ) Jean-Didier Legat - UCL Louvain-la-Neuve (B) Hans-Peter Mallot - Max-Planck Institut (D) Eddy Mayoraz - IDIAP Martigny (CH) Jean Arcady Meyer - Ecole Normale Superieure Paris (F) Jose Mira-Mira - UNED (E) Pietro Morasso - Univ. of Genoa (I) Jean-Pierre Nadal - Ecole Normale Superieure Paris (F) Erkki Oja - Helsinky University of Technology (FIN) Gilles Pages - Universite Paris VI (F) Helene Paugam-Moisy - Ecole Normale Superieure Lyon (F) Alberto Prieto - Universitad de Granada (E) Pierre Puget - LETI Grenoble (F) Ronan Reilly - University College Dublin (IRE) Tamas Roska - Hungarian Academy of Science (H) Jean-Pierre Rospars - INRA Versailles (F) Jean-Pierre Royet - Universite Lyon 1 (F) John Stonham - Brunel University (UK) John Taylor - King's College London (UK) Vincent Torre - Universita di Genova (I) Claude Touzet - IUSPIM Marseilles (F) Marc Van Hulle - KUL Leuven (B) Christian Wellekens - Eurecom Sophia-Antipolis (F) Reply form __________ If you wish to receive the final program of ESANN'96, for any address change, or to add one of your colleagues in our database, please send this form to the conference secretariat: D facto conference services 45 rue Masui B - 1210 Brussels (Belgium) phone: + 32 2 245 43 63 Fax: + 32 2 245 46 94 E-mail: esann at dice.ucl.ac.be Please indicate if you wish to receive the advanced registration form. ---------------------------------------------------------------- Name: .......................................................... First Name: .................................................... University or Company: ......................................... Address: ....................................................... ZIP: ................... Town: ................................ Country: ....................................................... Tel: ........................................................... Fax: ........................................................... E-mail: ........................................................ O Please send me the "advanced registration form" ---------------------------------------------------------------- _____________________________ D facto publications - conference services 45 rue Masui 1210 Brussels Belgium tel: +32 2 245 43 63 fax: +32 2 245 46 94 _____________________________ From jacobs at psych.stanford.edu Tue Nov 7 12:58:05 1995 From: jacobs at psych.stanford.edu (Robert Jacobs) Date: Tue, 7 Nov 1995 09:58:05 -0800 (PST) Subject: paper available: Gibbs sampling for HME Message-ID: <199511071758.JAA12308@aragorn.Stanford.EDU> The following paper is available via anonymous ftp from the neuroprose archive. The paper has been accepted for publication in the "Journal of the American Statistical Association." The manuscript is 26 pages. (Unfortunately, hardcopies are not available.) FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/jacobs.hme_gibbs.ps.Z Bayesian Inference in Mixtures-of-Experts and Hierarchical Mixtures-of-Experts Models With an Application to Speech Recognition Fengchun Peng, Robert A. Jacobs, and Martin A. Tanner Machine classification of acoustic waveforms as speech events is often difficult due to context-dependencies. A vowel recognition task with multiple speakers is studied in this paper via the use of a class of modular and hierarchical systems referred to as mixtures-of-experts and hierarchical mixtures-of-experts models. The statistical model underlying the systems is a mixture model in which both the mixture coefficients and the mixture components are generalized linear models. A full Bayesian approach is used as a basis of inference and prediction. Computations are performed using Markov chain Monte Carlo methods. A key benefit of this approach is the ability to obtain a sample from the posterior distribution of any functional of the parameters of the given model. In this way, more information is obtained than provided by a point estimate. Also avoided is the need to rely on a normal approximation to the posterior as the basis of inference. This is particularly important in cases where the posterior is skewed or multimodal. Comparisons between a hierarchical mixtures-of-experts model and other pattern classification systems on the vowel recognition task are reported. The results indicate that this model showed good classification performance, and also gave the additional benefit of providing for the opportunity to assess the degree of certainty of the model in its classification predictions. From esann at dice.ucl.ac.be Tue Nov 7 13:24:14 1995 From: esann at dice.ucl.ac.be (esann@dice.ucl.ac.be) Date: Tue, 7 Nov 1995 19:24:14 +0100 Subject: mistake in ESANN'96 address Message-ID: <199511071822.TAA05245@ns1.dice.ucl.ac.be> Dear Colleagues, This message is to bring to your attention that the WWW server address included in the message announcing the ESANN'96 conference was wrong. The ESANN'96 conference (European Symposium on Artificial Neural Networks) will be held in Bruges (Belgium) on April 24-25-26, 1996. All information can be obtained through FTP or WWW, at the following addresses: - WWW: http://www.dice.ucl.ac.be/neural-nets/ESANN/ESANN96.html - FTP: server: ftp.dice.ucl.ac.be directory: /pub/neural-nets/ESANN login: anonymous password: your e-mail address The WWW address in the previous message: - WWW: http://www.dice.ucl.ac.be/neural-nets/ESANN/ESANN'96.html was WRONG !!! Sorry for the inconvenience. Sincerely yours, Michel Verleysen _____________________________ D facto publications - conference services 45 rue Masui 1210 Brussels Belgium tel: +32 2 245 43 63 fax: +32 2 245 46 94 _____________________________ From pjs at aig.jpl.nasa.gov Tue Nov 7 12:46:39 1995 From: pjs at aig.jpl.nasa.gov (Padhraic J. Smyth) Date: Tue, 7 Nov 95 09:46:39 PST Subject: AISTATS-97: Preliminary Announcement Message-ID: <9511071746.AA00528@amorgos.jpl.nasa.gov> Preliminary Announcement Sixth International Workshop on Artificial Intelligence and Statistics (AISTATS-97) January 4-7, 1997 Ft. Lauderdale, Florida This is the sixth in a series of workshops which has brought together researchers in Artificial Intelligence (AI) and in Statistics to discuss problems of mutual interest. The exchange has broadened research in both fields and has strongly encouraged interdisciplinary work. Papers on all aspects of the interface between AI & Statistics are encouraged. For more details consult the AISTATS-97 Web page at: http://www.stat.washington.edu/aistats97/ A full Call for Papers will be released in early 1996. The paper submission deadline will be July 1st 1996. The workshop is organized under the auspices of the Society for Artificial Intelligence and Statistics. Program Chair: David Madigan, University of Washington General Chair: Padhraic Smyth, JPL and UCI From piuri at elet.polimi.it Wed Nov 8 16:01:20 1995 From: piuri at elet.polimi.it (Vincenzo Piuri) Date: Wed, 8 Nov 1995 21:01:20 GMT Subject: call for papers Message-ID: <9511082101.AA06232@ipmel2.elet.polimi.it> ========================================================= ETIM'96 1996 International Workshop on Emergent Technologies for Instrumentation and Measurements Como, Italy - 10-11 June 1996 ========================================================= Organized by the IEEE Instrumentation and Measurement Society (Technical Committee on Emergent Technologies) CALL FOR PAPERS This workshop is directed to create a unique synergetic discussion forum on the emergent technologies and a strong link between the theoretical researchers and the practitioners in the application fields related to instrumentation and measurements. The two-days single-session schedule will provide the ideal environment for in-depth analysis and discussions concerning the theoretical aspects of the applications and the use of new technologies in the practice. Researchers and practitioners are invited to submit papers concerning theoretical foundations, experimental results, or practical applications related to the use of advanced technologies for instrumentation and measurements. Papers are sollicited on, but not limited to, the following topics: neural networks, fuzzy logic, genetic algorithms, virtual instruments, optical technologies, laser, advanced digital signal/image processing, advanced analog signal processing, wavelets, sensor technologies, remote sensing, distributed systems, fault tolerance, adaptive systems. Interested authors should submit an extended summary or the full paper (limited to 20 double-spaced pages including figures and tables) to the program chair by January 15, 1996 (PostScript email or readable fax submissions are strongly encouraged). Submissions should contain: the corresponding author, affiliation, complete address, possible fax and email addresses. Submission implies the willingness of at least one of the authors to register and attend at the workshop and to present the paper. The corresponding author will be notified by February 16, 1996. The camera- ready version is limited to 10 one-column IEEE-book- standard pages and is due by May 1, 1996. The workshop will be held at the "A. Volta" Research Center - Villa Olmo, in Como, Italy. It is a two-hundred years-old villa in the pleasant scenario of one of the most attractive lakes around the nothern Italy, near Milan. Easy and frequent connections by train and airplane are available from Milan and all the main cities in Europe; flights from US and Asia arrive to the Malpensa international airport, connected by bus to Milan. The registration fee will be 120 US$, including lunches, coffe breaks and one copy of the proceedings. Hotel reservation will be managed directly by the Research Center to provide highly discounted rates. Program Chair prof. Vincenzo Piuri Dept of Electronics and Information Politecnico di Milano piazza L. da Vinci 32 I-20133 Milano, Italy phone +39-2-2399-3606 fax +39-2-2399-3411 e-mail piuri at elet.polimi.it American Co-Chair prof. Emil Petriu Dept. of Electrical Engineering University of Ottawa Ottawa, Ontario, Canada K1N 6N5 phone +1-613-564-2497 fax +1-613-564-6882 email petriu at trix.genie.uottawa.ca Asian Co-Chair prof. Kenzo Watanabe Research Inst. of Electronics Shizuoka University 3-5-1 Johoku, Hamamatsu 432, Japan phone +81-534-71-1171-573 fax +81-534-74-0630 email watanabe-k at rie.shizuoka.ac.jp Workshop Secretariat Ms. Laura Caldirola Politecnico di Milano phone +39-2-2399-3623 fax +39-2-2399-3411 caldirol at elet.polimi.it ========================================================= From simonpe at aisb.ed.ac.uk Wed Nov 8 07:32:38 1995 From: simonpe at aisb.ed.ac.uk (simonpe@aisb.ed.ac.uk) Date: Wed, 8 Nov 1995 12:32:38 +0000 Subject: No Free Lunch? (Delayed reply) In-Reply-To: <9511061603.AA05736@gaudi.ac.upc.es> References: <9511061603.AA05736@gaudi.ac.upc.es> Message-ID: <1520.9511081232@twain.aisb.ed.ac.uk> Pau Bofill writes: >Theorems for Search, and I was surprised >by his (and D.H. Wolpert's) statement that > > "any two search algorithms have exactly the same expected > performance, over the space of all fitness functions." > >which seemed to invalidate statements like > > "my search algorithm beats your search algorithm according > to a particular performance measure for the following fitness > function." Thess statements aren't contradictory (as you point out in your message). Averaged over the space of all possible optimization functions then sure, NFL says that all optimization algorithms are equal. But of course for any _particular_ optimization problem that you might be interested in, some algorithms are defintely much better than others. I think the importance of the NFL theorem is that it destroys the idea that there's such a thing as a `super-powerful all-purpose general learning algorithm' that can optimize any problem quickly. Instead, if we want a learning algorithm to work well, we have to analyze the particular problems we're interested in and tailor the learning algorithm to suit. Essentially we have to find ways of building prior domain knowledge about a problem into learning algorithms to solve that problem effectively. On the other hand, some people claim that there _is_ a general sub-class of problems for which it's at all feasible to find a solution - perhaps problems whose solutions have low komologrov complexity or something - and this might well mean that there _is_ a general good learning algorithm for this class of `interesting problems'. Any comments? << Simon Perkins >> Dept of AI, Edinburgh University S.Perkins at ed.ac.uk http://www.dai.ed.ac.uk/students/simonpe/ Tel: +44 131 650 3084 From zhuh at helios.aston.ac.uk Wed Nov 8 07:07:49 1995 From: zhuh at helios.aston.ac.uk (zhuh) Date: Wed, 8 Nov 1995 12:07:49 +0000 Subject: No Free Lunch? (Delayed reply) Message-ID: <321.9511081207@sun.aston.ac.uk> Pau Bofill paraphrases Wolpert, et al.'s paper "No Free Lunch..." [1] as > "any two search algorithms have exactly the same expected > performance, over the space of all optimization problems > that can be defined whithin a search space." This captures the trivial part of the argument. However, Wolpert et. al. also show that you can never cook up a method which will be good for an arbitrary non-uniform prior, because for every prior with which it works better than it would on a uniform prior, there is another with which it is worse by the same amount. This has important implications, given that neural net researchers are not in the habit of being precise about the distribution of optimisation problems their algorithms are directed at. This is an issue worth raising, and there is a principled way of dealing with it. The essential point is that without specifying the prior P(f) over the space of target distributions that a learning algorithm is meant to cope with, there is no objective way of comparing one algorithm with another. If you want to claim that good performance on a selection of problems from some domain implies that good performance should be expected on other problems from that domain, then you need to know enough about the prior distribution of problems to claim that your test suite forms a representative sample of it. Scientifically verifiable claims about a learning algorithm should specify at least the following three things: 1. the prior over target distributions. 2. the loss function. 3. the model. The first two are combined in Wolpert's work as the "prior on the fitness function". Conversely, it is shown [2,3,4] that 1. Given the first specification, there is always a uniquely defined "ideal estimator". 2. If the loss function is chosen as the information divergence [5] then the ideal estimate keeps all the information in the prior and the training data. Any learning algorithm must be a function of these ideal estimators. 3. If the purpose of a learning algorithm is to keep information, then it must approximate the ideal estimate. 4. For any computational model, the optimal estimate within the model is always an appropriate projection of the ideal estimate onto the model. In essence, the endavour to design better learning rules is fundamentally identical to the activity of finding good priors for application problems. Furthermore, the usual argument that in practice the prior is something vague and cannot be quantified is not substantiated. It is shown [6] that even in the case of training on data sets retrieved by ftp from the Internet, with little or no description of the problems, a reasonably good prior is still available. REFERENCES: [1] Wolpert, D. H. and Macready W. G.: No Free Lunch Theorems for Search, Santa Fe Inst. SFI-TR-95-02-010 ftp://ftp.santafe.edu/pub/wgm/nfl.ps [2] Zhu, H. and Rohwer, R.: Bayesian Invariant Measurements of Generalisation, 1995 ftp://cs.aston.ac.uk/neural/zhuh/letter.ps.Z [3] Zhu, H. and Rohwer, R.: Measurements of Generalisation Based on Information, MANNA conference, Oxford, 1995, (to appear in Ann. Math. Artif. Intell.) ftp://cs.aston.ac.uk/neural/zhuh/generalisation-manna.ps.Z [4] Zhu, H. and Rohwer, R.: A Bayesian Geometric Theory of Statistical Inference, 1995 ftp://cs.aston.ac.uk/neural/zhuh/stat.ps.Z [5] Amari, S.: Differential-Geometrical Methods in Statistics, Springer-Verlag, 1985. [6] Zhu, H. and Rohwer, R.: Bayesian regression filters and the issue of priors, 1995 ftp://cs.aston.ac.uk/neural/zhuh/reg_fil_prior.ps.Z -- Dr. Huaiyu Zhu zhuh at aston.ac.uk Neural Computing Research Group Dept of Computer Sciences and Applied Mathematics Aston University, Birmingham B4 7ET, UK From rgoldsto at bronze.ucs.indiana.edu Wed Nov 8 19:03:23 1995 From: rgoldsto at bronze.ucs.indiana.edu (rgoldsto@bronze.ucs.indiana.edu) Date: Wed, 8 Nov 1995 19:03:23 -0500 (EST) Subject: Faculty Position at Indiana University Message-ID: <199511090003.TAA23088@roatan.ucs.indiana.edu> INDIANA UNIVERSITY-BLOOMINGTON COGNITIVE SCIENCE POSITION The Cognitive Science Program and the Computer Science Department at Indiana University-Bloomington seek applicants for a joint faculty position, with rank open. Start date may be as early as Fall 1996, pending funding approval. We are looking for outstanding researchers at the forefront of the field, with ability to contribute to both Cognitive Science and Computer Science. Research area is open, including, for instance, neural net modeling, logic, reasoning, representation and information, language and discourse, robotics, computational vision and speech, visual inference, machine learning, and human-computer interaction. Applications from women and minority members are specifically encouraged. Indiana University is an Affirmative Action/Equal Opportunity Employer. The prospective faculty's office and laboratory will be based in the Computer Science department, which occupies a recently renovated spacious limestone building, and has extensive state-of-the-art computing facilities. Responsibilities will be shared with the Cognitive Science Program, one of the largest and most esteemed programs in the world today. The attractive wooded campus of Indiana University is located in Bloomington, voted one of the most cultural and livable small cities in the US, and a mere 45 minute drive from the Indianapolis airport. To be given full consideration applications must be received by February 15, 1996. The application should contain a detailed CV, copies of recent publications, brief statement of interests and future directions, and either three letters of recommendations, or a list of three references. Two copies of the application file must be sent (letters of reference may be sent to either address). Cognitive Science Search Cognitive Science Search Computer Science Department Cognitive Science Program Indiana University Psychology Department Bloomington, IN 47405 Indiana University Bloomington, IN 47405 Internet: cogsci-search at cs.indiana.edu or iucogsci at indiana.edu ____________________________ Rob Goldstone Department of Psychology/Program in Cognitive Science Indiana University Bloomington, IN. 47405 rgoldsto at indiana.edu Web site: http://cognitrn.psych.indiana.edu/ From dhw at santafe.edu Wed Nov 8 20:26:57 1995 From: dhw at santafe.edu (dhw@santafe.edu) Date: Wed, 8 Nov 95 18:26:57 MST Subject: No subject Message-ID: <9511090126.AA20987@santafe> Pau Bofill writes >>> I happened to read the message from Bill Macready on Free Lunch Theorems for Search, and I was surprised by his (and D.H. Wolpert's) statement that "any two search algorithms have exactly the same expected performance, over the space of all fitness functions." which seemed to invalidate statements like "my search algorithm beats your search algorithm according to a particular performance measure for the following fitness function." >>> We don't see how the two statements could be interpreted as contradictory. The first one explicitly is probabilistic (note the term "expected") whereas the second one is not. They explicitly refer to different things. >>> As far as I can see, the goal of a particular optimization PROBLEM, is to find a target POINT in search space whith specific properties. The role of any fitness function, then, is to assign a cost value to each point in search space in such a way that it helps the algorithm beeing used to find the target point. >>> Not at all! In the traveling salesperson problem, for example, the goal is to find a low tour-distance - if you can find the lowest, fine, but in practice you always must settle for something suboptimal. The choice of using tour-distance as one's fitness function was never made to "help the algorithm .. find the target point". (There is not even a single "target point", per se!) Rather it is made because ... that is what is of interest. The problem *is* the fitness function. We would argue that this is the case for almost any real-world problem. Now one's algorithm can perform transformations of the function so as to "help the algorithm ... find (a good) point". But that's another issue entirely. Aside to reader: Note that Bofill does not use "target" the way a machine-learner would. >>> If I understood them properly, Wolpert and Macready define the space of all possible fitness functions as the set of ALL possible assignments of cost values to points in search space. >>> Or to use Bofill's language, the space of all possible problems; they are synonymous. >>> And they use as a generalized perfomace measure the histogram of cost values, regardless of the points where they were found. >>> In most (almost all?) real world problems the set of points sampled (d_X values in our terminology) has essentially implications for the efficacy of the search algorithm used. Fitness values at those points, and maybe computational complexity of the algorithm, have such implications. But not the points themselves. Indeed, if there were some aspect of the points that *were* important, then (by definition!) it would go into the fitness function. Again consider TSP. The performance of an algorithm is determined by the tour-lengths (fitnesses) of the tours the algorithm has constructed, not by the tours themselves. >>> and their statement is equivalent to, "any two search algorithms have exactly the same expected performance, over the space of all optimization problems that can be defined whithin a search space." >>> Loosely speaking. >>> which stated otherwise would mean, "If the target doesn't matter, all algorithms perform alike." >>> This makes no sense. Of course the target matters - it matters more than anything else. The point is that *unless you take explicit consideration of the target (or more generally the fitness function) into account when using your algorithm*, all algorithms are alike. And *many* users of genetic algorithms, tabu search, hill-climbing, simulated annealing, etc., do not take into account the fitness function. (In fact, in many competitions, knowledge concerning the fitness function is considered cheating!) This - relatively minor - point of the paper is a cry for people to do the obvious: incorporate knowledge of the fitness function into their algorithm. On any problem, best results will accrue to such a strategy (e.g., the winners at TSP are those algorithms that are tailored to TSP.) Yet people still try to use fixed algorithms as though they were "general purpose". The NFL results point out the lack of formal justifiability of such strategies. >>> In particular, algorithms should be compared on PROBLEMS, not on fitness functions, with properly defined performance measures. >>> This is a content-free statement. In the real world, problems *are* fitness functions. Bill and David From tsioutsias-dimitris at CS.YALE.EDU Wed Nov 8 23:54:04 1995 From: tsioutsias-dimitris at CS.YALE.EDU (Dimitris I. Tsioutsias) Date: Wed, 8 Nov 1995 23:54:04 -0500 (EST) Subject: No Free Lunch? (Delayed reply) Message-ID: <199511090454.XAA21775@nebula.systemsz.cs.yale.edu> > From: simonpe at aisb.ed.ac.uk > Date: Wed, 8 Nov 1995 12:32:38 +0000 > .... > > I think the importance of the NFL theorem is that it destroys the idea > that there's such a thing as a `super-powerful all-purpose general > learning algorithm' that can optimize any problem quickly. Instead, if > we want a learning algorithm to work well, we have to analyze the > particular problems we're interested in and tailor the learning > algorithm to suit. Essentially we have to find ways of building prior > domain knowledge about a problem into learning algorithms to solve > that problem effectively. > ..... As any thoughtful person in the greater mathematical programming community might point out, there's no general optimization method that could outperform any other on any kind of problem. Rather it's the researcher's task to have an understanding of the problem domain, the computational requirements (and available resources), and the most intuitively promising avenue of attaining a ``good'' solution in the most efficient way... --Dimitris From bruno at redwood.psych.cornell.edu Wed Nov 8 23:52:46 1995 From: bruno at redwood.psych.cornell.edu (Bruno A. Olshausen) Date: Wed, 8 Nov 1995 23:52:46 -0500 Subject: sparse coding Message-ID: <199511090452.XAA13170@redwood.psych.cornell.edu> The following paper is available via http://redwood.psych.cornell.edu/bruno/papers.html or ftp://redwood.psych.cornell.edu/pub/papers/sparse-coding.ps.Z Sparse coding of natural images produces localized, oriented, bandpass receptive fields Bruno A. Olshausen and David J. Field Department of Psychology, Uris Hall Cornell University Ithaca, New York 14853 The images we typically view, or natural scenes, constitute a minuscule fraction of the space of all possible images. It seems reasonable that the visual cortex, which has evolved and developed to effectively cope with these images, has discovered efficient coding strategies for representing their structure. Here, we explore the hypothesis that the coding strategy employed at the earliest stage of the mammalian visual cortex maximizes the sparseness of the representation. We show that a learning algorithm that attempts to find linear sparse codes for natural scenes will develop receptive fields that are localized, oriented, and bandpass, much like those in the visual system. These receptive fields produce a more efficient image representation for later stages of processing because sparseness reduces the entropies of individual outputs, which in turn reduces the redundancy due to complex statistical dependencies among unit activities. From rosen at unr.edu Thu Nov 9 02:23:15 1995 From: rosen at unr.edu (David Rosen) Date: Wed, 8 Nov 1995 23:23:15 -0800 Subject: "How Good Were Those Probability Predictions?" -- Paper Available Message-ID: <199511090717.HAA05459@solstice.ccs.unr.edu> Announcing the following paper available in the neuroprose archive: How Good Were Those Probability Predictions? The Expected Recommendation Loss (ERL) Scoring Rule David B. Rosen To appear in: Maximum Entropy and Bayesian Methods. (Proceedings of the Thirteenth International Workshop, August 1993.) G. Heidbreder, ed. Kluwer, Dordrecht, The Netherlands, 1996. 8 pages. We present a new way to choose an appropriate scoring rule for evaluating the performance of a "soft classifier", i.e. of a supplier of predicted (inferred/estimated/learned/guessed) probabilities. A scoring rule (probability loss function) is a function of a single such prediction and the corresponding outcome event (true class); its expectation over the data space is the generalization performance of ultimate interest, while its sum or average over some benchmark test data set is an empirical performance measure. A user of probability predictions can apply his own decision threshold, preferring to err on one side, for example, to the extent that the consequences of an erroneous decision are more severe on the other side; this process is the subject of decision theory/analysis. We are not able to specify in advance, with certainty, these relative consequences, i.e. the user's cost matrix (indexed by decision and outcome event) defining his decision-making problem. So we represent this uncertainty itself by a distribution, from which we think of the cost matrix as being drawn. Specifying this distribution determines a uniquely appropriate scoring rule. We can interpret and characterize common scoring rules, such as the logarithmic (cross-entropy), quadratic (squared error or Brier), and the "0-1" misclassification score, as representing different assumptions about the probability that the predictions will be used in various decision-making problems. We discuss the connection to the theory of proper (truth- or honesty-rewarding) scoring rules. PostScript and plain-text versions are available via this Web page: http://www.scs.unr.edu/~cbmr/people/rosen/erl/ The paper is in Jordan Pollack's NEUROPROSE anonymous ftp archive as: ftp://archive.cis.ohio-state.edu/pub/neuroprose/rosen.exp-rec-loss.ps.Z (This supersedes an unannounced early version rosen.scoring.ps.Z) Hardcopies cannot be provided. -- David B Rosen OR New York Medical College From robtag at dia.unisa.it Thu Nov 9 05:55:21 1995 From: robtag at dia.unisa.it (Tagliaferri Roberto) Date: Thu, 9 Nov 1995 11:55:21 +0100 Subject: WIRN 96 Message-ID: <9511091055.AA11466@udsab.dia.unisa.it> ***************** CALL FOR PAPERS ***************** The 8-th Italian Workshop on Neural Nets WIRN VIETRI-96 May 23-25, 1996 Vietri Sul Mare, Salerno ITALY **************** FIRST ANNOUNCEMENT ***************** Organizing - Scientific Committee -------------------------------------------------- B. Apolloni (Univ. Milano) A. Bertoni ( Univ. Milano) D. D. Caviglia ( Univ. Genova) P. Campadelli ( Univ. Milano) M. Ceccarelli ( CNR Napoli) A. Colla (ELSAG Bailey Genova) M. Frixione ( I.I.A.S.S.) C. Furlanello (IRST Trento) G. M. Guazzo ( I.I.A.S.S.) M. Gori ( Univ. Firenze) F. Lauria ( Univ. Napoli) M. Marinaro ( Univ. Salerno) F. Masulli (Univ. Genova) P. Morasso (Univ. Genova) G. Orlandi ( Univ. Roma) E. Pasero ( Politecnico Torino ) A. Petrosino ( I.I.A.S.S.) M. Protasi ( Univ. Roma II) S. Rampone ( Univ. Salerno ) R. Serra ( Gruppo Ferruzzi Ravenna) F. Sorbello ( Univ. Palermo) R. Stefanelli ( Politecnico Milano) R. Tagliaferri ( Univ. Salerno) R. Vaccaro ( CNR Napoli) Topics ---------------------------------------------------- Mathematical Models Architectures and Algorithms Hardware and Software Design Hybrid Systems Pattern Recognition and Signal Processing Industrial and Commercial Applications Fuzzy Tecniques for Neural Networks Schedule ----------------------- Papers Due: January 31, 1996 Replies to Authors: March 31, 1996 Revised Papers Due: May 23, 1996 Sponsors ------------------------------------------------------------------------------ International Institute for Advanced Scientific Studies (IIASS) Dept. of Fisica Teorica, University of Salerno Dept. of Informatica e Applicazioni, University of Salerno Dept. of Scienze dell'Informazione, University of Milano Istituto per la Ricerca dei Sistemi Informatici Paralleli (IRSIP - CNR) Societa' Italiana Reti Neuroniche (SIREN) The 8-th Italian Workshop on Neural Nets (WIRN VIETRI-96) will take place in Vietri Sul Mare, Salerno ITALY, May 23-25, 1996. The conference will bring together scientists who are studying several topics related to neural networks. The three-day conference, to be held in the I.I.A.S.S., will feature both introductory tutorials and original, refereed papers, to be published by World Scientific Publishing. Papers should be 6 pages,including title, figures, tables, and bibliography. The first page should give keywords, postal and electronic mailing addresses, telephone, and FAX numbers. The camera ready format will be sent with the acceptation letter of the referees. Submit 3 copies and a 1 page abstract (containing keywords, postal and electronic mailing addresses, telephone, and FAX numbers with no more than 300 words) to the address shown (WIRN 96 c/o IIASS). An electronic copy of the abstract should be sent to the E-mail address below. During the Workshop the "Premio E.R. Caianiello" will be assigned to the best Ph.D. thesis in the area of Neural Nets and related fields of Italian researchers. The amount is of 2.000.000 Italian Lire. The interested researchers (with a thesis of 1993,1994, 1995 until February 29 1996) must send 3 copies of a c.v. and of the thesis to "Premio Caianiello" WIRN 96 c/o IIASS before February 29,1996. For more information, contact the Secretary of I.I.A.S.S. I.I.A.S.S Via G.Pellegrino, 19 84019 Vietri Sul Mare (SA) ITALY Tel. +39 89 761167 Fax +39 89 761189 E-Mail robtag at udsab.dia.unisa.it or the www pages at the address below: http:://www-dsi.ing.unifi.it/neural ***************************************************************** From juergen at idsia.ch Thu Nov 9 10:52:24 1995 From: juergen at idsia.ch (Juergen Schmidhuber) Date: Thu, 9 Nov 95 16:52:24 +0100 Subject: No Free Lunch? (Delayed reply) Message-ID: <9511091552.AA03224@fava.idsia.ch> In his response to NFL issues, Simon Perkins writes: > On the other hand, some people claim that there _is_ a general > sub-class of problems for which it's at all feasible to find a > solution - perhaps problems whose solutions have low komologrov > complexity or something - and this might well mean that there _is_ a > general good learning algorithm for this class of `interesting > problems'. A comment on this: We already know that for a wide variety of non-incremental search problems, there *is* a theoretically optimal algorithm: Levin's universal search algorithm (LS) (Ref.: L. A. Levin, Universal sequential search problems, Problems of Information Transmission, 9(3):265--266, 1973). Essentially, LS generates and tests solution candidates in order of their Levin complexities, until a solution is found (Levin complexity is a time-bounded restriction of Kolmogorov complexity). For instance, suppose there is an algorithm that solves a certain type of maze task in O(n^3) steps, where $n$ is a positive integer representing problem size. Then LS will solve the same task in at most O(n^3) steps (you may worry about the constant factor buried in the O-notation, though). Of course, there are huge classes of search problems that cannot be solved efficiently (say, in polynomial time), neither by LS nor by any other method. But most problems in the set of all possible, well-defined problems are indeed ``uninteresting''. Admittedly, I do not have a good objective definition of what's ``interesting''. The best I can come up with in the current context is, somewhat circularily, a possible superset of interesting problems: ``problems that can be solved efficiently by an optimal search algorithm''. Anyway, LS' existence fuels hope: just like there is an optimal, general search algorithm for many *non-incremental* search problems, there may be an optimal, general learning algorithm for *incremental* search problems (``incremental'' means: you may try to use experience with previous tasks to improve performance on new tasks). LS by itself is *not* necessarily optimal in incremental learning situations. For this reason, Marco Wiering and I are currently combining LS and the recent technique of ``environment-independent reinforcement acceleration'' (EIRA), currently the only method that guarantees a lifelong history of success accelerations, even in unrestricted environments (write-up may follow soon -- related publications in my home page). Juergen Schmidhuber IDSIA, Corso Elvezia 36, Ch-6900-Lugano http://www.idsia.ch/~juergen From black at signal.dra.hmg.gb Fri Nov 10 11:59:22 1995 From: black at signal.dra.hmg.gb (John V. Black) Date: Fri, 10 Nov 95 16:59:22 +0000 Subject: WWW page for the Pattern and Information Processing Group at DRA (UK) Message-ID: Dear Connectionits, Announcing a new WWW home page that covers the work of the Pattern and Information Processing Group of the Defence Research Agency (DRA) in the United Kingdom. The group consists of some 18 people, who together have a wide experience of pattern and information processing techniques and problems, which include: Analogue Systems for Information Processing Bayesian Methods Classification, Identification and Recognition Data and Information Fusion Data Analysis and Exploration Decision and Game Theory Information Processing Architectures Neural Network Techniques and Architectures Radar (Array) Signal Processing Self-Organising Systems Sensor Signal Processing Applications Statistical Pattern Processing Time-Series Analysis Tracking Uncertainty Handling (Bayesian Networks, Fuzzy Logic) The URL of the home page is http://www.dra.hmg.gb/cis5pip/Welcome.html John Black (black at signal.dra.hmg.gb) From isca at interpath.com Mon Nov 13 11:43:24 1995 From: isca at interpath.com (Mary Ann Sullivan) Date: Mon, 13 Nov 1995 11:43:24 -0500 Subject: CALL FOR PAPERS: 5th Intelligent Systems Conf. - Reno, Nevada June 19-21, 1996 (Formerly GWICS) Message-ID: <199511131643.LAA13980@mail-hub.interpath.net> ******************************************************************************* This message is being sent to multiple addressees. If you wish to have your address removed from our mailing list, please reply to the sender and we will promptly honor your request. ******************************************************************************* CALL FOR PAPERS Fifth International Conference on Intelligent Systems (formerly GWICS) June 19 - 21, 1996 Flamingo Hilton, Reno, Nevada, U.S.A. Sponsored by the International Society for Computers and Their Applications (ISCA) ============================================================================== CONFERENCE CHAIR PROGRAM CHAIR Carl Looney Frederick C. Harris, Jr. (Univ. of Nevada, Reno) (Univ. of Nevada, Reno) ============================================================================== The International Conference on Intelligent Systems seeks quality international submissions in all areas of intelligent systems including but not limited to: Logic and Inference Cognitive Science Artificial Neural Networks Reasoning Distributed Intelligent Systems Artificial Life Case-Based Reasoning Knowledge-Based Systems Vision, Image Processing Interpretation Machine Learning and Adaptive Sys. Cellular Automata Fuzzy Systems Robotics, Control and Planning Multimedia and Human Computer Evolutionary Computation Interaction (GA,GP,ES,EP) Autonomous Agents Recognition and Classification Search Instructions to Authors: Authors must submit 5 copies of an extended abstract (at least 4 pages) or complete paper (no more than 10 double spaced pages). Please include one separate cover page containing title, author's name(s), address, affiliation, e-mail address, telephone number, and topic area. To help us assign reviewers to papers, use the topics in the list above as a guide. In cases of multiple authors, all correspondence will be sent to the first author unless otherwise requested. Abstracts may be submitted via E-mail. Submit your paper by February 15, 1996 to the program chair: Dr. Frederick C. Harris, Jr. Telephone: (702) 784-6571 University of Nevada Fax: (702) 784-1766 Dept. of Computer Science E-mail: fredh at cs.unr.edu Reno, Nevada 89557 IMPORTANT DATES: Deadline for extended summary/paper submission: February 15, 1996 Notification of acceptance: April 10, 1996 Camera ready papers due: May 10, 1996 PROGRAM COMMITTEE G. Antoniou (U. of Newcastle) A. Barducci (CNR-IROE,Italy) M. Boden (U. of Skovde) A. Canas (U. of West Florida) M. Cohen (Cal State, Fresno) D. Egbert (U. of Nevada, Reno) S. Fadali (U. of Nevada, Reno) J. Fisher (Cal State Pomona) K. Ford (U. of West Florida) P. Geril (U. of Ghent) J. Gero (U. of Sydney) D. Hudson (U. of Cal, San Fran.) P. Jog (DePaul U.) S. Kawata (Tokyo Metropolitan U.) V. R. Kumar (Fujitsu, Australia) D. Leake (Indiana U.) F. Lin (Santa Clara U.) S. Louis (U. of Nevada, Reno) J. McDonnell (NRaD, San Diego) A. McRae (Appalachian State U.) S. Narayan (UNC-Wilmington) T. Oren (U. of Ottawa) V. Patel (McGill U.) D. Pheanis (Arizona State U.) V. Piuri (Politecnico Di Milano) R. Reynolds (Wayne State U.) M. Rosenman (U. of Sydney) A. Sangster (Aberdeen U.) R. Smith (U. of Alabama) R. Sun (U. of Alabama) A. Yfantis (UNLV) S. Yoon (Widener U.) ISCA Headquarters 8820 Six Forks Road, Raleigh, NC 27615 (USA) Ph: (919) 847-3747 Fax: (919) 676-0666 E-mail: isca at interpath.com URL= http://www.isca-hq.org/isca From edelman at wisdom.weizmann.ac.il Tue Nov 14 01:54:55 1995 From: edelman at wisdom.weizmann.ac.il (Edelman Shimon) Date: Tue, 14 Nov 1995 06:54:55 GMT Subject: TR available: RFs From Hyperacuity to Recognition Message-ID: <199511140654.GAA19020@lachesis.wisdom.weizmann.ac.il> Retrieval information: FTP-host: eris.wisdom.weizmann.ac.il (132.76.80.53) FTP-pathname: /pub/watt-rfs.ps.Z URL: ftp://eris.wisdom.weizmann.ac.il/pub/watt-rfs.ps.Z 28 pages; 519 KB compressed, 2.6 MB uncompressed. Comments welcome at URL mailto:edelman at wisdom.weizmann.ac.il ---------------------------------------------------------------------- Receptive Fields for Vision: from Hyperacuity to Object Recognition Weizmann Institute CS-TR 95-29, 1995; to appear in VISION, R. J. Watt, ed., MIT Press, 1996. Shimon Edelman Dept. of Applied Mathematics and Computer Science The Weizmann Institute of Science Rehovot 76100, ISRAEL http://eris.wisdom.weizmann.ac.il/~edelman Many of the lower-level areas in the mammalian visual system are organized retinotopically, that is, as maps which preserve to a certain degree the topography of the retina. A unit that is a part of such a retinotopic map normally responds selectively to stimulation in a well-delimited part of the visual field, referred to as its {\em receptive field} (RF). Receptive fields are probably the most prominent and ubiquitous computational mechanism employed by biological information processing systems. This paper surveys some of the possible computational reasons behind the ubiquity of RFs, by discussing examples of RF-based solutions to problems in vision, from spatial acuity, through sensory coding, to object recognition. ---------------------------------------------------------------------- -Shimon From dhw at santafe.edu Tue Nov 14 13:24:58 1995 From: dhw at santafe.edu (David Wolpert) Date: Tue, 14 Nov 95 11:24:58 MST Subject: Response to no-free-lunch discussion Message-ID: <9511141824.AA04471@sfi.santafe.edu> Some quick comments on the recent discussion of no-free-lunch (NFL) issues. As an aside, it's interesting to note how "all over the map" the discussion is, from people who whole-heartedly agree with NFL, to people who make claims diametrically opposed to it. **** Simon Perkins writes: >>> some people claim that there _is_ a general sub-class of problems for which it's at all feasible to find a solution - perhaps problems whose solutions have low komologrov complexity or something - and this might well mean that there _is_ a general good learning algorithm for this class of `interesting problems'. Any comments? >>> There's no disputing this. Restricting attention to a sub-class of problems is formally equivalent to placing a restriction on the target and/or prior over targets (in the latter case, placing a restriction on the prior's support). Certainly once things are restricted this way, we are in the domain of Bayesian analysis (and/or some versions of PAC), and not all algorithms are the same. However we have *no* a priori reason to believe that targets with "low Kolmogorov complexity" (or anything else) are/not likely to occur in the real world. Well, nothing can be proven from first principles to work well, you might say. This actually isn't always true (the loss function is important, there are minimax issues, etc.) But even in the simple scenarios in which this sentiment is essentially correct (i.e., the scenarios in which NFL holds), there is a huge body of literature which purports to prove from first principles that some algorithms *do* work better than others, without any assumption about the targets: *** E.g., claims that so long as the VC dimension of your algorithm is low, the training set large, and the misclassification rate on the training set small, then *independent of assumptions concerning the target*, you can bound how large the generalization error is. Or claims that boosting can only help generalization error, regardless of the prior over targets. Or the PAC "proof" of Occam's razor (which - absurdly - "holds" for any and all complexity measures). NFL results show up all such claims as problematic, at best. The difficulty is not with the math behind these claims, but rather with the interpretations of what that math means. *** Dimitris Tsioutsias writes: >>> As any thoughtful person in the greater mathematical programming community might point out, there's no general optimization method that could outperform any other on any kind of problem. >>> Obviously. But to give a simple example, before now, no such "thoughtful person" would have had any idea of whether there might be a "general optimization method" that only rarely performs worse than random search. Addressing such issues is one of the (more trivial) things NFL can do for you. *** Finally, Juergen Schmidhuber writes: >>> We already know that for a wide variety of non-incremental search problems, there *is* a theoretically optimal algorithm: Levin's universal search algorithm >>> I have lots of respect for Juergen's work, but on this issue, I have to disagree with him. Simply put, supervised learning (and in many regards search) is an exercise in statistics, not algorithmic information complexity theory. NFL *proves* this. In practice, it may (or may not) be a good idea to use an algorithm that searches for low Levin complexity rather than one that works by other means. But there is simply no first principles reason for believing so. Will such an algorithm beat backprop? Maybe, maybe not. It depends on things that can not be proven from first principles. (As well as a precise definition of "beat", etc.) The distribution over the set of problems we encounter in the real world is governed by an extraordinarily complicated interplay between physics, chemistry, biology, psychology, sociology and economics. There is no a priori reason for believing that this interplay respects notion of algorithmic information complexity, time-bounded or otherwise. David Wolpert From copelli at onsager.if.usp.br Tue Nov 14 17:30:34 1995 From: copelli at onsager.if.usp.br (Mauro Copelli da Silva) Date: Tue, 14 Nov 1995 20:30:34 -0200 (EDT) Subject: On-line learning paper Message-ID: <199511142230.UAA01640@curie.if.usp.br> FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/copelli.equivalence.ps.Z *** PAPER ANNOUNCEMENT *** The following paper is available by anonymous ftp from the pub/neuroprose directory of the archive.cis.ohio-state.edu host (see instructions below). It is 27 pages long and has been submitted to Physical Review E. Comments are welcomed. EQUIVALENCE BETWEEN LEARNING IN PERCEPTRONS WITH NOISY EXAMPLES AND TREE COMMITTEE MACHINES Mauro Copelli, Osame Kinouchi and Nestor Caticha Instituto de Fisica, Universidade de Sao Paulo CP 66318, 05389-970 Sao Paulo, SP, Brazil e-mail: copelli,osame,nestor at if.usp.br Abstract We study learning from single presentation of examples ({\em incremental} or {\em on-line} learning) in single-layer perceptrons and tree committee machines (TCMs). Lower bounds for the perceptron generalization error as a function of the noise level $\epsilon$ in the teacher output are calculated. We find that optimal local learning in a TCM with $K$ hidden units is simply related to optimal learning in a simple perceptron with a corresponding noise level $\epsilon(K)$. For large number of examples and finite $K$ the generalization error decays as $\alpha_{cm}^{-1}$, where $\alpha_{cm}$ is the number of examples per adjustable weight in the TCM. We also show that on-line learning is possible even in the $K\rightarrow\infty$ limit, but with the generalization error decaying as $\alpha_{cm}^{-1/2}$. The simple Hebb rule can also be applied to the TCM, but now the error decays as $\alpha_{cm}^{-1/2}$ for finite $K$ and $\alpha_{cm}^{-1/4}$ for $K\rightarrow\infty$. Exponential decay of the generalization error in both the perceptron learning from noisy examples and in the TCM is obtained by using the learning by queries strategy. ****************** How to obtain a copy ************************* unix> ftp archive.cis.ohio-state.edu User: anonymous Password: (type your e-mail address) ftp> cd pub/neuroprose ftp> binary ftp> get copelli.equivalence.ps.Z ftp> quit unix> uncompress copelli.equivalence.ps.Z unix> lpr copelli.equivalence.ps (or however you print PostScript files) **PLEASE DO NOT REPLY DIRECTLY TO THIS MESSAGE** From cas-cns at PARK.BU.EDU Wed Nov 15 11:20:00 1995 From: cas-cns at PARK.BU.EDU (CAS/CNS) Date: Wed, 15 Nov 1995 11:20:00 -0500 Subject: BU - Cognitive & Neural Systems Message-ID: <199511151618.LAA28690@cns.bu.edu> ************************************************************** DEPARTMENT OF COGNITIVE AND NEURAL SYSTEMS (CNS) AT BOSTON UNIVERSITY ************************************************************** Ennio Mingolla, Acting Chairman, 1995-96 Stephen Grossberg, Chairman Gail A. Carpenter, Director of Graduate Studies The Boston University Department of Cognitive and Neural Systems offers comprehensive graduate training in the neural and computational principles, mechanisms, and architectures that underlie human and animal behavior, and the application of neural network architectures to the solution of technological problems. Applications for Fall, 1996, admission and financial aid are now being accepted for both the MA and PhD degree programs. To obtain a brochure describing the CNS Program and a set of application materials, write, telephone, or fax: DEPARTMENT OF COGNITIVE & NEURAL SYSTEMS 677 Beacon Street Boston, MA 02215 617/353-9481 (phone) 617/353-7755 (fax) or send via email your full name and mailing address to: rll at cns.bu.edu Applications for admission and financial aid should be received by the Graduate School Admissions Office no later than January 15. Late applications will be considered until May 1; after that date applications will be considered only as special cases. Applicants are required to submit undergraduate (and, if applicable, graduate) transcripts, three letters of recommendation, and Graduate Record Examination (GRE) scores. The Advanced Test should be in the candidate's area of departmental specialization. GRE scores may be waived for MA candidates and, in exceptional cases, for PhD candidates, but absence of these scores may decrease an applicant's chances for admission and financial aid. Non-degree students may also enroll in CNS courses on a part-time basis. Description of the CNS Department: The Department of Cognitive and Neural Systems (CNS) provides advanced training and research experience for graduate students interested in the neural and computational principles, mechanisms, and architectures that underlie human and animal behavior, and the application of neural network architectures to the solution of technological problems. Students are trained in a broad range of areas concerning cognitive and neural systems, including vision and image processing; speech and language understanding; adaptive pattern recognition; cognitive information processing; self-organization; associative learning and long-term memory; computational neuroscience; nerve cell biophysics; cooperative and competitive network dynamics and short-term memory; reinforcement, motivation, and attention; adaptive sensory-motor control and robotics; active vision; and biological rhythms; as well as the mathematical and computational methods needed to support advanced modeling research and applications. The CNS Department awards MA, PhD, and BA/MA degrees. The CNS Department embodies a number of unique offerings. It has developed a curriculum that features 15 interdisciplinary graduate courses each of which integrates the psychological, neurobiological, mathematical, and computational information needed to theoretically investigate fundamental issues concerning mind and brain processes and the applications of neural networks to technology. Each course is typically taught once a week in the evening to make the program available to qualified students, including working professionals, throughout the Boston area. Nine additional research course are also offered. In these courses, one or two students meet regularly with one or two professors to pursue advanced reading and collaborative research. Students develop a coherent area of expertise by designing a program that includes courses in areas such as Biology, Computer Science, Engineering, Mathematics, and Psychology, in addition to courses in the CNS Department. The CNS Department prepares students for PhD thesis research with scientists in one of several Boston University research centers or groups, and with Boston-area scientists collaborating with these centers. The unit most closely linked to the department is the Center for Adaptive Systems (CAS). Students interested in neural network hardware work with researchers in CNS, the College of Engineering, and at MIT Lincoln Laboratory. Other research resources include distinguished research groups in neurophysiology, neuroanatomy, and neuropharmacology at the Medical School and the Charles River campus; in sensory robotics, biomedical engineering, computer and systems engineering, and neuromuscular research within the Engineering School; in dynamical systems within the Mathematics Department; in theoretical computer science within the Computer Science Department; and in biophysics and computational physics within the Physics Department. In addition to its basic research and training program, the Department offers a colloquium series, seminars, conferences, and special interest groups which bring many additional scientists from both experimental and theoretical disciplines into contact with the students. The CNS Department is moving in October, 1995 into its own new four-story building, which features a full range of offices, laboratories, classrooms, library, lounge, and related facilities for exclusive CNS use. 1995-96 CAS MEMBERS and CNS FACULTY: Jelle Atema Professor of Biology Director, Boston University Marine Program (BUMP) PhD, University of Michigan Sensory physiology and behavior Aijaz Baloch Research Associate of Cognitive and Neural Systems PhD, Electrical Engineering, Boston University Neural modeling of role of visual attention of recognition, learning and motor control, computational vision, adaptive control systems, reinforcement learning Helen Barbas Associate Professor, Department of Health Sciences, Boston University PhD, Physiology/Neurophysiology, McGill University Organization of the prefrontal cortex, evolution of the neocortex Jacob Beck Research Professor of Cognitive and Neural Systems PhD, Psychology, Cornell University Visual Perception, Psychophysics, Computational Models Daniel H. Bullock Associate Professor of Cognitive and Neural Systems and Psychology PhD, Psychology, Stanford University Real-time neural systems, sensory-motor learning and control, evolution of intelligence, cognitive development Gail A. Carpenter Professor of Cognitive and Neural Systems and Mathematics Director of Graduate Studies, Department of Cognitive and Neural Systems PhD, Mathematics, University of Wisconsin, Madison Pattern recognition, categorization, machine learning, differential equations Laird Cermak Professor of Neuropsychology, School of Medicine Professor of Occupational Therapy, Sargent College Director, Memory Disorders Research Center, Boston Veterans Affairs Medical Center PhD, Ohio State University Michael A. Cohen Associate Professor of Cognitive and Neural Systems and Computer Science Director, CAS/CNS Computation Labs PhD, Psychology, Harvard University Speech and language processing, measurement theory, neural modeling, dynamical systems H. Steven Colburn Professor of Biomedical Engineering PhD, Electrical Engineering, Massachusetts Institute of Technology Audition, binaural interaction, signal processing models of hearing William D. Eldred III Associate Professor of Biology BS, University of Colorado; PhD, University of Colorado, Health Science Center Visual neural biology Paolo Gaudiano Assistant Professor of Cognitive and Neural Systems PhD, Cognitive and Neural Systems, Boston University Computational and neural models of vision and adaptive sensory-motor control Jean Berko Gleason Professor of Psychology AB, Radcliffe College; AM, PhD, Harvard University Psycholinguistics Douglas Greve Research Associate of Cognitive and Neural Systems PhD, Cognitive and Neural Systems, Boston University Stephen Grossberg Wang Professor of Cognitive and Neural Systems Professor of Mathematics, Psychology, and Biomedical Engineering Director, Center for Adaptive Systems Chairman, Department of Cognitive and Neural Systems PhD, Mathematics, Rockefeller University Theoretical biology, theoretical psychology, dynamical systems, applied mathematics Frank Guenther Assistant Professor of Cognitive and Neural Systems PhD, Cognitive and Neural Systems, Boston University Biological sensory-motor control, spatial representation, speech production Thomas G. Kincaid Chairman and Professor of Electrical, Computer and Systems Engineering, College of Engineering PhD, Electrical Engineering, Massachusetts Institute of Technology Signal and image processing, neural networks, non-destructive testing Nancy Kopell Professor of Mathematics PhD, Mathematics, University of California at Berkeley Dynamical systems, mathematical physiology, pattern formation in biological/physical systems Ennio Mingolla Associate Professor of Cognitive and Neural Systems and Psychology Acting Chairman 1995-96, Department of Cognitive and Neural Systems PhD, Psychology, University of Connecticut Visual perception, mathematical modeling of visual processes Alan Peters Chairman and Professor of Anatomy and Neurobiology, School of Medicine PhD, Zoology, Bristol University, United Kingdom Organization of neurons in the cerebral cortex, effects of aging on the primate brain, fine structure of the nervous system Andrzej Przybyszewski Senior Research Associate of Cognitive and Neural Systems MSc, Technical Warsaw University; MA, University of Warsaw; PhD, Warsaw Medical Academy Adam Reeves Adjunct Professor of Cognitive and Neural Systems Professor of Psychology, Northeastern University PhD, Psychology, City University of New York Psychophysics, cognitive psychology, vision William Ross Research Associate of Cognitive and Neural Systems BSc, Cornell University; MA, PhD, Boston University Mark Rubin Research Assistant Professor of Cognitive and Neural Systems Research Physicist, Naval Air Warfare Center, China Lake, CA (on leave) PhD, Physics, University of Chicago Neural networks for vision, pattern recognition, and motor control Robert Savoy Adjunct Associate Professor of Cognitive and Neural Systems Scientist, Rowland Institute for Science PhD, Experimental Psychology, Harvard University Computational neuroscience; visual psychophysics of color, form, and motion perception Eric Schwartz Professor of Cognitive and Neural Systems; Electrical, Computer and Systems Engineering; and Anatomy and Neurobiology PhD, High Energy Physics, Columbia University Computational neuroscience, machine vision, neuroanatomy, neural modeling Robert Sekuler Adjunct Professor of Cognitive and Neural Systems Research Professor of Biomedical Engineering, College of Engineering, BioMolecular Engineering Research Center Jesse and Louis Salvage Professor of Psychology, Brandeis University AB,MA, Brandeis University; Sc.M., PhD, Brown University Allen Waxman Adjunct Associate Professor of Cognitive and Neural Systems Senior Staff Scientist, MIT Lincoln Laboratory PhD, Astrophysics, University of Chicago Visual system modeling, mobile robotic systems, parallel computing, optoelectronic hybrid architectures James Williamson Research Associate of Cognitive and Neural Systems PhD, Cognitive and Neural Systems, Boston University Image processing and object recognition. Particular interests are: dynamic binding, self-organization, shape representation, and classification Jeremy Wolfe Adjunct Associate Professor of Cognitive and Neural Systems Associate Professor of Ophthalmology, Harvard Medical School Psychophysicist, Brigham & Women's Hospital, Surgery Dept. Director of Psychophysical Studies, Center for Clinical Cataract Research PhD, Massachusetts Institute of Technology From pfbaldi at cco.caltech.edu Wed Nov 15 13:45:09 1995 From: pfbaldi at cco.caltech.edu (Pierre Baldi) Date: Wed, 15 Nov 1995 10:45:09 -0800 (PST) Subject: Tal Grossman Memorial Workshop in Vail (NIPS95) Message-ID: NIPS95 TAL GROSSMAN MEMORIAL WORKSHOP MACHINE LEARNING APPROACHES IN COMPUTATIONAL MOLECULAR BIOLOGY December 1, 1995 Vail, CO CURRENT LIST OF SCHEDULED PRESENTATIONS: Alan Lapedes Neural Network Representations of Empirical Protein Potentials. Gary Stormo The Use of Neural Networks for Identification of Common Domains by Maximizing Specificity. Ajay N. Jain Machine Learning Techniques for Drug Design: Lead Discovery, Lead Optimization, and Screening Strategies. Anders Krogh Maximum Entropy Weighting of Aligned Sequences of Proteins or DNA. Paul Stolorz Applying Dynamic Programming Ideas to Monte Carlo Sampling. Soren Brunak Bendability of Exons and Introns in Human DNA. Pierre Baldi Mining Data Bases of Fragments with HMMs. CURRENT LIST OF ABSTRACTS: Alan Lapedes (Los Alamos National Laboratory) asl at t13.lanl.gov Neural Network Representations of Empirical Protein Potentials. Recently, there has been considerable interest in deriving and applying knowledge-based, empirical potential functions for proteins. These empirical potentials have been derived from the statistics of interacting, spatially neighboring residues, as may be obtained from databases of known protein crystal structures. We employ neural networks to redefine empirical potential functions from the point of view of discrimination functions. This approach generalizes previous work, in which simple frequency counting statistics are used on a database of known protein structures. This generalization allows us to avoid restriction to strictly pairwise interactions. Instead of frequency counting to fix adjustable parameters, one now optimizes an objective function involving a parameterized probability distribution. We show how our method reduces to previous work in special situations, illustrating in this context the relationship of neural networks to statistical methodology. A key feature in the approach we advocate is the development of a representation to describe the location of interacting residues that exist in a sphere of small fixed radius around each residue. This is a natural ``shape representation'' for the interaction neighborhoods of protein residues. We demonstrate that this shape representation and the network's improved abilities enhances discrimination over that obtained by previous methodologies. This work is with Robert Farber and the late Tal Grossman (Los Alamos National Laboratory). Gary Stormo (University of Colorado, Boulder) stormo at exon.biotech.washington.edu The Use of Neural Networks for Identification of Common Domains by Maximizing Specificity. We describe an unsupervised learning procedure in which the objective to be maximized is ``specificity'', defined as the probability of obtaining a particular set of strings within a much larger collection of background strings. We demonstrate its use for identifying protein binding sites on unaligned DNA sequences, common sequence/structure motifs in RNA and common motifs in protein sequences. The idea behind the ``specificity'' criterion it to discover a probability distribution for strings such that the difference between the probabilities of the particular strings and the background strings is maximized. Both the probability distribution and the set of particular strings need to be discovered; the probability distribution can be any allowable distribution over the string alphabet, and the particular strings are contained within a set of longer strings, but their locations are not known in advance. Previous methods have viewed this problem as one of multiple alignment, whereas our method is more flexible in the types of patterns that can be allowed and in the treatment of the background strings. When the patterns are linearly separable from the background, a simple Perceptron works well to identify the patterns. We are currently testing more complicated networks for more complicated patterns. This work is in collaboration with Alan Lapedes of Los Alamos National Laboratory and the Santa Fe Institute, and John Heumann of Hewlett-Packard. Ajay N. Jain (Arris Pharmaceutical Corporation) jain at arris.com Machine Learning Techniques for Drug Design: Lead Discovery, Lead Optimization, and Screening Strategies. At its core, the drug discovery process involves designing small organic molecules that satisfy the physical constraints of binding to a specific site on a particular protein (usually an enzyme or receptor). Machine learning techniques can play a significant role in all phases of the process. When the structure of the protein is known, it is possible to "dock" candidate molecules into the structure and compute the likelihood that a molecule will bind well. Fundamentally, this is a thermodynamic event that is too complicated to simulate accurately. Machine learning techniques can be used to empirically construct functions that are predictive of binding affinities. Similarly, when no protein structure is known, but there exists some data on molecules exhibiting a range of binding affinities, it is possible to use machine learning techniques to capture the 3D pattern that is responsible for binding. Lastly, in cases where one has capacity to make large numbers of small molecules (libraries) to screen against multiple diverse protein targets, one can use clustering techniques to design maximally diverse libraries. This talk will briefly discuss each of these techniques in the context of drug discovery at Arris Pharmaceutical Corporation. Anders Krogh (The Sanger Centre) krogh at sanger.ac.uk Maximum Entropy Weighting of Aligned Sequences of Proteins or DNA. In a family of proteins or other biological sequences like DNA the various subfamilies are often very unevenly represented. For this reason a scheme for assigning weights to each sequence can greatly improve performance at tasks such as database searching with profiles or other consensus models based on multiple alignments. A new weighting scheme for this type of database search is proposed. In a statistical description of the searching problem it is derived from the maximum entropy principle. It can be proved that, in a certain sense, it corrects for uneven representation. It is shown that finding the maximum entropy weights is an easy optimization problem for which standard techniques are applicable. Paul Stolorz (Jet Propulsion Laboratory, Caltech) stolorz at telerobotics.jpl.nasa.gov Applying Dynamic Programming Ideas to Monte Carlo Sampling. Monte Carlo sampling methods developed originally for physics and chemistry calculations have turned out to be very useful heuristics for problems in fields such as computational biology, traditional computer science and statistics. Macromolecular structure prediction and alignment, combinatorial optimization, and more recently probabilistic inference, are classic examples of their use. This talk will swim against the tide a bit by showing that computer science, in the guise of dynamic programming, can in turn supply substantial insight into the Monte Carlo process. This insight allows the construction of powerful novel Monte Carlo methods for a range of calculations in areas such as computational biology, computational vision and statistical inference. The methods are especially useful for problems plagued by multiple modes in the integrand, and for problems containing important, though not overwhelming, long-range information. Applications to protein folding, and to generalized Hidden Markov Models, will be described to illustrate how to systematically implement and test these algorithms. Soren Brunak (The Technical University of Denmark) brunak at cbs.dtu.dk Bendability of Exons and Introns in Human DNA. We analyze the sequential structure of human exons and introns by hidden Markov models. We find that exons -- besides the reading frame -- hold a specific periodic pattern. The pattern has the triplet consensus: non-T(A/T)G and a minimal periodicity of roughly 10 nucleotides. It is not a consequence of the nucleotide statistics in the three codon positions, nor of the previously well known periodicity caused by the encoding of alpha-helices in proteins. Using DNA triplet bendability parameters from DNase I experiments, we show that the pattern corresponds to a periodic `in-phase' bending potential towards the major groove of the DNA. Similarly, nucleosome positioning data show that the consensus triplets have a preference for locations on a bent double helix where the major groove faces inward and is compressed. We discuss the relation between the bending potential of coding regions and its importance for the recognition of genes by the transcriptional machinery. This work is in collaboration with P. Baldi (Caltech), Y. Chauvin (Net-ID, Inc.), Anders Krogh (The Sanger Centre). Pierre Baldi (Caltech) pfbaldi at ccosun.caltech.edu Mining Data Bases of Fragments with HMMs. Hidden Markov Model (HMM) techniques are applied to the problem of mining large data bases of protein fragments. The study is focused on one particular protein family, the G-Protein-Coupled Receptors (GPCR). A large data base is first constructed, by randomly extracting fragments from the entire SWISS-PROT data base, at different lengths, positions, and simulated noise levels, in a way that roughly matches other existing, but not always publicly accessible, data bases. A HMM trained on the GPCR family is then used to score all the fragments, in terms of their negative log-likelihood. The discrimination power of the HMM is assessed, and quantitative results are derived on how performance degrades, as a function of fragment length, truncation position, and noise level, and on how to set discrimination thresholds. The raw score performance is further improved by deriving additional filters, based on the structure of the alignments of the fragments to the HMM. This work is in collaboration with Y. Chauvin (Net-ID, Inc.), F. Tobin and A. Williams (SmithKline Beecham). From tibs at utstat.toronto.edu Wed Nov 15 15:40:00 1995 From: tibs at utstat.toronto.edu (tibs@utstat.toronto.edu) Date: Wed, 15 Nov 95 15:40 EST Subject: new tech report available Message-ID: Model search and inference by bootstrap ``bumping'' Robert Tibshirani and Keith Knight University of Toronto We propose a bootstrap-based method for searching through a space of models. The technique is well suited to complex, adaptively fitted models: it provides a convenient method for finding better local minima, for resistant fitting, and for optimization under constraints. Applications to regression, classification and density estimation are described. The collection of models can also be used to form a confidence set for the true underlying model, using a generalization of Efron's percentile interval. We also provide results on the asymptotic behaviour of bumping estimates. Available at http://utstat.toronto.edu/reports/tibs or ftp: utstat.toronto.edu in pub/tibs/bumping.ps ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Rob Tibshirani, Dept of Preventive Med & Biostats, and Dept of Statistics Univ of Toronto, Toronto, Canada M5S 1A8. Phone: 416-978-4642 (PMB), 416-978-0673 (stats). FAX: 416 978-8299 tibs at utstat.toronto.edu. ftp: //utstat.toronto.edu/pub/tibs http://www.utstat.toronto.edu/~tibs/home.html +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ From scheler at ICSI.Berkeley.EDU Thu Nov 16 12:44:40 1995 From: scheler at ICSI.Berkeley.EDU (Gabriele Scheler) Date: Thu, 16 Nov 1995 09:44:40 -0800 Subject: Job Notice Message-ID: <199511161744.JAA15103@tiramisu.ICSI.Berkeley.EDU> ***********JOB NOTICE ******** 1 - 2 full-time positions for research associates are available in the area of connectionist natural language modeling in the Department of Computer Science, Technical University of Munich, Germany. One project concerns text-based learning, the other (pending final approval) combines modeling of language acquisition with situated learning. Several short-term or half-time paid positions for graduate students are also available. Knowledge of German is helpful, but not essential. Positions may start as early as February 1, 1996. A full job description with reference to relevant www-sites will be mailed to interested persons in Mid-December. Anyone interested in these positions should direct an informal request for further details to Dr Gabriele Scheler ICSI 1947 Center Street, Berkeley 94704-1198 scheler at icsi.berkeley.edu From barberd at helios.aston.ac.uk Wed Nov 15 13:40:48 1995 From: barberd at helios.aston.ac.uk (barberd) Date: Wed, 15 Nov 1995 18:40:48 +0000 (GMT) Subject: Paper available Message-ID: <6923.9511151840@sun.aston.ac.uk> The following paper, (a version of which was submitted to Europhysics Letters) is available by anonymous ftp (instructions below). FINITE SIZE EFFECTS IN ON-LINE LEARNING OF MULTI-LAYER NEURAL NETWORKS David Barber{2}, Peter Sollich{1} and David Saad{2} {1} Department of Physics, University of Edinburgh, EH9 3JZ, UK {2} Neural Computing Research Group, Aston University, Birmingham B4 7ET, United Kingdom email: D.Barber at aston.ac.uk Abstract We complement the recent progress in thermodynamic limit analyses of mean on-line gradient descent learning dynamics in multi-layer networks by calculating the fluctuations possessed by finite dimensional systems. Fluctuations from the mean dynamics are largest at the onset of specialisation as student hidden unit weight vectors begin to imitate specific teacher vectors, and increase with the degree of symmetry of the initial conditions. Including a term to stimulate asymmetry in the learning process typically significantly decreases finite size effects and training time. Ftp instructions ftp cs.aston.ac.uk User: anonymous Password: (type your e-mail address) ftp> cd neural/barberd ftp> binary ftp> get online.ps.Z ftp> quit unix> uncompress online.ps.Z **PLEASE DO NOT REPLY DIRECTLY TO THIS MESSAGE** From saadd at helios.aston.ac.uk Thu Nov 16 06:44:25 1995 From: saadd at helios.aston.ac.uk (saadd) Date: Thu, 16 Nov 1995 11:44:25 +0000 (GMT) Subject: NIPS workshop: The Dynamics Of On-Line Learning Message-ID: <644.9511161144@sun.aston.ac.uk> THE DYNAMICS OF ON-LINE LEARNING NIPS workshop, Friday and Saturday, December 2-3, 1995 7:30AM to 9:30AM -- 4:30PM to 6:30PM Organizers: Sara A. Solla (CONNECT, The Niels Bohr Institute) and David Saad (Aston University) On-line learning refers to a scenario in which the couplings of the learning machine are updated after the presentation of each example. The current hypothesis is used to predict an output for the current input; the corresponding error signal is used for weight modification, and the modified hypothesis is used for output prediction at the subsequent time step. This type of algorithm addresses general questions of learning dynamics, and has attracted the attention of both the computational learning theory and the statistical physics communities. Recent progress has provided tools that allow for the investigation of learning scenarios that incorporate many of the aspects of the learning of complex tasks: multilayer architectures, noisy data, regularization through weight decay, the use of momentum, tracking changing environments, presentation order when cycling repeatedly through a finite training set... An open and somewhat controversial question to be discussed in the workshop is the role of the learning rate in controlling the evolution and convergence of the learning process. The purpose of the workshop is to review the theoretical tools available for the analysis of on-line learning, to evaluate the current state of research in the field, and to predict possible contributions to the understanding and description of real world learning scenarios. We also seek to identify future research directions using these methods, their limitations and expected difficulties. The topics to be addressed in this workshop can be grouped as follows: 1) The investigation of on-line learning from the point of view of stochastic approximation theory. This approach is based on formulating a master equation to describe the dynamical evolution of a probability density which describes the ensemble of trained networks in the space of weights of the student network. (Todd Leen, Bert Kappen, Jenny Orr) 2) The investigation of on-line learning from the point of view of statistical mechanics. This approach is based on the derivation of dynamical equations for the overlaps among the weight vectors associated with the various hidden units in both student and teacher networks. The dynamical evolution of the overlaps provides a detailed characterization of the learning process and determines the generalization error. (Sara Solla, David Saad, Peter Riegler, David Barber, Ansgar West, Naama Barkai, Jason Freeman, Adam Prugel-Bennett) 3) The identification of optimal strategies for on-line learning. Most of the work has concentrated on the learning of classification tasks. A recent Bayesian formulation of the problem provides a unified derivation of optimal on-line equations. (Shun-ichi Amari, Nestor Caticha, Manfred Opper) The names in parenthesis identify the speakers. A list of additional participants includes Michael Kearns, Yann Le Cun, Yoshiyuki Kabashima, Mauro Copelli, Noboru Murata, Klaus Mueller. PROGRAM FOR THE NIPS WORKSHOP ON "THE DYNAMICS OF ON-LINE LEARNING" Friday ------ Morning: 7:30AM to 9:30AM Introduction: Todd Leen Speakers: Jenny Orr Bert Kappen Afternoon: 4:30PM to 6:30PM Speakers: Shun-ichi Amari Nestor Caticha Manfred Opper Saturday -------- Morning: 7:30AM to 9:30AM Introduction: Sara Solla/David Saad Speakers: David Barber Ansgar West Peter Riegler Afternoon: 4:30PM to 6:30PM Speakers: Adam Prugel-Bennett Jason Freeman Peter Riegler Naama Barkai More details, abstracts and references can be found on: http://neural-server.aston.ac.uk/nips95/workshop.html From thrun+ at HEAVEN.LEARNING.CS.CMU.EDU Wed Nov 15 20:39:04 1995 From: thrun+ at HEAVEN.LEARNING.CS.CMU.EDU (thrun+@HEAVEN.LEARNING.CS.CMU.EDU) Date: Wed, 15 Nov 95 20:39:04 EST Subject: 2 papers available Message-ID: Dear Colleagues: I am happy to announce two new papers: Lifelong Learning: A Case Study Sebastian Thrun Machine learning has not yet succeeded in the design of robust learning algorithms that generalize well from very small datasets. In contrast, humans often generalize correctly from only a single training example, even if the number of potentially relevant features is large. To do so, they successfully exploit knowledge acquired in previous learning tasks, to bias subsequent learning. This paper investigates learning in a lifelong context. Lifelong learning addresses situations where a learner faces a stream of learning tasks. Such scenarios provide the opportunity for synergetic effects that arise if knowledge is transferred across multiple learning tasks. To study the utility of transfer, several approaches to lifelong learning are proposed and evaluated in an object recognition domain. It is shown that all these algorithms generalize consistently more accurately from scarce training data than comparable "single-task" approaches. World Wide Web URL: http://www.cs.cmu.edu/~thrun/papers/thrun.lll_case_study.ps.Z ------------------------------------------------------------------- Clustering Learning Tasks and the Selective Cross-Task Transfer of Knowledge Sebastian Thrun and Joseph O'Sullivan Recently, there has been an increased interest in machine learning methods that learn from more than one learning task. Such methods have repeatedly found to outperform conventional, single-task learning algorithms when learning tasks are appropriately related. To increase robustness of these approaches, methods are desirable that can reason about the relatedness of individual learning tasks, in order to avoid the danger arising from tasks that are unrelated and thus potentially misleading. This paper describes the task-clustering (TC) algorithm. TC clusters learning tasks into classes of mutually related tasks. When facing a new thing to learn, TC first determines the most related task cluster, then exploits information selectively from this task cluster only. An empirical study carried out in a mobile robot domain shows that TC outperforms its unselective counterpart in situations where only a small number of tasks is relevant. World Wide Web URL: http://www.cs.cmu.edu/~thrun/papers/thrun.TC.ps.Z ------------------------------------------------------------------- INSTRUCTIONS FOR RETRIEVAL: (a) If you have access to the World Wide Web, you can retrieve the documents from my homepage (URL: http://www.cs.cmu.edu/~thrun, follow the paper link) or access them directly: netscape http://www.cs.cmu.edu/~thrun/papers/thrun.lll_case_study.ps.Z netscape http://www.cs.cmu.edu/~thrun/papers/thrun.TC.ps.Z (b) If you instead wish to retrieve the documents via anonymous ftp, follow these instructions: unix> ftp uran.informatik.uni-bonn.de user: anonymous passwd: aaa at bbb.ccc ftp> cd pub/user/thrun ftp> bin ftp> get thrun.lll_case_study.ps.Z ftp> get thrun.TC.ps.Z ftp> bye unix> uncompress thrun.lll_case_study.ps.Z unix> uncompress thrun.TC.ps.Z unix> lpr thrun.lll_case_study.ps.Z unix> lpr thrun.TC.ps.Z (c) Hard-copies can be obtained directly from Technical Reports Computer Science Department Carnegie Mellon University 5000 Forbes Ave Pittsburgh, PA 15213 Email: reports at cs.cmu.edu Please refer to the first paper as TR CMU-CS-95-208 and the second paper as TR CMU-CS-95-209 Comments are welcome! Sebastian Thrun From juergen at idsia.ch Wed Nov 15 04:35:42 1995 From: juergen at idsia.ch (Juergen Schmidhuber) Date: Wed, 15 Nov 95 10:35:42 +0100 Subject: response: Levin search Message-ID: <9511150935.AA03192@fava.idsia.ch> My response to David Wolpert's response to my response: Although it is true that ``we have *no* a priori reason to believe that targets with low Kolmogorov complexity (or anything else) are/not likely to occur in the real world'', it's also true that Levin search (LS) has the optimal order of search complexity for a broad class of *non-incremental* search problems (David did not agree). This is a well-known fact of theoretical computer science. Why is LS as efficient as any other search algorithm? Because LS effectively *runs* all the other search algorithms, but in a smart way that prevents it from loosing too much time with "wrong" search algorithms. David is right, however, by saying that ``In practice, it may (or may not) be a good idea to use an algorithm that searches for low Levin complexity rather than one that works by other means.'' One reason for this is: in practice, your problem size is always limited, and you *do* have to worry about the (possibly huge) constant buried in the notion of "optimal order of search complexity". Note that all my recent comments refer to *non-incremental* search --- for the moment, I am not addressing generalization issues. Can LS help us to improve certain kinds of *incremental* search and learning? We are currently trying to figure this one out. Juergen Schmidhuber From schwenk at robo.jussieu.fr Fri Nov 17 15:28:07 1995 From: schwenk at robo.jussieu.fr (Holger Schwenk) Date: Fri, 17 Nov 1995 20:28:07 +0000 (WET) Subject: paper available (OCR, discriminant tangent distance) Message-ID: <951117202807.22380000.adc15034@lea.robo.jussieu.fr> **DO NOT FORWARD TO OTHER GROUPS** FTP-host: ftp.robo.jussieu.fr FTP-filename: /pub/papers/schwenk.icann95.ps.gz (6 pages, 31k) The following paper, published in International Conference on Artificial Neural Networks (ICANN*95), Springer Verlag, is available via anonymous FTP at the above location. The paper is 6 pages long. Sorry, no hardcopies available. =========================================================================== H. Schwenk and M. Milgram PARC - boite 164 Universite Pierre et Marie Curie 4, place Jussieu 75252 Paris cedex 05, FRANCE ABSTRACT Transformation invariance is known to be fundamental for excellent performances in pattern recognition. One of the most successful approach is tangent distance, originally proposed for a nearest-neighbor algorithm (Simard et al.,1995). The resulting classifier, however, has a very high computational complexity and, perhaps more important, lacks discrimination capabilities. We present a discriminant learning algorithm for a modular classifier based on several autoassociative neural networks. Tangent distance as objective function guarantees efficient incorporation of transformation invariance. The system achieved a raw error rate of 2.6% and a rejection rate of 3.6% on the NIST uppercase letters. ============================================================================ FTP instructions: unix> ftp ftp.robo.jussieu.fr Name: anonymous Password: your full email address ftp> cd pub/papers ftp> bin ftp> get schwenk.icann95.ps.gz ftp> quit unix> gunzip schwenk.icann95.ps.gz unix> lp schwenk.icann95.ps (or however you print postscript) I welcome your comments. --------------------------------------------------------------------- Holger Schwenk PARC - boite 164 tel: (+33 1) 44.27.63.08 Universite Pierre et Marie Curie fax: (+33 1) 44.27.62.14 4, place Jussieu 75252 Paris cedex 05 email: schwenk at robo.jussieu.fr FRANCE --------------------------------------------------------------------- From schwenk at robo.jussieu.fr Fri Nov 17 15:28:07 1995 From: schwenk at robo.jussieu.fr (Holger Schwenk) Date: Fri, 17 Nov 1995 20:28:07 +0000 (WET) Subject: paper available (OCR, discriminant tangent distance) Message-ID: <951117202807.22380000.adc15034@lea.robo.jussieu.fr> **DO NOT FORWARD TO OTHER GROUPS** FTP-host: ftp.robo.jussieu.fr FTP-filename: /pub/papers/schwenk.icann95.ps.gz (6 pages, 31k) The following paper, published in International Conference on Artificial Neural Networks (ICANN*95), Springer Verlag, is available via anonymous FTP at the above location. The paper is 6 pages long. Sorry, no hardcopies available. =========================================================================== H. Schwenk and M. Milgram PARC - boite 164 Universite Pierre et Marie Curie 4, place Jussieu 75252 Paris cedex 05, FRANCE ABSTRACT Transformation invariance is known to be fundamental for excellent performances in pattern recognition. One of the most successful approach is tangent distance, originally proposed for a nearest-neighbor algorithm (Simard et al.,1995). The resulting classifier, however, has a very high computational complexity and, perhaps more important, lacks discrimination capabilities. We present a discriminant learning algorithm for a modular classifier based on several autoassociative neural networks. Tangent distance as objective function guarantees efficient incorporation of transformation invariance. The system achieved a raw error rate of 2.6% and a rejection rate of 3.6% on the NIST uppercase letters. ============================================================================ FTP instructions: unix> ftp ftp.robo.jussieu.fr Name: anonymous Password: your full email address ftp> cd pub/papers ftp> bin ftp> get schwenk.icann95.ps.gz ftp> quit unix> gunzip schwenk.icann95.ps.gz unix> lp schwenk.icann95.ps (or however you print postscript) I welcome your comments. --------------------------------------------------------------------- Holger Schwenk PARC - boite 164 tel: (+33 1) 44.27.63.08 Universite Pierre et Marie Curie fax: (+33 1) 44.27.62.14 4, place Jussieu 75252 Paris cedex 05 email: schwenk at robo.jussieu.fr FRANCE --------------------------------------------------------------------- From horne at research.nj.nec.com Fri Nov 17 15:16:49 1995 From: horne at research.nj.nec.com (Bill Horne) Date: Fri, 17 Nov 1995 15:16:49 -0500 Subject: NIPS*95 Workshop on Neural Networks for Signal Processing Message-ID: <9511171516.ZM2540@telluride> **** FINAL SCHEDULE **** NIPS*95 Workshop Neural Networks for Signal Processing Friday Dec 1, 1995 Marriott Vail Mountain Resort, Colorado ORGANIZERS Andrew D. Back C. Lee Giles and Bill G. Horne University of Queensland NEC Research Institute back at elec.uq.edu.au {giles,horne}@research.nj.nec.com WORKSHOP AIMS Nonlinear signal processing methods using neural network models form a topic of some recent interest. A common goal is for neural network models to outperform traditional linear and nonlinear models. Many researchers are interested in understanding, analysing and improving the performance of these nonlinear models by drawing from the well established base of linear systems theory and existing knowledge in other areas. How can this be best achieved? In the context of neural network models, a variety of methods have been proposed for capturing the time-dependence of signals. A common approach is to use recurrent connections or time-delays within the network structure. On the other hand, many signal processing techniques have been well developed over the last few decades. Recently, a strong interest has developed in understanding how better signal processing techniques can be developed by considering these different approaches. A major aim of this workshop is to obtain a better understanding of how well this development is proceeding. For example, the different model structures raise the question, "how suitable are the various neural networks for signal processing problems?". The success of some neural network models in signal processing problems indicate that they form a class of potentially powerful modeling methods, yet relatively little is understood about these architectures in the context of signal processing. As an outcome of the workshop it is intended that there should be a summary of current progress and goals for future work in this research area. SCHEDULE OF TALKS ** Session 1 - Speech Focus ** 7:30-7:40: Introduction 7:40-8:10: Herve Bourlard, ICSI and Faculte Polytechnique de Mons, "Hybrid use of hidden Markov models and neural networks for improving state-of-the-art speech recognition systems" 8:10-8:40: John Hogden, Los Alamos National Laboratory, "A maximum likelihood approach to estimating speech articulator positions from speech acoustics" 8:40-9:10: Shun-ichi Amari, A.Cichocki and H. Yang, RIKEN, "Blind separation of signals - Information geometric point of view" 9:10-9:30: Discussion ** Session 2 - Recurrent Network Focus ** 4:30-4:40: Introduction 4:40-5:10: Andrew Back, University of Queensland, "Issues in signal processing relevant to dynamic neural networks" 5:10-5:40: John Steele and Aaron Gordon, Colorado School of Mines, "Hierarchies of recurrent neural networks for signal interpretation with applications" 5:40-6:10: Stephen Piche, Pavilion Technologies, "Discrete Event Recurrent Neural Networks" 6:10-6:30: Open Forum, discussion time. For more information about the workshop see the workshop homepage: http://www.elec.uq.edu.au/~back/nips95ws/nips95ws.html or contact: Andrew D. Back Department of Electrical and Computer Engineering, University of Queensland, Brisbane, Qld 4072. Australia Ph: +61 7 365 3965 Fax: +61 7 365 4999 back at .elec.uq.edu.au C. Lee Giles, Bill G. Horne NEC Research Institute 4 Independence Way Princeton, NJ 08540. USA Ph: 609 951 2642, 2676 Fax: 609 951 2482 {giles,horne}@research.nj.nec.com -- Bill Horne horne at research.nj.nec.com http://www.neci.nj.nec.com/homepages/horne.html PHN: (609) 951-2676 FAX: (609) 951-2482 NEC Research Institute, 4 Independence Way, Princeton, NJ 08540 From rosen at unr.edu Fri Nov 17 18:59:23 1995 From: rosen at unr.edu (David B. Rosen) Date: Fri, 17 Nov 1995 15:59:23 -0800 Subject: Missing Data Workshop -- Final Announcement Message-ID: <199511180003.QAA24231@archive.ccs.unr.edu> This is the final email announcement (with updated list of presentations) for: MISSING DATA: METHODS AND MODELS A NIPS*95 Workshop Friday, December 1, 1995 INTRODUCTION Incomplete or missing data, typically unobserved or unavailable features in supervised learning, is an important problem often encountered in real-world data sets and applications. Assumptions about the missing-data mechanism are often not stated explicitly, for example independence between this mechanism and the values of the (missing or other) features themselves. In the important case of incomplete ~training~ data, one often discards incomplete rows or columns of the data matrix, throwing out some useful information along with the missing data. Ad hoc or univariate methods such as imputing the mean or mode are dangerous as they can sometimes give much worse results than simple discarding. Overcoming the problem of missing data often requires that we model not just the dependence of the output on the inputs, but the inputs among themselves as well. THE WORKSHOP This one-day workshop should provide a valuable opportunity to share and discuss methods and models used for missing data. The following short talks will be presented, with questions and discussion following each. o Leo Breiman, U.C. Berkeley Formal and ad hoc ways of handling missing data o Zoubin Ghahramani, U. Toronto Mixture models and missing data o Steffen Lauritzen and Bo Thiesson, Aalborg U. Learning Bayesian networks from incomplete data o Brian Ripley, Oxford U. Multiple imputation and simulation methods o Brian Ripley, Oxford U. Multiple imputation for neural nets and classification trees o Robert Tibshirani and Geoffrey Hinton, U. Toronto ``Coaching'' variables for regression and classification o Volker Tresp, Siemens AG Missing data: A fundamental problem in learning FURTHER INFORMATION The above is a snapshot of the workshop's web page: http://www.scs.unr.edu/~cbmr/nips/workshop-missing.html A schedule of presentation times is not yet available as of today. Sincerely, (the organizers:) Harry Burke David Rosen New York Medical College, Department of Medicine, Valhalla NY 10595 USA From at at cogsci.soton.ac.uk Sat Nov 18 05:34:40 1995 From: at at cogsci.soton.ac.uk (Adriaan Tijsseling) Date: Sat, 18 Nov 1995 10:34:40 GMT Subject: Changes to CogPsy Mailinglist Message-ID: <9511181034.AA05833@cogsci.soton.ac.uk> Dear Colleagues, There are a few changes to the CogPsy Mailinglist, but before I outline the changes, first a small description of the mailing list: The cogpsy mailing list is intended for discussion of issues and the dissemination of information important to researchers in all fields of cognitive science, especially connectionist cognitive psychology. Contributions could include: announcements of new techreports, dissertations, theses, conferences, seminars or courses discussions of research issues (including those arising from articles in Noetica) requests for information about bibliographic issues reviews of software or hardware packages The changes are: a new address: contributions should be send to cogpsy at neuro.psy.soton.ac.uk; subscriptions to cogpsy-request at neuro.psy.soton.ac.uk (make sure the Subject: field contains the word "subscribe"). a fusion with Noetica, a electronic journal on cognitive science, the url of which is: http://psych.psy.uq.oz.au/CogPsych/Noetica/ two accompanying webpages: one containing the latest contributions to the list, which is on http://www.soton.ac.uk/~coglab/coglab/CogPsy/ the other is a database of current projects in the field of cognitive science, listed by discipline and containing all information about the projects, including email- and URL-addresses. Feel free to add your own project!. It's on: http://neuro.psy.soton.ac.uk/~at/ With kind regards, Adriaan Tijsseling, CogPsy Moderator From koza at CS.Stanford.EDU Thu Nov 16 09:44:51 1995 From: koza at CS.Stanford.EDU (John Koza) Date: Thu, 16 Nov 95 6:44:51 PST Subject: CFP: Genetic Programming 1996 Conference (GP-96) Message-ID: ------------------------------------------------ Paper Submission Deadline: January 10, 1996 (Wednesday) ------------------------------------------------ CALL FOR PAPERS AND PARTICIPATION Genetic Programming 1996 Conference (GP-96) July 28 - 31 (Sunday - Wednesday), 1996 Fairchild Auditorium - Stanford University - Stanford, California Proceedings will be published by The MIT Press In cooperation with the Association for Computing Machinery (ACM), SIGART, the IEEE Neural Network Council, and the American Association for Artificial Intelligence. Genetic programming is a domain-independent method for evolving computer programs that solve, or approximately solve, problems. Starting with a primordial ooze of thousands of randomly created programs composed of functions and terminals appropriate to a problem, a genetic population is progressively evolved over many generations by applying the Darwinian principle of survival of the fittest, a sexual recombination operation, and occasional mutation. This first genetic programming conference will feature contributed papers, tutorials, invited speakers, and informal meetings. Topics include, but are not limited to, applications of genetic programming, theoretical foundations of genetic programming, implementation issues, parallelization techniques, technique extensions, implementations of memory and state, representation issues, new operators, architectural evolution, evolution of mental models, cellular encoding, evolution of machine language programs, evolvable hardware, combinations with other machine learning techniques, and relations to biology and cognitive systems. ------------------------------------------------- HONORARY CHAIR: John Holland, University of Michigan INVITED SPEAKERS: John Holland, University of Michigan and David E. Goldberg, University of Illinois GENERAL CHAIR: John Koza, Stanford University PUBLICITY CHAIR: Patrick Tufts, Brandeis University ------------------------------------------------- SPECIAL PROGRAM CHAIRS: The main focus of the conference (and about two-thirds of the papers) will be on genetic programming. In addition, papers describing recent developments in the following closely related areas of evolutionary computation (particularly those addressing issues common to various areas of evolutionary computation) will be reviewed by special program committees appointed and supervised by the following special program chairs. - GENETIC ALGORITHMS: David E. Goldberg, University of Illinois, Urbana, Illinois - CLASSIFIER SYSTEMS: Rick Riolo, University of Michigan - EVOLUTIONARY PROGRAMMING AND EVOLUTION STRATEGIES: David Fogel, University of California, San Diego, California ------------------------------------------------- TUTORIALS -Sunday July 28 9:15 AM - 11:30 AM - Genetic Algorithms - David E. Goldberg, University of Illinois - Machine Language Genetic Programming - Peter Nordin, University of Dortmund, Germany - Genetic Programming using Mathematica P Robert Nachbar P Merck Research Laboratories - Introduction to Genetic Programming - John Koza, Stanford University ------------------------------------------------- Sunday July 28 1:00 PM - 3: 15 PM - Classifier Systems- Robert Elliott Smith, University of Alabama - Evolutionary Computation for Constraint Optimization - Zbigniew Michalewicz, University of North Carolina - Advanced Genetic Programming - John Koza, Stanford University ------------------------------------------------- Sunday July 28 3:45 PM - 6 PM - Evolutionary Programming and Evolution Strategies - David Fogel, University of California, San Diego - Cellular Encoding P Frederic Gruau, Stanford University (via videotape) and David Andre, Stanford University (in person) - Genetic Programming with Linear Genomes (one hour) - Wolfgang Banzhaf, University of Dortmund, Germany -JECHO - Terry Jones, Santa Fe Institute ------------------------------------------------- Tuesday July 30 - 3 PM - 5:15PM - Neural Networks - David E. Rumelhart, Stanford University - Machine Learning - Pat Langley, Stanford University -JMolecular Biology for Computer Scientists - Russ B. Altman, Stanford University ------------------------------------------------- INFORMATION FOR SUBMITTING PAPERS The deadline for receipt at the physical mail address below of seven (7) copies of each submitted paper is Wednesday, January 10, 1996. Papers are to be in single-spaced, 12- point type on 8 1/2" x 11" or A4 paper (no e-mail or fax) with full 1" margins at top, bottom, left, and right. Papers are to contain ALL of the following 9 items, within a maximum of 10 pages, IN THIS ORDER: (1) title of paper, (2) author name(s), (3) author physical address(es), (4) author e-mail address(es), (5) author phone number(s), (6) a 100-200 word abstract of the paper, (7) the paper's category (chosen from one of the following five alternatives: genetic programming, genetic algorithms, classifier systems, evolutionary programming, or evolution strategies), (8) the text of the paper (including all figures and tables), and (9) bibliography. All other elements of the paper (e.g., acknowledgments, appendices, if any) must come within the maximum of 10 pages. Review criteria will include significance of the work, novelty, sufficiency of information to permit replication (if applicable), clarity, and writing quality. The first-named (or other designated) author will be notified of acceptance or rejection by approximately Monday February 26, 1996. The style of the camera-ready paper will be identical to that of the 1994 Simulation of Adaptive Behavior conference published by the MIT Press. Depending on the number, subject, and content of the submitted papers, the program committee may decide to allocate different number of pages to various accepted papers. The deadline for the camera-ready, revised version of accepted papers will be announced, but will be approximately Wednesday March 20, 1996. Proceedings will be published by The MIT Press and will be available at the conference (and, if requested, by priority mail to registered conference attendees with U.S. addresses just prior to the conference). One author will be expected to present each accepted paper at the conference. ------------------------------------------------- FOR MORE INFORMATION ABOUT THE GP-96 CONFERENCE: On the World Wide Web: http://www.cs.brandeis.edu/~zippy/gp-96.html or via e-mail at gp at aaai.org. Conference operated by Genetic Programming Conferences, Inc. (a California not-for-profit corporation). ------------------------------------------------- FOR MORE INFORMATION ABOUT GENETIC PROGRAMMING IN GENERAL: http://www-cs- faculty.stanford.edu/~koza/. ------------------------------------------------- FOR MORE INFORMATION ABOUT DISCOUNTED TRAVEL : For further information regarding special GP-96 airline and car rental rates, please contact Conventions in America at e-mail flycia at balboa.com; or phone 1-800-929-4242; or phone 619-678-3600; or FAX 619-678-3699. ------------------------------------------------- FOR MORE INFORMATION ABOUT THE SAN FRANCISCO BAY AREA AND SILICON VALLEY AREA SIGHTS: Try the Stanford University home page at http://www.stanford.edu/, the Hyperion Guide at http://www.hyperion.com/ba/sfbay.html; the Palo Alto weekly at http://www.service.com/PAW/home.html; the California Virtual Tourist at http://www.research.digital.com/SRC/virtual- tourist/California.html; and the Yahoo Guide of San Francisco at http://www.yahoo.com/Regional_Information/States/Califo rnia/San_Francisco. ------------------------------------------------- FOR MORE INFORMATION ABOUT CONTEMPORANEOUS WEST COAST CONFERENCES: Information about the AAAI-96 conference on August 4 P 8 (Sunday P Thursday), 1996, in Portland, Oregon can be found at http://www.aaai.org/. For information on the International Conference on Knowledge Discovery and Data Mining (KDD-96) in Portland, Oregon, on August 3- 5, 1996: http://www-aig.jpl.nasa.gov/kdd96. Information about the Foundations of Genetic Algorithms (FOGA) workshop on August 3 P 5 (Saturday P Monday), 1996, in San Diego, California can be found at http://www.aic.nrl.navy.mil/galist/foga/ or by contacting belew at cs.wisc.edu. ------------------------------------------------- FOR MORE INFORMATION ABOUT MEMBERSHIP IN THE ACM, AAAI, or IEEE: For information about ACM membership, try http://www.acm.org/; for information about SIGART, try http://sigart.acm.org/; for AAAI membership, go to http://www.aaai.org/; and for membership in the IEEE Computer Society, go to http://www.computer.org. ------------------------------------------------- PHYSICAL MAIL ADDRESS FOR GP-96: GP-96 Conference, c/o American Association for Artificial Intelligence, 445 Burgess Drive, Menlo Park, CA 94025. PHONE: 415-328-3123. FAX: 415-321-4457. WWW: http://www.aaai.org/. E-MAIL: gp at aaai.org. ------------------------------------------------ REGISTRATION FORM FOR GENETIC PROGRAMMING 1996 CONFERENCE TO BE HELD ON JULY 28 P 31, 1996 AT STANFORD UNIVERSITY First Name _________________________ Last Name_______________ Affiliation________________________________ Address__________________________________ ________________________________________ City__________________________ State/Province _________________ Zip/Postal Code____________________ Country__________________ Daytime telephone__________________________ E-Mail address_____________________________ Conference registration fee includes copy of proceedings, attendance at 4 tutorials of your choice, syllabus books for 4 tutorials, conference reception, and admission to conference. Students must send legible proof of full-time student status. Conference proceedings will be mailed to registered attendees with U.S. mailing addresses via 2-day U.S. priority mail 1 P 2 weeks prior to the conference at no extra charge (at addressee's risk). If you are uncertain as to whether you will be at that address at that time or DO NOT WANT YOUR PROCEEDINGS MAILED to you at the above address for any other reason, your copy of the proceedings will be held for you at the conference registration desk if you CHECK HERE ____. Postmarked by May 15, 1996: Student P ACM, IEEE, or AAAI Member $195 Regular P ACM, IEEE, or AAAI Member $395 Student P Non-member $215 Regular P Non-member $415 Postmarked by June 26, 1996: Student P ACM, IEEE, or AAAI Member $245 Regular P ACM, IEEE, or AAAI Member $445 Student P Non-member $265 Regular P Non-member $465 Postmarked later or on-site: Student P ACM, IEEE, or AAAI Member $295 Regular P ACM, IEEE, or AAAI Member $495 Student P Non-member $315 Regular P Non-member $515 Member number: ACM # ___________ IEEE # _________ AAAI # _________ Total fee (enter appropriate amount) $ _________ __ Check or money order made payable to "AAAI" (in U.S. funds) __ Mastercard __ Visa __ American Express Credit card number __________________________________________ Expiration Date ___________ Signature _________________________ TUTORIALS: Check off a box for one tutorial from each of the 4 columns: Sunday July 28, 1996 P 9:15 AM - 11:30 AM __ Genetic Algorithms __ Machine Language GP __ GP using Mathematica __ Introductory GP Sunday July 28, 1996 P 1:00 PM - 3: 15 PM __ Classifier Systems __ EC for Constraint Optimization __ Advanced GP Sunday July 28, 1996 P 3:45 PM - 6 PM __ Evolutionary Programming and Evolution Strategies __ Cellular Encoding __ GP with Linear Genomes __ ECHO Tuesday July 30, 1996 P3:00 PM - 5:15PM __ Neural Networks __ Machine Learning __ Molecular Biology for Computer Scientists __ Check here for information about housing and meal package at Stanford University. __ Check here for information on student travel grants. No refunds will be made; however, we will transfer your registration to a person you designate upon notification. SEND TO: GP-96 Conference, c/o American Association for Artificial Intelligence, 445 Burgess Drive, Menlo Park, CA 94025. PHONE: 415- 328-3123. FAX: 415-321-4457. E-MAIL: gp at aaai.org. WWW: http://www.aaai.org/. ------------------------------------------------- PROGRAM COMMITTEE Russell J. Abbott California State University, Los Angeles and The Aerospace Corporation Hojjat Adeli Ohio State University Dennis Allison Stanford University Lee Altenberg Hawaii Institute of Geophysics and Planetology David Andre Stanford University Peter J. Angeline Loral Federal Systems Wolfgang Banzhaf University of Dortmund, Germany Rik Belew University of California at San Diego Samy Bengio Centre National d'Etudes des Telecommunications, France Forrest H. Bennett III Genetic Algorithms Technology Corporation Scott Brave Stanford University Bill P. Buckles Tulane University Walter Cedeno Primavera Systems Inc. Nichael Lynn Cramer BBN System and Technologies Jason Daida University of Michigan Patrik D'haeseleer University of New Mexico Marco Dorigo Universite' Libre de Bruxelles Bertrand Daniel Dunay System Dynamics International Andrew N. Edmonds Science in Finance Ltd., UK H.H. Ehrenburg CWI, The Netherlands Frank D. Francone FRISEC P Francone & Raymond Institute for the Study of Evolutionary Computation, Germany Adam P. Fraser University of Salford Alex Fukunaga University of California, Los Angeles Frederic Gruau Stanford University Richard J. Hampo Ford Motor Company Simon Handley Stanford University Thomas D. Haynes The University of Tulsa Hitoshi Hemmi ATR, Kyoto, Japan Vasant Honavar Iowa State University Thomas Huang University of Illinois Hitoshi Iba Electrotechnical Laboratory, Japan Christian Andrew Johnson Department of Economics, University of Santiago Martin A. Keane Econometrics Inc. Mike Keith Allen Bradley Controls Maarten Keijzer Kenneth E. Kinnear, Jr. Adaptive Computing Technology W. B. Langdon University College, London David Levine Argonne National Laboratory Kenneth Marko Ford Motor Company Martin C. Martin Carnegie Mellon University Sidney R Maxwell III Nicholas Freitag McPhee University of Minnesota, Morris David Montana BBN System and Technologies Heinz Muehlenbein GMD Research Center, Germany Robert B. Nachbar Merck Research Laboratories Peter Nordin University of Dortmund, Germany Howard Oakley Institute of Naval Medicine, UK Franz Oppacher Carleton University, Ottawa Una-May O`Reilly Carleton University, Ottawa Michael Papka Argonne National Laboratory Timothy Perkis Frederick E. Petry Tulane University Bill Punch Michigan State University Justinian P. Rosca University of Rochester Conor Ryan University College Cork, Ireland Malcolm Shute University of Brighton, UK Eric V. Siegel Columbia University Karl Sims Andrew Singleton Creation Mechanics Lee Spector Hampshire College Walter Alden Tackett Neuromedia Astro Teller Carnegie Mellon University Marco Tomassini Ecole Polytechnique Federale de Lausanne Patrick Tufts Brandeis University V. Rao Vemuri University of Califonia at Davis Peter A. Whigham Australia Darrell Whitley Colorado State University Man Leung Wong Chinese University of Hong Kong Alden H. Wright University of Montana Byoung-Tak Zhang GMD, Germany From maass at igi.tu-graz.ac.at Sun Nov 19 11:31:42 1995 From: maass at igi.tu-graz.ac.at (Wolfgang Maass) Date: Sun, 19 Nov 95 17:31:42 +0100 Subject: computing with noisy spiking neurons: paper in neuroprose Message-ID: <199511191631.AA28958@figids02.tu-graz.ac.at> The file maass.noisy-spiking.ps.Z is now available for copying from the Neuroprose repository. This is a 9-page long paper. Hardcopies are not available. FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/maass.noisy-spiking.ps.Z On the Computational Power of Noisy Spiking Neurons Wolfgang Maass Institute for Theoretical Computer Science Technische Universitaet Graz Klosterwiesgasse 32/2 A-8010 Graz, Austria e-mail: maass at igi.tu-graz.ac.at Abstract This article provides positive results about the computational power of neural networks that are based on a neuron model ("noisy spiking neuron") which is acceptable to most neurobiologists as being reasonably realistic for a biological neuron. In fact: this model tends to underestimate the computational capabilities of a biological neuron, since it simplifies dendritic integration. Biological neurons communicate via spike-trains, i.e. via sequences of stereotyped pulses (spikes) that encode information in their time- differences ("temporal coding"). In addition it is wellknown that biological neurons are quite "noisy". There is some "jitter" in their firing times, and neurons (as well as synapses) my fail to fire with a certain probability. It has remained unknown whether one can in principle carry out reliable computation in networks of noisy spiking neurons. This article presents rigorous constructions for simulating in real-time arbitrary given boolean circuits and finite automata on such networks. In addition we show that with the help of "shunting inhibition" such networks can simulate in real-time any McCulloch-Pitts neuron (or "threshold gate"), and therefore any multilayer perceptron (or "threshold circuit") in a reliable manner. In view of the tremendous computational power of threshold circuits (even with few layers), this construction provides a possible explanation for the fact that biological neural systems can carry out quite complex computations within 100 msec. It turns out that the assumptions that these constructions require about the shape of the EPSP's and the behaviour of the noise are surprisingly weak. This article continues the related work from NIPS '94, where we had considered computations on networks of spiking neurons without noise. The current paper will appear in Advances in Neural Information Processing Systems, vol. 8 (= Proc. of NIPS '95) . ************ How to obtain a copy ***************** Via Anonymous FTP: unix> ftp archive.cis.ohio-state.edu Name: anonymous Password: (type your email address) ftp> cd pub/neuroprose ftp> binary ftp> get maass.noisy-spiking.ps.Z ftp> quit unix> uncompress maass.noisy-spiking.ps.Z unix> lpr maass.noisy-spiking.ps (or what you normally do to print PostScript) From piuri at elet.polimi.it Sun Nov 19 11:35:55 1995 From: piuri at elet.polimi.it (Vincenzo Piuri) Date: Sun, 19 Nov 1995 17:35:55 +0100 Subject: NICROSP'96 - call for papers Message-ID: <9511191635.AA06093@ipmel2.elet.polimi.it> ====================================================================== NICROSP'96 1996 International Workshop on Neural Networks for Identification, Control, Robotics, and Signal/Image Processing Venice, Italy - 21-23 August 1996 ====================================================================== Sponsored by the IEEE Computer Society and the IEEE CS Technical Committee on Pattern Analysis and Machine Intelligence. In cooperation with: ACM SIGART (pending), IEEE Circuits and Systems Society, IEEE Control Systems Society, IEEE Instrumentation and Measurement Society, IEEE Neural Network Council, IEEE North-Italy Section, IEEE Region 8, IEEE Robotics and Automation Society (pending), IEEE Signal Processing Society (pending), IEEE System, Man, and Cybernetics Society, IMACS, INNS (pending), ISCA, AEI, AICA, ANIPLA, FAST. CALL FOR PAPERS This workshop is directed to create a unique synergetic discussion forum and a strong link between theoretical researchers and practitioners in the application fields of identification, control, robotics, and signal/image processing by using neural techniques. The three-days single-session schedule will provide the ideal environment for in-depth analysis and discussions concerning the theoretical aspects of the applications and the use of neural networks in the practice. Invited talks in each area will provide a starting point for the discussion and give the state of the art in the corresponding field. Panels will provide an interactive discussion. Researchers and practitioners are invited to submit papers concerning theoretical foundations of neural computation, experimental results or practical applications related to the specific workshop's areas. Interested authors should submit a half-page abstract to the program chair by e-mail or fax by February 1, 1996, for review planning. Then, an extended summary or the full paper (limited to 20 double-spaced pages including figures and tables) must be sent to the program chair by February 16, 1996 (PostScript email submission is strongly encouraged). Submissions should contain: the corresponding author, affiliation, complete address, fax, email, and the preferred workshop track (identification, control, robotics, signal processing, image processing). Submission implies the willingness of at least one of the authors to register, attend the workshop and present the paper. Papers' selection is based on the full paper: the corresponding author will be notified by March 30, 1996. The camera-ready version, limited to 10 one-column IEEE-book-standard pages, is due by May 1, 1996. Proceedings will be published by the IEEE Computer Society Press. The extended version of selected papers will be considered for publication in special issues of international journals. General Chair Prof. Edgar Sanchez-Sinencio Department of Electrical Engineering Texas A&M University College Station, TX 77843-3128 USA phone (409) 845-7498 fax (409) 845-7161 email sanchez at eesun1.tamu.edu Program Chair Prof. Vincenzo Piuri Department of Electronics and Information Politecnico di Milano piazza L. da Vinci 32, I-20133 Milano, Italy phone +39-2-2399-3606 fax +39-2-2399-3411 email piuri at elet.polimi.it Publication Chair Dr. Jose' Pineda de Gyvez Department of Electrical Engineering Texas A&M University Publicity, Registr. & Local Arrangment Chair Dr. Cesare Alippi Department of Electronics and Information Politecnico di Milano Workshop Secretariat Ms. Laura Caldirola Department of Electronics and Information Politecnico di Milano phone +39-2-2399-3623 fax +39-2-2399-3411 email caldirol at elet.polimi.it Program Committee (preliminary list) Shun-Ichi Amari, University of Tokyo, Japan Magdy Bayoumi, University of Southwestern Louisiana, USA James C. Bezdek, University of West Florida, USA Pierre Borne, Ecole Politechnique de Lille, France Luiz Caloba, Universidad Federal de Rio de Janeiro, Brazil Chris De Silva, University of Western Australia, Australia Laurene Fausett, Florida Institute of Technology, USA C. Lee Giles, NEC, USA Karl Goser, University of Dortmund, Germany Simon Jones, University of Loughborough, UK Michael Jordan, Massachussets Institute of Technology, USA Robert J. Marks II, University of Washington, USA Jean D. Nicoud, EPFL, Switzerland Eros Pasero, Politecnico di Torino, Italy Emil M. Petriu, University of Ottawa, Canada Alberto Prieto, Universidad de Granada, Spain Gianguido Rizzotto, SGS-Thomson, Italy Edgar Sanchez-Sinencio, A&M University, USA Bernd Schuermann, Siemens, Germany Earl E. Swartzlander, University of Texas at Austin, USA Philip Treleaven, University College London, UK Kenzo Watanabe, Shizuoka University, Japan Michel Weinfeld, Ecole Politechnique de Paris, France ====================================================================== From bap at sloan.salk.edu Sun Nov 19 18:03:37 1995 From: bap at sloan.salk.edu (Barak Pearlmutter) Date: Sun, 19 Nov 1995 15:03:37 -0800 Subject: Response to no-free-lunch discussion In-Reply-To: <9511141824.AA04471@sfi.santafe.edu> (message from David Wolpert on Tue, 14 Nov 95 11:24:58 MST) Message-ID: <199511192303.PAA07609@valaga.salk.edu> Reviewing the theory of Kolmogorov complexity, we see that not having low Kolmogorov complexity is equivalent to being random. In other words, the minimal description of anything that does not have low Kolmogorov complexity is "nothing but noise." When you write this We have *no* a priori reason to believe that targets with "low Kolmogorov complexity" (or anything else) are/not likely to occur in the real world. it is precisely equivalent to saying We have *no* a priori reason to believe that targets with any non-random structure whatsoever are likely to occur in the real world. The NFL theory shows conclusively that there are no search algorithms which are particularly good at finding minima of such random functions. However, as scientists, we have had some success by positing the existence of non-random structure in the real world. So it seems to me that there is at least some reason to believe that the functions we optimize in practice are not completely random, in this Kolmogorov complexity sense. From bernabe at cnm.us.es Mon Nov 20 06:37:13 1995 From: bernabe at cnm.us.es (Bernabe Linares B.) Date: Mon, 20 Nov 95 12:37:13 +0100 Subject: No subject Message-ID: <9511201137.AA12650@cnm1.cnm.us.es> NIPS'95 WORKSHOP ANNOUNCEMENT Vail, Colorado. Friday, December 1st, 1995 Title: NEURAL HARDWARE ENGINEERING: From Sensory Data Adquisition to High-Level Intelligent Processing Organizers: Bernabe Linares-Barranco and Angel Rodriguez-Vazquez Dept. of Analog and Mixed-Signal Circuit Design Microelectronics National Center, Sevilla Ed. CICA, Av. Reina Mercedes s/n, 41012 Sevilla, SPAIN FAX: 34-5-4624506; Phone: 34-5-4239923; email: bernabe at cnm.us.es DESCRIPTION OF THE WORKSHOP: Developing hardware for neural applications is a task that hardware engineers have faced during the past years using two distinct main approaches: (a) producing "general purpose" digital neuro-computing systems or "neural- -accelerators" with a certain degree of flexibility to emulate different neural architectures and learning rules. (b) developing "special purpose" neuro-chips, mostly using analog or mixed analog/digital circuit techniques, intended to solve a specific problem with very high speed and efficiency. Usually hardware of task (b) is used for the front end of a neural processing system, such as sensory data (image/sound) adquisition and sometimes with some extra (pre)processing functionality (noise removal, automatic gain, dimensionality reduction). On the other hand, hardware of task (a) is employed for more "intelligent" or higher level processing such as learning, clustering, recognition, abstraction, and conceptualization. However, the limits between hardware of type (a) and (b) are not very clear, and as more hardware is developed the overlap between the two approaches increases. Digital technology provides larger accuracy in the realization of mathematical operations and offers great flexibility to change learning paradigms, learning rules, or to tune critical parameters. Analog technology, on the other hand, provides very high area and power efficiency, but is less accurate and flexible. It is clear that people that are developing neural algorithms need to have some type of digital neurocomputer system where they can change rapidly the neural architecture, the topology, the learning rules, try different mathematical functions, and all that with sufficient flexibility and computing power. On the other hand, when neural systems require image or sound adquisition capabilities (retinas or cochleas) analog technology offers very high power and chip area efficiency, so that this approach seems to be the preferred one. However, what happens when it comes to develop a hardware system that needs to handle the sensory data, perform some basic processing, and continue processing up to higher level stages where data segmentation has to be performed, recognition on the segments has to be achieved, and learning and abstraction should be fulfilled? Is there any clear border among analog and digital techniques as we proceed upwards in the processing cycle from signal acquisition to conceptualization? Is it possible to take advantage of the synergy between analog and digital? How? Are these conclusions the same for vision, hearing, olfactory, or intelligent control applications? We believe it is a good point in time to make a debate between representatives of the two approaches, since both have evolved independently into a large enough degree of development and maturity as to enable pros and counters be discussed on the basis of objective, rather than subjective considerations. LIST OF SPEAKERS: 1. Nelson Morgan, University of California, Berkeley, U.S.A. "Using A Fixed-point Vector Microprocessor for Connectionist Speech Recognition Training" 2. Yuzo Hirai, Institute of Information Sciences and Electronics, University of Tsukuba, Japan. "PDM Digital Neural Networks" 3. Taher Daud, Jet Propulsion Laboratory, Pasadena, California, U.S.A. "Focal Plane Imaging Array-Integrated 3-D Neuroprocessor" 4. Ulrich Ramacher, University of Technology, Dresden, Germany. "The Siemens Electronic Eye Proyect" 5. Xavier Arreguit, CSEM, Neuchatel, Switzerland "Analog VLSI for Perceptive Systems" 6. Andreas Andreou, John Hopkins University, Baltimore, Maryland, U.S.A. "Silicon Retinas for Contrast Sensitivity and Polarization Sensing" 7. Marwan Jabri, University of Sydney, Australia. "On-Chip Learning in Analog VLSI and its Application in Biomedical Implant Devices" 8. Tadashi Shibata, Tohoku University, Sendai, Japan. "Neuron-MOS Binary-Analog Merged Hardware Computation for Intelligent Information Processing" From stefano at kant.irmkant.rm.cnr.it Mon Nov 20 13:15:32 1995 From: stefano at kant.irmkant.rm.cnr.it (stefano@kant.irmkant.rm.cnr.it) Date: Mon, 20 Nov 1995 18:15:32 GMT Subject: Paper available: Learning to adapt to changing environments Message-ID: <9511201815.AA16840@kant.irmkant.rm.cnr.it> Papers available via WWW / FTP: Keywords: Learning, Adaptation to changing environments, Evolutionary Robotics Neural Networks, Genetic Algorithms, ------------------------------------------------------------------------------ LEARNING TO ADAPT TO CHANGING ENVIRONMENTS IN EVOLVING NEURAL NETWORKS Stefano Nolfi & Domenico Parisi Institute of Psychology, C.N.R., Rome. In order to study learning as an adaptive process it is necessary to take into consideration the role of evolution which is the primary adaptive process. In addition, learning should be studied in (artificial) organisms that live in an independent physical environment in such a way that the input from the environment can be at least partially controlled by the organisms' behavior. To explore these issues we used a genetic algorithm to simulate the evolution of a population of neural networks each controlling the behavior of a small mobile robot that must explore efficiently an environment surrounded by walls. Since the environment changes from one generation to the next each network must learn during its life to adapt to the particular environment it happens to be born in. We found that evolved networks incorporate a genetically inherited predisposition to learn that can be described as: (a) the presence of initial conditions that tend to canalize learning in the right directions; (b) the tendency to behave in a way that enhances the perceived differences between different environments and determines input stimuli that facilitate the learning of adaptive changes, and (c) the ability to reach desirable stable states. http://kant.irmkant.rm.cnr.it/public.html or ftp-server: kant.irmkant.rm.cnr.it (150.146.7.5) ftp-file : /pub/econets/nolfi.changing.ps.Z for the homepage of our research group with most of our publications available online and pointers to ALIFE resources see: http://kant.irmkant.rm.cnr.it/gral.html ---------------------------------------------------------------------------- Stefano Nolfi Institute of Psychology National Research Council e-mail: stefano at kant.irmkant.rm.cnr.it From dhw at santafe.edu Mon Nov 20 13:54:43 1995 From: dhw at santafe.edu (David Wolpert) Date: Mon, 20 Nov 95 11:54:43 MST Subject: Some more on NFL Message-ID: <9511201854.AA11261@sfi.santafe.edu> Barak Pearlmutter writes: >>> not having low Kolmogorov complexity is equivalent to being random... (the) description of anything that does not have low Kolmogorov complexity is "nothing but noise." >>> I don't necessarily disagree with such sentiments, but one should definitely be careful about them; there are *many* definitions of "random". (Seth Lloyd has counted about 30 versions of its flip side, "amount of information".) High Kolmogorov complexity is only one of them. To illustrate just one of the possible objections to measuring randomness with Kolmogorov complexity: Would you say that a macroscopic gas with a specified temperature is "random"? To describe it exactly takes a huge Kolmogorov complexity. And certainly in many regards its position in phase space is "nothing but noise". (Indeed, in a formal sense, its position is a random sample of the Boltzmann distribution.) Yet Physicists can (and do) make extraordinarilly accurate predictions about such creatures with ease. Another important (and related) point is that encoding constant that gets buried in the definition of Kolmogorov complexity - in practive it can be very important. To put it another way, one person's "random" is another person's "highly regular"; this is precisely why the basis you use in supervised learning matters so much. >>> as scientists, we have had some success by positing the existence of non-random structure in the real world. So it seems to me that there is at least some reason to believe that the functions we optimize in practice are not completely random, in this Kolmogorov complexity sense. >>> Oh, most definitely. To give one simple example: Cross-validation works quite well in practice. However either 1) for *any* fixed target (i.e., for any prior over targets), averaged over all sets of generalizers you're choosing among, cross-validation works no better than anti-cross-validation (choose the generalizer in the set at hand having *largest* cross-validation error); and 2) for a fixed set of generalizers, averaged over all targets, it works no better than anti-cross-validation. So for cross-validation to work requires a very subtle inter-relationship between the prior over targets and the set of generalizers you're choosing among. In particular, cross-validation cannot be given a Bayesian justification without regard to the set of generalizers. Nonetheless, I (and every other evenly marginally rational statistician) have used cross-validation in the past, and will do so again in the future. From marshall at cs.unc.edu Mon Nov 20 12:58:45 1995 From: marshall at cs.unc.edu (Jonathan Marshall) Date: Mon, 20 Nov 1995 13:58:45 -0400 Subject: CFP: Biologically Inspired Autonomous Systems Message-ID: <199511201758.NAA20951@marshall.cs.unc.edu> ---------------------------------------------------------------------------- Biologically Inspired Autonomous Systems: Computation, Cognition, and Action March 4-5, 1996 Washington Duke Hotel (Duke University) Durham, North Carolina Co-Sponsored by the Duke University Departments of Electrical and Computer Engineering, Neurobiology, Biomedical Engineering, and Experimental Psychology The dramatic evolution of computer technology has caused a return to the biological paradigms which inspired many of the early pioneers of information science such as John von Neumann, Stephen Kleene and Marvin Minsky. Similarly, many fields of the life and human sciences have been influenced by paradigms initiated in systems theory, computation and control Engineering. The purpose of this workshop is to pursue this fruitful interaction of engineering and the exact sciences, with the life and human sciences, by investigating the processes which can provide systems, both artificial and natural, with autonomous and adaptive behavior. Topics of interest include Autonomous behavior of biophysically and cognitively inspired models Autonomous agents and mobile systems Collective behaviour by semi-autonomous agents Self repair and regeneration in computational and artificial structures Autonomous image understanding Brain imaging and Functional MRI Keynote Speakers: Stephen Grossberg (Boston University), Daniel Mange (EPFL), Jean-Arcady Meyer (ENS, Paris), Heinz Muehlenbein (GMD, Bonn), John Taylor (University College, London). Speakers will include: Paul Bourgine (Ecole Polytechnique), Bernadette Dorizzi (INT, Evry), Warren Hall (Duke), Ivan Havel (Center for Theoretical Studies and Prague University), Petr Lansky (Center for Theoretical Studies and Prague University), Miguel Nicollelis (Duke), Richard Palmer (Duke), David Rubin (Duke) , Nestor Schmajuk (Duke), John Staddon (Duke), John Taylor (University College, London), Ed Ueberbacher (Oak Ridge National Laboratory), Paul Werbos (NSF). Paper submissions, in the form of four page extended abstracts, are solicited in areas of relevance to this workshop. They should be sent before January 15, 1996 to one of the workshop Co-Chairs. The Workshop Proceedings will be published in book form with full papers. Workshop Co-Chairs: Erol Gelenbe Nestor Schmajuk Department of Electrical and Department of Experimental Psychology Computer Engineering Duke University Duke University Durham, NC 27708, USA Durham, NC 27708-0291, USA nestor at acpub.duke.edu erol at ee.duke.edu ---------------------------------------------------------------------------- From niranjan at eng.cam.ac.uk Mon Nov 20 23:56:44 1995 From: niranjan at eng.cam.ac.uk (niranjan@eng.cam.ac.uk) Date: Tue, 21 Nov 95 04:56:44 GMT Subject: JOB JOB JOB Message-ID: <9511210456.9837@baby.eng.cam.ac.uk> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Research Assistant Position for One Year A Research Assistant position is available in Cambridge to investigate the use of: Neural Networks in the Prediction of Risk in Pregnancy Euro-PUNCH is a collaborative Project funded by the Human Capital and Mobility Programme of the Commission of the European Communities. Thus, the post is available only to a citizen of a European Union Member State (but not British), who wishes to come to work in the United Kingdom. From Olivier.Michel at alto.unice.fr Tue Nov 21 10:02:52 1995 From: Olivier.Michel at alto.unice.fr (Olivier MICHEL) Date: Tue, 21 Nov 1995 16:02:52 +0100 Subject: Announcement: Khepera Simulator 1.0 Message-ID: <199511211502.QAA16789@alto.unice.fr> * ANNOUNCEMENT OF NEW PUBLIC DOMAIN SOFTWARE PACKAGE * ------------------------------------------------------------- - Khepera Simulator version 1.0 - ------------------------------------------------------------- Khepera Simulator is a public domain software package written by Olivier MICHEL. It allows to write your own controller for the mobile robot Khepera using C or C++ languages, to test them in a simulated environment and features a nice colorful X11 graphical interface. Moreover, if you own a Khepera robot, it can drive the real robot using the same control algorithm. It is mainly destinated to researchers studying autonomous agents. o Requirements: UNIX system, X11 library. o User Manual and examples of controllers included. o This software is free of charge for research and teaching. o Commercial use is forbidden. o Khepera is a mini mobile robot developped at EPFL by Francesco Mondada, Edo Franzi and Andre Guignard (K-Team). o You can download it from the following web site: http://wwwi3s.unice.fr/~om/khep-sim.html Olivier MICHEL om at alto.unice.fr http://wwwi3s.unice.fr/~om/ From tds at ai.mit.edu Tue Nov 21 12:10:47 1995 From: tds at ai.mit.edu (Terence D. Sanger) Date: Tue, 21 Nov 95 12:10:47 EST Subject: NIPS workshop announcement Message-ID: <9511211710.AA08251@dentate.ai.mit.edu> NIPS*95 Post-Conference Workshop "Vertebrate Neurophysiology and Neural Networks: Can the teacher learn from the student?" Saturday December 2, 7:30-9:30AM, 4:30-6:30PM Organizer: Terence Sanger, MIT. SUMMARY Results from neurophysiological investigations continue to guide the development of artificial neural network models that have been shown to have wide applicability in solving difficult computational problems. This workshop addresses the question of whether artificial neural network models can be applied to understanding neurophysiological results and guiding further experimental investigations. Recent work on close modelling of vertebrate neurophysiology will be presented, so as to give a survey of some of the results in this field. We will concentrate on examples for which artificial neural network models have been constructed to mimic the structure as well as the function of their biological counterparts. Clearly, this can be done at many different levels of abstraction. The goal is to discuss models that have explanatory and predictive power for neurophysiology. The following questions will serve as general discussion topics: 1. Do artificial neural network models have any relationship to ``real'' Neurophysiology? 2. Have any such models been used to guide new biological research? 3. Is Neurophysiology really useful for designing artificial networks, or does it just provide a vague ``inspiration''? 4. How faithfully do models need to address ultrastructural or membrane properties of neurons and neural circuits in order to generate realistic predictions of function? 5. Are there any artificial network models that have applicability across different regions of the central nervous system devoted to varied sensory and motor modalities? 6. To what extent do theoretical models address more than one of David Marr's levels of algorithmic abstraction (general approach, specific algorithm, and hardware implementation)? Selected examples of Neural Network models for Neurophysiological results will be presented, and active audience participation and discussion will be encouraged. SCHEDULE Saturday, December 2 7:30 - T. Sanger: Introduction and Overview 8:00 - T. Sejnowski: "Bee Foraging in Uncertain Environments using Predictive Hebbian Learning" 8:30 - A. Pouget: "Spatial Representations in the Parietal Cortex may use Basis Functions" 9:00 - Discussion --- Break --- 4:30 - S. Giszter: "Spinal Primitives and their Dynamics in Vertebrate Limb Control: A Biological Perspective" 5:00 - G. Goodhill: "Modelling the Development of Primary Visual Cortex: Determinants of Ocular Dominance Column Periodicity" 5:30 - Discussion From prechelt at ira.uka.de Tue Nov 21 13:34:24 1995 From: prechelt at ira.uka.de (Lutz Prechelt) Date: Tue, 21 Nov 1995 19:34:24 +0100 Subject: NIPS Workshop: Benchmarking of NN learning algorithms Message-ID: <"iraun1.ira.532:21.11.95.18.34.54"@ira.uka.de> X-URL: http://wwwipd.ira.uka.de/~prechelt/NIPS_bench.html NIPS Workshop: Benchmarking of NN learning algorithms ******************************************************** Abstract: Proper benchmarking of neural network learning architectures is a prerequisite for orderly progress in this field. In many published papers deficiencies can be observed in the benchmarking that is performed. The workshop addresses the status quo of benchmarking, common errors and how to avoid them, currently existing benchmark collections, and, most prominently, a new benchmarking facility including a results database. The workshop goal is to improve benchmarking practices and to improve the comparability of benchmark tests. Workshop Chairs: o Thomas G. Dietterich , o Geoffrey Hinton , o Wolfgang Maass , o Lutz Prechelt [communicating chair] o Terry Sejnowski From caruana+ at cs.cmu.edu Tue Nov 21 14:20:09 1995 From: caruana+ at cs.cmu.edu (Rich Caruana) Date: Tue, 21 Nov 95 14:20:09 EST Subject: NIPS*95 Workshop on Transfer in Inductive Systems Message-ID: <9072.816981609@GS79.SP.CS.CMU.EDU> Post-NIPS*95 Workshop, December 1-2, 1995, Vail, Colorado TITLE: "Learning to Learn: Knowledge Consolidation and Transfer in Inductive Systems" ORGANIZERS: Rich Caruana (co-chair), Danny Silver (co-chair), Jon Baxter, Tom Mitchell, Lori Pratt, Sebastian Thrun INVITED TALKS BY: Leo Breiman (Berkeley) Tom Mitchell (CMU) Tomaso Poggio (MIT) Noel Sharkey (Sheffield) Jude Shavlik (Wisconsin) WEB PAGE: http://www.cs.cmu.edu/afs/cs/usr/caruana/pub/transfer.html DESCRIPTION: Because the power of tabula rasa learning is limited, interest is increasing in methods that capitalize on previously acquired domain knowledge. Examples of these methods include: o using symbolic domain theories to bias connectionist networks o using extra outputs on a connectionist network to bias the hidden layer representation towards more predictive features o using unsupervised learning on a large corpus of unlabelled data to learn features useful for subsequent supervised learning on a smaller labelled corpus o using models previously learned for other problems as a bias when learning new, but related, problems There are many approaches: hints, knowledge-based artificial neural nets (KBANN), explanation-based neural nets (EBNN), multitask learning (MTL), knowledge consolidation, ... What they all have in common is the attempt to transfer knowledge from other sources to benefit the current inductive task. The goal of this workshop is to provide an opportunity for researchers and practitioners to discuss problems and progress in knowledge transfer in learning. We hope to identify research directions, debate theories and approaches, discover unifying principles, and begin to start answering questions like: o when will transfer help -- or hinder? o what should be transferred and how? o what are the benefits of transfer? o in what domains is transfer most useful? o is there evidence for transfer in nature? From avrama at wo.erim.org Wed Nov 8 22:03:20 1995 From: avrama at wo.erim.org (Avrama Blackwell) Date: Tue, 21 Nov 1995 15:03:20 +30000 Subject: position open Message-ID: PRE- OR POST-DOCTORAL FELLOWSHIP IN NEUROCOMPUTING Applications are invited for the position of Pre- or Post-doctoral Fellow. The Fellow will be an integral member of a team collaborating with NIH/NINDS in the development of advanced models of associative learning and visual information processing. Position requires interest and experience in computational neurobiology or development of neural network models as well as good 'C' or 'C++' programming skills. For a review of recent activities of the group see: Alkon et al. In Neural Networks 7: 1005 (1994). The appointment, for one year with possibility of renewal, will be a joint appointment at ERIM (Environmental Research Institute of Michigan, Washington Office) and George Mason University. If interested, either contact Tom Vogl at the NIPS conference in Denver (Dr. Vogl will NOT be at the Workshops) or send statement of interest and CV to tvogl at erim.org. From omlinc at research.nj.nec.com Tue Nov 21 15:53:59 1995 From: omlinc at research.nj.nec.com (Christian Omlin) Date: Tue, 21 Nov 1995 15:53:59 -0500 Subject: paper available Message-ID: <199511212053.PAA12137@arosa> The following paper is available on the website http://www.neci.nj.nec.com/homepages/omlin/omlin.html The paper gives an overview of our work and contains an extensive bibliography on the representation of discrete dynamical systems in recurrent neural networks. -Christian =================================================================== Learning, Representation, and Synthesis of Discrete Dynamical Systems in Continuous Recurrent Neural Networks (*) C. Lee Giles (a,b) and Christian W. Omlin (a) (a) NEC Research Institute 4 Independence Way Princeton, NJ 08540 (b) Institute for Advanced Computer Studies University of Maryland College Park, MD 20742 ABSTRACT This paper gives an overview on learning and representation of discrete-time, discrete-space dynamical systems in discrete-time, continuous-space recurrent neural networks. We limit our discussion to dynamical systems (recurrent neural networks) which can be represented as finite-state machines (e.g. discrete event systems ). In particular, we discuss how a symbolic representation of the learned states and dynamics can be extracted from trained neural networks, and how (partially) known deterministic finite-state automata (DFAs) can be encoded in recurrent networks. While the DFAs that can be learned exactly with recurrent neural networks are generally small (on the order of 20 states), there exist subclasses of DFAs with on the order of 1000 states that can be learned by small recurrent networks. However, recent work in natural language processing implies that recurrent networks can possibly learn larger state systems. (*) Appeared in Proceedings of the IEEE Workshop on Architectures for Semiotic Modeling and Situation Analysis in Large Complex Systems, Monterey, CA, August 27-29, 1995. Copyright IEEE Press. From heckerma at microsoft.com Tue Nov 21 14:55:14 1995 From: heckerma at microsoft.com (David Heckerman) Date: Tue, 21 Nov 95 14:55:14 TZ Subject: NIPS95 workshop Message-ID: <199511212254.OAA29657@imail1.microsoft.com> **** TENTATIVE SCHEDULE **** NIPS*95 Workshop Learning in Bayesian Networks and Other Graphical Models Friday and Saturday, December 1-2, 1995 Marriott Vail Mountain Resort, Colorado http://www.cs.cmu.edu/afs/cs/project/cnbc/nips/NIPS.html (conference) http://www.ai.mit.edu/people/jordan/workshop.html (workshop) Topic and Purpose of the Workshop: A Bayesian network is a directed graphical representation of probabilistic relationships that people find easy to understand and use, often because the relationships have a causal interpretation. A network for a given domain defines a joint probability distribution for that domain, and algorithms exist for efficiently manipulating the joint distribution to determine probability distributions of interest. Over the last decade, the Bayesian network has become a popular representation for encoding uncertain expert knowledge in expert systems. More recently, researchers have developed methods for learning Bayesian networks from data. These approaches will be the focus of this workshop. Issues to be discussed include (1) the opposing roles of prediction and explanation; (2) search, model selection, and capacity control; (3) representation issues, including extensions of the Bayesian network (e.g., chain graphs), the role of ``hidden'' or ``latent'' variables in learning, the modeling of temporal systems, and the assessment of priors; (4) optimization and approximation methods, including gradient-based methods, EM algorithms, stochastic sampling, and the mean field algorithms. In addition, we plan to discuss known relationships among Bayesian networks, Markov random fields, Boltzmann machines, loglinear models for contingency tables, Hidden Markov models, decision trees, and feedforward neural networks, as well as to uncover previously unknown ties. Tentative Schedule: Friday, Dec 1 ------------- 7:30am - 8:50am Tutorial on Graphical Models Ross Shachter, Stanford University Bruce D'Ambrosio, Oregon State University Michael Jordan, MIT 8:50am - 9:30am Decomposable graphical models and their use in learning algorithms Steffen Lauritzen, Aalborg University Decomposable, or triangulated, graphical models occur for example as basic computational structures in probabilistic expert systems. I will first give a brief description of properties of decomposable graphical models and then present some unfinished ideas about their possible use in automatic learning procedures. 9:30am - 9:40am break 9:40am - 10:20am Likelihoods and Priors for Learning Bayesian Networks David Heckerman, Microsoft Dan Geiger, Technion I will discuss simple methods for constructing likelihoods and parameter priors for learning about the parameters and structure of a Bayesian network. In particular, I will introduce several assumptions that permit the construction of likelihoods and parameter priors for a large number of Bayesian-network structures from a small set of assessments. Two notable assumptions are parameter independence, which says that the parameters associated with each variable in a structure are independent, and likelihood equivalence, which (roughly speaking) says that data should not help to discriminate structures that represent the same assertions of conditional independence. In addition to explicating methods for likelihood and prior construction, I will show how the assumptions lead to characterizations of well-known prior distributions for the parameters of multivariate distributions. For example, when the joint likelihood is an unrestricted discrete distribution, parameter independence and likelihood equivalence imply that the parameter prior must be a Dirichlet distribution. 10:20am - 11:00am Bayesian model averaging for Markov equivalence classes of acyclic digraphs David Madigan, Michael D. Perlman, and Chris T. Volinsky, University of Washington, Seattle Acyclic digraphs (ADGs) are widely used to describe dependencies among variables in multivariate distributions. There may, however, be many ADGs that determine the same dependence (= Markov) model. Thus, the family of all ADGs with a given set of vertices is naturally partitioned into Markov-equivalence classes, each class being associated with a unique statistical model. Statistical procedures, such as model selection or model averaging, that fail to take into account these equivalence classes, may incur substantial computational or other inefficiencies. Recent results have shown that each Markov-equivalence class is uniquely determined by a single chain graph, the essential graph, that is itself Markov-equivalent simultaneously to all ADGs in the equivalence class. Here we propose two stochastic Bayesian model averaging and selection algorithms for essential graphs and apply them to the analysis of a number of discrete- variable data sets. 11:00am - 11:40am discussion ----mid-day break---- 4:30pm - 5:10pm Automated Causal Inference Peter Spirtes, Carnegie Mellon University Directed acyclic graphs can be used to represent both families of probability distributions and causal relationships. We introduce two axioms (Markov and Faithfulness) that are widely but implicitly assumed by statisticians and that relate causal structures with families of probability distributions. We then use these axioms to develop algorithms that infer some features of causal graphs given a probability distribution and optional background knowledge as input. The algorithms are correct in the large sample limit, even when latent variables and selection bias may be present. In the worst case, the algorithms are exponential, but in many cases they has been able to handle up to 100 variables. We will also present Monte Carlo simulation results on various sample sizes. 5:10pm - 5:50pm HELMHOLTZ MACHINES Geoffrey E. Hinton, Peter Dayan, Brendan Frey, Radford Neal University of Toronto and MIT For hierarchical generative models that use distributed representations in their hidden variables, there are exponentially many ways in which the model can produce each data point. It is therefore intractable to compute the posterior distribution over the hidden distributed representations given a datapoint and so there is no obvious way to use EM or gradient methods for fitting the model to data. A Helmholtz machine consists of a generative model that uses distributed representations and a recognition model that computes an approximation to the posterior distribution over representations. The machine is trained to minimize a Helmholtz free energy which is equal to the negative log probability of the data if the recognition model computes the correct posterior distribution. If the recognition model computes a more tractable, but incorrect distribution, the Helmholtz free energy is an upper bound on the negative log probability of the data, so it acts as a tractable and useful Lyapunov function for learning a good generative model. It also encourages generative models that give rise to nice simple posterior distributions, which makes perception a lot easier. Several different methods have been developed for minimizing the Helmholtz free energy. I will focus on the "wake-sleep" algorithm, which is easy to implement with neurons, and give some examples of it learning probability density functions in high dimensional spaces. 5:50pm - 6:30pm Bounding Log Likelihoods in Sigmoid Belief Networks Lawrence K. Saul, Tommi Jaakkola, and Michael I. Jordan, MIT Sigmoid belief nets define graphical models with useful probabilistic semantics. We show how to calculate a lower bound on the log likelihood of any partial instantiation of a sigmoid belief net. The bound can be used as a basis for inference and learning provided it is sufficiently tight; in practice we have found this often to be the case. The bound is computed by approximating the true posterior distribution over uninstantiated nodes, $P$, by a more tractable distribution, $Q$. Parameterized forms for $Q$ include factorial distributions, mixture models, and hierarchical distributions that exploit the presence of tractable substructures in the original belief net. Saturday, Dec 2 --------------- 7:30am - 8:10am A Method for Learning the Structure of a Neural Network from Data Gregory F. Cooper and Sankaran Rajagopalan, University of Pittsburgh A neural network can be viewed as consisting of a set of arcs and a parameterization of those arcs. Most neural network learning has focused on parameterizing a user-specified set of arcs. Relatively less work has addressed automatically learning from data which arcs to include in the network (i.e., the neural network structure). We will present a method for learning neural network structures from data (call it LNNS). The method takes as input a database and a set of priors over possible neural network structures, and it outputs the neural network structure that is most probable as found by a heuristic search procedure. We will describe the close relationship between the LNNS method and current Bayesian methods for learning Bayesian belief networks. We also will show how we can apply the LNNS method to learn hidden nodes in Bayesian belief networks. 8:10am - 8:50am Local learning in probabilistic networks with hidden variables Stuart Russell, John Binder, Daphne Koller, Keiji Kanazawa University of California, Berkeley We show that general probabilistic (Bayesian) networks with fixed structure containing hidden variables can be learned automatically from data using a gradient-descent mechanism similar to that used in neural networks. The gradient can be computed locally from information available as a direct by-product of the normal inference process in the network. Because probabilistic networks provide explicit representations of causal structure, human experts can easily contribute prior knowledge to the training process, thereby significantly improving the sample complexity. The method can be extended to networks with intensionally represented distributions, including networks with continuous variables and dynamic probabilistic networks (DPNs). Because DPNs provide a decomposed representation of state, they may have some advantages over HMMs as a way of learning certain types of stochastic temporal processes with hidden state. 8:50am - 9:30am Asymptotic Bayes Factors for Directed Networks with Hidden Variables Dan Geiger, Technion David Heckerman, Microsoft Chris Meek, Carnegie Mellon University 9:30am - 9:40am break 9:40am - 10:20am Bayesian Estimation of Gaussian Bayes Networks Richard Scheines, Carnegie Mellon University The Gibbs sampler can be used to draw a sample from the posterior distribution over the parameters in a linear causal model ( Gaussian network, structural equation model, or LISREL model). I show how this can be done, and provide several examples that demonstrate its utility. These include: estimating under-identified models and making correct small sample inferences about the parameters when the likelihood surface is non-normal, contrary to the assumptions of the asymptotic theory that all current techniques rely on. In fact the likelihood surface for structural equation models (SEMs) with latent variables is often multi-modal. I give an example of such a case, and show how the Gibbs sampler is still informative and useful in this case in contrast to standard SEM software like LISREL. 10:20am - 11:00am Learning Stochastic Grammars Stephen M. Omohundro Bayesian networks represent the distribution of a fixed set of random variables using conditional independence information. Many important application domains (eg. speech, vision, planning, etc.) have state spaces that don't naturally decompose into a fixed set of random variables. In this talk, I'll present stochastic grammars as a tractable class of probabilistic models over this kind of domain. I'll describe an algorithm by Stolcke and myself for learning Hidden Markov Models and stochastic regular grammars from training strings which is closely related to algorithms for learning Bayesian networks. I'll discuss connections between stochastic grammars and graphical probability models and argue for the need for richer structures which encompass certain properties of both classes of model. 11:00am - 11:20am Variable selection using the theory of Bayesian networks Christopher Meek, Carnegie Mellon University This talk will describe two approaches to variable selection based upon the theory of Bayesian networks. In a study about the prediction of mortality in hospital patients with pneumonia these two methods were used to select a set of variables upon which predictive models were developed. In addition several neural network methods were used to develop predictive models from the same large database of cases. The mortality models developed in this study are distinguished more by the number of variables and parameters that they contain than by their error rates. I will offer an explanation of why the two methods described lead to small sets of variables and models with fewer parameters. In addition the variable sets selected by the Bayesian network approaches are amenable to future implementation in a paper-based form and, for several of the models, are strikingly similar to variable sets hand selected by physicians. 11:20am - 11:40am Methods for Learning Hidden Variables Joel Martin Hidden variables can be learned in many ways. Which method should be used? Some have better theoretical justifications, some seem to have better pragmatic justifications. I will describe a simple class of probabilistic models and will compare several methods for learn hidden variables from data. The methods compared are EM, gradient descent, simulated annealing, genetic search, and a variety of incremental techniques. I will discuss the results by considering particular applications. ----mid-day break---- 4:30pm - 5:10pm Brains, Nets, Feedback and Time Series Clark Glymour, Thomas Richardson and Peter Spirtes, Carnegie Mellon Neural networks have been used extensively to model the differences between normal cognitive behavior and the behavior of brain damaged subjects. A network is trained to simulate normal behavior, lesioned, and then simulates, or under re-training simulates, brain-damaged behavior. A common objection to this explanatory strategy (see for example comments on Martha Farah's recent contribution in Behavioral and Brain Sciences) is that it can "explain anything." The complaint alleges that for any mathematically possible pairing of normal and brain-damaged behavior, there exists a neural net that simulates the normal and when lesioned simulates the abnormal. Is that so? We can model the normal and abnormal behaviors as probability distributions on a set of nodes, and the network as a set of simultaneous (generally non-recursive) equations with independent noises. The network and joint probability distribution then describe a cyclic graph and associated probability distribution. What is the connection between the probability distribution and the graph topology? It is easy to show that the (local) Markov condition fails for linear networks of this sort. Spirtes (and independently J. Koster, in a forthcoming paper in Annals of Statistics) has shown that the conditional independencies implied by a linear cyclic network are characterized by d-separation (in either Pearl's or Lauritzen's versions). Spirtes has also shown that d-separation fails for non-linear cyclic networks with independent errors, but has given another characterization for the non-linear case. These results can be directly applied to the methodological disputes about brains and neural net models. Assuming faithfulness, it follows that a lesioned neural net represented as a cyclic Bayes net, whether linear or non-linear, must preserve the conditional independence relations in the original network. Hence not every pairing of normal and abnormal behavior is possible according to the neural net hypothesis as formulated. This connection between Bayes nets and work on neural models suggests that we might look for other applications of Bayesian networks in cognitive neuropsychology. We might hope to see applications of discovery methods for Bayes nets to multiple single cell recordings, or to functional MRI data. Such appplications would be aided by discovery methods for cyclic Bayes nets as models of recurrent neural nets. Several important theoretical steps have been taken towards that goal by Richardson, who has found a polynomial time decision procedure for the equivlaence of linear cyclic graphs, and a polynomial (in sparse graphs) time, asymptotically correct and complete procedure for discovering equivalence classes of linear cyclic graphs (without latent variables). Research is under way on search procedures that parallel the FCI procedure of Spirtes Glymour and Scheines (1993) in allowing latent variables. The questions of equivalence and discovery for non-linear systems are relatively untouched. It may be that the proper way to model a recurrent neural net is not by a cyclic graphical model, but by a time series. That suggestion raises the question of the connections between time-series and cyclic graphical models, a matter under investigation by Richardson. Results in this area would have implications for econometrics as well as for neuropsychology, since the econometric tradition has treated feedback systems in both ways, by simultaneous linear equations and by time series, without fully characterizing the relations between the representations. While there are situations in which equilibrium models, such as cyclic graphical models appear applicable, these models, since they are not dynamic, make no predictions about the dynamic behavior of a system (time series) if it is pushed out of equilibrium. 5:10pm - 5:50pm Compiling Probabilistic Networks and Some Questions this Poses Wray Buntine Probabilistic networks (or similar) provide a high-level language that can be used as the input to a compiler for generating a learning or inference algorithm. Example compilers are BUGS (inputs a Bayes net with plates) by Gilks, Spiegelhalter, et al., and MultiClass (inputs a dataflow graph) by Roy. This talk will cover three parts: (1) an outline of the arguments for such compilers for probabilistic networks, (2) an introduction to some compilation techniques, and (3) the presentation of some theoretical challenges that compilation poses. High-level language compilers are usually justified as a rapid prototyping tool. In learning, rapid prototyping arises for the following reasons: good priors for complex networks are not obvious and experimentation can be required to understand them; several algorithms may suggest themselves and experimentation is required for comparative evaluation. These and other justifications will be described in the context of some current research on learning probabilistic networks, and past research on learning classification trees and feed-forward neural networks. Techniques for compilation include the data flow graph, automatic differentiation, Monte Carlo Markov Chain samplers of various kinds, and the generation of C code for certain exact inference tasks. With this background, I will then pose a number of important research questions to the audience. 5:50pm - 6:30pm discussion Organizers: Wray Buntine Greg Cooper Dan Geiger Clark Glymour David Heckerman Geoffry Hinton Mike Jordan Steffen Lauritzen David Madigan Radford Neal Steve Omohundro Judea Pearl Stuart Russell Richard Scheines Peter Spirtes From juergen at idsia.ch Wed Nov 22 04:32:17 1995 From: juergen at idsia.ch (Juergen Schmidhuber) Date: Wed, 22 Nov 95 10:32:17 +0100 Subject: compressibility, Kolmogorov, learning Message-ID: <9511220932.AA22891@fava.idsia.ch> In response to Barak's and David's recent messages: David writes: >>> To illustrate just one of the possible objections to measuring randomness with Kolmogorov complexity: Would you say that a macroscopic gas with a specified temperature is "random"? To describe it exactly takes a huge Kolmogorov complexity. And certainly in many regards its position in phase space is "nothing but noise". (Indeed, in a formal sense, its position is a random sample of the Boltzmann distribution.) Yet Physicists can (and do) make extraordinarilly accurate predictions about such creatures with ease. <<< 1. Is real gas random in the Kolmogorov sense? At least the idealized microscopic gas models we are studying are not random. For simplicity, let us consider a typical discrete time gas model. Each system state can be computed by a short algorithm, given the previous state. This allows for enormous compressibility of the system history, even if the initial state had high complexity (and even more so if the initial state was simple). Even if we assume that the deterministic model is corrupted by random processes, we won't end up with a system history with maximal Kolmogorov complexity: for instance, using standard compression techiques, we can provide short codes for likely next states and long codes for unlikely next states. The only requirement is that the random processes are not *completely* random. But in the real world they are not, as can be deduced from macroscopic gas properties (considering only abstract properties of the state, such as temperature and pressure) --- as David indicated, there are simple algorithms for predicting next macroscopic states from previous macroscopic states. In case of true randomness, this would not be the case. 2. Only where there is compressibility, there is room for non-trivial learning and generalization. Unfortunately, almost all possible histories of possible universes are random and incompressible. There is no miraculous universal learning algorithm for arbitrary universes (that's more or less the realm of NFL). As has been observed repeatedly, however, our own universe appears to be one of the relatively few (but still infinitely many) compressible ones (every electron behaves the same way, etc.). In fact, much of the previous work on machine learning can be thought of exploiting compressibility: ``chunking'', for instance, exploits the possibility of re-using subprograms. Methods for finding factorial (statistically non-redundant) codes of image data exploit the enormous redundancy and compressibility of visual inputs. Similarly for ``learning by analogy'' etc. In the context of PAC learning, Ming Li and Paul Vitanyi address related issues in an interesting paper from 1989: A theory of Learning Simple Concepts Under Simple Distributions and Average Case Complexity for the Universal Distribution, Proc. 30th American IEEE Symposium on Foundations of Computer Science, pages 34-39. 3. An intriguing possibility is: there may be something like a universal learning algorithm for compressible, low-complexity universes. I am the first to admit, however, that neither the concept of Kolmogorov complexity by itself nor the universal Solomonoff-Levin distribution provide all the necessary ingredients. Clearly, a hypothetical universal learning algorithm would have to take into account the fact that computational resources are limited. This is driving much of the current work at our lab. Juergen Schmidhuber IDSIA From tds at ai.mit.edu Wed Nov 22 09:43:42 1995 From: tds at ai.mit.edu (Terence D. Sanger) Date: Wed, 22 Nov 95 09:43:42 EST Subject: off on a tangent... Message-ID: <9511221443.AA08557@dentate.ai.mit.edu> David Wolpert writes: > To illustrate just one of the possible objections to measuring > randomness with Kolmogorov complexity: Would you say that a > macroscopic gas with a specified temperature is "random"? To describe > it exactly takes a huge Kolmogorov complexity. And certainly in many > regards its position in phase space is "nothing but noise". (Indeed, > in a formal sense, its position is a random sample of the Boltzmann > distribution.) Yet Physicists can (and do) make extraordinarilly > accurate predictions about such creatures with ease. In thinking about this, it seems that David is right: the gas is, in some important sense, highly structured. In particular, its *statistics* are stationary no matter how they are sampled. This means that a temperature sample from any part of the gas will predict temperatures in other parts of the gas, according to the law of large numbers. Consider a different statistical model: Choose a random number to be the mean of a distribution on a finite set of random variables. Divide the set in half, choose a new random number and add it to the left half's mean and subtract it from the right half's mean. Divide the half-sets in half, choose two new random numbers, and continue to split each half by adding and subtracting random numbers until no more splits are possible. Now we have: 1) All random variables are independent and identically distributed (this is basically a random walk). 2) The mean of the distribution is equal to the first random number chosen (at each step, the mean does not change). 3) The mean of any "binary" region (half, quarter, eighth, etc.) is a poor predictor of the means of neighboring regions. 4) Sample means do not converge uniformly to the ensemble mean, unless samples are chosen randomly across region boundaries. In some sense, this is a very highly structured field of random variables. Yet prediction is much harder than for the random gas. (In case anyone is interested, this distribution arises as a model for genetic diseases of mitochondria. If a cell has N mitochondria, M of which are defective, then at cell division it will pass a random number of normal and defective mitochondria to each daughter cell, where the total number of defective ones passed on is conserved and is equal to 2M. The daughter cells, in turn, will do the same thing. The problem is that a local biopsy to count the number of defective mitochondria will not predict biopsy results from other sites.) Terry Sanger tds at ai.mit.edu From kak at gate.ee.lsu.edu Wed Nov 22 11:55:45 1995 From: kak at gate.ee.lsu.edu (Subhash Kak) Date: Wed, 22 Nov 95 10:55:45 CST Subject: Paper Message-ID: <9511221655.AA22783@gate.ee.lsu.edu> The following paper ON GENERALIZATION BY NEURAL NETWORKS by Subhash C. Kak Abstract: We report new results on the corner classification approach to training feedforward neural networks. It is shown that a prescriptive learning procedure where the weights are simply read off based on the training data can provide adequate generalization. The paper also deals with the relations between the number of separable regions and the size of the training set for a binary data network. was recently presented at the Joint Conference on Information Science. You may ftp the paper at the following address: ftp://gate.ee.lsu.edu/pub/kak/gen.ps From geoff at salk.edu Wed Nov 22 13:18:18 1995 From: geoff at salk.edu (Geoff Goodhill) Date: Wed, 22 Nov 95 10:18:18 PST Subject: Topographic Mappings - Tech Report available Message-ID: <9511221818.AA23843@salk.edu> The following paper is available via ftp://salk.edu/pub/geoff/goodhill_finch_sejnowski_tech95.ps.Z or http://cnl.salk.edu/~geoff QUANTIFYING NEIGHBOURHOOD PRESERVATION IN TOPOGRAPHIC MAPPINGS Geoffrey J. Goodhill(1), Steven Finch(2) & Terrence J. Sejnowski(3) (1) The Salk Institute for Biological Studies 10010 North Torrey Pines Road, La Jolla, CA 92037, USA (2) Human Communication Research Centre University of Edinburgh, 2 Buccleuch Place Edinburgh EH8 9LW, GREAT BRITAIN (3) The Howard Hughes Medical Institute The Salk Institute for Biological Studies 10010 North Torrey Pines Road, La Jolla, CA 92037, USA & Department of Biology University of California San Diego, La Jolla, CA 92037, USA, Institute for Neural Computation Technical Report Series INC-9505, November 1995 ABSTRACT Mappings that preserve neighbourhood relationships are relevant in both practical and biological contexts. It is important to be clear about precisely what preserving neighbourhoods could mean. We give a definition of a ``perfectly neighbourhood preserving'' map, which we call a topographic homeomorphism, and prove that this has certain desirable properties. When a topographic homeomorphism does not exist (the usual case), many choices are available for quantifying the quality of a map. We introduce a particular measure, C, which has the form of a quadratic assignment problem. We also discuss other measures that have been proposed, some of which are related to C. A comparison of seven measures applied to the same simple mapping problem reveals interesting similarities and differences between the measures, and challenges common intuitions as to what constitutes a ``good'' map. 17 pages, uncompressed postscript = 154K From scott at cpl_mmag.nhrc.navy.mil Wed Nov 22 14:37:41 1995 From: scott at cpl_mmag.nhrc.navy.mil (Scott Makeig) Date: Wed, 22 Nov 95 11:37:41 -0800 Subject: Online-NIPS post-NIPS workshop: programme Message-ID: <9511221937.AA24212@cpl_mmag.nhrc.navy.mil> ****** PROGRAMME ****** NIPS*95 Workshop on ONLINE NEURAL INFORMATION PROCESSING SYSTEMS: Prospects for Neural Human-Machine Interfaces Date: Saturday, Dec. 2, 1995 Place: NIPS*95 Workshops, Vail, Colorado www: http://128.49.52.9/~www/nips.html Organizer: Scott Makeig (NHRC/UCSD) scott at salk.edu There is rapidly growing interest in the development of intelligent interfaces in which operator state information derived from psycho- physiological and/or video-based measures of the human operator is used directly to inform, interact with, or control computer-based systems. Adequate signal processing power is now available at reasonable cost to implement in near-real time a wide range of spectral, neural network, and dynamic systems algorithms for extracting information about psychological state or intent from multidimensional EEG signals, video images of the eyes and face, and other psychophysiological and/or behavioral data. This NIPS*95 conference workshop will give an opportunity for interested researchers from signal processing, neuroscience, neural networks, cognitive science, and computer design to discuss near- and medium-term prospects for, and obstacles to, practical neural human-systems interfaces (NHSI) technology for monitoring cognitive state and for using operator state information to give operator feedback, control adaptive automation or perform brain-actuated control. Aspects of cognitive state that might be monitored using NHSI technology include alertness, perception, attention, workload, intention and emotion: Programme (Saturday, Dec. 2, Vail Marriott): 7:30 Scott Makeig (NHRC/UCSD) NHSI Overview Sandy Pentland (MIT Media Lab) Video-based human-computer interaction Alan Gevins (EEG Systems Labs) EEG-based cognitive monitoring Discussion 8:30 Babak A. Taheri (SRI International) Active EEG electrode technology Tzyy-Ping Jung (Salk Institute) EEG-based alertness monitoring Magnus Stensmo (Salk Institute) Monitoring alertness via eye closures General Discussion 9:30 [free time] 12:30 Lunch (optional) 1:30 [free time] 4:30 Georg Dorffner (ANNDEE project group, Austria) Brain-actuated control Grant McMillan (Wright-Patterson Air Force Base ) Brain-actuated control Andrew Junker (CyberLink) EMG/EEG-actuated control Discussion 5:30 Curtis Padgett (UCSD) Video-based emotion monitoring Jose Principe (University of Florida) EEG-based communication Charles W. Anderson (Colorado State University) Mental task monitoring General Discussion The workshop will review ongoing progress and challenges in computational and technology areas, and discuss prospects for short- and medium-term implementations. Abstracts are available at: http://128.49.52.9/~www/nips.html NIPS*95 conference and post-conference workshop information: http://www.cs.cmu.edu/Web/Groups/NIPS/nips95.html or via ftp/email from: psyche.mit.edu in /pub/NIPS95 nips95 at mines.colorado.edu From stavrosz at med.auth.gr Wed Nov 22 18:06:25 1995 From: stavrosz at med.auth.gr (Stavros Zanos) Date: Thu, 23 Nov 1995 01:06:25 +0200 (EET) Subject: Paper on LTP and Learning Algorithms In-Reply-To: Message-ID: (Neural Nets: Foundations to Applications) The following paper is now available to anyone who sends a request at the following adress (use the word "reqLTP3" at the subject field): stavrosz at antigoni.med.auth.gr ********* AU: Zanos Stavros, 3rd year medical student AT: University of Thessaloniki School of Medicine Thessaloniki, Greece TI: Quantal Analysis of Hippocampal Long-Term Synaptic Potentiation , and Application to the Design of Biologically Plausible Learning Algorithms for Artificial Neural Networks AB: Quantal analysis (QA) of synaptic function has been used to examine whether the expression of long-term potentiation (LTP) in central synapses is mediated by a pre- or postsynaptic mechanism. However, it can also be used as a physiological model of synaptic transmission and plasticity; use of physiological models in network simulations provides reasonably accurate approximates of various biological parameters in a computationally efficient manner. We describe a stochastic algorithm of synaptic transmission and plasticity based on QA data from CA1 hippocampus LTP experiments. We also describe the application of such an algorithm in a typical CA1-region simulation (a simple self-organizing competitive matrix), and discuss the possible benefits of using noisy network elements (in this case, "synapses"). We show that the fluctuations in postsynaptic responses under constant static synaptic weights introduced by such an algorithm increase the storing capacity and the ability of the network to orthogonalize input vectors. A decrease in the number of required iterations for every learned input vector is also reported. Finally we examine the issue of a hypothetical "computational equivalence" of different optimization techniques when applied to similar problems, often met in the literature, since our simulation studies suggest that even small differences in the learning algorithms used could provide the network with a kind of "preference" to specific patterns of performance. ********* The above paper will appear at the 2nd European Conference of Medical Students (May 96), and it has been edited using MS Word-7 (for Win95). Those who adressed a request will receive the paper through email as an attachment compressed file. Detailed mathematical formalizations used in the simulations are available upon request. We welcome questions and/or remarks. Zanos Stavros Aristotle University of Thessaloniki School of Life Sciences, Faculty of Medicine From ken at phy.ucsf.edu Wed Nov 22 20:44:34 1995 From: ken at phy.ucsf.edu (Ken Miller) Date: Wed, 22 Nov 1995 17:44:34 -0800 Subject: Postdoctoral and Predoctoral Positions in Theoretical Neurobiology Message-ID: <9511230144.AA04384@coltrane.ucsf.edu> POSTDOCTORAL AND PREDOCTORAL POSITIONS SLOAN CENTER FOR THEORETICAL NEUROBIOLOGY UNIVERSITY OF CALIFORNIA, SAN FRANCISCO INFORMATION ON THE UCSF SLOAN CENTER AND FACULTY AND THE POSTDOCTORAL AND PREDOCTORAL POSITIONS IS AVAILABLE THROUGH OUR WWW SITE: http://keck.ucsf.edu/sloan. E-mail inquiries should be sent to sloan-info at phy.ucsf.edu. Below is basic information on the program: The Sloan Center for Theoretical Neurobiology at UCSF solicits applications for pre- and post-doctoral fellowships, with the goal of bringing theoretical approaches to bear on neuroscience. Applicants should have a strong background and education in a theoretical discipline, such as physics, mathematics, or computer science, and commitment to a future research career in neuroscience. Prior biological or neuroscience training is not required. The Sloan Center will offer opportunities to combine theoretical and experimental approaches to understanding the operation of the intact brain. The research undertaken by the trainees may be theoretical, experimental, or a combination. The RESIDENT FACULTY of the Sloan Center and their research interests are: Allison Doupe: Development of song recognition and production in songbirds. Stephen Lisberger: Learning and memory in a simple motor reflex, the vestibulo-ocular reflex, and visual guidance of smooth pursuit eye movements by the cerebral cortex. Michael Merzenich: Experience-dependent plasticity underlying learning in the adult cerebral cortex and the neurological bases of learning disabilities in children. Kenneth Miller: Mechanisms of self-organization of the cerebral cortex; circuitry and computational mechanisms underlying cortical function; computational neuroscience. Roger Nicoll: Synaptic and cellular mechanisms of learning and memory in the hippocampus. Christoph Schreiner: Cortical mechanisms of perception of complex sounds such as speech in adults, and plasticity of speech recognition in children and adults. Michael Stryker: Mechanisms that guide development of the visual cortex. All of these resident faculty are members of UCSF's W.M. Keck Foundation Center for Integrative Neuroscience, a new center (opened January, 1994) for systems neuroscience that includes extensive shared research resources within a newly renovated space designed to promote interaction and collaboration. The unusually collaborative and interactive nature of the Keck Center will facilitate the training of theorists in a variety of approaches to systems neuroscience. In addition to the resident faculty, there are a series of VISITING FACULTY who are in residence at UCSF for times ranging from 1-8 weeks each year. These faculty, and their research interests, include: Laurence Abbott, Brandeis University: Neural coding, relations between firing rate models and biophysical models, self-organization at the cellular level William Bialek, NEC Research Institute: Physical limits to sensory signal processing, reliability and information capacity in neural coding; Sebastian Seung, ATT Bell Labs: models of collective computation in neural systems; David Sparks, University of Pennsylvania: understanding the superior colliculus as a "model cortex" that guides eye movements; Steven Zucker, McGill University: Neurally based models of vision, visual psychophysics, mathematical characterization of neuroanatomical complexity. PREDOCTORAL applicants seeking to BEGIN a Ph.D. program should apply directly to the UCSF Neuroscience Ph.D. program. Contact Patricia Arrandale, patricia at phy.ucsf.edu, to obtain application materials. THE APPLICATION DEADLINE IS Jan. 5, 1996. Also send a letter to Steve Lisberger (address below) indicating that you are applying to the UCSF Neuroscience program with a desire to join the Sloan Center. POSTDOCTORAL applicants, or PREDOCTORAL applicants seeking to do research at the Sloan Center as part of a Ph.D. program in progress in a theoretical discipline elsewhere, should apply as follows: Send a curriculum vitae, a statement of previous research and research goals, up to three relevant publications, and have two letters of recommendation sent to us. THE APPLICATION DEADLINE IS February 1, 1996. UC San Francisco is an Equal Opportunity Employer. Send applications to: Steve Lisberger Sloan Center for Theoretical Neurobiology at UCSF Department of Physiology University of California 513 Parnassus Ave. San Francisco, CA 94143-0444 From radford at cs.toronto.edu Wed Nov 22 21:18:03 1995 From: radford at cs.toronto.edu (Radford Neal) Date: Wed, 22 Nov 1995 21:18:03 -0500 Subject: Performance evaluations: request for comments Message-ID: <95Nov22.211807edt.1585@neuron.ai.toronto.edu> Announcing a draft document on ASSESSING LEARNING PROCEDURES USING DELVE The DELVE development group, University of Toronto http://www.cs.utoronto.ca/neuron/delve/delve.html The DELVE development group requests comments on the draft manual for the DELVE environment from researchers who are interested in how to assess the performance of learning procedures. This manual is available via the DELVE homepage, at the URL above. Carl Rasmussen and Geoffrey Hinton will be talking about the DELVE environment at the NIPS workshop on Benchmarking of Neural Net Learning Algorithms. We would be pleased to hear any comments that attendees of this workshop, or other interested researchers, might have on the current design of the DELVE environment, as described in this draft manual. Here is the introduction to the DELVE manual: DELVE --- Data for Evaluating Learning in Valid Experiments --- is a collection of datasets from many sources, and an environment within which this data can be used to assess the performance of procedures that learn relationships using such data. Many procedures for learning from empirical data have been developed by researchers in statistics, pattern recognition, artificial intelligence, neural networks, and other fields. Learning procedures in common use include simple linear models, nearest neighbor methods, decision trees, multilayer perceptron networks, and many others of varying degrees of complexity. Comparing the performance of these learning procedures in realistic contexts is a surprisingly difficult task, requiring both an extensive collection of real-world data, and a carefully-designed scheme for performing experiments. The aim of DELVE is to help researchers and potential users to assess learning procedures in a way which is relevant to real-world problems and which allows for statistically-valid comparisons of different procedures. Improved assessments will make it easier to determine which learning procedures work best for various applications, and will promote the development of better learning procedures by allowing researchers to easily determine how the performance of a new procedure compares to that of existing procedures. This manual describes the DELVE environment in detail. First, however, we provide an overview of DELVE's capabilities, describe briefly how DELVE organizes datasets and learning tasks, and give an example of how DELVE can be used to assess the performance of a learning procedure. --------------------------------------------------------------------------- Members of the DELVE Development Group: G. E. Hinton R. M. Neal R. Tibshirani M. Revow C. E. Rasmussen D. van Camp R. Kustra Z. Ghahramani ---------------------------------------------------------------------------- Radford M. Neal radford at cs.toronto.edu Dept. of Statistics and Dept. of Computer Science radford at utstat.toronto.edu University of Toronto http://www.cs.toronto.edu/~radford ---------------------------------------------------------------------------- From lemm at LORENTZ.UNI-MUENSTER.DE Thu Nov 23 07:26:30 1995 From: lemm at LORENTZ.UNI-MUENSTER.DE (Joerg_Lemm) Date: Thu, 23 Nov 1995 13:26:30 +0100 Subject: NFL for NFL Message-ID: <9511231226.AA11633@xtp141.uni-muenster.de> I would like to make a comment to the NFL discussion from my point of view. (Thanks to David Wolpert for remaining active in this discussion he initiated.) 1.) If there is no relation between the function values on the test and training set (i.e. P(f(x_j)=y|Data) equal to the unconditional P(f(x_j)=y) ), then, having only training examples y_i = f(x_i) (=data) from a given function, it is clear that I cannot learn anything about values of the function at different arguments, (i.e. for f(x_j), with x_j not equal to any x_i = nonoverlapping test set). 2.) We are considering two of those (influence) relations P(f(x_j)=y|Data): one, named A, for the true nature (=target) and one, named B, for our model under study (=generalizer). Let P(A and B) be the joint probability distribution for the influence relations for target and generalizer. 3.) Of course, we do not know P(A and B), but in good old Bayesian tradition, we can construct a (hyper-)prior P(C) over the family of probability distributions of the joint distributions C = P(A and B). 4.) NFL now uses the very special prior assumption P(A and B) = P(A)P(B), or equivalently P(B|A)=P(B), which means NFL postulates that there is (on average) no relation between nature and model. No wonder that (averaging over targets P(A) or over generalizers P(B) ) cross-validation works as well (or as bad) as anti-cross-validation or anything else in such cases. 5.) But target and generalizer live on the same planet (sharing the same laws, environment, history and maybe even building blocks) so we have very good reasons to assume a bias for (hyper-)priors towards correlated P(A and B) not equal to the uncorrelated product P(A)P(B)! But that's not all: We do have information which is not of the form y=f(x). We know that the probability for many relations in nature to be continuous on certain scales seems to be high (->regularization). We can have additional information about other properties of the function, e.g.symmetries, compare Abu-Mostafa's concept of hints, which produces correlations between A (=target) and B (=model). In this sense I aggree with David Wolpert on looking >>> how to characterize the needed relationship between the set of generalizers and the prior that allows cross-validation to work. >>> To summarize it in a provocative way: There is no free lunch for NFL: Only if you assume that no relation between target and model exists, then you don't find a relation between target and model! And to be precise, I say that it is rational to believe (and David does so too, I think) that in real life cross-validation works better in more cases than anti-cross-validation. Joerg Lemm From jagota at ponder.csci.unt.edu Thu Nov 23 15:22:51 1995 From: jagota at ponder.csci.unt.edu (Jagota Arun Kumar) Date: Thu, 23 Nov 95 14:22:51 -0600 Subject: compressibility, Kolmogorov, learning Message-ID: <9511232022.AA04987@ponder> A short addition to the ongoing discussion thread of Juergen, Barak, and David: The following short paper is what intrigued us (K. Regan and I) to experimentally investigate the performance of algorithms on random versus compressible (i.e., structured) instances of the maximum clique optimization problem. @article{LiV92, author = "Li, M. and P.M.B. Vitanyi", title = "Average case complexity under the universal distribution equals worst-case complexity", pages = "145--149", journal = "Information Processing Letters", volume = 42, month = "May", year = 1992 } In other words, roughly speaking, if one samples uniformly from the universal distribution, one exhibits worst-case behavior of an algorithm. I announced our (experimental, max-clique) paper on Connectionists a few months back, so won't do it again. However, this will be the subject of my talk at the NIPS workshop on optimization (7:30--8:00, Dec 1, Vail). Stop by and jump all over me. Arun Jagota From nin at cns.brown.edu Sat Nov 25 10:41:21 1995 From: nin at cns.brown.edu (Nathan Intrator) Date: Sat, 25 Nov 95 10:41:21 EST Subject: New preprint: multi-task training Message-ID: <9511251541.AA28777@cns.brown.edu> Making a low-dimensional representation suitable for diverse tasks Nathan Intrator Shimon Edelman Tel-Aviv U. Weizmann Ins. We introduce a new approach to the training of classifiers for performance on multiple tasks. The proposed hybrid training method leads to improved generalization via a better low-dimensional representation of the problem space. The quality of the representation is assessed by embedding it in a 2D space using multidimensional scaling, allowing a direct visualization of the results. The performance of the approach is demonstrated on a highly nonlinear image classification task. The paper will be described at the comming Transfer workshop at NIPS url=ftp://cns.brown.edu/nin/papers/mds.ps.Z or it can be accessed through our hope pages: http://www.wisdom.weizmann.ac.il/~edelman/shimon.html http://www.physics.brown.edu/~nin Comments are most welcome. - Nathan Intrator From emj at cs.ucsd.edu Sun Nov 26 02:03:22 1995 From: emj at cs.ucsd.edu (Eric Mjolsness) Date: Sat, 25 Nov 95 23:03:22 -0800 Subject: NIPS 95 workshop schedule Message-ID: <9511260703.AA04033@triangulum> NIPS-95 Workshop, ** Tentative Schedule ** Statistical and Structural Models in Network Vision Friday, November 1, 1995 Organizers: Eric Mjolsness and Anand Rangarajan URL: http://www-cse.ucsd.edu/users/emj/workshop95.html Overview: Neural network and model-based approaches to vision are usually regarded as opposing tendencies. Whereas neural net methods often focus on images and learned feature detectors, model-based methods concentrate on matching high-level representations of objects and their parts and other intrinsic properties. It is possible that the two approaches can be integrated in the context of statistical models which have the flexibility to represent patterns in both image space and in higher-level object feature spaces. The workshop will examine the possibilities for and progress in formulating such models for vision problems, particularly those models which can result in neural network architectures. *Tentative Schedule* 7:30am - 7:50am Eric Mjolsness, "Workshop Overview" 7:50am - 8:15am Chris Bregler, "Soft Features for Soft Classifiers" 8:15am - 8:40am Hayit Greenspan, "Preprocessing and Learning in Rotation-Invariant Texture Recognition" 8:40am - 9:05am Alan Yuille, "Deformable Templates for Object Recognition: Geometry and Lighting" 9:05am - 9:30am Anand Rangarajan, "Bayesian Tomographic Reconstruction using Mechanical Models as Priors" 9:30am - 4:30pm Excercises 4:30pm - 4:50pm Paul Viola, "Recognition by Complex Features" 4:50pm - 5:15pm Lawrence Staib, "Model-based Parametrically Deformable Boundary Finding" 5:15pm - 5:40pm Steven Gold, "Recognizing Objects with Recurrent Neural Networks by Matching Structural Models" 5:40pm - 6:05pm Robert M. Haralick, "Annotated Computer Vision Data Sets" 6:05pm - 6:30pm Eric Saund, "Representations for Perceptual Level Chunks in Line Drawings" *Abstracts* Chris Bregler, U.C.Berkeley, "Soft Features for Soft Classifiers" Most connectionist approaches that are applied to visual domains either make little use of any preprocessing or are based on very high level input representations. The former solutions are motivated by the concern not to lose any useful information for the final classification and show how powerful such algorithms are in extracting relevant features automatically. "Hard" decisions like edge detectors, line-finders, etc. don't fit into the philosophy of adaptability across all levels. We attempt to find a balance between both extrema and show how mature "soft"-preprocessing techniques like rich sets of scaled and rotated gaussian derivatives, second moment texture statistics, and Hierachical Mixtures of Experts can be applied to the domain of car classification. Steven Gold, Yale University, "Recognizing Objects with Recurrent Neural Networks by Matching Structural Models" Attributed Relational Graphs (ARG) are used to create structural models of objects. Recently developed optimization techniques that have emerged out of the neural network/statistical physics framework are then used to construct algorithms to match ARGs. Experiments conducted on ARGs generated from images are presented. Hayit Greenspan, Caltech, "Preprocessing and Learning in Rotation-Invariant Texture Recognition" A number of texture recognition systems have been recently proposed in the literature, giving very high-accuracy classification rates. Almost all of these systems fail miserably when invariances, such as in rotation or scale, are to be included; invariant recognition is clearly the next major challenge in recognition systems. Rotation invariance can be achieved in one of two ways, either by extracting rotation-invariant features, or by appropriate training of the classifier to make it "learn" invariant properties. Learning invariances from the raw data is substantially influenced by the rotation angles that have been included in the system's training set. The more examples, the better the performance. We compare to that a mechanism to extract the rotation-invariant features {\em prior} to the learning phase. We introduce a powerful image representation space with the use of a steerable filter set; along with a new encoding scheme to extract the invariant features. Our results strongly indicate the advantage of extracting a powerful image-representation prior to the learning process; with savings in both storage and computational complexity. Rotation-invariant texture recognition results and demo will be shown. Robert M. Haralick, University of Washington, "Annotated Computer Vision Data Sets" Recognizing features with a protocol that learns from examples, requires that there be many example instances. In this talk, we describe the 78 image annotated data set of RADIUS images, of 2 different 3D scenes, which we have prepared for CDROM distribution. The feature annotation includes building edges, shadow edges, clutter edges, and corner positions. As well the image data set has photogrammetric data of corresponding 3D and 2D points and corresponding 2D pass points. The interior orientation and exterior orientation parameters for all images are given. The availability of this data set makes possible comparison of different algorithms and makes possible very careful experiments of feature extraction using neural net approaches. Eric Mjolsness, "Workshop Overview" I will introduce the workshop and discuss possibilities for integration between some of the research directions represented by the participants. Anand Rangarajan, Yale University, "Bayesian Tomographic Reconstruction using mechanical models as priors" We introduce a new prior---the weak plate---to tomographic reconstruction. MAP estimates are obtained via a deterministic annealing Generalized EM algorithm which avoids poor local minima. Bias/variance simulation results on an autoradiograph phantom demonstrate the superiority of the weak plate prior over other first-order priors used in the literature. Eric Saund, Xerox Palo Alto Research Center, "Representations for Perceptual Level Chunks in Line Drawings" In a line drawing, what makes a box a box? A perfectly drawn box is easy to recognize because it presents a remarkable conjunction of crisp spatial properties yielding a wealth of necessary and sufficient conditions to test. But if it is drawn sloppily, many ideal properties such as closure, squareness, and straightness of the sides, give way. In addressing this problem, most attendants of NIPS probably would look to the supple adaptability and warm fuzziness of statistical approaches over the cold fragility of logic-based specifications. Even in doing this, however, some representations generalize better than others. This talk will address alternative representations for visual objects presented at an elemental level as curvilinear lines, with a look to densely overlapping distributed representations in which a large number of properties can negotiate their relative significance. Lawrence Staib, Yale University, "Model-based Parametrically Deformable Boundary Finding" This work describes a global shape parametrization for smoothly deformable curves and surfaces. This representation is used as a model for geometric matching to image data. The parametrization represents the curve or surface using sinusoidal basis functions and allows a wide variety of smooth boundaries to be described with a small number of parameters. Extrinsic model-based information is incorporated by introducing prior probabilities on the parameters based on a sample of objects. Boundary finding is then formulated as an optimization problem. The method has been applied to synthetic images and three-dimensional medical images of the heart and brain. Paul Viola, Salk Institute, "Recognition by Complex Features" From gem at cogsci.indiana.edu Sun Nov 26 18:45:01 1995 From: gem at cogsci.indiana.edu (Gary McGraw) Date: Sun, 26 Nov 95 18:45:01 EST Subject: Letter recognition thesis Message-ID: Announcing the availability of my thesis on the web... Letter Spirit (part one): Emergent High-level Perception of Letters Using Fluid Concepts Gary McGraw Center for Research on Concepts and Cognition 510 North Fess Street Indiana University Bloomington, IN 47405 This thesis presents initial work on the Letter Spirit project, with a cognitive model of letter perception as its centerpiece. The Letter Spirit project is an attempt to model central aspects of human high-level perception and creativity on a computer, focusing on the creative act of artistic letter-design. The aim is to model the process of rendering the 26 lowercase letters of the roman alphabet in many different, internally coherent styles. Two important and orthogonal aspects of letterforms are basic to the project: the categorical sameness} possessed by instances of a single letter in various styles (e.g., the letter `a' in Times, Palatino, and Helvetica) and the stylistic sameness} possessed by instances of various letters in a single style, or spirit (e.g., the letters `a', `k', and `q' in Times, alone). Starting with one or more seed letters representing the beginnings of a style, the program will attempt to create the rest of the alphabet in such a way that all 26 letters share the same style, or spirit. Letters in the domain are formed exclusively from straight segments on a grid in order to make decisions smaller in number and more discrete. This restriction allows much of low-level vision to be bypassed and forces concentration on higher-level cognitive processing, particularly the abstract and context-dependent character of concepts. The overall architecture of the Letter Spirit project, based on the principles of emergent computation and flexible context-sensitive concepts, has been carefully developed and is presented in Part I. Creating a gridfont is an iterative process of guesswork and evaluation --- the ``central feedback loop of creativity''. The notion of evaluation and its necessary foundation in perception have been largely overlooked in many cognitive models of creativity. In order to be truly creative, a program must do its own evaluating and perceiving. To this end, we have focused initial Letter Spirit work on a high-level perceptual task --- letter perception. We have developed an emergent model of letter perception based on the hypothesis that letter categories are made up of conceptual constituents, called roles}, which exert clear top-down influence on the segmentation of letterforms into structural components. Part II introduces the role hypothesis, presents experimental psychological evidence supporting it, and then introduces a complex cognitive model of letter perception that is consistent with the empirical results. Because we are interested ultimately in the design of letters (and the creative process as whole) an effort was made to develop a model rich enough to be able to recognize and conceptualize a large range of letters including letters at the fringes of their categories. -------------------------------------------------------------------------- The ~400 page thesis is available on the web at URL: It may also be retrieved through ftp to ftp.cogsci.indiana.edu as the file pub/mcgraw.thesis.ps.gz Hardcopy is generally not available due to prohibitive costs. However, if you are having trouble retrieving the text, send me e-mail (gem at cogsci.indiana.edu). Comments and questions are welcome. Gary McGraw *---------------------------------------------------------------------------* | Gary McGraw gem at cogsci.indiana.edu | (__) | |--------------------------------------------------| (oo) | | Center for Research on Concepts and Cognition | /-------\/ | | Department of Computer Science | / | || | | Indiana University (812) 855-6966 | * ||----|| | | | ^^ ^^ | *---------------------------------------------------------------------------* From sayegh at CVAX.IPFW.INDIANA.EDU Sun Nov 26 19:53:57 1995 From: sayegh at CVAX.IPFW.INDIANA.EDU (sayegh@CVAX.IPFW.INDIANA.EDU) Date: Sun, 26 Nov 1995 19:53:57 EST Subject: NN distance learning course announcement Message-ID: <00999FC1.6320AA7C.51@CVAX.IPFW.INDIANA.EDU> FOUNDATIONS AND APPLICATIONS OF NEURAL NETWORKS Course announcement This course is to be offered in the Spring of 1996. Students at remote sites will receive and view lecture tapes at their convenience. Handouts, homework and other assignments will be handled via a web site. This is a 500 level course open to both seniors and graduate students in the Sciences, Mathematics, Engineering, Computer Science, and Psychology or professionals interested in the topic, provided they meet the prerequisites or obtain the instructor's permission. The course is listed as PHYS 525 at Purdue. Please contact the instructor if you are interested. Instructor: Dr. Samir Sayegh sayegh at cvax.ipfw.indiana.edu Phone: (219) 481-6157 FAX: (219) 481-6800 Description: In the last ten years Neural Networks have become both a powerful practical tool to approach difficult classification, optimization and signal processing problems as well as a serious paradigm for computation in parallel machines and biological networks. This course is an introduction to the main concepts and algorithms of neural networks. Lengthy derivations and formal proofs are avoided or minimized and an attempt is made to emphasize the connection between the "artificial" network approaches and their neurobiological counterparts. In order to help achieve that latter goal, the text "A Vision of the Brain" by Semir Zeki is required reading, in addition to the main text "Introduction to the Theory of Neural Computation" by Herz, Krogh and Palmer, and the instructor's (valuable) notes. Zeki's book recounts the fascinating (hi)story of the discovery of the color center in the human visual cortex and emphasizes very general organizational principles of neuroanatomy and neurophysiology, which are highly relevant to any serious computational approach. The following classic topics are covered: - Introduction to the brain and its simpler representations - Neural Computational Elements and Algorithms - Perceptron - Adaptive Linear Element - Backpropagation - Hopfield Model, Associative Memory and Optimization - Kohonen networks - Unsupervised Hebbian learning and principal component analysis - Applications in signals, speech, robotics and forecasting. - Introduction to Computational Neuroscience - Introduction to functional neuroanatomy and functional imaging - Introduction to the visual pathways and computation in retina and visual cortex. Prerequisites: Calculus, matrix algebra and familiarity with a computer language Texts: "A Vision of the Brain" by Semir Zeki (Blackwell, 1993) "Introduction to the Theory of Neural Computation" by Herz, Krogh and Palmer (Addison Wesley, 1991) Instructor's (valuable) notes. Testing: Each lecture comes with a handout that includes a list of objectives and a set of multiple choice questions. The two take-home midterm exams and the final exam will be mostly multiple choice with the questions reflecting the lecture objectives. In addition each student will be expected to complete an individual project in the area of her/his interest. The project may or may not be part of the final grade depending on the project's progress. Software: While students are welcome to use the language of their choice, the high level language MATLAB and the associated toolbox for Neural Networks will be provided for the duration of the course at no additional charge. Cost (US $) Indiana Resident Non resident Undergraduate 249.45 644.450 Graduate 315. 60 751.05 Appendix (brief intro): Neural Networks provide a fruitful approach to a variety of engineering and scientific problems that have been traditionally considered difficult. While an exact definition remains elusive and different practitioners would emphasize one or another of the characteristics of NN, it is possible to list the most common and some of the most fundamental features of neural network solutions: 1) Adaptive 2) Parallel 3) Neurally inspired 4) Ability to handle non-linear problems in a transparent way Let us look at these in some detail: 1) Adaptive solutions are desirable in a number of situations. They present advantages of stability as well as the ability to deal with huge data sets with minimal memory requirements, as the patterns are presented "one at a time." The same advantage implies the possibility of developing real time on-line solutions where the totality of the data set is not available at the outset. 2) The formulation of neural networks solutions is inherently parallel. A large number of nodes share the burden of a computation and often can operate independent of information made available by other nodes. This clearly speeds up computation and allows implementation on highly efficient parallel hardware. 3) Though the extent is somewhat debated, it is clear that there is some similarities between current artificial neural algorithms and biological systems capable of intelligence. The fact that such biological systems still display pattern recognition capabilities far beyond those of our algorithms is a continuing incentive to maintain and further explore the neurobiological connection. 4) The ability to handle nonlinearity is a fundamental requirement of modern scientific and engineering approaches. In a number of fields, the nonlinear approaches are developed on a case by case basis and have often little connection to the better established linear techniques. On the other hand, with the general approach of formulating a neural network and endowing it with increasingly complex processing capabilities, it is possible to define a unified spectrum extending from linear networks (say a one weight-layer ADALINE) to highly nonlinear ones with powerful processing capabilities (say a multilayer backpropagation network). The combination of the properties outlined coupled to the near universal model of neural networks and the availability of software and hardware tools make NN one of the most attractive instruments of signal processing and pattern recognition available today. From rao at cs.rochester.edu Sun Nov 26 22:01:07 1995 From: rao at cs.rochester.edu (rao@cs.rochester.edu) Date: Sun, 26 Nov 1995 22:01:07 -0500 Subject: Paper Available: Dynamic Model of Visual Cortex Message-ID: <199511270301.WAA12985@vulture.cs.rochester.edu> Dynamic Model of Visual Memory Predicts Neural Response Properties In The Visual Cortex Rajesh P.N. Rao and Dana H. Ballard Department of Computer Science University of Rochester Rochester, NY 14627-0226, USA Technical Report 95.4 National Resource Laboratory for the study of Brain and Behavior (November 1995) Abstract Recent neurophysiological experiments have shown that the responses of visual cortical neurons in a monkey freely viewing a natural scene can differ substantially from those obtained when the same image subregions are flashed while the monkey performs a fixation task. Neurophysiological research in the past has been based predominantly on cell recordings obtained during fixation tasks, under the assumption that this data would be useful in predicting the responses in more general situations. It is thus important to understand the differences revealed by the new findings and their relevance to the study of visual perception. We describe a computational model of visual memory which dynamically combines input-driven bottom-up signals with expectation-driven top-down signals to achieve optimal estimation of current state by using a Kalman filter-based framework. Computer simulations of the proposed model are shown to correlate closely with the reported neurophysiological observations in both free-viewing and fixating conditions. The model posits a role for the hierarchical structure of the visual cortex and its reciprocal connections between adjoining visual areas in determining the response properties of visual cortical neurons. ======================================================================== Retrieval information: FTP-host: ftp.cs.rochester.edu FTP-pathname: /pub/u/rao/papers/dynmem.ps.Z URL: ftp://ftp.cs.rochester.edu/pub/u/rao/papers/dynmem.ps.Z 10 pages; 309K compressed, 646K uncompressed e-mail: rao at cs.rochester.edu Hardcopies available upon request at the above address or from the first author at NIPS*95. ========================================================================= From eric at research.nj.nec.com Mon Nov 27 10:51:01 1995 From: eric at research.nj.nec.com (Eric B. Baum) Date: Mon, 27 Nov 1995 10:51:01 -0500 Subject: Response to no-free-lunch discussion In-Reply-To: Barak Pearlmutter "Re: Response to no-free-lunch discussion" (Nov 19, 3:03pm) References: <199511192303.PAA07609@valaga.salk.edu> Message-ID: <9511271051.ZM13175@yin> Barak Pearlmutter remarked that saying We have *no* a priori reason to believe that targets with "low Kolmogorov complexity" (or anything else) are/not likely to occur in the real world. (which I gather was a quote from David Wolpert?) is akin to saying we have no a priori reason to believe there is non-random structure in the world, which is not true, since we make great predictions about the world. Wolpert replied: > To illustrate just one of the possible objections to measuring > randomness with Kolmogorov complexity: Would you say that a > macroscopic gas with a specified temperature is "random"? To describe > it exactly takes a huge Kolmogorov complexity. And certainly in many > regards its position in phase space is "nothing but noise". (Indeed, > in a formal sense, its position is a random sample of the Boltzmann > distribution.) Yet Physicists can (and do) make extraordinarilly > accurate predictions about such creatures with ease. Somebody else (Jurgen Schmidhuber I think?) argued that a gas does *not* have high Kolmogorov complexity, because its time evolution is predictable. So in a lattice gas model, given initial conditions (which are relatively compact, including compact pseudorandom number generator) one may be able to predict evolution of gas. Two comments: (1) While it may be that in classical Lattice gas models, a gas does not have high Kolmogorov complexity, this is not the origin of the predictability exploited by physicists. Statistical mechanics follows simply from the assumption that the gas is in a random one of the accessible states, i.e. the states with a given amount of energy. So *define* a *theoretical* gas as follows: Every time you observe it,it is in a random accessible state. Then its Kolmogorov complexity is huge (there are many accessible states) but its macroscopic behavior is predictable. (Actually this an excellent description of a real gas, given quantum mechanics.) (2) Point 1 is no solace to those arguing for the relevance of Wolpert's theorem, as I understand it. We observe above that non-randomness arises purely out of statistical ensemble effects. This is non-randomness none-the-less. Consider the problem of learning to predict the pressure of a gas from its temperature. Wolpert's theorem, and his faith in our lack of prior about the world, predict, that any learning algorithm whatever is as likely to be good as any other. This is not correct. Interestingly, Wolpert and Macready's results appear irrelevant/wrong here in an entirely *random*, *play* world. We see that learnable structure arises at a macroscopic level, and that our natural instincts about learning (e.g. linear relationships, cross-validation as opposed to anti-cross validation) hold. We don't need to appeal to experience with physical nature in this play world. We could prove theorems about the origin of structure. (This may even be a fruitful thing to do.) Creatures evolving in this "play world" would exploit this structure and understand their world in terms of it. There are other things they would find hard to predict. In fact, it may be mathematically valid to say that one could mathematically construct equally many functions on which these creatures would fail to make good predictions. But so what? So would their competition. This is not relevant to looking for one's key, which is best done under the lamppost, where one has a hope of finding it. In fact, it doesn't seem that the play world creatures would care about all these other functions at all. What was the Einstein quote wondering about the surprising utility of mathematics in understanding the natural world? Maybe mathematics itself provides an answer? -- ------------------------------------- Eric Baum NEC Research Institute, 4 Independence Way, Princeton NJ 08540 PHONE:(609) 951-2712, FAX:(609) 951-2482, Inet:eric at research.nj.nec.com http://www.neci.nj.nec.com:80/homepages/eric/eric.html From omlinc at research.nj.nec.com Mon Nov 27 15:15:28 1995 From: omlinc at research.nj.nec.com (Christian Omlin) Date: Mon, 27 Nov 1995 15:15:28 -0500 Subject: paper available Message-ID: <199511272015.PAA18113@arosa> Following the announcement of the paper by Wolfgang Maas on the computational power of networks consisting of neurons that communicate via spike trains, we thought the following paper may be of interest to the connectionist community. It can be retrieved from the website http://www.neci.nj.nec.com/homepages/omlin/omlin.html We welcome your comments. -Christian ======================================================================= Training Recurrent Neural Networks with Temporal Input Encodings C.W. Omlin (a,b), C.L. Giles (a,c), B.G. Horne (a) L.R. Leerink (d), T. Lin (a) (a) NEC Research Institute 4 Independence Way Princeton, NJ 08540 (b) CS Department RPI Troy, NY 12180 (c) UMIACS University of Maryland College Park, MD 20742 (d) EE Department The University of Sydney Sydney, NSW 2006 Abstract We investigate the learning of deterministic finite-state automata (DFA's) with recurrent networks with a single input neuron, where each input symbol is represented as a temporal pattern and strings as sequences of temporal patterns. We empirically demonstrate that obvious temporal encodings can make learning very difficult or even impossible. Based on preliminary results, we formulate some hypotheses about increase training time compared to training of networks with multiple input neurons. From jkim at FIZ.HUJI.AC.IL Mon Nov 27 14:38:09 1995 From: jkim at FIZ.HUJI.AC.IL (Jai Won Kim) Date: Mon, 27 Nov 1995 21:38:09 +0200 Subject: Preprint announcement: Online-Gibbs Learning Message-ID: <199511271938.AA15158@keter.fiz.huji.ac.il> Dear Jordan Pollack We would like to post an announcement of a new preprint on your network. We attach below the title, authors as well as the abstract of the paper. Subject: announcement of a new preprint: On-line Gibbs Learning FTP-host: keter.fiz.huji.ac.il FTP-file: pub/ON-LINE-LEARNING/online_gibbs.ps.Z The length of the paper: 4 pages. Thanking you in advance for your help. Regard, Haim Sompolinsky and Jaiwon Kim e-mails : haim at fiz.huji.ac.il, jkim at fiz.huji.ac.il _______________________________________________________________________________o On-line Gibbs Learning J. W. Kim and H. Sompolinsky Racah Institute of Physics and Center for Neural Computation, Hebrew University, Jerusalem 91904, Israel e-mails: jkim at fiz.huji.ac.il ; haim at fiz.huji.ac.il (Submitted to Physical Review Letters, Nov 95) ABSTRACT We propose a new model of on-line learning which is appropriate for learning of realizable and unrealizable, smooth as well as threshold, functions. Following each presentation of an example the new weights are chosen from a Gibbs distribution with an on-line energy that balances the need to minimize the instantaneous error against the need to minimize the change in the weights. We show that this algorithm finds the weights that minimize the generalization error in the limit of infinite number of examples. The asymptotic rate of convergence is similar to that of batch learning. From Weevey at cris.com Mon Nov 27 21:47:34 1995 From: Weevey at cris.com (WEEVEY) Date: Mon, 27 Nov 1995 21:47:34 -0500 (EST) Subject: Dissn Research Summary - Primary Visual Cortex In-Reply-To: <9511260703.AA04033@triangulum> Message-ID: The following is the abstract from my dissertation which was completed back in May. More information about this research may be found at the following URL: http://www.cris.com/~Weevey. Sincerely, Eva S. Simmons ************************************************************************ CIRCUITRY STUDIES OF A COMPUTATIONAL MODEL OF THE PRIMARY VISUAL CORTEX Eva Sabrina Simmons, Ph.D. University of Texas at Austin, 1995 Supervisor: Robert E. Wyatt The goals of this project include: (1) proving a procedure for circuit determination of any cytoarchietectonic area of the brain, given certain kinds of data are known as in the computational model being used here, and (2) one circuit of activity will be proposed with three variations on it by changing the connection strength from the standard. Applying the concept of the connection matrix and obtaining basic statistical data about the connec- tions present with respect to presynaptic cells, basic connection data are obtained as the specified anatomy of the cells and random placement in appro- priate layers has allowed. Also, by allowing activity over a period of 20 ms, time propagation data are produced. By keeping a record of activated and deactivated cells at each time step whose minor types have been read-in from a file and by figuring out exactly how each cell was activated, pieces of the circuits can be produced. Later a circuit diagram can be produced from this data. The sets used for this study are: 400 and 2000 cell sets for basic data, and 1000 and 2000 cell sets for variations of connection strength. The following conclusions can be made: (1) The data shows increase in cell type activity with an increase in cell count in the first two time intervals (0.00-5.00 ms). (2) The pattern seen over the time intervals is: The first time interval A (0.00-2.50 ms), is always a period of immense activity. During the second time interval, B (2.55-5.00 ms), the activity continues to be heavy, with no new cell types being activated. The following time inter- vals, C through H (5.05-20.00 ms), moderate activity occurs. (3) The pattern of activity, as found in experiment, is also found here. (4) A pattern of cell type activity is seen when comparing sets to some degree, with some changes depending on cell count and variations in connection strength. (5) The circuits that have been found were as expected in the literature. @}---------- THE SIMMONS FACTOR --------- EVA SABRINA SIMMONS, PH.D. -------{@ WWW Personal Page: http://www.cris.com/~Weevey/index.html @}---- @}---- @}---- WATCH IT, OR IT MIGHT ATTACK!! ;) ---{@ ---{@ ---{@ ---{@ From andreas at sabai.cs.colorado.edu Tue Nov 28 04:43:58 1995 From: andreas at sabai.cs.colorado.edu (Andreas Weigend) Date: Tue, 28 Nov 1995 02:43:58 -0700 (MST) Subject: NIPS Time Series Workshop / Final List Message-ID: <199511280943.CAA12163@sabai.cs.colorado.edu> A non-text attachment was scrubbed... Name: not available Type: text Size: 3165 bytes Desc: not available Url : https://mailman.srv.cs.cmu.edu/mailman/private/connectionists/attachments/00000000/e6297b88/attachment.ksh From A.Sharkey at dcs.shef.ac.uk Tue Nov 28 09:31:51 1995 From: A.Sharkey at dcs.shef.ac.uk (A.Sharkey@dcs.shef.ac.uk) Date: Tue, 28 Nov 95 14:31:51 GMT Subject: Special issue of Connection Science Message-ID: <9511281431.AA02615@entropy.dcs.shef.ac.uk> ************ COMBINING NEURAL NETS ************ CALL FOR PAPERS: Deadline February 14th 1996 Papers are sought for this special issue of Connection Science. The aim of this special issue is to examine when, how, and why neural nets should be combined. The reliability of neural nets can be increased through the use of both redundant and modular nets, (either trained on the same task under differing conditions, or on different subcomponents of a task). Questions about the exploitation of redundancy and modularity in the combination of nets, or estimators, have both an engineering and a biological relevance, and include the following: * how best to combine the outputs of several nets. * quantification of the benefits of combining. * how best to create redundant nets that generalise differently (e.g. active learning methods). * how to effectively subdivide a task. * communication between neural net modules. * increasing the reliability of nets. * the use of neural nets for safety critical applications. Special issue editor: Amanda Sharkey (Sheffield, UK) Editorial Board: Leo Breiman (Berkeley, USA) Nathan Intrator (Brown, USA) Robert Jacobs (Rochester, USA) Michael Jordan (MIT, USA) Paul Munro (Pittsburgh, USA) Michael Perrone (IBM, USA) David Wolpert (Santa Fe Institute, USA) We solicit either theoretical or experimental papers on this topic. Questions and submissions concerning this special issue should be sent by February 14th 1996 to: Dr Amanda Sharkey,Department of Computer Science,Regent Court,Portobello Street,University of Sheffield,Sheffield, S1 4DP,United Kingdom. Email: amanda at dcs.shef.ac.uk From ruppin at math.tau.ac.il Tue Nov 28 12:23:55 1995 From: ruppin at math.tau.ac.il (Eytan Ruppin) Date: Tue, 28 Nov 1995 19:23:55 +0200 Subject: MEMORY Message-ID: Adams Super Center for Brain Studies at Tel Aviv University =========================================================== Workshop Announcement MEMORY ORGANIZATION AND CONSOLIDATION: COGNITIVE AND COMPUTATIONAL PERSPECTIVES A workshop on Memory Organization and Consolidation will be held during May 28-30, 1996 at Tel-Aviv University, Israel. This meeting is sponsored by Mr. Branco Weiss. In the last two decades the field of memory research has grown tremendously. This rapid expansion has been manifested in the recognition of the multiplicity of memory systems and the rise in popularity of multiple-memory system analysis, in the ability to trace changes of brain activity during memory performance using novel molecular, electrophysiological and imaging techniques, and in the development of fairly complex models of memory. The planned workshop will address these issues and discuss how memory storage and retrieval processes are organized in the brain. In particular, we shall focus on memory consolidation. This process involves the alteration of memory traces from temporary, `short-term' storage to `long-term' memory stores. It is a fundamental and intriguing process, which is considered to be strongly connected to our ability to form generalizations by learning from examples, and may depend also upon the integrity of specific sleep stages. The process of consolidation has recently become accessible to formal analysis using novel computational neural models. The workshop will provide a meeting ground for both experimental and computational research approaches. Numerous questions arise with regard to consolidation theory: What explanations could it offer? What are its neuronal (molecular, neurochemical) foundations? What are the relations between the consolidation processes and the circadian cycle? What are modulators of consolidation? What insights can be gained from computational models and how can the predictions they make be tested experimentally? The multidisciplinary nature of memory consolidation research, together with recent advancements, make the proposed workshop a promising opportunity for a timely and fruitful exchange of ideas between researchers employing different research methodologies, but sharing common interests in the study of memory. The workshop will consist of a three day meeting, and will include a series of invited talks, a poster session, and discussion panels. We have invited speakers from different disciplines of the Neurosciences who will discuss psychological, neurological, physiological and computational perspectives of the subject. An informal atmosphere will be maintained, encouraging questions and discussions. CURRENTLY CONFIRMED SPEAKERS Martin Albert (Boston) Daniel Amit (Jerusalem and Rome) Yadin Dudai (Weizmann Institute) Yoram Feldon (Zurich and Tel Aviv) Mark Gluck (Rutgers) Michael Hasselmo (Harvard) Avi Karni (NIH) Amos Korzcyn (Tel Aviv) Jay McClelland (CMU) Bruce McNaughton (Arizona) Matti Mintz (Tel Aviv) Morris Moscovitch (Toronto) Richard Thompson (USC) CALL FOR ABSTRACTS Individuals wishing to present a poster related to any aspect of the workshop's theme should submit an abstract describing the nature of their presentation. The single page submission should include title, author(s), contact information (address and email/fax), and abstract, and will be reviewed by the Program Committee. Abstract submissions should ARRIVE by March 31st, 1996, and should be sent to Eytan Ruppin, Dept. of Computer-Science, Tel-Aviv University, Tel-Aviv, Israel, 69978. Program Committee: ----------------- David Horn, Michael Mislobodsky and Eytan Ruppin (Tel-Aviv). Registration and Further Information: ----------------------------------- To register for the workshop, please fill out the registration form attached below and send it to Mrs. Bila Lenczner Adams Super Center for Brain Studies Tel Aviv University Tel Aviv 69978, Israel Tel.: 972-3-6407377 Fax: 972-3-6407932 email:memory at neuron.tau.ac.il The workshop will take place at the Gordon faculty club of Tel Aviv University. The registration fee of $70 covers lunch and refreshments throughout the three days of the workshop. Optionally one may register for $30 covering refreshments only. Since the number of places is limited please register early to secure your participation. Further questions about conference administration should be directed to Mrs. Bila Lenczner. For questions about the workshop technical/scientific content or absract submissions, please contact Eytan Ruppin Dept. of Computer Science Tel-Aviv University, Tel-Aviv, 69978, Israel. Tel.: 972-3-6407864 Fax: 972-3-6409357 email: ruppin at math.tau.ac.il The final workshop program and updated information will be available on a WWW homepage at http://neuron.tau.ac.il/Adams/memory ================================================================== REGISTRATION FORM MEMORY WORKSHOP May 28-30, 1996 Name: ___________________________________________________ Affiliation: ________________________________________________ Address: _________________________________________________ _________________________________________________________ Telephone: ___________________________ Fax: ________________________________ e-mail: ______________________________ ___ $70 Registration fee including lunch ___ $30 Registration fee including refreshments only Amount Enclosed: $________________ MAKE CHECKS PAYABLE TO "Tel Aviv University" From jbower at bbb.caltech.edu Tue Nov 28 22:21:05 1995 From: jbower at bbb.caltech.edu (jbower@bbb.caltech.edu) Date: Tue, 28 Nov 95 19:21:05 PST Subject: Call for Papers -- CNS*96 Message-ID: ********************************************************************** CALL FOR PAPERS Fifth Annual Computational Neuroscience Meeting CNS*96 July 14 - 17, 1996 Boston, Massachusetts ................ DEADLINE FOR SUMMARIES AND ABSTRACTS: **>> January 25, 1996 <<** ^^^^^^^^^^^^^^^^ This is the fifth annual meeting of an interdisciplinary conference addressing a broad range of research approaches and issues involved in the field of computational neuroscience. These meetings bring together experimental and theoretical neurobiologists along with engineers, computer scientists, cognitive scientists, physicists, and mathematicians interested in the functioning of biological nervous systems. The peer reviewed papers presented at the conference are all related to understanding how nervous systems compute. As in previous years, CNS*96 will equally emphasize experimental, model-based, and more abstract theoretical approaches to understanding neurobiological computation. The meeting in 1996 will take place at the Cambridge Center Marriott Hotel and include plenary, contributed, and poster sessions. The first session starts at 9 am, Sunday July 14th and the last session ends at 5 pm on Wednesday, July 17th. There will be no parallel sessions and the full text of presented papers will be published in a proceedings volume. The meeting will include time for informal workshops focused on current issues in computational neuroscience. Travel funds will be available for students and postdoctoral fellows presenting papers at the meeting. Day care will be available for children. SUBMISSION INSTRUCTIONS: With this announcement we solicit the submission of papers for presentation. All papers will be refereed. Authors should send original research contributions in the form of a 1000-word (or less) summary and a separate single page 100 word abstract clearly stating their results. Summaries are for program committee use only. Abstracts will be published in the conference program. At the bottom of each abstract page and on the first summary page, indicate preference for oral or poster presentation and specify at least one appropriate category and theme from the following list: Presentation categories: A. Theory and Analysis B. Modeling and Simulation C. Experimental D. Tools and Techniques Themes: A. Development B. Cell Biology C. Excitable Membranes and Synaptic Mechanisms D. Neurotransmitters, Modulators, Receptors E. Sensory Systems 1. Somatosensory 2. Visual 3. Auditory 4. Olfactory 5. Other systems F. Motor Systems and Sensory Motor Integration G. Learning and Memory H. Behavior I. Cognition J. Disease Include addresses of all authors on the front of the summary and the abstract including the E-mail address for EACH author. Indicate on the front of the summary to which author correspondence should be addressed. Also, indicate first author. Program committee decisions will be sent to the corresponding author only. Submissions will not be considered if they lack category information, separate abstract sheets, author addresses, or are late. Submissions can be made by surface mail ONLY by sending 6 copies of the abstract and summary to: CNS*96 Submissions Division of Biology 216-76 Caltech Pasadena, CA 91125 ADDITIONAL INFORMATION can be obtained by: o Using our on-line WWW information and registration server at the URL: http://www.bbb.caltech.edu/cns96/cns96.html o ftp-ing to our ftp site: yourhost% ftp ftp.bbb.caltech.edu Name: ftp Password: yourname at yourhost.yoursite.yourdomain ftp> cd pub/cns96 ftp> ls ftp> get filename o Sending Email to: cns96 at smaug.bbb.caltech.edu CNS*96 ORGANIZING COMMITTEE: Co-meeting chair / logistics - Mike Hasselmo, Harvard University Co-meeting chair / finances - John Miller, UC Berkeley Co-meeting chair / program - Jim Bower, Caltech Program Committee: Charlie Anderson, Washington University Axel Borst, Max-Planck Inst., Tuebingen, Germany Dennis Glanzman, NIMH/NIH Nancy Kopell, Boston University Christiane Linster, Harvard University Mark Nelson, University of Illinois, Urbana Maureen Rush, California State University, Bakersfield Karen Sigvardt, University of California, Davis Philip Ulinski, University of Chicago Regional Organizers: Europe- Erik De Schutter (Belgium) Middle East - Idan Segev (Jerusalem) Down Under - Mike Paulin (New Zealand) South America - Renato Sabbatini (Brazil) Asia - Zhaoping Li (Hong Kong) ********************************************************************** *************************************** James M. Bower Division of Biology Mail code: 216-76 Caltech Pasadena, CA 91125 (818) 395-6817 (818) 449-0679 FAX NCSA Mosaic addresses for: laboratory http://www.bbb.caltech.edu/bowerlab GENESIS: http://www.bbb.caltech.edu/GENESIS science education reform http://www.caltech.edu/~capsi From tgelder at phil.indiana.edu Wed Nov 29 00:20:42 1995 From: tgelder at phil.indiana.edu (Tim van Gelder) Date: Wed, 29 Nov 1995 00:20:42 -0500 Subject: 'Mind as Motion' annct & web page Message-ID: Book announcement ::: Available now. `MIND AS MOTION: EXPLORATIONS IN THE DYNAMICS OF COGNITION' edited by Robert Port and Tim van Gelder Bradford Books/MIT Press. >From the dust jacket: `Mind as Motion is the first comprehensive presentation of the dynamical approach to cognition. It contains a representative sampling of original current research on topics such as perception, motor control, speech and language, decision making, and development. Included are chapters by pioneers of the approach, as well as others applying the tools of dynamics to a wide range of new problems. Throughout, particular attention is paid to the philosophical foundations of this radical new research program. Mind as Motion provides a conceptual and historical overview of the dynamical approach, a tutorial introduction to dynamics for cognitive scientists, and a glossary covering the most frequently used terms. Each chapter includes an introduction by the editors, outlining its main ideas and placing it in context, and a guide to further reading.' 668 pages, 139 illustrations ISBN 0-262-16150-8 $60.00 US For further information including the full text of the preface and a sample chapter introduction, see the web page at MIT Press: http://www-mitpress.mit.edu/mitp/recent-books/cog/mind-as-motion.html _________________________________________________ Chapter Titles and Authors 1.Tim van Gelder & Robert Port. Introduction: Its About Time: An Overview of the Dynamical Approach to Cognition. 2. Alec Norton. Dynamics: A Tutorial Introduction for Cognitive Scientists. 3. Esther Thelen. Time Scale Dynamics and the Development of an Embodied Cognition. 4. Jerome Busemeyer & James Townsend Dynamic Representation of Decision Making 5. Randall Beer Computational and Dynamical Languages for Autonomous Agents 6. Elliot Saltzman Dynamics and Coordinate Systems in Skilled Sensorimotor Activity 7. Catherine Browman & Louis Goldstein Dynamics and Articulatory Phonology 8. Jeffrey Elman Language as a Dynamical System 9. Jean Petitot Morphodynamics and Attractor Syntax 10. Jordan Pollack The Induction of Dynamical Recognizers 11. Paul van Geert Growth Dynamics in Development 12. Robert Port, Fred Cummins & Devin McAuley Naive Time, Temporal Patterns and Human Audition 13. Michael Turvey & Claudia Carello Some Dynamical Themes in Perception and Action 14. Geoffrey Bingham Dynamics and the Problem of Event Recognition 15. Stephen Grossberg Neural Dynamics of Motion Perception, Recognition Learning, and Spatial Attention 16. Mary Ann Metzger Multiprocess Models Applied to Cognitive and Behavioral Dynamics 17. Steven Reidbord & Dana Redington The Dynamics of Mind and Body During Clinical Interviews: Current Trends, Potential, and Future Directions 18. Marco Giunti Dynamical Models of Cognition 19. Glossary of Terminology in Dynamics _________________________________________________ At MIT Press, orders can be made by email at: mitpress-orders at mit.edu For general information re MIT Press, see: http://www-mitpress.mit.edu _________________________________________________ The editors welcome enquiries, discussion, critical feedback, etc. Robert Port Tim van Gelder Department of Linguistics Department of Philosophy Indiana University University of Melbourne Bloomington IN 47405 Parkville 3052 VIC USA AUSTRALIA port at indiana.edu tgelder at ariel.unimelb.edu.au From josh at faline.bellcore.com Wed Nov 29 13:19:47 1995 From: josh at faline.bellcore.com (Joshua Alspector) Date: Wed, 29 Nov 1995 13:19:47 -0500 Subject: Research associate in neuromorphic electronics Message-ID: <199511291819.NAA15771@faline.bellcore.com> RESEARCH ASSOCIATE IN NEUROMORPHIC ELECTRONICS There is an anticipated position in the electrical and computer engineering department at the University of Colorado at Colorado Springs for a postdoctoral research associate in the area of neural learning microchips. The successful candidate will have experience in analog and digital VLSI design and test and be comfortable working at the system level in a UNIX/C/C++ environment. The project will involve applying an existing VME-based neural network learning system to several demanding problems in signal processing. These include adaptive non-linear equalization of underwater acoustic communication channels and magnetic recording channels. It is likely also to involve integrating the learning electronics with micro-machined sonic transducers directly on silicon. Please send a curriculum vita, names and addresses of at least three referees, and and copies of some representative publications to: Prof. Joshua Alspector Univ. of Colorado at Col. Springs Dept. of Elec. & Comp. Eng. P.O. Box 7150 Colorado Springs, CO 80933-7150 From ajit at austin.ibm.com Wed Nov 29 14:46:04 1995 From: ajit at austin.ibm.com (Dingankar) Date: Wed, 29 Nov 1995 13:46:04 -0600 Subject: "Network Approximation of Dynamical Systems" - Neuroprose paper Message-ID: <9511291946.AA12676@ding.austin.ibm.com> **DO NOT FORWARD TO OTHER GROUPS** Sorry, no hardcopies available. 6 pages. Greetings! The following invited paper will be presented at NOLTA'95 next month. The compressed PostScript file is available in the Neuroprose archive; the details (URL, bibtex entry and abstract) follow. Thanks, Ajit ------------------------------------------------------------------------------ URL: ftp://archive.cis.ohio-state.edu/pub/neuroprose/dingankar.tensor-products.ps.Z BiBTeX entry: @INPROCEEDINGS{atd:nolta-95, AUTHOR ="Dingankar, Ajit T. and Sandberg, Irwin W.", TITLE ="{Network Approximation of Dynamical Systems}", BOOKTITLE ="Proceedings of the International Symposium on Nonlinear Theory and its Applications (NOLTA'95)", YEAR ="1995", EDITOR ="", PAGES ="", ORGANIZATION ="", PUBLISHER ="", ADDRESS ="Las Vegas, Nevada", MONTH ="December 10--14" } Network Approximation of Dynamical Systems ------------------------------------------ ABSTRACT We consider the problem of approximating any member of a large class of input-output operators of time-varying nonlinear dynamical systems. We introduce a family of ``tensor product" dynamical neural networks, and show that a certain continuity condition is necessary and sufficient for the existence of arbitrarily good approximations using this family. ------------------------------------------------------------------------------ Ajit T. Dingankar | ajit at austin.ibm.com IBM Corporation, Internal Zip 4359 | Work: (512) 838-6850 11400 Burnet Road, Austin, TX 78758 | Fax : (512) 838-5882 From Connectionists-Request at cs.cmu.edu Wed Nov 1 00:05:57 1995 From: Connectionists-Request at cs.cmu.edu (Connectionists-Request@cs.cmu.edu) Date: Wed, 01 Nov 95 00:05:57 EST Subject: Bi-monthly Reminder Message-ID: <25785.815202357@B.GP.CS.CMU.EDU> *** DO NOT FORWARD TO ANY OTHER LISTS *** This note was last updated September 9, 1994. This is an automatically posted bi-monthly reminder about how the CONNECTIONISTS list works and how to access various online resources. CONNECTIONISTS is a moderated forum for enlightened technical discussions and professional announcements. It is not a random free-for-all like comp.ai.neural-nets. Membership in CONNECTIONISTS is restricted to persons actively involved in neural net research. The following posting guidelines are designed to reduce the amount of irrelevant messages sent to the list. Before you post, please remember that this list is distributed to thousands of busy people who don't want their time wasted on trivia. Also, many subscribers pay cash for each kbyte; they shouldn't be forced to pay for junk mail. -- Dave Touretzky & Lisa Saksida --------------------------------------------------------------------- What to post to CONNECTIONISTS ------------------------------ - The list is primarily intended to support the discussion of technical issues relating to neural computation. - We encourage people to post the abstracts of their latest papers and tech reports. - Conferences and workshops may be announced on this list AT MOST twice: once to send out a call for papers, and once to remind non-authors about the registration deadline. A flood of repetitive announcements about the same conference is not welcome here. - Requests for ADDITIONAL references. This has been a particularly sensitive subject. Please try to (a) demonstrate that you have already pursued the quick, obvious routes to finding the information you desire, and (b) give people something back in return for bothering them. The easiest way to do both these things is to FIRST do the library work to find the basic references, then POST these as part of your query. Here's an example: WRONG WAY: "Can someone please mail me all references to cascade correlation?" RIGHT WAY: "I'm looking for references to work on cascade correlation. I've already read Fahlman's paper in NIPS 2, his NIPS 3 abstract, corresponded with him directly and retrieved the code in the nn-bench archive. Is anyone aware of additional work with this algorithm? I'll summarize and post results to the list." - Announcements of job openings related to neural computation. - Short reviews of new textbooks related to neural computation. To send mail to everyone on the list, address it to Connectionists at CS.CMU.EDU ------------------------------------------------------------------- What NOT to post to CONNECTIONISTS: ----------------------------------- - Requests for addition to the list, change of address and other administrative matters should be sent to: "Connectionists-Request at cs.cmu.edu" (note the exact spelling: many "connectionists", one "request"). If you mention our mailing list to someone who may apply to be added to it, please make sure they use the above and NOT "Connectionists at cs.cmu.edu". - Requests for e-mail addresses of people who are believed to subscribe to CONNECTIONISTS should be sent to postmaster at appropriate-site. If the site address is unknown, send your request to Connectionists-Request at cs.cmu.edu and we'll do our best to help. A phone call to the appropriate institution may sometimes be simpler and faster. - Note that in many mail programs a reply to a message is automatically "CC"-ed to all the addresses on the "To" and "CC" lines of the original message. If the mailer you use has this property, please make sure your personal response (request for a Tech Report etc.) is NOT broadcast over the net. ------------------------------------------------------------------------------- The CONNECTIONISTS Archive: --------------------------- All e-mail messages sent to "Connectionists at cs.cmu.edu" starting 27-Feb-88 are now available for public perusal. A separate file exists for each month. The files' names are: arch.yymm where yymm stand for the obvious thing. Thus the earliest available data are in the file: arch.8802 Files ending with .Z are compressed using the standard unix compress program. To browse through these files (as well as through other files, see below) you must FTP them to your local machine. The file "current" in the same directory contains the archives for the current month. ------------------------------------------------------------------------------- How to FTP Files from the CONNECTIONISTS Archive ------------------------------------------------ 1. Open an FTP connection to host B.GP.CS.CMU.EDU 2. Login as user anonymous with password your username. 3. 'cd' directly to the following directory: /afs/cs/project/connect/connect-archives The archive directory is the ONLY one you can access. You can't even find out whether any other directories exist. If you are using the 'cd' command you must cd DIRECTLY into this directory. Problems? - contact us at "Connectionists-Request at cs.cmu.edu". ------------------------------------------------------------------------------- Using Mosaic and the World Wide Web ----------------------------------- You can also access these files using the following url: http://www.cs.cmu.edu:8001/afs/cs/project/connect/connect-archives ---------------------------------------------------------------------- The NEUROPROSE Archive ---------------------- Anonymous FTP on archive.cis.ohio-state.edu (128.146.8.52) pub/neuroprose directory This directory contains technical reports as a public service to the connectionist and neural network scientific community which has an organized mailing list (for info: connectionists-request at cs.cmu.edu) Researchers may place electronic versions of their preprints in this directory, announce availability, and other interested researchers can rapidly retrieve and print the postscripts. This saves copying, postage and handling, by having the interested reader supply the paper. We strongly discourage the merger into the repository of existing bodies of work or the use of this medium as a vanity press for papers which are not of publication quality. PLACING A FILE To place a file, put it in the Inbox subdirectory, and send mail to pollack at cis.ohio-state.edu. Within a couple of days, I will move and protect it, and suggest a different name if necessary. Current naming convention is author.title.filetype.Z where title is just enough to discriminate among the files of the same author. The filetype is usually "ps" for postscript, our desired universal printing format, but may be tex, which requires more local software than a spooler. The Z indicates that the file has been compressed by the standard unix "compress" utility, which results in the .Z affix. To place or retrieve .Z files, make sure to issue the FTP command "BINARY" before transfering files. After retrieval, call the standard unix "uncompress" utility, which removes the .Z affix. An example of placing a file is in the appendix. Make sure your paper is single-spaced, so as to save paper, and include an INDEX Entry, consisting of 1) the filename, 2) the email contact for problems, 3) the number of pages and 4) a one sentence description. See the INDEX file for examples. ANNOUNCING YOUR PAPER It is the author's responsibility to invite other researchers to make copies of their paper. Before announcing, have a friend at another institution retrieve and print the file, so as to avoid easily found local postscript library errors. And let the community know how many pages to expect on their printer. Finally, information about where the paper will/might appear is appropriate inside the paper as well as in the announcement. In your subject line of your mail message, rather than "paper available via FTP," please indicate the subject or title, e.g. "paper available "Solving Towers of Hanoi with ART-4" Please add two lines to your mail header, or the top of your message, so as to facilitate the development of mailer scripts and macros which can automatically retrieve files from both NEUROPROSE and other lab-specific repositories: FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/filename.ps.Z When you announce a paper, you should consider whether (A) you want it automatically forwarded to other groups, like NEURON-DIGEST, (which gets posted to comp.ai.neural-networks) and if you want to provide (B) free or (C) prepaid hard copies for those unable to use FTP. To prevent forwarding, place a "**DO NOT FORWARD TO OTHER GROUPS**" at the top of your file. If you do offer hard copies, be prepared for a high cost. One author reported that when they allowed combination AB, the rattling around of their "free paper offer" on the worldwide data net generated over 2000 hardcopy requests! A shell script called Getps, written by Tony Plate, is in the directory, and can perform the necessary retrieval operations, given the file name. Functions for GNU Emacs RMAIL, and other mailing systems will also be posted as debugged and available. At any time, for any reason, the author may request their paper be updated or removed. For further questions contact: Jordan Pollack Associate Professor Computer Science Department Center for Complex Systems Brandeis University Phone: (617) 736-2713/* to fax Waltham, MA 02254 email: pollack at cs.brandeis.edu APPENDIX: Here is an example of naming and placing a file: unix> compress myname.title.ps unix> ftp archive.cis.ohio-state.edu Connected to archive.cis.ohio-state.edu. 220 archive.cis.ohio-state.edu FTP server ready. Name: anonymous 331 Guest login ok, send ident as password. Password:neuron 230 Guest login ok, access restrictions apply. ftp> binary 200 Type set to I. ftp> cd pub/neuroprose/Inbox 250 CWD command successful. ftp> put myname.title.ps.Z 200 PORT command successful. 150 Opening BINARY mode data connection for myname.title.ps.Z 226 Transfer complete. 100000 bytes sent in 1.414 seconds ftp> quit 221 Goodbye. unix> mail pollack at cis.ohio-state.edu Subject: file in Inbox. Jordan, I just placed the file myname.title.ps.Z in the Inbox. Here is the INDEX entry: myname.title.ps.Z mylogin at my.email.address 12 pages. A random paper which everyone will want to read Let me know when it is in place so I can announce it to Connectionists at cmu. ^D AFTER RECEIVING THE GO-AHEAD, AND HAVING A FRIEND TEST RETRIEVE THE FILE, HE DOES THE FOLLOWING: unix> mail connectionists Subject: TR announcement: Born Again Perceptrons FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/myname.title.ps.Z The file myname.title.ps.Z is now available for copying from the Neuroprose repository: Random Paper (12 pages) Somebody Somewhere Cornell University ABSTRACT: In this unpublishable paper, I generate another alternative to the back-propagation algorithm which performs 50% better on learning the exclusive-or problem. ~r.signature ^D ------------------------------------------------------------------------ How to FTP Files from the NN-Bench Collection --------------------------------------------- 1. Create an FTP connection from wherever you are to machine "ftp.cs.cmu.edu" (128.2.254.155). 2. Log in as user "anonymous" with password your username. 3. Change remote directory to "/afs/cs/project/connect/bench". Any subdirectories of this one should also be accessible. Parent directories should not be. Another valid directory is "/afs/cs/project/connect/code", where we store various supported and unsupported neural network simulators and related software. 4. At this point FTP should be able to get a listing of files in this directory and fetch the ones you want. Problems? - contact us at "neural-bench at cs.cmu.edu". From marks at u.washington.edu Wed Nov 1 19:40:55 1995 From: marks at u.washington.edu (Robert Marks) Date: Wed, 1 Nov 95 16:40:55 -0800 Subject: NN Press Release Message-ID: <9511020040.AA13754@carson.u.washington.edu> IEEE Neural Networks Council PRESS RELEASE Detroit, Michigan, November 1, 1995 Awards Committee, IEEE Neural Networks Council 1995 IEEE Neural Networks Council Pioneer Awards Professors Michael A. Arbib and Nils J. Nilsson and Dr. Paul J. Werbos have been selected to receive the 1995 IEEE Neural Networks Council Pioneer Awards. The awards will be presented at the Banquet of the 1995 IEEE International Conference on Neural Networks (ICNN '95) in Perth (at the Tumbulgum Farm), Western Australia, on Thursday November 30, 1995. The IEEE Neural Networks Council Pioneer Awards have been established to recognize and honor the vision of those people whose efforts resulted in significant contributions to the early concepts and developments in the neural networks field. 1995 marks the fifth year for this award, which is to be presented to outstanding individuals for contributions made at least fifteen years earlier. The three individuals receiving Pioneer Awards in 1995 are internationally recognized experts who have made pioneering technical contributions in the neural networks field. The following is a brief description of the awardeesU pioneering contributions that the Pioneer Award recognize and biographies which provide an overview of the distinguished careers of the awardees. Michael A. Arbib pioneered Neural Networks in Australia, writing his first paper on the subject as an undergraduate at Sydney University in 1960, and basing his first book (Brains, Machines and Mathematics, McGraw- Hill 1964) on lectures presented at the University of New South Wales. He is being honored for his pioneering work on the development of a system-theoretic approach to the brain in the early sixties. He has very actively advanced the notion that the brain is not a computer in the recent technological sense, but that we can learn much about brains from studying machines, and much about machines from studying brains. His thoughts have influenced, encouraged, and encharmed many researchers in the field of neural networks. Arbib is Professor of Computer Science, Neurobiology and Physiology, as well as of Biomedical Engineering, Electrical Engineering, and Psychology at the University of Southern California, which he joined in September of 1986. Born in England in 1940, he grew up in Australia where he had earned his B.Sc. (Hons.) from Sydney University. Dr. Arbib later moved to the U.S. where he received his Ph.D. in Mathematics from MIT in 1963, spending two years as a Research Assistant to Warren McCulloch. After five years at Stanford University (as Assistant Professor and later as Associate Professor), he became Chairman of the Department of Computer and Information Science at the University of Massachusetts at Amherst in 1970, and remained in the Department until August of 1986. Dr. Arbib currently directs a major interdisciplinary project on "Neural Plasticity: Data and Computational Structures", integrating studies of the brain with new approaches to databases, visualization, simulation, and the World Wide Web. His own research focuses on mechanisms underlying the coordination of perception and action. The author of twenty books and the editor of eleven more, Arbib has most recently edited The Handbook of Brain Theory and Neural Networks (The MIT Press, 1995). Nils J. Nilsson is being honored with the IEEE Neural Networks Council Pioneer Award for his contribution to the theory of perceptrons and learning machines. His outstanding contribution in the area of neural networks was his 1965 Pioneering book, Learning Machines: Foundations of Trainable Pattern-Classifying Systems. This was the definitive book on the subject during that decade. The book treated algorithms, learning, capacity, and multi-layer perceptrons. He did it in an accessible manner, which influenced a whole decade of research in the area. Nils J. Nilsson is Professor of Computer Science at Stanford University. Born in Saginaw, Michigan, in 1933, Nilsson's early education was in schools in Michigan and Southern California. He attended Stanford University both as an undergraduate and a graduate student and earned M.S. and Ph.D. degrees in Electrical Engineering, in 1956 and 1958, respectively. For three years after his Ph.D., he served as an Air Force lieutenant at the Rome Air Development Center in Rome, New York, where he performed research in radar signal detection. Dr. Nilsson joined SRI International (then called the Stanford Research Institute) in 1961. His early work there was on statistical and neural-network approaches to pattern recognition and led to his influential book Learning Machines: Foundations of Trainable Pattern-Classifying Systems (McGraw- Hill, 1965). Later at SRI, Dr. Nilsson became interested in broader aspects of AI which led to the publication of his two books: Problem Solving Methods in Artificial Intelligence (McGraw-Hill, 1971), and Principles of Artificial Intelligence (Morgan Kaufmann, San Francisco, CA 1980). Dr. Nilsson also led the project that developed the SRI robot RShakeyS and served as the director of the SRI Artificial Intelligence Center for the years 1980 to 1984. Professor Nilsson returned to Stanford in 1985 as the Chairman of the Department of Computer Science, a position he held until August 1990. At Stanford, he coauthored (with Michael Genesereth) the book Logical Foundations of Artificial Intelligence (Morgan Kaufmann, San Francisco, CA 1987). His most recent research is on the problem of combining deliberate (and sometimes slow) robot-reasoning processes with mechanisms for making more rapid, stero-typical responses to dynamic, time-critical situations. He is also interested in applying machine learning and adaptive computation techniques to this problem. Professor Nilsson served as the AI Area Editor for the Journal of the Association for Computing Machinery, is on the editorial board of the journal Artificial Intelligence, and is a past-president and Fellow of the American Association for Artificial Intelligence. He is also a fellow of the American Association for the Advancement of Science and has been elected as a foreign member of the Royal Swedish Academy of Engineering Sciences. He helped found and is on the board of directors of Morgan Kaufmann Publishers, Inc. Paul J. Werbos is being honored for doing much of the ground work, in the early seventies, for what has now emerged as the practical back-propagation learning algorithm in multi-layer networks; and for his continuing and sustained contributions to current advances in neurocontrol. Dr. Werbos holds four degrees from Harvard University and the London School of Economics, covering economics, mathematical physics, decision and control. His 1974 Harvard Ph.D. thesis presented the Rbackpropagation methodS for the first time, permitting the efficient calculation of derivatives and adaptation of all kinds of nonlinear sparse structures, including neural networks; it has been reprinted in its entirety in his book, The Roots of Backpropagation (Wiley, 1994) along with several related seminal and tutorial papers. In these and other more recent papers, he has described how backpropagation may be incorporated into new intelligent control designs with extensive parallels to the structure of the human brain. Dr. Werbos runs the Neuroengineering program and the SBIR Next Generation Vehicle program at the National Science Foundation. He is Past President of the International Neural Network Society, and a member of the IEEE Control and SMC Societies. Prior to NSF, he worked at the University of Maryland and the U.S. Department of Energy. He was born in 1947 near Philadelphia, Pennsylvania. His publications range from neural networks to quantum foundations, energy economics, and issues of consciousness. Mohamad H. Hassoun Professor Department of Electrical and Computer Engineering Wayne State University 5050 Anthony Wayne Drive Detroit, MI 48202 Tel. (313) 577-3920 Fax. (313) 577-1101 From malaka at ira.uka.de Thu Nov 2 05:09:47 1995 From: malaka at ira.uka.de (Rainer Malaka) Date: Thu, 02 Nov 1995 11:09:47 +0100 Subject: (p)reprints and WWW-site on olfactory modelling Message-ID: <"irafs2.ira.414:02.11.95.10.10.06"@ira.uka.de> Dear connectionists, several re- and preprints on olfactory modelling, classical conditioning and spiking neural networks are available on our WWW-server: http://i11www.ira.uka.de:80/~malaka/publications.html including the following papers: R. Malaka Dynamical odor coding in a model of the antennal lobe. In Proceedings of the International Conference on Artificial Neural Networks (ICANN`95), Paris , volume 2 Abstract: A model for the insect antennal lobe is presented. The model is embedded into a framework covering chemosensory input and associative learning of odors. The resulting dynamic representation of odors in spatio-temporal activity patterns corresponds to response patterns observed in the generalistic olfactory systems. We discuss the meaning of symmetrical and asymmetrical connections and temporal coding for classical conditioning, and demonstrate, that non-converging activity patterns can be learned and discriminated. R. Malaka, M. Hammer (1996), Real-time models of classical conditioning. Submitted to the ICNN`96 conference, Washington Abstract: Real-time models of classical conditioning simulate features of associative learning including its dependence on the timing of stimuli. We present the Sutton/Barto model, the TD model, the CP model, the drive-reinforcement model, and the SOP model in a framework of reinforcement learning rules. The role of eligibility and reinforcement is analyzed and the ability of the models to simulate time-dependent learning (e.g. inhibitory backward conditioning) and other conditioning phenomena is compared. A new model is introduced, that is mathematically simple, and overcomes weaknesses of the other models. This model combines the two antagonistic US traces of the SOP model with the reinforcement term of the TD model. R. Malaka, T. Ragg, M. Hammer (1995) A Model for Chemosensory Reception, In G. Tesauro, D.S. Touretzky, T.K. Leen (eds), Advances in Neural Information Processing Systems, Vol. 7 Abstract: A new model for chemosensory reception is presented. It models reactions between odor molecules and receptor proteins and the activation of second messenger by receptor proteins. The mathematical formulation of the reaction kinetics is transformed into an artificial neural network (ANN). The resulting feed-forward network provides a powerful means for parameter fitting by applying learning algorithms. The weights of the network corresponding to chemical parameters can be trained by presenting experimental data. We demonstrate the simulation capabilities of the model with experimental data from honey bee chemosensory neurons. It can be shown that our model is sufficient to rebuild the observed data and that simpler models are not able to do this task. R. Malaka, U. Koelsch (1994) Pattern Segmentation in Recurrent Networks of Biologically Plausible Neural Elements. In Intelligent Engineering Systems Through Artificial Neural Networks, Vol. 4 Abstract: We introduce a neural network model using spiking neurons. The neuron model is a biological neuron with Hodgkin-Huxley channels. We compare the network's ability of auto-associative pattern recognition with to that of the Hopfield network. The model recognizes patterns by converging into dynamic stable states of synchonous firing activity. This activity can last for arbitrary time or return to a resting activation after stimulus offset. If one presents overlayed patterns to the network, the network is able to separate the components. The single components are encoded by synchronous firing patterns. and some others. Yours, Rainer Malaka ------------------------------------------------------------------------------ Rainer Malaka /| phone: (+49) (721) 608-4212 Universitaet Karlsruhe | | /| fax : (+49) (721) 608-4211 Institut fuer Logik, Komplexitaet /||/ | | csnet: malaka at ira.uka.de und Deduktionssysteme | | /||/ P.O.-Box 6980 |/ | | WWW : D-76128 Karlsruhe, Germany |/ http://i11www.ira.uka.de/~malaka/ ------------------------------------------------------------------------------ From marwan at sedal.usyd.edu.AU Thu Nov 2 09:10:08 1995 From: marwan at sedal.usyd.edu.AU (Marwan A. Jabri, Sydney Univ. Elec. Eng., Tel: +61-2 692 2240) Date: Fri, 3 Nov 1995 01:10:08 +1100 Subject: Girling Watson Research Fellowship Message-ID: <199511021410.BAA08045@sedal.sedal.su.OZ.AU> GIRLING WATSON RESEARCH FELLOWSHIP (Renewable) Reference No. B42/01 Systems Engineering & Design Automation Laboratory Department of Electrical Engineering The University of Sydney Applications are invited for a Girling Watson Research Fellowship at Sydney University Electrical Engineering. The Fellow will work with SEDAL (Systems Engineering and Design Automation Laboratory). SEDAL has research projects on optical character recognition, adiabatic computing, pattern recognition for implantable devices, VLSI implementation of connectionist architectures, intra-cardiac electrogram classification, software and hardware implementations for video-telephony, pattern recognition for cochlear implants, audio localisation and time series prediction. SEDAL collaborates with Australian and multinational companies in many of these research projects. The Fellow should have a PhD and an excellent research background in one of the following areas: Pattern recognition and analysis; connectionist architectures; analog and digital microelectronics; or time series modelling. The Fellow is expected to play a leading role in research, post graduate student supervision and at providing occasional teaching support in his/her area of expertise. The appointment is available for a period of three years. There is the possibility of further offers of employment for another three years, subject to funding and need. Membership of a University approved superannuation scheme is a condition of employment for new appointees. For further information contact Marwan Jabri tel (+61-2) 351 2240, fax 660 1228, Email: marwan at sedal.su.oz.au Salary: Research Fellow $42,198 - $50,111 p.a. Closing: 7 December 1995 Applications should be sent to the Personeel Services at the address below. Applications should quote the reference number B42/01, a CV describing research achievements and the names, addresses, teplephones, fax and email of three referees who can comment on research performance, Personnel Services Academic Group B J13 The University of Sydney NSW 2006 Australia From maja at cs.brandeis.edu Thu Nov 2 10:28:44 1995 From: maja at cs.brandeis.edu (Maja Mataric) Date: Thu, 2 Nov 1995 10:28:44 -0500 Subject: CFP - Adaptive Behavior Journal Message-ID: <199511021528.KAA13997@garnet.cs.brandeis.edu> CALL FOR PAPERS (http://www.cs.brandeis.edu:80/~maja/abj-special-issue/) ADAPTIVE BEHAVIOR Journal Special Issue on COMPLETE AGENT LEARNING IN COMPLEX ENVIRONMENTS Guest editors: Maja J Mataric Submission Deadline: June 1, 1996. Adaptive Behavior is an international journal published by MIT Press; Editor-in-Chief: Jean-Arcady Meyer, Ecole Normale Superieure, Paris. In the last decade, the problems being treated in AI, Alife, and Robotics have witnessed an increase in complexity as the domains under investigation have transitioned from theoretically clean scenarios to more complex dynamic environments. Agents that must adapt in environments such as the physical world, an active ecology or economy, and the World Wide Web, challenge traditional assumptions and approaches to learning. As a consequence, novel methods for automated adaptation, action selection, and new behavior acquisition have become the focus of much research in the field. This special issue of Adaptive Behavior will focus on situated agent learning in challenging environments that feature noise, uncertainty, and complex dynamics. We are soliciting papers describing finished work on autonomous learning and adaptation during the lifetime of a complete agent situated in a dynamic environment. We encourage submissions that address several of the following topics within a whole agent learning system: * learning from ambiguous perceptual inputs * learning with noisy/uncertain action/motor outputs * learning from sparse, irregular, inconsistent, and noisy reinforcement/feedback * learning in real time * combining built-in and learned knowledge * learning in complex environments requiring generalization in state representation * learning from incremental and delayed feedback * learning in smoothly or discontinuously changing environments We invite submissions from all areas in AI, Alife, and Robotics that treat either complete synthetic systems or models of biological adaptive systems situated in complex environments. Submitted papers should be delivered by June 1, 1996. Authors intending to submit a manuscript should contact the guest editor as soon as possible to discuss paper ideas and suitability for this issue. Use maja at cs.brandeis.edu or tel: (617) 736-2708 or fax: (617) 736-2741. Manuscripts should be typed or laser-printed in English (with American spelling preferred) and double-spaced. Both paper and electronic submission are possible, as described below. Copies of the complete Adaptive Behavior Instructions to Contributors are available on request--also see the Adaptive Behavior journal's home page at: http://www.ens.fr:80/bioinfo/www/francais/AB.html. For paper submissions, send five (5) copies of submitted papers (hard-copy only) to: Maja Mataric Volen Center for Complex Systems Computer Science Department Brandeis University Waltham, MA 02254-9110, USA For electronic submissions, use Postscript format, ftp the file to ftp.cs.brandeis.edu/incoming, and send an email notification to maja at cs.brandeis.edu. For a Web page of this call, and detailed ftp directions, see: http://www.cs.brandeis.edu/~maja/abj-special-issue/ \end{document} From szepes at sol.cc.u-szeged.hu Thu Nov 2 09:02:44 1995 From: szepes at sol.cc.u-szeged.hu (Csaba Szepesvari) Date: Thu, 2 Nov 1995 14:02:44 +0000 Subject: PA: Approximate Geometry Representations and Sensory Fusion Message-ID: <9511021303.AA22334@sol.cc.u-szeged.hu> *****************Pre-print Available via FTP ******************* URL ftp:// iserv.iki.kfki.hu/pub/papers/szepes.fusion.ps.Z WWW http://iserv.iki.kfki.hu/adaptlab.html Title: Approximate Geometry Representations and Sensory Fusion Keywords: self-organizing networks, sensory fusion, geometry representation, topographical mapping, Kohonen network Csaba Szepesvari^* Andras Lorincz Adaptive Systems Laboratory, Insitute of Isotopes, Hungarian Academny of Sciences *Bolyai Institute of Mathematics, Jozsef Attila University of Szeged Knowledge of the geometry of the world external to a system is essential in cases such as navigation when for predicting the trajectories of moving objects. It may also play a role in recognition tasks, particularly when the procedures used for image segmentation and feature extraction utilize information on the geometry. A more abstract example is function approximation, where this information is used to create better interpolation. This paper summarizes the recent advances in the theory of self-organizing development of approximate geometry representations based on the use of neural networks. Part of this work is based on the theoretical approach of (Szepesvari, 1993), which is different from that of (Martinetz, 1993) and also is somewhat more general. The Martinetz approach treats signals provided by artificial neuron-like entities whereas the present work uses the entities of the external world as its starting point. The relationship between the present work and the Martinetz approach will be detailed. We approach the problem of approximate geometry representations by first examining the problem of sensory fusion, i.e., the problem of fusing information from different transductors. A straightforward solution is the simultaneous discretization of the output of all transductors, which means the discretization of a space defined as the product of the individual transductor output spaces. However, the geometry relations are defined for the **external world** only, so it is still an open question how to define the metrics on the product of output spaces. It will be shown that **simple Hebbian learning** can result in the formation of a correct geometry representation. Some mathematical considerations will be presented to help us clarify the underlying concepts and assumptions. The mathematical framework gives rise to a corollary on the "topographical mappings" realized by Kohonen networks. In fact, the present work as well as (Martinetz, 1993) may be considered as a generalization of Kohonen's topographic maps. We develop topographic maps with self-organizing interneural connections. *****************Pre-print Available via FTP ******************* ========================================================================== Csaba Szepesvari ---------------------------------------+---------------------------------- Bolyai Institute of Mathematics |e-mail: szepes at math.u-szeged.hu "Jozsef Attila" University of Szeged |http://www.inf.u-szeged.hu/~szepes Szeged 6720 | Aradi vrt tere 1. | HUNGARY | Tel.: (36-62) 311-622/3706 |phone at home (tel/fax): Tel/Fax: (36-62) 326-246 | (36-62) 494-225 ---------------------------------------+---------------------------------- From dnoelle at cs.ucsd.edu Thu Nov 2 20:00:49 1995 From: dnoelle at cs.ucsd.edu (David Noelle) Date: Thu, 2 Nov 95 17:00:49 -0800 Subject: Cognitive Science '96 Message-ID: <9511030100.AA22084@beowulf> Eighteenth Annual Conference of the COGNITIVE SCIENCE SOCIETY July 12-15, 1996 University of California, San Diego La Jolla, California CALL FOR PAPERS DUE DATE: Thursday, February 1, 1996 The Annual Cognitive Science Conference began with the La Jolla Conference on Cognitive Science in August of 1979. The organizing committee of the Eighteenth Annual Conference would like to welcome members home to La Jolla. We plan to recapture the pioneering spirit of the original conference, extending our welcome to fields on the expanding frontier of Cognitive Science, including Artificial Life, Cognitive and Computational Neuroscience, Evolutionary Psychology, as well as the core areas of Anthropology, Computer Science, Linguistics, Neuroscience, Philosophy, and Psychology. As a change this year, we follow the example of Psychonomics and the Neuroscience Conferences and invite Members of the Society to submit one-page abstracts for guaranteed poster presentation at the conference. A second change is that all papers accepted as posters will only get one page in the proceedings. The conference will feature plenary addresses by invited speakers, invited symposia by leaders in their fields, technical paper and a poster sessions, a banquet, and a Blues Party. San Diego is the home of the world-famous San Diego Zoo and Wild Animal Park, Sea World, the historic all-wooden Hotel Del Coronado, beautiful beaches, mountain areas and deserts, is a short drive from Mexico, and features a high Cappuccino Index. Bring the whole family and stay a while! GUIDELINES FOR PAPER SUBMISSIONS Novel research papers are invited on any topic related to cognition. Members of the Society may submit a one page abstract for poster presentation, which will be automatically accepted for publication in the proceedings. Submitted full-length papers will be evaluated through peer review with respect to several criteria, including originality, quality, and significance of research, relevance to a broad audience of cognitive science researchers, and clarity of presentation. Papers will either be accepted for publication in the proceedings, or (if the author is a Society member) will be accepted as a poster, and a one-page abstract will be published. Such authors will get a chance to flesh out the abstract to a page when submitting their camera ready copy. Poster abstracts from non-members will be accepted, but the presenter should join the Society prior to presenting the poster. Accepted papers will be presented at the conference as talks. Papers may present results from completed research as well as report on current research with an emphasis on novel approaches, methods, ideas, and perspectives. Posters may report on recent work to be published elsewhere that has not been previously presented at the conference. Authors should submit five (5) copies of the paper in hard copy form by Thursday, February 1, 1996, to: Dr. Garrison W. Cottrell Computer Science and Engineering 0114 FED EX ONLY: 3250 Applied Physics and Math University of California San Diego La Jolla, Ca. 92093-0114 phone for FED EX: 619-534-5948 (my secretary, Marie Kreider) If confirmation of receipt is desired, please use certified mail or enclose a self-addressed stamped envelope or postcard. DAVID MARR MEMORIAL PRIZES FOR EXCELLENT STUDENT PAPERS Papers with a student first author are eligible to compete for a David Marr Memorial Prize for excellence in research and presentation. The David Marr Prizes are accompanied by a $300.00 honorarium, and are funded by an anonymous donor. LENGTH Papers must be a maximum of eleven (11) pages long (excluding only the cover page but including figures and references), with 1 inch margins on all sides (i.e., the text should be 6.5 inches by 9 inches, including footnotes but excluding page numbers), double-spaced, and in 12-point type. Each page should be numbered (excluding the cover page). Template and style files conforming to these specifications for several text formatting programs, including LaTeX, Framemaker, Word, Word Perfect, and HTML will be available by anonymous FTP after November 15th, 1995. (Check "http://www.cse.ucsd.edu/events/cogsci96/" for details). Submitted abstracts should be one page in length, with the same margins as full papers. Style files for these will be available at the same location as above. Final versions of papers and poster abstracts will be required only after authors are notified of acceptance; accepted papers will be published in a CD-ROM version of the proceedings. Abstracts will be available before the meeting from a WWW server. Final versions must follow the HTML style guidelines referred to above. This year we will continue to publish the proceedings in two modalities, paper and a CD-ROM version. When the procedures for efficient HTML submission stabilize, we will be switching from paper to CD-ROM publication in order to control escalating costs and permit use of search software. [Comments on this change should be directed to "alan at lrdc4.lrdc.pitt.edu" (Alan Lesgold, Secretary/Treasurer).] COVER PAGE Each copy of the submitted paper must include a cover page, separate from the body of the paper, which includes: 1. Title of paper. 2. Full names, postal addresses, phone numbers, and e-mail addresses of all authors. 3. An abstract of no more than 200 words. 4. Three to five keywords in decreasing order of relevance. The keywords will be used in the index for the proceedings. 5. Preference for presentation format: Talk or poster, talk only, poster only. Poster only submissions should follow paper format, but be no more than 2 pages in this format (final poster abstracts will follow the same 2 column format as papers). Accepted papers will be presented as talks. Submitted posters by Society Members will be accepted for poster presentation, but may, at the discretion of the Program Committee, be invited for oral presentation. Non-members may join the Society at the time of submission. 6. A note stating if the paper is eligible to compete for a Marr Prize. DEADLINE Papers must be received by Thursday, February 1, 1996. Papers received after this date will be recycled. CALL FOR SYMPOSIA In addition to technical papers, posters, plenary sessions, and invited symposia, the conference will accept submitted research symposia. Proposals for symposia are invited and should indicate: 1. A brief description of the topic; 2. How the symposium would address a broad cognitive science audience, and some evidence of interest; 3. Names of symposium organizer(s); 4. List of potential speakers, their topics, and some estimate of their likelihood of participation; 5. Proposed symposium format (designed to last 90 minutes). Symposium proposals should be sent as soon as possible, but no later than January 1, 1996. Abstracts of the symposium talks will be included in the proceedings and must be made available in HTML format, as above. CONFERENCE CHAIRS Edwin Hutchins and Walter Savitch hutchins at cogsci.ucsd.edu savitch at cogsci.ucsd.edu PROGRAM CHAIR Garrison W. Cottrell gary at cs.ucsd.edu From steven.young at psy.ox.ac.uk Fri Nov 3 09:40:37 1995 From: steven.young at psy.ox.ac.uk (Steven Young) Date: Fri, 3 Nov 1995 14:40:37 +0000 (GMT) Subject: Oxford Summer School on Connectionist Modelling Message-ID: <199511031440.OAA04054@cogsci2.psych.ox.ac.uk> The call for participation for the 1996 Oxford Summer School on Connectionist Modelling follows. Please pass on this information to people you know you would be interested. -------- OXFORD SUMMER SCHOOL ON CONNECTIONIST MODELLING Department of Experimental Psychology University of Oxford 21 July - 2nd August 1996 Applications are invited for participation in a 2-week residential Summer School on techniques in connectionist modelling. The course is aimed primarily at researchers who wish to exploit neural network models in their teaching and/or research and it will provide a general introduction to connectionist modelling through lectures and exercises on Power PCs. The course is interdisciplinary in content though many of the illustrative examples are taken from cognitive and developmental psychology, and cognitive neuroscience. The instructors with primary responsibility for teaching the course are Kim Plunkett and Edmund Rolls. No prior knowledge of computational modelling will be required though simple word processing skills will be assumed. Participants will be encouraged to start work on their own modelling projects during the Summer School. The cost of participation in the Summer School is #750 to include accommodation (bed and breakfast at St. John's College) and registration. Participants will be expected to cover their own travel and meal costs. A small number of graduate student scholarships providing partial funding may be available. Applicants should indicate whether they wish to be considered for a graduate student scholarship but are advised to seek their own funding as well, since in previous years the number of graduate student applications has far exceeded the number of scholarships available. There is a Summer School World Wide Web page describing the contents of the 1995 Summer School available on: http://cogsci1.psych.ox.ac.uk/summer-school/ Further information about contents of the course can be obtained from Steven.Young at psy.ox.ac.uk If you are interested in participating in the Summer School, please contact: Mrs Sue King Department of Experimental Psychology University of Oxford South Parks Road Oxford OX1 3UD Tel: (01865) 271353 Email: susan.king at psy.oxford.ac.uk Please send a brief description of your background with an explanation of why you would like to attend the Summer School (one page maximum) no later than 31st January 1996. Regards, Steven Young. -- Facility for Computational Modelling in Cognitive Science McDonnell-Pew Centre for Cognitive Neuroscience, Oxford From Dave_Touretzky at DST.BOLTZ.CS.CMU.EDU Sat Nov 4 05:06:55 1995 From: Dave_Touretzky at DST.BOLTZ.CS.CMU.EDU (Dave_Touretzky@DST.BOLTZ.CS.CMU.EDU) Date: Sat, 04 Nov 95 05:06:55 EST Subject: Neural Processes in Cognition training program Message-ID: <9295.815479615@DST.BOLTZ.CS.CMU.EDU> Applications are being accepted for both pre- and postdoctoral training in Neural Processes in Cognition, a joint program of the Center for the Neural Basis of Cognition operated by the University of Pittsburgh, its School of Medicine, and Carnegie Mellon University. This is an interdisciplinary program investigating the neurobiology of cognition, and utilizing neuroanatomical, neurophysiological, behavioral, and computer simulation techniques. Some of the departments offering training as part of the Neural Processes in Cognition program are: CMU: Psychology, Computer Science, Biological Sciences, Robotics University of Pittsburgh: Psychology, Neuroscience, Neurobiology, Mathematics, Information Science, Radiology, Neurology Research facilities include: computerized microscopy, human and animal electrophysiological instrumentation, behavioral assessment laboratories, brain imaging, the Pittsburgh Supercomputing Center, and access to human clinical populations. The application deadline is February 1, 1996. Additional details about the Center for the Neural Basis of Cognition, the Neural Processes in Cognition program, and how to apply to the program can be found on the World-Wide Web: http://neurocog.lrdc.pitt.edu/npc/ http://www.cs.cmu.edu/Web/Groups/CNBC/CNBC.html For more information contact: Professor Walter Schneider University of Pittsburgh 3939 O'Hara Street Pittsburgh, PA 15260 tel. 412-624-7061 E-mail: NEUROCOG at PITTVMS.BITNET From b-wah at uiuc.edu Sun Nov 5 15:03:58 1995 From: b-wah at uiuc.edu (Benjamin Wah) Date: Sun, 05 Nov 95 14:03:58 CST Subject: ICNN'96 Extension of Submission Deadline Message-ID: <199511052003.AA06874@teacher.crhc.uiuc.edu> International Conference on Neural Networks IMPORTANT EXTENSION Deadline for submission to ICNN'96 has been extended to December 29, 1995 Authors interested to submit papers must have their papers received by the Program Chair (see address to the left) by the deadline. Papers received after that date will be returned unopened. Papers submitted must be in final publishable form and will be reviewed by senior researchers in the field using the same standard as papers submitted before the October 16, 1995, deadline. However, papers submitted after the October 16, 1995, deadline will be either accepted or rejected, and authors of accepted papers will not have a chance to revise their papers. (Authors of accepted papers submitted before the October 16 deadline will be allowed to revise their papers.) Authors submitting papers late will be notified of the final decision by February 15, 1996. Six copies (one original and five copies) of the paper must be submitted. Papers must be camera-ready on 8 1/2-by-11 white paper, one-column format in Times or similar font style, 10 points or larger with one inch margins on all four sides. Do not fold or staple the original camera-ready copy. Four pages are encouraged; however, the paper must not exceed six pages, including figures, tables, and refer- ences, and should be written in English. Submissions that do not adhere to the guidelines above will be returned unre- viewed. Centered at the top of the first page should be the complete title, author name(s) and postal and electronic mailing addresses. In the accompanying letter, the following infor- mation must be included: (a) full title of paper; (b) pre- sentation preferred (oral or poster); (c) audio visual requirements (e.g. 35 mm slide, OHP, VCR); (d) corresponding author (name, postal and e-mail addresses, telephone & fax numbers); and (e) presenter (name, postal and e-mail addresses, telephone & fax numbers). For further information on ICNN'96, please consult our World Wide Web home page at http://www-ece.rice.edu/96icnn, or send electronic mail to icnn96 at manip.crhc.uiuc.edu. From pau at ac.upc.es Mon Nov 6 11:03:33 1995 From: pau at ac.upc.es (Pau Bofill) Date: Mon, 6 Nov 1995 16:03:33 +0000 (METDST) Subject: No Free Lunch? (Delayed reply) Message-ID: <9511061603.AA05736@gaudi.ac.upc.es> (I've just been added to the Connectionists list, and therefore I apologize if I'm repeating something that's been said before.) I happened to read the message from Bill Macready on Free Lunch Theorems for Search, and I was surprised by his (and D.H. Wolpert's) statement that "any two search algorithms have exactly the same expected performance, over the space of all fitness functions." which seemed to invalidate statements like "my search algorithm beats your search algorithm according to a particular performance measure for the following fitness function." Thus, I picked the papers anounced on that message (see below) and tryied to find out what did they mean. As far as I can see, the goal of a particular optimization PROBLEM, is to find a target POINT in search space whith specific properties. The role of any fitness function, then, is to assign a cost value to each point in search space in such a way that it helps the algorithm beeing used to find the target point. In particular, for optimization purposes, a necessary condition for the validity of a fitness function is that it assigns the maximum (or minimum) cost value to all valid target points, and only to target points. If I understood them properly, Wolpert and Macready define the space of all possible fitness functions as the set of ALL possible assignments of cost values to points in search space. And they use as a generalized perfomace measure the histogram of cost values, regardless of the points where they were found. Then, if what I understood is right, they are ignoring the previous necessary condition on fitness functions and their statement is equivalent to, "any two search algorithms have exactly the same expected performance, over the space of all optimization problems that can be defined whithin a search space." which stated otherwise would mean, "If the target doesn't matter, all algorithms perform alike." I don't doubt that Wolpert & Macready's "No Free Lunch Theorem" can be useful as a tool for deriving further results, but one should be carefull when considering its "physical" meaning. Thus, I believe it is very important to define carefully the meaning of, "My algorithm performs better than yours." In particular, algorithms should be compared on PROBLEMS, not on fitness functions, with properly defined performance measures. Probably, the most fair one-to-one comparision would use the best (found) fitness function for each algorithm (finding the best fitness function for a particular problem AND algorithm is, in turn, an optimization problem). Averaging over algorithms in order to measure problem hardness is "dangerous" in the same sense. Pau Bofill Universitat Politecnica de Catalunya. ************************************************************************** The papers mentioned are at ftp.santafe.edu /pub/wgm/nfl.ps /hard.ps From esann at dice.ucl.ac.be Mon Nov 6 13:52:37 1995 From: esann at dice.ucl.ac.be (esann@dice.ucl.ac.be) Date: Mon, 6 Nov 1995 19:52:37 +0100 Subject: ESANN'96 Annoucement and Call for Papers Message-ID: <199511061850.TAA10706@ns1.dice.ucl.ac.be> Dear Colleagues, You will find below the Announcement and Call for Papers of ESANN'96, the fourth European Symposium on Artificial Neural Networks, which will be held in Bruges (Belgium), on April 24-26, 1996. All information concerning this conference is also available on the following servers: - WWW: http://www.dice.ucl.ac.be/neural-nets/ESANN/ESANN'96.html - FTP: server: ftp.dice.ucl.ac.be directory: /pub/neural-nets/ESANN login: anonymous password: your e-mail address Please don't hesitate to connect to these servers, and to contact the conference secretariat (see address below) for any supplementary information. Looking forward to meeting you during ESANN'96, Sincerely yours, Michel Verleysen _____________________________________________________________________ ******************************************************** * ESANN96 * * * * 4th European Symposium on Artificial Neural Networks * * Bruges - April 24-25-26, 1996 * * * ******************************************************** First announcement and call for papers ______________________________________ Invitation and Scope of the conference ______________________________________ The fourth European Symposium on Artificial Neural Networks will be organized in Bruges, Belgium, in April 1996. The three first successful editions, organized in Brussels, each gathered between 90 and 110 scientists, coming from Western and Eastern Europe, but also from USA, Japan, Australia, New Zealand, South America... The field of Artificial Neural Networks includes a lot of different disciplines, from mathematics and statistics to robotics and electronics. For this reason, actual studies concern various aspects of the field, sometimes to the detriment of strong, well established foundations for these researches; it is obvious that a better knowledge of the basic concepts and opportunities of neurocomputing, and more effective comparisons with other computing methods are strongly needed for a profitable long-term use of neural networks in applications. The purpose of the ESANN conferences is to present the latest results in the fundamental aspects of artificial neural networks. Invited and survey talks will also present a comprehensive view of particular topics of the conference. The program committee of ESANN'96 welcomes papers covering new results or being of tutorial nature, and dealing with theoretical, biological or mathematical aspects of artificial neural networks, or with the relations between neural networks and other fields. Presentation of results from large research project (ESPRIT,...) is also encouraged. The following is a non-exhaustive list of topics which will be covered during ESANN'96 : + theory + models and architectures + mathematics + learning algorithms + biologically plausible artificial networks + formal models of biological phenomena + neurobiological systems + approximation of functions + identification of non-linear dynamic systems + adaptive behavior + adaptive control + signal processing + statistics + self-organization + evolutive learning Invited talks will cover several of the above topics; an invited talk will be given by Prof. Nicolas Franceschini (CNRS Marseilles, France). Other invited talks are to be announced. The conference will be held in Bruges (also called "Venice of the North"), one of the most beautiful towns in Europe. Bruges can be reached by train from Brussels in less than one hour (frequent trains). The town of Bruges is worldwide known, and famous both by its architectural style, its canals, and its pleasant atmosphere. Call for contributions ______________________ Prospective authors are invited to submit six originals of their contribution before December 8, 1995. Working language of the conference (including proceedings) is English. Papers should not exceed six A4 pages (including figures and references). Printing area will be 12.2 x 19.3 cm (centered on the A4 page); left, right, top and bottom margins will thus respectively be 4.4, 4.4, 5.2 and 5.2 cm. 10-point Times font will be used for the main text; headings will be in bold characters (but not underlined), and will be separated from the main text by two blank lines before and one after. Manuscripts prepared in this format will be reproduced in the same size in the book. The first page will be typed in the format indicated in the figure file ESANN_format.eps that can be found on the anonymous FTP ESANN'96 server (see below). The next pages will be similar, except the heading which will be omitted. Originals of the figures will be pasted into the manuscript and centered between the margins. The lettering of the figures should be in 10-point Times font size. Figures should be numbered. The legends also should be centered between the margins and be written in 9-point Times font size as follows: Fig. 3. Text follows ... The pages of the manuscript will not be numbered (numbering decided by the editor). A separate page (not included in the manuscript) will indicate: + the title of the manuscript + author(s) name(s) + the complete address (including phone & fax numbers and E-mail) of the corresponding author + a list of five keywords or topics On the same page, the authors will copy and sign the following paragraph: "in case of acceptation of the paper for presentation at ESANN 96: - at least one of the authors will register to the conference and will present the paper - the author(s) give their rights up over the paper to the organizers of ESANN 96, for the proceedings and any publication that could directly be generated by the conference - if the paper does not match the format requirements for the proceedings, the author(s) will send a revised version within two weeks of the notification of acceptation." Contributions must be sent to the conference secretariat. Examples of camera-ready contributions can be obtained by writing to the same address. Registration fees _________________ registration before registration after February 1st, 1996 February 1st, 1996 Universities BEF 15500 BEF 16500 Industries BEF 19500 BEF 20500 An "advanced registration form" is available by writing to the conference secretariat (see reply form below). Please ask for this form in order to benefit from the reduced registration fee before February 1st, 1996. Deadlines _________ Submission of papers December 8, 1995 Notification of acceptance January 31, 1996 Symposium April 24-261, 1996 Grants ______ We regret that no grants are available this year, because of a lack of funding from the European Community. Conference secretariat ______________________ Dr. Michel Verleysen D facto conference services 45 rue Masui B - 1210 Brussels (Belgium) phone: + 32 2 245 43 63 Fax: + 32 2 245 46 94 E-mail: esann at dice.ucl.ac.be Information is also available through WWW and anonymous FTP on the following sites: URL: http://www.dice.ucl.ac.be/neural-nets/ESANN/ESANN96.html FTP: ftp.dice.ucl.ac.be, directory /pub/neural-nets/ESANN Steering committee __________________ Francois Blayo - Univ. Paris I (F) Marie Cottrell - Univ. Paris I (F) Jeanny Herault - INPG Grenoble (F) Joos Vandewalle - KUL Leuven (B) Michel Verleysen - UCL Louvain-la-Neuve (B) Scientific committee ____________________ (to be confirmed) Agnes Babloyantz - Univ. Libre Bruxelles (Belgium) Herve Bourlard - ICSI Berkeley (USA) Joan Cabestany - Univ. Polit. de Catalunya (E) Dave Cliff - University of Sussex (UK) Pierre Comon - Thomson-Sintra Sophia Antipolis (F) Holk Cruse - Universitat Bielefeld (D) Dante Del Corso - Politecnico di Torino (I) Wlodek Duch - Nicholas Copernicus Univ. (PL) Marc Duranton - Philips / LEP (F) Jean-Claude Fort - Universite Nancy I (F) Bernd Fritzke - Ruhr-Universitat Bochum (D) Karl Goser - Universitat Dortmund (D) Manuel Grana - UPV San Sebastian (E) Martin Hasler - EPFL Lausanne (CH) Kurt Hornik - Techische Univ. Wien (A) Christian Jutten - INPG Grenoble (F) Vera Kurkova - Acad. of Science of the Czech Rep. (CZ) Petr Lansky - Acad. of Science of the Czech Rep. (CZ) Jean-Didier Legat - UCL Louvain-la-Neuve (B) Hans-Peter Mallot - Max-Planck Institut (D) Eddy Mayoraz - IDIAP Martigny (CH) Jean Arcady Meyer - Ecole Normale Superieure Paris (F) Jose Mira-Mira - UNED (E) Pietro Morasso - Univ. of Genoa (I) Jean-Pierre Nadal - Ecole Normale Superieure Paris (F) Erkki Oja - Helsinky University of Technology (FIN) Gilles Pages - Universite Paris VI (F) Helene Paugam-Moisy - Ecole Normale Superieure Lyon (F) Alberto Prieto - Universitad de Granada (E) Pierre Puget - LETI Grenoble (F) Ronan Reilly - University College Dublin (IRE) Tamas Roska - Hungarian Academy of Science (H) Jean-Pierre Rospars - INRA Versailles (F) Jean-Pierre Royet - Universite Lyon 1 (F) John Stonham - Brunel University (UK) John Taylor - King's College London (UK) Vincent Torre - Universita di Genova (I) Claude Touzet - IUSPIM Marseilles (F) Marc Van Hulle - KUL Leuven (B) Christian Wellekens - Eurecom Sophia-Antipolis (F) Reply form __________ If you wish to receive the final program of ESANN'96, for any address change, or to add one of your colleagues in our database, please send this form to the conference secretariat: D facto conference services 45 rue Masui B - 1210 Brussels (Belgium) phone: + 32 2 245 43 63 Fax: + 32 2 245 46 94 E-mail: esann at dice.ucl.ac.be Please indicate if you wish to receive the advanced registration form. ---------------------------------------------------------------- Name: .......................................................... First Name: .................................................... University or Company: ......................................... Address: ....................................................... ZIP: ................... Town: ................................ Country: ....................................................... Tel: ........................................................... Fax: ........................................................... E-mail: ........................................................ O Please send me the "advanced registration form" ---------------------------------------------------------------- _____________________________ D facto publications - conference services 45 rue Masui 1210 Brussels Belgium tel: +32 2 245 43 63 fax: +32 2 245 46 94 _____________________________ From jacobs at psych.stanford.edu Tue Nov 7 12:58:05 1995 From: jacobs at psych.stanford.edu (Robert Jacobs) Date: Tue, 7 Nov 1995 09:58:05 -0800 (PST) Subject: paper available: Gibbs sampling for HME Message-ID: <199511071758.JAA12308@aragorn.Stanford.EDU> The following paper is available via anonymous ftp from the neuroprose archive. The paper has been accepted for publication in the "Journal of the American Statistical Association." The manuscript is 26 pages. (Unfortunately, hardcopies are not available.) FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/jacobs.hme_gibbs.ps.Z Bayesian Inference in Mixtures-of-Experts and Hierarchical Mixtures-of-Experts Models With an Application to Speech Recognition Fengchun Peng, Robert A. Jacobs, and Martin A. Tanner Machine classification of acoustic waveforms as speech events is often difficult due to context-dependencies. A vowel recognition task with multiple speakers is studied in this paper via the use of a class of modular and hierarchical systems referred to as mixtures-of-experts and hierarchical mixtures-of-experts models. The statistical model underlying the systems is a mixture model in which both the mixture coefficients and the mixture components are generalized linear models. A full Bayesian approach is used as a basis of inference and prediction. Computations are performed using Markov chain Monte Carlo methods. A key benefit of this approach is the ability to obtain a sample from the posterior distribution of any functional of the parameters of the given model. In this way, more information is obtained than provided by a point estimate. Also avoided is the need to rely on a normal approximation to the posterior as the basis of inference. This is particularly important in cases where the posterior is skewed or multimodal. Comparisons between a hierarchical mixtures-of-experts model and other pattern classification systems on the vowel recognition task are reported. The results indicate that this model showed good classification performance, and also gave the additional benefit of providing for the opportunity to assess the degree of certainty of the model in its classification predictions. From esann at dice.ucl.ac.be Tue Nov 7 13:24:14 1995 From: esann at dice.ucl.ac.be (esann@dice.ucl.ac.be) Date: Tue, 7 Nov 1995 19:24:14 +0100 Subject: mistake in ESANN'96 address Message-ID: <199511071822.TAA05245@ns1.dice.ucl.ac.be> Dear Colleagues, This message is to bring to your attention that the WWW server address included in the message announcing the ESANN'96 conference was wrong. The ESANN'96 conference (European Symposium on Artificial Neural Networks) will be held in Bruges (Belgium) on April 24-25-26, 1996. All information can be obtained through FTP or WWW, at the following addresses: - WWW: http://www.dice.ucl.ac.be/neural-nets/ESANN/ESANN96.html - FTP: server: ftp.dice.ucl.ac.be directory: /pub/neural-nets/ESANN login: anonymous password: your e-mail address The WWW address in the previous message: - WWW: http://www.dice.ucl.ac.be/neural-nets/ESANN/ESANN'96.html was WRONG !!! Sorry for the inconvenience. Sincerely yours, Michel Verleysen _____________________________ D facto publications - conference services 45 rue Masui 1210 Brussels Belgium tel: +32 2 245 43 63 fax: +32 2 245 46 94 _____________________________ From pjs at aig.jpl.nasa.gov Tue Nov 7 12:46:39 1995 From: pjs at aig.jpl.nasa.gov (Padhraic J. Smyth) Date: Tue, 7 Nov 95 09:46:39 PST Subject: AISTATS-97: Preliminary Announcement Message-ID: <9511071746.AA00528@amorgos.jpl.nasa.gov> Preliminary Announcement Sixth International Workshop on Artificial Intelligence and Statistics (AISTATS-97) January 4-7, 1997 Ft. Lauderdale, Florida This is the sixth in a series of workshops which has brought together researchers in Artificial Intelligence (AI) and in Statistics to discuss problems of mutual interest. The exchange has broadened research in both fields and has strongly encouraged interdisciplinary work. Papers on all aspects of the interface between AI & Statistics are encouraged. For more details consult the AISTATS-97 Web page at: http://www.stat.washington.edu/aistats97/ A full Call for Papers will be released in early 1996. The paper submission deadline will be July 1st 1996. The workshop is organized under the auspices of the Society for Artificial Intelligence and Statistics. Program Chair: David Madigan, University of Washington General Chair: Padhraic Smyth, JPL and UCI From piuri at elet.polimi.it Wed Nov 8 16:01:20 1995 From: piuri at elet.polimi.it (Vincenzo Piuri) Date: Wed, 8 Nov 1995 21:01:20 GMT Subject: call for papers Message-ID: <9511082101.AA06232@ipmel2.elet.polimi.it> ========================================================= ETIM'96 1996 International Workshop on Emergent Technologies for Instrumentation and Measurements Como, Italy - 10-11 June 1996 ========================================================= Organized by the IEEE Instrumentation and Measurement Society (Technical Committee on Emergent Technologies) CALL FOR PAPERS This workshop is directed to create a unique synergetic discussion forum on the emergent technologies and a strong link between the theoretical researchers and the practitioners in the application fields related to instrumentation and measurements. The two-days single-session schedule will provide the ideal environment for in-depth analysis and discussions concerning the theoretical aspects of the applications and the use of new technologies in the practice. Researchers and practitioners are invited to submit papers concerning theoretical foundations, experimental results, or practical applications related to the use of advanced technologies for instrumentation and measurements. Papers are sollicited on, but not limited to, the following topics: neural networks, fuzzy logic, genetic algorithms, virtual instruments, optical technologies, laser, advanced digital signal/image processing, advanced analog signal processing, wavelets, sensor technologies, remote sensing, distributed systems, fault tolerance, adaptive systems. Interested authors should submit an extended summary or the full paper (limited to 20 double-spaced pages including figures and tables) to the program chair by January 15, 1996 (PostScript email or readable fax submissions are strongly encouraged). Submissions should contain: the corresponding author, affiliation, complete address, possible fax and email addresses. Submission implies the willingness of at least one of the authors to register and attend at the workshop and to present the paper. The corresponding author will be notified by February 16, 1996. The camera- ready version is limited to 10 one-column IEEE-book- standard pages and is due by May 1, 1996. The workshop will be held at the "A. Volta" Research Center - Villa Olmo, in Como, Italy. It is a two-hundred years-old villa in the pleasant scenario of one of the most attractive lakes around the nothern Italy, near Milan. Easy and frequent connections by train and airplane are available from Milan and all the main cities in Europe; flights from US and Asia arrive to the Malpensa international airport, connected by bus to Milan. The registration fee will be 120 US$, including lunches, coffe breaks and one copy of the proceedings. Hotel reservation will be managed directly by the Research Center to provide highly discounted rates. Program Chair prof. Vincenzo Piuri Dept of Electronics and Information Politecnico di Milano piazza L. da Vinci 32 I-20133 Milano, Italy phone +39-2-2399-3606 fax +39-2-2399-3411 e-mail piuri at elet.polimi.it American Co-Chair prof. Emil Petriu Dept. of Electrical Engineering University of Ottawa Ottawa, Ontario, Canada K1N 6N5 phone +1-613-564-2497 fax +1-613-564-6882 email petriu at trix.genie.uottawa.ca Asian Co-Chair prof. Kenzo Watanabe Research Inst. of Electronics Shizuoka University 3-5-1 Johoku, Hamamatsu 432, Japan phone +81-534-71-1171-573 fax +81-534-74-0630 email watanabe-k at rie.shizuoka.ac.jp Workshop Secretariat Ms. Laura Caldirola Politecnico di Milano phone +39-2-2399-3623 fax +39-2-2399-3411 caldirol at elet.polimi.it ========================================================= From simonpe at aisb.ed.ac.uk Wed Nov 8 07:32:38 1995 From: simonpe at aisb.ed.ac.uk (simonpe@aisb.ed.ac.uk) Date: Wed, 8 Nov 1995 12:32:38 +0000 Subject: No Free Lunch? (Delayed reply) In-Reply-To: <9511061603.AA05736@gaudi.ac.upc.es> References: <9511061603.AA05736@gaudi.ac.upc.es> Message-ID: <1520.9511081232@twain.aisb.ed.ac.uk> Pau Bofill writes: >Theorems for Search, and I was surprised >by his (and D.H. Wolpert's) statement that > > "any two search algorithms have exactly the same expected > performance, over the space of all fitness functions." > >which seemed to invalidate statements like > > "my search algorithm beats your search algorithm according > to a particular performance measure for the following fitness > function." Thess statements aren't contradictory (as you point out in your message). Averaged over the space of all possible optimization functions then sure, NFL says that all optimization algorithms are equal. But of course for any _particular_ optimization problem that you might be interested in, some algorithms are defintely much better than others. I think the importance of the NFL theorem is that it destroys the idea that there's such a thing as a `super-powerful all-purpose general learning algorithm' that can optimize any problem quickly. Instead, if we want a learning algorithm to work well, we have to analyze the particular problems we're interested in and tailor the learning algorithm to suit. Essentially we have to find ways of building prior domain knowledge about a problem into learning algorithms to solve that problem effectively. On the other hand, some people claim that there _is_ a general sub-class of problems for which it's at all feasible to find a solution - perhaps problems whose solutions have low komologrov complexity or something - and this might well mean that there _is_ a general good learning algorithm for this class of `interesting problems'. Any comments? << Simon Perkins >> Dept of AI, Edinburgh University S.Perkins at ed.ac.uk http://www.dai.ed.ac.uk/students/simonpe/ Tel: +44 131 650 3084 From zhuh at helios.aston.ac.uk Wed Nov 8 07:07:49 1995 From: zhuh at helios.aston.ac.uk (zhuh) Date: Wed, 8 Nov 1995 12:07:49 +0000 Subject: No Free Lunch? (Delayed reply) Message-ID: <321.9511081207@sun.aston.ac.uk> Pau Bofill paraphrases Wolpert, et al.'s paper "No Free Lunch..." [1] as > "any two search algorithms have exactly the same expected > performance, over the space of all optimization problems > that can be defined whithin a search space." This captures the trivial part of the argument. However, Wolpert et. al. also show that you can never cook up a method which will be good for an arbitrary non-uniform prior, because for every prior with which it works better than it would on a uniform prior, there is another with which it is worse by the same amount. This has important implications, given that neural net researchers are not in the habit of being precise about the distribution of optimisation problems their algorithms are directed at. This is an issue worth raising, and there is a principled way of dealing with it. The essential point is that without specifying the prior P(f) over the space of target distributions that a learning algorithm is meant to cope with, there is no objective way of comparing one algorithm with another. If you want to claim that good performance on a selection of problems from some domain implies that good performance should be expected on other problems from that domain, then you need to know enough about the prior distribution of problems to claim that your test suite forms a representative sample of it. Scientifically verifiable claims about a learning algorithm should specify at least the following three things: 1. the prior over target distributions. 2. the loss function. 3. the model. The first two are combined in Wolpert's work as the "prior on the fitness function". Conversely, it is shown [2,3,4] that 1. Given the first specification, there is always a uniquely defined "ideal estimator". 2. If the loss function is chosen as the information divergence [5] then the ideal estimate keeps all the information in the prior and the training data. Any learning algorithm must be a function of these ideal estimators. 3. If the purpose of a learning algorithm is to keep information, then it must approximate the ideal estimate. 4. For any computational model, the optimal estimate within the model is always an appropriate projection of the ideal estimate onto the model. In essence, the endavour to design better learning rules is fundamentally identical to the activity of finding good priors for application problems. Furthermore, the usual argument that in practice the prior is something vague and cannot be quantified is not substantiated. It is shown [6] that even in the case of training on data sets retrieved by ftp from the Internet, with little or no description of the problems, a reasonably good prior is still available. REFERENCES: [1] Wolpert, D. H. and Macready W. G.: No Free Lunch Theorems for Search, Santa Fe Inst. SFI-TR-95-02-010 ftp://ftp.santafe.edu/pub/wgm/nfl.ps [2] Zhu, H. and Rohwer, R.: Bayesian Invariant Measurements of Generalisation, 1995 ftp://cs.aston.ac.uk/neural/zhuh/letter.ps.Z [3] Zhu, H. and Rohwer, R.: Measurements of Generalisation Based on Information, MANNA conference, Oxford, 1995, (to appear in Ann. Math. Artif. Intell.) ftp://cs.aston.ac.uk/neural/zhuh/generalisation-manna.ps.Z [4] Zhu, H. and Rohwer, R.: A Bayesian Geometric Theory of Statistical Inference, 1995 ftp://cs.aston.ac.uk/neural/zhuh/stat.ps.Z [5] Amari, S.: Differential-Geometrical Methods in Statistics, Springer-Verlag, 1985. [6] Zhu, H. and Rohwer, R.: Bayesian regression filters and the issue of priors, 1995 ftp://cs.aston.ac.uk/neural/zhuh/reg_fil_prior.ps.Z -- Dr. Huaiyu Zhu zhuh at aston.ac.uk Neural Computing Research Group Dept of Computer Sciences and Applied Mathematics Aston University, Birmingham B4 7ET, UK From rgoldsto at bronze.ucs.indiana.edu Wed Nov 8 19:03:23 1995 From: rgoldsto at bronze.ucs.indiana.edu (rgoldsto@bronze.ucs.indiana.edu) Date: Wed, 8 Nov 1995 19:03:23 -0500 (EST) Subject: Faculty Position at Indiana University Message-ID: <199511090003.TAA23088@roatan.ucs.indiana.edu> INDIANA UNIVERSITY-BLOOMINGTON COGNITIVE SCIENCE POSITION The Cognitive Science Program and the Computer Science Department at Indiana University-Bloomington seek applicants for a joint faculty position, with rank open. Start date may be as early as Fall 1996, pending funding approval. We are looking for outstanding researchers at the forefront of the field, with ability to contribute to both Cognitive Science and Computer Science. Research area is open, including, for instance, neural net modeling, logic, reasoning, representation and information, language and discourse, robotics, computational vision and speech, visual inference, machine learning, and human-computer interaction. Applications from women and minority members are specifically encouraged. Indiana University is an Affirmative Action/Equal Opportunity Employer. The prospective faculty's office and laboratory will be based in the Computer Science department, which occupies a recently renovated spacious limestone building, and has extensive state-of-the-art computing facilities. Responsibilities will be shared with the Cognitive Science Program, one of the largest and most esteemed programs in the world today. The attractive wooded campus of Indiana University is located in Bloomington, voted one of the most cultural and livable small cities in the US, and a mere 45 minute drive from the Indianapolis airport. To be given full consideration applications must be received by February 15, 1996. The application should contain a detailed CV, copies of recent publications, brief statement of interests and future directions, and either three letters of recommendations, or a list of three references. Two copies of the application file must be sent (letters of reference may be sent to either address). Cognitive Science Search Cognitive Science Search Computer Science Department Cognitive Science Program Indiana University Psychology Department Bloomington, IN 47405 Indiana University Bloomington, IN 47405 Internet: cogsci-search at cs.indiana.edu or iucogsci at indiana.edu ____________________________ Rob Goldstone Department of Psychology/Program in Cognitive Science Indiana University Bloomington, IN. 47405 rgoldsto at indiana.edu Web site: http://cognitrn.psych.indiana.edu/ From dhw at santafe.edu Wed Nov 8 20:26:57 1995 From: dhw at santafe.edu (dhw@santafe.edu) Date: Wed, 8 Nov 95 18:26:57 MST Subject: No subject Message-ID: <9511090126.AA20987@santafe> Pau Bofill writes >>> I happened to read the message from Bill Macready on Free Lunch Theorems for Search, and I was surprised by his (and D.H. Wolpert's) statement that "any two search algorithms have exactly the same expected performance, over the space of all fitness functions." which seemed to invalidate statements like "my search algorithm beats your search algorithm according to a particular performance measure for the following fitness function." >>> We don't see how the two statements could be interpreted as contradictory. The first one explicitly is probabilistic (note the term "expected") whereas the second one is not. They explicitly refer to different things. >>> As far as I can see, the goal of a particular optimization PROBLEM, is to find a target POINT in search space whith specific properties. The role of any fitness function, then, is to assign a cost value to each point in search space in such a way that it helps the algorithm beeing used to find the target point. >>> Not at all! In the traveling salesperson problem, for example, the goal is to find a low tour-distance - if you can find the lowest, fine, but in practice you always must settle for something suboptimal. The choice of using tour-distance as one's fitness function was never made to "help the algorithm .. find the target point". (There is not even a single "target point", per se!) Rather it is made because ... that is what is of interest. The problem *is* the fitness function. We would argue that this is the case for almost any real-world problem. Now one's algorithm can perform transformations of the function so as to "help the algorithm ... find (a good) point". But that's another issue entirely. Aside to reader: Note that Bofill does not use "target" the way a machine-learner would. >>> If I understood them properly, Wolpert and Macready define the space of all possible fitness functions as the set of ALL possible assignments of cost values to points in search space. >>> Or to use Bofill's language, the space of all possible problems; they are synonymous. >>> And they use as a generalized perfomace measure the histogram of cost values, regardless of the points where they were found. >>> In most (almost all?) real world problems the set of points sampled (d_X values in our terminology) has essentially implications for the efficacy of the search algorithm used. Fitness values at those points, and maybe computational complexity of the algorithm, have such implications. But not the points themselves. Indeed, if there were some aspect of the points that *were* important, then (by definition!) it would go into the fitness function. Again consider TSP. The performance of an algorithm is determined by the tour-lengths (fitnesses) of the tours the algorithm has constructed, not by the tours themselves. >>> and their statement is equivalent to, "any two search algorithms have exactly the same expected performance, over the space of all optimization problems that can be defined whithin a search space." >>> Loosely speaking. >>> which stated otherwise would mean, "If the target doesn't matter, all algorithms perform alike." >>> This makes no sense. Of course the target matters - it matters more than anything else. The point is that *unless you take explicit consideration of the target (or more generally the fitness function) into account when using your algorithm*, all algorithms are alike. And *many* users of genetic algorithms, tabu search, hill-climbing, simulated annealing, etc., do not take into account the fitness function. (In fact, in many competitions, knowledge concerning the fitness function is considered cheating!) This - relatively minor - point of the paper is a cry for people to do the obvious: incorporate knowledge of the fitness function into their algorithm. On any problem, best results will accrue to such a strategy (e.g., the winners at TSP are those algorithms that are tailored to TSP.) Yet people still try to use fixed algorithms as though they were "general purpose". The NFL results point out the lack of formal justifiability of such strategies. >>> In particular, algorithms should be compared on PROBLEMS, not on fitness functions, with properly defined performance measures. >>> This is a content-free statement. In the real world, problems *are* fitness functions. Bill and David From tsioutsias-dimitris at CS.YALE.EDU Wed Nov 8 23:54:04 1995 From: tsioutsias-dimitris at CS.YALE.EDU (Dimitris I. Tsioutsias) Date: Wed, 8 Nov 1995 23:54:04 -0500 (EST) Subject: No Free Lunch? (Delayed reply) Message-ID: <199511090454.XAA21775@nebula.systemsz.cs.yale.edu> > From: simonpe at aisb.ed.ac.uk > Date: Wed, 8 Nov 1995 12:32:38 +0000 > .... > > I think the importance of the NFL theorem is that it destroys the idea > that there's such a thing as a `super-powerful all-purpose general > learning algorithm' that can optimize any problem quickly. Instead, if > we want a learning algorithm to work well, we have to analyze the > particular problems we're interested in and tailor the learning > algorithm to suit. Essentially we have to find ways of building prior > domain knowledge about a problem into learning algorithms to solve > that problem effectively. > ..... As any thoughtful person in the greater mathematical programming community might point out, there's no general optimization method that could outperform any other on any kind of problem. Rather it's the researcher's task to have an understanding of the problem domain, the computational requirements (and available resources), and the most intuitively promising avenue of attaining a ``good'' solution in the most efficient way... --Dimitris From bruno at redwood.psych.cornell.edu Wed Nov 8 23:52:46 1995 From: bruno at redwood.psych.cornell.edu (Bruno A. Olshausen) Date: Wed, 8 Nov 1995 23:52:46 -0500 Subject: sparse coding Message-ID: <199511090452.XAA13170@redwood.psych.cornell.edu> The following paper is available via http://redwood.psych.cornell.edu/bruno/papers.html or ftp://redwood.psych.cornell.edu/pub/papers/sparse-coding.ps.Z Sparse coding of natural images produces localized, oriented, bandpass receptive fields Bruno A. Olshausen and David J. Field Department of Psychology, Uris Hall Cornell University Ithaca, New York 14853 The images we typically view, or natural scenes, constitute a minuscule fraction of the space of all possible images. It seems reasonable that the visual cortex, which has evolved and developed to effectively cope with these images, has discovered efficient coding strategies for representing their structure. Here, we explore the hypothesis that the coding strategy employed at the earliest stage of the mammalian visual cortex maximizes the sparseness of the representation. We show that a learning algorithm that attempts to find linear sparse codes for natural scenes will develop receptive fields that are localized, oriented, and bandpass, much like those in the visual system. These receptive fields produce a more efficient image representation for later stages of processing because sparseness reduces the entropies of individual outputs, which in turn reduces the redundancy due to complex statistical dependencies among unit activities. From rosen at unr.edu Thu Nov 9 02:23:15 1995 From: rosen at unr.edu (David Rosen) Date: Wed, 8 Nov 1995 23:23:15 -0800 Subject: "How Good Were Those Probability Predictions?" -- Paper Available Message-ID: <199511090717.HAA05459@solstice.ccs.unr.edu> Announcing the following paper available in the neuroprose archive: How Good Were Those Probability Predictions? The Expected Recommendation Loss (ERL) Scoring Rule David B. Rosen To appear in: Maximum Entropy and Bayesian Methods. (Proceedings of the Thirteenth International Workshop, August 1993.) G. Heidbreder, ed. Kluwer, Dordrecht, The Netherlands, 1996. 8 pages. We present a new way to choose an appropriate scoring rule for evaluating the performance of a "soft classifier", i.e. of a supplier of predicted (inferred/estimated/learned/guessed) probabilities. A scoring rule (probability loss function) is a function of a single such prediction and the corresponding outcome event (true class); its expectation over the data space is the generalization performance of ultimate interest, while its sum or average over some benchmark test data set is an empirical performance measure. A user of probability predictions can apply his own decision threshold, preferring to err on one side, for example, to the extent that the consequences of an erroneous decision are more severe on the other side; this process is the subject of decision theory/analysis. We are not able to specify in advance, with certainty, these relative consequences, i.e. the user's cost matrix (indexed by decision and outcome event) defining his decision-making problem. So we represent this uncertainty itself by a distribution, from which we think of the cost matrix as being drawn. Specifying this distribution determines a uniquely appropriate scoring rule. We can interpret and characterize common scoring rules, such as the logarithmic (cross-entropy), quadratic (squared error or Brier), and the "0-1" misclassification score, as representing different assumptions about the probability that the predictions will be used in various decision-making problems. We discuss the connection to the theory of proper (truth- or honesty-rewarding) scoring rules. PostScript and plain-text versions are available via this Web page: http://www.scs.unr.edu/~cbmr/people/rosen/erl/ The paper is in Jordan Pollack's NEUROPROSE anonymous ftp archive as: ftp://archive.cis.ohio-state.edu/pub/neuroprose/rosen.exp-rec-loss.ps.Z (This supersedes an unannounced early version rosen.scoring.ps.Z) Hardcopies cannot be provided. -- David B Rosen OR New York Medical College From robtag at dia.unisa.it Thu Nov 9 05:55:21 1995 From: robtag at dia.unisa.it (Tagliaferri Roberto) Date: Thu, 9 Nov 1995 11:55:21 +0100 Subject: WIRN 96 Message-ID: <9511091055.AA11466@udsab.dia.unisa.it> ***************** CALL FOR PAPERS ***************** The 8-th Italian Workshop on Neural Nets WIRN VIETRI-96 May 23-25, 1996 Vietri Sul Mare, Salerno ITALY **************** FIRST ANNOUNCEMENT ***************** Organizing - Scientific Committee -------------------------------------------------- B. Apolloni (Univ. Milano) A. Bertoni ( Univ. Milano) D. D. Caviglia ( Univ. Genova) P. Campadelli ( Univ. Milano) M. Ceccarelli ( CNR Napoli) A. Colla (ELSAG Bailey Genova) M. Frixione ( I.I.A.S.S.) C. Furlanello (IRST Trento) G. M. Guazzo ( I.I.A.S.S.) M. Gori ( Univ. Firenze) F. Lauria ( Univ. Napoli) M. Marinaro ( Univ. Salerno) F. Masulli (Univ. Genova) P. Morasso (Univ. Genova) G. Orlandi ( Univ. Roma) E. Pasero ( Politecnico Torino ) A. Petrosino ( I.I.A.S.S.) M. Protasi ( Univ. Roma II) S. Rampone ( Univ. Salerno ) R. Serra ( Gruppo Ferruzzi Ravenna) F. Sorbello ( Univ. Palermo) R. Stefanelli ( Politecnico Milano) R. Tagliaferri ( Univ. Salerno) R. Vaccaro ( CNR Napoli) Topics ---------------------------------------------------- Mathematical Models Architectures and Algorithms Hardware and Software Design Hybrid Systems Pattern Recognition and Signal Processing Industrial and Commercial Applications Fuzzy Tecniques for Neural Networks Schedule ----------------------- Papers Due: January 31, 1996 Replies to Authors: March 31, 1996 Revised Papers Due: May 23, 1996 Sponsors ------------------------------------------------------------------------------ International Institute for Advanced Scientific Studies (IIASS) Dept. of Fisica Teorica, University of Salerno Dept. of Informatica e Applicazioni, University of Salerno Dept. of Scienze dell'Informazione, University of Milano Istituto per la Ricerca dei Sistemi Informatici Paralleli (IRSIP - CNR) Societa' Italiana Reti Neuroniche (SIREN) The 8-th Italian Workshop on Neural Nets (WIRN VIETRI-96) will take place in Vietri Sul Mare, Salerno ITALY, May 23-25, 1996. The conference will bring together scientists who are studying several topics related to neural networks. The three-day conference, to be held in the I.I.A.S.S., will feature both introductory tutorials and original, refereed papers, to be published by World Scientific Publishing. Papers should be 6 pages,including title, figures, tables, and bibliography. The first page should give keywords, postal and electronic mailing addresses, telephone, and FAX numbers. The camera ready format will be sent with the acceptation letter of the referees. Submit 3 copies and a 1 page abstract (containing keywords, postal and electronic mailing addresses, telephone, and FAX numbers with no more than 300 words) to the address shown (WIRN 96 c/o IIASS). An electronic copy of the abstract should be sent to the E-mail address below. During the Workshop the "Premio E.R. Caianiello" will be assigned to the best Ph.D. thesis in the area of Neural Nets and related fields of Italian researchers. The amount is of 2.000.000 Italian Lire. The interested researchers (with a thesis of 1993,1994, 1995 until February 29 1996) must send 3 copies of a c.v. and of the thesis to "Premio Caianiello" WIRN 96 c/o IIASS before February 29,1996. For more information, contact the Secretary of I.I.A.S.S. I.I.A.S.S Via G.Pellegrino, 19 84019 Vietri Sul Mare (SA) ITALY Tel. +39 89 761167 Fax +39 89 761189 E-Mail robtag at udsab.dia.unisa.it or the www pages at the address below: http:://www-dsi.ing.unifi.it/neural ***************************************************************** From juergen at idsia.ch Thu Nov 9 10:52:24 1995 From: juergen at idsia.ch (Juergen Schmidhuber) Date: Thu, 9 Nov 95 16:52:24 +0100 Subject: No Free Lunch? (Delayed reply) Message-ID: <9511091552.AA03224@fava.idsia.ch> In his response to NFL issues, Simon Perkins writes: > On the other hand, some people claim that there _is_ a general > sub-class of problems for which it's at all feasible to find a > solution - perhaps problems whose solutions have low komologrov > complexity or something - and this might well mean that there _is_ a > general good learning algorithm for this class of `interesting > problems'. A comment on this: We already know that for a wide variety of non-incremental search problems, there *is* a theoretically optimal algorithm: Levin's universal search algorithm (LS) (Ref.: L. A. Levin, Universal sequential search problems, Problems of Information Transmission, 9(3):265--266, 1973). Essentially, LS generates and tests solution candidates in order of their Levin complexities, until a solution is found (Levin complexity is a time-bounded restriction of Kolmogorov complexity). For instance, suppose there is an algorithm that solves a certain type of maze task in O(n^3) steps, where $n$ is a positive integer representing problem size. Then LS will solve the same task in at most O(n^3) steps (you may worry about the constant factor buried in the O-notation, though). Of course, there are huge classes of search problems that cannot be solved efficiently (say, in polynomial time), neither by LS nor by any other method. But most problems in the set of all possible, well-defined problems are indeed ``uninteresting''. Admittedly, I do not have a good objective definition of what's ``interesting''. The best I can come up with in the current context is, somewhat circularily, a possible superset of interesting problems: ``problems that can be solved efficiently by an optimal search algorithm''. Anyway, LS' existence fuels hope: just like there is an optimal, general search algorithm for many *non-incremental* search problems, there may be an optimal, general learning algorithm for *incremental* search problems (``incremental'' means: you may try to use experience with previous tasks to improve performance on new tasks). LS by itself is *not* necessarily optimal in incremental learning situations. For this reason, Marco Wiering and I are currently combining LS and the recent technique of ``environment-independent reinforcement acceleration'' (EIRA), currently the only method that guarantees a lifelong history of success accelerations, even in unrestricted environments (write-up may follow soon -- related publications in my home page). Juergen Schmidhuber IDSIA, Corso Elvezia 36, Ch-6900-Lugano http://www.idsia.ch/~juergen From black at signal.dra.hmg.gb Fri Nov 10 11:59:22 1995 From: black at signal.dra.hmg.gb (John V. Black) Date: Fri, 10 Nov 95 16:59:22 +0000 Subject: WWW page for the Pattern and Information Processing Group at DRA (UK) Message-ID: Dear Connectionits, Announcing a new WWW home page that covers the work of the Pattern and Information Processing Group of the Defence Research Agency (DRA) in the United Kingdom. The group consists of some 18 people, who together have a wide experience of pattern and information processing techniques and problems, which include: Analogue Systems for Information Processing Bayesian Methods Classification, Identification and Recognition Data and Information Fusion Data Analysis and Exploration Decision and Game Theory Information Processing Architectures Neural Network Techniques and Architectures Radar (Array) Signal Processing Self-Organising Systems Sensor Signal Processing Applications Statistical Pattern Processing Time-Series Analysis Tracking Uncertainty Handling (Bayesian Networks, Fuzzy Logic) The URL of the home page is http://www.dra.hmg.gb/cis5pip/Welcome.html John Black (black at signal.dra.hmg.gb) From isca at interpath.com Mon Nov 13 11:43:24 1995 From: isca at interpath.com (Mary Ann Sullivan) Date: Mon, 13 Nov 1995 11:43:24 -0500 Subject: CALL FOR PAPERS: 5th Intelligent Systems Conf. - Reno, Nevada June 19-21, 1996 (Formerly GWICS) Message-ID: <199511131643.LAA13980@mail-hub.interpath.net> ******************************************************************************* This message is being sent to multiple addressees. If you wish to have your address removed from our mailing list, please reply to the sender and we will promptly honor your request. ******************************************************************************* CALL FOR PAPERS Fifth International Conference on Intelligent Systems (formerly GWICS) June 19 - 21, 1996 Flamingo Hilton, Reno, Nevada, U.S.A. Sponsored by the International Society for Computers and Their Applications (ISCA) ============================================================================== CONFERENCE CHAIR PROGRAM CHAIR Carl Looney Frederick C. Harris, Jr. (Univ. of Nevada, Reno) (Univ. of Nevada, Reno) ============================================================================== The International Conference on Intelligent Systems seeks quality international submissions in all areas of intelligent systems including but not limited to: Logic and Inference Cognitive Science Artificial Neural Networks Reasoning Distributed Intelligent Systems Artificial Life Case-Based Reasoning Knowledge-Based Systems Vision, Image Processing Interpretation Machine Learning and Adaptive Sys. Cellular Automata Fuzzy Systems Robotics, Control and Planning Multimedia and Human Computer Evolutionary Computation Interaction (GA,GP,ES,EP) Autonomous Agents Recognition and Classification Search Instructions to Authors: Authors must submit 5 copies of an extended abstract (at least 4 pages) or complete paper (no more than 10 double spaced pages). Please include one separate cover page containing title, author's name(s), address, affiliation, e-mail address, telephone number, and topic area. To help us assign reviewers to papers, use the topics in the list above as a guide. In cases of multiple authors, all correspondence will be sent to the first author unless otherwise requested. Abstracts may be submitted via E-mail. Submit your paper by February 15, 1996 to the program chair: Dr. Frederick C. Harris, Jr. Telephone: (702) 784-6571 University of Nevada Fax: (702) 784-1766 Dept. of Computer Science E-mail: fredh at cs.unr.edu Reno, Nevada 89557 IMPORTANT DATES: Deadline for extended summary/paper submission: February 15, 1996 Notification of acceptance: April 10, 1996 Camera ready papers due: May 10, 1996 PROGRAM COMMITTEE G. Antoniou (U. of Newcastle) A. Barducci (CNR-IROE,Italy) M. Boden (U. of Skovde) A. Canas (U. of West Florida) M. Cohen (Cal State, Fresno) D. Egbert (U. of Nevada, Reno) S. Fadali (U. of Nevada, Reno) J. Fisher (Cal State Pomona) K. Ford (U. of West Florida) P. Geril (U. of Ghent) J. Gero (U. of Sydney) D. Hudson (U. of Cal, San Fran.) P. Jog (DePaul U.) S. Kawata (Tokyo Metropolitan U.) V. R. Kumar (Fujitsu, Australia) D. Leake (Indiana U.) F. Lin (Santa Clara U.) S. Louis (U. of Nevada, Reno) J. McDonnell (NRaD, San Diego) A. McRae (Appalachian State U.) S. Narayan (UNC-Wilmington) T. Oren (U. of Ottawa) V. Patel (McGill U.) D. Pheanis (Arizona State U.) V. Piuri (Politecnico Di Milano) R. Reynolds (Wayne State U.) M. Rosenman (U. of Sydney) A. Sangster (Aberdeen U.) R. Smith (U. of Alabama) R. Sun (U. of Alabama) A. Yfantis (UNLV) S. Yoon (Widener U.) ISCA Headquarters 8820 Six Forks Road, Raleigh, NC 27615 (USA) Ph: (919) 847-3747 Fax: (919) 676-0666 E-mail: isca at interpath.com URL= http://www.isca-hq.org/isca From edelman at wisdom.weizmann.ac.il Tue Nov 14 01:54:55 1995 From: edelman at wisdom.weizmann.ac.il (Edelman Shimon) Date: Tue, 14 Nov 1995 06:54:55 GMT Subject: TR available: RFs From Hyperacuity to Recognition Message-ID: <199511140654.GAA19020@lachesis.wisdom.weizmann.ac.il> Retrieval information: FTP-host: eris.wisdom.weizmann.ac.il (132.76.80.53) FTP-pathname: /pub/watt-rfs.ps.Z URL: ftp://eris.wisdom.weizmann.ac.il/pub/watt-rfs.ps.Z 28 pages; 519 KB compressed, 2.6 MB uncompressed. Comments welcome at URL mailto:edelman at wisdom.weizmann.ac.il ---------------------------------------------------------------------- Receptive Fields for Vision: from Hyperacuity to Object Recognition Weizmann Institute CS-TR 95-29, 1995; to appear in VISION, R. J. Watt, ed., MIT Press, 1996. Shimon Edelman Dept. of Applied Mathematics and Computer Science The Weizmann Institute of Science Rehovot 76100, ISRAEL http://eris.wisdom.weizmann.ac.il/~edelman Many of the lower-level areas in the mammalian visual system are organized retinotopically, that is, as maps which preserve to a certain degree the topography of the retina. A unit that is a part of such a retinotopic map normally responds selectively to stimulation in a well-delimited part of the visual field, referred to as its {\em receptive field} (RF). Receptive fields are probably the most prominent and ubiquitous computational mechanism employed by biological information processing systems. This paper surveys some of the possible computational reasons behind the ubiquity of RFs, by discussing examples of RF-based solutions to problems in vision, from spatial acuity, through sensory coding, to object recognition. ---------------------------------------------------------------------- -Shimon From dhw at santafe.edu Tue Nov 14 13:24:58 1995 From: dhw at santafe.edu (David Wolpert) Date: Tue, 14 Nov 95 11:24:58 MST Subject: Response to no-free-lunch discussion Message-ID: <9511141824.AA04471@sfi.santafe.edu> Some quick comments on the recent discussion of no-free-lunch (NFL) issues. As an aside, it's interesting to note how "all over the map" the discussion is, from people who whole-heartedly agree with NFL, to people who make claims diametrically opposed to it. **** Simon Perkins writes: >>> some people claim that there _is_ a general sub-class of problems for which it's at all feasible to find a solution - perhaps problems whose solutions have low komologrov complexity or something - and this might well mean that there _is_ a general good learning algorithm for this class of `interesting problems'. Any comments? >>> There's no disputing this. Restricting attention to a sub-class of problems is formally equivalent to placing a restriction on the target and/or prior over targets (in the latter case, placing a restriction on the prior's support). Certainly once things are restricted this way, we are in the domain of Bayesian analysis (and/or some versions of PAC), and not all algorithms are the same. However we have *no* a priori reason to believe that targets with "low Kolmogorov complexity" (or anything else) are/not likely to occur in the real world. Well, nothing can be proven from first principles to work well, you might say. This actually isn't always true (the loss function is important, there are minimax issues, etc.) But even in the simple scenarios in which this sentiment is essentially correct (i.e., the scenarios in which NFL holds), there is a huge body of literature which purports to prove from first principles that some algorithms *do* work better than others, without any assumption about the targets: *** E.g., claims that so long as the VC dimension of your algorithm is low, the training set large, and the misclassification rate on the training set small, then *independent of assumptions concerning the target*, you can bound how large the generalization error is. Or claims that boosting can only help generalization error, regardless of the prior over targets. Or the PAC "proof" of Occam's razor (which - absurdly - "holds" for any and all complexity measures). NFL results show up all such claims as problematic, at best. The difficulty is not with the math behind these claims, but rather with the interpretations of what that math means. *** Dimitris Tsioutsias writes: >>> As any thoughtful person in the greater mathematical programming community might point out, there's no general optimization method that could outperform any other on any kind of problem. >>> Obviously. But to give a simple example, before now, no such "thoughtful person" would have had any idea of whether there might be a "general optimization method" that only rarely performs worse than random search. Addressing such issues is one of the (more trivial) things NFL can do for you. *** Finally, Juergen Schmidhuber writes: >>> We already know that for a wide variety of non-incremental search problems, there *is* a theoretically optimal algorithm: Levin's universal search algorithm >>> I have lots of respect for Juergen's work, but on this issue, I have to disagree with him. Simply put, supervised learning (and in many regards search) is an exercise in statistics, not algorithmic information complexity theory. NFL *proves* this. In practice, it may (or may not) be a good idea to use an algorithm that searches for low Levin complexity rather than one that works by other means. But there is simply no first principles reason for believing so. Will such an algorithm beat backprop? Maybe, maybe not. It depends on things that can not be proven from first principles. (As well as a precise definition of "beat", etc.) The distribution over the set of problems we encounter in the real world is governed by an extraordinarily complicated interplay between physics, chemistry, biology, psychology, sociology and economics. There is no a priori reason for believing that this interplay respects notion of algorithmic information complexity, time-bounded or otherwise. David Wolpert From copelli at onsager.if.usp.br Tue Nov 14 17:30:34 1995 From: copelli at onsager.if.usp.br (Mauro Copelli da Silva) Date: Tue, 14 Nov 1995 20:30:34 -0200 (EDT) Subject: On-line learning paper Message-ID: <199511142230.UAA01640@curie.if.usp.br> FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/copelli.equivalence.ps.Z *** PAPER ANNOUNCEMENT *** The following paper is available by anonymous ftp from the pub/neuroprose directory of the archive.cis.ohio-state.edu host (see instructions below). It is 27 pages long and has been submitted to Physical Review E. Comments are welcomed. EQUIVALENCE BETWEEN LEARNING IN PERCEPTRONS WITH NOISY EXAMPLES AND TREE COMMITTEE MACHINES Mauro Copelli, Osame Kinouchi and Nestor Caticha Instituto de Fisica, Universidade de Sao Paulo CP 66318, 05389-970 Sao Paulo, SP, Brazil e-mail: copelli,osame,nestor at if.usp.br Abstract We study learning from single presentation of examples ({\em incremental} or {\em on-line} learning) in single-layer perceptrons and tree committee machines (TCMs). Lower bounds for the perceptron generalization error as a function of the noise level $\epsilon$ in the teacher output are calculated. We find that optimal local learning in a TCM with $K$ hidden units is simply related to optimal learning in a simple perceptron with a corresponding noise level $\epsilon(K)$. For large number of examples and finite $K$ the generalization error decays as $\alpha_{cm}^{-1}$, where $\alpha_{cm}$ is the number of examples per adjustable weight in the TCM. We also show that on-line learning is possible even in the $K\rightarrow\infty$ limit, but with the generalization error decaying as $\alpha_{cm}^{-1/2}$. The simple Hebb rule can also be applied to the TCM, but now the error decays as $\alpha_{cm}^{-1/2}$ for finite $K$ and $\alpha_{cm}^{-1/4}$ for $K\rightarrow\infty$. Exponential decay of the generalization error in both the perceptron learning from noisy examples and in the TCM is obtained by using the learning by queries strategy. ****************** How to obtain a copy ************************* unix> ftp archive.cis.ohio-state.edu User: anonymous Password: (type your e-mail address) ftp> cd pub/neuroprose ftp> binary ftp> get copelli.equivalence.ps.Z ftp> quit unix> uncompress copelli.equivalence.ps.Z unix> lpr copelli.equivalence.ps (or however you print PostScript files) **PLEASE DO NOT REPLY DIRECTLY TO THIS MESSAGE** From cas-cns at PARK.BU.EDU Wed Nov 15 11:20:00 1995 From: cas-cns at PARK.BU.EDU (CAS/CNS) Date: Wed, 15 Nov 1995 11:20:00 -0500 Subject: BU - Cognitive & Neural Systems Message-ID: <199511151618.LAA28690@cns.bu.edu> ************************************************************** DEPARTMENT OF COGNITIVE AND NEURAL SYSTEMS (CNS) AT BOSTON UNIVERSITY ************************************************************** Ennio Mingolla, Acting Chairman, 1995-96 Stephen Grossberg, Chairman Gail A. Carpenter, Director of Graduate Studies The Boston University Department of Cognitive and Neural Systems offers comprehensive graduate training in the neural and computational principles, mechanisms, and architectures that underlie human and animal behavior, and the application of neural network architectures to the solution of technological problems. Applications for Fall, 1996, admission and financial aid are now being accepted for both the MA and PhD degree programs. To obtain a brochure describing the CNS Program and a set of application materials, write, telephone, or fax: DEPARTMENT OF COGNITIVE & NEURAL SYSTEMS 677 Beacon Street Boston, MA 02215 617/353-9481 (phone) 617/353-7755 (fax) or send via email your full name and mailing address to: rll at cns.bu.edu Applications for admission and financial aid should be received by the Graduate School Admissions Office no later than January 15. Late applications will be considered until May 1; after that date applications will be considered only as special cases. Applicants are required to submit undergraduate (and, if applicable, graduate) transcripts, three letters of recommendation, and Graduate Record Examination (GRE) scores. The Advanced Test should be in the candidate's area of departmental specialization. GRE scores may be waived for MA candidates and, in exceptional cases, for PhD candidates, but absence of these scores may decrease an applicant's chances for admission and financial aid. Non-degree students may also enroll in CNS courses on a part-time basis. Description of the CNS Department: The Department of Cognitive and Neural Systems (CNS) provides advanced training and research experience for graduate students interested in the neural and computational principles, mechanisms, and architectures that underlie human and animal behavior, and the application of neural network architectures to the solution of technological problems. Students are trained in a broad range of areas concerning cognitive and neural systems, including vision and image processing; speech and language understanding; adaptive pattern recognition; cognitive information processing; self-organization; associative learning and long-term memory; computational neuroscience; nerve cell biophysics; cooperative and competitive network dynamics and short-term memory; reinforcement, motivation, and attention; adaptive sensory-motor control and robotics; active vision; and biological rhythms; as well as the mathematical and computational methods needed to support advanced modeling research and applications. The CNS Department awards MA, PhD, and BA/MA degrees. The CNS Department embodies a number of unique offerings. It has developed a curriculum that features 15 interdisciplinary graduate courses each of which integrates the psychological, neurobiological, mathematical, and computational information needed to theoretically investigate fundamental issues concerning mind and brain processes and the applications of neural networks to technology. Each course is typically taught once a week in the evening to make the program available to qualified students, including working professionals, throughout the Boston area. Nine additional research course are also offered. In these courses, one or two students meet regularly with one or two professors to pursue advanced reading and collaborative research. Students develop a coherent area of expertise by designing a program that includes courses in areas such as Biology, Computer Science, Engineering, Mathematics, and Psychology, in addition to courses in the CNS Department. The CNS Department prepares students for PhD thesis research with scientists in one of several Boston University research centers or groups, and with Boston-area scientists collaborating with these centers. The unit most closely linked to the department is the Center for Adaptive Systems (CAS). Students interested in neural network hardware work with researchers in CNS, the College of Engineering, and at MIT Lincoln Laboratory. Other research resources include distinguished research groups in neurophysiology, neuroanatomy, and neuropharmacology at the Medical School and the Charles River campus; in sensory robotics, biomedical engineering, computer and systems engineering, and neuromuscular research within the Engineering School; in dynamical systems within the Mathematics Department; in theoretical computer science within the Computer Science Department; and in biophysics and computational physics within the Physics Department. In addition to its basic research and training program, the Department offers a colloquium series, seminars, conferences, and special interest groups which bring many additional scientists from both experimental and theoretical disciplines into contact with the students. The CNS Department is moving in October, 1995 into its own new four-story building, which features a full range of offices, laboratories, classrooms, library, lounge, and related facilities for exclusive CNS use. 1995-96 CAS MEMBERS and CNS FACULTY: Jelle Atema Professor of Biology Director, Boston University Marine Program (BUMP) PhD, University of Michigan Sensory physiology and behavior Aijaz Baloch Research Associate of Cognitive and Neural Systems PhD, Electrical Engineering, Boston University Neural modeling of role of visual attention of recognition, learning and motor control, computational vision, adaptive control systems, reinforcement learning Helen Barbas Associate Professor, Department of Health Sciences, Boston University PhD, Physiology/Neurophysiology, McGill University Organization of the prefrontal cortex, evolution of the neocortex Jacob Beck Research Professor of Cognitive and Neural Systems PhD, Psychology, Cornell University Visual Perception, Psychophysics, Computational Models Daniel H. Bullock Associate Professor of Cognitive and Neural Systems and Psychology PhD, Psychology, Stanford University Real-time neural systems, sensory-motor learning and control, evolution of intelligence, cognitive development Gail A. Carpenter Professor of Cognitive and Neural Systems and Mathematics Director of Graduate Studies, Department of Cognitive and Neural Systems PhD, Mathematics, University of Wisconsin, Madison Pattern recognition, categorization, machine learning, differential equations Laird Cermak Professor of Neuropsychology, School of Medicine Professor of Occupational Therapy, Sargent College Director, Memory Disorders Research Center, Boston Veterans Affairs Medical Center PhD, Ohio State University Michael A. Cohen Associate Professor of Cognitive and Neural Systems and Computer Science Director, CAS/CNS Computation Labs PhD, Psychology, Harvard University Speech and language processing, measurement theory, neural modeling, dynamical systems H. Steven Colburn Professor of Biomedical Engineering PhD, Electrical Engineering, Massachusetts Institute of Technology Audition, binaural interaction, signal processing models of hearing William D. Eldred III Associate Professor of Biology BS, University of Colorado; PhD, University of Colorado, Health Science Center Visual neural biology Paolo Gaudiano Assistant Professor of Cognitive and Neural Systems PhD, Cognitive and Neural Systems, Boston University Computational and neural models of vision and adaptive sensory-motor control Jean Berko Gleason Professor of Psychology AB, Radcliffe College; AM, PhD, Harvard University Psycholinguistics Douglas Greve Research Associate of Cognitive and Neural Systems PhD, Cognitive and Neural Systems, Boston University Stephen Grossberg Wang Professor of Cognitive and Neural Systems Professor of Mathematics, Psychology, and Biomedical Engineering Director, Center for Adaptive Systems Chairman, Department of Cognitive and Neural Systems PhD, Mathematics, Rockefeller University Theoretical biology, theoretical psychology, dynamical systems, applied mathematics Frank Guenther Assistant Professor of Cognitive and Neural Systems PhD, Cognitive and Neural Systems, Boston University Biological sensory-motor control, spatial representation, speech production Thomas G. Kincaid Chairman and Professor of Electrical, Computer and Systems Engineering, College of Engineering PhD, Electrical Engineering, Massachusetts Institute of Technology Signal and image processing, neural networks, non-destructive testing Nancy Kopell Professor of Mathematics PhD, Mathematics, University of California at Berkeley Dynamical systems, mathematical physiology, pattern formation in biological/physical systems Ennio Mingolla Associate Professor of Cognitive and Neural Systems and Psychology Acting Chairman 1995-96, Department of Cognitive and Neural Systems PhD, Psychology, University of Connecticut Visual perception, mathematical modeling of visual processes Alan Peters Chairman and Professor of Anatomy and Neurobiology, School of Medicine PhD, Zoology, Bristol University, United Kingdom Organization of neurons in the cerebral cortex, effects of aging on the primate brain, fine structure of the nervous system Andrzej Przybyszewski Senior Research Associate of Cognitive and Neural Systems MSc, Technical Warsaw University; MA, University of Warsaw; PhD, Warsaw Medical Academy Adam Reeves Adjunct Professor of Cognitive and Neural Systems Professor of Psychology, Northeastern University PhD, Psychology, City University of New York Psychophysics, cognitive psychology, vision William Ross Research Associate of Cognitive and Neural Systems BSc, Cornell University; MA, PhD, Boston University Mark Rubin Research Assistant Professor of Cognitive and Neural Systems Research Physicist, Naval Air Warfare Center, China Lake, CA (on leave) PhD, Physics, University of Chicago Neural networks for vision, pattern recognition, and motor control Robert Savoy Adjunct Associate Professor of Cognitive and Neural Systems Scientist, Rowland Institute for Science PhD, Experimental Psychology, Harvard University Computational neuroscience; visual psychophysics of color, form, and motion perception Eric Schwartz Professor of Cognitive and Neural Systems; Electrical, Computer and Systems Engineering; and Anatomy and Neurobiology PhD, High Energy Physics, Columbia University Computational neuroscience, machine vision, neuroanatomy, neural modeling Robert Sekuler Adjunct Professor of Cognitive and Neural Systems Research Professor of Biomedical Engineering, College of Engineering, BioMolecular Engineering Research Center Jesse and Louis Salvage Professor of Psychology, Brandeis University AB,MA, Brandeis University; Sc.M., PhD, Brown University Allen Waxman Adjunct Associate Professor of Cognitive and Neural Systems Senior Staff Scientist, MIT Lincoln Laboratory PhD, Astrophysics, University of Chicago Visual system modeling, mobile robotic systems, parallel computing, optoelectronic hybrid architectures James Williamson Research Associate of Cognitive and Neural Systems PhD, Cognitive and Neural Systems, Boston University Image processing and object recognition. Particular interests are: dynamic binding, self-organization, shape representation, and classification Jeremy Wolfe Adjunct Associate Professor of Cognitive and Neural Systems Associate Professor of Ophthalmology, Harvard Medical School Psychophysicist, Brigham & Women's Hospital, Surgery Dept. Director of Psychophysical Studies, Center for Clinical Cataract Research PhD, Massachusetts Institute of Technology From pfbaldi at cco.caltech.edu Wed Nov 15 13:45:09 1995 From: pfbaldi at cco.caltech.edu (Pierre Baldi) Date: Wed, 15 Nov 1995 10:45:09 -0800 (PST) Subject: Tal Grossman Memorial Workshop in Vail (NIPS95) Message-ID: NIPS95 TAL GROSSMAN MEMORIAL WORKSHOP MACHINE LEARNING APPROACHES IN COMPUTATIONAL MOLECULAR BIOLOGY December 1, 1995 Vail, CO CURRENT LIST OF SCHEDULED PRESENTATIONS: Alan Lapedes Neural Network Representations of Empirical Protein Potentials. Gary Stormo The Use of Neural Networks for Identification of Common Domains by Maximizing Specificity. Ajay N. Jain Machine Learning Techniques for Drug Design: Lead Discovery, Lead Optimization, and Screening Strategies. Anders Krogh Maximum Entropy Weighting of Aligned Sequences of Proteins or DNA. Paul Stolorz Applying Dynamic Programming Ideas to Monte Carlo Sampling. Soren Brunak Bendability of Exons and Introns in Human DNA. Pierre Baldi Mining Data Bases of Fragments with HMMs. CURRENT LIST OF ABSTRACTS: Alan Lapedes (Los Alamos National Laboratory) asl at t13.lanl.gov Neural Network Representations of Empirical Protein Potentials. Recently, there has been considerable interest in deriving and applying knowledge-based, empirical potential functions for proteins. These empirical potentials have been derived from the statistics of interacting, spatially neighboring residues, as may be obtained from databases of known protein crystal structures. We employ neural networks to redefine empirical potential functions from the point of view of discrimination functions. This approach generalizes previous work, in which simple frequency counting statistics are used on a database of known protein structures. This generalization allows us to avoid restriction to strictly pairwise interactions. Instead of frequency counting to fix adjustable parameters, one now optimizes an objective function involving a parameterized probability distribution. We show how our method reduces to previous work in special situations, illustrating in this context the relationship of neural networks to statistical methodology. A key feature in the approach we advocate is the development of a representation to describe the location of interacting residues that exist in a sphere of small fixed radius around each residue. This is a natural ``shape representation'' for the interaction neighborhoods of protein residues. We demonstrate that this shape representation and the network's improved abilities enhances discrimination over that obtained by previous methodologies. This work is with Robert Farber and the late Tal Grossman (Los Alamos National Laboratory). Gary Stormo (University of Colorado, Boulder) stormo at exon.biotech.washington.edu The Use of Neural Networks for Identification of Common Domains by Maximizing Specificity. We describe an unsupervised learning procedure in which the objective to be maximized is ``specificity'', defined as the probability of obtaining a particular set of strings within a much larger collection of background strings. We demonstrate its use for identifying protein binding sites on unaligned DNA sequences, common sequence/structure motifs in RNA and common motifs in protein sequences. The idea behind the ``specificity'' criterion it to discover a probability distribution for strings such that the difference between the probabilities of the particular strings and the background strings is maximized. Both the probability distribution and the set of particular strings need to be discovered; the probability distribution can be any allowable distribution over the string alphabet, and the particular strings are contained within a set of longer strings, but their locations are not known in advance. Previous methods have viewed this problem as one of multiple alignment, whereas our method is more flexible in the types of patterns that can be allowed and in the treatment of the background strings. When the patterns are linearly separable from the background, a simple Perceptron works well to identify the patterns. We are currently testing more complicated networks for more complicated patterns. This work is in collaboration with Alan Lapedes of Los Alamos National Laboratory and the Santa Fe Institute, and John Heumann of Hewlett-Packard. Ajay N. Jain (Arris Pharmaceutical Corporation) jain at arris.com Machine Learning Techniques for Drug Design: Lead Discovery, Lead Optimization, and Screening Strategies. At its core, the drug discovery process involves designing small organic molecules that satisfy the physical constraints of binding to a specific site on a particular protein (usually an enzyme or receptor). Machine learning techniques can play a significant role in all phases of the process. When the structure of the protein is known, it is possible to "dock" candidate molecules into the structure and compute the likelihood that a molecule will bind well. Fundamentally, this is a thermodynamic event that is too complicated to simulate accurately. Machine learning techniques can be used to empirically construct functions that are predictive of binding affinities. Similarly, when no protein structure is known, but there exists some data on molecules exhibiting a range of binding affinities, it is possible to use machine learning techniques to capture the 3D pattern that is responsible for binding. Lastly, in cases where one has capacity to make large numbers of small molecules (libraries) to screen against multiple diverse protein targets, one can use clustering techniques to design maximally diverse libraries. This talk will briefly discuss each of these techniques in the context of drug discovery at Arris Pharmaceutical Corporation. Anders Krogh (The Sanger Centre) krogh at sanger.ac.uk Maximum Entropy Weighting of Aligned Sequences of Proteins or DNA. In a family of proteins or other biological sequences like DNA the various subfamilies are often very unevenly represented. For this reason a scheme for assigning weights to each sequence can greatly improve performance at tasks such as database searching with profiles or other consensus models based on multiple alignments. A new weighting scheme for this type of database search is proposed. In a statistical description of the searching problem it is derived from the maximum entropy principle. It can be proved that, in a certain sense, it corrects for uneven representation. It is shown that finding the maximum entropy weights is an easy optimization problem for which standard techniques are applicable. Paul Stolorz (Jet Propulsion Laboratory, Caltech) stolorz at telerobotics.jpl.nasa.gov Applying Dynamic Programming Ideas to Monte Carlo Sampling. Monte Carlo sampling methods developed originally for physics and chemistry calculations have turned out to be very useful heuristics for problems in fields such as computational biology, traditional computer science and statistics. Macromolecular structure prediction and alignment, combinatorial optimization, and more recently probabilistic inference, are classic examples of their use. This talk will swim against the tide a bit by showing that computer science, in the guise of dynamic programming, can in turn supply substantial insight into the Monte Carlo process. This insight allows the construction of powerful novel Monte Carlo methods for a range of calculations in areas such as computational biology, computational vision and statistical inference. The methods are especially useful for problems plagued by multiple modes in the integrand, and for problems containing important, though not overwhelming, long-range information. Applications to protein folding, and to generalized Hidden Markov Models, will be described to illustrate how to systematically implement and test these algorithms. Soren Brunak (The Technical University of Denmark) brunak at cbs.dtu.dk Bendability of Exons and Introns in Human DNA. We analyze the sequential structure of human exons and introns by hidden Markov models. We find that exons -- besides the reading frame -- hold a specific periodic pattern. The pattern has the triplet consensus: non-T(A/T)G and a minimal periodicity of roughly 10 nucleotides. It is not a consequence of the nucleotide statistics in the three codon positions, nor of the previously well known periodicity caused by the encoding of alpha-helices in proteins. Using DNA triplet bendability parameters from DNase I experiments, we show that the pattern corresponds to a periodic `in-phase' bending potential towards the major groove of the DNA. Similarly, nucleosome positioning data show that the consensus triplets have a preference for locations on a bent double helix where the major groove faces inward and is compressed. We discuss the relation between the bending potential of coding regions and its importance for the recognition of genes by the transcriptional machinery. This work is in collaboration with P. Baldi (Caltech), Y. Chauvin (Net-ID, Inc.), Anders Krogh (The Sanger Centre). Pierre Baldi (Caltech) pfbaldi at ccosun.caltech.edu Mining Data Bases of Fragments with HMMs. Hidden Markov Model (HMM) techniques are applied to the problem of mining large data bases of protein fragments. The study is focused on one particular protein family, the G-Protein-Coupled Receptors (GPCR). A large data base is first constructed, by randomly extracting fragments from the entire SWISS-PROT data base, at different lengths, positions, and simulated noise levels, in a way that roughly matches other existing, but not always publicly accessible, data bases. A HMM trained on the GPCR family is then used to score all the fragments, in terms of their negative log-likelihood. The discrimination power of the HMM is assessed, and quantitative results are derived on how performance degrades, as a function of fragment length, truncation position, and noise level, and on how to set discrimination thresholds. The raw score performance is further improved by deriving additional filters, based on the structure of the alignments of the fragments to the HMM. This work is in collaboration with Y. Chauvin (Net-ID, Inc.), F. Tobin and A. Williams (SmithKline Beecham). From tibs at utstat.toronto.edu Wed Nov 15 15:40:00 1995 From: tibs at utstat.toronto.edu (tibs@utstat.toronto.edu) Date: Wed, 15 Nov 95 15:40 EST Subject: new tech report available Message-ID: Model search and inference by bootstrap ``bumping'' Robert Tibshirani and Keith Knight University of Toronto We propose a bootstrap-based method for searching through a space of models. The technique is well suited to complex, adaptively fitted models: it provides a convenient method for finding better local minima, for resistant fitting, and for optimization under constraints. Applications to regression, classification and density estimation are described. The collection of models can also be used to form a confidence set for the true underlying model, using a generalization of Efron's percentile interval. We also provide results on the asymptotic behaviour of bumping estimates. Available at http://utstat.toronto.edu/reports/tibs or ftp: utstat.toronto.edu in pub/tibs/bumping.ps ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Rob Tibshirani, Dept of Preventive Med & Biostats, and Dept of Statistics Univ of Toronto, Toronto, Canada M5S 1A8. Phone: 416-978-4642 (PMB), 416-978-0673 (stats). FAX: 416 978-8299 tibs at utstat.toronto.edu. ftp: //utstat.toronto.edu/pub/tibs http://www.utstat.toronto.edu/~tibs/home.html +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ From scheler at ICSI.Berkeley.EDU Thu Nov 16 12:44:40 1995 From: scheler at ICSI.Berkeley.EDU (Gabriele Scheler) Date: Thu, 16 Nov 1995 09:44:40 -0800 Subject: Job Notice Message-ID: <199511161744.JAA15103@tiramisu.ICSI.Berkeley.EDU> ***********JOB NOTICE ******** 1 - 2 full-time positions for research associates are available in the area of connectionist natural language modeling in the Department of Computer Science, Technical University of Munich, Germany. One project concerns text-based learning, the other (pending final approval) combines modeling of language acquisition with situated learning. Several short-term or half-time paid positions for graduate students are also available. Knowledge of German is helpful, but not essential. Positions may start as early as February 1, 1996. A full job description with reference to relevant www-sites will be mailed to interested persons in Mid-December. Anyone interested in these positions should direct an informal request for further details to Dr Gabriele Scheler ICSI 1947 Center Street, Berkeley 94704-1198 scheler at icsi.berkeley.edu From barberd at helios.aston.ac.uk Wed Nov 15 13:40:48 1995 From: barberd at helios.aston.ac.uk (barberd) Date: Wed, 15 Nov 1995 18:40:48 +0000 (GMT) Subject: Paper available Message-ID: <6923.9511151840@sun.aston.ac.uk> The following paper, (a version of which was submitted to Europhysics Letters) is available by anonymous ftp (instructions below). FINITE SIZE EFFECTS IN ON-LINE LEARNING OF MULTI-LAYER NEURAL NETWORKS David Barber{2}, Peter Sollich{1} and David Saad{2} {1} Department of Physics, University of Edinburgh, EH9 3JZ, UK {2} Neural Computing Research Group, Aston University, Birmingham B4 7ET, United Kingdom email: D.Barber at aston.ac.uk Abstract We complement the recent progress in thermodynamic limit analyses of mean on-line gradient descent learning dynamics in multi-layer networks by calculating the fluctuations possessed by finite dimensional systems. Fluctuations from the mean dynamics are largest at the onset of specialisation as student hidden unit weight vectors begin to imitate specific teacher vectors, and increase with the degree of symmetry of the initial conditions. Including a term to stimulate asymmetry in the learning process typically significantly decreases finite size effects and training time. Ftp instructions ftp cs.aston.ac.uk User: anonymous Password: (type your e-mail address) ftp> cd neural/barberd ftp> binary ftp> get online.ps.Z ftp> quit unix> uncompress online.ps.Z **PLEASE DO NOT REPLY DIRECTLY TO THIS MESSAGE** From saadd at helios.aston.ac.uk Thu Nov 16 06:44:25 1995 From: saadd at helios.aston.ac.uk (saadd) Date: Thu, 16 Nov 1995 11:44:25 +0000 (GMT) Subject: NIPS workshop: The Dynamics Of On-Line Learning Message-ID: <644.9511161144@sun.aston.ac.uk> THE DYNAMICS OF ON-LINE LEARNING NIPS workshop, Friday and Saturday, December 2-3, 1995 7:30AM to 9:30AM -- 4:30PM to 6:30PM Organizers: Sara A. Solla (CONNECT, The Niels Bohr Institute) and David Saad (Aston University) On-line learning refers to a scenario in which the couplings of the learning machine are updated after the presentation of each example. The current hypothesis is used to predict an output for the current input; the corresponding error signal is used for weight modification, and the modified hypothesis is used for output prediction at the subsequent time step. This type of algorithm addresses general questions of learning dynamics, and has attracted the attention of both the computational learning theory and the statistical physics communities. Recent progress has provided tools that allow for the investigation of learning scenarios that incorporate many of the aspects of the learning of complex tasks: multilayer architectures, noisy data, regularization through weight decay, the use of momentum, tracking changing environments, presentation order when cycling repeatedly through a finite training set... An open and somewhat controversial question to be discussed in the workshop is the role of the learning rate in controlling the evolution and convergence of the learning process. The purpose of the workshop is to review the theoretical tools available for the analysis of on-line learning, to evaluate the current state of research in the field, and to predict possible contributions to the understanding and description of real world learning scenarios. We also seek to identify future research directions using these methods, their limitations and expected difficulties. The topics to be addressed in this workshop can be grouped as follows: 1) The investigation of on-line learning from the point of view of stochastic approximation theory. This approach is based on formulating a master equation to describe the dynamical evolution of a probability density which describes the ensemble of trained networks in the space of weights of the student network. (Todd Leen, Bert Kappen, Jenny Orr) 2) The investigation of on-line learning from the point of view of statistical mechanics. This approach is based on the derivation of dynamical equations for the overlaps among the weight vectors associated with the various hidden units in both student and teacher networks. The dynamical evolution of the overlaps provides a detailed characterization of the learning process and determines the generalization error. (Sara Solla, David Saad, Peter Riegler, David Barber, Ansgar West, Naama Barkai, Jason Freeman, Adam Prugel-Bennett) 3) The identification of optimal strategies for on-line learning. Most of the work has concentrated on the learning of classification tasks. A recent Bayesian formulation of the problem provides a unified derivation of optimal on-line equations. (Shun-ichi Amari, Nestor Caticha, Manfred Opper) The names in parenthesis identify the speakers. A list of additional participants includes Michael Kearns, Yann Le Cun, Yoshiyuki Kabashima, Mauro Copelli, Noboru Murata, Klaus Mueller. PROGRAM FOR THE NIPS WORKSHOP ON "THE DYNAMICS OF ON-LINE LEARNING" Friday ------ Morning: 7:30AM to 9:30AM Introduction: Todd Leen Speakers: Jenny Orr Bert Kappen Afternoon: 4:30PM to 6:30PM Speakers: Shun-ichi Amari Nestor Caticha Manfred Opper Saturday -------- Morning: 7:30AM to 9:30AM Introduction: Sara Solla/David Saad Speakers: David Barber Ansgar West Peter Riegler Afternoon: 4:30PM to 6:30PM Speakers: Adam Prugel-Bennett Jason Freeman Peter Riegler Naama Barkai More details, abstracts and references can be found on: http://neural-server.aston.ac.uk/nips95/workshop.html From thrun+ at HEAVEN.LEARNING.CS.CMU.EDU Wed Nov 15 20:39:04 1995 From: thrun+ at HEAVEN.LEARNING.CS.CMU.EDU (thrun+@HEAVEN.LEARNING.CS.CMU.EDU) Date: Wed, 15 Nov 95 20:39:04 EST Subject: 2 papers available Message-ID: Dear Colleagues: I am happy to announce two new papers: Lifelong Learning: A Case Study Sebastian Thrun Machine learning has not yet succeeded in the design of robust learning algorithms that generalize well from very small datasets. In contrast, humans often generalize correctly from only a single training example, even if the number of potentially relevant features is large. To do so, they successfully exploit knowledge acquired in previous learning tasks, to bias subsequent learning. This paper investigates learning in a lifelong context. Lifelong learning addresses situations where a learner faces a stream of learning tasks. Such scenarios provide the opportunity for synergetic effects that arise if knowledge is transferred across multiple learning tasks. To study the utility of transfer, several approaches to lifelong learning are proposed and evaluated in an object recognition domain. It is shown that all these algorithms generalize consistently more accurately from scarce training data than comparable "single-task" approaches. World Wide Web URL: http://www.cs.cmu.edu/~thrun/papers/thrun.lll_case_study.ps.Z ------------------------------------------------------------------- Clustering Learning Tasks and the Selective Cross-Task Transfer of Knowledge Sebastian Thrun and Joseph O'Sullivan Recently, there has been an increased interest in machine learning methods that learn from more than one learning task. Such methods have repeatedly found to outperform conventional, single-task learning algorithms when learning tasks are appropriately related. To increase robustness of these approaches, methods are desirable that can reason about the relatedness of individual learning tasks, in order to avoid the danger arising from tasks that are unrelated and thus potentially misleading. This paper describes the task-clustering (TC) algorithm. TC clusters learning tasks into classes of mutually related tasks. When facing a new thing to learn, TC first determines the most related task cluster, then exploits information selectively from this task cluster only. An empirical study carried out in a mobile robot domain shows that TC outperforms its unselective counterpart in situations where only a small number of tasks is relevant. World Wide Web URL: http://www.cs.cmu.edu/~thrun/papers/thrun.TC.ps.Z ------------------------------------------------------------------- INSTRUCTIONS FOR RETRIEVAL: (a) If you have access to the World Wide Web, you can retrieve the documents from my homepage (URL: http://www.cs.cmu.edu/~thrun, follow the paper link) or access them directly: netscape http://www.cs.cmu.edu/~thrun/papers/thrun.lll_case_study.ps.Z netscape http://www.cs.cmu.edu/~thrun/papers/thrun.TC.ps.Z (b) If you instead wish to retrieve the documents via anonymous ftp, follow these instructions: unix> ftp uran.informatik.uni-bonn.de user: anonymous passwd: aaa at bbb.ccc ftp> cd pub/user/thrun ftp> bin ftp> get thrun.lll_case_study.ps.Z ftp> get thrun.TC.ps.Z ftp> bye unix> uncompress thrun.lll_case_study.ps.Z unix> uncompress thrun.TC.ps.Z unix> lpr thrun.lll_case_study.ps.Z unix> lpr thrun.TC.ps.Z (c) Hard-copies can be obtained directly from Technical Reports Computer Science Department Carnegie Mellon University 5000 Forbes Ave Pittsburgh, PA 15213 Email: reports at cs.cmu.edu Please refer to the first paper as TR CMU-CS-95-208 and the second paper as TR CMU-CS-95-209 Comments are welcome! Sebastian Thrun From juergen at idsia.ch Wed Nov 15 04:35:42 1995 From: juergen at idsia.ch (Juergen Schmidhuber) Date: Wed, 15 Nov 95 10:35:42 +0100 Subject: response: Levin search Message-ID: <9511150935.AA03192@fava.idsia.ch> My response to David Wolpert's response to my response: Although it is true that ``we have *no* a priori reason to believe that targets with low Kolmogorov complexity (or anything else) are/not likely to occur in the real world'', it's also true that Levin search (LS) has the optimal order of search complexity for a broad class of *non-incremental* search problems (David did not agree). This is a well-known fact of theoretical computer science. Why is LS as efficient as any other search algorithm? Because LS effectively *runs* all the other search algorithms, but in a smart way that prevents it from loosing too much time with "wrong" search algorithms. David is right, however, by saying that ``In practice, it may (or may not) be a good idea to use an algorithm that searches for low Levin complexity rather than one that works by other means.'' One reason for this is: in practice, your problem size is always limited, and you *do* have to worry about the (possibly huge) constant buried in the notion of "optimal order of search complexity". Note that all my recent comments refer to *non-incremental* search --- for the moment, I am not addressing generalization issues. Can LS help us to improve certain kinds of *incremental* search and learning? We are currently trying to figure this one out. Juergen Schmidhuber From schwenk at robo.jussieu.fr Fri Nov 17 15:28:07 1995 From: schwenk at robo.jussieu.fr (Holger Schwenk) Date: Fri, 17 Nov 1995 20:28:07 +0000 (WET) Subject: paper available (OCR, discriminant tangent distance) Message-ID: <951117202807.22380000.adc15034@lea.robo.jussieu.fr> **DO NOT FORWARD TO OTHER GROUPS** FTP-host: ftp.robo.jussieu.fr FTP-filename: /pub/papers/schwenk.icann95.ps.gz (6 pages, 31k) The following paper, published in International Conference on Artificial Neural Networks (ICANN*95), Springer Verlag, is available via anonymous FTP at the above location. The paper is 6 pages long. Sorry, no hardcopies available. =========================================================================== H. Schwenk and M. Milgram PARC - boite 164 Universite Pierre et Marie Curie 4, place Jussieu 75252 Paris cedex 05, FRANCE ABSTRACT Transformation invariance is known to be fundamental for excellent performances in pattern recognition. One of the most successful approach is tangent distance, originally proposed for a nearest-neighbor algorithm (Simard et al.,1995). The resulting classifier, however, has a very high computational complexity and, perhaps more important, lacks discrimination capabilities. We present a discriminant learning algorithm for a modular classifier based on several autoassociative neural networks. Tangent distance as objective function guarantees efficient incorporation of transformation invariance. The system achieved a raw error rate of 2.6% and a rejection rate of 3.6% on the NIST uppercase letters. ============================================================================ FTP instructions: unix> ftp ftp.robo.jussieu.fr Name: anonymous Password: your full email address ftp> cd pub/papers ftp> bin ftp> get schwenk.icann95.ps.gz ftp> quit unix> gunzip schwenk.icann95.ps.gz unix> lp schwenk.icann95.ps (or however you print postscript) I welcome your comments. --------------------------------------------------------------------- Holger Schwenk PARC - boite 164 tel: (+33 1) 44.27.63.08 Universite Pierre et Marie Curie fax: (+33 1) 44.27.62.14 4, place Jussieu 75252 Paris cedex 05 email: schwenk at robo.jussieu.fr FRANCE --------------------------------------------------------------------- From schwenk at robo.jussieu.fr Fri Nov 17 15:28:07 1995 From: schwenk at robo.jussieu.fr (Holger Schwenk) Date: Fri, 17 Nov 1995 20:28:07 +0000 (WET) Subject: paper available (OCR, discriminant tangent distance) Message-ID: <951117202807.22380000.adc15034@lea.robo.jussieu.fr> **DO NOT FORWARD TO OTHER GROUPS** FTP-host: ftp.robo.jussieu.fr FTP-filename: /pub/papers/schwenk.icann95.ps.gz (6 pages, 31k) The following paper, published in International Conference on Artificial Neural Networks (ICANN*95), Springer Verlag, is available via anonymous FTP at the above location. The paper is 6 pages long. Sorry, no hardcopies available. =========================================================================== H. Schwenk and M. Milgram PARC - boite 164 Universite Pierre et Marie Curie 4, place Jussieu 75252 Paris cedex 05, FRANCE ABSTRACT Transformation invariance is known to be fundamental for excellent performances in pattern recognition. One of the most successful approach is tangent distance, originally proposed for a nearest-neighbor algorithm (Simard et al.,1995). The resulting classifier, however, has a very high computational complexity and, perhaps more important, lacks discrimination capabilities. We present a discriminant learning algorithm for a modular classifier based on several autoassociative neural networks. Tangent distance as objective function guarantees efficient incorporation of transformation invariance. The system achieved a raw error rate of 2.6% and a rejection rate of 3.6% on the NIST uppercase letters. ============================================================================ FTP instructions: unix> ftp ftp.robo.jussieu.fr Name: anonymous Password: your full email address ftp> cd pub/papers ftp> bin ftp> get schwenk.icann95.ps.gz ftp> quit unix> gunzip schwenk.icann95.ps.gz unix> lp schwenk.icann95.ps (or however you print postscript) I welcome your comments. --------------------------------------------------------------------- Holger Schwenk PARC - boite 164 tel: (+33 1) 44.27.63.08 Universite Pierre et Marie Curie fax: (+33 1) 44.27.62.14 4, place Jussieu 75252 Paris cedex 05 email: schwenk at robo.jussieu.fr FRANCE --------------------------------------------------------------------- From horne at research.nj.nec.com Fri Nov 17 15:16:49 1995 From: horne at research.nj.nec.com (Bill Horne) Date: Fri, 17 Nov 1995 15:16:49 -0500 Subject: NIPS*95 Workshop on Neural Networks for Signal Processing Message-ID: <9511171516.ZM2540@telluride> **** FINAL SCHEDULE **** NIPS*95 Workshop Neural Networks for Signal Processing Friday Dec 1, 1995 Marriott Vail Mountain Resort, Colorado ORGANIZERS Andrew D. Back C. Lee Giles and Bill G. Horne University of Queensland NEC Research Institute back at elec.uq.edu.au {giles,horne}@research.nj.nec.com WORKSHOP AIMS Nonlinear signal processing methods using neural network models form a topic of some recent interest. A common goal is for neural network models to outperform traditional linear and nonlinear models. Many researchers are interested in understanding, analysing and improving the performance of these nonlinear models by drawing from the well established base of linear systems theory and existing knowledge in other areas. How can this be best achieved? In the context of neural network models, a variety of methods have been proposed for capturing the time-dependence of signals. A common approach is to use recurrent connections or time-delays within the network structure. On the other hand, many signal processing techniques have been well developed over the last few decades. Recently, a strong interest has developed in understanding how better signal processing techniques can be developed by considering these different approaches. A major aim of this workshop is to obtain a better understanding of how well this development is proceeding. For example, the different model structures raise the question, "how suitable are the various neural networks for signal processing problems?". The success of some neural network models in signal processing problems indicate that they form a class of potentially powerful modeling methods, yet relatively little is understood about these architectures in the context of signal processing. As an outcome of the workshop it is intended that there should be a summary of current progress and goals for future work in this research area. SCHEDULE OF TALKS ** Session 1 - Speech Focus ** 7:30-7:40: Introduction 7:40-8:10: Herve Bourlard, ICSI and Faculte Polytechnique de Mons, "Hybrid use of hidden Markov models and neural networks for improving state-of-the-art speech recognition systems" 8:10-8:40: John Hogden, Los Alamos National Laboratory, "A maximum likelihood approach to estimating speech articulator positions from speech acoustics" 8:40-9:10: Shun-ichi Amari, A.Cichocki and H. Yang, RIKEN, "Blind separation of signals - Information geometric point of view" 9:10-9:30: Discussion ** Session 2 - Recurrent Network Focus ** 4:30-4:40: Introduction 4:40-5:10: Andrew Back, University of Queensland, "Issues in signal processing relevant to dynamic neural networks" 5:10-5:40: John Steele and Aaron Gordon, Colorado School of Mines, "Hierarchies of recurrent neural networks for signal interpretation with applications" 5:40-6:10: Stephen Piche, Pavilion Technologies, "Discrete Event Recurrent Neural Networks" 6:10-6:30: Open Forum, discussion time. For more information about the workshop see the workshop homepage: http://www.elec.uq.edu.au/~back/nips95ws/nips95ws.html or contact: Andrew D. Back Department of Electrical and Computer Engineering, University of Queensland, Brisbane, Qld 4072. Australia Ph: +61 7 365 3965 Fax: +61 7 365 4999 back at .elec.uq.edu.au C. Lee Giles, Bill G. Horne NEC Research Institute 4 Independence Way Princeton, NJ 08540. USA Ph: 609 951 2642, 2676 Fax: 609 951 2482 {giles,horne}@research.nj.nec.com -- Bill Horne horne at research.nj.nec.com http://www.neci.nj.nec.com/homepages/horne.html PHN: (609) 951-2676 FAX: (609) 951-2482 NEC Research Institute, 4 Independence Way, Princeton, NJ 08540 From rosen at unr.edu Fri Nov 17 18:59:23 1995 From: rosen at unr.edu (David B. Rosen) Date: Fri, 17 Nov 1995 15:59:23 -0800 Subject: Missing Data Workshop -- Final Announcement Message-ID: <199511180003.QAA24231@archive.ccs.unr.edu> This is the final email announcement (with updated list of presentations) for: MISSING DATA: METHODS AND MODELS A NIPS*95 Workshop Friday, December 1, 1995 INTRODUCTION Incomplete or missing data, typically unobserved or unavailable features in supervised learning, is an important problem often encountered in real-world data sets and applications. Assumptions about the missing-data mechanism are often not stated explicitly, for example independence between this mechanism and the values of the (missing or other) features themselves. In the important case of incomplete ~training~ data, one often discards incomplete rows or columns of the data matrix, throwing out some useful information along with the missing data. Ad hoc or univariate methods such as imputing the mean or mode are dangerous as they can sometimes give much worse results than simple discarding. Overcoming the problem of missing data often requires that we model not just the dependence of the output on the inputs, but the inputs among themselves as well. THE WORKSHOP This one-day workshop should provide a valuable opportunity to share and discuss methods and models used for missing data. The following short talks will be presented, with questions and discussion following each. o Leo Breiman, U.C. Berkeley Formal and ad hoc ways of handling missing data o Zoubin Ghahramani, U. Toronto Mixture models and missing data o Steffen Lauritzen and Bo Thiesson, Aalborg U. Learning Bayesian networks from incomplete data o Brian Ripley, Oxford U. Multiple imputation and simulation methods o Brian Ripley, Oxford U. Multiple imputation for neural nets and classification trees o Robert Tibshirani and Geoffrey Hinton, U. Toronto ``Coaching'' variables for regression and classification o Volker Tresp, Siemens AG Missing data: A fundamental problem in learning FURTHER INFORMATION The above is a snapshot of the workshop's web page: http://www.scs.unr.edu/~cbmr/nips/workshop-missing.html A schedule of presentation times is not yet available as of today. Sincerely, (the organizers:) Harry Burke David Rosen New York Medical College, Department of Medicine, Valhalla NY 10595 USA From at at cogsci.soton.ac.uk Sat Nov 18 05:34:40 1995 From: at at cogsci.soton.ac.uk (Adriaan Tijsseling) Date: Sat, 18 Nov 1995 10:34:40 GMT Subject: Changes to CogPsy Mailinglist Message-ID: <9511181034.AA05833@cogsci.soton.ac.uk> Dear Colleagues, There are a few changes to the CogPsy Mailinglist, but before I outline the changes, first a small description of the mailing list: The cogpsy mailing list is intended for discussion of issues and the dissemination of information important to researchers in all fields of cognitive science, especially connectionist cognitive psychology. Contributions could include: announcements of new techreports, dissertations, theses, conferences, seminars or courses discussions of research issues (including those arising from articles in Noetica) requests for information about bibliographic issues reviews of software or hardware packages The changes are: a new address: contributions should be send to cogpsy at neuro.psy.soton.ac.uk; subscriptions to cogpsy-request at neuro.psy.soton.ac.uk (make sure the Subject: field contains the word "subscribe"). a fusion with Noetica, a electronic journal on cognitive science, the url of which is: http://psych.psy.uq.oz.au/CogPsych/Noetica/ two accompanying webpages: one containing the latest contributions to the list, which is on http://www.soton.ac.uk/~coglab/coglab/CogPsy/ the other is a database of current projects in the field of cognitive science, listed by discipline and containing all information about the projects, including email- and URL-addresses. Feel free to add your own project!. It's on: http://neuro.psy.soton.ac.uk/~at/ With kind regards, Adriaan Tijsseling, CogPsy Moderator From koza at CS.Stanford.EDU Thu Nov 16 09:44:51 1995 From: koza at CS.Stanford.EDU (John Koza) Date: Thu, 16 Nov 95 6:44:51 PST Subject: CFP: Genetic Programming 1996 Conference (GP-96) Message-ID: ------------------------------------------------ Paper Submission Deadline: January 10, 1996 (Wednesday) ------------------------------------------------ CALL FOR PAPERS AND PARTICIPATION Genetic Programming 1996 Conference (GP-96) July 28 - 31 (Sunday - Wednesday), 1996 Fairchild Auditorium - Stanford University - Stanford, California Proceedings will be published by The MIT Press In cooperation with the Association for Computing Machinery (ACM), SIGART, the IEEE Neural Network Council, and the American Association for Artificial Intelligence. Genetic programming is a domain-independent method for evolving computer programs that solve, or approximately solve, problems. Starting with a primordial ooze of thousands of randomly created programs composed of functions and terminals appropriate to a problem, a genetic population is progressively evolved over many generations by applying the Darwinian principle of survival of the fittest, a sexual recombination operation, and occasional mutation. This first genetic programming conference will feature contributed papers, tutorials, invited speakers, and informal meetings. Topics include, but are not limited to, applications of genetic programming, theoretical foundations of genetic programming, implementation issues, parallelization techniques, technique extensions, implementations of memory and state, representation issues, new operators, architectural evolution, evolution of mental models, cellular encoding, evolution of machine language programs, evolvable hardware, combinations with other machine learning techniques, and relations to biology and cognitive systems. ------------------------------------------------- HONORARY CHAIR: John Holland, University of Michigan INVITED SPEAKERS: John Holland, University of Michigan and David E. Goldberg, University of Illinois GENERAL CHAIR: John Koza, Stanford University PUBLICITY CHAIR: Patrick Tufts, Brandeis University ------------------------------------------------- SPECIAL PROGRAM CHAIRS: The main focus of the conference (and about two-thirds of the papers) will be on genetic programming. In addition, papers describing recent developments in the following closely related areas of evolutionary computation (particularly those addressing issues common to various areas of evolutionary computation) will be reviewed by special program committees appointed and supervised by the following special program chairs. - GENETIC ALGORITHMS: David E. Goldberg, University of Illinois, Urbana, Illinois - CLASSIFIER SYSTEMS: Rick Riolo, University of Michigan - EVOLUTIONARY PROGRAMMING AND EVOLUTION STRATEGIES: David Fogel, University of California, San Diego, California ------------------------------------------------- TUTORIALS -Sunday July 28 9:15 AM - 11:30 AM - Genetic Algorithms - David E. Goldberg, University of Illinois - Machine Language Genetic Programming - Peter Nordin, University of Dortmund, Germany - Genetic Programming using Mathematica P Robert Nachbar P Merck Research Laboratories - Introduction to Genetic Programming - John Koza, Stanford University ------------------------------------------------- Sunday July 28 1:00 PM - 3: 15 PM - Classifier Systems- Robert Elliott Smith, University of Alabama - Evolutionary Computation for Constraint Optimization - Zbigniew Michalewicz, University of North Carolina - Advanced Genetic Programming - John Koza, Stanford University ------------------------------------------------- Sunday July 28 3:45 PM - 6 PM - Evolutionary Programming and Evolution Strategies - David Fogel, University of California, San Diego - Cellular Encoding P Frederic Gruau, Stanford University (via videotape) and David Andre, Stanford University (in person) - Genetic Programming with Linear Genomes (one hour) - Wolfgang Banzhaf, University of Dortmund, Germany -JECHO - Terry Jones, Santa Fe Institute ------------------------------------------------- Tuesday July 30 - 3 PM - 5:15PM - Neural Networks - David E. Rumelhart, Stanford University - Machine Learning - Pat Langley, Stanford University -JMolecular Biology for Computer Scientists - Russ B. Altman, Stanford University ------------------------------------------------- INFORMATION FOR SUBMITTING PAPERS The deadline for receipt at the physical mail address below of seven (7) copies of each submitted paper is Wednesday, January 10, 1996. Papers are to be in single-spaced, 12- point type on 8 1/2" x 11" or A4 paper (no e-mail or fax) with full 1" margins at top, bottom, left, and right. Papers are to contain ALL of the following 9 items, within a maximum of 10 pages, IN THIS ORDER: (1) title of paper, (2) author name(s), (3) author physical address(es), (4) author e-mail address(es), (5) author phone number(s), (6) a 100-200 word abstract of the paper, (7) the paper's category (chosen from one of the following five alternatives: genetic programming, genetic algorithms, classifier systems, evolutionary programming, or evolution strategies), (8) the text of the paper (including all figures and tables), and (9) bibliography. All other elements of the paper (e.g., acknowledgments, appendices, if any) must come within the maximum of 10 pages. Review criteria will include significance of the work, novelty, sufficiency of information to permit replication (if applicable), clarity, and writing quality. The first-named (or other designated) author will be notified of acceptance or rejection by approximately Monday February 26, 1996. The style of the camera-ready paper will be identical to that of the 1994 Simulation of Adaptive Behavior conference published by the MIT Press. Depending on the number, subject, and content of the submitted papers, the program committee may decide to allocate different number of pages to various accepted papers. The deadline for the camera-ready, revised version of accepted papers will be announced, but will be approximately Wednesday March 20, 1996. Proceedings will be published by The MIT Press and will be available at the conference (and, if requested, by priority mail to registered conference attendees with U.S. addresses just prior to the conference). One author will be expected to present each accepted paper at the conference. ------------------------------------------------- FOR MORE INFORMATION ABOUT THE GP-96 CONFERENCE: On the World Wide Web: http://www.cs.brandeis.edu/~zippy/gp-96.html or via e-mail at gp at aaai.org. Conference operated by Genetic Programming Conferences, Inc. (a California not-for-profit corporation). ------------------------------------------------- FOR MORE INFORMATION ABOUT GENETIC PROGRAMMING IN GENERAL: http://www-cs- faculty.stanford.edu/~koza/. ------------------------------------------------- FOR MORE INFORMATION ABOUT DISCOUNTED TRAVEL : For further information regarding special GP-96 airline and car rental rates, please contact Conventions in America at e-mail flycia at balboa.com; or phone 1-800-929-4242; or phone 619-678-3600; or FAX 619-678-3699. ------------------------------------------------- FOR MORE INFORMATION ABOUT THE SAN FRANCISCO BAY AREA AND SILICON VALLEY AREA SIGHTS: Try the Stanford University home page at http://www.stanford.edu/, the Hyperion Guide at http://www.hyperion.com/ba/sfbay.html; the Palo Alto weekly at http://www.service.com/PAW/home.html; the California Virtual Tourist at http://www.research.digital.com/SRC/virtual- tourist/California.html; and the Yahoo Guide of San Francisco at http://www.yahoo.com/Regional_Information/States/Califo rnia/San_Francisco. ------------------------------------------------- FOR MORE INFORMATION ABOUT CONTEMPORANEOUS WEST COAST CONFERENCES: Information about the AAAI-96 conference on August 4 P 8 (Sunday P Thursday), 1996, in Portland, Oregon can be found at http://www.aaai.org/. For information on the International Conference on Knowledge Discovery and Data Mining (KDD-96) in Portland, Oregon, on August 3- 5, 1996: http://www-aig.jpl.nasa.gov/kdd96. Information about the Foundations of Genetic Algorithms (FOGA) workshop on August 3 P 5 (Saturday P Monday), 1996, in San Diego, California can be found at http://www.aic.nrl.navy.mil/galist/foga/ or by contacting belew at cs.wisc.edu. ------------------------------------------------- FOR MORE INFORMATION ABOUT MEMBERSHIP IN THE ACM, AAAI, or IEEE: For information about ACM membership, try http://www.acm.org/; for information about SIGART, try http://sigart.acm.org/; for AAAI membership, go to http://www.aaai.org/; and for membership in the IEEE Computer Society, go to http://www.computer.org. ------------------------------------------------- PHYSICAL MAIL ADDRESS FOR GP-96: GP-96 Conference, c/o American Association for Artificial Intelligence, 445 Burgess Drive, Menlo Park, CA 94025. PHONE: 415-328-3123. FAX: 415-321-4457. WWW: http://www.aaai.org/. E-MAIL: gp at aaai.org. ------------------------------------------------ REGISTRATION FORM FOR GENETIC PROGRAMMING 1996 CONFERENCE TO BE HELD ON JULY 28 P 31, 1996 AT STANFORD UNIVERSITY First Name _________________________ Last Name_______________ Affiliation________________________________ Address__________________________________ ________________________________________ City__________________________ State/Province _________________ Zip/Postal Code____________________ Country__________________ Daytime telephone__________________________ E-Mail address_____________________________ Conference registration fee includes copy of proceedings, attendance at 4 tutorials of your choice, syllabus books for 4 tutorials, conference reception, and admission to conference. Students must send legible proof of full-time student status. Conference proceedings will be mailed to registered attendees with U.S. mailing addresses via 2-day U.S. priority mail 1 P 2 weeks prior to the conference at no extra charge (at addressee's risk). If you are uncertain as to whether you will be at that address at that time or DO NOT WANT YOUR PROCEEDINGS MAILED to you at the above address for any other reason, your copy of the proceedings will be held for you at the conference registration desk if you CHECK HERE ____. Postmarked by May 15, 1996: Student P ACM, IEEE, or AAAI Member $195 Regular P ACM, IEEE, or AAAI Member $395 Student P Non-member $215 Regular P Non-member $415 Postmarked by June 26, 1996: Student P ACM, IEEE, or AAAI Member $245 Regular P ACM, IEEE, or AAAI Member $445 Student P Non-member $265 Regular P Non-member $465 Postmarked later or on-site: Student P ACM, IEEE, or AAAI Member $295 Regular P ACM, IEEE, or AAAI Member $495 Student P Non-member $315 Regular P Non-member $515 Member number: ACM # ___________ IEEE # _________ AAAI # _________ Total fee (enter appropriate amount) $ _________ __ Check or money order made payable to "AAAI" (in U.S. funds) __ Mastercard __ Visa __ American Express Credit card number __________________________________________ Expiration Date ___________ Signature _________________________ TUTORIALS: Check off a box for one tutorial from each of the 4 columns: Sunday July 28, 1996 P 9:15 AM - 11:30 AM __ Genetic Algorithms __ Machine Language GP __ GP using Mathematica __ Introductory GP Sunday July 28, 1996 P 1:00 PM - 3: 15 PM __ Classifier Systems __ EC for Constraint Optimization __ Advanced GP Sunday July 28, 1996 P 3:45 PM - 6 PM __ Evolutionary Programming and Evolution Strategies __ Cellular Encoding __ GP with Linear Genomes __ ECHO Tuesday July 30, 1996 P3:00 PM - 5:15PM __ Neural Networks __ Machine Learning __ Molecular Biology for Computer Scientists __ Check here for information about housing and meal package at Stanford University. __ Check here for information on student travel grants. No refunds will be made; however, we will transfer your registration to a person you designate upon notification. SEND TO: GP-96 Conference, c/o American Association for Artificial Intelligence, 445 Burgess Drive, Menlo Park, CA 94025. PHONE: 415- 328-3123. FAX: 415-321-4457. E-MAIL: gp at aaai.org. WWW: http://www.aaai.org/. ------------------------------------------------- PROGRAM COMMITTEE Russell J. Abbott California State University, Los Angeles and The Aerospace Corporation Hojjat Adeli Ohio State University Dennis Allison Stanford University Lee Altenberg Hawaii Institute of Geophysics and Planetology David Andre Stanford University Peter J. Angeline Loral Federal Systems Wolfgang Banzhaf University of Dortmund, Germany Rik Belew University of California at San Diego Samy Bengio Centre National d'Etudes des Telecommunications, France Forrest H. Bennett III Genetic Algorithms Technology Corporation Scott Brave Stanford University Bill P. Buckles Tulane University Walter Cedeno Primavera Systems Inc. Nichael Lynn Cramer BBN System and Technologies Jason Daida University of Michigan Patrik D'haeseleer University of New Mexico Marco Dorigo Universite' Libre de Bruxelles Bertrand Daniel Dunay System Dynamics International Andrew N. Edmonds Science in Finance Ltd., UK H.H. Ehrenburg CWI, The Netherlands Frank D. Francone FRISEC P Francone & Raymond Institute for the Study of Evolutionary Computation, Germany Adam P. Fraser University of Salford Alex Fukunaga University of California, Los Angeles Frederic Gruau Stanford University Richard J. Hampo Ford Motor Company Simon Handley Stanford University Thomas D. Haynes The University of Tulsa Hitoshi Hemmi ATR, Kyoto, Japan Vasant Honavar Iowa State University Thomas Huang University of Illinois Hitoshi Iba Electrotechnical Laboratory, Japan Christian Andrew Johnson Department of Economics, University of Santiago Martin A. Keane Econometrics Inc. Mike Keith Allen Bradley Controls Maarten Keijzer Kenneth E. Kinnear, Jr. Adaptive Computing Technology W. B. Langdon University College, London David Levine Argonne National Laboratory Kenneth Marko Ford Motor Company Martin C. Martin Carnegie Mellon University Sidney R Maxwell III Nicholas Freitag McPhee University of Minnesota, Morris David Montana BBN System and Technologies Heinz Muehlenbein GMD Research Center, Germany Robert B. Nachbar Merck Research Laboratories Peter Nordin University of Dortmund, Germany Howard Oakley Institute of Naval Medicine, UK Franz Oppacher Carleton University, Ottawa Una-May O`Reilly Carleton University, Ottawa Michael Papka Argonne National Laboratory Timothy Perkis Frederick E. Petry Tulane University Bill Punch Michigan State University Justinian P. Rosca University of Rochester Conor Ryan University College Cork, Ireland Malcolm Shute University of Brighton, UK Eric V. Siegel Columbia University Karl Sims Andrew Singleton Creation Mechanics Lee Spector Hampshire College Walter Alden Tackett Neuromedia Astro Teller Carnegie Mellon University Marco Tomassini Ecole Polytechnique Federale de Lausanne Patrick Tufts Brandeis University V. Rao Vemuri University of Califonia at Davis Peter A. Whigham Australia Darrell Whitley Colorado State University Man Leung Wong Chinese University of Hong Kong Alden H. Wright University of Montana Byoung-Tak Zhang GMD, Germany From maass at igi.tu-graz.ac.at Sun Nov 19 11:31:42 1995 From: maass at igi.tu-graz.ac.at (Wolfgang Maass) Date: Sun, 19 Nov 95 17:31:42 +0100 Subject: computing with noisy spiking neurons: paper in neuroprose Message-ID: <199511191631.AA28958@figids02.tu-graz.ac.at> The file maass.noisy-spiking.ps.Z is now available for copying from the Neuroprose repository. This is a 9-page long paper. Hardcopies are not available. FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/maass.noisy-spiking.ps.Z On the Computational Power of Noisy Spiking Neurons Wolfgang Maass Institute for Theoretical Computer Science Technische Universitaet Graz Klosterwiesgasse 32/2 A-8010 Graz, Austria e-mail: maass at igi.tu-graz.ac.at Abstract This article provides positive results about the computational power of neural networks that are based on a neuron model ("noisy spiking neuron") which is acceptable to most neurobiologists as being reasonably realistic for a biological neuron. In fact: this model tends to underestimate the computational capabilities of a biological neuron, since it simplifies dendritic integration. Biological neurons communicate via spike-trains, i.e. via sequences of stereotyped pulses (spikes) that encode information in their time- differences ("temporal coding"). In addition it is wellknown that biological neurons are quite "noisy". There is some "jitter" in their firing times, and neurons (as well as synapses) my fail to fire with a certain probability. It has remained unknown whether one can in principle carry out reliable computation in networks of noisy spiking neurons. This article presents rigorous constructions for simulating in real-time arbitrary given boolean circuits and finite automata on such networks. In addition we show that with the help of "shunting inhibition" such networks can simulate in real-time any McCulloch-Pitts neuron (or "threshold gate"), and therefore any multilayer perceptron (or "threshold circuit") in a reliable manner. In view of the tremendous computational power of threshold circuits (even with few layers), this construction provides a possible explanation for the fact that biological neural systems can carry out quite complex computations within 100 msec. It turns out that the assumptions that these constructions require about the shape of the EPSP's and the behaviour of the noise are surprisingly weak. This article continues the related work from NIPS '94, where we had considered computations on networks of spiking neurons without noise. The current paper will appear in Advances in Neural Information Processing Systems, vol. 8 (= Proc. of NIPS '95) . ************ How to obtain a copy ***************** Via Anonymous FTP: unix> ftp archive.cis.ohio-state.edu Name: anonymous Password: (type your email address) ftp> cd pub/neuroprose ftp> binary ftp> get maass.noisy-spiking.ps.Z ftp> quit unix> uncompress maass.noisy-spiking.ps.Z unix> lpr maass.noisy-spiking.ps (or what you normally do to print PostScript) From piuri at elet.polimi.it Sun Nov 19 11:35:55 1995 From: piuri at elet.polimi.it (Vincenzo Piuri) Date: Sun, 19 Nov 1995 17:35:55 +0100 Subject: NICROSP'96 - call for papers Message-ID: <9511191635.AA06093@ipmel2.elet.polimi.it> ====================================================================== NICROSP'96 1996 International Workshop on Neural Networks for Identification, Control, Robotics, and Signal/Image Processing Venice, Italy - 21-23 August 1996 ====================================================================== Sponsored by the IEEE Computer Society and the IEEE CS Technical Committee on Pattern Analysis and Machine Intelligence. In cooperation with: ACM SIGART (pending), IEEE Circuits and Systems Society, IEEE Control Systems Society, IEEE Instrumentation and Measurement Society, IEEE Neural Network Council, IEEE North-Italy Section, IEEE Region 8, IEEE Robotics and Automation Society (pending), IEEE Signal Processing Society (pending), IEEE System, Man, and Cybernetics Society, IMACS, INNS (pending), ISCA, AEI, AICA, ANIPLA, FAST. CALL FOR PAPERS This workshop is directed to create a unique synergetic discussion forum and a strong link between theoretical researchers and practitioners in the application fields of identification, control, robotics, and signal/image processing by using neural techniques. The three-days single-session schedule will provide the ideal environment for in-depth analysis and discussions concerning the theoretical aspects of the applications and the use of neural networks in the practice. Invited talks in each area will provide a starting point for the discussion and give the state of the art in the corresponding field. Panels will provide an interactive discussion. Researchers and practitioners are invited to submit papers concerning theoretical foundations of neural computation, experimental results or practical applications related to the specific workshop's areas. Interested authors should submit a half-page abstract to the program chair by e-mail or fax by February 1, 1996, for review planning. Then, an extended summary or the full paper (limited to 20 double-spaced pages including figures and tables) must be sent to the program chair by February 16, 1996 (PostScript email submission is strongly encouraged). Submissions should contain: the corresponding author, affiliation, complete address, fax, email, and the preferred workshop track (identification, control, robotics, signal processing, image processing). Submission implies the willingness of at least one of the authors to register, attend the workshop and present the paper. Papers' selection is based on the full paper: the corresponding author will be notified by March 30, 1996. The camera-ready version, limited to 10 one-column IEEE-book-standard pages, is due by May 1, 1996. Proceedings will be published by the IEEE Computer Society Press. The extended version of selected papers will be considered for publication in special issues of international journals. General Chair Prof. Edgar Sanchez-Sinencio Department of Electrical Engineering Texas A&M University College Station, TX 77843-3128 USA phone (409) 845-7498 fax (409) 845-7161 email sanchez at eesun1.tamu.edu Program Chair Prof. Vincenzo Piuri Department of Electronics and Information Politecnico di Milano piazza L. da Vinci 32, I-20133 Milano, Italy phone +39-2-2399-3606 fax +39-2-2399-3411 email piuri at elet.polimi.it Publication Chair Dr. Jose' Pineda de Gyvez Department of Electrical Engineering Texas A&M University Publicity, Registr. & Local Arrangment Chair Dr. Cesare Alippi Department of Electronics and Information Politecnico di Milano Workshop Secretariat Ms. Laura Caldirola Department of Electronics and Information Politecnico di Milano phone +39-2-2399-3623 fax +39-2-2399-3411 email caldirol at elet.polimi.it Program Committee (preliminary list) Shun-Ichi Amari, University of Tokyo, Japan Magdy Bayoumi, University of Southwestern Louisiana, USA James C. Bezdek, University of West Florida, USA Pierre Borne, Ecole Politechnique de Lille, France Luiz Caloba, Universidad Federal de Rio de Janeiro, Brazil Chris De Silva, University of Western Australia, Australia Laurene Fausett, Florida Institute of Technology, USA C. Lee Giles, NEC, USA Karl Goser, University of Dortmund, Germany Simon Jones, University of Loughborough, UK Michael Jordan, Massachussets Institute of Technology, USA Robert J. Marks II, University of Washington, USA Jean D. Nicoud, EPFL, Switzerland Eros Pasero, Politecnico di Torino, Italy Emil M. Petriu, University of Ottawa, Canada Alberto Prieto, Universidad de Granada, Spain Gianguido Rizzotto, SGS-Thomson, Italy Edgar Sanchez-Sinencio, A&M University, USA Bernd Schuermann, Siemens, Germany Earl E. Swartzlander, University of Texas at Austin, USA Philip Treleaven, University College London, UK Kenzo Watanabe, Shizuoka University, Japan Michel Weinfeld, Ecole Politechnique de Paris, France ====================================================================== From bap at sloan.salk.edu Sun Nov 19 18:03:37 1995 From: bap at sloan.salk.edu (Barak Pearlmutter) Date: Sun, 19 Nov 1995 15:03:37 -0800 Subject: Response to no-free-lunch discussion In-Reply-To: <9511141824.AA04471@sfi.santafe.edu> (message from David Wolpert on Tue, 14 Nov 95 11:24:58 MST) Message-ID: <199511192303.PAA07609@valaga.salk.edu> Reviewing the theory of Kolmogorov complexity, we see that not having low Kolmogorov complexity is equivalent to being random. In other words, the minimal description of anything that does not have low Kolmogorov complexity is "nothing but noise." When you write this We have *no* a priori reason to believe that targets with "low Kolmogorov complexity" (or anything else) are/not likely to occur in the real world. it is precisely equivalent to saying We have *no* a priori reason to believe that targets with any non-random structure whatsoever are likely to occur in the real world. The NFL theory shows conclusively that there are no search algorithms which are particularly good at finding minima of such random functions. However, as scientists, we have had some success by positing the existence of non-random structure in the real world. So it seems to me that there is at least some reason to believe that the functions we optimize in practice are not completely random, in this Kolmogorov complexity sense. From bernabe at cnm.us.es Mon Nov 20 06:37:13 1995 From: bernabe at cnm.us.es (Bernabe Linares B.) Date: Mon, 20 Nov 95 12:37:13 +0100 Subject: No subject Message-ID: <9511201137.AA12650@cnm1.cnm.us.es> NIPS'95 WORKSHOP ANNOUNCEMENT Vail, Colorado. Friday, December 1st, 1995 Title: NEURAL HARDWARE ENGINEERING: From Sensory Data Adquisition to High-Level Intelligent Processing Organizers: Bernabe Linares-Barranco and Angel Rodriguez-Vazquez Dept. of Analog and Mixed-Signal Circuit Design Microelectronics National Center, Sevilla Ed. CICA, Av. Reina Mercedes s/n, 41012 Sevilla, SPAIN FAX: 34-5-4624506; Phone: 34-5-4239923; email: bernabe at cnm.us.es DESCRIPTION OF THE WORKSHOP: Developing hardware for neural applications is a task that hardware engineers have faced during the past years using two distinct main approaches: (a) producing "general purpose" digital neuro-computing systems or "neural- -accelerators" with a certain degree of flexibility to emulate different neural architectures and learning rules. (b) developing "special purpose" neuro-chips, mostly using analog or mixed analog/digital circuit techniques, intended to solve a specific problem with very high speed and efficiency. Usually hardware of task (b) is used for the front end of a neural processing system, such as sensory data (image/sound) adquisition and sometimes with some extra (pre)processing functionality (noise removal, automatic gain, dimensionality reduction). On the other hand, hardware of task (a) is employed for more "intelligent" or higher level processing such as learning, clustering, recognition, abstraction, and conceptualization. However, the limits between hardware of type (a) and (b) are not very clear, and as more hardware is developed the overlap between the two approaches increases. Digital technology provides larger accuracy in the realization of mathematical operations and offers great flexibility to change learning paradigms, learning rules, or to tune critical parameters. Analog technology, on the other hand, provides very high area and power efficiency, but is less accurate and flexible. It is clear that people that are developing neural algorithms need to have some type of digital neurocomputer system where they can change rapidly the neural architecture, the topology, the learning rules, try different mathematical functions, and all that with sufficient flexibility and computing power. On the other hand, when neural systems require image or sound adquisition capabilities (retinas or cochleas) analog technology offers very high power and chip area efficiency, so that this approach seems to be the preferred one. However, what happens when it comes to develop a hardware system that needs to handle the sensory data, perform some basic processing, and continue processing up to higher level stages where data segmentation has to be performed, recognition on the segments has to be achieved, and learning and abstraction should be fulfilled? Is there any clear border among analog and digital techniques as we proceed upwards in the processing cycle from signal acquisition to conceptualization? Is it possible to take advantage of the synergy between analog and digital? How? Are these conclusions the same for vision, hearing, olfactory, or intelligent control applications? We believe it is a good point in time to make a debate between representatives of the two approaches, since both have evolved independently into a large enough degree of development and maturity as to enable pros and counters be discussed on the basis of objective, rather than subjective considerations. LIST OF SPEAKERS: 1. Nelson Morgan, University of California, Berkeley, U.S.A. "Using A Fixed-point Vector Microprocessor for Connectionist Speech Recognition Training" 2. Yuzo Hirai, Institute of Information Sciences and Electronics, University of Tsukuba, Japan. "PDM Digital Neural Networks" 3. Taher Daud, Jet Propulsion Laboratory, Pasadena, California, U.S.A. "Focal Plane Imaging Array-Integrated 3-D Neuroprocessor" 4. Ulrich Ramacher, University of Technology, Dresden, Germany. "The Siemens Electronic Eye Proyect" 5. Xavier Arreguit, CSEM, Neuchatel, Switzerland "Analog VLSI for Perceptive Systems" 6. Andreas Andreou, John Hopkins University, Baltimore, Maryland, U.S.A. "Silicon Retinas for Contrast Sensitivity and Polarization Sensing" 7. Marwan Jabri, University of Sydney, Australia. "On-Chip Learning in Analog VLSI and its Application in Biomedical Implant Devices" 8. Tadashi Shibata, Tohoku University, Sendai, Japan. "Neuron-MOS Binary-Analog Merged Hardware Computation for Intelligent Information Processing" From stefano at kant.irmkant.rm.cnr.it Mon Nov 20 13:15:32 1995 From: stefano at kant.irmkant.rm.cnr.it (stefano@kant.irmkant.rm.cnr.it) Date: Mon, 20 Nov 1995 18:15:32 GMT Subject: Paper available: Learning to adapt to changing environments Message-ID: <9511201815.AA16840@kant.irmkant.rm.cnr.it> Papers available via WWW / FTP: Keywords: Learning, Adaptation to changing environments, Evolutionary Robotics Neural Networks, Genetic Algorithms, ------------------------------------------------------------------------------ LEARNING TO ADAPT TO CHANGING ENVIRONMENTS IN EVOLVING NEURAL NETWORKS Stefano Nolfi & Domenico Parisi Institute of Psychology, C.N.R., Rome. In order to study learning as an adaptive process it is necessary to take into consideration the role of evolution which is the primary adaptive process. In addition, learning should be studied in (artificial) organisms that live in an independent physical environment in such a way that the input from the environment can be at least partially controlled by the organisms' behavior. To explore these issues we used a genetic algorithm to simulate the evolution of a population of neural networks each controlling the behavior of a small mobile robot that must explore efficiently an environment surrounded by walls. Since the environment changes from one generation to the next each network must learn during its life to adapt to the particular environment it happens to be born in. We found that evolved networks incorporate a genetically inherited predisposition to learn that can be described as: (a) the presence of initial conditions that tend to canalize learning in the right directions; (b) the tendency to behave in a way that enhances the perceived differences between different environments and determines input stimuli that facilitate the learning of adaptive changes, and (c) the ability to reach desirable stable states. http://kant.irmkant.rm.cnr.it/public.html or ftp-server: kant.irmkant.rm.cnr.it (150.146.7.5) ftp-file : /pub/econets/nolfi.changing.ps.Z for the homepage of our research group with most of our publications available online and pointers to ALIFE resources see: http://kant.irmkant.rm.cnr.it/gral.html ---------------------------------------------------------------------------- Stefano Nolfi Institute of Psychology National Research Council e-mail: stefano at kant.irmkant.rm.cnr.it From dhw at santafe.edu Mon Nov 20 13:54:43 1995 From: dhw at santafe.edu (David Wolpert) Date: Mon, 20 Nov 95 11:54:43 MST Subject: Some more on NFL Message-ID: <9511201854.AA11261@sfi.santafe.edu> Barak Pearlmutter writes: >>> not having low Kolmogorov complexity is equivalent to being random... (the) description of anything that does not have low Kolmogorov complexity is "nothing but noise." >>> I don't necessarily disagree with such sentiments, but one should definitely be careful about them; there are *many* definitions of "random". (Seth Lloyd has counted about 30 versions of its flip side, "amount of information".) High Kolmogorov complexity is only one of them. To illustrate just one of the possible objections to measuring randomness with Kolmogorov complexity: Would you say that a macroscopic gas with a specified temperature is "random"? To describe it exactly takes a huge Kolmogorov complexity. And certainly in many regards its position in phase space is "nothing but noise". (Indeed, in a formal sense, its position is a random sample of the Boltzmann distribution.) Yet Physicists can (and do) make extraordinarilly accurate predictions about such creatures with ease. Another important (and related) point is that encoding constant that gets buried in the definition of Kolmogorov complexity - in practive it can be very important. To put it another way, one person's "random" is another person's "highly regular"; this is precisely why the basis you use in supervised learning matters so much. >>> as scientists, we have had some success by positing the existence of non-random structure in the real world. So it seems to me that there is at least some reason to believe that the functions we optimize in practice are not completely random, in this Kolmogorov complexity sense. >>> Oh, most definitely. To give one simple example: Cross-validation works quite well in practice. However either 1) for *any* fixed target (i.e., for any prior over targets), averaged over all sets of generalizers you're choosing among, cross-validation works no better than anti-cross-validation (choose the generalizer in the set at hand having *largest* cross-validation error); and 2) for a fixed set of generalizers, averaged over all targets, it works no better than anti-cross-validation. So for cross-validation to work requires a very subtle inter-relationship between the prior over targets and the set of generalizers you're choosing among. In particular, cross-validation cannot be given a Bayesian justification without regard to the set of generalizers. Nonetheless, I (and every other evenly marginally rational statistician) have used cross-validation in the past, and will do so again in the future. From marshall at cs.unc.edu Mon Nov 20 12:58:45 1995 From: marshall at cs.unc.edu (Jonathan Marshall) Date: Mon, 20 Nov 1995 13:58:45 -0400 Subject: CFP: Biologically Inspired Autonomous Systems Message-ID: <199511201758.NAA20951@marshall.cs.unc.edu> ---------------------------------------------------------------------------- Biologically Inspired Autonomous Systems: Computation, Cognition, and Action March 4-5, 1996 Washington Duke Hotel (Duke University) Durham, North Carolina Co-Sponsored by the Duke University Departments of Electrical and Computer Engineering, Neurobiology, Biomedical Engineering, and Experimental Psychology The dramatic evolution of computer technology has caused a return to the biological paradigms which inspired many of the early pioneers of information science such as John von Neumann, Stephen Kleene and Marvin Minsky. Similarly, many fields of the life and human sciences have been influenced by paradigms initiated in systems theory, computation and control Engineering. The purpose of this workshop is to pursue this fruitful interaction of engineering and the exact sciences, with the life and human sciences, by investigating the processes which can provide systems, both artificial and natural, with autonomous and adaptive behavior. Topics of interest include Autonomous behavior of biophysically and cognitively inspired models Autonomous agents and mobile systems Collective behaviour by semi-autonomous agents Self repair and regeneration in computational and artificial structures Autonomous image understanding Brain imaging and Functional MRI Keynote Speakers: Stephen Grossberg (Boston University), Daniel Mange (EPFL), Jean-Arcady Meyer (ENS, Paris), Heinz Muehlenbein (GMD, Bonn), John Taylor (University College, London). Speakers will include: Paul Bourgine (Ecole Polytechnique), Bernadette Dorizzi (INT, Evry), Warren Hall (Duke), Ivan Havel (Center for Theoretical Studies and Prague University), Petr Lansky (Center for Theoretical Studies and Prague University), Miguel Nicollelis (Duke), Richard Palmer (Duke), David Rubin (Duke) , Nestor Schmajuk (Duke), John Staddon (Duke), John Taylor (University College, London), Ed Ueberbacher (Oak Ridge National Laboratory), Paul Werbos (NSF). Paper submissions, in the form of four page extended abstracts, are solicited in areas of relevance to this workshop. They should be sent before January 15, 1996 to one of the workshop Co-Chairs. The Workshop Proceedings will be published in book form with full papers. Workshop Co-Chairs: Erol Gelenbe Nestor Schmajuk Department of Electrical and Department of Experimental Psychology Computer Engineering Duke University Duke University Durham, NC 27708, USA Durham, NC 27708-0291, USA nestor at acpub.duke.edu erol at ee.duke.edu ---------------------------------------------------------------------------- From niranjan at eng.cam.ac.uk Mon Nov 20 23:56:44 1995 From: niranjan at eng.cam.ac.uk (niranjan@eng.cam.ac.uk) Date: Tue, 21 Nov 95 04:56:44 GMT Subject: JOB JOB JOB Message-ID: <9511210456.9837@baby.eng.cam.ac.uk> -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- Research Assistant Position for One Year A Research Assistant position is available in Cambridge to investigate the use of: Neural Networks in the Prediction of Risk in Pregnancy Euro-PUNCH is a collaborative Project funded by the Human Capital and Mobility Programme of the Commission of the European Communities. Thus, the post is available only to a citizen of a European Union Member State (but not British), who wishes to come to work in the United Kingdom. From Olivier.Michel at alto.unice.fr Tue Nov 21 10:02:52 1995 From: Olivier.Michel at alto.unice.fr (Olivier MICHEL) Date: Tue, 21 Nov 1995 16:02:52 +0100 Subject: Announcement: Khepera Simulator 1.0 Message-ID: <199511211502.QAA16789@alto.unice.fr> * ANNOUNCEMENT OF NEW PUBLIC DOMAIN SOFTWARE PACKAGE * ------------------------------------------------------------- - Khepera Simulator version 1.0 - ------------------------------------------------------------- Khepera Simulator is a public domain software package written by Olivier MICHEL. It allows to write your own controller for the mobile robot Khepera using C or C++ languages, to test them in a simulated environment and features a nice colorful X11 graphical interface. Moreover, if you own a Khepera robot, it can drive the real robot using the same control algorithm. It is mainly destinated to researchers studying autonomous agents. o Requirements: UNIX system, X11 library. o User Manual and examples of controllers included. o This software is free of charge for research and teaching. o Commercial use is forbidden. o Khepera is a mini mobile robot developped at EPFL by Francesco Mondada, Edo Franzi and Andre Guignard (K-Team). o You can download it from the following web site: http://wwwi3s.unice.fr/~om/khep-sim.html Olivier MICHEL om at alto.unice.fr http://wwwi3s.unice.fr/~om/ From tds at ai.mit.edu Tue Nov 21 12:10:47 1995 From: tds at ai.mit.edu (Terence D. Sanger) Date: Tue, 21 Nov 95 12:10:47 EST Subject: NIPS workshop announcement Message-ID: <9511211710.AA08251@dentate.ai.mit.edu> NIPS*95 Post-Conference Workshop "Vertebrate Neurophysiology and Neural Networks: Can the teacher learn from the student?" Saturday December 2, 7:30-9:30AM, 4:30-6:30PM Organizer: Terence Sanger, MIT. SUMMARY Results from neurophysiological investigations continue to guide the development of artificial neural network models that have been shown to have wide applicability in solving difficult computational problems. This workshop addresses the question of whether artificial neural network models can be applied to understanding neurophysiological results and guiding further experimental investigations. Recent work on close modelling of vertebrate neurophysiology will be presented, so as to give a survey of some of the results in this field. We will concentrate on examples for which artificial neural network models have been constructed to mimic the structure as well as the function of their biological counterparts. Clearly, this can be done at many different levels of abstraction. The goal is to discuss models that have explanatory and predictive power for neurophysiology. The following questions will serve as general discussion topics: 1. Do artificial neural network models have any relationship to ``real'' Neurophysiology? 2. Have any such models been used to guide new biological research? 3. Is Neurophysiology really useful for designing artificial networks, or does it just provide a vague ``inspiration''? 4. How faithfully do models need to address ultrastructural or membrane properties of neurons and neural circuits in order to generate realistic predictions of function? 5. Are there any artificial network models that have applicability across different regions of the central nervous system devoted to varied sensory and motor modalities? 6. To what extent do theoretical models address more than one of David Marr's levels of algorithmic abstraction (general approach, specific algorithm, and hardware implementation)? Selected examples of Neural Network models for Neurophysiological results will be presented, and active audience participation and discussion will be encouraged. SCHEDULE Saturday, December 2 7:30 - T. Sanger: Introduction and Overview 8:00 - T. Sejnowski: "Bee Foraging in Uncertain Environments using Predictive Hebbian Learning" 8:30 - A. Pouget: "Spatial Representations in the Parietal Cortex may use Basis Functions" 9:00 - Discussion --- Break --- 4:30 - S. Giszter: "Spinal Primitives and their Dynamics in Vertebrate Limb Control: A Biological Perspective" 5:00 - G. Goodhill: "Modelling the Development of Primary Visual Cortex: Determinants of Ocular Dominance Column Periodicity" 5:30 - Discussion From prechelt at ira.uka.de Tue Nov 21 13:34:24 1995 From: prechelt at ira.uka.de (Lutz Prechelt) Date: Tue, 21 Nov 1995 19:34:24 +0100 Subject: NIPS Workshop: Benchmarking of NN learning algorithms Message-ID: <"iraun1.ira.532:21.11.95.18.34.54"@ira.uka.de> X-URL: http://wwwipd.ira.uka.de/~prechelt/NIPS_bench.html NIPS Workshop: Benchmarking of NN learning algorithms ******************************************************** Abstract: Proper benchmarking of neural network learning architectures is a prerequisite for orderly progress in this field. In many published papers deficiencies can be observed in the benchmarking that is performed. The workshop addresses the status quo of benchmarking, common errors and how to avoid them, currently existing benchmark collections, and, most prominently, a new benchmarking facility including a results database. The workshop goal is to improve benchmarking practices and to improve the comparability of benchmark tests. Workshop Chairs: o Thomas G. Dietterich , o Geoffrey Hinton , o Wolfgang Maass , o Lutz Prechelt [communicating chair] o Terry Sejnowski From caruana+ at cs.cmu.edu Tue Nov 21 14:20:09 1995 From: caruana+ at cs.cmu.edu (Rich Caruana) Date: Tue, 21 Nov 95 14:20:09 EST Subject: NIPS*95 Workshop on Transfer in Inductive Systems Message-ID: <9072.816981609@GS79.SP.CS.CMU.EDU> Post-NIPS*95 Workshop, December 1-2, 1995, Vail, Colorado TITLE: "Learning to Learn: Knowledge Consolidation and Transfer in Inductive Systems" ORGANIZERS: Rich Caruana (co-chair), Danny Silver (co-chair), Jon Baxter, Tom Mitchell, Lori Pratt, Sebastian Thrun INVITED TALKS BY: Leo Breiman (Berkeley) Tom Mitchell (CMU) Tomaso Poggio (MIT) Noel Sharkey (Sheffield) Jude Shavlik (Wisconsin) WEB PAGE: http://www.cs.cmu.edu/afs/cs/usr/caruana/pub/transfer.html DESCRIPTION: Because the power of tabula rasa learning is limited, interest is increasing in methods that capitalize on previously acquired domain knowledge. Examples of these methods include: o using symbolic domain theories to bias connectionist networks o using extra outputs on a connectionist network to bias the hidden layer representation towards more predictive features o using unsupervised learning on a large corpus of unlabelled data to learn features useful for subsequent supervised learning on a smaller labelled corpus o using models previously learned for other problems as a bias when learning new, but related, problems There are many approaches: hints, knowledge-based artificial neural nets (KBANN), explanation-based neural nets (EBNN), multitask learning (MTL), knowledge consolidation, ... What they all have in common is the attempt to transfer knowledge from other sources to benefit the current inductive task. The goal of this workshop is to provide an opportunity for researchers and practitioners to discuss problems and progress in knowledge transfer in learning. We hope to identify research directions, debate theories and approaches, discover unifying principles, and begin to start answering questions like: o when will transfer help -- or hinder? o what should be transferred and how? o what are the benefits of transfer? o in what domains is transfer most useful? o is there evidence for transfer in nature? From avrama at wo.erim.org Wed Nov 8 22:03:20 1995 From: avrama at wo.erim.org (Avrama Blackwell) Date: Tue, 21 Nov 1995 15:03:20 +30000 Subject: position open Message-ID: PRE- OR POST-DOCTORAL FELLOWSHIP IN NEUROCOMPUTING Applications are invited for the position of Pre- or Post-doctoral Fellow. The Fellow will be an integral member of a team collaborating with NIH/NINDS in the development of advanced models of associative learning and visual information processing. Position requires interest and experience in computational neurobiology or development of neural network models as well as good 'C' or 'C++' programming skills. For a review of recent activities of the group see: Alkon et al. In Neural Networks 7: 1005 (1994). The appointment, for one year with possibility of renewal, will be a joint appointment at ERIM (Environmental Research Institute of Michigan, Washington Office) and George Mason University. If interested, either contact Tom Vogl at the NIPS conference in Denver (Dr. Vogl will NOT be at the Workshops) or send statement of interest and CV to tvogl at erim.org. From omlinc at research.nj.nec.com Tue Nov 21 15:53:59 1995 From: omlinc at research.nj.nec.com (Christian Omlin) Date: Tue, 21 Nov 1995 15:53:59 -0500 Subject: paper available Message-ID: <199511212053.PAA12137@arosa> The following paper is available on the website http://www.neci.nj.nec.com/homepages/omlin/omlin.html The paper gives an overview of our work and contains an extensive bibliography on the representation of discrete dynamical systems in recurrent neural networks. -Christian =================================================================== Learning, Representation, and Synthesis of Discrete Dynamical Systems in Continuous Recurrent Neural Networks (*) C. Lee Giles (a,b) and Christian W. Omlin (a) (a) NEC Research Institute 4 Independence Way Princeton, NJ 08540 (b) Institute for Advanced Computer Studies University of Maryland College Park, MD 20742 ABSTRACT This paper gives an overview on learning and representation of discrete-time, discrete-space dynamical systems in discrete-time, continuous-space recurrent neural networks. We limit our discussion to dynamical systems (recurrent neural networks) which can be represented as finite-state machines (e.g. discrete event systems ). In particular, we discuss how a symbolic representation of the learned states and dynamics can be extracted from trained neural networks, and how (partially) known deterministic finite-state automata (DFAs) can be encoded in recurrent networks. While the DFAs that can be learned exactly with recurrent neural networks are generally small (on the order of 20 states), there exist subclasses of DFAs with on the order of 1000 states that can be learned by small recurrent networks. However, recent work in natural language processing implies that recurrent networks can possibly learn larger state systems. (*) Appeared in Proceedings of the IEEE Workshop on Architectures for Semiotic Modeling and Situation Analysis in Large Complex Systems, Monterey, CA, August 27-29, 1995. Copyright IEEE Press. From heckerma at microsoft.com Tue Nov 21 14:55:14 1995 From: heckerma at microsoft.com (David Heckerman) Date: Tue, 21 Nov 95 14:55:14 TZ Subject: NIPS95 workshop Message-ID: <199511212254.OAA29657@imail1.microsoft.com> **** TENTATIVE SCHEDULE **** NIPS*95 Workshop Learning in Bayesian Networks and Other Graphical Models Friday and Saturday, December 1-2, 1995 Marriott Vail Mountain Resort, Colorado http://www.cs.cmu.edu/afs/cs/project/cnbc/nips/NIPS.html (conference) http://www.ai.mit.edu/people/jordan/workshop.html (workshop) Topic and Purpose of the Workshop: A Bayesian network is a directed graphical representation of probabilistic relationships that people find easy to understand and use, often because the relationships have a causal interpretation. A network for a given domain defines a joint probability distribution for that domain, and algorithms exist for efficiently manipulating the joint distribution to determine probability distributions of interest. Over the last decade, the Bayesian network has become a popular representation for encoding uncertain expert knowledge in expert systems. More recently, researchers have developed methods for learning Bayesian networks from data. These approaches will be the focus of this workshop. Issues to be discussed include (1) the opposing roles of prediction and explanation; (2) search, model selection, and capacity control; (3) representation issues, including extensions of the Bayesian network (e.g., chain graphs), the role of ``hidden'' or ``latent'' variables in learning, the modeling of temporal systems, and the assessment of priors; (4) optimization and approximation methods, including gradient-based methods, EM algorithms, stochastic sampling, and the mean field algorithms. In addition, we plan to discuss known relationships among Bayesian networks, Markov random fields, Boltzmann machines, loglinear models for contingency tables, Hidden Markov models, decision trees, and feedforward neural networks, as well as to uncover previously unknown ties. Tentative Schedule: Friday, Dec 1 ------------- 7:30am - 8:50am Tutorial on Graphical Models Ross Shachter, Stanford University Bruce D'Ambrosio, Oregon State University Michael Jordan, MIT 8:50am - 9:30am Decomposable graphical models and their use in learning algorithms Steffen Lauritzen, Aalborg University Decomposable, or triangulated, graphical models occur for example as basic computational structures in probabilistic expert systems. I will first give a brief description of properties of decomposable graphical models and then present some unfinished ideas about their possible use in automatic learning procedures. 9:30am - 9:40am break 9:40am - 10:20am Likelihoods and Priors for Learning Bayesian Networks David Heckerman, Microsoft Dan Geiger, Technion I will discuss simple methods for constructing likelihoods and parameter priors for learning about the parameters and structure of a Bayesian network. In particular, I will introduce several assumptions that permit the construction of likelihoods and parameter priors for a large number of Bayesian-network structures from a small set of assessments. Two notable assumptions are parameter independence, which says that the parameters associated with each variable in a structure are independent, and likelihood equivalence, which (roughly speaking) says that data should not help to discriminate structures that represent the same assertions of conditional independence. In addition to explicating methods for likelihood and prior construction, I will show how the assumptions lead to characterizations of well-known prior distributions for the parameters of multivariate distributions. For example, when the joint likelihood is an unrestricted discrete distribution, parameter independence and likelihood equivalence imply that the parameter prior must be a Dirichlet distribution. 10:20am - 11:00am Bayesian model averaging for Markov equivalence classes of acyclic digraphs David Madigan, Michael D. Perlman, and Chris T. Volinsky, University of Washington, Seattle Acyclic digraphs (ADGs) are widely used to describe dependencies among variables in multivariate distributions. There may, however, be many ADGs that determine the same dependence (= Markov) model. Thus, the family of all ADGs with a given set of vertices is naturally partitioned into Markov-equivalence classes, each class being associated with a unique statistical model. Statistical procedures, such as model selection or model averaging, that fail to take into account these equivalence classes, may incur substantial computational or other inefficiencies. Recent results have shown that each Markov-equivalence class is uniquely determined by a single chain graph, the essential graph, that is itself Markov-equivalent simultaneously to all ADGs in the equivalence class. Here we propose two stochastic Bayesian model averaging and selection algorithms for essential graphs and apply them to the analysis of a number of discrete- variable data sets. 11:00am - 11:40am discussion ----mid-day break---- 4:30pm - 5:10pm Automated Causal Inference Peter Spirtes, Carnegie Mellon University Directed acyclic graphs can be used to represent both families of probability distributions and causal relationships. We introduce two axioms (Markov and Faithfulness) that are widely but implicitly assumed by statisticians and that relate causal structures with families of probability distributions. We then use these axioms to develop algorithms that infer some features of causal graphs given a probability distribution and optional background knowledge as input. The algorithms are correct in the large sample limit, even when latent variables and selection bias may be present. In the worst case, the algorithms are exponential, but in many cases they has been able to handle up to 100 variables. We will also present Monte Carlo simulation results on various sample sizes. 5:10pm - 5:50pm HELMHOLTZ MACHINES Geoffrey E. Hinton, Peter Dayan, Brendan Frey, Radford Neal University of Toronto and MIT For hierarchical generative models that use distributed representations in their hidden variables, there are exponentially many ways in which the model can produce each data point. It is therefore intractable to compute the posterior distribution over the hidden distributed representations given a datapoint and so there is no obvious way to use EM or gradient methods for fitting the model to data. A Helmholtz machine consists of a generative model that uses distributed representations and a recognition model that computes an approximation to the posterior distribution over representations. The machine is trained to minimize a Helmholtz free energy which is equal to the negative log probability of the data if the recognition model computes the correct posterior distribution. If the recognition model computes a more tractable, but incorrect distribution, the Helmholtz free energy is an upper bound on the negative log probability of the data, so it acts as a tractable and useful Lyapunov function for learning a good generative model. It also encourages generative models that give rise to nice simple posterior distributions, which makes perception a lot easier. Several different methods have been developed for minimizing the Helmholtz free energy. I will focus on the "wake-sleep" algorithm, which is easy to implement with neurons, and give some examples of it learning probability density functions in high dimensional spaces. 5:50pm - 6:30pm Bounding Log Likelihoods in Sigmoid Belief Networks Lawrence K. Saul, Tommi Jaakkola, and Michael I. Jordan, MIT Sigmoid belief nets define graphical models with useful probabilistic semantics. We show how to calculate a lower bound on the log likelihood of any partial instantiation of a sigmoid belief net. The bound can be used as a basis for inference and learning provided it is sufficiently tight; in practice we have found this often to be the case. The bound is computed by approximating the true posterior distribution over uninstantiated nodes, $P$, by a more tractable distribution, $Q$. Parameterized forms for $Q$ include factorial distributions, mixture models, and hierarchical distributions that exploit the presence of tractable substructures in the original belief net. Saturday, Dec 2 --------------- 7:30am - 8:10am A Method for Learning the Structure of a Neural Network from Data Gregory F. Cooper and Sankaran Rajagopalan, University of Pittsburgh A neural network can be viewed as consisting of a set of arcs and a parameterization of those arcs. Most neural network learning has focused on parameterizing a user-specified set of arcs. Relatively less work has addressed automatically learning from data which arcs to include in the network (i.e., the neural network structure). We will present a method for learning neural network structures from data (call it LNNS). The method takes as input a database and a set of priors over possible neural network structures, and it outputs the neural network structure that is most probable as found by a heuristic search procedure. We will describe the close relationship between the LNNS method and current Bayesian methods for learning Bayesian belief networks. We also will show how we can apply the LNNS method to learn hidden nodes in Bayesian belief networks. 8:10am - 8:50am Local learning in probabilistic networks with hidden variables Stuart Russell, John Binder, Daphne Koller, Keiji Kanazawa University of California, Berkeley We show that general probabilistic (Bayesian) networks with fixed structure containing hidden variables can be learned automatically from data using a gradient-descent mechanism similar to that used in neural networks. The gradient can be computed locally from information available as a direct by-product of the normal inference process in the network. Because probabilistic networks provide explicit representations of causal structure, human experts can easily contribute prior knowledge to the training process, thereby significantly improving the sample complexity. The method can be extended to networks with intensionally represented distributions, including networks with continuous variables and dynamic probabilistic networks (DPNs). Because DPNs provide a decomposed representation of state, they may have some advantages over HMMs as a way of learning certain types of stochastic temporal processes with hidden state. 8:50am - 9:30am Asymptotic Bayes Factors for Directed Networks with Hidden Variables Dan Geiger, Technion David Heckerman, Microsoft Chris Meek, Carnegie Mellon University 9:30am - 9:40am break 9:40am - 10:20am Bayesian Estimation of Gaussian Bayes Networks Richard Scheines, Carnegie Mellon University The Gibbs sampler can be used to draw a sample from the posterior distribution over the parameters in a linear causal model ( Gaussian network, structural equation model, or LISREL model). I show how this can be done, and provide several examples that demonstrate its utility. These include: estimating under-identified models and making correct small sample inferences about the parameters when the likelihood surface is non-normal, contrary to the assumptions of the asymptotic theory that all current techniques rely on. In fact the likelihood surface for structural equation models (SEMs) with latent variables is often multi-modal. I give an example of such a case, and show how the Gibbs sampler is still informative and useful in this case in contrast to standard SEM software like LISREL. 10:20am - 11:00am Learning Stochastic Grammars Stephen M. Omohundro Bayesian networks represent the distribution of a fixed set of random variables using conditional independence information. Many important application domains (eg. speech, vision, planning, etc.) have state spaces that don't naturally decompose into a fixed set of random variables. In this talk, I'll present stochastic grammars as a tractable class of probabilistic models over this kind of domain. I'll describe an algorithm by Stolcke and myself for learning Hidden Markov Models and stochastic regular grammars from training strings which is closely related to algorithms for learning Bayesian networks. I'll discuss connections between stochastic grammars and graphical probability models and argue for the need for richer structures which encompass certain properties of both classes of model. 11:00am - 11:20am Variable selection using the theory of Bayesian networks Christopher Meek, Carnegie Mellon University This talk will describe two approaches to variable selection based upon the theory of Bayesian networks. In a study about the prediction of mortality in hospital patients with pneumonia these two methods were used to select a set of variables upon which predictive models were developed. In addition several neural network methods were used to develop predictive models from the same large database of cases. The mortality models developed in this study are distinguished more by the number of variables and parameters that they contain than by their error rates. I will offer an explanation of why the two methods described lead to small sets of variables and models with fewer parameters. In addition the variable sets selected by the Bayesian network approaches are amenable to future implementation in a paper-based form and, for several of the models, are strikingly similar to variable sets hand selected by physicians. 11:20am - 11:40am Methods for Learning Hidden Variables Joel Martin Hidden variables can be learned in many ways. Which method should be used? Some have better theoretical justifications, some seem to have better pragmatic justifications. I will describe a simple class of probabilistic models and will compare several methods for learn hidden variables from data. The methods compared are EM, gradient descent, simulated annealing, genetic search, and a variety of incremental techniques. I will discuss the results by considering particular applications. ----mid-day break---- 4:30pm - 5:10pm Brains, Nets, Feedback and Time Series Clark Glymour, Thomas Richardson and Peter Spirtes, Carnegie Mellon Neural networks have been used extensively to model the differences between normal cognitive behavior and the behavior of brain damaged subjects. A network is trained to simulate normal behavior, lesioned, and then simulates, or under re-training simulates, brain-damaged behavior. A common objection to this explanatory strategy (see for example comments on Martha Farah's recent contribution in Behavioral and Brain Sciences) is that it can "explain anything." The complaint alleges that for any mathematically possible pairing of normal and brain-damaged behavior, there exists a neural net that simulates the normal and when lesioned simulates the abnormal. Is that so? We can model the normal and abnormal behaviors as probability distributions on a set of nodes, and the network as a set of simultaneous (generally non-recursive) equations with independent noises. The network and joint probability distribution then describe a cyclic graph and associated probability distribution. What is the connection between the probability distribution and the graph topology? It is easy to show that the (local) Markov condition fails for linear networks of this sort. Spirtes (and independently J. Koster, in a forthcoming paper in Annals of Statistics) has shown that the conditional independencies implied by a linear cyclic network are characterized by d-separation (in either Pearl's or Lauritzen's versions). Spirtes has also shown that d-separation fails for non-linear cyclic networks with independent errors, but has given another characterization for the non-linear case. These results can be directly applied to the methodological disputes about brains and neural net models. Assuming faithfulness, it follows that a lesioned neural net represented as a cyclic Bayes net, whether linear or non-linear, must preserve the conditional independence relations in the original network. Hence not every pairing of normal and abnormal behavior is possible according to the neural net hypothesis as formulated. This connection between Bayes nets and work on neural models suggests that we might look for other applications of Bayesian networks in cognitive neuropsychology. We might hope to see applications of discovery methods for Bayes nets to multiple single cell recordings, or to functional MRI data. Such appplications would be aided by discovery methods for cyclic Bayes nets as models of recurrent neural nets. Several important theoretical steps have been taken towards that goal by Richardson, who has found a polynomial time decision procedure for the equivlaence of linear cyclic graphs, and a polynomial (in sparse graphs) time, asymptotically correct and complete procedure for discovering equivalence classes of linear cyclic graphs (without latent variables). Research is under way on search procedures that parallel the FCI procedure of Spirtes Glymour and Scheines (1993) in allowing latent variables. The questions of equivalence and discovery for non-linear systems are relatively untouched. It may be that the proper way to model a recurrent neural net is not by a cyclic graphical model, but by a time series. That suggestion raises the question of the connections between time-series and cyclic graphical models, a matter under investigation by Richardson. Results in this area would have implications for econometrics as well as for neuropsychology, since the econometric tradition has treated feedback systems in both ways, by simultaneous linear equations and by time series, without fully characterizing the relations between the representations. While there are situations in which equilibrium models, such as cyclic graphical models appear applicable, these models, since they are not dynamic, make no predictions about the dynamic behavior of a system (time series) if it is pushed out of equilibrium. 5:10pm - 5:50pm Compiling Probabilistic Networks and Some Questions this Poses Wray Buntine Probabilistic networks (or similar) provide a high-level language that can be used as the input to a compiler for generating a learning or inference algorithm. Example compilers are BUGS (inputs a Bayes net with plates) by Gilks, Spiegelhalter, et al., and MultiClass (inputs a dataflow graph) by Roy. This talk will cover three parts: (1) an outline of the arguments for such compilers for probabilistic networks, (2) an introduction to some compilation techniques, and (3) the presentation of some theoretical challenges that compilation poses. High-level language compilers are usually justified as a rapid prototyping tool. In learning, rapid prototyping arises for the following reasons: good priors for complex networks are not obvious and experimentation can be required to understand them; several algorithms may suggest themselves and experimentation is required for comparative evaluation. These and other justifications will be described in the context of some current research on learning probabilistic networks, and past research on learning classification trees and feed-forward neural networks. Techniques for compilation include the data flow graph, automatic differentiation, Monte Carlo Markov Chain samplers of various kinds, and the generation of C code for certain exact inference tasks. With this background, I will then pose a number of important research questions to the audience. 5:50pm - 6:30pm discussion Organizers: Wray Buntine Greg Cooper Dan Geiger Clark Glymour David Heckerman Geoffry Hinton Mike Jordan Steffen Lauritzen David Madigan Radford Neal Steve Omohundro Judea Pearl Stuart Russell Richard Scheines Peter Spirtes From juergen at idsia.ch Wed Nov 22 04:32:17 1995 From: juergen at idsia.ch (Juergen Schmidhuber) Date: Wed, 22 Nov 95 10:32:17 +0100 Subject: compressibility, Kolmogorov, learning Message-ID: <9511220932.AA22891@fava.idsia.ch> In response to Barak's and David's recent messages: David writes: >>> To illustrate just one of the possible objections to measuring randomness with Kolmogorov complexity: Would you say that a macroscopic gas with a specified temperature is "random"? To describe it exactly takes a huge Kolmogorov complexity. And certainly in many regards its position in phase space is "nothing but noise". (Indeed, in a formal sense, its position is a random sample of the Boltzmann distribution.) Yet Physicists can (and do) make extraordinarilly accurate predictions about such creatures with ease. <<< 1. Is real gas random in the Kolmogorov sense? At least the idealized microscopic gas models we are studying are not random. For simplicity, let us consider a typical discrete time gas model. Each system state can be computed by a short algorithm, given the previous state. This allows for enormous compressibility of the system history, even if the initial state had high complexity (and even more so if the initial state was simple). Even if we assume that the deterministic model is corrupted by random processes, we won't end up with a system history with maximal Kolmogorov complexity: for instance, using standard compression techiques, we can provide short codes for likely next states and long codes for unlikely next states. The only requirement is that the random processes are not *completely* random. But in the real world they are not, as can be deduced from macroscopic gas properties (considering only abstract properties of the state, such as temperature and pressure) --- as David indicated, there are simple algorithms for predicting next macroscopic states from previous macroscopic states. In case of true randomness, this would not be the case. 2. Only where there is compressibility, there is room for non-trivial learning and generalization. Unfortunately, almost all possible histories of possible universes are random and incompressible. There is no miraculous universal learning algorithm for arbitrary universes (that's more or less the realm of NFL). As has been observed repeatedly, however, our own universe appears to be one of the relatively few (but still infinitely many) compressible ones (every electron behaves the same way, etc.). In fact, much of the previous work on machine learning can be thought of exploiting compressibility: ``chunking'', for instance, exploits the possibility of re-using subprograms. Methods for finding factorial (statistically non-redundant) codes of image data exploit the enormous redundancy and compressibility of visual inputs. Similarly for ``learning by analogy'' etc. In the context of PAC learning, Ming Li and Paul Vitanyi address related issues in an interesting paper from 1989: A theory of Learning Simple Concepts Under Simple Distributions and Average Case Complexity for the Universal Distribution, Proc. 30th American IEEE Symposium on Foundations of Computer Science, pages 34-39. 3. An intriguing possibility is: there may be something like a universal learning algorithm for compressible, low-complexity universes. I am the first to admit, however, that neither the concept of Kolmogorov complexity by itself nor the universal Solomonoff-Levin distribution provide all the necessary ingredients. Clearly, a hypothetical universal learning algorithm would have to take into account the fact that computational resources are limited. This is driving much of the current work at our lab. Juergen Schmidhuber IDSIA From tds at ai.mit.edu Wed Nov 22 09:43:42 1995 From: tds at ai.mit.edu (Terence D. Sanger) Date: Wed, 22 Nov 95 09:43:42 EST Subject: off on a tangent... Message-ID: <9511221443.AA08557@dentate.ai.mit.edu> David Wolpert writes: > To illustrate just one of the possible objections to measuring > randomness with Kolmogorov complexity: Would you say that a > macroscopic gas with a specified temperature is "random"? To describe > it exactly takes a huge Kolmogorov complexity. And certainly in many > regards its position in phase space is "nothing but noise". (Indeed, > in a formal sense, its position is a random sample of the Boltzmann > distribution.) Yet Physicists can (and do) make extraordinarilly > accurate predictions about such creatures with ease. In thinking about this, it seems that David is right: the gas is, in some important sense, highly structured. In particular, its *statistics* are stationary no matter how they are sampled. This means that a temperature sample from any part of the gas will predict temperatures in other parts of the gas, according to the law of large numbers. Consider a different statistical model: Choose a random number to be the mean of a distribution on a finite set of random variables. Divide the set in half, choose a new random number and add it to the left half's mean and subtract it from the right half's mean. Divide the half-sets in half, choose two new random numbers, and continue to split each half by adding and subtracting random numbers until no more splits are possible. Now we have: 1) All random variables are independent and identically distributed (this is basically a random walk). 2) The mean of the distribution is equal to the first random number chosen (at each step, the mean does not change). 3) The mean of any "binary" region (half, quarter, eighth, etc.) is a poor predictor of the means of neighboring regions. 4) Sample means do not converge uniformly to the ensemble mean, unless samples are chosen randomly across region boundaries. In some sense, this is a very highly structured field of random variables. Yet prediction is much harder than for the random gas. (In case anyone is interested, this distribution arises as a model for genetic diseases of mitochondria. If a cell has N mitochondria, M of which are defective, then at cell division it will pass a random number of normal and defective mitochondria to each daughter cell, where the total number of defective ones passed on is conserved and is equal to 2M. The daughter cells, in turn, will do the same thing. The problem is that a local biopsy to count the number of defective mitochondria will not predict biopsy results from other sites.) Terry Sanger tds at ai.mit.edu From kak at gate.ee.lsu.edu Wed Nov 22 11:55:45 1995 From: kak at gate.ee.lsu.edu (Subhash Kak) Date: Wed, 22 Nov 95 10:55:45 CST Subject: Paper Message-ID: <9511221655.AA22783@gate.ee.lsu.edu> The following paper ON GENERALIZATION BY NEURAL NETWORKS by Subhash C. Kak Abstract: We report new results on the corner classification approach to training feedforward neural networks. It is shown that a prescriptive learning procedure where the weights are simply read off based on the training data can provide adequate generalization. The paper also deals with the relations between the number of separable regions and the size of the training set for a binary data network. was recently presented at the Joint Conference on Information Science. You may ftp the paper at the following address: ftp://gate.ee.lsu.edu/pub/kak/gen.ps From geoff at salk.edu Wed Nov 22 13:18:18 1995 From: geoff at salk.edu (Geoff Goodhill) Date: Wed, 22 Nov 95 10:18:18 PST Subject: Topographic Mappings - Tech Report available Message-ID: <9511221818.AA23843@salk.edu> The following paper is available via ftp://salk.edu/pub/geoff/goodhill_finch_sejnowski_tech95.ps.Z or http://cnl.salk.edu/~geoff QUANTIFYING NEIGHBOURHOOD PRESERVATION IN TOPOGRAPHIC MAPPINGS Geoffrey J. Goodhill(1), Steven Finch(2) & Terrence J. Sejnowski(3) (1) The Salk Institute for Biological Studies 10010 North Torrey Pines Road, La Jolla, CA 92037, USA (2) Human Communication Research Centre University of Edinburgh, 2 Buccleuch Place Edinburgh EH8 9LW, GREAT BRITAIN (3) The Howard Hughes Medical Institute The Salk Institute for Biological Studies 10010 North Torrey Pines Road, La Jolla, CA 92037, USA & Department of Biology University of California San Diego, La Jolla, CA 92037, USA, Institute for Neural Computation Technical Report Series INC-9505, November 1995 ABSTRACT Mappings that preserve neighbourhood relationships are relevant in both practical and biological contexts. It is important to be clear about precisely what preserving neighbourhoods could mean. We give a definition of a ``perfectly neighbourhood preserving'' map, which we call a topographic homeomorphism, and prove that this has certain desirable properties. When a topographic homeomorphism does not exist (the usual case), many choices are available for quantifying the quality of a map. We introduce a particular measure, C, which has the form of a quadratic assignment problem. We also discuss other measures that have been proposed, some of which are related to C. A comparison of seven measures applied to the same simple mapping problem reveals interesting similarities and differences between the measures, and challenges common intuitions as to what constitutes a ``good'' map. 17 pages, uncompressed postscript = 154K From scott at cpl_mmag.nhrc.navy.mil Wed Nov 22 14:37:41 1995 From: scott at cpl_mmag.nhrc.navy.mil (Scott Makeig) Date: Wed, 22 Nov 95 11:37:41 -0800 Subject: Online-NIPS post-NIPS workshop: programme Message-ID: <9511221937.AA24212@cpl_mmag.nhrc.navy.mil> ****** PROGRAMME ****** NIPS*95 Workshop on ONLINE NEURAL INFORMATION PROCESSING SYSTEMS: Prospects for Neural Human-Machine Interfaces Date: Saturday, Dec. 2, 1995 Place: NIPS*95 Workshops, Vail, Colorado www: http://128.49.52.9/~www/nips.html Organizer: Scott Makeig (NHRC/UCSD) scott at salk.edu There is rapidly growing interest in the development of intelligent interfaces in which operator state information derived from psycho- physiological and/or video-based measures of the human operator is used directly to inform, interact with, or control computer-based systems. Adequate signal processing power is now available at reasonable cost to implement in near-real time a wide range of spectral, neural network, and dynamic systems algorithms for extracting information about psychological state or intent from multidimensional EEG signals, video images of the eyes and face, and other psychophysiological and/or behavioral data. This NIPS*95 conference workshop will give an opportunity for interested researchers from signal processing, neuroscience, neural networks, cognitive science, and computer design to discuss near- and medium-term prospects for, and obstacles to, practical neural human-systems interfaces (NHSI) technology for monitoring cognitive state and for using operator state information to give operator feedback, control adaptive automation or perform brain-actuated control. Aspects of cognitive state that might be monitored using NHSI technology include alertness, perception, attention, workload, intention and emotion: Programme (Saturday, Dec. 2, Vail Marriott): 7:30 Scott Makeig (NHRC/UCSD) NHSI Overview Sandy Pentland (MIT Media Lab) Video-based human-computer interaction Alan Gevins (EEG Systems Labs) EEG-based cognitive monitoring Discussion 8:30 Babak A. Taheri (SRI International) Active EEG electrode technology Tzyy-Ping Jung (Salk Institute) EEG-based alertness monitoring Magnus Stensmo (Salk Institute) Monitoring alertness via eye closures General Discussion 9:30 [free time] 12:30 Lunch (optional) 1:30 [free time] 4:30 Georg Dorffner (ANNDEE project group, Austria) Brain-actuated control Grant McMillan (Wright-Patterson Air Force Base ) Brain-actuated control Andrew Junker (CyberLink) EMG/EEG-actuated control Discussion 5:30 Curtis Padgett (UCSD) Video-based emotion monitoring Jose Principe (University of Florida) EEG-based communication Charles W. Anderson (Colorado State University) Mental task monitoring General Discussion The workshop will review ongoing progress and challenges in computational and technology areas, and discuss prospects for short- and medium-term implementations. Abstracts are available at: http://128.49.52.9/~www/nips.html NIPS*95 conference and post-conference workshop information: http://www.cs.cmu.edu/Web/Groups/NIPS/nips95.html or via ftp/email from: psyche.mit.edu in /pub/NIPS95 nips95 at mines.colorado.edu From stavrosz at med.auth.gr Wed Nov 22 18:06:25 1995 From: stavrosz at med.auth.gr (Stavros Zanos) Date: Thu, 23 Nov 1995 01:06:25 +0200 (EET) Subject: Paper on LTP and Learning Algorithms In-Reply-To: Message-ID: (Neural Nets: Foundations to Applications) The following paper is now available to anyone who sends a request at the following adress (use the word "reqLTP3" at the subject field): stavrosz at antigoni.med.auth.gr ********* AU: Zanos Stavros, 3rd year medical student AT: University of Thessaloniki School of Medicine Thessaloniki, Greece TI: Quantal Analysis of Hippocampal Long-Term Synaptic Potentiation , and Application to the Design of Biologically Plausible Learning Algorithms for Artificial Neural Networks AB: Quantal analysis (QA) of synaptic function has been used to examine whether the expression of long-term potentiation (LTP) in central synapses is mediated by a pre- or postsynaptic mechanism. However, it can also be used as a physiological model of synaptic transmission and plasticity; use of physiological models in network simulations provides reasonably accurate approximates of various biological parameters in a computationally efficient manner. We describe a stochastic algorithm of synaptic transmission and plasticity based on QA data from CA1 hippocampus LTP experiments. We also describe the application of such an algorithm in a typical CA1-region simulation (a simple self-organizing competitive matrix), and discuss the possible benefits of using noisy network elements (in this case, "synapses"). We show that the fluctuations in postsynaptic responses under constant static synaptic weights introduced by such an algorithm increase the storing capacity and the ability of the network to orthogonalize input vectors. A decrease in the number of required iterations for every learned input vector is also reported. Finally we examine the issue of a hypothetical "computational equivalence" of different optimization techniques when applied to similar problems, often met in the literature, since our simulation studies suggest that even small differences in the learning algorithms used could provide the network with a kind of "preference" to specific patterns of performance. ********* The above paper will appear at the 2nd European Conference of Medical Students (May 96), and it has been edited using MS Word-7 (for Win95). Those who adressed a request will receive the paper through email as an attachment compressed file. Detailed mathematical formalizations used in the simulations are available upon request. We welcome questions and/or remarks. Zanos Stavros Aristotle University of Thessaloniki School of Life Sciences, Faculty of Medicine From ken at phy.ucsf.edu Wed Nov 22 20:44:34 1995 From: ken at phy.ucsf.edu (Ken Miller) Date: Wed, 22 Nov 1995 17:44:34 -0800 Subject: Postdoctoral and Predoctoral Positions in Theoretical Neurobiology Message-ID: <9511230144.AA04384@coltrane.ucsf.edu> POSTDOCTORAL AND PREDOCTORAL POSITIONS SLOAN CENTER FOR THEORETICAL NEUROBIOLOGY UNIVERSITY OF CALIFORNIA, SAN FRANCISCO INFORMATION ON THE UCSF SLOAN CENTER AND FACULTY AND THE POSTDOCTORAL AND PREDOCTORAL POSITIONS IS AVAILABLE THROUGH OUR WWW SITE: http://keck.ucsf.edu/sloan. E-mail inquiries should be sent to sloan-info at phy.ucsf.edu. Below is basic information on the program: The Sloan Center for Theoretical Neurobiology at UCSF solicits applications for pre- and post-doctoral fellowships, with the goal of bringing theoretical approaches to bear on neuroscience. Applicants should have a strong background and education in a theoretical discipline, such as physics, mathematics, or computer science, and commitment to a future research career in neuroscience. Prior biological or neuroscience training is not required. The Sloan Center will offer opportunities to combine theoretical and experimental approaches to understanding the operation of the intact brain. The research undertaken by the trainees may be theoretical, experimental, or a combination. The RESIDENT FACULTY of the Sloan Center and their research interests are: Allison Doupe: Development of song recognition and production in songbirds. Stephen Lisberger: Learning and memory in a simple motor reflex, the vestibulo-ocular reflex, and visual guidance of smooth pursuit eye movements by the cerebral cortex. Michael Merzenich: Experience-dependent plasticity underlying learning in the adult cerebral cortex and the neurological bases of learning disabilities in children. Kenneth Miller: Mechanisms of self-organization of the cerebral cortex; circuitry and computational mechanisms underlying cortical function; computational neuroscience. Roger Nicoll: Synaptic and cellular mechanisms of learning and memory in the hippocampus. Christoph Schreiner: Cortical mechanisms of perception of complex sounds such as speech in adults, and plasticity of speech recognition in children and adults. Michael Stryker: Mechanisms that guide development of the visual cortex. All of these resident faculty are members of UCSF's W.M. Keck Foundation Center for Integrative Neuroscience, a new center (opened January, 1994) for systems neuroscience that includes extensive shared research resources within a newly renovated space designed to promote interaction and collaboration. The unusually collaborative and interactive nature of the Keck Center will facilitate the training of theorists in a variety of approaches to systems neuroscience. In addition to the resident faculty, there are a series of VISITING FACULTY who are in residence at UCSF for times ranging from 1-8 weeks each year. These faculty, and their research interests, include: Laurence Abbott, Brandeis University: Neural coding, relations between firing rate models and biophysical models, self-organization at the cellular level William Bialek, NEC Research Institute: Physical limits to sensory signal processing, reliability and information capacity in neural coding; Sebastian Seung, ATT Bell Labs: models of collective computation in neural systems; David Sparks, University of Pennsylvania: understanding the superior colliculus as a "model cortex" that guides eye movements; Steven Zucker, McGill University: Neurally based models of vision, visual psychophysics, mathematical characterization of neuroanatomical complexity. PREDOCTORAL applicants seeking to BEGIN a Ph.D. program should apply directly to the UCSF Neuroscience Ph.D. program. Contact Patricia Arrandale, patricia at phy.ucsf.edu, to obtain application materials. THE APPLICATION DEADLINE IS Jan. 5, 1996. Also send a letter to Steve Lisberger (address below) indicating that you are applying to the UCSF Neuroscience program with a desire to join the Sloan Center. POSTDOCTORAL applicants, or PREDOCTORAL applicants seeking to do research at the Sloan Center as part of a Ph.D. program in progress in a theoretical discipline elsewhere, should apply as follows: Send a curriculum vitae, a statement of previous research and research goals, up to three relevant publications, and have two letters of recommendation sent to us. THE APPLICATION DEADLINE IS February 1, 1996. UC San Francisco is an Equal Opportunity Employer. Send applications to: Steve Lisberger Sloan Center for Theoretical Neurobiology at UCSF Department of Physiology University of California 513 Parnassus Ave. San Francisco, CA 94143-0444 From radford at cs.toronto.edu Wed Nov 22 21:18:03 1995 From: radford at cs.toronto.edu (Radford Neal) Date: Wed, 22 Nov 1995 21:18:03 -0500 Subject: Performance evaluations: request for comments Message-ID: <95Nov22.211807edt.1585@neuron.ai.toronto.edu> Announcing a draft document on ASSESSING LEARNING PROCEDURES USING DELVE The DELVE development group, University of Toronto http://www.cs.utoronto.ca/neuron/delve/delve.html The DELVE development group requests comments on the draft manual for the DELVE environment from researchers who are interested in how to assess the performance of learning procedures. This manual is available via the DELVE homepage, at the URL above. Carl Rasmussen and Geoffrey Hinton will be talking about the DELVE environment at the NIPS workshop on Benchmarking of Neural Net Learning Algorithms. We would be pleased to hear any comments that attendees of this workshop, or other interested researchers, might have on the current design of the DELVE environment, as described in this draft manual. Here is the introduction to the DELVE manual: DELVE --- Data for Evaluating Learning in Valid Experiments --- is a collection of datasets from many sources, and an environment within which this data can be used to assess the performance of procedures that learn relationships using such data. Many procedures for learning from empirical data have been developed by researchers in statistics, pattern recognition, artificial intelligence, neural networks, and other fields. Learning procedures in common use include simple linear models, nearest neighbor methods, decision trees, multilayer perceptron networks, and many others of varying degrees of complexity. Comparing the performance of these learning procedures in realistic contexts is a surprisingly difficult task, requiring both an extensive collection of real-world data, and a carefully-designed scheme for performing experiments. The aim of DELVE is to help researchers and potential users to assess learning procedures in a way which is relevant to real-world problems and which allows for statistically-valid comparisons of different procedures. Improved assessments will make it easier to determine which learning procedures work best for various applications, and will promote the development of better learning procedures by allowing researchers to easily determine how the performance of a new procedure compares to that of existing procedures. This manual describes the DELVE environment in detail. First, however, we provide an overview of DELVE's capabilities, describe briefly how DELVE organizes datasets and learning tasks, and give an example of how DELVE can be used to assess the performance of a learning procedure. --------------------------------------------------------------------------- Members of the DELVE Development Group: G. E. Hinton R. M. Neal R. Tibshirani M. Revow C. E. Rasmussen D. van Camp R. Kustra Z. Ghahramani ---------------------------------------------------------------------------- Radford M. Neal radford at cs.toronto.edu Dept. of Statistics and Dept. of Computer Science radford at utstat.toronto.edu University of Toronto http://www.cs.toronto.edu/~radford ---------------------------------------------------------------------------- From lemm at LORENTZ.UNI-MUENSTER.DE Thu Nov 23 07:26:30 1995 From: lemm at LORENTZ.UNI-MUENSTER.DE (Joerg_Lemm) Date: Thu, 23 Nov 1995 13:26:30 +0100 Subject: NFL for NFL Message-ID: <9511231226.AA11633@xtp141.uni-muenster.de> I would like to make a comment to the NFL discussion from my point of view. (Thanks to David Wolpert for remaining active in this discussion he initiated.) 1.) If there is no relation between the function values on the test and training set (i.e. P(f(x_j)=y|Data) equal to the unconditional P(f(x_j)=y) ), then, having only training examples y_i = f(x_i) (=data) from a given function, it is clear that I cannot learn anything about values of the function at different arguments, (i.e. for f(x_j), with x_j not equal to any x_i = nonoverlapping test set). 2.) We are considering two of those (influence) relations P(f(x_j)=y|Data): one, named A, for the true nature (=target) and one, named B, for our model under study (=generalizer). Let P(A and B) be the joint probability distribution for the influence relations for target and generalizer. 3.) Of course, we do not know P(A and B), but in good old Bayesian tradition, we can construct a (hyper-)prior P(C) over the family of probability distributions of the joint distributions C = P(A and B). 4.) NFL now uses the very special prior assumption P(A and B) = P(A)P(B), or equivalently P(B|A)=P(B), which means NFL postulates that there is (on average) no relation between nature and model. No wonder that (averaging over targets P(A) or over generalizers P(B) ) cross-validation works as well (or as bad) as anti-cross-validation or anything else in such cases. 5.) But target and generalizer live on the same planet (sharing the same laws, environment, history and maybe even building blocks) so we have very good reasons to assume a bias for (hyper-)priors towards correlated P(A and B) not equal to the uncorrelated product P(A)P(B)! But that's not all: We do have information which is not of the form y=f(x). We know that the probability for many relations in nature to be continuous on certain scales seems to be high (->regularization). We can have additional information about other properties of the function, e.g.symmetries, compare Abu-Mostafa's concept of hints, which produces correlations between A (=target) and B (=model). In this sense I aggree with David Wolpert on looking >>> how to characterize the needed relationship between the set of generalizers and the prior that allows cross-validation to work. >>> To summarize it in a provocative way: There is no free lunch for NFL: Only if you assume that no relation between target and model exists, then you don't find a relation between target and model! And to be precise, I say that it is rational to believe (and David does so too, I think) that in real life cross-validation works better in more cases than anti-cross-validation. Joerg Lemm From jagota at ponder.csci.unt.edu Thu Nov 23 15:22:51 1995 From: jagota at ponder.csci.unt.edu (Jagota Arun Kumar) Date: Thu, 23 Nov 95 14:22:51 -0600 Subject: compressibility, Kolmogorov, learning Message-ID: <9511232022.AA04987@ponder> A short addition to the ongoing discussion thread of Juergen, Barak, and David: The following short paper is what intrigued us (K. Regan and I) to experimentally investigate the performance of algorithms on random versus compressible (i.e., structured) instances of the maximum clique optimization problem. @article{LiV92, author = "Li, M. and P.M.B. Vitanyi", title = "Average case complexity under the universal distribution equals worst-case complexity", pages = "145--149", journal = "Information Processing Letters", volume = 42, month = "May", year = 1992 } In other words, roughly speaking, if one samples uniformly from the universal distribution, one exhibits worst-case behavior of an algorithm. I announced our (experimental, max-clique) paper on Connectionists a few months back, so won't do it again. However, this will be the subject of my talk at the NIPS workshop on optimization (7:30--8:00, Dec 1, Vail). Stop by and jump all over me. Arun Jagota From nin at cns.brown.edu Sat Nov 25 10:41:21 1995 From: nin at cns.brown.edu (Nathan Intrator) Date: Sat, 25 Nov 95 10:41:21 EST Subject: New preprint: multi-task training Message-ID: <9511251541.AA28777@cns.brown.edu> Making a low-dimensional representation suitable for diverse tasks Nathan Intrator Shimon Edelman Tel-Aviv U. Weizmann Ins. We introduce a new approach to the training of classifiers for performance on multiple tasks. The proposed hybrid training method leads to improved generalization via a better low-dimensional representation of the problem space. The quality of the representation is assessed by embedding it in a 2D space using multidimensional scaling, allowing a direct visualization of the results. The performance of the approach is demonstrated on a highly nonlinear image classification task. The paper will be described at the comming Transfer workshop at NIPS url=ftp://cns.brown.edu/nin/papers/mds.ps.Z or it can be accessed through our hope pages: http://www.wisdom.weizmann.ac.il/~edelman/shimon.html http://www.physics.brown.edu/~nin Comments are most welcome. - Nathan Intrator From emj at cs.ucsd.edu Sun Nov 26 02:03:22 1995 From: emj at cs.ucsd.edu (Eric Mjolsness) Date: Sat, 25 Nov 95 23:03:22 -0800 Subject: NIPS 95 workshop schedule Message-ID: <9511260703.AA04033@triangulum> NIPS-95 Workshop, ** Tentative Schedule ** Statistical and Structural Models in Network Vision Friday, November 1, 1995 Organizers: Eric Mjolsness and Anand Rangarajan URL: http://www-cse.ucsd.edu/users/emj/workshop95.html Overview: Neural network and model-based approaches to vision are usually regarded as opposing tendencies. Whereas neural net methods often focus on images and learned feature detectors, model-based methods concentrate on matching high-level representations of objects and their parts and other intrinsic properties. It is possible that the two approaches can be integrated in the context of statistical models which have the flexibility to represent patterns in both image space and in higher-level object feature spaces. The workshop will examine the possibilities for and progress in formulating such models for vision problems, particularly those models which can result in neural network architectures. *Tentative Schedule* 7:30am - 7:50am Eric Mjolsness, "Workshop Overview" 7:50am - 8:15am Chris Bregler, "Soft Features for Soft Classifiers" 8:15am - 8:40am Hayit Greenspan, "Preprocessing and Learning in Rotation-Invariant Texture Recognition" 8:40am - 9:05am Alan Yuille, "Deformable Templates for Object Recognition: Geometry and Lighting" 9:05am - 9:30am Anand Rangarajan, "Bayesian Tomographic Reconstruction using Mechanical Models as Priors" 9:30am - 4:30pm Excercises 4:30pm - 4:50pm Paul Viola, "Recognition by Complex Features" 4:50pm - 5:15pm Lawrence Staib, "Model-based Parametrically Deformable Boundary Finding" 5:15pm - 5:40pm Steven Gold, "Recognizing Objects with Recurrent Neural Networks by Matching Structural Models" 5:40pm - 6:05pm Robert M. Haralick, "Annotated Computer Vision Data Sets" 6:05pm - 6:30pm Eric Saund, "Representations for Perceptual Level Chunks in Line Drawings" *Abstracts* Chris Bregler, U.C.Berkeley, "Soft Features for Soft Classifiers" Most connectionist approaches that are applied to visual domains either make little use of any preprocessing or are based on very high level input representations. The former solutions are motivated by the concern not to lose any useful information for the final classification and show how powerful such algorithms are in extracting relevant features automatically. "Hard" decisions like edge detectors, line-finders, etc. don't fit into the philosophy of adaptability across all levels. We attempt to find a balance between both extrema and show how mature "soft"-preprocessing techniques like rich sets of scaled and rotated gaussian derivatives, second moment texture statistics, and Hierachical Mixtures of Experts can be applied to the domain of car classification. Steven Gold, Yale University, "Recognizing Objects with Recurrent Neural Networks by Matching Structural Models" Attributed Relational Graphs (ARG) are used to create structural models of objects. Recently developed optimization techniques that have emerged out of the neural network/statistical physics framework are then used to construct algorithms to match ARGs. Experiments conducted on ARGs generated from images are presented. Hayit Greenspan, Caltech, "Preprocessing and Learning in Rotation-Invariant Texture Recognition" A number of texture recognition systems have been recently proposed in the literature, giving very high-accuracy classification rates. Almost all of these systems fail miserably when invariances, such as in rotation or scale, are to be included; invariant recognition is clearly the next major challenge in recognition systems. Rotation invariance can be achieved in one of two ways, either by extracting rotation-invariant features, or by appropriate training of the classifier to make it "learn" invariant properties. Learning invariances from the raw data is substantially influenced by the rotation angles that have been included in the system's training set. The more examples, the better the performance. We compare to that a mechanism to extract the rotation-invariant features {\em prior} to the learning phase. We introduce a powerful image representation space with the use of a steerable filter set; along with a new encoding scheme to extract the invariant features. Our results strongly indicate the advantage of extracting a powerful image-representation prior to the learning process; with savings in both storage and computational complexity. Rotation-invariant texture recognition results and demo will be shown. Robert M. Haralick, University of Washington, "Annotated Computer Vision Data Sets" Recognizing features with a protocol that learns from examples, requires that there be many example instances. In this talk, we describe the 78 image annotated data set of RADIUS images, of 2 different 3D scenes, which we have prepared for CDROM distribution. The feature annotation includes building edges, shadow edges, clutter edges, and corner positions. As well the image data set has photogrammetric data of corresponding 3D and 2D points and corresponding 2D pass points. The interior orientation and exterior orientation parameters for all images are given. The availability of this data set makes possible comparison of different algorithms and makes possible very careful experiments of feature extraction using neural net approaches. Eric Mjolsness, "Workshop Overview" I will introduce the workshop and discuss possibilities for integration between some of the research directions represented by the participants. Anand Rangarajan, Yale University, "Bayesian Tomographic Reconstruction using mechanical models as priors" We introduce a new prior---the weak plate---to tomographic reconstruction. MAP estimates are obtained via a deterministic annealing Generalized EM algorithm which avoids poor local minima. Bias/variance simulation results on an autoradiograph phantom demonstrate the superiority of the weak plate prior over other first-order priors used in the literature. Eric Saund, Xerox Palo Alto Research Center, "Representations for Perceptual Level Chunks in Line Drawings" In a line drawing, what makes a box a box? A perfectly drawn box is easy to recognize because it presents a remarkable conjunction of crisp spatial properties yielding a wealth of necessary and sufficient conditions to test. But if it is drawn sloppily, many ideal properties such as closure, squareness, and straightness of the sides, give way. In addressing this problem, most attendants of NIPS probably would look to the supple adaptability and warm fuzziness of statistical approaches over the cold fragility of logic-based specifications. Even in doing this, however, some representations generalize better than others. This talk will address alternative representations for visual objects presented at an elemental level as curvilinear lines, with a look to densely overlapping distributed representations in which a large number of properties can negotiate their relative significance. Lawrence Staib, Yale University, "Model-based Parametrically Deformable Boundary Finding" This work describes a global shape parametrization for smoothly deformable curves and surfaces. This representation is used as a model for geometric matching to image data. The parametrization represents the curve or surface using sinusoidal basis functions and allows a wide variety of smooth boundaries to be described with a small number of parameters. Extrinsic model-based information is incorporated by introducing prior probabilities on the parameters based on a sample of objects. Boundary finding is then formulated as an optimization problem. The method has been applied to synthetic images and three-dimensional medical images of the heart and brain. Paul Viola, Salk Institute, "Recognition by Complex Features" From gem at cogsci.indiana.edu Sun Nov 26 18:45:01 1995 From: gem at cogsci.indiana.edu (Gary McGraw) Date: Sun, 26 Nov 95 18:45:01 EST Subject: Letter recognition thesis Message-ID: Announcing the availability of my thesis on the web... Letter Spirit (part one): Emergent High-level Perception of Letters Using Fluid Concepts Gary McGraw Center for Research on Concepts and Cognition 510 North Fess Street Indiana University Bloomington, IN 47405 This thesis presents initial work on the Letter Spirit project, with a cognitive model of letter perception as its centerpiece. The Letter Spirit project is an attempt to model central aspects of human high-level perception and creativity on a computer, focusing on the creative act of artistic letter-design. The aim is to model the process of rendering the 26 lowercase letters of the roman alphabet in many different, internally coherent styles. Two important and orthogonal aspects of letterforms are basic to the project: the categorical sameness} possessed by instances of a single letter in various styles (e.g., the letter `a' in Times, Palatino, and Helvetica) and the stylistic sameness} possessed by instances of various letters in a single style, or spirit (e.g., the letters `a', `k', and `q' in Times, alone). Starting with one or more seed letters representing the beginnings of a style, the program will attempt to create the rest of the alphabet in such a way that all 26 letters share the same style, or spirit. Letters in the domain are formed exclusively from straight segments on a grid in order to make decisions smaller in number and more discrete. This restriction allows much of low-level vision to be bypassed and forces concentration on higher-level cognitive processing, particularly the abstract and context-dependent character of concepts. The overall architecture of the Letter Spirit project, based on the principles of emergent computation and flexible context-sensitive concepts, has been carefully developed and is presented in Part I. Creating a gridfont is an iterative process of guesswork and evaluation --- the ``central feedback loop of creativity''. The notion of evaluation and its necessary foundation in perception have been largely overlooked in many cognitive models of creativity. In order to be truly creative, a program must do its own evaluating and perceiving. To this end, we have focused initial Letter Spirit work on a high-level perceptual task --- letter perception. We have developed an emergent model of letter perception based on the hypothesis that letter categories are made up of conceptual constituents, called roles}, which exert clear top-down influence on the segmentation of letterforms into structural components. Part II introduces the role hypothesis, presents experimental psychological evidence supporting it, and then introduces a complex cognitive model of letter perception that is consistent with the empirical results. Because we are interested ultimately in the design of letters (and the creative process as whole) an effort was made to develop a model rich enough to be able to recognize and conceptualize a large range of letters including letters at the fringes of their categories. -------------------------------------------------------------------------- The ~400 page thesis is available on the web at URL: It may also be retrieved through ftp to ftp.cogsci.indiana.edu as the file pub/mcgraw.thesis.ps.gz Hardcopy is generally not available due to prohibitive costs. However, if you are having trouble retrieving the text, send me e-mail (gem at cogsci.indiana.edu). Comments and questions are welcome. Gary McGraw *---------------------------------------------------------------------------* | Gary McGraw gem at cogsci.indiana.edu | (__) | |--------------------------------------------------| (oo) | | Center for Research on Concepts and Cognition | /-------\/ | | Department of Computer Science | / | || | | Indiana University (812) 855-6966 | * ||----|| | | | ^^ ^^ | *---------------------------------------------------------------------------* From sayegh at CVAX.IPFW.INDIANA.EDU Sun Nov 26 19:53:57 1995 From: sayegh at CVAX.IPFW.INDIANA.EDU (sayegh@CVAX.IPFW.INDIANA.EDU) Date: Sun, 26 Nov 1995 19:53:57 EST Subject: NN distance learning course announcement Message-ID: <00999FC1.6320AA7C.51@CVAX.IPFW.INDIANA.EDU> FOUNDATIONS AND APPLICATIONS OF NEURAL NETWORKS Course announcement This course is to be offered in the Spring of 1996. Students at remote sites will receive and view lecture tapes at their convenience. Handouts, homework and other assignments will be handled via a web site. This is a 500 level course open to both seniors and graduate students in the Sciences, Mathematics, Engineering, Computer Science, and Psychology or professionals interested in the topic, provided they meet the prerequisites or obtain the instructor's permission. The course is listed as PHYS 525 at Purdue. Please contact the instructor if you are interested. Instructor: Dr. Samir Sayegh sayegh at cvax.ipfw.indiana.edu Phone: (219) 481-6157 FAX: (219) 481-6800 Description: In the last ten years Neural Networks have become both a powerful practical tool to approach difficult classification, optimization and signal processing problems as well as a serious paradigm for computation in parallel machines and biological networks. This course is an introduction to the main concepts and algorithms of neural networks. Lengthy derivations and formal proofs are avoided or minimized and an attempt is made to emphasize the connection between the "artificial" network approaches and their neurobiological counterparts. In order to help achieve that latter goal, the text "A Vision of the Brain" by Semir Zeki is required reading, in addition to the main text "Introduction to the Theory of Neural Computation" by Herz, Krogh and Palmer, and the instructor's (valuable) notes. Zeki's book recounts the fascinating (hi)story of the discovery of the color center in the human visual cortex and emphasizes very general organizational principles of neuroanatomy and neurophysiology, which are highly relevant to any serious computational approach. The following classic topics are covered: - Introduction to the brain and its simpler representations - Neural Computational Elements and Algorithms - Perceptron - Adaptive Linear Element - Backpropagation - Hopfield Model, Associative Memory and Optimization - Kohonen networks - Unsupervised Hebbian learning and principal component analysis - Applications in signals, speech, robotics and forecasting. - Introduction to Computational Neuroscience - Introduction to functional neuroanatomy and functional imaging - Introduction to the visual pathways and computation in retina and visual cortex. Prerequisites: Calculus, matrix algebra and familiarity with a computer language Texts: "A Vision of the Brain" by Semir Zeki (Blackwell, 1993) "Introduction to the Theory of Neural Computation" by Herz, Krogh and Palmer (Addison Wesley, 1991) Instructor's (valuable) notes. Testing: Each lecture comes with a handout that includes a list of objectives and a set of multiple choice questions. The two take-home midterm exams and the final exam will be mostly multiple choice with the questions reflecting the lecture objectives. In addition each student will be expected to complete an individual project in the area of her/his interest. The project may or may not be part of the final grade depending on the project's progress. Software: While students are welcome to use the language of their choice, the high level language MATLAB and the associated toolbox for Neural Networks will be provided for the duration of the course at no additional charge. Cost (US $) Indiana Resident Non resident Undergraduate 249.45 644.450 Graduate 315. 60 751.05 Appendix (brief intro): Neural Networks provide a fruitful approach to a variety of engineering and scientific problems that have been traditionally considered difficult. While an exact definition remains elusive and different practitioners would emphasize one or another of the characteristics of NN, it is possible to list the most common and some of the most fundamental features of neural network solutions: 1) Adaptive 2) Parallel 3) Neurally inspired 4) Ability to handle non-linear problems in a transparent way Let us look at these in some detail: 1) Adaptive solutions are desirable in a number of situations. They present advantages of stability as well as the ability to deal with huge data sets with minimal memory requirements, as the patterns are presented "one at a time." The same advantage implies the possibility of developing real time on-line solutions where the totality of the data set is not available at the outset. 2) The formulation of neural networks solutions is inherently parallel. A large number of nodes share the burden of a computation and often can operate independent of information made available by other nodes. This clearly speeds up computation and allows implementation on highly efficient parallel hardware. 3) Though the extent is somewhat debated, it is clear that there is some similarities between current artificial neural algorithms and biological systems capable of intelligence. The fact that such biological systems still display pattern recognition capabilities far beyond those of our algorithms is a continuing incentive to maintain and further explore the neurobiological connection. 4) The ability to handle nonlinearity is a fundamental requirement of modern scientific and engineering approaches. In a number of fields, the nonlinear approaches are developed on a case by case basis and have often little connection to the better established linear techniques. On the other hand, with the general approach of formulating a neural network and endowing it with increasingly complex processing capabilities, it is possible to define a unified spectrum extending from linear networks (say a one weight-layer ADALINE) to highly nonlinear ones with powerful processing capabilities (say a multilayer backpropagation network). The combination of the properties outlined coupled to the near universal model of neural networks and the availability of software and hardware tools make NN one of the most attractive instruments of signal processing and pattern recognition available today. From rao at cs.rochester.edu Sun Nov 26 22:01:07 1995 From: rao at cs.rochester.edu (rao@cs.rochester.edu) Date: Sun, 26 Nov 1995 22:01:07 -0500 Subject: Paper Available: Dynamic Model of Visual Cortex Message-ID: <199511270301.WAA12985@vulture.cs.rochester.edu> Dynamic Model of Visual Memory Predicts Neural Response Properties In The Visual Cortex Rajesh P.N. Rao and Dana H. Ballard Department of Computer Science University of Rochester Rochester, NY 14627-0226, USA Technical Report 95.4 National Resource Laboratory for the study of Brain and Behavior (November 1995) Abstract Recent neurophysiological experiments have shown that the responses of visual cortical neurons in a monkey freely viewing a natural scene can differ substantially from those obtained when the same image subregions are flashed while the monkey performs a fixation task. Neurophysiological research in the past has been based predominantly on cell recordings obtained during fixation tasks, under the assumption that this data would be useful in predicting the responses in more general situations. It is thus important to understand the differences revealed by the new findings and their relevance to the study of visual perception. We describe a computational model of visual memory which dynamically combines input-driven bottom-up signals with expectation-driven top-down signals to achieve optimal estimation of current state by using a Kalman filter-based framework. Computer simulations of the proposed model are shown to correlate closely with the reported neurophysiological observations in both free-viewing and fixating conditions. The model posits a role for the hierarchical structure of the visual cortex and its reciprocal connections between adjoining visual areas in determining the response properties of visual cortical neurons. ======================================================================== Retrieval information: FTP-host: ftp.cs.rochester.edu FTP-pathname: /pub/u/rao/papers/dynmem.ps.Z URL: ftp://ftp.cs.rochester.edu/pub/u/rao/papers/dynmem.ps.Z 10 pages; 309K compressed, 646K uncompressed e-mail: rao at cs.rochester.edu Hardcopies available upon request at the above address or from the first author at NIPS*95. ========================================================================= From eric at research.nj.nec.com Mon Nov 27 10:51:01 1995 From: eric at research.nj.nec.com (Eric B. Baum) Date: Mon, 27 Nov 1995 10:51:01 -0500 Subject: Response to no-free-lunch discussion In-Reply-To: Barak Pearlmutter "Re: Response to no-free-lunch discussion" (Nov 19, 3:03pm) References: <199511192303.PAA07609@valaga.salk.edu> Message-ID: <9511271051.ZM13175@yin> Barak Pearlmutter remarked that saying We have *no* a priori reason to believe that targets with "low Kolmogorov complexity" (or anything else) are/not likely to occur in the real world. (which I gather was a quote from David Wolpert?) is akin to saying we have no a priori reason to believe there is non-random structure in the world, which is not true, since we make great predictions about the world. Wolpert replied: > To illustrate just one of the possible objections to measuring > randomness with Kolmogorov complexity: Would you say that a > macroscopic gas with a specified temperature is "random"? To describe > it exactly takes a huge Kolmogorov complexity. And certainly in many > regards its position in phase space is "nothing but noise". (Indeed, > in a formal sense, its position is a random sample of the Boltzmann > distribution.) Yet Physicists can (and do) make extraordinarilly > accurate predictions about such creatures with ease. Somebody else (Jurgen Schmidhuber I think?) argued that a gas does *not* have high Kolmogorov complexity, because its time evolution is predictable. So in a lattice gas model, given initial conditions (which are relatively compact, including compact pseudorandom number generator) one may be able to predict evolution of gas. Two comments: (1) While it may be that in classical Lattice gas models, a gas does not have high Kolmogorov complexity, this is not the origin of the predictability exploited by physicists. Statistical mechanics follows simply from the assumption that the gas is in a random one of the accessible states, i.e. the states with a given amount of energy. So *define* a *theoretical* gas as follows: Every time you observe it,it is in a random accessible state. Then its Kolmogorov complexity is huge (there are many accessible states) but its macroscopic behavior is predictable. (Actually this an excellent description of a real gas, given quantum mechanics.) (2) Point 1 is no solace to those arguing for the relevance of Wolpert's theorem, as I understand it. We observe above that non-randomness arises purely out of statistical ensemble effects. This is non-randomness none-the-less. Consider the problem of learning to predict the pressure of a gas from its temperature. Wolpert's theorem, and his faith in our lack of prior about the world, predict, that any learning algorithm whatever is as likely to be good as any other. This is not correct. Interestingly, Wolpert and Macready's results appear irrelevant/wrong here in an entirely *random*, *play* world. We see that learnable structure arises at a macroscopic level, and that our natural instincts about learning (e.g. linear relationships, cross-validation as opposed to anti-cross validation) hold. We don't need to appeal to experience with physical nature in this play world. We could prove theorems about the origin of structure. (This may even be a fruitful thing to do.) Creatures evolving in this "play world" would exploit this structure and understand their world in terms of it. There are other things they would find hard to predict. In fact, it may be mathematically valid to say that one could mathematically construct equally many functions on which these creatures would fail to make good predictions. But so what? So would their competition. This is not relevant to looking for one's key, which is best done under the lamppost, where one has a hope of finding it. In fact, it doesn't seem that the play world creatures would care about all these other functions at all. What was the Einstein quote wondering about the surprising utility of mathematics in understanding the natural world? Maybe mathematics itself provides an answer? -- ------------------------------------- Eric Baum NEC Research Institute, 4 Independence Way, Princeton NJ 08540 PHONE:(609) 951-2712, FAX:(609) 951-2482, Inet:eric at research.nj.nec.com http://www.neci.nj.nec.com:80/homepages/eric/eric.html From omlinc at research.nj.nec.com Mon Nov 27 15:15:28 1995 From: omlinc at research.nj.nec.com (Christian Omlin) Date: Mon, 27 Nov 1995 15:15:28 -0500 Subject: paper available Message-ID: <199511272015.PAA18113@arosa> Following the announcement of the paper by Wolfgang Maas on the computational power of networks consisting of neurons that communicate via spike trains, we thought the following paper may be of interest to the connectionist community. It can be retrieved from the website http://www.neci.nj.nec.com/homepages/omlin/omlin.html We welcome your comments. -Christian ======================================================================= Training Recurrent Neural Networks with Temporal Input Encodings C.W. Omlin (a,b), C.L. Giles (a,c), B.G. Horne (a) L.R. Leerink (d), T. Lin (a) (a) NEC Research Institute 4 Independence Way Princeton, NJ 08540 (b) CS Department RPI Troy, NY 12180 (c) UMIACS University of Maryland College Park, MD 20742 (d) EE Department The University of Sydney Sydney, NSW 2006 Abstract We investigate the learning of deterministic finite-state automata (DFA's) with recurrent networks with a single input neuron, where each input symbol is represented as a temporal pattern and strings as sequences of temporal patterns. We empirically demonstrate that obvious temporal encodings can make learning very difficult or even impossible. Based on preliminary results, we formulate some hypotheses about increase training time compared to training of networks with multiple input neurons. From jkim at FIZ.HUJI.AC.IL Mon Nov 27 14:38:09 1995 From: jkim at FIZ.HUJI.AC.IL (Jai Won Kim) Date: Mon, 27 Nov 1995 21:38:09 +0200 Subject: Preprint announcement: Online-Gibbs Learning Message-ID: <199511271938.AA15158@keter.fiz.huji.ac.il> Dear Jordan Pollack We would like to post an announcement of a new preprint on your network. We attach below the title, authors as well as the abstract of the paper. Subject: announcement of a new preprint: On-line Gibbs Learning FTP-host: keter.fiz.huji.ac.il FTP-file: pub/ON-LINE-LEARNING/online_gibbs.ps.Z The length of the paper: 4 pages. Thanking you in advance for your help. Regard, Haim Sompolinsky and Jaiwon Kim e-mails : haim at fiz.huji.ac.il, jkim at fiz.huji.ac.il _______________________________________________________________________________o On-line Gibbs Learning J. W. Kim and H. Sompolinsky Racah Institute of Physics and Center for Neural Computation, Hebrew University, Jerusalem 91904, Israel e-mails: jkim at fiz.huji.ac.il ; haim at fiz.huji.ac.il (Submitted to Physical Review Letters, Nov 95) ABSTRACT We propose a new model of on-line learning which is appropriate for learning of realizable and unrealizable, smooth as well as threshold, functions. Following each presentation of an example the new weights are chosen from a Gibbs distribution with an on-line energy that balances the need to minimize the instantaneous error against the need to minimize the change in the weights. We show that this algorithm finds the weights that minimize the generalization error in the limit of infinite number of examples. The asymptotic rate of convergence is similar to that of batch learning. From Weevey at cris.com Mon Nov 27 21:47:34 1995 From: Weevey at cris.com (WEEVEY) Date: Mon, 27 Nov 1995 21:47:34 -0500 (EST) Subject: Dissn Research Summary - Primary Visual Cortex In-Reply-To: <9511260703.AA04033@triangulum> Message-ID: The following is the abstract from my dissertation which was completed back in May. More information about this research may be found at the following URL: http://www.cris.com/~Weevey. Sincerely, Eva S. Simmons ************************************************************************ CIRCUITRY STUDIES OF A COMPUTATIONAL MODEL OF THE PRIMARY VISUAL CORTEX Eva Sabrina Simmons, Ph.D. University of Texas at Austin, 1995 Supervisor: Robert E. Wyatt The goals of this project include: (1) proving a procedure for circuit determination of any cytoarchietectonic area of the brain, given certain kinds of data are known as in the computational model being used here, and (2) one circuit of activity will be proposed with three variations on it by changing the connection strength from the standard. Applying the concept of the connection matrix and obtaining basic statistical data about the connec- tions present with respect to presynaptic cells, basic connection data are obtained as the specified anatomy of the cells and random placement in appro- priate layers has allowed. Also, by allowing activity over a period of 20 ms, time propagation data are produced. By keeping a record of activated and deactivated cells at each time step whose minor types have been read-in from a file and by figuring out exactly how each cell was activated, pieces of the circuits can be produced. Later a circuit diagram can be produced from this data. The sets used for this study are: 400 and 2000 cell sets for basic data, and 1000 and 2000 cell sets for variations of connection strength. The following conclusions can be made: (1) The data shows increase in cell type activity with an increase in cell count in the first two time intervals (0.00-5.00 ms). (2) The pattern seen over the time intervals is: The first time interval A (0.00-2.50 ms), is always a period of immense activity. During the second time interval, B (2.55-5.00 ms), the activity continues to be heavy, with no new cell types being activated. The following time inter- vals, C through H (5.05-20.00 ms), moderate activity occurs. (3) The pattern of activity, as found in experiment, is also found here. (4) A pattern of cell type activity is seen when comparing sets to some degree, with some changes depending on cell count and variations in connection strength. (5) The circuits that have been found were as expected in the literature. @}---------- THE SIMMONS FACTOR --------- EVA SABRINA SIMMONS, PH.D. -------{@ WWW Personal Page: http://www.cris.com/~Weevey/index.html @}---- @}---- @}---- WATCH IT, OR IT MIGHT ATTACK!! ;) ---{@ ---{@ ---{@ ---{@ From andreas at sabai.cs.colorado.edu Tue Nov 28 04:43:58 1995 From: andreas at sabai.cs.colorado.edu (Andreas Weigend) Date: Tue, 28 Nov 1995 02:43:58 -0700 (MST) Subject: NIPS Time Series Workshop / Final List Message-ID: <199511280943.CAA12163@sabai.cs.colorado.edu> A non-text attachment was scrubbed... Name: not available Type: text Size: 3165 bytes Desc: not available Url : https://mailman.srv.cs.cmu.edu/mailman/private/connectionists/attachments/00000000/e6297b88/attachment-0001.ksh From A.Sharkey at dcs.shef.ac.uk Tue Nov 28 09:31:51 1995 From: A.Sharkey at dcs.shef.ac.uk (A.Sharkey@dcs.shef.ac.uk) Date: Tue, 28 Nov 95 14:31:51 GMT Subject: Special issue of Connection Science Message-ID: <9511281431.AA02615@entropy.dcs.shef.ac.uk> ************ COMBINING NEURAL NETS ************ CALL FOR PAPERS: Deadline February 14th 1996 Papers are sought for this special issue of Connection Science. The aim of this special issue is to examine when, how, and why neural nets should be combined. The reliability of neural nets can be increased through the use of both redundant and modular nets, (either trained on the same task under differing conditions, or on different subcomponents of a task). Questions about the exploitation of redundancy and modularity in the combination of nets, or estimators, have both an engineering and a biological relevance, and include the following: * how best to combine the outputs of several nets. * quantification of the benefits of combining. * how best to create redundant nets that generalise differently (e.g. active learning methods). * how to effectively subdivide a task. * communication between neural net modules. * increasing the reliability of nets. * the use of neural nets for safety critical applications. Special issue editor: Amanda Sharkey (Sheffield, UK) Editorial Board: Leo Breiman (Berkeley, USA) Nathan Intrator (Brown, USA) Robert Jacobs (Rochester, USA) Michael Jordan (MIT, USA) Paul Munro (Pittsburgh, USA) Michael Perrone (IBM, USA) David Wolpert (Santa Fe Institute, USA) We solicit either theoretical or experimental papers on this topic. Questions and submissions concerning this special issue should be sent by February 14th 1996 to: Dr Amanda Sharkey,Department of Computer Science,Regent Court,Portobello Street,University of Sheffield,Sheffield, S1 4DP,United Kingdom. Email: amanda at dcs.shef.ac.uk From ruppin at math.tau.ac.il Tue Nov 28 12:23:55 1995 From: ruppin at math.tau.ac.il (Eytan Ruppin) Date: Tue, 28 Nov 1995 19:23:55 +0200 Subject: MEMORY Message-ID: Adams Super Center for Brain Studies at Tel Aviv University =========================================================== Workshop Announcement MEMORY ORGANIZATION AND CONSOLIDATION: COGNITIVE AND COMPUTATIONAL PERSPECTIVES A workshop on Memory Organization and Consolidation will be held during May 28-30, 1996 at Tel-Aviv University, Israel. This meeting is sponsored by Mr. Branco Weiss. In the last two decades the field of memory research has grown tremendously. This rapid expansion has been manifested in the recognition of the multiplicity of memory systems and the rise in popularity of multiple-memory system analysis, in the ability to trace changes of brain activity during memory performance using novel molecular, electrophysiological and imaging techniques, and in the development of fairly complex models of memory. The planned workshop will address these issues and discuss how memory storage and retrieval processes are organized in the brain. In particular, we shall focus on memory consolidation. This process involves the alteration of memory traces from temporary, `short-term' storage to `long-term' memory stores. It is a fundamental and intriguing process, which is considered to be strongly connected to our ability to form generalizations by learning from examples, and may depend also upon the integrity of specific sleep stages. The process of consolidation has recently become accessible to formal analysis using novel computational neural models. The workshop will provide a meeting ground for both experimental and computational research approaches. Numerous questions arise with regard to consolidation theory: What explanations could it offer? What are its neuronal (molecular, neurochemical) foundations? What are the relations between the consolidation processes and the circadian cycle? What are modulators of consolidation? What insights can be gained from computational models and how can the predictions they make be tested experimentally? The multidisciplinary nature of memory consolidation research, together with recent advancements, make the proposed workshop a promising opportunity for a timely and fruitful exchange of ideas between researchers employing different research methodologies, but sharing common interests in the study of memory. The workshop will consist of a three day meeting, and will include a series of invited talks, a poster session, and discussion panels. We have invited speakers from different disciplines of the Neurosciences who will discuss psychological, neurological, physiological and computational perspectives of the subject. An informal atmosphere will be maintained, encouraging questions and discussions. CURRENTLY CONFIRMED SPEAKERS Martin Albert (Boston) Daniel Amit (Jerusalem and Rome) Yadin Dudai (Weizmann Institute) Yoram Feldon (Zurich and Tel Aviv) Mark Gluck (Rutgers) Michael Hasselmo (Harvard) Avi Karni (NIH) Amos Korzcyn (Tel Aviv) Jay McClelland (CMU) Bruce McNaughton (Arizona) Matti Mintz (Tel Aviv) Morris Moscovitch (Toronto) Richard Thompson (USC) CALL FOR ABSTRACTS Individuals wishing to present a poster related to any aspect of the workshop's theme should submit an abstract describing the nature of their presentation. The single page submission should include title, author(s), contact information (address and email/fax), and abstract, and will be reviewed by the Program Committee. Abstract submissions should ARRIVE by March 31st, 1996, and should be sent to Eytan Ruppin, Dept. of Computer-Science, Tel-Aviv University, Tel-Aviv, Israel, 69978. Program Committee: ----------------- David Horn, Michael Mislobodsky and Eytan Ruppin (Tel-Aviv). Registration and Further Information: ----------------------------------- To register for the workshop, please fill out the registration form attached below and send it to Mrs. Bila Lenczner Adams Super Center for Brain Studies Tel Aviv University Tel Aviv 69978, Israel Tel.: 972-3-6407377 Fax: 972-3-6407932 email:memory at neuron.tau.ac.il The workshop will take place at the Gordon faculty club of Tel Aviv University. The registration fee of $70 covers lunch and refreshments throughout the three days of the workshop. Optionally one may register for $30 covering refreshments only. Since the number of places is limited please register early to secure your participation. Further questions about conference administration should be directed to Mrs. Bila Lenczner. For questions about the workshop technical/scientific content or absract submissions, please contact Eytan Ruppin Dept. of Computer Science Tel-Aviv University, Tel-Aviv, 69978, Israel. Tel.: 972-3-6407864 Fax: 972-3-6409357 email: ruppin at math.tau.ac.il The final workshop program and updated information will be available on a WWW homepage at http://neuron.tau.ac.il/Adams/memory ================================================================== REGISTRATION FORM MEMORY WORKSHOP May 28-30, 1996 Name: ___________________________________________________ Affiliation: ________________________________________________ Address: _________________________________________________ _________________________________________________________ Telephone: ___________________________ Fax: ________________________________ e-mail: ______________________________ ___ $70 Registration fee including lunch ___ $30 Registration fee including refreshments only Amount Enclosed: $________________ MAKE CHECKS PAYABLE TO "Tel Aviv University" From jbower at bbb.caltech.edu Tue Nov 28 22:21:05 1995 From: jbower at bbb.caltech.edu (jbower@bbb.caltech.edu) Date: Tue, 28 Nov 95 19:21:05 PST Subject: Call for Papers -- CNS*96 Message-ID: ********************************************************************** CALL FOR PAPERS Fifth Annual Computational Neuroscience Meeting CNS*96 July 14 - 17, 1996 Boston, Massachusetts ................ DEADLINE FOR SUMMARIES AND ABSTRACTS: **>> January 25, 1996 <<** ^^^^^^^^^^^^^^^^ This is the fifth annual meeting of an interdisciplinary conference addressing a broad range of research approaches and issues involved in the field of computational neuroscience. These meetings bring together experimental and theoretical neurobiologists along with engineers, computer scientists, cognitive scientists, physicists, and mathematicians interested in the functioning of biological nervous systems. The peer reviewed papers presented at the conference are all related to understanding how nervous systems compute. As in previous years, CNS*96 will equally emphasize experimental, model-based, and more abstract theoretical approaches to understanding neurobiological computation. The meeting in 1996 will take place at the Cambridge Center Marriott Hotel and include plenary, contributed, and poster sessions. The first session starts at 9 am, Sunday July 14th and the last session ends at 5 pm on Wednesday, July 17th. There will be no parallel sessions and the full text of presented papers will be published in a proceedings volume. The meeting will include time for informal workshops focused on current issues in computational neuroscience. Travel funds will be available for students and postdoctoral fellows presenting papers at the meeting. Day care will be available for children. SUBMISSION INSTRUCTIONS: With this announcement we solicit the submission of papers for presentation. All papers will be refereed. Authors should send original research contributions in the form of a 1000-word (or less) summary and a separate single page 100 word abstract clearly stating their results. Summaries are for program committee use only. Abstracts will be published in the conference program. At the bottom of each abstract page and on the first summary page, indicate preference for oral or poster presentation and specify at least one appropriate category and theme from the following list: Presentation categories: A. Theory and Analysis B. Modeling and Simulation C. Experimental D. Tools and Techniques Themes: A. Development B. Cell Biology C. Excitable Membranes and Synaptic Mechanisms D. Neurotransmitters, Modulators, Receptors E. Sensory Systems 1. Somatosensory 2. Visual 3. Auditory 4. Olfactory 5. Other systems F. Motor Systems and Sensory Motor Integration G. Learning and Memory H. Behavior I. Cognition J. Disease Include addresses of all authors on the front of the summary and the abstract including the E-mail address for EACH author. Indicate on the front of the summary to which author correspondence should be addressed. Also, indicate first author. Program committee decisions will be sent to the corresponding author only. Submissions will not be considered if they lack category information, separate abstract sheets, author addresses, or are late. Submissions can be made by surface mail ONLY by sending 6 copies of the abstract and summary to: CNS*96 Submissions Division of Biology 216-76 Caltech Pasadena, CA 91125 ADDITIONAL INFORMATION can be obtained by: o Using our on-line WWW information and registration server at the URL: http://www.bbb.caltech.edu/cns96/cns96.html o ftp-ing to our ftp site: yourhost% ftp ftp.bbb.caltech.edu Name: ftp Password: yourname at yourhost.yoursite.yourdomain ftp> cd pub/cns96 ftp> ls ftp> get filename o Sending Email to: cns96 at smaug.bbb.caltech.edu CNS*96 ORGANIZING COMMITTEE: Co-meeting chair / logistics - Mike Hasselmo, Harvard University Co-meeting chair / finances - John Miller, UC Berkeley Co-meeting chair / program - Jim Bower, Caltech Program Committee: Charlie Anderson, Washington University Axel Borst, Max-Planck Inst., Tuebingen, Germany Dennis Glanzman, NIMH/NIH Nancy Kopell, Boston University Christiane Linster, Harvard University Mark Nelson, University of Illinois, Urbana Maureen Rush, California State University, Bakersfield Karen Sigvardt, University of California, Davis Philip Ulinski, University of Chicago Regional Organizers: Europe- Erik De Schutter (Belgium) Middle East - Idan Segev (Jerusalem) Down Under - Mike Paulin (New Zealand) South America - Renato Sabbatini (Brazil) Asia - Zhaoping Li (Hong Kong) ********************************************************************** *************************************** James M. Bower Division of Biology Mail code: 216-76 Caltech Pasadena, CA 91125 (818) 395-6817 (818) 449-0679 FAX NCSA Mosaic addresses for: laboratory http://www.bbb.caltech.edu/bowerlab GENESIS: http://www.bbb.caltech.edu/GENESIS science education reform http://www.caltech.edu/~capsi From tgelder at phil.indiana.edu Wed Nov 29 00:20:42 1995 From: tgelder at phil.indiana.edu (Tim van Gelder) Date: Wed, 29 Nov 1995 00:20:42 -0500 Subject: 'Mind as Motion' annct & web page Message-ID: Book announcement ::: Available now. `MIND AS MOTION: EXPLORATIONS IN THE DYNAMICS OF COGNITION' edited by Robert Port and Tim van Gelder Bradford Books/MIT Press. >From the dust jacket: `Mind as Motion is the first comprehensive presentation of the dynamical approach to cognition. It contains a representative sampling of original current research on topics such as perception, motor control, speech and language, decision making, and development. Included are chapters by pioneers of the approach, as well as others applying the tools of dynamics to a wide range of new problems. Throughout, particular attention is paid to the philosophical foundations of this radical new research program. Mind as Motion provides a conceptual and historical overview of the dynamical approach, a tutorial introduction to dynamics for cognitive scientists, and a glossary covering the most frequently used terms. Each chapter includes an introduction by the editors, outlining its main ideas and placing it in context, and a guide to further reading.' 668 pages, 139 illustrations ISBN 0-262-16150-8 $60.00 US For further information including the full text of the preface and a sample chapter introduction, see the web page at MIT Press: http://www-mitpress.mit.edu/mitp/recent-books/cog/mind-as-motion.html _________________________________________________ Chapter Titles and Authors 1.Tim van Gelder & Robert Port. Introduction: Its About Time: An Overview of the Dynamical Approach to Cognition. 2. Alec Norton. Dynamics: A Tutorial Introduction for Cognitive Scientists. 3. Esther Thelen. Time Scale Dynamics and the Development of an Embodied Cognition. 4. Jerome Busemeyer & James Townsend Dynamic Representation of Decision Making 5. Randall Beer Computational and Dynamical Languages for Autonomous Agents 6. Elliot Saltzman Dynamics and Coordinate Systems in Skilled Sensorimotor Activity 7. Catherine Browman & Louis Goldstein Dynamics and Articulatory Phonology 8. Jeffrey Elman Language as a Dynamical System 9. Jean Petitot Morphodynamics and Attractor Syntax 10. Jordan Pollack The Induction of Dynamical Recognizers 11. Paul van Geert Growth Dynamics in Development 12. Robert Port, Fred Cummins & Devin McAuley Naive Time, Temporal Patterns and Human Audition 13. Michael Turvey & Claudia Carello Some Dynamical Themes in Perception and Action 14. Geoffrey Bingham Dynamics and the Problem of Event Recognition 15. Stephen Grossberg Neural Dynamics of Motion Perception, Recognition Learning, and Spatial Attention 16. Mary Ann Metzger Multiprocess Models Applied to Cognitive and Behavioral Dynamics 17. Steven Reidbord & Dana Redington The Dynamics of Mind and Body During Clinical Interviews: Current Trends, Potential, and Future Directions 18. Marco Giunti Dynamical Models of Cognition 19. Glossary of Terminology in Dynamics _________________________________________________ At MIT Press, orders can be made by email at: mitpress-orders at mit.edu For general information re MIT Press, see: http://www-mitpress.mit.edu _________________________________________________ The editors welcome enquiries, discussion, critical feedback, etc. Robert Port Tim van Gelder Department of Linguistics Department of Philosophy Indiana University University of Melbourne Bloomington IN 47405 Parkville 3052 VIC USA AUSTRALIA port at indiana.edu tgelder at ariel.unimelb.edu.au From josh at faline.bellcore.com Wed Nov 29 13:19:47 1995 From: josh at faline.bellcore.com (Joshua Alspector) Date: Wed, 29 Nov 1995 13:19:47 -0500 Subject: Research associate in neuromorphic electronics Message-ID: <199511291819.NAA15771@faline.bellcore.com> RESEARCH ASSOCIATE IN NEUROMORPHIC ELECTRONICS There is an anticipated position in the electrical and computer engineering department at the University of Colorado at Colorado Springs for a postdoctoral research associate in the area of neural learning microchips. The successful candidate will have experience in analog and digital VLSI design and test and be comfortable working at the system level in a UNIX/C/C++ environment. The project will involve applying an existing VME-based neural network learning system to several demanding problems in signal processing. These include adaptive non-linear equalization of underwater acoustic communication channels and magnetic recording channels. It is likely also to involve integrating the learning electronics with micro-machined sonic transducers directly on silicon. Please send a curriculum vita, names and addresses of at least three referees, and and copies of some representative publications to: Prof. Joshua Alspector Univ. of Colorado at Col. Springs Dept. of Elec. & Comp. Eng. P.O. Box 7150 Colorado Springs, CO 80933-7150 From ajit at austin.ibm.com Wed Nov 29 14:46:04 1995 From: ajit at austin.ibm.com (Dingankar) Date: Wed, 29 Nov 1995 13:46:04 -0600 Subject: "Network Approximation of Dynamical Systems" - Neuroprose paper Message-ID: <9511291946.AA12676@ding.austin.ibm.com> **DO NOT FORWARD TO OTHER GROUPS** Sorry, no hardcopies available. 6 pages. Greetings! The following invited paper will be presented at NOLTA'95 next month. The compressed PostScript file is available in the Neuroprose archive; the details (URL, bibtex entry and abstract) follow. Thanks, Ajit ------------------------------------------------------------------------------ URL: ftp://archive.cis.ohio-state.edu/pub/neuroprose/dingankar.tensor-products.ps.Z BiBTeX entry: @INPROCEEDINGS{atd:nolta-95, AUTHOR ="Dingankar, Ajit T. and Sandberg, Irwin W.", TITLE ="{Network Approximation of Dynamical Systems}", BOOKTITLE ="Proceedings of the International Symposium on Nonlinear Theory and its Applications (NOLTA'95)", YEAR ="1995", EDITOR ="", PAGES ="", ORGANIZATION ="", PUBLISHER ="", ADDRESS ="Las Vegas, Nevada", MONTH ="December 10--14" } Network Approximation of Dynamical Systems ------------------------------------------ ABSTRACT We consider the problem of approximating any member of a large class of input-output operators of time-varying nonlinear dynamical systems. We introduce a family of ``tensor product" dynamical neural networks, and show that a certain continuity condition is necessary and sufficient for the existence of arbitrarily good approximations using this family. ------------------------------------------------------------------------------ Ajit T. Dingankar | ajit at austin.ibm.com IBM Corporation, Internal Zip 4359 | Work: (512) 838-6850 11400 Burnet Road, Austin, TX 78758 | Fax : (512) 838-5882