From Connectionists-Request at cs.cmu.edu Sat Jan 1 00:05:12 1994 From: Connectionists-Request at cs.cmu.edu (Connectionists-Request@cs.cmu.edu) Date: Sat, 01 Jan 94 00:05:12 EST Subject: Bi-monthly Reminder Message-ID: <16894.757400712@B.GP.CS.CMU.EDU> *** DO NOT FORWARD TO ANY OTHER LISTS *** This note was last updated January 4, 1993. This is an automatically posted bi-monthly reminder about how the CONNECTIONISTS list works and how to access various online resources. CONNECTIONISTS is not an edited forum like the Neuron Digest, or a free-for-all newsgroup like comp.ai.neural-nets. It's somewhere in between, relying on the self-restraint of its subscribers. Membership in CONNECTIONISTS is restricted to persons actively involved in neural net research. The following posting guidelines are designed to reduce the amount of irrelevant messages sent to the list. Before you post, please remember that this list is distributed to over a thousand busy people who don't want their time wasted on trivia. Also, many subscribers pay cash for each kbyte; they shouldn't be forced to pay for junk mail. Happy hacking. -- Dave Touretzky & David Redish --------------------------------------------------------------------- What to post to CONNECTIONISTS ------------------------------ - The list is primarily intended to support the discussion of technical issues relating to neural computation. - We encourage people to post the abstracts of their latest papers and tech reports. - Conferences and workshops may be announced on this list AT MOST twice: once to send out a call for papers, and once to remind non-authors about the registration deadline. A flood of repetitive announcements about the same conference is not welcome here. - Requests for ADDITIONAL references. This has been a particularly sensitive subject lately. Please try to (a) demonstrate that you have already pursued the quick, obvious routes to finding the information you desire, and (b) give people something back in return for bothering them. The easiest way to do both these things is to FIRST do the library work to find the basic references, then POST these as part of your query. Here's an example: WRONG WAY: "Can someone please mail me all references to cascade correlation?" RIGHT WAY: "I'm looking for references to work on cascade correlation. I've already read Fahlman's paper in NIPS 2, his NIPS 3 abstract, and found the code in the nn-bench archive. Is anyone aware of additional work with this algorithm? I'll summarize and post results to the list." - Announcements of job openings related to neural computation. - Short reviews of new text books related to neural computation. To send mail to everyone on the list, address it to Connectionists at CS.CMU.EDU ------------------------------------------------------------------- What NOT to post to CONNECTIONISTS: ----------------------------------- - Requests for addition to the list, change of address and other administrative matters should be sent to: "Connectionists-Request at cs.cmu.edu" (note the exact spelling: many "connectionists", one "request"). If you mention our mailing list to someone who may apply to be added to it, please make sure they use the above and NOT "Connectionists at cs.cmu.edu". - Requests for e-mail addresses of people who are believed to subscribe to CONNECTIONISTS should be sent to postmaster at appropriate-site. If the site address is unknown, send your request to Connectionists-Request at cs.cmu.edu and we'll do our best to help. A phone call to the appropriate institution may sometimes be simpler and faster. - Note that in many mail programs a reply to a message is automatically "CC"-ed to all the addresses on the "To" and "CC" lines of the original message. If the mailer you use has this property, please make sure your personal response (request for a Tech Report etc.) is NOT broadcast over the net. - Do NOT tell a friend about Connectionists at cs.cmu.edu. Tell him or her only about Connectionists-Request at cs.cmu.edu. This will save your friend from public embarrassment if she/he tries to subscribe. ------------------------------------------------------------------------------- The CONNECTIONISTS Archive: --------------------------- All e-mail messages sent to "Connectionists at cs.cmu.edu" starting 27-Feb-88 are now available for public perusal. A separate file exists for each month. The files' names are: arch.yymm where yymm stand for the obvious thing. Thus the earliest available data are in the file: arch.8802 Files ending with .Z are compressed using the standard unix compress program. To browse through these files (as well as through other files, see below) you must FTP them to your local machine. ------------------------------------------------------------------------------- How to FTP Files from the CONNECTIONISTS Archive ------------------------------------------------ 1. Open an FTP connection to host B.GP.CS.CMU.EDU (Internet address 128.2.242.8). 2. Login as user anonymous with password your username. 3. 'cd' directly to one of the following directories: /usr/connect/connectionists/archives /usr/connect/connectionists/bibliographies 4. The archives and bibliographies directories are the ONLY ones you can access. You can't even find out whether any other directories exist. If you are using the 'cd' command you must cd DIRECTLY into one of these two directories. Access will be denied to any others, including their parent directory. 5. The archives subdirectory contains back issues of the mailing list. Some bibliographies are in the bibliographies subdirectory. Problems? - contact us at "Connectionists-Request at cs.cmu.edu". Anonymous FTP on archive.cis.ohio-state.edu (128.146.8.52) pub/neuroprose directory This directory contains technical reports as a public service to the connectionist and neural network scientific community which has an organized mailing list (for info: connectionists-request at cs.cmu.edu) Researchers may place electronic versions of their preprints in this directory, announce availability, and other interested researchers can rapidly retrieve and print the postscripts. This saves copying, postage and handling, by having the interested reader supply the paper. We strongly discourage the merger into the repository of existing bodies of work or the use of this medium as a vanity press for papers which are not of publication quality. PLACING A FILE To place a file, put it in the Inbox subdirectory, and send mail to pollack at cis.ohio-state.edu. Within a couple of days, I will move and protect it, and suggest a different name if necessary. Current naming convention is author.title.filetype.Z where title is just enough to discriminate among the files of the same author. The filetype is usually "ps" for postscript, our desired universal printing format, but may be tex, which requires more local software than a spooler. The Z indicates that the file has been compressed by the standard unix "compress" utility, which results in the .Z affix. To place or retrieve .Z files, make sure to issue the FTP command "BINARY" before transfering files. After retrieval, call the standard unix "uncompress" utility, which removes the .Z affix. An example of placing a file is in the appendix. Make sure your paper is single-spaced, so as to save paper, and include an INDEX Entry, consisting of 1) the filename, 2) the email contact for problems, 3) the number of pages and 4) a one sentence description. See the INDEX file for examples. ANNOUNCING YOUR PAPER It is the author's responsibility to invite other researchers to make copies of their paper. Before announcing, have a friend at another institution retrieve and print the file, so as to avoid easily found local postscript library errors. And let the community know how many pages to expect on their printer. Finally, information about where the paper will/might appear is appropriate inside the paper as well as in the announcement. Please add two lines to your mail header, or the top of your message, so as to facilitate the development of mailer scripts and macros which can automatically retrieve files from both NEUROPROSE and other lab-specific repositories: FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/filename.ps.Z When you announce a paper, you should consider whether (A) you want it automatically forwarded to other groups, like NEURON-DIGEST, (which gets posted to comp.ai.neural-networks) and if you want to provide (B) free or (C) prepaid hard copies for those unable to use FTP. To prevent forwarding, place a "**DO NOT FORWARD TO OTHER GROUPS**" at the top of your file. If you do offer hard copies, be prepared for a high cost. One author reported that when they allowed combination AB, the rattling around of their "free paper offer" on the worldwide data net generated over 2000 hardcopy requests! A shell script called Getps, written by Tony Plate, is in the directory, and can perform the necessary retrieval operations, given the file name. Functions for GNU Emacs RMAIL, and other mailing systems will also be posted as debugged and available. At any time, for any reason, the author may request their paper be updated or removed. For further questions contact: Jordan Pollack Assistant Professor CIS Dept/OSU Laboratory for AI Research 2036 Neil Ave Email: pollack at cis.ohio-state.edu Columbus, OH 43210 Phone: (614) 292-4890 APPENDIX: Here is an example of naming and placing a file: gvax> cp i-was-right.txt.ps rosenblatt.reborn.ps gvax> compress rosenblatt.reborn.ps gvax> ftp cheops.cis.ohio-state.edu Connected to cheops.cis.ohio-state.edu. 220 cheops.cis.ohio-state.edu FTP server ready. Name: anonymous 331 Guest login ok, send ident as password. Password:neuron 230 Guest login ok, access restrictions apply. ftp> binary 200 Type set to I. ftp> cd pub/neuroprose/Inbox 250 CWD command successful. ftp> put rosenblatt.reborn.ps.Z 200 PORT command successful. 150 Opening BINARY mode data connection for rosenblatt.reborn.ps.Z 226 Transfer complete. 100000 bytes sent in 3.14159 seconds ftp> quit 221 Goodbye. gvax> mail pollack at cis.ohio-state.edu Subject: file in Inbox. Jordan, I just placed the file rosenblatt.reborn.ps.Z in the Inbox. Here is the INDEX entry: rosenblatt.reborn.ps.Z rosenblatt at gvax.cs.cornell.edu 17 pages. Boastful statements by the deceased leader of the neurocomputing field. Let me know when it is in place so I can announce it to Connectionists at cmu. Frank ^D AFTER FRANK RECEIVES THE GO-AHEAD, AND HAS A FRIEND TEST RETRIEVE THE FILE, HE DOES THE FOLLOWING: gvax> mail connectionists Subject: TR announcement: Born Again Perceptrons FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/rosenblatt.reborn.ps.Z The file rosenblatt.reborn.ps.Z is now available for copying from the Neuroprose repository: Born Again Perceptrons (17 pages) Frank Rosenblatt Cornell University ABSTRACT: In this unpublished paper, I review the historical facts regarding my death at sea: Was it an accident or suicide? Moreover, I look over the past 23 years of work and find that I was right in my initial overblown assessments of the field of neural networks. ~r.signature ^D ------------------------------------------------------------------------ How to FTP Files from the NN-Bench Collection --------------------------------------------- 1. Create an FTP connection from wherever you are to machine "pt.cs.cmu.edu" (128.2.254.155). 2. Log in as user "anonymous" with password your username. 3. Change remote directory to "/afs/cs/project/connect/bench". Any subdirectories of this one should also be accessible. Parent directories should not be. 4. At this point FTP should be able to get a listing of files in this directory and fetch the ones you want. Problems? - contact us at "nn-bench-request at cs.cmu.edu". From terry at salk.edu Mon Jan 3 19:57:12 1994 From: terry at salk.edu (Terry Sejnowski) Date: Mon, 3 Jan 94 16:57:12 PST Subject: NEURAL COMPUTATION 6:1 Message-ID: <9401040057.AA21592@salk.edu> Neural Computation -- January, 1994 -- Volume 6 Number 1 Article: Cortical Map Reorganization as a Competitve Process Granger G. Sutton III, James A. Reggia, Steven L. Armentrout, and C. Lynne D'Autrechy Note: An Efficient Method for Computing Synaptic Conductances Based on a Kinetic Model of Receptor Binding A. Destexhe, Z. F. Mainen and T. J. Sejnowski Letters: A Neural Network for Coding of Trajectories by Time Series of Neuronal Population Vectors Apostolos P. Georgopoulos and Alexander V. Lukashin Theoretical Considerations for the Analysis of Population Coding in Motor Cortex Terence D. Sanger Neural Network Model of the Cerebellum: Temporal Discrimination and the Timing of Motor Responses Dean V. Buonomano and Michael D. Mauk Computational Aspects of the Respiratory Pattern Generator Allan Gottschalk, Malcolm D. Ogilvie Diethelm W. Richter and Allan I. Pack Subharmonic Coordination in Networks of Neurons with Slow Conductances Thomas LoFaro, Nancy Kopell, Eve Marder and Scott L. Hooper Setting the Activity Level in Sparse Random Networks Ali A. Minai and William B. Levy The Role of Constraints in Hebbian Learning Kenneth D. Miller and David J. C. MacKay Towards a Theory of the Striate Cortex Zhaoping Li and Joseph J. Atick Fast Exact Multiplication by the Hessian Barak A. Pearlmutter Polyhedral Combinatorics and Neural Networks Andrew H. Gee and Richard W. Prager From linster at neurones.espci.fr Mon Jan 3 10:12:03 1994 From: linster at neurones.espci.fr (Christiane LINSTER) Date: Mon, 3 Jan 94 10:12:03 MET Subject: Please distribute widely (fwd) Message-ID: <9401030918.AA24787@Hugo.espci.fr> > > > First Announcement and Call for Papers > > THE FIRST EUROPEAN CONFERENCE ON > "COGNITIVE SCIENCE IN INDUSTRY" > > 28th - 30th September 1994 - Luxembourg > > > > OBJETCIVES > > The study of human operational strategies and human processes is a particularly > well developed area of Cognitive Science. This particular domain belongs to the > larger field of Information Science, which considers semantic and qualitative > aspects of information, rather than digital and qualitative ones. This > represents an important issue for industries concerned with human beings in a > work place context because Cognitive Science provides a great number of > additional dimensions allowing to conceive more complete systems. > > It is the purpose of the European Conference on Cognitive Science to assemble > theoreticians and practitioners from industry and academic institutions to > discuss questions related to the practical application of Cognitive Science in > industrial settings. The conference will be a forum for those who recognize the > need to develop theories, methods, and systems to ensure the transfer of > laboratory results into real-life problem situations. Furthermore, the European > Conference on Cognitive Science will facilitate the exchange of expertise among > practitioners. > > The organizing committee plans to hold the conference in collaboration with the > following institutions: > > GDR-CNRS 957 Sciences Cognitives de Paris, France > Ecole Nationale Superieure de Telecommunication de Bretagne, France > Universite Paris Sorbonne, France > Centre de Recherche Public - CU, Luxembourg > Centre Universitaire, Luxembourg > DFKI, Germany > Universitat Kaiserslautern, Germany > Universitat Freiburg, Germany > > > CALL FOR PAPERS > > Two types of papers are solicited: those discussing on cognitive theories in > the context of industrial practice and those presenting industrial applications. > Suitable conferences and panel discussion themes include: > > o Cognitive robotics > o Man-Machine cooperation > o Cognitive organization and cooperation > o Intelligent assistance for decision > o Knowledge engineering, acquisition and modeling > o Distributed cognition and multi agents system > o Perception, recognition, interpretation and action > o Planning > o Intelligent management of multimedia documents > o Piloting of important projects > > > Applications, ongoing developments and relevant experimentations > in the areas of: > o Transports > o Bank > o Assurances > o Health > o Telecommunications > o Environment protection > o Production > o Supervision and control > > > SUBMISSION DETAILS > > Two classes of submission are solicited - long papers, which should not exceed > 30 pages, and short papers, which should be up to 10 pages. Reports of > industrial applications should ideally be presented as short papers. The program > committee is seeking for original papers in all areas concerning the use of > cognitive science for industrial problem solving. Papers could either describe > industrial cases, significant results from ongoing research or user experience, > or offer a critical analysis of current theories, tools or techniques in this > domain. > > > IMPORTANT DATES > > Papers should be submitted by the 7th March 1994. Notification of acceptance > will be by the 30th May 1994. The final version of accepted papers must be > received by the 4th July. > > > GENERAL CHAIRMAN > > Jean-Pierre Barthelemy Telecom Bretagne, France > > > PROGRAM CO-CHAIRMEN > > Raymond Bisdorff CRP-CU, Luxembourg > Jean-Pierre Descles ISHA Paris Sorbonne, France > > > ADVISORY COMMITTEE > > Actual open list: > > Nath. Aussenac-Gilles IRIT U. Sabatier, Toulouse, France > Beatrice Bacconet Thomson - CSF, France > Jean-Pierre Barthelemy Telecom Bretagne, France > Raymond Bisdorff CRP-CU, Luxembourg > Regine Bourgine CNRS (GRID-ENS), France > Jean-Pierre Descles ISHA Paris Sorbonne, France > M. Founeau E.D.F. Chatou, France > Fernand Grosber TrefilARBED, Luxembourg > Jean-Michel Le Bot Credit Mutuel de Bretagne, France > Marc Linster DEC, USA > Chr. de Maindreville Alcatel Alsthom, France > Jacques Mathieu LAFORIA Paris Jussieu, France > Olivier Paillet Alcatel Alsthom Recherche, France > Fernand Reinig CRP-CU, Luxembourg > Michael Richter Universitat Kaiserslautern, Germany > Susan Spirgi Swiss Bank Corp. Zuerich, Switzerland > Patrice Taillibert Dassault Electronique, France > Gerhard Strube Universitat Freiburg, Germany > Christian Tora EDIAT, France > > > LOCAL ORGANIZATION COMMITTEE > > Fernand Reinig local committee president CRP-CU Luxembourg > Raymond Bisdorff, Sophie Laurent, Emmanuel Pichon CRP-CU Luxembourg > Pierre Seck Centre Universitaire Luxembourg > > For more information and correspondence, please contact: > > R. Bisdorff > Centre de Recherche Public - Centre Universitaire > 162a, avenue de la Faiencerie > L-1511 Luxembourg > Tel.: (+352) 47 02 61 1 or 44 01 95 > Fax: (+352) 47 02 64 > Email: bisdorff at crpcu.lu > > > SPONSORSHIP > > The following sponsorships are envisaged: the Cogniscience project (CNRS Paris), > the European Commission (Luxembourg), the FEDIL (Luxembourg) and the Luxembourg > Governement (Departments of Economy and National Education). > > > ------------------------------------------------------------------------------- > > If you want to attend the meeting, or if you wish to submit a paper, please > complete and return the following registration form: > > > Name: > > Organisation: > > Department: > > Address: > > Telephone: > > Fax & Email: > > > > I want to attend the meeting: yes/no > > I wish to submit a paper: yes/no > Its title will be: > From gary at cs.ucsd.edu Tue Jan 4 10:33:56 1994 From: gary at cs.ucsd.edu (Gary Cottrell) Date: Tue, 4 Jan 94 07:33:56 -0800 Subject: Cognitive Science Conference: REVISED DEADLINE! Message-ID: <9401041533.AA00789@desi> Sixteenth Annual Conference of the COGNITIVE SCIENCE SOCIETY August 13-16, 1994 Georgia Institute of Technology Atlanta, Georgia CALL FOR PAPERS Revised due date: Tuesday, February 15, 1994 As Cognitive Science has matured over the years, it has broadened its scope in order to address fundamental issues of cognition embedded within culturally, socially, and technologically rich environments. The Sixteenth Annual Conference of the Cognitive Science Society aims at broad coverage of the many topics, methodologies, and disciplines that comprise Cognitive Science. The conference will highlight new ideas, theories, methods and results in a wide range of research areas relating to cognition. The conference will feature plenary addresses by invited speakers, technical paper and poster sessions, research symposia and panels, and a banquet. The conference will be held at Georgia Tech in Atlanta, home of the Civil Rights movement, the 1996 Olympics, and the Dogwood Festival. GUIDELINES FOR PAPER SUBMISSIONS Novel research papers are invited on any topic related to cognition. Reports of research that cuts across traditional disciplinary boundaries and investigations of cognition within cultural, social and technological contexts are encouraged. To create a high-quality program representing the newest ideas and results in the field, submitted papers will be evaluated through peer review with respect to several criteria, including originality, quality, and significance of research, relevance to a broad audience of cognitive science researchers, and clarity of presentation. Accepted papers will be presented at the conference as talks or posters, as appropriate. Papers may present results from completed research as well as report on current research with an emphasis on novel approaches, methods, ideas, and perspectives. Authors should submit five (5) copies of the paper in hard copy form by Tuesday, February 15, 1994, to: Prof. Ashwin Ram Cognitive Science 1994 Submissions Georgia Institute of Technology College of Computing 801 Atlantic Drive Atlanta, Georgia 30332-0280 If confirmation of receipt is desired, please use certified mail or enclose a self-addressed stamped envelope or postcard. DAVID MARR MEMORIAL PRIZES FOR EXCELLENT STUDENT PAPERS Papers with a student first author are eligible to compete for a David Marr Memorial Prize for excellence in research and presentation. The David Marr Prizes are accompanied by a $300.00 honorarium, and are funded by an anonymous donor. LENGTH Papers must be a maximum of eleven (11) pages long (excluding only the cover page but including figures and references), with 1 inch margins on all sides (i.e., the text should be 6.5 inches by 9 inches, including footnotes but excluding page numbers), double-spaced, and in 12-point type. Each page should be numbered (excluding the cover page). Template and style files conforming to these specifications for several text formatting programs, including LaTeX, Framemaker, Word, and Word Perfect, are available by anonymous FTP from ftp.cc.gatech.edu:/pub/cogsci94/submission-templates. (Camera-ready papers will be required only after authors are notified of acceptance; accepted papers will be allotted six proceedings pages in the usual double-column camera-ready format.) COVER PAGE Each copy of the paper must include a cover page, separate from the body of the paper, which includes: 1. Title of paper. 2. Full names, postal addresses, phone numbers, and e-mail addresses of all authors. 3. An abstract of no more than 200 words. 4. Three to five keywords in decreasing order of relevance. The keywords will be used in the index for the proceedings. 5. Preference for presentation format: Talk or poster, talk only, poster only. Accepted papers will be presented either as talks or posters, depending on authors' preferences and reviewers' recommendations about which would be more suitable, and will not reflect the quality of the papers. 6. A note stating if the paper is eligible to compete for a Marr Prize. DEADLINE Papers must be received by Tuesday, February 15, 1994. Papers received after this date will be recycled. CALL FOR SYMPOSIA In addition to the technical paper and poster sessions, the conference will feature research symposia, panels, and workshops. Proposals for symposia are invited. Proposals should indicate: 1. A brief description of the topic; 2. How the symposium would address a broad cognitive science audience, and some evidence of interest; 3. Names of symposium organizer(s); 4. List of potential speakers, their topics, and some estimate of their likelihood of participation; 5. Proposed symposium format (designed to last 90 minutes). Symposium proposals should be sent as soon as possible, but no later than January 14, 1994. Abstracts of the symposium talks will be published in the proceedings. CONFERENCE CHAIRS Kurt Eiselt and Ashwin Ram STEERING COMMITTEE Dorrit Billman, Mike Byrne, Alex Kirlik, Janet Kolodner (chair), Nancy Nersessian, Mimi Recker, and Tony Simon PLEASE ADDRESS ALL CORRESPONDENCE TO: Prof. Kurt Eiselt Cognitive Science 1994 Conference Georgia Institute of Technology Cognitive Science Program Atlanta, Georgia 30332-0505 E-mail: cogsci94 at cc.gatech.edu From Dave_Touretzky at DST.BOLTZ.CS.CMU.EDU Wed Jan 5 20:33:46 1994 From: Dave_Touretzky at DST.BOLTZ.CS.CMU.EDU (Dave_Touretzky@DST.BOLTZ.CS.CMU.EDU) Date: Wed, 05 Jan 94 20:33:46 EST Subject: notes for current NIPS authors and future submitters Message-ID: <23036.757820026@DST.BOLTZ.CS.CMU.EDU> 1. For current authors (those who presented papers at NIPS-93 and are now preparing the final camera-ready copy), a message from Jack Cowan: The deadline conditions for papers sent to Morgan Kauffman has been modified. For papers mailed in the US or Canada, a postmark of January 7 will suffice. For papers mailed from elsewhere, the receipt date of January 7 still applies. Jack Cowan, NIPS*93 General Chair 2. For future NIPS submitters: Unlike in previous years, NIPS-94 submissions will require full drafts of papers rather than 1000-word extended abstracts. People are encouraged (but not required) to submit their papers using the NIPS proceedings style. (Authors will still have a chance to revise their papers after the conference, as before.) LaTeX style files are available by anonymous FTP from: helper.systems.caltech.edu (131.215.68.12) in /pub/nips b.gp.cs.cmu.edu (128.2.242.8) in /usr/dst/public/nips Full details of the new submission requirements will be given in the NIPS*94 call for papers, which will appear later this month. -- Dave Touretzky, NIPS*94 Program Chair From N.Sharkey at dcs.shef.ac.uk Thu Jan 6 07:51:53 1994 From: N.Sharkey at dcs.shef.ac.uk (Noel Sharkey) Date: Thu, 6 Jan 94 12:51:53 GMT Subject: CNLP call Message-ID: <9401061251.AA16828@dcs.shef.ac.uk> **************************************************** * * * International Conference on * * New Methods in Language Processing * * * **************************************************** CALL FOR PAPERS Dates: 14-16th September 1994 (inclusive) Location: Centre for Computational Linguistics, UMIST, Manchester, UK. Purpose: In recent years there has been a steadily increasing interest in alternative theories and methodologies to the mainstream techniques of symbolic computational linguistics. This international conference will provide a forum for researchers in the broad area of new methods in NLP, i.e., symbolic and non-symbolic techniques of analogy-based, statistical, and connectionist processing, to present their most recent research and to discuss its implications. In order to focus the conference, however, it is intended to concentrate on research primarily involving written NLP. It is also hoped that the conference will promote discussion in general terms of what this branch of NLP hopes to achieve and how far this paradigm can take NLP in the future. Topics of Interest: * Example- and Memory-based MT * Corpus-based NLP * Bootstrapping techniques * Analogy-based NLP * Connectionist NLP * Statistical MT/NLP * Theoretical issues of sub-symbolic vs. symbolic NLP * Hybrid approaches Programme Committee: Co-chairs: Harold Somers, Daniel Jones (UMIST) Ken Church (AT&T) Hitoshi Iida (ATR) Sergei Nirenburg (CMU) David Powers (IMPACT) James Pustejovsky (Brandeis University) Satoshi Sato (JAIST) Noel Sharkey (Sheffield University) Royal Skousen (Brigham Young University) Jun-ichi Tsujii (UMIST) Susan Warwick-Armstrong (ISSCO) Yorick Wilks (Sheffield University) Preliminary paper submission deadline: 31st March 1994 Acceptance Notification by: 1st June 1994 Camera-ready copy due: 1st August 1994 Submission Requirements: Authors should submit FOUR *hard* copies of a preliminary version of the paper (NOT an outline or abstract) which should be no longer than 6 (A4) pages long, printed no smaller than 10-point. Papers should include a brief abstract, and a list of key words indicating which of the above topics are addressed. A contact address for the author(s) (preferably e-mail) should also be included. Send papers to: NeMLaP, Centre for Computational Linguistics, UMIST, Sackville Street, Manchester, UK. Enquiries : nemlap at ccl.umist.ac.uk From hutch at phz.com Thu Jan 6 18:26:32 1994 From: hutch at phz.com (Jim Hutchinson) Date: Thu, 6 Jan 94 18:26:32 EST Subject: Thesis available: RBF Approach to Financial Time Series Analysis Message-ID: <9401062326.AA01212@phz.com> My thesis, "A Radial Basis Function Approach to Financial Time Series Analysis" is now available from the MIT AI Lab Publications office as Technical Report 1457, both in hardcopy and in FTP-able compressed postscript form. Abstract follows. You may want to preview it before printing: it is 159 pages, and takes about 2MB of disk uncompressed. Comments and questions to hutch at phz.com are welcome! Jim Hutchinson Email: hutch at phz.com PHZ Partners Voice: +1 (617) 494-6000 One Cambridge Center FAX: +1 (617) 494-5332 Cambridge, MA 02142 USA ---------------------- Abstract ---------------------------------- A RADIAL BASIS FUNCTION APPROACH TO FINANCIAL TIME SERIES ANALYSIS Jim Hutchinson Billions of dollars flow through the world's financial markets every day, and market participants are understandably eager to accurately price financial instruments and understand relationships involving them. Nonlinear multivariate statistical modeling on fast computers offers the potential to capture more of the underlying dynamics of these high dimensional, noisy systems than traditional models while at the same time making fewer restrictive assumptions about them. For this style of exploratory, nonparametric modeling to be useful, however, care must be taken in fundamental estimation and confidence issues, especially concerns deriving from limited sample sizes. This thesis presents a collection of practical techniques to address these issues for a modeling methodology, Radial Basis Function networks. These techniques include efficient methods for parameter estimation and pruning, including a heuristic for setting good initial parameter values, a pointwise prediction error estimator for kernel type RBF networks, and a methodology for controlling the ``data mining'' problem. Novel applications in the finance area are described, including the derivation of customized, adaptive option pricing formulas that can distill information about the associated time varying systems that may not be readily captured by theoretical models. A second application area is stock price prediction, where models are found with lower out-of-sample error and better ``paper trading'' profitability than that of simpler linear and/or univariate models, although their true economic significance for real life trading is questionable. Finally, a case is made for fast computer implementations of these ideas to facilitate the necessary model searching and confidence testing, and related implementation issues are discussed. ------------------- FTP Retrieval Instructions ------------------- % ftp publications.ai.mit.edu Name (publications.ai.mit.edu:hutch): anonymous Password: (your email address) .. ftp> cd ai-publications/1993 ftp> binary ftp> get AITR-1457.ps.Z ftp> quit % uncompress AITR-1457.ps.Z % lpr AITR-1457.ps -------------------- For hardcopies contact ----------------------- Sally Richter MIT AI Laboratory Publications Office email: publications at ai.mit.edu phone: 617-253-6773 fax: 617-253-5060 Ask for TR-1457. From bahrami at cse.unsw.edu.au Fri Jan 7 01:22:19 1994 From: bahrami at cse.unsw.edu.au (Mohammad Bahrami) Date: Fri, 7 Jan 94 17:22:19 +1100 Subject: INTCON Message-ID: <940107062219.24210@cse.unsw.edu.au> To all neuro-control researchers, We are going to establish a special interest group called Intelligent Control (INTCON) which will be dedicated to subjects such as: Neuro-control, fuzzy logic control, reinforcement learning and other related subjects. Our objective is to provide a forum for communication and exchange of ideas among researchers in these fields. This will also provide a way for announcement of seminars, conferences, exhibitions, technical papers, programs, and so forth. If you are interested to join this group please send me an e-mail so I can put your name in the list of receivers of the e-mails sent to INTCON. Please inform others who might be interested. Thank you ------------------------------ Mohammad Bahrami University of New South Wales bahrami at syscon.ee.unsw.edu.au *** DO NOT "REPLY" BY USING "R" OR OTHER MEANS. *** *** SEND THE MAIL TO THE ABOVE ADDRESS. *** From MASULLI at GENOVA.INFN.IT Fri Jan 7 07:42:00 1994 From: MASULLI at GENOVA.INFN.IT (F. Masulli Dept. Physics Univ. Genova-It...) Date: Fri, 7 JAN 94 12:42 GMT Subject: Post-Doc Fellowship Message-ID: <5641@GENOVA.INFN.IT> ============= Post-Doc Fellowship in Soft-Computing =============== The Lab of Neural Networks of the Research Unit of Genoa (Italy) of the INFM (National Consortium for Matter Physics) could host a Post-Doc Fellow with a grant of the Program Human Capital and Mobility of the European Community. The research will be carried out in Application of Neural Networks and Fuzzy Systems (Soft Computing) to one of the following topics: - Image Understanding; - On-Line Handwriting Recognition. Applicant must be a Citizen of a member state of EEC or EFTA (with the exception of Italy) and hold a Doctoral Degree. The grant amount of the fellowship is interesting. Apply before Jan 20th, 1994 to Dr. Francesco Masulli (by fax or Email). Include a research program, a curriculum vitae, and the names and addresses of two referees. _____ Dr. Francesco Masulli Email: masulli at genova.infn.it Assistant Professor Fax: +39 10 314218 UdR INFM Genoa Via Dodecaneso 33 16146 Genova - Italy From nowlan at cajal.synaptics.com Fri Jan 7 12:54:57 1994 From: nowlan at cajal.synaptics.com (Steven J. Nowlan) Date: Fri, 07 Jan 94 09:54:57 -0800 Subject: NIPS preprint available via Neuroprose (ftp only) Message-ID: <9401071754.AA10849@cajal.> ****** PAPER AVAILABLE VIA NEUROPROSE *************************************** ****** AVAILABLE VIA FTP ONLY *********************************************** ****** PLEASE DO NOT FORWARD TO OTHER MAILING LISTS OR BOARDS. THANK YOU. ** FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/nowlan.nips94.ps.Z The following paper has been placed in the Neuroprose archive at Ohio State. The file is nowlan.nips94.ps.Z. Only the electronic version of this paper is available. This paper is 8 pages in length. This is a preprint of the paper to appear in Advance in Neural Information Processing Systems 6. This file contains 5 embedded postscript figures and is 1.8 Mbytes uncompressed. It has been successfully printed at remote sites, but it may not print on some printers with limited memory. ----------------------------------------------------- Mixtures of Controllers for Jump Linear and Non-linear Plants Steven J. Nowlan Synaptics, Inc. 2698 Orchard Parkway San Jose, CA 95134 ABSTRACT: We describe an extension to the Mixture of Experts architecture for modelling and controlling dynamical systems which exhibit multiple modes of behavior. This extension is based on a Markov process model, and suggests a recurrent network for gating a set of linear or non-linear controllers. The new architecture is demonstrated to be capable of learning effective control strategies for jump linear and non-linear plants with multiple modes of behavior. ----------------------------------------------------- Steven J. Nowlan Synaptics, Inc. 2698 Orchard Parkway San Jose, CA 95134 e-mail: nowlan at synaptics.com phone: (408) 434-0110 x118 From isabelle at neural.att.com Fri Jan 7 13:41:23 1994 From: isabelle at neural.att.com (Isabelle Guyon) Date: Fri, 7 Jan 94 13:41:23 EST Subject: No subject Message-ID: <9401071841.AA06106@neural> ************************************************************************** SPECIAL ISSUE OF THE INTERNATIONAL JOURNAL OF PATTERN RECOGNITION AND ARTIFICIAL INTELLIGENCE ON NEURAL NETWORKS ************************************************************************** ISSN: 0218-0014 Advances in Pattern Recognition Systems using Neural Networks, Eds. I. Guyon and P.S.P. Wang, IJPRAI, vol. 7, number 4, August 1993. ************************************************************************** Among the many applications that have been proposed for neural networks, pattern recognition has been one of the most successful ones, why? This collection of papers give will satisfy your curiosity! The commonplace rationale behind using Neural Networks is that a machine which architecture imitates that of the brain should inherit its remarquable intelligence. This logic usually contrasts with the reality of the performance of Neural Networks. In this special issue, however, the authors have kept some distance with the biological foundations of Neural Networks. The success of their applications relies, to a large extend, on careful engineering. For instance, many novel aspects of the works presented here are concerned with combining Neural Networks with other ``non neural'' modules. With: [ [1] ] Y. Bengio. A Connectionist Approach to Speech Recognition. [ [2] ] J. Bromley, J. W. Bentz, L. Bottou, I. Guyon, L. Jackel, Y. Le Cun, C. Moore, E. Sackinger, and R. Shah. Signature Verification with a Siamese TDNN. [ [3] ] C. Burges, J. Ben, Y. Le Cun, J. Denker and C. Nohl. Off-line Recognition of Handwritten Postal Words using Neural Networks. [ [4] ] H. Drucker, Robert Schapire and Patrice Simard. Boosting Performance in Neural Networks. [ [5] ] F. Fogelman, B. Lamy and E. Viennet. Multi-Modular Neural Network Architectures for Pattern Recognition: Applications in Optical Character Recognition and Human Face Recognition. [ [6] ] A. Gupta, M. V. Nagendraprasad, A. Liu, P. S. P. Wang and S. Ayyadurai. An Integrated Architecture for Recognition of Totally Unconstrained Handwritten Numerals. [ [7] ] E. K. Kim, J. T. Wu, S. Tamura, R. Close, H. Taketani, H. Kawai, M. Inoue and K. Ono. Comparison of Neural Network and K-NN Classification Methods in Vowel and Patellar Subluxation Image Recognitions. [ [8] ] E. Levin, R. Pieraccini and E. Bocchieri. Time-Warping Network: A Neural Approach to Hidden Markov Model based Speech Recognition. [ [9] ] H. Li and J. Wang. Computing Optical Flow with a Recurrent Neural Network. [ [10] ] W. Li and N. Nasrabadi. Invariant Object recognition Based on Neural Network of Cascaded RCE Nets. [ [11] ] G. Martin, M. Rashid and J. Pittman. Integrated Segmentation and Recognition Through Exhaustive Scans or Learned Saccadic Jumps. [ [12] ] C. B. Miller and C. L. Giles. Experimental Comparison of the Effect of Order in Recurrent Neural Networks. [ [13] ] L. Miller and A. Gorin. Structured Networks, for Adaptive Language Acquisition. [ [14] ] N. Morgan, H. Bourlard, S. Renals M. Cohen and H. Franco. Hybrid Neural Network / Hidden Markov Model Systems for Continuous Speech Recognition. [ [15] ] K. Peleg and U. Ben-Hanan. Adaptive Classification by Neural Net Based Prototype Populations. [ [16] ] L. Wiskott and C. von der Malsburg. A Neural System for the Recognition of Partially Occluded Objects in Cluttered Scenes - A Pilot Study. [ [17] ] G. Zavaliagkos, S. Austin, J. Makhoul and R. Schwartz. A Hybrid Continuous Speech Recognition System Using Segmental Neural Nets with Hidden Markov Models. From hutch at phz.com Fri Jan 7 17:36:34 1994 From: hutch at phz.com (Jim Hutchinson) Date: Fri, 7 Jan 94 17:36:34 EST Subject: Thesis available: RBF Approach to Financial Time Series Analysis In-Reply-To: Jim Hutchinson's message of Thu, 6 Jan 94 18:26:32 EST <9401062326.AA01212@phz.com> Message-ID: <9401072236.AA00442@phz.com> Aparently my thesis announcement yesterday generated quite a few requests to the MIT AI Lab Publications office for hardcopies, so they have asked me to forward the following message to everyone, encouraging the FTP route. My apologies for not getting this right the first time. Jim Hutchinson Email: hutch at phz.com PHZ Partners Voice: (617) 494-6000 One Cambridge Center FAX: (617) 494-5332 Cambridge, MA 02142 --------------- You can ftp this paper which is the fastest way to get it. FTP instructions are included below. You may prefer to order the hardcopy. In that case, it is necessary to send prepayment for report plus shipping. FOREIGN ------- AITR-1457 $9.00 shipping $5.50 SURFACE +$10.00 for AIRMAIL ------ Total $14.50 SURFACE $24.50 AIRMAIL Please specify SURFACE or AIRMAIL. Checks should be in US dollars and drawn on a US bank and made payable to MIT AI Lab. DOMESTIC -------- AITR-1457 $9.00 shipping $2.50 ------ Total $11.50 Please send your request and payment to the following address: MIT AI Lab Publications, NE43-818 545 Technology Square Cambridge, MA 02139 USA FTP INSTRUCTIONS ---------------- To ftp and download Jim Hutchinson's thesis `A Radial Basis Function Approach to Financial Time Series Analysis' (AITR-1457) type the following text within quotes at your promt (minus the quotes): 1. `ftp publications.ai.mit.edu' 2. login `anonymous' 3. password `your login' 4. `cd ai-publications/1993' 5. `get AITR-1457.ps.Z' You should see a message stating port command successful. If you do not type `bin' or `binary' before you try to `get file' again. If you need more details on any of the files or instructions listed above, please refer to the README file upon entry to our public ftp site or contact me. We hope this information is helpful to you in your research. Sally Richter MIT AI Laboratory Publications Office email: publications at ai.mit.edu phone: 617-253-6773 fax: 617-253-5060 From nowlan at cajal.synaptics.com Fri Jan 7 22:15:49 1994 From: nowlan at cajal.synaptics.com (Steven J. Nowlan) Date: Fri, 07 Jan 94 19:15:49 -0800 Subject: CORRECTION: Re: NIPS preprint available via Neuroprose (ftp only) Message-ID: <9401080315.AA11536@cajal.> I apologize for the extra bandwidth, but the first announcement of this paper omitted the name of the first author. My apologies to Tim, who did all the hard work. -S. ****** PAPER AVAILABLE VIA NEUROPROSE *************************************** ****** AVAILABLE VIA FTP ONLY *********************************************** ****** PLEASE DO NOT FORWARD TO OTHER MAILING LISTS OR BOARDS. THANK YOU. ** FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/nowlan.nips94.ps.Z The following paper has been placed in the Neuroprose archive at Ohio State. The file is nowlan.nips94.ps.Z. Only the electronic version of this paper is available. This paper is 8 pages in length. This is a preprint of the paper to appear in Advance in Neural Information Processing Systems 6. This file contains 5 embedded postscript figures and is 1.8 Mbytes uncompressed. It may not print on some printers with limited memory. ----------------------------------------------------- Mixtures of Controllers for Jump Linear and Non-linear Plants Timothy W. Cacciatore Steven J. Nowlan Department of Neurosciences Synaptics, Inc. University of California at San Diego 2698 Orchard Parkway La Jolla, CA 92093 San Jose, CA 95134 ABSTRACT: We describe an extension to the Mixture of Experts architecture for modelling and controlling dynamical systems which exhibit multiple modes of behavior. This extension is based on a Markov process model, and suggests a recurrent network for gating a set of linear or non-linear controllers. The new architecture is demonstrated to be capable of learning effective control strategies for jump linear and non-linear plants with multiple modes of behavior. ----------------------------------------------------- Steven J. Nowlan Synaptics, Inc. 2698 Orchard Parkway San Jose, CA 95134 e-mail: nowlan at synaptics.com phone: (408) 434-0110 x118 From bengioy at IRO.UMontreal.CA Sat Jan 8 11:53:18 1994 From: bengioy at IRO.UMontreal.CA (Yoshua Bengio) Date: Sat, 8 Jan 1994 11:53:18 -0500 Subject: Colloquium Message-ID: <9401081653.AA24867@saguenay.IRO.UMontreal.CA> COLLOQUIUM to be held at the 62nd congress of ACFAS (Association Canadienne Francaise pour l'Avancement des Sciences) (French Canadian Association for the Advancement of Science) LEARNING AND ARTIFICIAL NEURAL NETWORKS The next ACFAS congress will be held in Montreal from May 16th to May 20th 1994. Colloquia in many areas of science are organized to allow researchers in a specific area to meet, exchange views and make their work better known. Talks are given in FRENCH (no posters). Abstracts of accepted contributions that were submitted before February 1st will be published in the proceedings of the congress. Submission procedure: ******************** A short French abstract (150-300 words) which could fit on a 10cm x 15cm rectangle should be submitted by February 1st. Electronic submissions (plain ascii or latex) are much preferred: cloutier at iro.umontreal.ca or bengioy at iro.umontreal.ca J. Cloutier or Y. Bengio Dept. I.R.O., Universite de Montreal, C.P. 6128 Succ. A, Montreal, Qc, Canada, H3C3J7 Topics of interest: ****************** LEARNING AND ARTIFICIAL NEURAL NETWORKS - Learning algorithms - Generalization - Applications of theoretical results to practical problems - How accelerate learning algorithms - Links between learning algorithms and generalization - Use of a-priori knowledge in the design of learning systems - Combinations of artificial neural networks with other techniques -- Yoshua Bengio E-mail: bengioy at iro.umontreal.ca Fax: (514) 343-5834 Tel: (514) 343-6804. Residence: (514) 738-6206 Y-219, Universite de Montreal, Dept. IRO, CP 6128, Succ. A, 2900 Edouard-Montpetit, Montreal, Quebec, Canada, H3T 1J4 From becker at cs.toronto.edu Sun Jan 9 13:58:54 1994 From: becker at cs.toronto.edu (Sue Becker) Date: Sun, 9 Jan 1994 13:58:54 -0500 Subject: 3 job advertisements, McMaster University Message-ID: <94Jan9.135902edt.191@neuron.ai.toronto.edu> The Department of Psychology at McMaster University is advertising tenure-track faculty positions in the following three areas: cognitive psychology, sensation/perception and behavioural neuroscience. The 3 ads are below. Applicants whose research combines one of these areas with connectionist modelling are strongly encouraged. Sue Becker Department of Psychology McMaster University Hamilton, Ontario Canada L8S 4K1 email: becker at hypatia.psychology.mcmaster.ca _________________________________________________________________________ McMaster University's Psychology Department invites applications for a tenure-track position at the assistant or associate level commencing no earlier than July 1, 1994. This position is subject to final budgetary approval. We are seeking someone with an established record of independent research who will provide links between our group in cognitive psychology and other researchers in the department, such as an expert in computational models of dyslexia, the neuropsychology of attention, etc. McMaster University is committed to Employment Equity and encourages applications from all qualified candidates, including persons with disabilities, members of visible minorities and women. In accordance with Canadian immigration regulations, this advertisement is directed to Canadian citizens and landed immigrants in the first instance. To apply, send a CV, short statement of research interests, a publication list with selected reprints, and arrange to have three letters of reference sent to Dr. L. R. Brooks, Chair of the Search Committee, Department of Psychology, McMaster University, Hamilton, Ontario, Canada L8S 4K1. ________________________________________________________________________ McMaster University's Psychology Department seeks a faculty member for a tenure-track position at the senior assistant or associate level commencing no earlier than July 1, 1994, with interest in some aspect of sensation or perception. This position is subject to final budgetary approval. The candidate should have strong quantitative skills and an established record of independent research, which may involve anatomy, physiology, behaviour, or modelling. McMaster University is committed to Employment Equity and encourages applications from all qualified candidates, including persons with disabilities, members of visible minorities and women. In accordance with Canadian immigration regulations, this advertisement is directed to Canadian citizens and landed immigrants in the first instance. To apply, send a CV, short statement of research interests, a publication list with selected reprints, and arrange to have three letters of reference sent to Dr. J. R. Platt, Chair of the Search Committee, Department of Psychology, McMaster University, Hamilton, Ontario, Canada L8S 4K1. ________________________________________________________________________ The Department of Psychology at McMaster University invites applications for a tenure-track position at the assistant professor level in the area of behavioural neuroscience commencing no earlier than July 1, 1994. This position is subject to final budgetary approval. The long-term goal of the applicant's research must be an understanding of a behavioural problem preferably in, but not restricted to, the areas of learning/memory, motivation, or perception. Preference will be given to applicants who can exploit new and innovative biological methods (e.g., molecular or imaging techniques) or other methods not currently represented in the department. McMaster University is committed to Employment Equity and encourages applications from all qualified candidates, including persons with disabilities, members of visible minorities and women. In accordance with Canadian immigreation regulations, this advertisment is directed to Canadian citizens and landed immigrants in the fisrt instance. To apply, send a CV, short statement of research interests, a publication list with selected reprints, and arrange to have three letters of reference to Dr. R. Racine, Chair of the Search Committee, Department of Psychology, McMaster University, Hamilton, Ontario, Canada L8S 4K1. __________________________________________________________________________ From hzs at cns.brown.edu Mon Jan 10 15:25:07 1994 From: hzs at cns.brown.edu (Harel Z. Shouval) Date: Mon, 10 Jan 1994 15:25:07 -0500 (EST) Subject: paper available in neuroprose Message-ID: <9401102025.AA09095@cns.brown.edu> A non-text attachment was scrubbed... Name: not available Type: text Size: 2164 bytes Desc: not available Url : https://mailman.srv.cs.cmu.edu/mailman/private/connectionists/attachments/00000000/2059d051/attachment.ksh From zemel at salk.edu Mon Jan 10 19:52:40 1994 From: zemel at salk.edu (Richard S. Zemel) Date: Mon, 10 Jan 94 16:52:40 PST Subject: Thesis on neuroprose Message-ID: <9401110052.AA18791@broca> **DO NOT FORWARD TO OTHER GROUPS** A postscript copy of my PhD thesis has been placed in the neuroprose archive. It prints on 138 pages. The abstract is given below, followed by retrieval instructions. Rich Zemel e-mail: zemel at salk.edu ----------------------------------------------------------------------------- A Minimum Description Length Framework for Unsupervised Learning ABSTRACT A fundamental problem in learning and reasoning about a set of information is finding the right representation. The primary goal of an unsupervised learning procedure is to optimize the quality of a system's internal representation. In this thesis, we present a general framework for describing unsupervised learning procedures based on the Minimum Description Length (MDL) principle. The MDL principle states that the best model is one that minimizes the summed description length of the model and the data with respect to the model. Applying this approach to the unsupervised learning problem makes explicit a key trade off between the accuracy of a representation (i.e., how concise a description of the input may be generated from it) and its succinctness (i.e., how compactly the representation itself can be described). Viewing existing unsupervised learning procedures in terms of the framework exposes their implicit assumptions about the type of structure assumed to underlie the data. While these existing algorithms typically minimize the data description using a fixed-length representation, we use the framework to derive a class of objective functions for training self-supervised neural networks, where the goal is to minimize the description length of the representation simultaneously with that of the data. Formulating a description of the representation forces assumptions about the structure of the data to be made explicit, which in turn leads to a particular network configuration as well as an objective function that can be used to optimize the network parameters. We describe three new learning algorithms derived in this manner from the MDL framework. Each algorithm embodies a different scheme for describing the internal representation, and is therefore suited to a range of datasets based on the structure underlying the data. Simulations demonstrate the applicability of these algorithms on some simple computational vision tasks. ----------------------------------------------------------------------------- I divided the thesis for retrieval purposes into 3 chunks. Also, it is in book style, so it will look better if you print it out on a double-sided printer if you have access to one. To retrieve from neuroprose: unix> ftp cheops.cis.ohio-state.edu Name (cheops.cis.ohio-state.edu:zemel): anonymous Password: (use your email address) ftp> cd pub/neuroprose ftp> get zemel.thesis1.ps.Z ftp> get zemel.thesis2.ps.Z ftp> get zemel.thesis3.ps.Z ftp> quit unix> uncompress zemel* unix> lpr zemel.thesis1.ps unix> lpr zemel.thesis2.ps unix> lpr zemel.thesis3.ps From hamps at richibucto.jpl.nasa.gov Mon Jan 10 20:37:05 1994 From: hamps at richibucto.jpl.nasa.gov (John B. Hampshire II) Date: Mon, 10 Jan 94 17:37:05 -0800 Subject: Efficient Learning Message-ID: <9401110137.AA09840@richibucto.jpl.nasa.gov> A Differential Theory of Learning for Efficient Statistical Pattern Recognition J. B. Hampshire II Jet Propulsion Laboratory, M/S 238-420 California Institute of Technology 4800 Oak Grove Drive Pasadena, CA 91109-8099 hamps at bvd.jpl.nasa.gov ABSTRACT -------- There is more to learning stochastic concepts for robust statistical pattern recognition than the learning itself: computational resources must be allocated and information must be obtained. Therein lies the key to a learning strategy that is efficient, requiring the fewest resources and the least information necessary to produce classifiers that generalize well. Probabilistic learning strategies currently used with connectionist (as well as most traditional) classifiers are often inefficient, requiring high classifier complexity and large training sample sizes to ensure good generalization. An asymptotically efficient **differential learning strategy** is set forth. It guarantees the best generalization allowed by the choice of classifier paradigm as long as the training sample size is large; this guarantee also holds for small training sample sizes when the classifier is an ``improper parametric model'' of the data (as it often is). Differential learning requires the classifier with the minimum functional complexity necessary --- under a broad range of accepted complexity measures --- for Bayesian (i.e., minimum probability-of-error) discrimination. The theory is demonstrated in several real-world machine learning/pattern recognition tasks associated with Fisher's Iris data, optical character recognition, medical diagnosis, and airborne remote sensing imagery interpretation. These applications focus on the implementation of differential learning and illustrate its advantages and limitations in a series of experiments that complement the theory. The experiments demonstrate that differentially-generated classifiers consistently generalize better than their probabilistically-generated counterparts across a wide range of real-world learning-and-classification tasks. The discrimination improvements range from moderate to significant, depending on the statistical nature of the learning task and its relationship to the functional basis of the classifier used. ============================================================ RETRIEVING DOCUMENTS: To obtain a list of the materials/documents that can be retrieved electronically, use anonymous ftp as follows (the IP address of speech1 is 128.2.254.145): > ftp speech1.cs.cmu.edu > user: anonymous > passwd: > cd /usr0/hamps/public > get README Read the file README and choose what you want to retrieve. All files are in /usr0/hamps/public and /usr0/hamps/public/thesis. I welcome your comments and constructive criticism. Happy reading. -JBH2 From tap at cs.toronto.edu Tue Jan 11 08:15:42 1994 From: tap at cs.toronto.edu (Tony Plate) Date: Tue, 11 Jan 1994 08:15:42 -0500 Subject: NIPS preprint available Message-ID: <94Jan11.081543edt.197@neuron.ai.toronto.edu> Preprint Available: To appear in J. D. Cowan, G. Tesauro, and J. Alspector, editors, {\it Advances in Neural Information Processing Systems - 6 - (NIPS*93)}, Morgan Kaufmann, San Mateo, CA Estimating analogical similarity by dot-products of Holographic Reduced Representations. Tony A. Plate Department of Computer Science University of Toronto Toronto, M5S 1A4 Canada tap at ai.utoronto.ca ABSTRACT Models of analog retrieval require a computationally cheap method of estimating similarity between a probe and the candidates in a large pool of memory items. The vector dot-product operation would be ideal for this purpose if it were possible to encode complex structures as vector representations in such a way that the superficial similarity of vector representations reflected underlying structural similarity. This paper describes how such an encoding is provided by Holographic Reduced Representations (HRRs), which are a method for encoding nested relational structures as fixed-width distributed representations. The conditions under which structural similarity is reflected in the dot-product rankings of HRRs are discussed. [This paper is possibly relevant to the recent discussion of the binding problem on this list. In HRRs, I use convolution (which can be thought of as a compressed conjunctive code) to bind roles and fillers, and build up distributed representations of hierarchical predicate structures. This representation preserves the natural similarity structure of predicates and objects.] - Obtain by ftp from archive.cis.ohio-state.edu in pub/neuroprose. - No hardcopy available. - Software to perform the simulations available. - FTP procedure: unix> ftp archive.cis.ohio-state.edu (or 128.146.8.52) Name: anonymous Password: neuron ftp> cd pub/neuroprose ftp> binary ftp> get plate.nips93.ps.Z ftp> quit unix> uncompress plate.nips93.ps.Z unix> lpr plate.nips93.ps (or however you print postscript) From kolen-j at cis.ohio-state.edu Tue Jan 11 18:02:04 1994 From: kolen-j at cis.ohio-state.edu (john kolen) Date: Tue, 11 Jan 1994 18:02:04 -0500 Subject: Reprint Announcement Message-ID: <199401112302.SAA27441@pons.cis.ohio-state.edu> This is an announcement of a newly available paper in neuroprose: Fool's Gold: Extracting Finite State Machines From Recurrent Network Dynamics John F. Kolen Laboratory for Artificial Intelligence Research Department of Computer and Information Science The Ohio State University Columbus, OH 43210 kolen-j at cis.ohio-state.edu Abstract Several recurrent networks have been proposed as representations for the task of formal language learning. After training a recurrent network rec- ognize a formal language or predict the next symbol of a sequence, the next logical step is to understand the information processing carried out by the network. Some researchers have begun to extracting finite state machines from the internal state trajectories of their recurrent networks. This paper describes how sensitivity to initial conditions and discrete measurements can trick these extraction methods to return illusory finite state descriptions. This paper will appear in NIPS 6. ************************ How to obtain a copy ************************ Via Anonymous FTP: unix> ftp archive.cis.ohio-state.edu Name: anonymous Password: (type your email address) ftp> cd pub/neuroprose ftp> binary ftp> get kolen.foolsgold.ps.Z ftp> quit unix> uncompress kolen.foolsgold.ps.Z unix> lpr kolen.foolsgold.ps (or what you normally do to print PostScript) From reza at ai.mit.edu Wed Jan 12 09:38:12 1994 From: reza at ai.mit.edu (Reza Shadmehr) Date: Wed, 12 Jan 94 09:38:12 EST Subject: Rerport available: Human Adaptive Control Message-ID: <9401121438.AA12470@corpus-callosum> The following report is available from neuropose: fTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/shadmehr.elements.ps.Z number of pages: 8 contact: reza at ai.mit.edu ---------------------------------------------------- Computational Elements of the Adaptive Controller of the Human Arm Reza Shadmehr and Ferdinando Mussa-Ivaldi Dept. of Brain and Cognitive Sciences M. I. T. We consider the problem of how the CNS learns to control dynamics of a mechanical system. By using a paradigm where a subject's hand interacts with a virtual mechanical environment, we show that learning control is via composition of a model of the imposed dynamics. Some properties of the computational elements with which the CNS composes this model are inferred through the generalization capabilities of the subject outside the training data. with best wishes, Reza Shadmehr reza at ai.mit.edu From singer at cs.huji.ac.il Wed Jan 12 11:32:35 1994 From: singer at cs.huji.ac.il (Yoram Singer) Date: Wed, 12 Jan 1994 18:32:35 +0200 Subject: Preprint announcement Message-ID: <199401121632.AA03417@minuet.cs.huji.ac.il> *************** PAPERS AVAILABLE **************** *** DO NOT FORWARD TO ANY OTHER LISTS *** ************************************************* The following papers have been placed in cs.huji.ac.il (132.65.16.10). The files are vmm.ps.Z and cursive.ps.Z . Ftp instructions follow the abstracts. These are preprints of the papers to appear in the NIPS 6 proceedings. ----------------------------------------------------- Decoding Cursive Scripts Yoram Singer and Naftali Tishby Institute of Computer Science and Center for Neural Computation Hebrew University, Jerusalem 91904, Israel ABSTRACT: Online cursive handwriting recognition is currently one of the most intriguing challenges in pattern recognition. This study presents a novel approach to this problem which is composed of two complementary phases. The first is dynamic encoding of the writing trajectory into a compact sequence of discrete motor control symbols. In this compact representation we largely remove the redundancy of the script, while preserving most of its intelligible components. In the second phase these control sequences are used to train adaptive probabilistic acyclic automata (PAA) for the important ingredients of the writing trajectories e.g. letters. We present a new and efficient learning algorithm for such stochastic a automata, and demonstrate its utility for spotting and segmentation of cursive scripts. Our experiments show that over 90% of the letters are correctly spotted and identified, prior to any higher level language model. Moreover, both the training and recognition algorithms are very efficient compared to other modeling methods and the models are `on-line' adaptable to other writers and styles. ----------------------------------------------------- The Power of Amnesia Dana Ron Yoram Singer Naftali Tishby Institute of Computer Science and Center for Neural Computation Hebrew University, Jerusalem 91904, Israel ABSTRACT: We propose a learning algorithm for a variable memory length Markov process. Human communication, whether given as text, handwriting, or speech, has multi characteristic time scales. On short scales it is characterized mostly by the dynamics that generate the process, whereas on large scales, more syntactic and semantic information is carried. For that reason the conventionally used fixed memory Markov models cannot capture effectively the complexity of such structures. On the other hand using long memory models uniformly is not practical even for as short memory as four. The algorithm we propose is based on minimizing the statistical prediction error by extending the memory, or state length, adaptively, until the total prediction error is sufficiently small. We demonstrate the algorithm by learning the structure of natural English text and applying the learned model to the correction of corrupted text. Using less than 3000 states the model's performance is far superior to that of fixed memory models with similar number of states. We also show how the algorithm can be applied to intergenic E.coli DNA base prediction with results comparable to HMM-based methods. ----------------------------------------------------- FTP INSTRUCTIONS unix> ftp cs.huji.ac.il (or 132.65.16.10) Name: anonymous Password: your_full_email_address ftp> cd singer ftp> binary ftp> get vmm.ps.Z ftp> get cursive.ps.Z ftp> quit unix> uncompress vmm.ps.Z cursive.ps.Z unix> lpr -P vmm.ps cursive.ps From soller at asylum.cs.utah.edu Wed Jan 12 17:03:29 1994 From: soller at asylum.cs.utah.edu (Jerome Soller) Date: Wed, 12 Jan 94 15:03:29 -0700 Subject: Neural Plasticity and Control Modelling Faculty Position in Bioengineering Message-ID: <9401122203.AA12335@asylum.cs.utah.edu> I was recently made aware of the following job announcement from the University of Utah Department of Bioengineering that was advertised nationally. They have two tenure track faculty openings, one of which may focus on models of neural plasticity and control. I repeat their annoucement verbatim below, and I apologize for the short notice (responses must be made by Jan. 15th). For further information, contact, Dr. Richard Normann of Bioengineering. Sincerely, Jerome Soller (soller at asylum.cs.utah.edu) U. of Utah Dept. of Computer Science and VA Geriatric, Research, Education and Clinical Center ------------------------------------------------------------- Tenure Track Faculty Positions Department of Bioengineering University of Utah Applications are invited for two tenure-track positions in the area of "Biobased Engineering." Candidates must have an earned doctorate and a strong physical science or engineering background, with a specific biological direction to their research. We are specifically seeking candidates with demonstrated expertise in one of the following three areas. 1. Micro/Nano Fabrication of Inorganic, Organic, and Biomolecular Materials: use of inorganic materials, compliant biomaterials, or composite inorganic/polymeric materials to fabricate microsensors and microactuators, and the application of microsystems to problems in the life sciences. 2. Cellular Bioengineering: cytoskeletal biomechanics, mechanisms of cell attachment, locomotion, and bioenergetics. 3. Neural Plasticity and Control: information processing and plasticity in higher neural centers, neural network architectures, and adaptive and hierarchical control systems. Appointees will be expected to develop significant research programs, to assist in the development of new teaching laboratories, and to reach graduate classes in their area of specialization. A complete CV, names of three references, and brief career goals/objectives statement should be sent to Dr. R. Normann, Chair, Department of Bioengineering, 2480 MEB, University of Utah, Salt Lake City, UT 84112 (phone 801-581-8528, FAX 801-585-5361) by January 15, 1994, or until qualified applicants applications are identified. The University is an AA/EO employer, encourages applications from women and minorities, and provides reasonable accomodation to the known disabilities of applicants and employees. From gherrity at io.nosc.mil Thu Jan 13 21:02:31 1994 From: gherrity at io.nosc.mil (Mike Gherrity) Date: Thu, 13 Jan 94 18:02:31 PST Subject: Thesis available on neuroprose Message-ID: <199401140202.SAA02329@io.nosc.mil> FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/gherrity.thesis.ps.Z A postscript copy of my PhD thesis has been placed in the neuroprose archive. It prints on 110 pages. The abstract is given below, followed by retrieval instructions. Mike Gherrity e-mail: gherrity at nosc.mil ----------------------------------------------------------------------------- A Game-Learning Machine ABSTRACT This disertation describes a program which learns good strategies for two-person, deterministic, zero-sum board games of perfect information. The program learns by simply playing the game against either a human or computer opponent. The results of the program's learning the games of tic-tac-toe, connect-four, and chess are reported. The program consists of a game-independent kernel and a game-specific move generator module. Only the move generator is modified to reflect the rules of the game to be played. The kernel remains unchanged for different games. The kernal uses a temporal difference procedure combined with a backpropagation neural network to learn good evaluation functions for the game being played. Central to the performance of the program is the consistency search procedure. This is a game-independent generalization of the capture tree search used in most successful chess playing programs. It is based on the idea of using search to correct errors in evaluations of positions. This procedure is described, analyzed, tested, and implemented in the game-learning program. Both the test results and the performance of the program confirm the results of the analysis which indicate that consistency search improves game playing performance for sufficiently accurate evaluation functions. ----------------------------------------------------------------------------- To retrieve from neuroprose: unix> ftp cheops.cis.ohio-state.edu Name (cheops.cis.ohio-state.edu:zemel): anonymous Password: (use your email address) ftp> cd pub/neuroprose ftp> get gherrity.thesis.ps.Z ftp> quit unix> uncompress gherrity.thesis.ps.Z unix> lpr gherrity.thesis.ps From D.Gorse at cs.ucl.ac.uk Fri Jan 14 11:08:47 1994 From: D.Gorse at cs.ucl.ac.uk (D.Gorse@cs.ucl.ac.uk) Date: Fri, 14 Jan 94 16:08:47 +0000 Subject: Preprint available - reinforcement learning for continuous functions Message-ID: FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/gorse.reinforce.ps.Z The file gorse.reinforce.ps.Z is now available for copying from the Neuroprose archive. This is a 6 page paper, submitted to WCNN '94 San Diego. A longer and more detailed paper describing this work is in preparation and will be available soon. --------------------------------------------------------------------------- A PULSE-BASED REINFORCEMENT ALGORITHM FOR LEARNING CONTINUOUS FUNCTIONS D Gorse Department of Computer Science University College, Gower Street, London WC1E 6BT, UK J G Taylor Department of Mathematics T G Clarkson Department of Electrical and Electronic Engineering King's College, Strand, London WC2R 2LS, UK ABSTRACT: An algorithm is presented which allows continuous functions to be learned by a neural network using spike-based reinforcement learning. Both the mean and the variance of the weights are changed during training; the latter is accomplished by manipulating the lengths of the spike trains used to represent real-valued quantities. The method is here applied to the probabilistic RAM (pRAM) model, but it may be adapted for use with any pulse-based stochastic model in which individual weights behave as random variables. Denise Gorse (D.Gorse at cs.ucl.ac.uk) ---------------------------------------------------------------------------- To obtain a copy: ftp archive.cis.ohio-state.edu login: anonymous password: cd pub/neuroprose binary get gorse.reinforce.ps.Z quit Then at your system: uncompress gorse.reinforce.ps.Z lpr -P gorse.reinforce.ps From lfausett at zach.fit.edu Fri Jan 14 11:28:59 1994 From: lfausett at zach.fit.edu ( Laurene V. Fausett) Date: Fri, 14 Jan 94 11:28:59 -0500 Subject: book announcement Message-ID: <9401141628.AA15672@zach.fit.edu> BOOK ANNOUNCEMENT Title: Fundamentals of Neural Networks: Architectures, Algorithms, and Applications Author: Laurene V. Fausett Publisher: Prentice Hall Ordering Information: Price $49.00 ISBN 0-13-334186-0 To order, call Prentice-Hall Customer Service at 1-800-922-0579 or your local Prentice-Hall representative. This book has also been published in paperback as a Prentice Hall International Edition with ISBN 0-13-042250-9 for distribution outside of the U.S.A., Canada, and Mexico. Brief Description: Written with the beginner in mind, this volume offers an exceptionally clear and thorough introduction to neural networks at an elementary level. Systematic discussion of all major neural nets features presentation of the architectures, detailed algorithms, and examples of simple applications - in many cases variations on a theme. Each chapter concludes with suggestions for further study, including numerous exercises and computer projects. An instructor's manual with solutions and sample software (in Fortran and C) will be available later this spring. Table of Contents Chapter 1 INTRODUCTION; 1.1 Why neural networks, and why now?; 1.2 What is a neural net?; 1.3 Where are neural nets being used?; 1.4 How are neural networks used?; 1.5 Who is developing neural networks?; 1.6 When neural nets began - the McCulloch-Pitts neuron. Chapter 2 SIMPLE NEURAL NETS FOR PATTERN CLASSIFICATION; 2.1 General discussion; 2.2 Hebb net; 2.3 Perceptron; 2.4 Adaline. Chapter 3 PATTERN ASSOCIATION; 3.1 Training algorithms for pattern association; 3.2 Heteroassociative memory neural network; 3.3 Autoassociative net; 3.4 Iterative autoassociative net; 3.5 Bidirectional associative memory (BAM). Chapter 4 NEURAL NETWORKS BASED ON COMPETITION; 4.1 Fixed-weight competitive nets; 4.2 Kohonen self-organizing maps; 4.3 Learning vector quantization; 4.4 Counterpropagation. Chapter 5 ADAPTIVE RESONANCE THEORY; 5.1 Introduction; 5.2 ART1; 5.3 ART2. Chapter 6 BACKPROPAGATION NEURAL NET; 6.1 Standard backpropagation; 6.2 Variations; 6.3 Theoretical results. Chapter 7 A SAMPLER OF OTHER NEURAL NETS; 7.1 Fixed weight nets for constrained optimization; 7.2 A few more nets that learn; 7.3 Adaptive architectures; 7.4 Neocognitron. Glossary; References; Index. From ilya at cheme.seas.upenn.edu Fri Jan 14 02:26:05 1994 From: ilya at cheme.seas.upenn.edu (Ilya Rybak) Date: Fri, 14 Jan 94 02:26:05 -0500 Subject: AHP conductance descriptions Message-ID: <9401140726.AA23250@cheme.seas.upenn.edu> Dear Connectionists, I am developing models of respiratory and baroreflex neural networks on the base of H-H style model of single neuron. I have a problem with description of AHP conductance, that play a very important role in my models. I would be very thankful to everybody for any help. The problem consists in the following. Let's consider the description of AHP conductance in the paper of Yamada et al."Multiple channels and calcium dynamics"(In Methods in oNeuronal Modeling.Eds. Koch and Segev, The MIT press, 1989,97-133.) In the page 132, you can see for AHP conductance that tau=1000/(f(Ca)+b) (1) m=f(Ca)/(f(Ca)+b) (2) In the page 133, you can see that f(Ca)=1.25*10^8*[Ca++]^2 and b=2.5 (3) ----- Ca is measured by mM. (4) In the page 132, you can also see that "the midpoint for m is 44.7 nM and for m^2 is 69.5 nM" (5) It is very simple to chech that (1)-(4) do not correspond to (5). I think, that it is mistake. To correct this mistake we have to change (3). There are two ways for this: 1) f(Ca) is the same as in (3), but b=0.25 (6) ------- 2) f(Ca)=1.25*10^9*[Ca++]^2 and b is the same as in (3) (7) -------- I do not know what is correct (3) or (6) or (7). But the behavior of the behavior of my model depends on this very strong. I have tried to compare the discription of AHP conductance in Huguenard's and McCormick's Manual for VClamp and CClamp. In the page 28 of the Manual the authors refered to the same paper and wrote: alfa=1.2*10^9*[Ca++]^2 and betta=0.001 (8) Taking into account that f(Ca)=alfa*1000 and b=betta*1000 (9) and that Ca is measured in M (10) we can get the following f(Ca)=1.2*10^6*[Ca++]^2 and b=1 (11) ------- This is absolutely different from (3) as well as from (6) as well as from (7). The expresions (3), (6), (7) and (11) are too different. Because of this the behavior of my neuron and network models is absolutaly differnt depending on which one AHP description I use. I cannot go ahead without finding out which one of descriptions is correct. I will be very tankful to everybody for any explaination of this mysterious. Sinceraly, Ilya Rybak Dept. of Neuroscience University of Pennsylvania and Neural Computation Group at DuPont Com. ilya at cheme.seas.upenn.edu From lyle at ai.mit.edu Sat Jan 15 12:05:04 1994 From: lyle at ai.mit.edu (Lyle J. Borg-Graham) Date: Sat, 15 Jan 94 12:05:04 EST Subject: AHP conductance descriptions In-Reply-To: Ilya Rybak's message of Fri, 14 Jan 94 02:26:05 -0500 <9401140726.AA23250@cheme.seas.upenn.edu> Message-ID: <9401151705.AA25287@peduncle> Ilya, I developed a model for I_ahp for hippocampus that is described in: Borg-Graham, L., {\it Modelling the Somatic Electrical Behavior of Hippocampal Pyramidal Neurons}. MIT AI Lab Technical Report 1161, 1989 (290pp). This can be obtained from the MIT AI Lab Publications office (richter at ai.mit.edu). A description of the general extended H-H channel model (including V and Ca dependencies) that I used is found in: Borg-Graham, L., {\it Modelling the Non-Linear Conductances of Excitable Membranes}. Chapter in {\it Cellular Neurobiology: A Practical Approach}, edited by J.\ Chad and H.\ Wheal, IRL Press at Oxford University Press, 1991. I have lifted some of the Latex material that I have on AHP and included it below - I hope it can be of some use! Note that the shell.1 and shell.2 [Ca] that are referred to below are from a proposal in TR1161 that $I_C$ channels are co-localized with Ca channels, and that $I_{AHP}$ channels are evenly distributed across the somatic membrane. The motivation for this inhomogeneous arrangement was to account for the apparent fast and transient Ca-dependence of $I_C$, in contrast with the slower, more intergrative Ca-dependence of $I_{AHP}$. Simulations with a simple 3-compartment model of Ca accumulation (shell.1 being a fraction of the submembrane space in the vicinity of the co-localized CA and C channels, shell.2 being the remainder of the submembrane space which in turn supplies [Ca] for the AHP channels, and a constant [Ca] core compartment) show that this arrangement reproduces the Ca-dependent behavior of the fAHP (from $I_C$) and the slow AHP (from $I_{AHP}$). The parameters listed for the Ca-dep $w$ particle result in a (increasing/decreasing) sigmoidal dependence of the (steady-state/time constant) on the log[Ca], with a half point at 0.004mM and (0.1/0.9) points on the curves at about (0.002/0.009) mM. The maximum tau is 100ms. Also, the specific parameter values referenced below should be used mainly as guidelines; they are currently under revision. **************************************************************** **************************************************************** **************************************************************** **************************************************************** **************************************************************** % for laTeX \documentstyle[11pt]{report} \begin{document} \noindent Summary of $I_{AHP}$, extracted from Chapter 7 of:\\ \noindent Borg-Graham, L., {\it Modelling the Somatic Electrical Behavior of Hippocampal Pyramidal Neurons}. MIT AI Lab Technical Report 1161, 1989 (290pp). . . . . In the case of $I_{C}$ and $I_{AHP}$, little voltage clamp data is available for either their steady state or temporal properties of any presumed activation/inactivation parameters. In addition, describing these currents is complicated by the fact that they are presumably mediated by intracellular $Ca^{2+}$. Little quantitative data is available on this interaction for either current, and there is at present no consensus among workers in this field as to the mechanisms involved. As introduced in the previous chapter and which shall be expanded upon later, I have made the simple assumption (like that used by other workers, e.g. Tra-Lli-79) that $I_{C}$ and $I_{AHP}$ are dependent on a power of the concentration of $Ca^{2+}$ either directly beneath the membrane or in a secondary ``compartment''. This is a highly speculative model, as discussed in the previous chapter. The parameters of this description are based primarily on heuristics, specifically the simulation of the fAHP and the AHP that is observed in HPC. Making the situation more difficult is the fact that there are no protocols to date in which $I_C$ or $I_{AHP}$ are activated without the concomitment presence of other currents, thereby inextricably tying the behavior of any set of estimated parameters for these currents to those of other currents. To a first approximation, the actions of $I_C$ and $I_{AHP}$ are independent of one another. $I_C$ is transient over a time span of a few milliseconds during the spike, and the evidence indicates that this a large current. On the other hand, $I_{AHP}$ activates more slowly, is small, and may last from 0.5 to several seconds. However, since both these currents are dependent on $Ca^{2+}$ entry, their estimation was tied to the description of $I_{Ca}$ and the mechanisms regulating $[Ca^{2+}]_{shell.1}$ and $[Ca^{2+}]_{shell.2}$. Therefore, while the behavior of the $I_C$ or $I_{AHP}$ descriptions could be evaluated independently, whenever the $Ca^{2+}$ mechanisms were modified to alter one of the current's action, the effect of the modification on the other current had to be checked. \section{$Ca^{2+}$-Mediation of $K^+$ Currents by $Ca^{2+}$~-~binding Gating Particle $w$} In order to cause $I_C$ and $I_{AHP}$ to be mediated by intracellular $Ca^{2+}$, I incorporated a $Ca^{2+}$-binding gating particle in the expressions for both of these currents. Several workers have postulated mechanisms for such an interaction between intracellular $Ca^{2+}$ and different ion channels, ranging from complex multi-state kinetic models based on experimental data to very simple descriptions for modelling studies (Tra-Lli-79). In light of the paucity of quantitative data on such mechanisms in HPC, my goals for the description of a putative, generic $Ca^{2+}$-binding gating particle were as follows: \begin{itemize} \item Relationship between $Ca^{2+}$ concentration and particle activation allowing for non-degenerate kinetics considering the range of $Ca^{2+}$ concentrations during various cell responses. \item Binding kinetics based on a simple but reasonable model. \item Kinetic description that could be easily modified to yield significantly different behavior, that is a description that could be modified to suit a wide range of desired behaviors. \end{itemize} To this end the following description for a $Ca^{2+}$ -binding gating particle, $w$, was used. Each $w$ particle can be in one of two states, open or closed, just as the case for the Hodgkin-Huxley-like voltage-dependent activation and inactivation gating particles. Each $w$ particle is assumed to have $n$ $Ca^{2+}$ binding sites, all of which must be bound in order for the particle to be in the open state. Binding is cooperative in a sense that reflects the two states available to a given particle, i.e. either a particle has no $Ca^{2+}$ ions bound to it, and therefore it is in the closed state, or all $n$ binding sites are filled, and the particle is in the open state. The state diagram for this reaction is as follows: $$ w_{closed} + n\, Ca^{2+}_{in} \buildrel \alpha, \beta \over \rightleftharpoons w_{open}^*$$ \noindent where the $*$ notation means that the particle is bound to all $n$ (intracellular) $Ca^{2+}$ ions. $\alpha$ and $\beta$ are the forward and backward rate constants, respectively. This scheme results in the following differential equation for $w$, where now $w$ is the fraction of particles in the open state, assuming that the concentration of $Ca^{2+}$ is large enough that the reaction does not significantly change the store of intracellular $Ca^{2+}$: $$ { {\rm d}w \over {\rm dt}} = (\alpha (1 - w)[Ca^{2+}]_{in})^n - \beta w$$ The steady state value for $w$ ( the fraction of particles in the open state) as a function of the intracellular $Ca^{2+}$ concentration is then: $$ w_{\infty} = {(\alpha [Ca^{2+}]_{in})^n \over (\alpha [Ca^{2+}]_{in})^n + \beta} $$ The time constant for the differential equation is: $$ \tau_w = ((\alpha [Ca^{2+}]_{in})^n + \beta)^{-1} $$ The order of the binding reaction,$n$, that is the number of $Ca^{2+}$ binding sites per $w$ particle, determines the steepness of the previous two expressions, as a function of $ [Ca^{2+}]_{in}$. Given the constraints on the range for $[Ca^{2+}]_{shell.1}$ and $[Ca^{2+}]_{shell.2}$ during single and repetitive firing, $n$ was set to three for both the $I_C$ $w$ particle and the $I_{AHP}$ $w$ particle. On the other hand, as shall be presented shortly, the range of $Ca^{2+}$ concentrations for which the $I_{AHP}$ $w$ particle is activated is set to about one order of magnitude lower than that for the $I_C$ $w$ particle, since $I_C$ was exposed to the larger $[Ca^{2+}]_{shell.1}$ . \section{AHP Potassium Current - $I_{AHP}$} $I_{AHP}$ is a slow, $Ca^{2+}$-mediated $K^+$ current that underlies the long afterhyperpolarization (AHP). Typically the AHP is about 1 to 2 millivolts and lasts from 0.5 -- 3 seconds after a single spike. Adding $Ca^{2+}$ blockers or noradrenaline to the extracellular medium eliminates the AHP, and likewise markedly reduces the cell's accommodation to tonic stimulus. Since most of the data on the proposed $I_{AHP}$ is derived from various current clamp protocols, the model description of this current is based on that used in other models (Koch and Adams, 1986) and from heuristics derived from the properties of other currents, in particular $I_{Ca}$ and $I_{DR}$. The important relationship between the $I_{AHP}$ and $I_{DR}$ parameters arose when I attempted to simulate both the mAHP (mediated by $I_{DR}$) and the AHP according to data from Storm (). In addition, since $I_{AHP}$ is dependent on $Ca^{2+}$ entry, the derivation of this current and the dynamics of $[Ca]_{shell.1}$ and $[Ca]_{shell.2}$ was done simultaneously. In fact, it was determined that in order for the activation of $I_{AHP}$ to be delayed from the onset of the spike, it was necessary to introduce the second intracellular space (shell) that was described in Chapter 6. Such a relationship between $Ca^{2+}$ influx and the subsequent delayed activation of $I_{AHP}$ has been suggested in the literature (Lan-Ada-86). \subsection{Results} I propose that the conductance underlying $I_{AHP}$ is dependent both on $Ca^{2+}$ and voltage. The $Ca^{2+}$ dependence of this current is clearly demonstrated since the AHP is removed when $Ca^{2+}$ blockers are added, and construction of a reasonable model of $Ca^{2+}$ dynamics such that $I_{AHP}$ may be dependent on this is possible. The mechanism that I use for $Ca^{2+}$-mediation of $I_{AHP}$ is similar to that for $I_C$, that is the $I_{AHP}$ channel includes a single $Ca^{2+}$-binding $w$ particle, with the same binding reaction as shown in Equation x. Voltage-clamp studies (Lan-Ada-86) indicate that there is no voltage-dependent activation of $I_{AHP}$, however. This puts a greater constraint on the $Ca^{2+}$-mediated mechanism for this current since the activation necessary to underly the long, small hyperpolarization after a single spike is significantly less than that required to squelch rapid spikes after some delay in response to tonic stimulus. In particular, these requirements provided rather restricted constraints on the buildup of $Ca^{2+}$ during each spike in region of the $I_{AHP}$ channels, $shell.2$, and likewise the dependence of the $I_{AHP}$ $w$ particle on this localized concentration of $Ca^{2+}$ . On the other hand I have included two inactivation gating particles, $y$ and $z$. The rationale for the $y$ particle is based on two pieces of evidence. First, it has been reported that $Ca^{2+}$ spikes are insensitive to noradrenaline in protocols where $I_{DR}$ and $I_A$ have been blocked by TEA and 4-AP, respectively (Segal and Barker). The fact that these spikes are unchanged with the addition of noradrenaline implies that under this protocol $I_{AHP}$ is inactivated by some other mechanism, since presumably $I_{AHP}$ has not been disabled. Since the protocol involves a long (approximately 30 milliseconds) depolarization of the cell before the $Ca^{2+}$ spike, it was possible to include an inactivation particle for $I_{AHP}$ that was (a) fast enough to disable $I_{AHP}$ under these conditions, but (b) was slow enough so that normal spiking did not cause the $y$ particle to change states. A second indication for the voltage-dependent inactivation particle $y$ is consistent with the previous evidence, that is the amplitude and rate of rise of action potentials singly or in trains appears independent of the presence $I_{AHP}$. In particular, the size of the $I_{AHP}$ conductance necessary to repress repetitive firing is large enough to significantly effect the spike once threshold is achieved if this conductance remained during the spike. Such a role for $I_{AHP}$ has not been demonstrated. $y$ therefore causes $I_{AHP}$ to shut off during an action potential so that this current does not reduce the amplitude of the spike. The second inactivation particle, $z$, was included to account for the delayed peak seen in the large afterhyperpolarization that occurs after a long (greater than 100 ms) stimulus (Madison and Nicoll, 1982 and others). At rest, $z$ is partially closed. With a large, lengthy hyperpolarization the $z$ particle becomes more open, thereby slowly increasing $I_{AHP}$ and the magnitude of the sAHP, until the $Ca^{2+}$ in $shell.2$ eventually drains down to its resting level and subsequently shutting off $w$. The time constant for $z$ was set very slow above rest so that it did not change appreciably during firing. Below about -75 mV, however, the time constant approaches 120 milliseconds so that the desired role of $z$ during the sAHP is obtained. No voltage-dependence for $I_{AHP}$ has been noted in the literature. However, the dependence of $I_{AHP}$ on $Ca^{2+}$ influx may have precluded voltage-clamp experiments which might verify the voltage-dependencies indicated by the simulations. With the present formulation for $I_{AHP}$, this current plays an important role during repetitive firing by shutting off the spike train after several hundred milliseconds. This occurs primarily through the dependence of $I_{AHP}$ on $[Ca]_{shell.2}$, which slowly increases during repetitive firing. Eventually the rise of $[Ca]_{shell.2}$ causes $I_{AHP}$ to provide sufficient outward rectification for counter-acting the stimulus current and thus stop the cell from firing (Figure~\ref{f:ahp-spks}). The fact that $I_{AHP}$ is strongly activated by this protocol is indicated by the long hyperpolarization at the end of the stimulus (Madison and Nicoll, 1982, and see simulation of their results in Figure~\ref{f:ahp-spks}). Madison and Nicoll, 1982 Mad-Nic-82 report that noradrenaline blocks accommodation by selectively blocking $I_{AHP}$. The characteristics demonstrated by the model $I_{AHP}$ are in qualitative agreement with many of the characteristics reported in the literature (e.g. Lan-Ada-86 , Seg-Bar-86),, including the increased activation of $I_{AHP}$ with increasing numbers of spikes in a single train, delayed activation from onset of $Ca^{2+}$ influx, the role of $I_{AHP}$ in modulating repetitive firing, time constant for inactivation/deactivation of greater than one second, the apparent voltage insensitivity (the transition of $y$ and $z$ with sub-threshold depolarization is slow, and once $x$ is activated deactivation takes several seconds. The equation for $I_{AHP}$ is - $$I_{AHP}= y_{AHP}^2 \, z_{AHP}\, w_{ahp}\, (V - E_K)$$ \noindent where $$\overline g_{AHP} = 0.4 \, \mu \rm S$$ \begin{table} \centering \begin{tabular}{|c||c|c|c|c|c|c|c|}\hline Gating Variable & $z$ & $\gamma$ & $\alpha_0$ & $V_{{1 \over 2}}\,$(mV) & $\tau_0\,$(ms) & $\alpha_{Ca}^*$ & $\beta_{Ca}^{**}$ \\ \hline\hline $y$ (inactivation)& -15 & 0.2 & 0.01 & -50.0 & 1.0 & - & - \\ \hline $z$ (inactivation)& -12 & 1.0 & 0.0002 & -72.0 & 100.0 & - & - \\ \hline $w$ ($Ca^{2+}$-activation) & - & - & - & - & - & $50$ & 0.01 \\ \hline \end{tabular}\caption[Parameters of $I_{AHP}$ Gating Variables] {Parameters of $I_{AHP}$ Gating Variables. $* = (ms^{-1/3}mM^{-1})$, $** = (ms^{-1})$}\label{t:ahp}\end{table} Table \ref{t:ahp} lists the parameters for the $I_{AHP}$ gating variables. These are the rate functions for the activation variable, $x$, of $I_{AHP}$- $$\alpha_{y,AHP} = 0.015 \exp \biggl({(V + 50) 0.8 \cdot -15 \cdot F \over R T} \biggr)$$ $$\beta_{y,AHP} = 0.015 \exp \biggl({(-50 - V) 0.2 \cdot -15 \cdot F \over R T} \biggr)$$ These are the rate functions for the activation variable, $y$, of $I_{AHP}$- $$\alpha_{z,AHP} = 0.0002 \, (\gamma = 0)$$ $$\beta_{z,AHP} = 0.0002 \exp \biggl({(-72 - V) \cdot -12 \cdot F \over R T} \biggr)$$ Again, each $w$ particle was assumed to have three non-competitive $Ca^{2+}$ binding sites, all of which were either empty (corresponding to the closed state) or filled (corresponding to the open state). \end{document} From ingber at alumni.cco.caltech.edu Sat Jan 15 13:26:51 1994 From: ingber at alumni.cco.caltech.edu (Lester Ingber) Date: Sat, 15 Jan 1994 10:26:51 -0800 Subject: Preprint: Path-integral evolution of short-term memory Message-ID: <199401151826.KAA18504@alumni.cco.caltech.edu> The following is an abstract from a paper accepted for publication in Physical Review E. The preprint may be retrieved via anonymous ftp from ftp.caltech.edu [131.215.48.151] in the pub/ingber directory using instructions given below. The file, smni94_stm.ps.gz, is about 0.9 MBytes. Statistical mechanics of neocortical interactions: Path-integral evolution of short-term memory Lester Ingber Lester Ingber Research, P.O. Box 857, McLean, VA 22101 ingber at alumni.caltech.edu Previous papers in this series of statistical mechanics of neocortical interactions (SMNI) have detailed a development from the relatively microscopic scales of neurons up to the macroscopic scales as recorded by electroencephalography (EEG), requiring an intermediate mesocolumnar scale to be developed at the scale of minicolumns (~10^2 neurons) and macrocolumns (~10^5 neurons). Opportunity was taken to view SMNI as sets of statistical constraints, not necessarily describing specific synaptic or neuronal mechanisms, on neuronal interactions, on some aspects of short-term memory (STM), e.g., its capacity, stability and duration. A recently developed C-language code, PATHINT, provides a non-Monte Carlo technique for calculating the dynamic evolution of arbitrary-dimension (subject to computer resources) nonlinear Lagrangians, such as derived for the two- variable SMNI problem. Here, PATHINT is used to explicitly detail the evolution of the SMNI constraints on STM. Interactively [brackets signify machine prompts]: [your_machine%] ftp ftp.caltech.edu [Name (...):] anonymous [Password:] your_e-mail_address [ftp>] cd pub/ingber [ftp>] binary [ftp>] ls [ftp>] get smni94_stm.ps.gz [ftp>] quit This directory also contains the Adaptive Simulated Annealing (ASA) code, now at version 2.8, in ASA-shar, ASA-shar.Z, ASA.tar.gz, and ASA.zip formats. The 00index file contains an index of the other (p)reprints and information on getting gzip and unshar for DOS, MAC, UNIX, and VMS systems. To get on or off the ASA_list e-mailings, just send an e-mail to asa-request at alumni.caltech.edu with your request. If you do not have ftp access, get information on the FTPmail service by: mail ftpmail at decwrl.dec.com, and send only the word "help" in the body of the message. If any of the above are not possible, and if your mailer can handle large files (please test this first), the code or papers you require can be sent as uuencoded compressed files via electronic mail. If you have gzip, resulting in smaller files, please state this. Sorry, I cannot assume the task of mailing out hardcopies of code or papers. Lester || Prof. Lester Ingber || || Lester Ingber Research || || P.O. Box 857 E-Mail: ingber at alumni.caltech.edu || || McLean, VA 22101 Archive: ftp.caltech.edu:/pub/ingber || From harnad at Princeton.EDU Sat Jan 15 20:22:50 1994 From: harnad at Princeton.EDU (Stevan Harnad) Date: Sat, 15 Jan 94 20:22:50 EST Subject: EEG Models: Chaotic and Linear: PSYCOLOQUY Call for Commentary Message-ID: <9401160122.AA26789@clarity.Princeton.EDU> Note: This is a PSYCOLOQUY Call for Commentators, *not* a BBS Call: You are invited to submit a formal commentary on the target article whose abstract appears below. It has just been published in the refereed electronic journal PSYCOLOQUY. Instructions for retrieving the full article and for preparing a PSYCOLOQUY commentary appear after the abstract. All commentaries are refereed. TARGET ARTICLE AUTHOR'S RATIONALE FOR SOLICITING COMMENTARY The target article attempts to reconcile attractor neural network (ANN) theory with certain current models for the generation of the EEG as a step toward integrating ANN theory with gross observations of brain function. Emphasis is placed on symmetry of cortical connections at a macroscopic level as compared to symmetry at a microscopic level. We hope to elicit commentary on (1) the methodology of the experiments and simulations on which the work is based, (2) any contradictory experimental findings, (3) quantitative methods in anatomy required for further development, (4) other critiques of ANN applicability to global brain function. psycoloquy.93.4.60.EEG-chaos.1.wright Thursday 23 December 1993 ISSN 1055-0143 (53 parags, 12 equations, 3 figs, 62 refs, 1092 lines) PSYCOLOQUY is sponsored by the American Psychological Association (APA) Copyright 1993 JJ Wright, RR Kydd & DTJ Liley EEG MODELS: CHAOTIC AND LINEAR J.J. Wright, R.R. Kydd, D.T.J. Liley Department of Psychiatry and Behavioural Science, School of Medicine, University of Auckland, Auckland, New Zealand jwright at ccu1.auckland.ac.nz jjw at brain.physics.swin.oz.au ABSTRACT: Two complementary EEG models are considered. The first (Freeman 1991) predicts 40+ Hz oscillation and chaotic local dynamics. The second (Wright 1990) predicts propagating EEG waves exhibiting linear superposition, nondispersive transmission, and near-equilibrium dynamics, on the millimetric scale. Anatomical considerations indicate that these models must apply, respectively, to cortical neurons which are very asymmetrically coupled and to symmetric average couplings. Aspects of both are reconciled in a simulation which explains wave velocities, EEG harmonics, the 1/f spectrum of desynchronised EEG, and frequency-wavenumber spectra. Local dynamics can be compared to the attractor model of Amit and Tsodyks (1990) applied in conditions of highly asymmetric coupling. Nonspecific cortical afferents may confer an adiabatic energy landscape to the large-scale dynamics of cortex. KEYWORDS: chaos, EEG simulation, electroencephalogram, linear dynamics, neocortex, network symmetry, neurodynamics, pyramidal cell, wave velocity. ------------------------------------------------------------------- INSTRUCTIONS FOR PSYCOLOQUY COMMENTATORS Accepted PSYCOLOQUY target articles have been judged by 5-8 referees to be appropriate for Open Peer Commentary, the special service provided by PSYCOLOQUY to investigators in psychology, neuroscience, behavioral biology, cognitive sciences and philosophy who wish to solicit multiple responses from an international group of fellow specialists within and across these disciplines to a particularly significant and controversial piece of work. If you feel that you can contribute substantive criticism, interpretation, elaboration or pertinent complementary or supplementary material on a PSYCOLOQUY target article, you are invited to submit a formal electronic commentary. Please note that although commentaries are solicited and most will appear, acceptance cannot, of course, be guaranteed. 1. Before preparing your commentary, please read carefully the Instructions for Authors and Commentators and examine recent numbers of PSYCOLOQUY. 2. Commentaries should be limited to 200 lines (1800 words, references included). PSYCOLOQUY reserves the right to edit commentaries for relevance and style. In the interest of speed, commentators will only be sent the edited draft for review when there have been major editorial changes. Where judged necessary by the Editor, commentaries will be formally refereed. 3. Please provide a title for your commentary. As many commentators will address the same general topic, your title should be a distinctive one that reflects the gist of your specific contribution and is suitable for the kind of keyword indexing used in modern bibliographic retrieval systems. Each commentary should have a brief (~50-60 word) abstract 4. All paragraphs should be numbered consecutively. Line length should not exceed 72 characters. The commentary should begin with the title, your name and full institutional address (including zip code) and email address. References must be prepared in accordance with the examples given in the Instructions. Please read the sections of the Instruction for Authors concerning style, preparation and editing. PSYCOLOQUY is a refereed electronic journal (ISSN 1055-0143) sponsored on an experimental basis by the American Psychological Association and currently estimated to reach a readership of 36,000. PSYCOLOQUY publishes brief reports of new ideas and findings on which the author wishes to solicit rapid peer feedback, international and interdisciplinary ("Scholarly Skywriting"), in all areas of psychology and its related fields (biobehavioral, cognitive, neural, social, etc.) All contributions are refereed by members of PSYCOLOQUY's Editorial Board. Target article length should normally not exceed 500 lines [c. 4500 words]. Commentaries and responses should not exceed 200 lines [c. 1800 words]. All target articles, commentaries and responses must have (1) a short abstract (up to 100 words for target articles, shorter for commentaries and responses), (2) an indexable title, (3) the authors' full name(s) and institutional address(es). In addition, for target articles only: (4) 6-8 indexable keywords, (5) a separate statement of the authors' rationale for soliciting commentary (e.g., why would commentary be useful and of interest to the field? what kind of commentary do you expect to elicit?) and (6) a list of potential commentators (with their email addresses). All paragraphs should be numbered in articles, commentaries and responses (see format of already published articles in the PSYCOLOQUY archive; line length should be < 80 characters, no hyphenation). It is strongly recommended that all figures be designed so as to be screen-readable ascii. If this is not possible, the provisional solution is the less desirable hybrid one of submitting them as postscript files (or in some other universally available format) to be printed out locally by readers to supplement the screen-readable text of the article. PSYCOLOQUY also publishes multiple reviews of books in any of the above fields; these should normally be the same length as commentaries, but longer reviews will be considered as well. Book authors should submit a 500-line self-contained Precis of their book, in the format of a target article; if accepted, this will be published in PSYCOLOQUY together with a formal Call for Reviews (of the book, not the Precis). The author's publisher must agree in advance to furnish review copies to the reviewers selected. Authors of accepted manuscripts assign to PSYCOLOQUY the right to publish and distribute their text electronically and to archive and make it permanently retrievable electronically, but they retain the copyright, and after it has appeared in PSYCOLOQUY authors may republish their text in any way they wish -- electronic or print -- as long as they clearly acknowledge PSYCOLOQUY as its original locus of publication. However, except in very special cases, agreed upon in advance, contributions that have already been published or are being considered for publication elsewhere are not eligible to be considered for publication in PSYCOLOQUY, Please submit all material to psyc at pucc.bitnet or psyc at pucc.princeton.edu Anonymous ftp archive is DIRECTORY pub/harnad/Psycoloquy HOST princeton.edu ------------------------------------------------------------- To retrieve the file by ftp from a Unix/Internet site, type either: ftp princeton.edu or ftp 128.112.128.1 When you are asked for your login, type: anonymous Enter password as per instructions (make sure to include the specified @), and then change directories with: cd /pub/harnad/Psycoloquy/1993.volume.4 To show the available files, type: ls Next, retrieve the file you want with (for example): get psyc.93.4.60.EEG-chaos.1.wright [or you can abbreviate with: mget *wright When you have the file(s) you want, type: quit In case of doubt or difficulty, consult your system manager. A more elaborate version of these instructions for the U.K. is available on request (thanks to Brian Josephson)> These files can also be retrieved using gopher, archie, veronica, etc. ---------- Where the above procedures are not available (e.g. from Bitnet or other networks), there are two fileservers: ftpmail at decwrl.dec.com and bitftp at pucc.bitnet that will do the transfer for you. To one or the other of them, send the following one line message: help for instructions (which will be similar to the above, but will be in the form of a series of lines in an email message that ftpmail or bitftp will then execute for you). From harnad at Princeton.EDU Sat Jan 15 21:15:02 1994 From: harnad at Princeton.EDU (Stevan Harnad) Date: Sat, 15 Jan 94 21:15:02 EST Subject: Beyond Modularity: BBS Call for Book Reviewers Message-ID: <9401160215.AA27062@clarity.Princeton.EDU> Below is the abstract of a book that will be accorded multiple book review in Behavioral and Brain Sciences (BBS), an international, interdisciplinary journal that provides Open Peer Commentary on important and controversial current research in the biobehavioral and cognitive sciences. Reviewers must be current BBS Associates or nominated by a current BBS Associate. To be considered as a reviewer for this book, to suggest other appropriate reviewers, or for information about how to become a BBS Associate, please send email to: harnad at clarity.princeton.edu or harnad at pucc.bitnet or write to: BBS, 20 Nassau Street, #240, Princeton NJ 08542 [tel: 609-921-7771] To help us put together a balanced list of reviewers, please give some indication of the aspects of the topic on which you would bring your areas of expertise to bear if you are selected as a reviewer. Please also indicate whether you already have a copy of the book or will need one if you are selected. The author's article-length precis of the book is available for inspection by anonymous ftp according to the instructions that follow after the abstract. ____________________________________________________________________ BBS Multiple Book Review of: BEYOND MODULARITY: A DEVELOPMENTAL PERSPECTIVE ON COGNITIVE SCIENCE Cambridge, MA: MIT Press 1992 (234 pp.) Annette Karmiloff-Smith Cognitive Development Unit, Medical Research Council, 4 Taviton Street, London WC1H 0BT, U.K. Electronic Mail: annette at cdu.ucl.ac.uk KEYWORDS: cognitive development, connectionism, constructivism, developmental stages, Fodor, modularity, nativism, Piaget, representational redescription, species differences. ABSTRACT: Beyond Modularity attempts a synthesis of Fodor's anti-constructivist nativism and Piaget's anti-nativist constructivism. Contra Fodor, I argue that: (1) the study of cognitive development is essential to cognitive science, (2) the module/central processing dichotomy is too rigid, and (3) the mind does not begin with prespecified modules, but that development involves a gradual process of modularization. Contra Piaget, I argue that: (1) development rarely involves stage-like domain-general change, and (2) domain-specific predispositions give development a small but significant kickstart by focusing the infant's attention on proprietary inputs. Development does not stop at efficient learning. A fundamental aspect of human development ("Representational Redescription") is the hypothesized process by which information that is IN a cognitive system becomes progressively explicit knowledge TO that system. Development thus involves two complementary processes of progressive modularization and rendering explicit. Empirical findings on the child as linguist, physicist, mathematician, psychologist and notator are discussed in support of the theoretical framework. Each chapter concentrates first on the initial state of the infant mind/brain and on subsequent domain-specific learning in infancy and early childhood. They then go on to explore data on older children's problem solving and theory building, with particular focus on evolving cognitive flexibility. Throughout the book there is an emphasis on the status of representations underlying different capacities and on the multiple levels at which knowledge is stored and accessible. Finally, consideration is given to the need for more formal developmental models, and the Representational Redescription framework is compared with connectionist simulations of development. The concluding sections consider what is special about human cognition and offer some speculations about the status of representations underlying the structure of behavior in other species. -------------------------------------------------------------- To help you decide whether you would be an appropriate reviewer for this book, an electronic precis is retrievable by anonymous ftp from princeton.edu according to the instructions below (the filename is bbs.karmsmith). Please let us know, after having inspected it, what relevant expertise you feel you would bring to bear on what aspect of the article. Note that only the book, not the Precis, is the object of the reviews. ------------------------------------------------------------- To retrieve a file by ftp from a Unix/Internet site, type either: ftp princeton.edu or ftp 128.112.128.1 When you are asked for your login, type: anonymous Enter password as per instructions (make sure to include the specified @), and then change directories with: cd /pub/harnad/BBS To show the available files, type: ls Next, retrieve the file you want with (for example): get bbs.karmsmith When you have the file(s) you want, type: quit In case of doubt or difficulty, consult your system manager. A more elaborate version of these instructions for the U.K. is available on request (thanks to Brian Josephson)> These files can also be retrieved using gopher, archie, veronica, etc. ---------- Where the above procedures are not available (e.g. from Bitnet or other networks), there are two fileservers: ftpmail at decwrl.dec.com and bitftp at pucc.bitnet that will do the transfer for you. To one or the other of them, send the following one line message: help for instructions (which will be similar to the above, but will be in the form of a series of lines in an email message that ftpmail or bitftp will then execute for you). From harnad at Princeton.EDU Sun Jan 16 22:43:24 1994 From: harnad at Princeton.EDU (Stevan Harnad) Date: Sun, 16 Jan 94 22:43:24 EST Subject: Pattern Recognition Nets: PSYC Call for Book Reviewers Message-ID: <9401170343.AA03667@clarity.Princeton.EDU> CALL FOR BOOK REVIEWERS Below is the Precis of NEURAL NETWORKS FOR PATTERN RECOGNITION, by Albert Niigrin. This book has been selected for multiple review in PSYCOLOQUY. If you wish to submit a formal book review (see Instructions following Precis) please write to psyc at pucc.bitnet indicating what expertise you would bring to bear on reviewing the book if you were selected to review it (if you have never reviewed for PSYCOLOQUY or Behavioral & Brain Sciences before, it would be helpful if you could also append a copy of your CV to your message). If you are selected as one of the reviewers, you will be sent a copy of the book directly by the publisher (please let us know if you have a copy already). Reviews may also be submitted without invitation, but all reviews will be refereed. The author will reply to all accepted reviews. ----------------------------------------------------------------------- psycoloquy.93.4.2.pattern-recognition.1.nigrin Sunday 16 January 1994 ISSN 1055-0143 (34 paragraphs, 1 appendix, 1 table, 6 refs, 468 lines) PSYCOLOQUY is sponsored by the American Psychological Association (APA) Copyright 1994 Albert Nigrin Precis of: NEURAL NETWORKS FOR PATTERN RECOGNITION Albert Nigrin (1993) 8 chapters, 413 pages, Cambridge MA: The MIT Press Albert Nigrin Department of Computer Science and Information Systems The American University 4400 Massachusetts Avenue NW Washington DC 20016-8116 (202) 885-3145 [fax (202) 885-3155] nigrin at american.edu ABSTRACT: This Precis provides an overview of the book "Neural Networks for Pattern Recognition." First, it presents a list of properties that the author believes autonomous pattern classifiers should achieve. (These thirteen properties are also briefly discussed at the end.) It then describes the evolution of a self-organizing neural network called SONNET that was designed to satisfy those properties. It details the organization of (1) tutorial chapters that describe previous work; (2) chapters that present working neural networks for the context sensitive recognition of both spatial and temporal patterns; and (3) chapters that reorganize the mechanisms for competition to allow future networks to deal with synonymous and homonymic patterns in a distributed fashion. KEYWORDS: context sensitivity, machine learning, neural networks, pattern recognition, self-organization, synonymy 1. This book presents a self-organizing neural network called SONNET that has been designed to perform real-time pattern recognition. The book attempts to discover, through gedanken experiments, the fundamental properties that any pattern classifier should satisfy (see Table 1 and Appendix A below). It then proceeds to construct, step by step, a new neural network framework to achieve these properties. Although the framework described has not yet been fully implemented, a prototype network called SONNET 1 does exist. Simulations show that SONNET 1 can be used as a general purpose pattern classifier that can learn to recognize arbitrary spatial patterns (static patterns as in a snapshot) and segment temporal patterns (changing patterns as in speech) in an unsupervised fashion. Furthermore, SONNET 1 can learn new patterns without degrading the representations of previously classified patterns, even when patterns are allowed to be embedded within larger patterns. 2. The book can be subdivided into three major sections. The first section provides an introduction to neural networks for a general audience and presents the previous work upon which SONNET is based. The second section describes the structure of SONNET 1 and presents simulations to illustrate the operation of the network. And the third section describes a reorganization of the competitive structure of SONNET 1 to create more powerful networks that will achieve additional important properties. 3. The first segment consists of Chapters 1 and 2. After presenting a simplified network to introduce the subject to novices, Chapter 1 presents one possible definition for neural networks and an approach to designing them. The chapter then describes many of the fundamental properties that a neural network should achieve when it is being used for pattern classification. These properties are listed in Table 1 (reproduced from Nigrin, 1993) and are each briefly discussed in Appendix A below. _________________________________________________________________________ | | | A classification system should be able to: | | | | 1) self-organize using unsupervised learning. | | 2) form stable category codes. | | 3) operate under the presence of noise. | | 4) operate in real-time. | | 5) perform fast and slow learning. | | 6) scale well to large problems. | | 7) use feedback expectancies to bias classifications. | | 8) create arbitrarily coarse or tight classifications | | that are distortion insensitive. | | 9) perform context-sensitive recognition. | | 10) process multiple patterns simultaneously. | | 11) combine existing representations to create categories | | for novel patterns. | | 12) perform synonym processing. | | 13) unlearn or modify categories when necessary. | | | | TABLE 1 | |_______________________________________________________________________| 4. I believe that before one can construct (or understand) autonomous agents that can operate in real-world environments, one must design classification networks that satisfy all of the properties in Table 1. It is not easy to see how any of these properties could be pushed off to other components in a system, regardless of whether the architecture is used to classify higher level structures such as sentences or visual scenes, or lower level structures such as phonemes or feature detectors. For example, consider the problem of modeling language acquisition and recognition. It is illuminating to attempt to push off any of the above properties to a subsystem other than the classifying system and still account for human behavior without resorting to a homunculus or to circular arguments. 5. With a description of the goals for the book in hand, Chapter 2 begins the process of describing neural network mechanisms for achieving them. Chapter 2 presents a tutorial overview of the foundations underlying the neural networks in the book. The book presents only those mechanisms that are essential to SONNET. Alternative approaches such as backpropagation, Hopfield networks, or Kohonen networks are not discussed. The discourse begins at the level of the building blocks and discusses basic components such as cells and weights. It then describes some essential properties that must be achieved in short term memory (STM) and long term memory (LTM) and presents architectures that achieve them. 6. Chapter 2 also discusses how to incorporate these architectures into different networks. The two major networks described in the chapter are the ART networks of Carpenter and Grossberg (1987a, 1987b) and the masking field networks of Cohen and Grossberg (1986, 1987). The ART networks completely or partially achieve many important properties. They can self-organize using unsupervised learning; form stable category codes; operate in noise; operate in real-time; perform fast or slow learning; use feedback; and create tight or coarse classifications. The masking field is also an important architecture. It achieves a framework for achieving properties such as context sensitive recognition and simultaneous classification of multiple patterns. 7. After presenting the necessary groundwork, the book begins the presentation of the real-time network called SONNET, which is its main focus. Due to its complexity, the complete network has not yet been fully implemented. Instead, the implemented network contains simplifications that allowed it to be slowly built up and analyzed. These simplifications were also useful to allow the network to be completed within a reasonable time frame. However, they had the drawback of preventing the satisfaction of some important properties that will be achievable by the full network. 8. Chapter 3 presents the basic version of the model called SONNET 1, as it pertains to spatial patterns. This network merged the properties of the ART networks with those of the masking field networks. SONNET 1 either partially or totally achieved all but four of the properties listed in Table 1. (It did not use feedback, form distributed categories, perform synonym processing or unlearn classifications.) After the network is described, simulations are presented that show its behavior. Furthermore, simple improvements are described that could increase network performance. 9. To allow SONNET 1 to achieve these properties, several novel features were incorporated into the network. These included (among others) the following: (1) The network used a non-linear summing rule to allow the classifying nodes to reach decisions in real-time. This non-linear rule was similar to those found in networks using sigma-pi units. (2) A learning rule was used to allow the inhibitory weights to self-organize so that classifying nodes only competed with other nodes that represented similar patterns. This allowed the network to classify multiple patterns simultaneously. (3) Each node encoded two independent values in its output signal. The first output value represented the activity of the cell while the second value represented a confidence value that indicated how well the cell represented the input. The use of two output values allowed the network to form stable categories, even when input patterns were embedded within larger patterns. 10. Chapter 4 incorporates SONNET 1 into a framework that allows it to process temporal patterns. This chapter has several aspects. First, it shows how to design input fields that convert temporal sequences of events into classifiable spatial patterns of activity. Then, it describes how the use of feedback expectancies can help segment the sequences into reasonable length lists, and allow arbitrarily long sequences of events to be processed. 11. After describing the network, Chapter 4 presents simulations that show its operation. One of the simulations consisted of presenting the following list to the network, where each number refers to a specific input line. The list was presented by activating each input line for a constant period of time upon the presentation of its item. After the last item in the list was presented, the first item was immediately presented again, with no breaks between any of the items. 0 1 2 3 4 5 24 25 26 6 7 8 9 0 1 2 10 11 12 13 24 25 26 14 15 16 0 1 2 17 18 19 24 25 26 20 21 22 23 12. In this list, items (0,1,2) and (24,25,26) appear in three different contexts. Because of this, the network learned to create categories for those lists and to segment them accordingly. Thus, it learned in a real-time environment. It was also clear that it performed classifications in real-time since each of the lists was classified approximately 2 items after it had been fully presented. For example, if the list 22 23 0 1 2 3 4 5 6 was presented, the list (0,1,2) would be classified while item 4 or 5 was being presented. Simulations have shown that the amount of equilibration time needed for classification would not increase significantly, even if multiple similar patterns were classified by the network. 13. Chapter 5 continues to discuss the classification of temporal patterns. (However, many elements in this chapter are also applicable to purely spatial patterns.) The chapter shows how to cascade multiple homologous layers to create a hierarchy of representations. It also shows how to use feedback to bias the network in favor of expected occurrences and how to use a nonspecific attention signal to increase the power of the network. As is the case with the networks in later chapters, these proposed modifications are presented but not simulated. 14. One major limitation of the networks presented in Chapters 4 and 5 is that items can be presented only once within a classified list. For example, the list $ABC$ can be classified by the network, but the list $ABA$ cannot, since the $A$ occurs repeatedly. This deficiency is due to the simplifications that were made in the construction of SONNET 1. To overcome this and other weaknesses, the simplifications needed to be removed. 15. This is accomplished in Chapter 6, which presents a gedanken experiment analyzing the way repeated items in a list could be properly represented and classified. The chapter begins by showing that multiple representations of the same item are needed to allow the network to unambiguously represent the repeated occurrence of an item. It then analyzes methods by which the classifying system could learn to classify lists composed of these different representations. 16. During this gedanken experiment, it quickly became clear that the problem of classifying repeated items in a list was actually a subproblem of a more general one, called the synonym problem: Often, different input representations actually refer to the same concept and should therefore be treated by classifying cells as equivalent. However, the problem is complicated by the fact that sometimes different patterns refer to the same concept while sometimes the same pattern may have multiple meanings (homonyms). 17. To address the synonym problem, Chapter 6 presents a way to radically alter the method of competition between categories. In SONNET 1 (as in most competitive networks), classifying nodes compete with each other for the right to classify signals on active input lines. Conversely, in the altered network, it is the input lines that will compete with each other, and they will do so for the right to activate their respective classifying nodes. The principles in Chapter 6 are far and away the most important new contribution in this book. 18. After showing how synonyms could be learned and represented, Chapter 6 also discusses general mechanisms for creating distributed representations. These mechanisms were designed to allow existing representations to combine in STM (short-term memory) to temporarily represent novel patterns. They were also designed to allow the novel categories to be permanently bound in LTM (long-term memory). 19. After establishing the new mechanisms and principles in Chapter 6, these mechanisms are used in Chapter 7 to create specific architectures that tackle previously unsolved problems. The first section discusses the first implementation of SONNET that uses competition between links rather than nodes; it and shows how multiple patterns could be learned simultaneously. To complement the discussion in the previous chapter, the discussion here is as specific as possible (given that the network was yet to be implemented). The second section discusses how the new formulation could allow networks to solve the twin problems of translation and size invariant recognition of objects. This shows how the new mechanisms could be used to solve an important previously unresolved issue. 20. Finally, Chapter 8 concludes the book. It describes which properties have already been satisfied by SONNET 1, which properties can be satisfied by simple extensions to SONNET 1, and which properties must wait until future versions of SONNET are implemented. This chapter gives the reader a good indication of the current state of the network and also indicates areas for future research. 21. The following briefly summarizes thirteen properties that SONNET is meant to satisfy. Although it is possible to find examples in many different areas to motivate each of the following properties, the examples are mainly chosen from the area of natural language processing. This is done because the problems in this area are the easiest to describe and are often the most compelling. However, the reader should keep in mind that equivalent properties also exist in other domains and that, at least initially, SONNET is meant to be used primarily for lower level classification problems. 22. The first property is that a neural network should self-organize using unsupervised learning. It should form its own categories in response to the invariances in the environment. This allows the network to operate in an autonomous fashion and is important because in many areas, such as lower level perception, no external teacher is available to guide the system. Furthermore, as shown in the ARTMAP network (Carpenter, Grossberg, and Reynolds, 1991), it is often the case that if a network can perform unsupervised learning then it can also be embedded in a framework that allows it to perform supervised learning (but not the reverse). 23. The second property is that a neural network should form stable category codes. Thus, a neural network should learn new categories without degrading previous categories it has established. Networks that achieve this property can operate using both fast and slow learning (see fifth property). Conversely, those that do not are restricted to using slow learning. In addition, networks that don't form stable category codes must shut off learning at some point in time to prevent the degradation of useful categories. 24. The third property is that neural networks should operate in the presence of noise. This is necessary to allow them to operate in real-world environments. Noise can occur in three different areas. It can be present within an object, within the background of an object, and within the components of the system. A network must handle noise in all of these areas. 25. The fourth property is that a neural network should operate in real-time. There are several aspects to this. The first and most often recognized is that a net must equilibrate at least as fast as the patterns appear. However, there are several additional aspects to this property. First, in many applications, such as speech recognition and motion detection, a network should not equilibrate too rapidly, but at a pace that matches the evolution of the patterns. Second, in real-world environments, events do not come pre-labeled with markers designating the beginnings and endings of the events. Instead, the networks themselves must determine the beginning and end to each event and act accordingly. 26. The fifth property is that a neural network should perform fast and slow learning. A network should perform fast learning to allow it to classify patterns as quickly as a single trial when it is clear exactly what should be learned and it is important that the network learn quickly. (For example, one should not have to touch a hot stove 500 times before learning one will be burnt.) Furthermore, a network should also perform slow learning to allow it to generalize over multiple different examples. 27. The sixth property is that a neural network should scale well to large problems. There are at least two aspects to this property. First, as the size of a problem grows, the size of the required network should not grow too quickly. (While modularity may help in this respect, it is not a panacea, because of problems with locality and simultaneous processing.) Second, as the number of different patterns in a training set increases, the number of required presentations for each pattern (to obtain successful classifications) should not increase too rapidly. 28. The seventh property is that a neural network should use feedback expectancies to bias classifications. This is necessary because it is often ambiguous how to bind features into a category unless there is some context with which to place the features. 29. The eighth property is that a neural network should create arbitrarily coarse or tight classifications that are distortion insensitive. Patterns in a category often differ from the prototype (average) of the category. A network should vary the acceptable distortion from the prototype in at least two ways. It should globally vary the acceptable overall error. It should also allow different amounts of variance at different dimensions of the input pattern (the different input lines). This would allow the network to create categories that are more complex than just the nearest neighbor variety. 30. The ninth property is that a neural network should perform context-sensitive recognition. Two aspects of this will be discussed here. First, a network should learn and detect patterns that are embedded within extraneous information. For example, if the patterns SEEITRUN, ITSAT, and MOVEIT are presented, a network should establish a category for IT and later recognize the pattern when it appears within extraneous information. The second aspect occurs when a smaller classified pattern is embedded within a larger classified pattern. Then, the category for the smaller pattern should be turned off when the larger pattern is classified. For example, if a network has a category for a larger word like ITALY, then the category for IT should be turned off when the larger word is presented. Otherwise the category for IT would lose much of its predictive power, because it would learn the contexts of many non-related words such as HIT, KIT, SPIT, FIT, LIT, SIT, etc. 31. The tenth property is that a neural network should process multiple patterns simultaneously. This is important, because objects in the real world do not appear in isolation. Instead, scenes are cluttered with multiple objects that often overlap. To have any hope of segmenting a scene in real time, multiple objects often need to be classified in parallel. Furthermore, the parallel classifications must interact with one another, since it is often true that the segmentation for an object can only be determined by defining it in relation to other objects in the field. (Thus, it is not sufficient to use multiple stand-alone systems that each attempt to classify a single object in some selected portion of the input field.) The easiest modality in which to observe this is continuous speech, which often has no clear breaks between any words. (However, analogous situations also occur in vision.) For example, when the phrase ALL TURN TO THE SPEAKER is spoken, there is usually no break in the speech signal between the words ALL and TURN. Still, those words are perceived, rather than the embedded word ALTER. This can only be done by processing multiple patterns simultaneously, since the word ALTER by itself would overshadow both ALL and TURN. 32. The eleventh property is that a neural network should combine existing representations to create categories for novel patterns. These types of representations are typically called distributed ones. A network must form temporary representations in short term memory (STM) and also permanent iones in long term memory (LTM). Distributed representations are useful because they can reduce hardware requirements and also allow novel patterns to be represented as a combination of constituent parts. 33. The twelfth property is that a neural network should perform synonym processing. This is true because patterns that have entirely different physical attributes often have the same meaning, while a single pattern may have multiple meanings (as in homonyms). This is especially recognized in natural language, where words like "mean" and "average" sometimes refer to the same concept, and sometimes do not. However, solving the synonym problem will also solve problems that occur in the processing of lists composed of repeated occurrences of the same symbol (consider the letters "a" and "n" in the word "banana"). This follows because the different storage locations of a symbol can be viewed as (exact) synonyms for each other and handled in exactly the same way as the general case. Synonym representation is also necessary in object recognition, manifesting itself in several different ways. First, it is possible for multiple versions of the same object to appear within a scene (similar to the problem of repeated letters in a word). Second, since an object may appear completely when viewed different from different perspectives, it is important to map the dissimilar representations of the object onto the same category. Finally, it is also possible for an object to appear in different portions of the visual field (translation-invariant recognition) or with different apparent sizes (size-invariant recognition). Despite the fact that in both cases the object will be represented by entirely different sets of cells, a network should still classify the object correctly. 34. The thirteenth property is that a neural network should unlearn or modify categories when necessary. It should modify its categories passively to allow it to track slow changes in the environment. A network should also quickly change the meanings for its categories when the environment changes and renders them either superfluous or wrong. This property is the one least that ius discussed in the book, because it is possible that much unlearning could take place under the guise of reinforcement learning. APPENDIX: Table of Contents 1 Introduction 2 Highlights of Adaptive Resonance Theory 3 Classifying Spatial Patterns 4 Classifying Temporal Patterns 5 Multilayer Networks and the Use of Attention 6 Representing Synonyms 7 Specific Architectures That Use Presynaptic Inhibition 8 Conclusion Appendices REFERENCES Carpenter, G. and Grossberg, S. 1987a. A Massively Parallel Architecture for a Self-organizing Neural Pattern Recognition Machine. Computer Vision, Graphics, and Image Processing, 37:54--115. Carpenter, G. and Grossberg, S. 1987b. ART 2: Self-organization of Stable Category Recognition Codes for Analog Input Patterns. Applied Optics, 26(23):4919--4930. Carpenter,G., Grossberg, S., and Reynolds, J. 1991. ARTMAP: Supervised Real-time Learning and Classification of Nonstationary Data by a Self-organizing Neural Network. Neural Networks, 4(5):565-588. Cohen, M. and Grossberg, S. 1986. Neural Dynamics of Speech and Language Coding: Developmental Programs, Perceptual Grouping, and Competition for Short-term Memory. Human Neurobiology, 5(1):1--22. Cohen, M. and Grossberg, S. 1987. Masking Fields: a Massively Parallel Neural Architecture for Learning, Recognizing, and Predicting Multiple Groupings of Data. Applied Optics, 26:1866--1891. Nigrin, A. 1993. Neural Networks for Pattern Recognition. The MIT Press, Cambridge MA. -------------------------------------------------------------------- PSYCOLOQUY Book Review Instructions The PSYCOLOQUY book review procedure is very similar to the commentary procedure except that it is the book itself, not a target article, that is under review. (The Precis summarizing the book is intended to permit PSYCOLOQUY readers who have not read the book to assess the exchange, but the reviews should address the book, not primarily the Precis.) Note that as multiple reviews will be co-appearing, you need only comment on the aspects of the book relevant to your own specialty and interests, not necessarily the book in its entirety. Any substantive comments and criticism -- including points calling for a detailed and substantive response from the author -- are appropriate. Hence, investigators who have already reviewed or intend to review this book elsewhere are still encouraged to submit a PSYCOLOQUY review specifically written with this specialized multilateral review-and-response feature in mind. 1. Before preparing your review, please read carefully the Instructions for Authors and Commentators and examine recent numbers of PSYCOLOQUY. 2. Reviews should not exceed 500 lines. Where judged necessary by the Editor, reviews will be formally refereed. 3. Please provide a title for your review. As many commentators will address the same general topic, your title should be a distinctive one that reflects the gist of your specific contribution and is suitable for the kind of keyword indexing used in modern bibliographic retrieval systems. Each review should also have a brief (~50-60 word) Abstract 4. All paragraphs should be numbered consecutively. Line length should not exceed 72 characters. The review should begin with the title, your name and full institutional address (including zip code) and email address. References must be prepared in accordance with the examples given in the Instructions. Please read the sections of the Instruction for Authors concerning style, preparation and editing. PSYCOLOQUY is a refereed electronic journal (ISSN 1055-0143) sponsored on an experimental basis by the American Psychological Association and currently estimated to reach a readership of 20,000. PSYCOLOQUY publishes brief reports of new ideas and findings on which the author wishes to solicit rapid peer feedback, international and interdisciplinary ("Scholarly Skywriting"), in all areas of psychology and its related fields (biobehavioral, cognitive, neural, social, etc.) All contributions are refereed by members of PSYCOLOQUY's Editorial Board. Target article length should normally not exceed 500 lines [c. 4500 words]. Commentaries and responses should not exceed 200 lines [c. 1800 words]. All target articles, commentaries and responses must have (1) a short abstract (up to 100 words for target articles, shorter for commentaries and responses), (2) an indexable title, (3) the authors' full name(s) and institutional address(es). In addition, for target articles only: (4) 6-8 indexable keywords, (5) a separate statement of the authors' rationale for soliciting commentary (e.g., why would commentary be useful and of interest to the field? what kind of commentary do you expect to elicit?) and (6) a list of potential commentators (with their email addresses). All paragraphs should be numbered in articles, commentaries and responses (see format of already published articles in the PSYCOLOQUY archive; line length should be < 80 characters, no hyphenation). It is strongly recommended that all figures be designed so as to be screen-readable ascii. If this is not possible, the provisional solution is the less desirable hybrid one of submitting them as postscript files (or in some other universally available format) to be printed out locally by readers to supplement the screen-readable text of the article. PSYCOLOQUY also publishes multiple reviews of books in any of the above fields; these should normally be the same length as commentaries, but longer reviews will be considered as well. Book authors should submit a 500-line self-contained Precis of their book, in the format of a target article; if accepted, this will be published in PSYCOLOQUY together with a formal Call for Reviews (of the book, not the Precis). The author's publisher must agree in advance to furnish review copies to the reviewers selected. Authors of accepted manuscripts assign to PSYCOLOQUY the right to publish and distribute their text electronically and to archive and make it permanently retrievable electronically, but they retain the copyright, and after it has appeared in PSYCOLOQUY authors may republish their text in any way they wish -- electronic or print -- as long as they clearly acknowledge PSYCOLOQUY as its original locus of publication. However, except in very special cases, agreed upon in advance, contributions that have already been published or are being considered for publication elsewhere are not eligible to be considered for publication in PSYCOLOQUY, Please submit all material to psyc at pucc.bitnet or psyc at pucc.princeton.edu Anonymous ftp archive is DIRECTORY pub/harnad/Psycoloquy HOST princeton.edu From qin at turtle.fisher.com Mon Jan 17 09:56:58 1994 From: qin at turtle.fisher.com (qin@turtle.fisher.com) Date: Mon, 17 Jan 94 09:56:58 CDT Subject: Call For Papers Message-ID: <00978AA7A931E520.68601F60@turtle.fisher.com> The IEEE International Conference on Neural Networks is to be held in Orlando, Florida, June 26 - July 2, 1994. This conference is part of the World Congress on Computational Intelligence. This Announcement is to call for papers for a Special Session on "Neural Networks for Control". Papers related to using neural networks for control, system identification, fault detection and diagnosis are welcome, but not limitted to these areas. The suggested paper length is four pages. The maximum paper length is 6 pages. The deadline for submitting papers is January 31, 1994. All papers should be submitted to one of the two session organizers: Dr. S. Joe Qin Fisher-Rosemount Systems, Inc. 1712 Centre Creek Drive Austin, TX 78754 Tel 512-832-3635 FAX 512-834-7200 qin at fisher.com Dr. Ching-Fang Lin, President American GNC Corporation 9131 Mason Avenue Chatsworth, CA 91311 Tel 818-407-0092 FAX 818-407-0093 american_gnc at cup.portal.com FAX submissions are acceptable. From rupa at dendrite.cs.colorado.edu Mon Jan 17 12:38:10 1994 From: rupa at dendrite.cs.colorado.edu (Sreerupa Das) Date: Mon, 17 Jan 1994 10:38:10 -0700 Subject: NIPS preprint available via neuroprose Message-ID: <199401171738.AA08039@pons.cs.Colorado.EDU> FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/das.dolce.ps.Z Number of pages: 8 The following paper is now available for copying from the Neuroprose archive. Only electronic version of this paper is available. This is a preprint of the paper to appear in J.D. Cowan, G. Tesauro, and J. Alspector (eds.) Advances in Neural Information Processing Systems 6, 1994. A Unified Gradient-Descent/Clustering Architecture for Finite State Machine Induction Sreerupa Das and Michael C. Mozer Department of Computer Science University of Colorado at Boulder CO 80309--0430 ABSTRACT Although recurrent neural nets have been moderately successful in learning to emulate finite-state machines (FSMs), the continuous internal state dynamics of a neural net are not well matched to the discrete behavior of an FSM. We describe an architecture, called DOLCE, that allows discrete states to evolve in a net as learning progresses. DOLCE consists of a standard recurrent neural net trained by gradient descent and an adaptive clustering technique that quantizes the state space. DOLCE is based on the assumption that a finite set of discrete internal states is required for the task, and that the actual network state belongs to this set but has been corrupted by noise due to inaccuracy in the weights. DOLCE learns to recover the discrete state with maximum a posteriori probability from the noisy state. Simulations show that DOLCE leads to a significant improvement in generalization performance over earlier neural net approaches to FSM induction. ====================================================================== FTP procedure: unix> ftp archive.cis.ohio-state.edu (or 128.146.8.52) Name: anonymous Password: your_email_address ftp> cd pub/neuroprose ftp> binary ftp> get das.dolce.ps.Z ftp> quit unix> uncompress das.dolce.ps.Z unix> lpr das.dolce.ps Thanks to Jordan Pollack for maintaining the archive! Sreerupa Das Department of Computer Science University of Colorado at Boulder CO 80309-0430 email: rupa at cs.colorado.edu From zemel at salk.edu Mon Jan 17 17:18:38 1994 From: zemel at salk.edu (Richard S. Zemel) Date: Mon, 17 Jan 94 14:18:38 PST Subject: 2 NIPS preprints on neuroprose Message-ID: <9401172218.AA29430@broca> **DO NOT FORWARD TO OTHER GROUPS** The following two papers have been placed in the neuroprose archive. The first prints on 9 pages, the second on 8. The abstracts are given below, followed by retrieval instructions. Only electronic versions of these papers are available. Both are to appear in J.D. Cowan, G. Tesauro, and J. Alspector (Eds.), Advances in Neural Information Processing Systems 6, San Mateo, CA: Morgan Kaufmann. Rich Zemel e-mail: zemel at salk.edu FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/hinton.autoencoders.ps.Z FTP-filename: /pub/neuroprose/zemel.pop-codes.ps.Z ----------------------------------------------------------------------------- Autoencoders, Minimum Description Length and Helmholtz Free Energy Geoffrey E. Hinton and Richard S. Zemel An autoencoder network uses a set of {\it recognition} weights to convert an input vector into a code vector. It then uses a set of {\it generative} weights to convert the code vector into an approximate reconstruction of the input vector. We derive an objective function for training autoencoders based on the Minimum Description Length (MDL) principle. The aim is to minimize the information required to describe both the code vector and the reconstruction error. We show that this information is minimized by choosing code vectors stochastically according to a Boltzmann distribution, where the generative weights define the energy of each possible code vector given the input vector. Unfortunately, if the code vectors use distributed representations, it is exponentially expensive to compute this Boltzmann distribution because it involves all possible code vectors. We show that the recognition weights of an autoencoder can be used to compute an approximation to the Boltzmann distribution and that this approximation gives an upper bound on the description length. Even when this bound is poor, it can be used as a Lyapunov function for learning both the generative and the recognition weights. We demonstrate that this approach can be used to learn factorial codes. ----------------------------------------------------------------------------- Developing Population Codes By Minimizing Description Length Richard S. Zemel and Geoffrey E. Hinton The Minimum Description Length principle (MDL) can be used to train the hidden units of a neural network to extract a representation that is cheap to describe but nonetheless allows the input to be reconstructed accurately. We show how MDL can be used to develop highly redundant population codes. Each hidden unit has a location in a low-dimensional {\em implicit} space. If the hidden unit activities form a bump of a standard shape in this space, they can be cheaply encoded by the center of this bump. So the weights from the input units to the hidden units in an autoencoder are trained to make the activities form a standard bump. The coordinates of the hidden units in the implicit space are also learned, thus allowing flexibility, as the network develops a discontinuous topography when presented with different input classes. Population-coding in a space other than the input enables a network to extract nonlinear higher-order properties of the inputs. ----------------------------------------------------------------------------- To retrieve from neuroprose: unix> ftp cheops.cis.ohio-state.edu Name (cheops.cis.ohio-state.edu:zemel): anonymous Password: (use your email address) ftp> cd pub/neuroprose ftp> get zemel.pop-codes.ps.Z ftp> get hinton.autoencoders.ps.Z ftp> quit unix> uncompress zemel.pop-codes.ps unix> uncompress hinton.autoencoders.ps unix> lpr zemel.pop-codes.ps unix> lpr hinton.autoencoders.ps From schraudo at salk.edu Mon Jan 17 23:53:20 1994 From: schraudo at salk.edu (Nici Schraudolph) Date: Mon, 17 Jan 94 20:53:20 PST Subject: NIPS preprint available Message-ID: <9401180453.AA15851@salk.edu> Temporal Difference Learning of Position Evaluation in the Game of Go --------------------------------------------------------------------- Nicol N. Schraudolph Peter Dayan Terrence J. Sejnowski Computational Neurobiology Laboratory The Salk Institute for Biological Studies San Diego, CA 92186-5800 Abstract: The game of Go has a high branching factor that defeats the tree search approach used in computer chess, and long-range spatiotemporal inter- actions that make position evaluation extremely difficult. Development of conventional Go programs is hampered by their knowledge-intensive nature. We demonstrate a viable alternative by training networks to evaluate Go positions via temporal difference (TD) learning. Our approach is based on network architectures that reflect the spatial organization of both input and reinforcement signals on the Go board, and training protocols that provide exposure to competent (though un- labelled) play. These techniques yield far better performance than undifferentiated networks trained by self-play alone. A network with less than 500 weights learned within 3,000 games of 9x9 Go a position evaluation function that enables a primitive one-ply search to defeat a commercial Go program at a low playing level. -------- A preprint of the above paper is available by anonymous ftp from salk.edu (192.31.153.101), file pub/schraudo/nips93.ps.Z. (If you do not have ftp access to the Internet, send the message "help" to ftpmail at decwrl.dec.com for information on ftp-by-email service.) From D.Gorse at cs.ucl.ac.uk Tue Jan 18 13:36:32 1994 From: D.Gorse at cs.ucl.ac.uk (D.Gorse@cs.ucl.ac.uk) Date: Tue, 18 Jan 94 18:36:32 +0000 Subject: New preprint in neuroprose - avoiding local minima by homotopy Message-ID: FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/gorse.homotopy.ps.Z The file gorse.homotopy.ps.Z is now available for copying from the Neuroprose archive. This is a 6 page paper, submitted to WCNN '94 San Diego. A longer and more detailed paper describing this work is in preparation and will be available soon. --------------------------------------------------------------------------- A CLASSICAL ALGORITHM FOR AVOIDING LOCAL MINIMA D Gorse and A Shepherd Department of Computer Science University College, Gower Street, London WC1E 6BT, UK J G Taylor Department of Mathematics King's College, Strand, London WC2R 2LS, UK ABSTRACT: Conventional methods of supervised learning are inevitably faced with the problem of local minima; evidence is presented that conjugate gradient and quasi-Newton techniques are particularly susceptible to being trapped in sub-optimal solutions. A new classical technique is presented which by the use of a homotopy on the range of the target outputs allows supervised learning methods to find a global minimum of the error function in almost every case. Denise Gorse (D.Gorse at cs.ucl.ac.uk) ---------------------------------------------------------------------------- To obtain a copy: ftp archive.cis.ohio-state.edu login: anonymous password: cd pub/neuroprose binary get gorse.homotopy.ps.Z quit Then at your system: uncompress gorse.homotopy.ps.Z lpr -P gorse.homotopy.ps From N.Sharkey at dcs.shef.ac.uk Tue Jan 18 07:14:03 1994 From: N.Sharkey at dcs.shef.ac.uk (N.Sharkey@dcs.shef.ac.uk) Date: Tue, 18 Jan 94 12:14:03 GMT Subject: ADDRESS CHANGE Message-ID: <9401181214.AA04945@entropy.dcs.shef.ac.uk> ********************************* * * * CONNECTION SCIENCE * * ADDRESS CHANGE * * * ********************************* Please note that the Headquarters of Journal: Connection Science has moved to Sheffield University. The Journal is now into Volume 6 and is still going strong thanks to all of the support from the connectionist community. Because of the move, Lyn Shakelton, who has served as assistant editor since the beginning, is no longer with us. She has been replaced by a new Editorial Assistant Julie Clarke. I am sorry for any delay in responding to correspondence or in dealing with manuscripts that the move has caused. Bear with us. SUBMISSIONS SHOULD NOW BE SENT TO: Julie Clarke Connection Science Department of Computer Science Regent Court University of Sheffield S1 4DP, Sheffield, UK j.clarke at dcs.shef.ac.uk (or username julie) VOLUNTEER REVIEWERS: We have had extensive help from a number of reviewers of the past 5 years and we have worn some of them down to the bone. We are now trying to update our review panel to give some of the others a bit of a rest. If you wish to volunteer please contact Julie at the above address. We will be eternally grateful for your assistance. For other queries please contact me ************************************ * * * Professor Noel Sharkey * * Department of Computer Science * * Regent Court * * University of Sheffield * * S1 4DP, Sheffield, UK * * * * N.Sharkey at dcs.shef.ac.uk * * * ************************************ From hzs at cns.brown.edu Tue Jan 18 16:51:48 1994 From: hzs at cns.brown.edu (Harel Z. Shouval) Date: Tue, 18 Jan 1994 16:51:48 -0500 (EST) Subject: Correction Message-ID: <9401182151.AA26962@cns.brown.edu> A non-text attachment was scrubbed... Name: not available Type: text Size: 709 bytes Desc: not available Url : https://mailman.srv.cs.cmu.edu/mailman/private/connectionists/attachments/00000000/f4256760/attachment.ksh From kirk at FIZ.HUJI.AC.IL Wed Jan 19 05:34:00 1994 From: kirk at FIZ.HUJI.AC.IL (Scott Kirkpatrick) Date: Wed, 19 Jan 1994 12:34:00 +0200 Subject: preprints available on Neuroprose Message-ID: <199401191034.AA15823@binah.fiz.huji.ac.il> **DO NOT FORWARD TO OTHER GROUPS** FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/kirkpatrick.critical.ps.Z FTP-file: pub/neuroprose/kirkpatrick.nips93-statistical.ps.Z Only soft copy is available. The two above preprints are available for anonymous ftp in the Neuroprose archive. Full titles and authors are: Critical Behavior at the k-Satisfaction Threshold (preprint), by Scott Kirkpatrick and Bart Selman The Statistical Mechanics of k-Satisfaction, (NIPS-6 preprint) by S. Kirkpatrick, G. Gyorgyi, N. Tishby and L. Troyansky abstracts follow: {Critical Behavior at the $k$-Satisfiability Threshold} The satisfiability of random Boolean formulae with precisely $k$ variables per clause is a popular testbed for the performance of search algorithms in artificial intelligence and computer science. For $k = 2$, formulae are almost aways satisfiable when the ratio of clauses to variables is less than 1; for ratios larger than 1, the formulae are almost never satisfiable. We present data showing a similar threshold behavior for higher values of $k$. We also show how finite-size scaling, a method from statistical physics, can be used to characterize size dependent effects near the threshold. Finally, we commment on the relationship between thresholds and computational complexity. {The Statistical Mechanics of $k$-Satisfaction} The satisfiability of random CNF formulae with precisely $k$ variables per clause (``$k$-SAT'') is a popular testbed for the performance of search algorithms. Formulae have $M$ clauses from $N$ variables, randomly negated, keeping the ratio $\alpha = M/N$ fixed. For $k = 2$, this model has been proven to have a sharp threshold at $\alpha = 1$ between formulae which are almost aways satisfiable and formulae which are almost never satisfiable as $N \rightarrow \infty$. Computer experiments for $k$ = 2, 3, 4, 5 and 6, (carried out in collaboration with B. Selman of ATT Bell Labs) show similar threshold behavior for each value of $k$. Finite-size scaling, a theory of the critical point phenomena used in statistical physics, is shown to characterize the size dependence near the threshold. Annealed and replica-based mean field theories give a good account of the results. From hzs at cns.brown.edu Wed Jan 19 10:18:15 1994 From: hzs at cns.brown.edu (Harel Z. Shouval) Date: Wed, 19 Jan 1994 10:18:15 -0500 (EST) Subject: Correction to Correction Message-ID: <9401191518.AA29437@cns.brown.edu> A non-text attachment was scrubbed... Name: not available Type: text Size: 758 bytes Desc: not available Url : https://mailman.srv.cs.cmu.edu/mailman/private/connectionists/attachments/00000000/ae7ff432/attachment.ksh From skalsky at aaai.org Fri Jan 21 10:17:27 1994 From: skalsky at aaai.org (Rick Skalsky) Date: Fri, 21 Jan 94 07:17:27 PST Subject: AAAI-94 Special Notice Message-ID: <9401211517.AA17186@aaai.org> ----------------------------------------------------------------------------- Due to the earthquake in Los Angeles, and the severe winter weather in the North-Eastern U.S. and Canada, the deadline for the receipt of AAAI papers is being extended to Friday, January 28, 1994, for those authors that are severely impacted by these events. In order to minimize disruption of the self-selection and review process, however, the title pages, including paper title, authors, addresses, content areas, and abstract, must still arrive at the AAAI office by Monday, January 24th. These should be sent electronically, to abstract at aaai.org, in the format specified in the call for papers. This extension applies only to those individuals whose ability to complete their papers on time was severely impacted by these events, and authors are requested and expected to be honest in their use of it. If an electronic title page is submitted, but a decision is later made not to submit the paper, please send a message to abstract at aaai.org informing us of this fact as soon as possible. Please forward this message to anyone you know who may have been impacted by the storms or earthquake. Thank you very much. Barbara Hayes-Roth (bhr at ksl.stanford.edu) and Richard Korf (korf at cs.ucla.edu) Program Co-Chairs, AAAI-94 From moody at chianti.cse.ogi.edu Sat Jan 22 17:29:49 1994 From: moody at chianti.cse.ogi.edu (John Moody) Date: Sat, 22 Jan 94 14:29:49 -0800 Subject: Call for Papers: NEURAL NETWORKS in the CAPITAL MARKETS Message-ID: <9401222229.AA04012@chianti.cse.ogi.edu> ******************************************************************* --- Preliminary Announcement and Call for Papers --- NNCM-94 Second International Workshop NEURAL NETWORKS in the CAPITAL MARKETS Thursday-Friday, November 17-18, 1994 The Ritz-Carlton Hotel, Pasadena, California, U.S.A. Sponsored by Caltech and London Business School Neural networks have now been applied to a number of live systems in the capital markets, and in many cases have demonstrated better performance than competing approaches. Because of the overwhelming interest in the first NNCM workshop held in London in November 1993, and after the success of this workshop, the second annual NNCM workshop is planned for November 17-18, 1994, in Pasadena, California. This is a research meeting where original, significant contributions to the field are presented and discussed. In addition, two introductory tutorials will be included to familiarize audiences of different backgrounds with the financial aspects, and the mathematical aspects, of the field. Areas of Interest: Bond and stock valuation and trading, asset allocation and risk management, foreign exchange rate predication, commodity price forecasting, portfolio management, univariate time series analysis, multivariate data analysis, classification and ranking, pattern recognition, and hybrid systems. Organizing Committee: Dr. Y. Abu-Mostafa, California Institute of Technology Dr. A. Atiya, Cairo University Dr. N. Biggs, London School of Economics Dr. D. Bunn, London Business School Dr. B. LeBaron, University of Wisconsin Dr. A. Lo, MIT Sloan School Dr. J. Moody, Oregon Graduate Institute Dr. A. Refenes, London Business School Dr. M. Steiner, Universitaet Munster Dr. A. Timermann, Brickbeck College, London Dr. A. Weigend, University of Colorado Dr. H. White, University of California, San Diego Submission of Papers: Original contributions representing new and significant research, development, and applications in the above areas of interest will be considered. Authors should send 5 copies of a 1000-word summary clearly stating their results to Dr. Y. Abu-Mostafa, Caltech 116-81, Pasadena, CA 91125, U.S.A. All submissions must be received before May 1, 1994. There will be a rigorous refereeing process to select the high-quality papers to be presented at the workshop. Location: The workshop will be held at the Ritz-Carlton Huntington Hotel in Pasadena, within two miles from the Caltech campus. The hotel is a 35-minute drive from Los Angeles International Airport (LAX) with nonstop flights from most major cities in North America, Europe, the Far East, Australia, and South America. Mailing List: If you wish to be added to the mailing list of NNCM-94, please send your postal address, e-mail address, and fax number to Dr. Y. Abu-Mostafa, Caltech 116-81, Pasadena, CA 91125, U.S.A. e-mail: yaser at caltech.edu , fax (818) 568-8437 ******************************************************************* From SCHNEIDER at vms.cis.pitt.edu Sun Jan 23 09:15:00 1994 From: SCHNEIDER at vms.cis.pitt.edu (SCHNEIDER@vms.cis.pitt.edu) Date: Sun, 23 Jan 1994 09:15 EST Subject: Pre & Postdocs in neural processes in cognition in Pittsburgh Message-ID: <01H80M7Q2HSG9UP6GJ@vms.cis.pitt.edu> Pre- and Postdoctoral Training in Neural Processes in Cognition at the University of Pittsburgh and Carnegie Mellon University The Pittsburgh Neural Processes in Cognition program, now in its fourth year, is providing interdisciplinary training in brain sciences. The National Science Foundation has established an innovative program for students investigating the neurobiology of cognition. The program's focus is the interpretation of cognitive functions in terms of neuroanatomical and neurophysiological data and computer simulations. Such functions include perceiving, attending, learning, planning, and remembering in humans and in animals. This is an interdisciplinary program that prepares each student to perform original research investigating cortical function at multiple levels of analysis. State of the art facilities include: computerized microscopy, human and animal electrophysiological instrumentation, behavioral assessment laboratories, fMRI and PET brain scanners, the Pittsburgh Supercomputing Center, and a regional medical center providing access to human clinical populations. This is a joint program between the University of Pittsburgh, its School of Medicine, and Carnegie Mellon University. Each student receives full financial support, travel allowances and workstation support. Applications are encouraged from students with interest in biology, psychology, engineering, physics, mathematics, or computer science. Last year's class included mathematicians, psychologists, and neuroscience researchers. Pittsburgh is one of America's most exciting and affordable cities, offering outstanding symphony, theater, professional sports, and outdoor recreation in the surrounding Allegheny mountains. More than ten thousand graduate students attend its universities. Core Faculty and interests and affiliation CARNEGIE MELLON UNIVERSITY Psychology- James McClelland, Marlene Behrmann, Jonathan Cohen, Mark Johnson Computer Science - David Touretzky UNIVERSITY OF PITTSBURGH Behavioral Neuroscience - German Barrinonuevo, Susan Sesack Biology - Teresa Chay Information Science - Paul Munro Mathematics - Bard Ermentrout, Xiao-Jing Wang Neurobiology - John Horn, Al Humphrey, Peter Land, Charles Scudder, Dan Simons Neurological Surgery - Don Krieger, Robert Sclabassi Neurology - Steven Small, Robert Stowe Otolaryngology & physiology - Robert Schor Psychiatry - David Lewis, Lisa Morrow, Stuart Steinhauer Psychology - Walter Schneider, Velma Dobson, Michael Pogue-Geile Physiology - Dan Simons Radiology - Mark Mintun Applications: To apply to the program contact the program office or one of the affiliated departments. Students are admitted jointly to a home department and the Neural Processes in Cognition Program. Postdoctoral applicants MUST HAVE HAVE A SPONSOR AMONG THE TRAINING FACULTY. Most of our funds are limited to United States residents although there may be an option to consider some non-residents. To receive full consideration applications SHOULD BE SUBMITTED BY FEBRUARY 15. For information contact: Professor Walter Schneider Program Director Neural Processes in Cognition University of Pittsburgh 3939 O'Hara St Pittsburgh, PA 15260 Or: call 412-624-7064 or Email to NEUROCOG at VMS.CIS.PITT.EDU In Email requests for application materials, please provide your address and an indication of which department(s) you might be interested in. We can Email the research interests of the faculty. From mm at santafe.edu Sun Jan 23 20:19:10 1994 From: mm at santafe.edu (Melanie Mitchell) Date: Sun, 23 Jan 94 18:19:10 MST Subject: paper available Message-ID: <9401240119.AA00700@wupatki> The final version of our paper "When Will a Genetic Algorithm Outperform Hill Climbing?" (to appear in NIPS 6) is now available in neuroprose: When Will a Genetic Algorithm Outperform Hill Climbing? Melanie Mitchell John H. Holland Stephanie Forrest Santa Fe Institute University of Michigan University of New Mexico Abstract We analyze a simple hill-climbing algorithm (RMHC) that was previously shown to outperform a genetic algorithm (GA) on a simple ``Royal Road'' function. We then analyze an ``idealized'' genetic algorithm (IGA) that is significantly faster than RMHC and that gives a lower bound for GA speed. We identify the features of the IGA that give rise to this speedup, and discuss how these features can be incorporated into a real GA. The paper is 9 pages. To obtain a copy: unix> ftp archive.cis.ohio-state.edu Name: anonymous Password: (type your email address) ftp> cd pub/neuroprose ftp> binary ftp> get mitchell.ga-hillclimb.ps.Z ftp> quit unix>uncompress mitchell.ga-hillclimb.ps.Z unix> lpr mitchell.ga-hillclimb.ps From PIURI at IPMEL1.POLIMI.IT Mon Jan 24 18:58:01 1994 From: PIURI at IPMEL1.POLIMI.IT (PIURI@IPMEL1.POLIMI.IT) Date: Mon, 24 Jan 1994 18:58:01 MET-DST Subject: call for papers Message-ID: <01H82KU3MIJ6935O0G@ICIL64.CILEA.IT> ***************************************************************************** 37th MIDWEST SYMPOSIUM ON CIRCUITS AND SYSTEMS Lafayette Hilton and Towers, Lafayette, Louisiana August 3-5, 1994 CALL FOR PAPERS FOR THE SPECIAL SESSION ON THE EVALUATION OF NEURAL NETWORKS VS. CONVENTIONAL SYSTEMS ***************************************************************************** The 1994 Midwest Symposium on Circuits and Systems is organized by the Center for Advanced Computer Studies, University of Southwestern Louisiana. The symposium is devoted to all aspects of theory, design, and applications of circuits and systems. Emphasis is on current and future challenges in these areas as well as their interdisciplinary impact. Topics may include but are not limited to: + Analog/Digital Circuit Design + Digital Signal Processing + Control Systems & Robotics + Microwave Circuits + Nonlinear Circuits & Systems + Analog/Digital VLSI Design + Power Electronics & Systems + Analog and Digital Filter Design + Neural Networks + Image Processing + Fuzzy Logic + Communication Circuits + Solid State Circuits + Computer Networks + Expert Systems + Fault Analysis + Computer Aided Design + Petri Nets + Biomedical Applications + Military Applications + Space Applications + Automotive Applications + Intelligent Systems + Manufacturing + System Integration & + Multimedia Prototyping The special session on the evaluation of neural networks vs. conventional computing system is mainly directed (but not limited): - to identify and discuss the figures of merit that can be used to evaluate cost and performances of the neural networks, - to explore the limits and the advantages of the neural computation with respect to the conventional algorithmic approach, - to provide criteria for choosing among neural systems and for selecting a neural system vs. a conventional computing architecture. Authors interested in this special session are invited to send a one-page summary (by fax or e-mail) to the Session Program Chair by February 28, 1994. An extended summary (at least 4 pages plus figures) or the full paper must be sent (fax and e-mail latex submissions are accepted) to the Session Program Chair by March 28, 1994. Notification of acceptance or rejection and author kits will be sent by April 15, 1994. Session Program Chair for the special session on the evaluation of neural networks vs. conventional system Vincenzo Piuri Department of Electronics and Information Politecnico di Milano piazza L. da Vinci 32 I-20133 Milano, Italy phone no. +39-2-2399-3606 secretariat no. +39-2-2399-3623 fax no. +39-2-2399-3411 e-mail piuri at ipmel2.elet.polimi.it ***************************************************************************** General Chair: Registration: Magdy A. Bayoumi Cathy Pomier The Center for Advanced The Center for Advanced Computer Studies, USL Computer Studies, USL email: mab at cacs.usl.edu email: cathy at cacs.usl.edu Technical Program Chair: Technical Program Co-Chair: W. Ken Jenkins Hussein Baher Coordinated Science Lab Electrical Engineering Dept. University of Illinois KFU of Petroleum and Minerals email: jenkins at uicsl.csl.uiuc.edu Special Sessions Chair: Proceedings: Dolores Etter Nian-Feng Tzeng Department of Electrical Engg. The Center for Advanced University of Colorado Computer Studies email: etter at boulder.colorado.edu email: tzeng at cacs.usl.edu ***************************************************************************** From KELLYFJ at vax1.tcd.ie Mon Jan 24 08:03:00 1994 From: KELLYFJ at vax1.tcd.ie (Frank Kelly) Date: Mon, 24 Jan 1994 13:03 GMT Subject: ART vs. Leader clustering algorithm? Message-ID: <01H828GOASG0001GFZ@vax1.tcd.ie> Hello, Recently I heard an argument against Gail Carpenter and Stephen Grossberg's ART(Adaptive resonance theory). The basic argument was that ART was simply the 'leader clustering algorithm' enclosed in a load of neural net terminology. I am not very familiar with the leader clustering algorithm and was wondering would anyone like to remark for/against this argument as I am very interested in ART. Does anyone know of any paper on this subject? (ART vs. Leader clustering, or even leader clustering on it's own?). Cheers, --Frank Kelly *********************************************************************** kellyfj at vax1.tcd.ie | Senior Sophister Computer Science kellyfj at unix1.tcd.ie | Trinity College Dublin. Ireland. ======================================================================= From yuhas at bellcore.com Mon Jan 24 16:59:56 1994 From: yuhas at bellcore.com (Ben Yuhas) Date: Mon, 24 Jan 1994 16:59:56 -0500 Subject: Book Announcement Message-ID: <199401242159.QAA00782@om.bellcore.com> Title: NEURAL NETWORKS IN TELECOMMUNICATIONS Editors: Ben Yuhas, Bellcore Nirwan Ansari, New Jersey Institute of Technology Publisher: Kluwer Academic Publishers 367 pp. To Order: Phone: 617-871-6600 Fax: 617-871-6528 email: kluwer at world.std.com ISBN 0-7923-9417-8 Price is $105, but Kluwer will extend a 20% discount to those on the Connectionist mailing list through the end of February. Tell them you saw the add here when ordering. NEURAL NETWORKS IN TELECOMMUNICATIONS consists of a tightly edited collection of chapters that provides an overview of a wide range of telecommunications tasks being addressed with neural networks. These tasks range from the design and control of the underlying transport network to the filtering, interpretation and manipulation of the transported media. The chapters focus on specific applications, describe specific solutions and demonstrate the benefits that neural networks can provide. By doing this, the authors have demonstrated why neural networks should be another tool in the telecommunications engineer's toolbox. The contents include: 1. Introduction/ B.Yuhas, N.Ansari 2. Neural Networks for Switching/ T.X. Brown 3. Routing in Random Multistage Interconnection Networks/ M.W.Goudreau, C.L. Giles 4. ATM Traffic Control using Neural Networks/ A. H. Hiramatsu 5. Learning from Rare Events: Dynamic Cell Scheduling for ATM Networks/ D.B. Schwartz 6. A Neural Network Model for Adaptive Congestion Control in Broadband ATM Networks/ X. Chen 7. Structure and Performance of Neural Networks in Broadband Admission Control/ P.Trans-Gia, OLiver Gropp 8. Neural Network Channel Equalization/ W.R.Kirkland, D.P.Taylor 9. Application of Neural Networks as Exciser for Spread Spectrum Communication Systems/ R.Bijjani, P. K. Das 10. Static and Dynamic Channel Assignment using Simulated Annealing/ M. Duque-Anton, D.Kunz, B.Ruber 11. Cellular Mobile Communication Design Using Self-organizing Feature Maps/ T.Fritsch 12. Automatic Language Identification using Telephone Speech/ Y.K.Muthusamy, R.A. Cole 13.Text-Independent Talker Verification using Cohort Normalized Scores/ D.Burr 14. Neural Network Applications in Character Recognition and Document Analysis/ L.D. Jackel et al. 15. Image Vector Quantization by Neural Networks/ R. Lancini 16. Managing the Infoglut: Information Filtering using Neural Networks/ T.John by Thomas John 17. Empirical Comparisons of Neural Networks and Statistical Methods for Classification and Regression/ D.Duffy, B.Yuhas, A.Jain, A.Buja 18. A Neurocomputing Approach to Optimizing the Performance of a Satellite Communication Network/N.Ansari INDEX From webber at signal.dra.hmg.gb Tue Jan 25 04:25:54 1994 From: webber at signal.dra.hmg.gb (Chris Webber) Date: Tue, 25 Jan 94 09:25:54 +0000 Subject: NeuroProse preprint announcement Message-ID: FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/webber.self-org.ps.Z The file "webber.self-org.ps.Z" is available for copying from the Neuroprose preprint archive: TITLE: Self-organization of transformation-invariant neural detectors for constituents which recur within different perceptual patterns AUTHOR: Chris J.S. Webber (Cambridge University) (21 pages, preprint of article submitted to "Network" journal.) ABSTRACT: A simple self-organizing dynamics for governing the adaptation of individual neural perception units to the statistics of their input patterns is presented. The dynamics has a single adjustable parameter associated with each neuron, which directly controls the proportion of the patterns experienced that can induce response in the neuron, and thereby controls the nature of the neuron's response-preferences after the convergence of its adaptation. Neurons are driven by this dynamics to develop into detectors for the various individual pattern-constituents that recur frequently within the different patterns experienced: the elementary building-blocks which, in various combinations, make up those patterns. A detector develops so as to respond invariantly to those patterns which contain its trigger constituent. The development of discriminating detectors for specific faces, through adaptation to many photo-montages of combinations of different faces, is demonstrated. The characteristic property observed in the convergent states of this dynamics is that a neuron's synaptic vector becomes aligned symmetrically between pattern-vectors to which the neuron responds, so that those patterns project equal lengths onto the synaptic vector. Consequently, the neuron's response becomes invariant under the transformations which relate those patterns to one another. Transformation invariances that can develop in multi-layered systems of neurons, adapting according to this dynamics, include shape tolerance and local position tolerance. This is demonstrated using a two-level hierarchy, adapted to montages of cartoon faces generated to exhibit variability in facial expression and shape: neurons at the higher level of this hierarchy can discriminate between different faces invariantly with respect to expression, shape deformation, and local shift in position. These tolerances develop so as to correspond to the variability experienced during adaptation: the development of transformation invariances is driven entirely by statistical associations within patterns from the environment, and is not enforced by any constraints imposed on the architecture of neural connections. From hinton at cs.toronto.edu Tue Jan 25 11:36:50 1994 From: hinton at cs.toronto.edu (Geoffrey Hinton) Date: Tue, 25 Jan 1994 11:36:50 -0500 Subject: limited term faculty jobs at Toronto Message-ID: <94Jan25.113652edt.228@neuron.ai.toronto.edu> LIMITED TERM FACULTY POSITIONS AVAILABLE AT TORONTO The Department of Computer Science at the University of Toronto has two or three limited term faculty positions available. Appointments will be for 2 or 3 years and will not be renewed as these are NOT tenure-track jobs. The teaching load is approximately 4 hours of lectures per week for both semesters. Applications are invited from all areas of Computer Science. The needs of the Department would be well fitted by an applicant who can teach numerical analysis and does research on neural networks, especially learning algorithms, time series prediction, or image interpretation. The neural networks group in the department currently consists of Geoff Hinton, Peter Dayan, Mike Revow, Drew van Camp and eight graduate students (Tony Plate, Radford Neal, Chris Williams, Evan Steeg, Sid Fels, Ed Rasmussen, Brendan Frey and Sageeve Oore). We have close ties with other researchers in computer vision, statistics, and psychology. We have our own Silicon Graphics multiprocessor containing four R4400 chips. Applicants from the area of neural networks should send their applications to Geoffrey Hinton Computer Science Department University of Toronto 6 Kings College Road Toronto, Ontario M5S 1A4 CANADA Please include a CV, the names and addresses of 3 references, an outline of your research interests and a description of your background in numerical analysis and your teaching experience. Applications should be received by Feb 10, 1994. In accordance with Canadian immigration requirements, this advertisement is directed to Canadian citizens and permanent residents of Canada, but if there is no suitable Canadian applicant it may be possible to appoint another applicant. In accordance with its Employment Equity Policy, the University of Toronto encourages applications from qualified women or men, members of visible minorities, aboriginal peoples, and persons with disabilities. From rsun at cs.ua.edu Tue Jan 25 15:01:14 1994 From: rsun at cs.ua.edu (Ron Sun) Date: Tue, 25 Jan 1994 14:01:14 -0600 Subject: No subject Message-ID: <9401252001.AA28757@athos.cs.ua.edu> A special issue of _Connection Science_ journal on "Integrating Neural and Symbolic Processes" is now available Guest Editors of the special issue are: Larry Bookman Sun Microsystem Lab. Chelmsford, MA 01824 Ron Sun The University of Alabama Tuscaloosa, AL 35487 ----------------------------------------- Table of Contents for Special Issue Editorial: Integrating Neural and Symbolic Processes by L.A. Bookman and R. Sun Reflexive Reasoning with Multiple Instantiation in a Connectionist Reasoning System with a Type Hierarchy by D.R. Mani \& L. Shastri A Scalable Architecture for Integrating Associative and Semantic Memory by L. A. Bookman A Connectionist Production System with a Partial Match and its Use for Approximate Reasoning by N.K. Kasabov \& S.I. Shishkov Extraction, Insertion, and Refinement of Symbolic Rules in Dynamically-Driven Recurrent Neural Networks by C.L. Giles \& W. Omlin Combining Connectionist and Symbolic Learning to Refine Certainty-Factor Rule-Bases by J.J. Mahoney \& R.J. Mooney Combining Prior Symbolic Knowledge and Constructive Neural Network Learning by J. Fletcher \& Z. Obradovic Integrating Neural and Symbolic Approaches: A Symbolic Learning Scheme for a Connectionist Associative Memory by J.P. Ueberla \& A. Jagota Linking Symbolic and Subsymbolic Computing by A. Wilson \& J. Hendler Published by Carfax Publishing Company P.O.Box 25, Arbingdon, Oxfordsshire OK143UE UK e-mail: Carfax at ibmpcug.co.uk Editor-in-chief of Connection Science: Noel Sharkey e-mail: N.Sharkey at dcs.shef.ac.uk From barb at ai.mit.edu Tue Jan 25 18:21:48 1994 From: barb at ai.mit.edu (Barbara K. Moore Bryant) Date: Tue, 25 Jan 94 18:21:48 EST Subject: ART 1 Message-ID: <9401252321.AA09513@billberry> I have a paper about ART 1 and pattern clustering and would be happy to send it to you if you give me your mailing address. Or, you can look it up in Moore, "ART 1 and Pattern Clustering," Proceedings of the 1988 Connectionist Summer School, Morgan-Kaufman publ., pp. 174-185. I'd love to hear what you think. In the paper I show that ART 1 does in fact implement the leader clustering algorithm. ART 1's "stability" and "plasticity" are a property of the clustering algorithm (and the fact that only binary strings are the input and stored pattern), not of the underlying "neural" components. A careful reading of the paper and perusal of the examples might suggest that a different choice of distance metric or clustering algorithm might make more sense in a particular application. In fact, the final clusters formed by ART 1 might be described by some as downright weird (see Fig. 6 in my paper). I show by an example that other choices can be implemented in a similar architecture: there is no algorithmic constraint embodied in the architectural components of ART 1. About stability: ART 1 is stable because stored binary patterns can only be changed in one "direction" (you can change 1's to 0's but not 0's to 1's). So you will never get a situation where a pattern cycles. Moreover, no two stored patterns can be the same in ART 1, so you can only have finitely many stored patterns (because they're binary), and after some number of presentations of the same training set, the patterns will be fixed. Note that ART 1 would *not* be stable for real-valued inputs! As with any incremental clustering algorithm, different orders of presentation of input vectors to ART 1 during learning can result in different clusters. It is not necessarily bad or a problem that ART 1 implements the leader clustering algorithm. It would be nice, however, if this were made clear by the architects in the somewhat complicated papers that have been written on the subject. It might actually be very interesting that such architectures can implement clustering algorithms. It might be interesting to see what happens when you relax the constraint that all the underlying dynamical systems reach equilibrium before presenting the next training input. A cleverly designed architecture might behave in a useful way, or a biological way. (Note: I am not the only one to have made these observations about ART 1, but the presentation in my paper is the clearest that I know of. The paper is written so that it can be understood by people who aren't familiar with clustering.) barb at ai.mit.edu Please cc me on responses. From mherrmann at informatik.uni-leipzig.d400.de Wed Jan 26 09:58:36 1994 From: mherrmann at informatik.uni-leipzig.d400.de (mherrmann@informatik.uni-leipzig.d400.de) Date: Wed, 26 Jan 1994 15:58:36 +0100 Subject: job announcement Message-ID: <940126155836*/S=mherrmann/OU=informatik/PRMD=UNI-LEIPZIG/ADMD=D400/C=DE/@MHS> |------------------------------------------------------------| | Open Research Post in the EC-Project | | | | "Principles of Cortical Computation" | | | | at Leipzig University | |------------------------------------------------------------| The project forms a part of the "Human Capital and Mobility" programme and is a cooperative network between the University of Stirling (W.A. Phillips, coordinator), NORDITA (J. Hertz), MPI for Brain Research Frankfurt (W. Singer), the Insitute of Neuroinformatics Bochum (C. von der Malsburg), and the University of Leipzig (R. Der). The central goal of this network is to advance the understanding of the basic principles upon which computation in the mammalian cortex is based. The work of the Leipzig group is devoted to the statistical mechanical theory and/or applications of learning and self- organizing systems, in particular the Kohonen feature map. We use self-organizing layered Kohonen maps and the neural gas algorithm for hierarchical feature classification, time series predictions, and modelling and control of nonlinear dynamical systems. Reinforcement and Q-learning algorithms are of particular interest for the control tasks. Recent activities focus on the use of neural networks for the control of chaotic systems and possible implications for modelling the dynamical storage in the brain. The project runs until September 1995. Duration of the employment is about ten months. Preferentially the beginning of the employment should be in the next three months. Salary is about 5000.- DM per month according to qualification. As a rule applicants should have a doctoral degree but qualified graduate students are also considered. The applicant has to be a citizen of an EC country except Germany. Applications should contain a curriculum vitae, names and addresses of two references, a list of publications, and a statement of interests and should be submitted as soon as possible. Dr. habil. R. Der Leipzig, January 1994 Universitaet Leipzig Institut fuer Informatik Augustusplatz 10 - 11 D-04109 Leipzig Tel +49-341-719 2214 Fax +49-341-719 2399 e-mail: DER at INFORMATIK.UNI-LEIPZIG.D400.DE ------------------------------ End of body part 2 From kuh at spectra.eng.hawaii.edu Wed Jan 26 10:17:14 1994 From: kuh at spectra.eng.hawaii.edu (Anthony Kuh) Date: Wed, 26 Jan 94 10:17:14 HST Subject: extra NOLTA proceedings Message-ID: <9401262017.AA27871@spectra.eng.hawaii.edu> From sloman at columbo.cog.brown.edu Wed Jan 26 17:27:48 1994 From: sloman at columbo.cog.brown.edu (Steven Sloman) Date: Wed, 26 Jan 94 17:27:48 EST Subject: temporary job announcement Message-ID: <9401262227.AA06487@columbo.cog.brown.edu> Brown University Department of Cognitive and Linguistic Sciences Two Visiting Faculty Positions The Brown University Department of Cognitive and Linguistic Sciences invites applications for two temporary visiting faculty positions for the academic year September, 1994 to June, 1995. Each position would be suited to either a senior sabbatical visitor who, in exchange for half-time salary support, would teach one or two courses at Brown or to a more junior applicant who would receive full salary support and teach three courses. All applicants must have received the Ph.D. degree or equivalent by the time of their application. Position 1, Vision: A candidate should have strong teaching and research interests in one or more of the following areas: visual perception, visual cognition, computational vision, or computational neuroscience related to vision. Position 2, Cognition: A candidate should have strong teaching and research interests in an area such as memory, attention, problem solving, judgment and decision making, or comparative cognition. Please send vitae, recent publications, three references, and a cover letter describing teaching and research interests and qualifications to: Search Committee or Search Committee Vision Cognition Department of Cognitive and Linguistic Sciences Box 1978 Brown University Providence, RI 02912 The initial deadline for applications is February 15, 1994, but applications will be accepted after that time until the temporary positions are filled. Brown is an Equal Opportunity/Affirmative Action employer. Women and minorities are especially encouraged to apply. From mjhealy at espresso.rt.cs.boeing.com Wed Jan 26 21:08:03 1994 From: mjhealy at espresso.rt.cs.boeing.com (Michael J. Healy 865-3123 (206)) Date: Wed, 26 Jan 94 18:08:03 PST Subject: ART 1 Message-ID: <9401270208.AA09839@espresso.rt.cs.boeing.com> > Recently I heard an argument against Gail Carpenter and Stephen > Grossberg's ART(Adaptive resonance theory). The basic argument was that ART > was simply the 'leader clustering algorithm' enclosed in a load of neural > net terminology. I am not very familiar with the leader clustering > algorithm and was wondering would anyone like to remark for/against this > argument as I am very interested in ART. Does anyone know of any paper on > this subject? (ART vs. Leader clustering, or even leader clustering on > it's own?). > I thought it would be informative to post my reply, since I have done some work with ART. I would like to make two points: First, it is incorrect to state that the binary pattern clustering algorithm implemented by ART1 is equivalent to the leader clustering algorithm (ART is much more general than the ART1 architecture. I assumed the reference was to ART1). There are two significant differences: 1. ART1 is meant to function as a real-time clustering algorithm. This means that it (1) accepts and clusters input patterns in sequence, as they would appear in an application requiring an online system that learns as it processes data, and (2) is capable of finding a representation of the inputs that is arguably general (see below). The leader clustering algorithm, as I understand it, is supposed to have all its inputs available at once so that it can scan the set globally to form clusters. Hardly a real-time algorithm in any sense of the word. 2. The leader clustering algorithm does not generalize about its inputs. To explain, the patterns that it uses to represent its clusters are simply the input patterns that initiate the clusters (the "leaders"). ART1, on the other hand, forms a synaptic (in the neurobiological sense of the word) memory consisting of patterns that are templates for the patterns in each of the (real-time, dynamic) clusters that it forms. It updates these templates as it processes its inputs. Each template is the bitwise AND of all the input patterns that have been assigned to the corresponding cluster at some time in the learning history of ART1. This bitwise AND is a consequence of the Hebbian-like (actually, Weber-Fechner law) learning at each synapse in the outstar of F2 ---> F1 feedback connections from the F2 node that represents the cluster. A corresponding change occurs in the F1 ---> F2 connections to that same node, which form an adaptive filter for screening the inputs that come in through the F1 layer. Whether an input pattern is adopted by a particular cluster or not depends upon two measures of input pattern/template similarity that the ART1 system computes. The first measure is a result of F2 layer competition through inhibitory interconnections (again, synaptic). The second is computed by F2 ---> F1 gain control and the vigilance mechanism. The F2 ---> F1 gain control and F1 ---> vigilance node inhibitory connections, input layer ---> vigilance node connections, and vigilance node ---> F2 connections (all synaptic) effect the computation. The result is (1) Generalization. In fact, if the F1 nodes are thought of as implementing predicates in a two-valued logic, it is possible to prove that the ART1 templates represent conjunctive generalizations about the objects or events represented by the input patterns that have been adopted by a cluster. That is, each ART1 cluster represents a concept class. Each template also corresponds to a formula about any future objects that might be recognized as members of its concept class. This is more complicated than a simple conjunction of F1 predicates, but can be broken down into component conjunctions. I have a technical report on this, but the following reference is more useful relative to ART1 and its algorithm: Healy, M. J., Caudell, T. P. and Smith, S. D. G., A Neural Architecture for Pattern Sequence Verification Through Inferencing, IEEE Transactions on Neural Networks, Vol 4, No. 1, 1993, pp. 9-20. Suppose it is important to stabilize the memory on a fixed set of training patterns. Suppose it is desirable to know how many cycles, repeatedly showing the set of patterns to the ART1 system, are necessary to accomplish this; that is, how many cycles until the templates do not change any more, and each input pattern is recognized consistently as corresponding to a single template? Further, can the patterns be presented in some randomized order each time, or do they have to be presented in a particular order? The answer is as follows: Suppose that the number of distinct sizes of patterns---size being the number of 1-bits in a binary pattern---is M (obviously, M <= N, where N is the number of training patterns). Then M cycles are required. Further, the order of presentation can be arbitrary, and can be different with each cycle. Reference: M. Georgiopoulos, G. L. Heileman, and J. Huang, Properties of Learning Related to Pattern Diversity in ART1, Neural Networks, Vol. 4, pp. 751-757, 1991. This does not mean that the FORM of the templates is independent of the order of presentation. In fact, learning in ART1 is order-dependent, as it is in all clustering algorithms. I'll bet that leader clustering, even though it views the training set all at once, is also order-dependent. The inputs still have to be processed in some order and then deleted from the training set on each cycle. You could redo the entire training process for all N! possible presentation orders, but you would still have to somehow find the "best" of all the N! clusterings. My second point addresses the relevance of the argument that ART (meaning ART1) is "simply the leader clustering algorithm enclosed in a load of neural net terminology": ART1 represents a neural network, complete with a dynamic system model. Watch for Heileman, G., A Dynamical Adaptive Resonance Architecture, IEEE Transactions on Neural Networks (soon to appear) Given the relevance of ART1 to neural systems, including those that may actually exist in the brain, and given the proven stability of the ART1 algorithm, it seems to me that any argument that ART1 is simply this, that or the other algorithm is a moot point. I hope this sheds some light on the relationship between ART1 and the leader clustering algorithm. My thanks to the author of the original posting. Mike Healy From jbower at smaug.bbb.caltech.edu Thu Jan 27 13:00:45 1994 From: jbower at smaug.bbb.caltech.edu (Jim Bower) Date: Thu, 27 Jan 94 10:00:45 PST Subject: ART@ Message-ID: <9401271800.AA20109@smaug.bbb.caltech.edu> > Given the relevance of ART1 to neural systems, including > those that may actually exist in the brain, and given the proven > stability of the ART1 algorithm, it seems to me that any argument that > ART1 is simply this, that or the other algorithm is a moot point. As a neurobiologist who works with anatomically and physiologically derived models of cerebral cortical circuits, and memory, there is very little in the posted description of ART@ that justifies this statement. Ultimately, proposed solutions to engineering problems must live and die on their usefulness, not on asserted similarities to computing devices we do not understand (the brain in this case). It seems to me that if there is any lesson from the last 10 years research in "neural networks", it is that a thorough investigation of related algorithms in other domains is useful and appropriate. After all, this is supposed to be at least partially an intellectual exercise, not completely a sales job. Jim Bower From brown at galab3.mh.ua.edu Thu Jan 27 13:00:18 1994 From: brown at galab3.mh.ua.edu (brown@galab3.mh.ua.edu) Date: Thu, 27 Jan 1994 12:00:18 -0600 (CST) Subject: Batch Backprop versus Incremental Message-ID: <9401271800.AA15191@galab3.mh.ua.edu> A non-text attachment was scrubbed... Name: not available Type: text Size: 1329 bytes Desc: not available Url : https://mailman.srv.cs.cmu.edu/mailman/private/connectionists/attachments/00000000/39eda707/attachment.ksh From 70712.3265 at CompuServe.COM Thu Jan 27 17:37:07 1994 From: 70712.3265 at CompuServe.COM (Morgan Downey) Date: 27 Jan 94 17:37:07 EST Subject: World Congress on Neural Networks 1994 Message-ID: <940127223706_70712.3265_FHP116-1@CompuServe.COM> WORLD CONGRESS ON NEURAL NETWORKS 1994 (WCNN '94) Annual Meeting of the International Neural Network Society Town and Country Hotel - San Diego, California, USA - June 4-9, 1994 Revised Call for Papers Due Date: Tuesday, February 15, 1994 The International Neural Network Society is pleased to announce that it can accept post-deadline papers and is inviting INNS members and non-members to submit papers for WCNN '94 by Tuesday, February 15, 1994. Papers will be reviewed by the Organizing Committee for acceptance and presentation format and will be published in the proceedings. INNS members can designate one paper that they have authored for automatic acceptance and publication in the proceedings. Papers submitted utilizing this deadline extension will not be eligible for revision. Papers previously accepted by journals, or publicly accessible Tech reports may be submitted for poster presentations. Submit three copies of the paper with a one page abstract of the talk which clearly cites the paper well enough to permit easy access to it. Only the abstract will be published. Submission Procedures (These procedures supersede the previously published Call for Papers information in the brochure.): o Six (6) copies (1 original, (5) copies) Do not fold or staple originals. o Six page limit in English. $20 per page for papers exceeding (6) pages (do not number pages). Checks for over length charges should be made out to INNS and must be included with submitted paper. o Format: camera-ready 8 1/2" x 11" white paper, 1" margins all aides, one column format, single spaced, in Times or similar type style of 10 points or larger, one side of paper only. Faxed copies are not acceptable. o Center at the top of first page: Full title of paper, author names(s), affiliation(s), and mailing address(es), followed by blank space, and abstract (us to 15 lines), and text. o Cover letter to accompany paper must include: full title of paper, corresponding author(s), and presenting author name, address, telephone and fax numbers, 1st and 2nd choices of Technical Session (see session topics), and INNS membership number if applicable. o Author agrees to the transfer of copyright to INNS for the conference proceedings. All submitted papers become the property of INNS. _________________________________ SCHEDULE: Saturday, June 4, 1994 and Sunday, June 5, 1994 INNS UNIVERSITY SHORT COURSES Monday, June 6, 1994 NEURAL NETWORK INDUSTRIAL EXPOSITION RECEPTION OPENING CEREMONY Tuesday, June 7, 1994 PARALLEL SESSIONS EXHIBITS SPECIAL SESSION ON "BIOMEDICAL APPLICATIONS" SPECIAL SESSION ON "COMMERCIAL AND INDUSTRIAL APPLICATIONS" PLENARY 1: LOTFI ZADEH PLENARY 2: PER BAK Wednesday, June 8, 1994 PARALLEL SESSIONS EXHIBITS SPECIAL SESSION ON "FINANCIAL AND ECONOMIC APPLICATIONS" PLENARY 1: BERNARD WIDROW PLENARY 2: MELANIE MITCHELL SPECIAL INTEREST GROUP (SIGINNS) SESSIONS Thursday, June 9, 1994 PARALLEL SESSIONS EXHIBITS SPECIAL SESSION ON "NEURAL NETWORKS IN CHEMICAL ENGINEERING" SPECIAL SESSION ON "MIND, BRAIN, AND CONSCIOUSNESS" PLENARY 1: PAUL WERBOS PLENARY 2: JOHN TAYLOR Friday, June 10, 1994 and Saturday, June 11, 1994 SATELLITE MEETING: WNN/FNN 94 SAN DIEGO - NEURAL NETWORKS AND FUZZY LOGIC Sponsoring Society: NASA (National Aeronautics and Space Administration) Cooperating: INNS, Society for Computer Simulation, SPIE and all other interested societies. For more information contact: Mary Lou Padgett, Auburn University, 1165 Owens Road, Auburn AL 36830 ph: 205-821-2472 or 3488; fax: 205-844-1809; e-mail: mpadgett at eng.auburn.edu -- NASA Rep: Robert Savely, NASA/JSC __________________________________________________ WCNN 1994 ORGANIZING COMMITTEE: Paul Werbos, Chair Harold Szu Bernard Widrow Liaison to the European Neural Network Society: John G. Taylor Liaison to the Japanese Neural Network Society: Kunihiko Fukushima PROGRAM COMMITTEE: Daniel Alkon Shun-ichi Amari James A. Anderson Richard Andersen Kaveh Ashenayi Andrew Barto David Brown Horacio Bouzas Gail Carpenter David Casasent Ralph Castain Cihan Dagli Joel Davis Judith Dayhoff Guido DeBoeck David Fong Judith Franklin Walter Freeman Kunihiko Fukushima Michael Georgiopoulos Lee Giles Stephen Grossberg Dan Hammerstrom Robert Hecht-Nielsen Robert Jannarone Jari Kangas Christof Koch Teuvo Kohonen Bart Kosko Clifford Lau Soo-Young Lee George Lendaris Daniel Levine Alianna Maren Kenneth Marko Thomas McAvoy Thomas McKenna Larry Medsker Len Neiberg Erkki Oja Robert Pap Rich Peterson David Rumelhart Mohammed Sayeh Dejan Sobajic Harold Szu John Taylor Brian Telfer Shiro Usui John Weinstein Bernard Widrow Takeshi Yamakawa Lotfi Zadeh Mona Zaghloul COOPERATING SOCIETIES/INSTITUTIONS: American Association for Artificial Intelligence American Institute for Chemical Engineers American Physical Society Center for Devices and Radiological Health, US FDA Cognitive Science Society European Neural Network Society International Fuzzy Systems Association Japanese Neural Network Society Korean Neural Network Society US National Institute of Allergy and Infectious Diseases US Office of Naval Research Society for Manufacturing Engineers SPIE - The International Society for Optical Engineering Division of Cancer Treatment, US National Cancer Institute ____________________________________________________ SESSIONS AND CHAIRS: 1 Biological Vision ... S. Grossberg Invited Talk: Stephen Grossberg - Recent Results in Biological Vision 2 Machine Vision ... K. Fukushima, R. Hecht-Nielsen Invited Talk: Kunihiko Fukushima - Visual Pattern Recognition with Selective Attention Invited Talk: Robert Hecht-Nielsen - Foveal Active Vision: Methods, Results, and Prospects 3 Speech and Language ... D. Rumelhart, T. Peterson 4 Biological Neural Networks ... T. McKenna, J. Davis Session One: From Biological Networks to Silicon Invited Speakers: Frank Werblin, UC Berkeley Richard Granger, UC Irvine Theodore Berger, USC Session Two: Real Neurons in Networks Invited Speakers: Jim Schwaber, DuPont Misha Mahowald, Oxford University David Stenger, NRL Session Three: Networks for Motor Control and Audition Invited Speakers: Randy Beer, Case Western Reserve University Daniel Bullock, Boston University Shihab Shamma, University of Maryland Session Four: Learning and Cognition and Biological Networks Invited Speakers: Mark Gluck, Rutgers University Nestor Schmajuk, Northwestern University Michael Hasselmo, Harvard University 5 Neurocontrol and Robotics ... A. Barto, K. Ashenayi 6 Supervised Learning ... G. Lendaris, S-Y. Lee Invited Talk: George Lendaris - Apriori Knowledge and NN Architectures Invited Talk: Soo-Young Lee - Error Minimization, Generalization, and Hardware Implementability of Supervised Learning 7 Unsupervised Learning ... G. Carpenter, R. Jannarone Invited Talk: Gail Carpenter - Distributed Recognition Codes and Catastrophic Forgetting Invited Talk: Robert Jannarone - Current Trends of Learning Algorithms 8 Pattern Recognition ... T. Kohonen, B. Telfer Invited Talk: Teuvo Kohonen - Physiological Model for the Self-Organizing Map Invited Talk: Brian Telfer - Challenges in Automatic Object Recognition: Adaptivity, Wavelets, Confidence 9 Prediction and System Identification ... P. Werbos, G. Deboeck Invited Talk: Guido Deboeck - Neural, Genetic, and Fuzzy Systems for Trading Chaotic Financial Markets 10 Cognitive Neuroscience ... D. Alkon, D. Fong 11 Links to Cognitive Science and Artificial Intelligence ... J. Anderson, L. Medsker Invited Talk: Larry Medsker - Hybrid Intelligent Systems: Research and Development Issues 12 Neural Fuzzy Systems ... L. Zadeh, B. Kosko 13 Signal Processing ... B. Widrow, H. Bouzas Invited Talk: Bernard Widrow - Nonlinear Adaptive Signal Processing 14 Neurodynamics and Chaos ... H. Szu, M. Zaghloul Invited Talk: Walter Freeman - Biological Neural Network Chaos Invited Talk: Harold Szu - Artificial Neural Network Chaos 15 Hardware Implementations ... C. Lau, R. Castain, M. Sayeh Invited Talk: Clifford Lau - Challenges in Neurocomputers Invited Talk: Mark Holler - High Performance Classifier Chip 16 Associative Memory ... J. Taylor, S. Usui Invited Talk: John G. Taylor - Where is Associative Memory Going? Invited Talk: Shiro Usui - Review of Associative Memory 17 Applications ... D. Casasent, B. Pap, D. Sobajic Invited Talk: David Casasent - Optical Neural Networks and Applications Invited Talk: Yoh-Han Pao - Mathematical Basis for the Power of the Functional-Link Net Approach: Applications to Semiconductor Processing Invited Talk: Mohammed Sayeh - Advances in Optical Neurocomputers 18 Neuroprocessing and Virtual Reality ... L. Giles, H. Hawkins Invited Talk: Harold Hawkins - Opportunities for Virtual Environment and Neuroprocessing 19 Circuits and System Neuroscience ... J. Dayhoff, C. Koch Invited Talk: Judith Dayhoff - Temporal Processing for Neurobiological Signal Processing Invited Talk: Christof Koch - Temporal Analysis of Spike Patterns in Monkeys and Artificial Neural Networks 20 Mathematical Foundations ... S-I. Amari, D. Levine Invited Talk: Shun-ichi Amari - Manifolds of Neural Networks and EM Algorithms Additional session invited talks to be determined. Session invited talks will not be scheduled to run concurrently at WCNN 1994. *Invited INNS wishes to acknowledge the US Office of Naval Research for its generous support of the Biological Neural Networks Session at WCNN 1994 ___________________________________________________________ NEW IN '94! INNS UNIVERSITY SHORT COURSES INNS is proud to announce the establishment of the INNS University Short Course format to replace the former tutorial program. The new 2-day, 4-hour per course format provides twice the instruction with much greater depth and detail. There will be six parallel tracks offered in three segments (morning, afternoon, and evening each day). INNS reserves the right to cancel Short Courses and refund payment should registration not meet the minimum number of persons required per Short Course. [Dates and times are listed after each instructor; course descriptions are available by contacting INNS.] A. Teuvo Kohonen, Helsinki University of Technology - SATURDAY, JUNE 4. 1994, 6-10 PM Advances in the Theory and Applications of Self-Organizing Maps B. James A. Anderson, Brown University - SATURDAY, JUNE 4. 1994, 1-5 PM Neural Network Computation as Viewed by Cognitive Science and Neuroscience C. Christof Koch, California Institute of Technology - SUNDAY, JUNE 5. 1994, 6-10 PM Vision Chips: Implementing Vision Algorithms with Analog VLSI Circuits D. Kunihiko Fukushima, Osaka University - SATURDAY, JUNE 4. 1994, 1-5 PM Visual Pattern Recognition with Neural Networks E. John G. Taylor, King's College London - SUNDAY, JUNE 5. 1994, 1-5 PM Stochastic Neural Computing: From Living Neurons to Hardware F. Harold Szu, Naval Surface Warfare Center - SATURDAY, JUNE 4. 1994, 6-10 PM Spatiotemporal Information Processing by Means of McCollouch-Pitts and Chaotic Neurons G. Shun-ichi Amari, University of Tokyo - SUNDAY, JUNE 5. 1994, 8 AM-12 PM Learning Curves, Generalization Errors and Model Selection H. Walter J. Freeman, University of California, Berkeley - SUNDAY, JUNE 5. 1994, 8 AM-12 PM Review of Neurobiology: From Single Neurons to Chaotic Dynamics of Cerebral Cortex I. Judith Dayhoff, University of Maryland - SATURDAY, JUNE 4. 1994, 8 AM-12 PM Neurodynamics of Temporal Processing J. Richard A. Anderson, Massachusetts Institute of Technology - SATURDAY, JUNE 4. 1994, 6-10 PM Neurobiologically Plausible Neural Networks K. Paul Werbos, National Science Foundation - SUNDAY, JUNE 5. 1994, 1-5 PM From stolcke at ICSI.Berkeley.EDU Thu Jan 27 21:35:32 1994 From: stolcke at ICSI.Berkeley.EDU (Andreas Stolcke) Date: Thu, 27 Jan 1994 18:35:32 PST Subject: TR on HMM induction available Message-ID: <199401280235.SAA07317@icsib30.ICSI.Berkeley.EDU> The following technical report is now available from ICSI. FTP instructions appear at the end of this message. Note that this report is a greatly expanded and revised follow-up to our paper in last year's NIPS volume. It replaces report TR-93-003 mentioned in that paper, which was never released as we decided to include substantial new material instead. We regret any confusion or inconvenience this may have caused. Andreas Stolcke Stephen Omohundro -------------------------------------------------------------------------- Best-first Model Merging for Hidden Markov Model Induction Andreas Stolcke and Stephen M. Omohundro TR-94-003 January 1994 Abstract: This report describes a new technique for inducing the structure of Hidden Markov Models from data which is based on the general `model merging' strategy (Omohundro 1992). The process begins with a maximum likelihood HMM that directly encodes the training data. Successively more general models are produced by merging HMM states. A Bayesian posterior probability criterion is used to determine which states to merge and when to stop generalizing. The procedure may be considered a heuristic search for the HMM structure with the highest posterior probability. We discuss a variety of possible priors for HMMs, as well as a number of approximations which improve the computational efficiency of the algorithm. We studied three applications to evaluate the procedure. The first compares the merging algorithm with the standard Baum-Welch approach in inducing simple finite-state languages from small, positive-only training samples. We found that the merging procedure is more robust and accurate, particularly with a small amount of training data. The second application uses labelled speech data from the TIMIT database to build compact, multiple-pronunciation word models that can be used in speech recognition. Finally, we describe how the algorithm was incorporated in an operational speech understanding system, where it is combined with neural network acoustic likelihood estimators to improve performance over single-pronunciation word models. -------------------------------------------------------------------------- Instructions for retrieving ICSI technical reports via ftp Replace YEAR and tr-XX-YYY with the appropriate year and TR number. If your name server is ignorant about ftp.icsi.berkeley.edu, use 128.32.201.55 instead. unix% ftp ftp.icsi.berkeley.edu Name (ftp.icsi.berkeley.edu:): anonymous Password: your_name at your_machine ftp> cd /pub/techreports/YEAR ftp> binary ftp> get tr-XX-YYY.ps.Z ftp> quit unix% uncompress tr-XX-YYY.ps.Z unix% lpr -Pyour_printer tr-XX-YYY.ps All files in this archive can also be obtained through an e-mail interface in case direct ftp is not available. Send mail containing the line `send help' to ftpmail at ICSI.Berkeley.EDU for instructions. As a last resort, hardcopies may be ordered for a small fee. Send mail to info at ICSI.Berkeley.EDU for more information. From sef+ at cs.cmu.edu Fri Jan 28 00:06:43 1994 From: sef+ at cs.cmu.edu (Scott E. Fahlman) Date: Fri, 28 Jan 94 00:06:43 EST Subject: Batch Backprop versus Incremental In-Reply-To: Your message of Thu, 27 Jan 94 12:00:18 -0600. <9401271800.AA15191@galab3.mh.ua.edu> Message-ID: ...a gentleman from the U.K. suggested that Batch mode learning could possibly be unstable in the long term for backpropagation. I did not know the gentleman and when I asked for a reference he could not provide one. Does anyone have any kind of proof stating that one method is better than another? Or that possibly batch backprop is unstable in <> sense? Those U.K. guys get some funny ideas. I think it's the intoxicating fumes from wet sheep. :-) Batch mode backprop is actually more stable (other things being equal) than online (also known as "stochastic") updating. In batch mode, each weight update is made with respect to the true error gradient, computed over the whole training set. In online, each update is made with respect to a single sample, and a few atypical samples in a row can take you very far afield, especially if you use one of the fast training methods that adapts step-size. In addition, online training never settles down into a stable minimum, since the network continues to be buffeted by the individual training cases as they arrive. (You can, of course, reduce the learning rate once the net seems to have found a solution.) Perhaps the origin of this myth about batch learning is that you need to scale down the gradient values (or the learning rate parameter) as the number of training cases goes up. If you don't, the effective learning rate can become arbitrarily large and the learning will be unstable. This isn't to say that batch learning is necessarily better. As long as you take small steps, online updating will usually be stable, and it can be much faster than batch updating if the training set is large and redundant. A net trained by online update might converge before even a single epoch has been completed. -- Scott =========================================================================== Scott E. Fahlman Internet: sef+ at cs.cmu.edu Senior Research Scientist Phone: 412 268-2575 School of Computer Science Fax: 412 681-5739 Carnegie Mellon University Latitude: 40:26:33 N 5000 Forbes Avenue Longitude: 79:56:48 W Pittsburgh, PA 15213 =========================================================================== From dfisher at vuse.vanderbilt.edu Fri Jan 28 07:39:55 1994 From: dfisher at vuse.vanderbilt.edu (Douglas H. Fisher) Date: Fri, 28 Jan 94 06:39:55 CST Subject: AI and Stats Workshop Message-ID: <9401281239.AA05663@aim.vuse> Call For Papers Fifth International Workshop on Artificial Intelligence and Statistics January 4-7, 1995 Ft. Lauderdale, Florida PURPOSE: This is the fifth in a series of workshops which has brought together researchers in Artificial Intelligence and in Statistics to discuss problems of mutual interest. The exchange has broadened research in both fields and has strongly encouraged interdisciplinary work. This workshop will have as its primary theme: ``Learning from data'' Papers on other topics at the interface of AI & Statistics are *strongly* encouraged as well (see TOPICS below). FORMAT: To encourage interaction and a broad exchange of ideas, the presentations will be limited to about 20 discussion papers in single session meetings over three days (Jan. 5-7). Focussed poster sessions will provide the means for presenting and discussing the remaining research papers. Papers for poster sessions will be treated equally with papers for presentation in publications. Attendance at the workshop will *not* be limited. The three days of research presentations will be preceded by a day of tutorials (Jan. 4). These are intended to expose researchers in each field to the methodology used in the other field. The Tutorial Chair is Prakash Shenoy. Suggestions on tutorial topics can be sent to him at pshenoy at ukanvm.bitnet. LANGUAGE: The language will be English. TOPICS OF INTEREST: The fifth workshop has a primary theme of ``Learning from data'' At least one third of the workshop schedule will be set aside for papers with this theme. Other themes will be developed according to the strength of the papers in other areas, including but not limited to: - integrated man-machine modeling methods - empirical discovery and statistical methods for knowledge acquisition - probability and search - uncertainty propagation - combined statistical and qualitative reasoning - inferring causation - quantitative programming tools and integrated software for data analysis and modeling. - discovery in databases - meta data and design of statistical data bases - automated data analysis and knowledge representation for statistics - connectionist approaches - cluster analysis SUBMISSION REQUIREMENTS: Three copies of an extended abstract (up to four pages) should be sent to H. Lenz, Program Chair or D. Fisher, General Chair 5th Int'l Workshop on AI & Stats 5th Int'l Workshop on AI & Stats Free University of Berlin Box 1679, Station B Department of Economics Department of Computer Science Institute for Statistics Vanderbilt University and Econometrics Nashville, Tennessee 37235 14185 Berlin, Garystr 21 USA Germany or electronically (postscript or latex documents preferred) to ai-stats-95 at vuse.vanderbilt.edu Submissions for discussion papers (and poster presentations) will be considered if *postmarked* by June 30, 1994. If the submission is electronic (e-mail), then it must be *received* by midnight June 30, 1994. Abstracts postmarked after this date but *before* July 31, 1994, will be considered for poster presentation *only*. Please indicate which topic(s) your abstract addresses and include an electronic mail address for correspondence. Receipt of all submissions will be confirmed via electronic mail. Acceptance notices will be mailed by September 1, 1994. Preliminary papers (up to 20 pages) must be returned by November 1, 1994. These preliminary papers will be copied and distributed at the workshop. PROGRAM COMMITTEE: General Chair: D. Fisher Vanderbilt U., USA Program Chair: H. Lenz Free U. Berlin, Germany Members: W. Buntine NASA (Ames), USA J. Catlett AT&T Bell Labs, USA P. Cheeseman NASA (Ames), USA P. Cohen U. of Mass., USA D. Draper U. of Bath, UK Wm. Dumouchel Columbia U., USA A. Gammerman U. of London, UK D. J. Hand Open U., UK P. Hietala U. Tampere, Finland R. Kruse TU Braunschweig, Germany S. Lauritzen Aalborg U., Denmark W. Oldford U. of Waterloo, Canada J. Pearl UCLA, USA D. Pregibon AT&T Bell Labs, USA E. Roedel Humboldt U., Germany G. Shafer Rutgers U., USA P. Smyth JPL, USA MORE INFORMATION: For more information write dfisher at vuse.vanderbilt.edu or write to ai-stats-request at watstat.uwaterloo.ca to subscribe to the AI and Statistics mailing list. Traditionally, the Workshop has attracted many with an interest in connectionism, and we encourage even more for the 1995 Workshop. From qin at turtle.fisher.com Fri Jan 28 08:49:46 1994 From: qin at turtle.fisher.com (qin@turtle.fisher.com) Date: Fri, 28 Jan 94 08:49:46 CDT Subject: Batch vs. Pattern Backprop Message-ID: <009793431874AAA0.686054A3@turtle.fisher.com> From: UUCP%"ds2100!galab3.mh.ua.edu!brown" 28-JAN-1994 06:35:06.88 To: cs.cmu.edu!Connectionists CC: Subj: Batch Backprop versus Incremental >From: ds2100!galab3.mh.ua.edu!brown >Message-Id: <9401271800.AA15191 at galab3.mh.ua.edu> >Subject: Batch Backprop versus Incremental >To: cs.cmu.edu!Connectionists >Date: Thu, 27 Jan 1994 12:00:18 -0600 (CST) >X-Mailer: ELM [version 2.4 PL22] >Content-Type: text >Content-Length: 1329 >Dear Connectionists, > I attended the 5th International Conference on Genetic Algorithms >this summer, and in one of the sessions on combinations of genetic >algorithms (GAs) and Neural Nets(ANNs) a gentleman from the U.K. >suggested that Batch mode learning could possibly be unstable in >the long term for backpropagation. I did not know the gentleman >and when I asked for a reference he could not provide one. >Does anyone have any kind of proof stating that one method is better >than another? Or that possibly batch backprop is unstable in <> >sense? >Any and all response are thanked for in advance, >Brown Cribbs =============================================================== Brown, I suggest the following paper for reference: S.Z. Qin, et al. (1992). Comparison of four neural net learning methods for system identification. IEEE TRANSACTIONS ON NEURAL NETWORKS. vol3, no.1 pp122-130. It is proven in the paper that the batch and pattern/incremental learning methods are equivalent given small learning rates. There is no result showing that batch learning is more unstable. However, one simulation in the paper shows that batch learning has ripples for large learning rates in a particular simulation. But the pattern learning does not. In other words, batch learning error decreases, then increases, and then decreases. Initially batch learning was a bit faster than pattern learning, but it has this kind of ripples. My guess for this observation is due to the particular error surface. In summary, there is no significant difference for small learning rate, but there is difference for large learning rates. Though there is one simulation example showing pattern learning is more stable than batch learning, it may not be generally true. S.Joe Qin Fisher-Rosemount Systems, Inc. 1712 Centre Creek Drive Austin, TX 78754 512-832-3635 words, The From BATTITI at itnvax.science.unitn.it Fri Jan 28 13:10:13 1994 From: BATTITI at itnvax.science.unitn.it (BATTITI@itnvax.science.unitn.it) Date: 28 Jan 1994 18:10:13 +0000 Subject: MEMORY, LEARNING & OPTIMIZATION - comments and preprint - Message-ID: <01H884B3Q7KIMAVS1W@itnvax.science.unitn.it> MEMORY, LEARNING & OPTIMIZATION Are Monte Carlo techniques appropriate? A research stream in Neural Networks proposes to use NN for Combinatorial Optimization (CO), with mixed results. Other streams consider state of the art optimization techniques and apply them to machine learning and NN tasks. In reality the areas of optimization and learning are deeply interrelated. In particular, recent CO algorithms use MEMORY and LEARNING to generate efficient search trajectories from a set of local moves. Unfortunately Markov Chain Monte Carlo methods are memory-less (by definition, the distribution of the random variable X(t+1) in a Markov chain is mediated entirely by the value of X(t)). The past history does NOT influence the current move. Is the importance assigned to Markov chain sampling methods in the area of optimization and learning justified by the theoretical results (whose practical interest is dubious, see the clear discussion about sa in [1]) or by the results obtained on significant tasks? We doubt it. =========================================================================== Now we describe a homework that we made after reading the interesting paper placed in the neuroprose archive by M. Mitchell [2] about a week ago. (to spare e-mail bytes we assume knowledge of [2]) On the "Royal Road" function R1 (Table 1 in the cited reference), GA converges in 61,334 function evaluations (mean), RMHC (a zero-temperature Monte Carlo method used by Forrest and Mitchell) in 6,179. In RMHC, a given string is mutated at a randomly chosen single locus. If the mutation leads to an equal or higher fitness, the new string replaces the old string. Now, if the last mutation increased the fitness, it is reasonable that it is NOT considered again in the next step (otherwise the successful mutation can be undone). Similarly, if the last mutation was not accepted because the fitness would have decreased, it is pointless to consider it again in the next step. Let us generalize and modify RMHC: if mutation (i) at iteration (t) either decreases or increases the fitness, (i) is prohibited for the next T iterations (the corresponding bit in the string is "frozen"). Nothing happens if the fit- ness remains unchanged. Here are the homework results as a function of T (100 runs for each value): T mean f.evals st. dev. 0 6,450 (289) ----(confirms results of [2]) 50 3,728 (148) 100 3,122 (104) 200 2,805 (94) 300 2,575 (88) 400 2,364 (83) 500 2,605 (79) The use of this simple form of memory can accelerate the convergence (about 1/3 function evaluations are sufficient when T=400). Bye the way, we would not be surprised if some biological mechanism could be used for the same purpose (the above modification of RMHC resembles a sort of "refractory period": if a mutation "fires" it does not fire again for T steps). No doubt there is noise in our biological neurons..., but is our gray matter generating Markov chains? Comments are welcome. Roberto Battiti Giampietro Tecchiolli Dip. di Matematica Univ. di Trento IRST 38050 Povo (Trento) - ITALY 38050 Povo (Trento) - ITALY e-mail: battiti at itnvax.science.unitn.it e-mail: tec at irst.it ***************************************************************************** A preprint with a benchmark of the Reactive Tabu Scheme and other popular techniques (including GAs) is available by ftp from our local archive ("Local Search with Memory: Benchmarking RTS", R. Battiti and G. Tecchiolli) To get a copy: unix> ftp volterra.science.unitn.it (130.186.34.16) Name: anonymous Password: (type your email address) ftp> cd pub/rts-neuro ftp> binary ftp> get rts-benchmark.ps.Z ftp> quit unix>uncompress rts-benchmark.ps.Z unix> lpr rts-benchmark.ps (34 pages) A limited number of hardcopies are available if printing is impossible. Additional background references are abstracted in the file: pub/rts-neuro/ABSTRACTS-TN of the above archive. ***************************************************************************** ftp references cited: [1] "Probabilistic inference using Markov chain Monte Carlo methods" R. M. Neal, posted 23 Nov 1993. ftp: ftp.cs.toronto.edu file: pub/radford/review.ps.Z). [2] " When Will a Genetic Algorithm Outperform Hill Climbing?" Melanie Mitchell, John H. Holland, Stephanie Forrest, posted 23 Jan 1994. ftp: archive.cis.ohio-state.edu file: pub/neuroprose/mitchell.ga-hillclimb.ps.Z). From arantza at cogs.susx.ac.uk Fri Jan 28 13:34:31 1994 From: arantza at cogs.susx.ac.uk (Arantza Etxeberria) Date: Fri, 28 Jan 94 18:34:31 GMT Subject: ECAL95 First Announcement Message-ID: First Announcement 3rd. EUROPEAN CONFERENCE ON ARTIFICIAL LIFE ECAL95 Granada, Spain, 4-6 June, 1995 It is a pleasure to announce the forthcoming 3rd European Conference on Artificial Life (ECAL95). Despite its short life, Artificial Life (AL) is already a mature scientific field. In trying to discover the rules of life and extract its essence so that it can be implemented in different media, AL research has led us to a better understanding of a large set of interesting biology-related problems, such as self organization, emergence, origins of life, self-reproduction, computer viruses, learning, growth and development, animal behavior, ecosystems, autonomous agents, adaptive robotics, etc. The Conference will be organized into Scientific Sessions, Demonstrations, Videos and Comercial Exhibits. Scientific Sessions will consist of Lectures (invited), Oral Presentations, and Posters. The site of ECAL95 will be the city of Granada, located in the south of Spain, in the region of Andalucia. Granada was the last moors site in the Iberic Peninsula, and it has the inheritance of their culture with the legacy of marvelous constructions such as the Alhambra and the Gardens of Generalife. ECAL95 will be organized in collaboration with the International Workshop on Artificial Neural Networks (IWANN95) to be held at Malaga (Costa del Sol, Spain), June 7-9, 1995. These places are only one hour apart by car. Special inscription fees will be offered to those attending both meetings. Scientific Sessions and Topics 1. Foundations and Epistemology: Philosophical Issues. Emergence. Levels of organization. Evolution of Hierarchical Systems. Evolvability. Computation and Dynamics. Ethical Problems. 2. Evolution: Prebiotic Evolution. Origins of Life. Evolution of Metabolism. Fitness Landscapes. Ecosystem Evolution. Biodiversity. Evolution of Sex. Natural Selection and Sexual selection. Units of Selection. 3. Adaptive and Cognitive Systems: Reaction, Neural and Immune Networks. Growth and Differentiation. Self-organization. Pattern Formation. Multicellulars and Development. Natural and Artificial Morphogenesis. 4. Artificial Worlds: Simulation of Adaptive and Cognitive Systems. System-Environment Correlation. Sensor-Effector Coordination. Environment Design. 5. Robotics and Emulation of Animal Behavior: Sensory and Motor Activity. Mobile Agents. Adaptive Robots. Autonomous Robots. Evolutionary Robotics. Ethology. 6. Societies and Collective Behavior: Swarm Intelligence. Cooperation and Communication among Animals and Robots. Evolution of Social Behavior. Social Organizations. Division of Tasks. 7. Applications and Common Tools: Optimization. Problem Solving. Virtual Reality and Computer Graphics. Genetic Algorithms. Neural Networks. Fuzzy Logic. Evolutionary Computation. Genetic Programming. Inscription / Information Those interested please send (mail/fax/e-mail) the Intention Form to the Programme Secretary, Juan J. Merelo, at the following address: Dept. Electronica | Facultad de Ciencias | Phone: +34-58-243162 Campus Fuentenueva | Fax: +34-58-243230 18071 Granada, Spain | E-mail: ecal95 at ugr.es Organization Committee Federico Moran. UCM. Madrid (E) Chair Alvaro Moreno. UPV. San Sebastian (E) Chair Arantza Etxeberria Univ. Sussex (UK) Julio Fernandez. UPV. San Sebastian (E) Francisco Montero. UCM. Madrid (E) Alberto Prieto. UGr. Granada (E) Carme Torras. UPC. Barcelona (E) Programm Committee Francisco Varela. CNRS/CREA. Paris (F) Chair Juan J. Merelo. UGr. Granada (E) Secretary (Definitive list of this Committee will be completed and announced in the forthcoming Call-for-Papers) -------------------------------- cut here -------------------------------- INTENTION FORM 3rd. EUROPEAN CONFERENCE ON ARTIFICIAL LIFE ECAL95 Granada, Spain, 4-6 June, 1995 Family Name: First Name: Institution: Address: Phone No.: Fax No.: e-mail: Signature: Date: From finnoff at predict.com Fri Jan 28 14:21:01 1994 From: finnoff at predict.com (William Finnoff) Date: Fri, 28 Jan 94 12:21:01 MST Subject: Batch Backprop versus Incremental Message-ID: <9401281921.AA28545@predict.com> H Cribbs writes: >Dear Connectionists, > I attended the 5th International Conference on Genetic Algorithms >this summer, and in one of the sessions on combinations of genetic >algorithms (GAs) and Neural Nets(ANNs) a gentleman from the U.K. >suggested that Batch mode learning could possibly be unstable in >the long term for backpropagation. I did not know the gentleman >and when I asked for a reference he could not provide one. >Does anyone have any kind of proof stating that one method is better >than another? Or that possibly batch backprop is unstable in <> >sense? As has been known for quite some time, incremental backprop can be viewed as a type of stochastic algorithm, which in the small step size limit will be essentially equivalent to the batch algorithm. Since the batch version of backprop is a type of gradient descent, there are only two stability issues involved. The first is the possibility that the weights will become infinitely large, which is possible if there is no genuine minimum, (local or global) which can be reached by the training process from a given starting point. An example where this occurs is when the function which one is trying to approximate with the network contains a discontinuity (a step function, say). This sort of behavior is sometimes called "converging to infinity". One should note here, that the error will still always decrease during training (though it won't necessarily converge), unless one encounters problems of "numerical instability". Problems of numerical instability are due to the fact that one is using only a discrete version of a "true" gradient descent. That is, the training algorithm with a constant step size can be viewed as the Euler approximation of the solution to a differential equation. For the solution to this differential equation, the error will always decrease and will either converge to a (perhaps local) minimum of the error function, or to infinity as described above. The error for the training may not always decrease and the training process can "explode" if the step size is choosen too "large". The question of what is "large" depends essentially on the size of the eigenvalues of the matrix of second derivatives of the error function, in particular, the smallest one. If the ratio between the largest and smallest eigenvalue is "large" the differential equation is referred to as being "stiff" or poorly conditioned. The trade off that has to be achieved is between stability (achieved by having a small step size) and speed of convergence (achieved by having a larger step size). It should be noted that the conditioning of the problem will also effect the stability of the incremental version of backprop, since it is also only an approximation of the solution to the same differential equation. The problems of numerical stability can be reduced by using Newton or Quasi-Newton methods (sometimes a problem for neural networks, due to the dimension of typical problems, where hundreds or thousands of parameters may be involved) or by regularization, which modifies the error function to improve the conditioning of the Hessian. The simplest type of conditioning is to simply add a quadratic term in the weights to the error function, i.e., if E(W) is the original error funtion (viewed as a function of the vector of network weights W = (w_i)_{i=1,,,.n}) then add a (penalty) term P_{\lambda}(W) = \lambda \sum_{i=1,...,n} w_i^2, which leads to a "weight decay" term in the training algorithm. This modification of the error function also has the effect of preventing the traing process from converging to infinity and will often improve the generalization ability of the network trained using the modified error function. The disadvantage with this is that one then has another parameter to choose (the weighting factor \lambda) and that the penalty term tends to create additional local minima (particularly around zero) in which one will frequently get stuck while searching for a "good" solution to the minimization problem, which brings us back to the incremental version of backprop. The advantages with using the incremental versions of backprop (in my opinion) have nothing to do with stability issues. First, and most obvious, is that it can be implemented online. Second is the question of efficiency: Since weight updates are made more frequently, the reduction in error can be much faster than with the batch algorithm, although this will depend on the specific nature of the data being used. Finally, due to its stochastic nature, the incremental version of backprop has a "quasi-annealling" character which makes it less likely to get stuck in local minima than the batch training process; (this statement can be made fairly rigorous, consult the references given below). References: 1) Battiti, R. (1992). First- and second order methods for learning: Between steepest descent and Newton's method, {\it Neural Computation} 4, 141-166. 2) Benveniste, A., M\'etivier, M. and Priouret, P., { \it Adaptive Algorithms and Stochastic Approximations}, Springer Verlag (1987). 3) Bouton C., Approximation Gaussienne d'algorithmes stochastiques a dynamique Markovienne. Thesis, Paris VI, (in French), (1985). 4) Darken C. and Moody J., Note on learning rate schedules for stochastic optimization, in{\it Advances in Neural Information Processing Systems 3}, Lippmann, R. Moody, J., and Touretzky, D., ed., Morgan Kaufmann, San Mateo, (1991). 5) Finnoff, W., Asymptotics for the constant step size backpropagation algorithm and resistance to local minima, {\it Neural Computation}, 6, pp. 285-293, 1994. 6) Finnoff, W., The dynamics of generalization for first and second order parameter adaptation methods, submitted to {\it IEEE Trans.~on Information Theory}. 7) Hornik, K. and Kuan, C.M., Convergence of learning algorithms with constant learning rates, {\it IEEE Trans. on Neural Networks} 2, pp. 484-489, (1991). 8) Krasovskii, N., {\it The Stability of Motion}, Stanford University Press, Stanford, (1963). 9) Le Cun, Y., Kanter I. and Solla, S. (1990). Second order properties of error surfaces: Learning time and generalization, In R. Lippman, J. Moody and D. Touretzy (Eds.), {\it Advances in Neural Information Processing III} (pp.918-924). San Mateo: Morgan Kaufman. 10) White, H. 1989. Learning in artificial neural networks: A statistical perspective, {\it Neural Computation} 1: 425-464. 11) White, H., Some asymptotic results for learning in single hidden-layer feedforward network models, Jour. Amer. Stat. Ass. 84, no. 408, pp. 1003-1013, (1989)I. Cheers William %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% William Finnoff Prediction Co. 320 Aztec St., Suite B Santa Fe, NM, 87501, USA Tel.: (505)-984-3123 Fax: (505)-983-0571 e-mail: finnoff at predict.com From yves at netid.com Fri Jan 28 14:43:02 1994 From: yves at netid.com (Yves Chauvin) Date: Fri, 28 Jan 94 11:43:02 PST Subject: HMMs and Molecular Biology Message-ID: <9401281943.AA00678@netid.com> **DO NOT FORWARD TO OTHER GROUPS** The following papers, "Hidden Markov Models of Biological Primary Sequence Information", to be published in the Proceedings of the National Academy of Sciences (USA), vol. 91, February 94. and "Hidden Markov Models for Human Genes", to be published in the Proceedings of the 1993 NIPS conference, vol. 6. have been placed on ftp site. Further information and retrieval instructions are given below. Yves Chauvin yves at netid.com ___________________________________________________________________________ Hidden Markov Models of Biological Primary Sequence Information Pierre Baldi Jet Propulsion Laboratory and Division of Biology, California Institute of Technology Pasadena, CA 91109 Yves Chauvin Net-ID, Inc. Tim Hunkapiller University of Washington Marcella A. McClure University of Nevada Hidden Markov Model (HMM) techniques are used to model families of biological sequences. A smooth and convergent algorithm is introduced to iteratively adapt the transition and emission parameters of the models from the examples in a given family. The HMM approach is applied to three protein families: globins, immunoglobulins and kinases. In all cases, the models derived capture the important statistical characteristic of the family and can be used for a number of tasks including: multiple alignments, motif detection and classification. For $K$ sequences of average length $N$, this approach yields an effective multiple alignment algorithm which requires $O(KN^2)$ operations, linear in the number of sequences. ___________________________________________________________________________ Hidden Markov Models for Human Genes Pierre Baldi Jet Propulsion Laboratory and Division of Biology, California Institute of Technology Pasadena, CA 91109 Soren Brunak The Technical University of Denmark Yves Chauvin Net-ID, Inc. Jacob Engelbrecht The Technical University of Denmark Anders Krogh The Technical University of Denmark Human genes are not continuous but rather consist of short coding regions (exons) interspersed with highly variable non-coding regions (introns). We apply HMMs to the problem of modeling exons, introns and detecting splice sites in the human genome. Our most interesting result so far is the detection of particular oscillatory patterns, with a minimal period of roughly 10 nucleotides, that seem to be characteristic of exon regions and may have significant biological implications. ___________________________________________________________________________ Retrieval instructions: The papers are "baldi.bioprimseq.ps.z" and "baldi.humgenes.ps.z". To retrieve these files: % ftp netcom.com Connected to netcom.com. 220 netcom FTP server (Version 2.0WU(10) [...] ready. Name (netcom.com:yourname): anonymous 331 Guest login ok, send your complete e-mail address as password. Password: .. ftp> cd pub/netid/papers ftp> ls ftp> binary ftp> get ftp> close .. % gunzip From hilario at cui.unige.ch Sat Jan 29 10:48:03 1994 From: hilario at cui.unige.ch (Hilario Melanie) Date: Sat, 29 Jan 1994 16:48:03 +0100 Subject: CFP: ECAI-94 WS on Combining Symbolic and Connectionist Processing Message-ID: <165*/S=hilario/OU=cui/O=unige/PRMD=switch/ADMD=arcom/C=ch/@MHS> Dear People, Please distribute the following CFP via the connectionists mailing list. Many thanks, Melanie Hilario ---------------------------------------------------------------------------- Call for Papers COMBINING SYMBOLIC AND CONNECTIONIST PROCESSING Workshop held in conjunction with ECAI-94 August 9, 1994 - Amsterdam, The Netherlands Until a few years ago, the history of AI has been marked by two parallel, often antagonistic streams of development -- classical or symbolic AI and connectionist processing. A recent research trend, premissed on the complementarity of these two paradigms, strives to build hybrid systems which combine the advantages of both to overcome the limitations of each. For instance, attempts have been made to accomplish complex tasks by blending neural networks with rule-based or case-based reasoning. This workshop will be the first Europe-wide effort to bring together researchers active in the area in view of laying the groundwork for a theory and methodology of symbolic/connectionist integration (SCI). The workshop will focus on the following topics: o theoretical (cognitive and computational) foundations of SCI o techniques and mechanisms for combining symbolic and neural processing methods (e.g. ways of improving and going beyond state-of-the-art rule compilation and extraction techniques) o outstanding problems encountered and issues involved in SCI (e.g. Which symbolic or connectionist representation schemes are best adapted to SCI? The vector space used in neural nets and the symbolic space have fundamental mathematical differences; how will these differences impact SCI? Do we have the conceptual tools needed to cope with this representation problem?) o profiles of application domains in which SCI has been (or can be) shown to perform better than traditional approaches o description, analysis and comparison of implemented symbolic/connectionist systems SUBMISSION REQUIREMENTS Prospective participants should submit an extended abstract to the contact person below, either via email in postscript format or via regular mail, in which case 3 copies are required. Each submission should include a separate information page containing the title of the paper, author names and affiliations, and the complete address (including telephone, fax and email) of the first author. The paper itself should not exceed 12 pages. Submission deadline is April 1, 1994. Each paper will be reviewed by at least two members of the Program Committee. Notification of acceptance or rejection will be sent to first authors by May 1, 1994. Camera-ready copies of accepted papers are due on June 1st and will be reproduced for distribution at the workshop. Those who wish to participate without presenting a paper should send a request describing their research interests and/or previous work in the field of SCI. Since attendance will be limited to ensure effective interaction, these requests will be considered after screening of submitted papers. All workshop participants are required to register for the main conference. PROGRAM COMMITTEE Bernard Amy (LIFIA-IMAG, Grenoble, France) Patrick Gallinari (LAFORIA, University of Paris 6, France) Franz Kurfess (Dept. Neural Information Processing, University of Ulm, Germany) Christian Pellegrini (CUI, University of Geneva, Switzerland) Alessandro Sperduti (CSD, University of Pisa, Italy) IMPORTANT DATES Submission deadline April 1, 1994 Notification of acceptance/rejection May 1, 1994 Final papers due June 1, 1994 Date of the workshop August 9, 1994 CONTACT PERSON Melanie Hilario CUI - University of Geneva 24 rue General Dufour CH-1211 Geneva 4 Voice: +41 22/705 7791 Fax: +41 22/320 2927 Email: hilario at cui.unige.ch From N.Sharkey at dcs.shef.ac.uk Sat Jan 29 08:08:29 1994 From: N.Sharkey at dcs.shef.ac.uk (N.Sharkey@dcs.shef.ac.uk) Date: Sat, 29 Jan 94 13:08:29 GMT Subject: IEE Colloquia Message-ID: <9401291308.AA10480@entropy.dcs.shef.ac.uk> IEE COLLOQUIUM IN LONDON, UK SYMBOLIC AND NEURAL COGNITIVE ENGINEERING Colloquium organised by Professional Group C4 (Artificial intelligence) of the Institute of Electical Engineers (IEE) to be held at Savoy Place on Monday, 14 February 1994 PROVISIONAL PROGRAMME 9.30 am Registration and coffee Chairman: Professor I Aleksander (Imperial College) 10.00 Chairman's introduction 1 10.10 A review of cognitive symbolic engineering: Professor B Richards (Imperial College) 2 10.40 The interplay of symbolic and adaptive techniques: R Garigliano and D J Nettleton (University of Durham) 3 11.10 Engineering cognitive systems - some conceptual issues: R Paton (University of Liverpool) 4 11.40 Variable binding in a neural network using a distributed representation: A Browne and J Pilkington (South Bank University) 12.10 LUNCH 5 1.30 Connectionist advances in natural language processing: Professor N Sharkey (University of Sheffield) 6 2.00 Systematicity and generalisation in connectionist models: L F Niklasson (University of Exeter) 7 2.30 Hierarchical symbolic structures and knowledge chunking: B K Purhoit (BT Laboratories) and J F Boyce (King's College London) 3.00 TEA 8 3.15 Relational computing: Professor J Taylor (King's College London) 9 3.45 The research challenge for symbolic and neural approaches: Professor I Aleksander (Imperial College) 4.15 Discussion 4.45 CLOSE The IEE is not, as a body, responsible for the views or opinions expressed by individual authors or speakers. 151/36/38 ref: 94/038 MB OTHER EVENTS ORGANISED BY THE COMPUTING AND CONTROL DIVISION TO BE HELD FEBRUARY 1994 1 Tue Colloquium on High performance applications of parallel PG C2 architectures 7 Mon Colloquium on Intelligent front-ends for existing systems PG C4 10 Thur 11TH COMPUTING AND CONTROL LECTURE Out of control into systems engineering research? By Professor C J Harris (University of Southampton) 21 Mon Colloquium on Modelling of controlled natural energy PG C6 systems 23 Wed Colloquium on Vehicle diagnostics in Europe PG C12 24 Thur Colloquium on Implications of the new legislation on PG C5 work with display screen equipment 28 Mon Colloquium on Molecular bioinformatics PG C4 Joint meeting with PAPAGENA Further details of the above events can be obtained from the Secretary, LS(D)CA, IEE, Savoy Place, London WC2R 0BL or by telephoning 071 240 1871 Ext: 2206 From srx014 at cck.coventry.ac.uk Mon Jan 31 10:59:00 1994 From: srx014 at cck.coventry.ac.uk (CRReeves) Date: Mon, 31 Jan 94 10:59:00 WET Subject: Batch Backprop versus Incremental (fwd) Message-ID: <15509.9401311059@cck.coventry.ac.uk> On 27th January, Brown Cribbs wrote: > > Dear Connectionists, > I attended the 5th International Conference on Genetic Algorithms > this summer, and in one of the sessions on combinations of genetic > algorithms (GAs) and Neural Nets(ANNs) a gentleman from the U.K. > suggested that Batch mode learning could possibly be unstable in > the long term for backpropagation. I did not know the gentleman > and when I asked for a reference he could not provide one. > > Does anyone have any kind of proof stating that one method is better > than another? Or that possibly batch backprop is unstable in <> > sense? > I thought a contribution from the UK was necessary, particularly in view of Scott Fahlman's later, somewhat provocative, remarks! I've seen a preprint of a paper by Steve Ellacott (University of Brighton) in which he considers just this problem. This may have been the paper referred to in the original question. He considers the question of {\em numerical\/} stability of batch and case-by-case training, and shows that in this sense there are conditions under which the delta rule is unstable for batch updates. He then proceeds to look at the generalized delta rule, with similar results at least in the neighbourhood of a local minimum. Of course, the choice of learning rate will affect the conclusions in a particular case. I understand these strange ideas have been expanded into a chapter (called "The Numerical Analysis Approach") of a book just published (end '93): Mathematical Approaches to Neural Networks (J.G.Taylor - Ed.) Elsevier Science Publishers ISBN 0-444-81692-5 -- ___________________________________________ | Colin Reeves | | Division of Statistics and OR | | School of Mathematical and Information | | Sciences | | Coventry University | | Priory St | | Coventry CV1 5FB | | tel :+44 (0)203 838979 | | fax :+44 (0)203 838585 | | email: CRReeves at uk.ac.cov.cck | | (alternative email: srx014 at cck.cov.ac.uk) | |___________________________________________| From presnik at caesar.East.Sun.COM Mon Jan 31 00:30:07 1994 From: presnik at caesar.East.Sun.COM (Philip Resnik - Sun Microsystems Labs BOS) Date: Mon, 31 Jan 1994 10:30:07 +0500 Subject: CFP: Combining symbolic and statistical approaches to language Message-ID: <9401311530.AA10552@caesar.East.Sun.COM> Hello, Although the following workshop does not specifically concern connectionist approaches, one of our goals is to initiate new discussions with the wide variety of researchers who have been thinking about similar issues. That includes many members of this list, and I would like to encourage connectionist researchers to participate. Philip ---------------------------------------------------------------- ***** CALL FOR PAPERS ****** THE BALANCING ACT: Combining Symbolic and Statistical Approaches to Language 1 July 1994 New Mexico State University Las Cruces, New Mexico, USA A workshop in conjunction with the 32nd Annual Meeting of the Association for Computational Linguistics (27-30 June 1994) A renaissance of interest in corpus-based statistical methods has rekindled old controversies -- rationalist vs. empiricist philosophies, theory-driven vs. data-driven methodologies, symbolic vs. statistical techniques. The aim of this workshop is to set aside a priori biases and explore the balancing act that must take place when symbolic and statistical approaches are brought together. We plan to accept papers from authors having a wide range of perspectives, and to initiate a discussion that includes philosophical, theoretical, and practical issues. Submissions to the workshop must describe research in which both symbolic and statistical methods play a part. All research of this kind requires that the researcher make choices: What knowledge will be represented symbolically and how will it be obtained? What assumptions underlie the statistical model? What is the researcher gaining by combining approaches? Questions like these, and the metaphor of the balancing act, will provide a unifying theme to draw contributions from a wide spectrum of language researchers. ORGANIZERS: Judith Klavans, Columbia Univerisity Philip Resnik, Sun Microsystems Laboratories, Inc. REQUIREMENTS: Papers should describe original work; they should clearly emphasize the type of paper to be presented (e.g. implementation, philosophical, etc.) and the state of completion of the research. A paper accepted for presentation cannot be presented or have been presented at any other meeting. In addition to the workshop proceedings, plans for publication as a book require that papers not have been published in any other publicly available proceedings. Papers submitted to other conferences will be considered, as long as this fact is clearly indicated in the submission. FORMAT FOR SUBMISSION: Following guidelines for the ACL meeting, authors should submit preliminary versions of their papers, not to exceed 3200 words (exclusive of references). Papers outside the specified length and formatting requirements are subject to rejection without review. Papers should be headed by a title page containing the paper title, a short (5 line) summary and a specification of the subject area(s). If the author wishes reviewing to be blind, a separate page with author identification information must be submitted. SUBMISSION MEDIA: Papers may be submitted electronically or in hard copy to either organizer at the addresses given below. Electronic submissions should be either self-contained LaTeX source or plain text. LaTeX submissions must use the ACL submission style (aclsub.sty) retrievable from the ACL LISTSERV server (access to which is described below) and should not refer to any external files or styles except for the standard styles for TeX 3.14 and LaTeX 2.09. A model submission modelsub.tex is also provided in the archive, as well as a bibliography style acl.bst. Note that the bibliography for a submission cannot be submitted as separate .bib file; the actual bibliography entries must be inserted in the submitted LaTeX source file. Be sure that e-mail submissions have no lines longer than 80 characters to avoid mailer problems. Hard copy submissions should consist of four (4) copies of the paper. A plain text version of the identification page should be sent separately by electronic mail if possible, giving the following information: title, author(s), address(es), abstract, content areas, word count. Schedule: Papers must be received by 15 March 1994. Late papers will not be considered. Notification of receipt will be mailed to the first author (or designated author) soon after receipt. Authors will be notified of acceptance by 10 April 1994. Camera-ready copies of final papers prepared in a double-column format, preferably using a laser printer, must be received by 10 May 1994, along with a signed copyright release statement. The ACL LaTeX proceedings format is available through the ACL LISTSERV. REGISTRATION: Registration fees are $25 for participants who register by 15 May 1994. Late registrations will be $30. Registration includes a copy of the proceedings, lunch, and refreshments during the day. Payment in US$ checks payable to ACL or credit card payment (Visa/Mastercard) can be sent to Philip Resnik at the address below. Please submit the following information along with payment: name affiliation postal address email method of payment (check or credit card) credit card info (name, card number, expiration date) dietary requirements (vegetarian, kosher, etc) ACL INFORMATION: For other information on the ACL conference which precedes the workshop and on the ACL more generally, please use the ACL LISTSERV, described below. ACL LISTSERV: Listserv is a facility to allow access to an electronic document archive by electronic mail. The ACL LISTSERV has been set up at Columbia University's Department of Computer Science. Requests from the archive should be sent as e-mail messages to listserv at cs.columbia.edu with an empty subject field and the message body containing the request command. The most useful requests are "help" for general help on using LISTSERV, "index acl-l" for the current contents of the ACL archive and "get acl-l " to get a particular file named >from the archive. For example, to get an ACL membership form, a message with the following body should be sent: get acl-l membership-form.txt Answers to requests are returned by e-mail. Since the server may have many requests for different archives to process, requests are queued up and may take a while (say, overnight) to be fulfilled. The ACL archive can also be accessed by anonymous FTP. Here is an example of how to get the same file by FTP (user typein is underlined): $ ftp cs.columbia.edu ------------------- Name (cs.columbia.edu:pereira): anonymous --------- Password:pereira at research.att.com << not echoed ------------------------ ftp> cd acl-l -------- ftp> get membership-form.txt.Z ------------------------- ftp> quit ---- $ uncompress membership-form.txt.Z -------------------------------- This file is listed under acl-l/ACL94/Workshop_balancing_act.ascii.Z. SPONSORSHIP: This workshop is sponsored by the Association for Computational Linguistics (ACL). It is organized by: Judith L. Klavans Philip Resnik Columbia University Sun Microsystems Laboratories, Inc. Department of Computer Science Mailstop UCHL03-207 500 W 120th Street Two Elizabeth Drive New York, NY 10027, USA Chelmsford, MA 01824-4195 USA klavans at cs.columbia.edu philip.resnik at east.sun.com Phone: (212) 939-7120 Phone: (508) 442-0841 Fax: (914) 478-1802 Fax: (508) 250-5067 [94-01-27] From harnad at Princeton.EDU Mon Jan 31 20:52:35 1994 From: harnad at Princeton.EDU (Stevan Harnad) Date: Mon, 31 Jan 94 20:52:35 EST Subject: Human Memory: BBS Call for Commentators Message-ID: <9402010152.AA22714@clarity.Princeton.EDU> Below is the abstract of a forthcoming target article by: MS Humphreys, J Wiles & S Dennis on: TOWARD A THEORY OF HUMAN MEMORY: DATA STRUCTURES AND ACCESS PROCESSES This article has been accepted for publication in Behavioral and Brain Sciences (BBS), an international, interdisciplinary journal providing Open Peer Commentary on important and controversial current research in the biobehavioral and cognitive sciences. Commentators must be current BBS Associates or nominated by a current BBS Associate. To be considered as a commentator for this article, to suggest other appropriate commentators, or for information about how to become a BBS Associate, please send email to: harnad at clarity.princeton.edu or harnad at pucc.bitnet or write to: BBS, 20 Nassau Street, #240, Princeton NJ 08542 [tel: 609-921-7771] To help us put together a balanced list of commentators, please give some indication of the aspects of the topic on which you would bring your areas of expertise to bear if you were selected as a commentator. An electronic draft of the full text is available for inspection by anonymous ftp according to the instructions that follow after the abstract. ____________________________________________________________________ TOWARD A THEORY OF HUMAN MEMORY: DATA STRUCTURES AND ACCESS PROCESSES Michael S. Humphreys, Department of Psychology Janet Wiles, Departments of Psychology and Computer Science Simon Dennis, Department of Computer Science University of Queensland QLD 4072 Australia mh at psych.psy.uq.oz.au KEYWORDS: amnesia, binding, context, data structure, lexical decision, memory access, perceptual identification, recall, recognition, representation. ABSTRACT: A theory of the data structures and access processes of human memory is proposed and demonstrated on 10 tasks. The two starting points are Marr's (1982) ideas about the levels at which we can understand an information processing device and the standard laboratory paradigms which demonstrate the power and complexity of human memory. The theory suggests how to capture the functional characteristics of human memory (e.g., analogies, reasoning, etc.) without having to be concerned with implementational details. Ours is not a performance theory. We specify what is computed by the memory system with a multidimensional task classification which encompasses existing classifications (e.g., the distinction between implicit and explicit, data driven and conceptually driven, and simple associative (2-way bindings) and higher order tasks (3-way bindings). This provides a broad basis for new experimentation. Our formal language clarifies the binding problem in episodic memory, the role of input pathways in both episodic and semantic (lexical) memory, the importance of the input set in episodic memory, and the ubiquitous calculation of an intersection in theories of episodic and lexical access. -------------------------------------------------------------- To help you decide whether you would be an appropriate commentator for this article, an electronic draft is retrievable by anonymous ftp from princeton.edu according to the instructions below (the filename is bbs.humphreys). Please do not prepare a commentary on this draft. Just let us know, after having inspected it, what relevant expertise you feel you would bring to bear on what aspect of the article. The file is also retrievable using archie, gopher, veronica, etc. ------------------------------------------------------------- To retrieve a file by ftp from an Internet site, type either: ftp princeton.edu or ftp 128.112.128.1 When you are asked for your login, type: anonymous Enter password as queried (your password is your actual userid: yourlogin at yourhost.whatever.whatever - be sure to include the "@") cd /pub/harnad/BBS To show the available files, type: ls Next, retrieve the file you want with (for example): get bbs.humphreys When you have the file(s) you want, type: quit These files can also be retrieved using gopher, archie, veronica, etc. ---------- Where the above procedure is not available there are two fileservers: ftpmail at decwrl.dec.com and bitftp at pucc.bitnet that will do the transfer for you. To one or the other of them, send the following one line message: help for instructions (which will be similar to the above, but will be in the form of a series of lines in an email message that ftpmail or bitftp will then execute for you). JANET users without ftp can instead utilise the file transfer facilities at sites uk.ac.ft-relay or uk.ac.nsf.sun. Full details are available on request. ------------------------------------------------------------- From Connectionists-Request at cs.cmu.edu Sat Jan 1 00:05:12 1994 From: Connectionists-Request at cs.cmu.edu (Connectionists-Request@cs.cmu.edu) Date: Sat, 01 Jan 94 00:05:12 EST Subject: Bi-monthly Reminder Message-ID: <16894.757400712@B.GP.CS.CMU.EDU> *** DO NOT FORWARD TO ANY OTHER LISTS *** This note was last updated January 4, 1993. This is an automatically posted bi-monthly reminder about how the CONNECTIONISTS list works and how to access various online resources. CONNECTIONISTS is not an edited forum like the Neuron Digest, or a free-for-all newsgroup like comp.ai.neural-nets. It's somewhere in between, relying on the self-restraint of its subscribers. Membership in CONNECTIONISTS is restricted to persons actively involved in neural net research. The following posting guidelines are designed to reduce the amount of irrelevant messages sent to the list. Before you post, please remember that this list is distributed to over a thousand busy people who don't want their time wasted on trivia. Also, many subscribers pay cash for each kbyte; they shouldn't be forced to pay for junk mail. Happy hacking. -- Dave Touretzky & David Redish --------------------------------------------------------------------- What to post to CONNECTIONISTS ------------------------------ - The list is primarily intended to support the discussion of technical issues relating to neural computation. - We encourage people to post the abstracts of their latest papers and tech reports. - Conferences and workshops may be announced on this list AT MOST twice: once to send out a call for papers, and once to remind non-authors about the registration deadline. A flood of repetitive announcements about the same conference is not welcome here. - Requests for ADDITIONAL references. This has been a particularly sensitive subject lately. Please try to (a) demonstrate that you have already pursued the quick, obvious routes to finding the information you desire, and (b) give people something back in return for bothering them. The easiest way to do both these things is to FIRST do the library work to find the basic references, then POST these as part of your query. Here's an example: WRONG WAY: "Can someone please mail me all references to cascade correlation?" RIGHT WAY: "I'm looking for references to work on cascade correlation. I've already read Fahlman's paper in NIPS 2, his NIPS 3 abstract, and found the code in the nn-bench archive. Is anyone aware of additional work with this algorithm? I'll summarize and post results to the list." - Announcements of job openings related to neural computation. - Short reviews of new text books related to neural computation. To send mail to everyone on the list, address it to Connectionists at CS.CMU.EDU ------------------------------------------------------------------- What NOT to post to CONNECTIONISTS: ----------------------------------- - Requests for addition to the list, change of address and other administrative matters should be sent to: "Connectionists-Request at cs.cmu.edu" (note the exact spelling: many "connectionists", one "request"). If you mention our mailing list to someone who may apply to be added to it, please make sure they use the above and NOT "Connectionists at cs.cmu.edu". - Requests for e-mail addresses of people who are believed to subscribe to CONNECTIONISTS should be sent to postmaster at appropriate-site. If the site address is unknown, send your request to Connectionists-Request at cs.cmu.edu and we'll do our best to help. A phone call to the appropriate institution may sometimes be simpler and faster. - Note that in many mail programs a reply to a message is automatically "CC"-ed to all the addresses on the "To" and "CC" lines of the original message. If the mailer you use has this property, please make sure your personal response (request for a Tech Report etc.) is NOT broadcast over the net. - Do NOT tell a friend about Connectionists at cs.cmu.edu. Tell him or her only about Connectionists-Request at cs.cmu.edu. This will save your friend from public embarrassment if she/he tries to subscribe. ------------------------------------------------------------------------------- The CONNECTIONISTS Archive: --------------------------- All e-mail messages sent to "Connectionists at cs.cmu.edu" starting 27-Feb-88 are now available for public perusal. A separate file exists for each month. The files' names are: arch.yymm where yymm stand for the obvious thing. Thus the earliest available data are in the file: arch.8802 Files ending with .Z are compressed using the standard unix compress program. To browse through these files (as well as through other files, see below) you must FTP them to your local machine. ------------------------------------------------------------------------------- How to FTP Files from the CONNECTIONISTS Archive ------------------------------------------------ 1. Open an FTP connection to host B.GP.CS.CMU.EDU (Internet address 128.2.242.8). 2. Login as user anonymous with password your username. 3. 'cd' directly to one of the following directories: /usr/connect/connectionists/archives /usr/connect/connectionists/bibliographies 4. The archives and bibliographies directories are the ONLY ones you can access. You can't even find out whether any other directories exist. If you are using the 'cd' command you must cd DIRECTLY into one of these two directories. Access will be denied to any others, including their parent directory. 5. The archives subdirectory contains back issues of the mailing list. Some bibliographies are in the bibliographies subdirectory. Problems? - contact us at "Connectionists-Request at cs.cmu.edu". Anonymous FTP on archive.cis.ohio-state.edu (128.146.8.52) pub/neuroprose directory This directory contains technical reports as a public service to the connectionist and neural network scientific community which has an organized mailing list (for info: connectionists-request at cs.cmu.edu) Researchers may place electronic versions of their preprints in this directory, announce availability, and other interested researchers can rapidly retrieve and print the postscripts. This saves copying, postage and handling, by having the interested reader supply the paper. We strongly discourage the merger into the repository of existing bodies of work or the use of this medium as a vanity press for papers which are not of publication quality. PLACING A FILE To place a file, put it in the Inbox subdirectory, and send mail to pollack at cis.ohio-state.edu. Within a couple of days, I will move and protect it, and suggest a different name if necessary. Current naming convention is author.title.filetype.Z where title is just enough to discriminate among the files of the same author. The filetype is usually "ps" for postscript, our desired universal printing format, but may be tex, which requires more local software than a spooler. The Z indicates that the file has been compressed by the standard unix "compress" utility, which results in the .Z affix. To place or retrieve .Z files, make sure to issue the FTP command "BINARY" before transfering files. After retrieval, call the standard unix "uncompress" utility, which removes the .Z affix. An example of placing a file is in the appendix. Make sure your paper is single-spaced, so as to save paper, and include an INDEX Entry, consisting of 1) the filename, 2) the email contact for problems, 3) the number of pages and 4) a one sentence description. See the INDEX file for examples. ANNOUNCING YOUR PAPER It is the author's responsibility to invite other researchers to make copies of their paper. Before announcing, have a friend at another institution retrieve and print the file, so as to avoid easily found local postscript library errors. And let the community know how many pages to expect on their printer. Finally, information about where the paper will/might appear is appropriate inside the paper as well as in the announcement. Please add two lines to your mail header, or the top of your message, so as to facilitate the development of mailer scripts and macros which can automatically retrieve files from both NEUROPROSE and other lab-specific repositories: FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/filename.ps.Z When you announce a paper, you should consider whether (A) you want it automatically forwarded to other groups, like NEURON-DIGEST, (which gets posted to comp.ai.neural-networks) and if you want to provide (B) free or (C) prepaid hard copies for those unable to use FTP. To prevent forwarding, place a "**DO NOT FORWARD TO OTHER GROUPS**" at the top of your file. If you do offer hard copies, be prepared for a high cost. One author reported that when they allowed combination AB, the rattling around of their "free paper offer" on the worldwide data net generated over 2000 hardcopy requests! A shell script called Getps, written by Tony Plate, is in the directory, and can perform the necessary retrieval operations, given the file name. Functions for GNU Emacs RMAIL, and other mailing systems will also be posted as debugged and available. At any time, for any reason, the author may request their paper be updated or removed. For further questions contact: Jordan Pollack Assistant Professor CIS Dept/OSU Laboratory for AI Research 2036 Neil Ave Email: pollack at cis.ohio-state.edu Columbus, OH 43210 Phone: (614) 292-4890 APPENDIX: Here is an example of naming and placing a file: gvax> cp i-was-right.txt.ps rosenblatt.reborn.ps gvax> compress rosenblatt.reborn.ps gvax> ftp cheops.cis.ohio-state.edu Connected to cheops.cis.ohio-state.edu. 220 cheops.cis.ohio-state.edu FTP server ready. Name: anonymous 331 Guest login ok, send ident as password. Password:neuron 230 Guest login ok, access restrictions apply. ftp> binary 200 Type set to I. ftp> cd pub/neuroprose/Inbox 250 CWD command successful. ftp> put rosenblatt.reborn.ps.Z 200 PORT command successful. 150 Opening BINARY mode data connection for rosenblatt.reborn.ps.Z 226 Transfer complete. 100000 bytes sent in 3.14159 seconds ftp> quit 221 Goodbye. gvax> mail pollack at cis.ohio-state.edu Subject: file in Inbox. Jordan, I just placed the file rosenblatt.reborn.ps.Z in the Inbox. Here is the INDEX entry: rosenblatt.reborn.ps.Z rosenblatt at gvax.cs.cornell.edu 17 pages. Boastful statements by the deceased leader of the neurocomputing field. Let me know when it is in place so I can announce it to Connectionists at cmu. Frank ^D AFTER FRANK RECEIVES THE GO-AHEAD, AND HAS A FRIEND TEST RETRIEVE THE FILE, HE DOES THE FOLLOWING: gvax> mail connectionists Subject: TR announcement: Born Again Perceptrons FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/rosenblatt.reborn.ps.Z The file rosenblatt.reborn.ps.Z is now available for copying from the Neuroprose repository: Born Again Perceptrons (17 pages) Frank Rosenblatt Cornell University ABSTRACT: In this unpublished paper, I review the historical facts regarding my death at sea: Was it an accident or suicide? Moreover, I look over the past 23 years of work and find that I was right in my initial overblown assessments of the field of neural networks. ~r.signature ^D ------------------------------------------------------------------------ How to FTP Files from the NN-Bench Collection --------------------------------------------- 1. Create an FTP connection from wherever you are to machine "pt.cs.cmu.edu" (128.2.254.155). 2. Log in as user "anonymous" with password your username. 3. Change remote directory to "/afs/cs/project/connect/bench". Any subdirectories of this one should also be accessible. Parent directories should not be. 4. At this point FTP should be able to get a listing of files in this directory and fetch the ones you want. Problems? - contact us at "nn-bench-request at cs.cmu.edu". From terry at salk.edu Mon Jan 3 19:57:12 1994 From: terry at salk.edu (Terry Sejnowski) Date: Mon, 3 Jan 94 16:57:12 PST Subject: NEURAL COMPUTATION 6:1 Message-ID: <9401040057.AA21592@salk.edu> Neural Computation -- January, 1994 -- Volume 6 Number 1 Article: Cortical Map Reorganization as a Competitve Process Granger G. Sutton III, James A. Reggia, Steven L. Armentrout, and C. Lynne D'Autrechy Note: An Efficient Method for Computing Synaptic Conductances Based on a Kinetic Model of Receptor Binding A. Destexhe, Z. F. Mainen and T. J. Sejnowski Letters: A Neural Network for Coding of Trajectories by Time Series of Neuronal Population Vectors Apostolos P. Georgopoulos and Alexander V. Lukashin Theoretical Considerations for the Analysis of Population Coding in Motor Cortex Terence D. Sanger Neural Network Model of the Cerebellum: Temporal Discrimination and the Timing of Motor Responses Dean V. Buonomano and Michael D. Mauk Computational Aspects of the Respiratory Pattern Generator Allan Gottschalk, Malcolm D. Ogilvie Diethelm W. Richter and Allan I. Pack Subharmonic Coordination in Networks of Neurons with Slow Conductances Thomas LoFaro, Nancy Kopell, Eve Marder and Scott L. Hooper Setting the Activity Level in Sparse Random Networks Ali A. Minai and William B. Levy The Role of Constraints in Hebbian Learning Kenneth D. Miller and David J. C. MacKay Towards a Theory of the Striate Cortex Zhaoping Li and Joseph J. Atick Fast Exact Multiplication by the Hessian Barak A. Pearlmutter Polyhedral Combinatorics and Neural Networks Andrew H. Gee and Richard W. Prager From linster at neurones.espci.fr Mon Jan 3 10:12:03 1994 From: linster at neurones.espci.fr (Christiane LINSTER) Date: Mon, 3 Jan 94 10:12:03 MET Subject: Please distribute widely (fwd) Message-ID: <9401030918.AA24787@Hugo.espci.fr> > > > First Announcement and Call for Papers > > THE FIRST EUROPEAN CONFERENCE ON > "COGNITIVE SCIENCE IN INDUSTRY" > > 28th - 30th September 1994 - Luxembourg > > > > OBJETCIVES > > The study of human operational strategies and human processes is a particularly > well developed area of Cognitive Science. This particular domain belongs to the > larger field of Information Science, which considers semantic and qualitative > aspects of information, rather than digital and qualitative ones. This > represents an important issue for industries concerned with human beings in a > work place context because Cognitive Science provides a great number of > additional dimensions allowing to conceive more complete systems. > > It is the purpose of the European Conference on Cognitive Science to assemble > theoreticians and practitioners from industry and academic institutions to > discuss questions related to the practical application of Cognitive Science in > industrial settings. The conference will be a forum for those who recognize the > need to develop theories, methods, and systems to ensure the transfer of > laboratory results into real-life problem situations. Furthermore, the European > Conference on Cognitive Science will facilitate the exchange of expertise among > practitioners. > > The organizing committee plans to hold the conference in collaboration with the > following institutions: > > GDR-CNRS 957 Sciences Cognitives de Paris, France > Ecole Nationale Superieure de Telecommunication de Bretagne, France > Universite Paris Sorbonne, France > Centre de Recherche Public - CU, Luxembourg > Centre Universitaire, Luxembourg > DFKI, Germany > Universitat Kaiserslautern, Germany > Universitat Freiburg, Germany > > > CALL FOR PAPERS > > Two types of papers are solicited: those discussing on cognitive theories in > the context of industrial practice and those presenting industrial applications. > Suitable conferences and panel discussion themes include: > > o Cognitive robotics > o Man-Machine cooperation > o Cognitive organization and cooperation > o Intelligent assistance for decision > o Knowledge engineering, acquisition and modeling > o Distributed cognition and multi agents system > o Perception, recognition, interpretation and action > o Planning > o Intelligent management of multimedia documents > o Piloting of important projects > > > Applications, ongoing developments and relevant experimentations > in the areas of: > o Transports > o Bank > o Assurances > o Health > o Telecommunications > o Environment protection > o Production > o Supervision and control > > > SUBMISSION DETAILS > > Two classes of submission are solicited - long papers, which should not exceed > 30 pages, and short papers, which should be up to 10 pages. Reports of > industrial applications should ideally be presented as short papers. The program > committee is seeking for original papers in all areas concerning the use of > cognitive science for industrial problem solving. Papers could either describe > industrial cases, significant results from ongoing research or user experience, > or offer a critical analysis of current theories, tools or techniques in this > domain. > > > IMPORTANT DATES > > Papers should be submitted by the 7th March 1994. Notification of acceptance > will be by the 30th May 1994. The final version of accepted papers must be > received by the 4th July. > > > GENERAL CHAIRMAN > > Jean-Pierre Barthelemy Telecom Bretagne, France > > > PROGRAM CO-CHAIRMEN > > Raymond Bisdorff CRP-CU, Luxembourg > Jean-Pierre Descles ISHA Paris Sorbonne, France > > > ADVISORY COMMITTEE > > Actual open list: > > Nath. Aussenac-Gilles IRIT U. Sabatier, Toulouse, France > Beatrice Bacconet Thomson - CSF, France > Jean-Pierre Barthelemy Telecom Bretagne, France > Raymond Bisdorff CRP-CU, Luxembourg > Regine Bourgine CNRS (GRID-ENS), France > Jean-Pierre Descles ISHA Paris Sorbonne, France > M. Founeau E.D.F. Chatou, France > Fernand Grosber TrefilARBED, Luxembourg > Jean-Michel Le Bot Credit Mutuel de Bretagne, France > Marc Linster DEC, USA > Chr. de Maindreville Alcatel Alsthom, France > Jacques Mathieu LAFORIA Paris Jussieu, France > Olivier Paillet Alcatel Alsthom Recherche, France > Fernand Reinig CRP-CU, Luxembourg > Michael Richter Universitat Kaiserslautern, Germany > Susan Spirgi Swiss Bank Corp. Zuerich, Switzerland > Patrice Taillibert Dassault Electronique, France > Gerhard Strube Universitat Freiburg, Germany > Christian Tora EDIAT, France > > > LOCAL ORGANIZATION COMMITTEE > > Fernand Reinig local committee president CRP-CU Luxembourg > Raymond Bisdorff, Sophie Laurent, Emmanuel Pichon CRP-CU Luxembourg > Pierre Seck Centre Universitaire Luxembourg > > For more information and correspondence, please contact: > > R. Bisdorff > Centre de Recherche Public - Centre Universitaire > 162a, avenue de la Faiencerie > L-1511 Luxembourg > Tel.: (+352) 47 02 61 1 or 44 01 95 > Fax: (+352) 47 02 64 > Email: bisdorff at crpcu.lu > > > SPONSORSHIP > > The following sponsorships are envisaged: the Cogniscience project (CNRS Paris), > the European Commission (Luxembourg), the FEDIL (Luxembourg) and the Luxembourg > Governement (Departments of Economy and National Education). > > > ------------------------------------------------------------------------------- > > If you want to attend the meeting, or if you wish to submit a paper, please > complete and return the following registration form: > > > Name: > > Organisation: > > Department: > > Address: > > Telephone: > > Fax & Email: > > > > I want to attend the meeting: yes/no > > I wish to submit a paper: yes/no > Its title will be: > From gary at cs.ucsd.edu Tue Jan 4 10:33:56 1994 From: gary at cs.ucsd.edu (Gary Cottrell) Date: Tue, 4 Jan 94 07:33:56 -0800 Subject: Cognitive Science Conference: REVISED DEADLINE! Message-ID: <9401041533.AA00789@desi> Sixteenth Annual Conference of the COGNITIVE SCIENCE SOCIETY August 13-16, 1994 Georgia Institute of Technology Atlanta, Georgia CALL FOR PAPERS Revised due date: Tuesday, February 15, 1994 As Cognitive Science has matured over the years, it has broadened its scope in order to address fundamental issues of cognition embedded within culturally, socially, and technologically rich environments. The Sixteenth Annual Conference of the Cognitive Science Society aims at broad coverage of the many topics, methodologies, and disciplines that comprise Cognitive Science. The conference will highlight new ideas, theories, methods and results in a wide range of research areas relating to cognition. The conference will feature plenary addresses by invited speakers, technical paper and poster sessions, research symposia and panels, and a banquet. The conference will be held at Georgia Tech in Atlanta, home of the Civil Rights movement, the 1996 Olympics, and the Dogwood Festival. GUIDELINES FOR PAPER SUBMISSIONS Novel research papers are invited on any topic related to cognition. Reports of research that cuts across traditional disciplinary boundaries and investigations of cognition within cultural, social and technological contexts are encouraged. To create a high-quality program representing the newest ideas and results in the field, submitted papers will be evaluated through peer review with respect to several criteria, including originality, quality, and significance of research, relevance to a broad audience of cognitive science researchers, and clarity of presentation. Accepted papers will be presented at the conference as talks or posters, as appropriate. Papers may present results from completed research as well as report on current research with an emphasis on novel approaches, methods, ideas, and perspectives. Authors should submit five (5) copies of the paper in hard copy form by Tuesday, February 15, 1994, to: Prof. Ashwin Ram Cognitive Science 1994 Submissions Georgia Institute of Technology College of Computing 801 Atlantic Drive Atlanta, Georgia 30332-0280 If confirmation of receipt is desired, please use certified mail or enclose a self-addressed stamped envelope or postcard. DAVID MARR MEMORIAL PRIZES FOR EXCELLENT STUDENT PAPERS Papers with a student first author are eligible to compete for a David Marr Memorial Prize for excellence in research and presentation. The David Marr Prizes are accompanied by a $300.00 honorarium, and are funded by an anonymous donor. LENGTH Papers must be a maximum of eleven (11) pages long (excluding only the cover page but including figures and references), with 1 inch margins on all sides (i.e., the text should be 6.5 inches by 9 inches, including footnotes but excluding page numbers), double-spaced, and in 12-point type. Each page should be numbered (excluding the cover page). Template and style files conforming to these specifications for several text formatting programs, including LaTeX, Framemaker, Word, and Word Perfect, are available by anonymous FTP from ftp.cc.gatech.edu:/pub/cogsci94/submission-templates. (Camera-ready papers will be required only after authors are notified of acceptance; accepted papers will be allotted six proceedings pages in the usual double-column camera-ready format.) COVER PAGE Each copy of the paper must include a cover page, separate from the body of the paper, which includes: 1. Title of paper. 2. Full names, postal addresses, phone numbers, and e-mail addresses of all authors. 3. An abstract of no more than 200 words. 4. Three to five keywords in decreasing order of relevance. The keywords will be used in the index for the proceedings. 5. Preference for presentation format: Talk or poster, talk only, poster only. Accepted papers will be presented either as talks or posters, depending on authors' preferences and reviewers' recommendations about which would be more suitable, and will not reflect the quality of the papers. 6. A note stating if the paper is eligible to compete for a Marr Prize. DEADLINE Papers must be received by Tuesday, February 15, 1994. Papers received after this date will be recycled. CALL FOR SYMPOSIA In addition to the technical paper and poster sessions, the conference will feature research symposia, panels, and workshops. Proposals for symposia are invited. Proposals should indicate: 1. A brief description of the topic; 2. How the symposium would address a broad cognitive science audience, and some evidence of interest; 3. Names of symposium organizer(s); 4. List of potential speakers, their topics, and some estimate of their likelihood of participation; 5. Proposed symposium format (designed to last 90 minutes). Symposium proposals should be sent as soon as possible, but no later than January 14, 1994. Abstracts of the symposium talks will be published in the proceedings. CONFERENCE CHAIRS Kurt Eiselt and Ashwin Ram STEERING COMMITTEE Dorrit Billman, Mike Byrne, Alex Kirlik, Janet Kolodner (chair), Nancy Nersessian, Mimi Recker, and Tony Simon PLEASE ADDRESS ALL CORRESPONDENCE TO: Prof. Kurt Eiselt Cognitive Science 1994 Conference Georgia Institute of Technology Cognitive Science Program Atlanta, Georgia 30332-0505 E-mail: cogsci94 at cc.gatech.edu From Dave_Touretzky at DST.BOLTZ.CS.CMU.EDU Wed Jan 5 20:33:46 1994 From: Dave_Touretzky at DST.BOLTZ.CS.CMU.EDU (Dave_Touretzky@DST.BOLTZ.CS.CMU.EDU) Date: Wed, 05 Jan 94 20:33:46 EST Subject: notes for current NIPS authors and future submitters Message-ID: <23036.757820026@DST.BOLTZ.CS.CMU.EDU> 1. For current authors (those who presented papers at NIPS-93 and are now preparing the final camera-ready copy), a message from Jack Cowan: The deadline conditions for papers sent to Morgan Kauffman has been modified. For papers mailed in the US or Canada, a postmark of January 7 will suffice. For papers mailed from elsewhere, the receipt date of January 7 still applies. Jack Cowan, NIPS*93 General Chair 2. For future NIPS submitters: Unlike in previous years, NIPS-94 submissions will require full drafts of papers rather than 1000-word extended abstracts. People are encouraged (but not required) to submit their papers using the NIPS proceedings style. (Authors will still have a chance to revise their papers after the conference, as before.) LaTeX style files are available by anonymous FTP from: helper.systems.caltech.edu (131.215.68.12) in /pub/nips b.gp.cs.cmu.edu (128.2.242.8) in /usr/dst/public/nips Full details of the new submission requirements will be given in the NIPS*94 call for papers, which will appear later this month. -- Dave Touretzky, NIPS*94 Program Chair From N.Sharkey at dcs.shef.ac.uk Thu Jan 6 07:51:53 1994 From: N.Sharkey at dcs.shef.ac.uk (Noel Sharkey) Date: Thu, 6 Jan 94 12:51:53 GMT Subject: CNLP call Message-ID: <9401061251.AA16828@dcs.shef.ac.uk> **************************************************** * * * International Conference on * * New Methods in Language Processing * * * **************************************************** CALL FOR PAPERS Dates: 14-16th September 1994 (inclusive) Location: Centre for Computational Linguistics, UMIST, Manchester, UK. Purpose: In recent years there has been a steadily increasing interest in alternative theories and methodologies to the mainstream techniques of symbolic computational linguistics. This international conference will provide a forum for researchers in the broad area of new methods in NLP, i.e., symbolic and non-symbolic techniques of analogy-based, statistical, and connectionist processing, to present their most recent research and to discuss its implications. In order to focus the conference, however, it is intended to concentrate on research primarily involving written NLP. It is also hoped that the conference will promote discussion in general terms of what this branch of NLP hopes to achieve and how far this paradigm can take NLP in the future. Topics of Interest: * Example- and Memory-based MT * Corpus-based NLP * Bootstrapping techniques * Analogy-based NLP * Connectionist NLP * Statistical MT/NLP * Theoretical issues of sub-symbolic vs. symbolic NLP * Hybrid approaches Programme Committee: Co-chairs: Harold Somers, Daniel Jones (UMIST) Ken Church (AT&T) Hitoshi Iida (ATR) Sergei Nirenburg (CMU) David Powers (IMPACT) James Pustejovsky (Brandeis University) Satoshi Sato (JAIST) Noel Sharkey (Sheffield University) Royal Skousen (Brigham Young University) Jun-ichi Tsujii (UMIST) Susan Warwick-Armstrong (ISSCO) Yorick Wilks (Sheffield University) Preliminary paper submission deadline: 31st March 1994 Acceptance Notification by: 1st June 1994 Camera-ready copy due: 1st August 1994 Submission Requirements: Authors should submit FOUR *hard* copies of a preliminary version of the paper (NOT an outline or abstract) which should be no longer than 6 (A4) pages long, printed no smaller than 10-point. Papers should include a brief abstract, and a list of key words indicating which of the above topics are addressed. A contact address for the author(s) (preferably e-mail) should also be included. Send papers to: NeMLaP, Centre for Computational Linguistics, UMIST, Sackville Street, Manchester, UK. Enquiries : nemlap at ccl.umist.ac.uk From hutch at phz.com Thu Jan 6 18:26:32 1994 From: hutch at phz.com (Jim Hutchinson) Date: Thu, 6 Jan 94 18:26:32 EST Subject: Thesis available: RBF Approach to Financial Time Series Analysis Message-ID: <9401062326.AA01212@phz.com> My thesis, "A Radial Basis Function Approach to Financial Time Series Analysis" is now available from the MIT AI Lab Publications office as Technical Report 1457, both in hardcopy and in FTP-able compressed postscript form. Abstract follows. You may want to preview it before printing: it is 159 pages, and takes about 2MB of disk uncompressed. Comments and questions to hutch at phz.com are welcome! Jim Hutchinson Email: hutch at phz.com PHZ Partners Voice: +1 (617) 494-6000 One Cambridge Center FAX: +1 (617) 494-5332 Cambridge, MA 02142 USA ---------------------- Abstract ---------------------------------- A RADIAL BASIS FUNCTION APPROACH TO FINANCIAL TIME SERIES ANALYSIS Jim Hutchinson Billions of dollars flow through the world's financial markets every day, and market participants are understandably eager to accurately price financial instruments and understand relationships involving them. Nonlinear multivariate statistical modeling on fast computers offers the potential to capture more of the underlying dynamics of these high dimensional, noisy systems than traditional models while at the same time making fewer restrictive assumptions about them. For this style of exploratory, nonparametric modeling to be useful, however, care must be taken in fundamental estimation and confidence issues, especially concerns deriving from limited sample sizes. This thesis presents a collection of practical techniques to address these issues for a modeling methodology, Radial Basis Function networks. These techniques include efficient methods for parameter estimation and pruning, including a heuristic for setting good initial parameter values, a pointwise prediction error estimator for kernel type RBF networks, and a methodology for controlling the ``data mining'' problem. Novel applications in the finance area are described, including the derivation of customized, adaptive option pricing formulas that can distill information about the associated time varying systems that may not be readily captured by theoretical models. A second application area is stock price prediction, where models are found with lower out-of-sample error and better ``paper trading'' profitability than that of simpler linear and/or univariate models, although their true economic significance for real life trading is questionable. Finally, a case is made for fast computer implementations of these ideas to facilitate the necessary model searching and confidence testing, and related implementation issues are discussed. ------------------- FTP Retrieval Instructions ------------------- % ftp publications.ai.mit.edu Name (publications.ai.mit.edu:hutch): anonymous Password: (your email address) .. ftp> cd ai-publications/1993 ftp> binary ftp> get AITR-1457.ps.Z ftp> quit % uncompress AITR-1457.ps.Z % lpr AITR-1457.ps -------------------- For hardcopies contact ----------------------- Sally Richter MIT AI Laboratory Publications Office email: publications at ai.mit.edu phone: 617-253-6773 fax: 617-253-5060 Ask for TR-1457. From bahrami at cse.unsw.edu.au Fri Jan 7 01:22:19 1994 From: bahrami at cse.unsw.edu.au (Mohammad Bahrami) Date: Fri, 7 Jan 94 17:22:19 +1100 Subject: INTCON Message-ID: <940107062219.24210@cse.unsw.edu.au> To all neuro-control researchers, We are going to establish a special interest group called Intelligent Control (INTCON) which will be dedicated to subjects such as: Neuro-control, fuzzy logic control, reinforcement learning and other related subjects. Our objective is to provide a forum for communication and exchange of ideas among researchers in these fields. This will also provide a way for announcement of seminars, conferences, exhibitions, technical papers, programs, and so forth. If you are interested to join this group please send me an e-mail so I can put your name in the list of receivers of the e-mails sent to INTCON. Please inform others who might be interested. Thank you ------------------------------ Mohammad Bahrami University of New South Wales bahrami at syscon.ee.unsw.edu.au *** DO NOT "REPLY" BY USING "R" OR OTHER MEANS. *** *** SEND THE MAIL TO THE ABOVE ADDRESS. *** From MASULLI at GENOVA.INFN.IT Fri Jan 7 07:42:00 1994 From: MASULLI at GENOVA.INFN.IT (F. Masulli Dept. Physics Univ. Genova-It...) Date: Fri, 7 JAN 94 12:42 GMT Subject: Post-Doc Fellowship Message-ID: <5641@GENOVA.INFN.IT> ============= Post-Doc Fellowship in Soft-Computing =============== The Lab of Neural Networks of the Research Unit of Genoa (Italy) of the INFM (National Consortium for Matter Physics) could host a Post-Doc Fellow with a grant of the Program Human Capital and Mobility of the European Community. The research will be carried out in Application of Neural Networks and Fuzzy Systems (Soft Computing) to one of the following topics: - Image Understanding; - On-Line Handwriting Recognition. Applicant must be a Citizen of a member state of EEC or EFTA (with the exception of Italy) and hold a Doctoral Degree. The grant amount of the fellowship is interesting. Apply before Jan 20th, 1994 to Dr. Francesco Masulli (by fax or Email). Include a research program, a curriculum vitae, and the names and addresses of two referees. _____ Dr. Francesco Masulli Email: masulli at genova.infn.it Assistant Professor Fax: +39 10 314218 UdR INFM Genoa Via Dodecaneso 33 16146 Genova - Italy From nowlan at cajal.synaptics.com Fri Jan 7 12:54:57 1994 From: nowlan at cajal.synaptics.com (Steven J. Nowlan) Date: Fri, 07 Jan 94 09:54:57 -0800 Subject: NIPS preprint available via Neuroprose (ftp only) Message-ID: <9401071754.AA10849@cajal.> ****** PAPER AVAILABLE VIA NEUROPROSE *************************************** ****** AVAILABLE VIA FTP ONLY *********************************************** ****** PLEASE DO NOT FORWARD TO OTHER MAILING LISTS OR BOARDS. THANK YOU. ** FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/nowlan.nips94.ps.Z The following paper has been placed in the Neuroprose archive at Ohio State. The file is nowlan.nips94.ps.Z. Only the electronic version of this paper is available. This paper is 8 pages in length. This is a preprint of the paper to appear in Advance in Neural Information Processing Systems 6. This file contains 5 embedded postscript figures and is 1.8 Mbytes uncompressed. It has been successfully printed at remote sites, but it may not print on some printers with limited memory. ----------------------------------------------------- Mixtures of Controllers for Jump Linear and Non-linear Plants Steven J. Nowlan Synaptics, Inc. 2698 Orchard Parkway San Jose, CA 95134 ABSTRACT: We describe an extension to the Mixture of Experts architecture for modelling and controlling dynamical systems which exhibit multiple modes of behavior. This extension is based on a Markov process model, and suggests a recurrent network for gating a set of linear or non-linear controllers. The new architecture is demonstrated to be capable of learning effective control strategies for jump linear and non-linear plants with multiple modes of behavior. ----------------------------------------------------- Steven J. Nowlan Synaptics, Inc. 2698 Orchard Parkway San Jose, CA 95134 e-mail: nowlan at synaptics.com phone: (408) 434-0110 x118 From isabelle at neural.att.com Fri Jan 7 13:41:23 1994 From: isabelle at neural.att.com (Isabelle Guyon) Date: Fri, 7 Jan 94 13:41:23 EST Subject: No subject Message-ID: <9401071841.AA06106@neural> ************************************************************************** SPECIAL ISSUE OF THE INTERNATIONAL JOURNAL OF PATTERN RECOGNITION AND ARTIFICIAL INTELLIGENCE ON NEURAL NETWORKS ************************************************************************** ISSN: 0218-0014 Advances in Pattern Recognition Systems using Neural Networks, Eds. I. Guyon and P.S.P. Wang, IJPRAI, vol. 7, number 4, August 1993. ************************************************************************** Among the many applications that have been proposed for neural networks, pattern recognition has been one of the most successful ones, why? This collection of papers give will satisfy your curiosity! The commonplace rationale behind using Neural Networks is that a machine which architecture imitates that of the brain should inherit its remarquable intelligence. This logic usually contrasts with the reality of the performance of Neural Networks. In this special issue, however, the authors have kept some distance with the biological foundations of Neural Networks. The success of their applications relies, to a large extend, on careful engineering. For instance, many novel aspects of the works presented here are concerned with combining Neural Networks with other ``non neural'' modules. With: [ [1] ] Y. Bengio. A Connectionist Approach to Speech Recognition. [ [2] ] J. Bromley, J. W. Bentz, L. Bottou, I. Guyon, L. Jackel, Y. Le Cun, C. Moore, E. Sackinger, and R. Shah. Signature Verification with a Siamese TDNN. [ [3] ] C. Burges, J. Ben, Y. Le Cun, J. Denker and C. Nohl. Off-line Recognition of Handwritten Postal Words using Neural Networks. [ [4] ] H. Drucker, Robert Schapire and Patrice Simard. Boosting Performance in Neural Networks. [ [5] ] F. Fogelman, B. Lamy and E. Viennet. Multi-Modular Neural Network Architectures for Pattern Recognition: Applications in Optical Character Recognition and Human Face Recognition. [ [6] ] A. Gupta, M. V. Nagendraprasad, A. Liu, P. S. P. Wang and S. Ayyadurai. An Integrated Architecture for Recognition of Totally Unconstrained Handwritten Numerals. [ [7] ] E. K. Kim, J. T. Wu, S. Tamura, R. Close, H. Taketani, H. Kawai, M. Inoue and K. Ono. Comparison of Neural Network and K-NN Classification Methods in Vowel and Patellar Subluxation Image Recognitions. [ [8] ] E. Levin, R. Pieraccini and E. Bocchieri. Time-Warping Network: A Neural Approach to Hidden Markov Model based Speech Recognition. [ [9] ] H. Li and J. Wang. Computing Optical Flow with a Recurrent Neural Network. [ [10] ] W. Li and N. Nasrabadi. Invariant Object recognition Based on Neural Network of Cascaded RCE Nets. [ [11] ] G. Martin, M. Rashid and J. Pittman. Integrated Segmentation and Recognition Through Exhaustive Scans or Learned Saccadic Jumps. [ [12] ] C. B. Miller and C. L. Giles. Experimental Comparison of the Effect of Order in Recurrent Neural Networks. [ [13] ] L. Miller and A. Gorin. Structured Networks, for Adaptive Language Acquisition. [ [14] ] N. Morgan, H. Bourlard, S. Renals M. Cohen and H. Franco. Hybrid Neural Network / Hidden Markov Model Systems for Continuous Speech Recognition. [ [15] ] K. Peleg and U. Ben-Hanan. Adaptive Classification by Neural Net Based Prototype Populations. [ [16] ] L. Wiskott and C. von der Malsburg. A Neural System for the Recognition of Partially Occluded Objects in Cluttered Scenes - A Pilot Study. [ [17] ] G. Zavaliagkos, S. Austin, J. Makhoul and R. Schwartz. A Hybrid Continuous Speech Recognition System Using Segmental Neural Nets with Hidden Markov Models. From hutch at phz.com Fri Jan 7 17:36:34 1994 From: hutch at phz.com (Jim Hutchinson) Date: Fri, 7 Jan 94 17:36:34 EST Subject: Thesis available: RBF Approach to Financial Time Series Analysis In-Reply-To: Jim Hutchinson's message of Thu, 6 Jan 94 18:26:32 EST <9401062326.AA01212@phz.com> Message-ID: <9401072236.AA00442@phz.com> Aparently my thesis announcement yesterday generated quite a few requests to the MIT AI Lab Publications office for hardcopies, so they have asked me to forward the following message to everyone, encouraging the FTP route. My apologies for not getting this right the first time. Jim Hutchinson Email: hutch at phz.com PHZ Partners Voice: (617) 494-6000 One Cambridge Center FAX: (617) 494-5332 Cambridge, MA 02142 --------------- You can ftp this paper which is the fastest way to get it. FTP instructions are included below. You may prefer to order the hardcopy. In that case, it is necessary to send prepayment for report plus shipping. FOREIGN ------- AITR-1457 $9.00 shipping $5.50 SURFACE +$10.00 for AIRMAIL ------ Total $14.50 SURFACE $24.50 AIRMAIL Please specify SURFACE or AIRMAIL. Checks should be in US dollars and drawn on a US bank and made payable to MIT AI Lab. DOMESTIC -------- AITR-1457 $9.00 shipping $2.50 ------ Total $11.50 Please send your request and payment to the following address: MIT AI Lab Publications, NE43-818 545 Technology Square Cambridge, MA 02139 USA FTP INSTRUCTIONS ---------------- To ftp and download Jim Hutchinson's thesis `A Radial Basis Function Approach to Financial Time Series Analysis' (AITR-1457) type the following text within quotes at your promt (minus the quotes): 1. `ftp publications.ai.mit.edu' 2. login `anonymous' 3. password `your login' 4. `cd ai-publications/1993' 5. `get AITR-1457.ps.Z' You should see a message stating port command successful. If you do not type `bin' or `binary' before you try to `get file' again. If you need more details on any of the files or instructions listed above, please refer to the README file upon entry to our public ftp site or contact me. We hope this information is helpful to you in your research. Sally Richter MIT AI Laboratory Publications Office email: publications at ai.mit.edu phone: 617-253-6773 fax: 617-253-5060 From nowlan at cajal.synaptics.com Fri Jan 7 22:15:49 1994 From: nowlan at cajal.synaptics.com (Steven J. Nowlan) Date: Fri, 07 Jan 94 19:15:49 -0800 Subject: CORRECTION: Re: NIPS preprint available via Neuroprose (ftp only) Message-ID: <9401080315.AA11536@cajal.> I apologize for the extra bandwidth, but the first announcement of this paper omitted the name of the first author. My apologies to Tim, who did all the hard work. -S. ****** PAPER AVAILABLE VIA NEUROPROSE *************************************** ****** AVAILABLE VIA FTP ONLY *********************************************** ****** PLEASE DO NOT FORWARD TO OTHER MAILING LISTS OR BOARDS. THANK YOU. ** FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/nowlan.nips94.ps.Z The following paper has been placed in the Neuroprose archive at Ohio State. The file is nowlan.nips94.ps.Z. Only the electronic version of this paper is available. This paper is 8 pages in length. This is a preprint of the paper to appear in Advance in Neural Information Processing Systems 6. This file contains 5 embedded postscript figures and is 1.8 Mbytes uncompressed. It may not print on some printers with limited memory. ----------------------------------------------------- Mixtures of Controllers for Jump Linear and Non-linear Plants Timothy W. Cacciatore Steven J. Nowlan Department of Neurosciences Synaptics, Inc. University of California at San Diego 2698 Orchard Parkway La Jolla, CA 92093 San Jose, CA 95134 ABSTRACT: We describe an extension to the Mixture of Experts architecture for modelling and controlling dynamical systems which exhibit multiple modes of behavior. This extension is based on a Markov process model, and suggests a recurrent network for gating a set of linear or non-linear controllers. The new architecture is demonstrated to be capable of learning effective control strategies for jump linear and non-linear plants with multiple modes of behavior. ----------------------------------------------------- Steven J. Nowlan Synaptics, Inc. 2698 Orchard Parkway San Jose, CA 95134 e-mail: nowlan at synaptics.com phone: (408) 434-0110 x118 From bengioy at IRO.UMontreal.CA Sat Jan 8 11:53:18 1994 From: bengioy at IRO.UMontreal.CA (Yoshua Bengio) Date: Sat, 8 Jan 1994 11:53:18 -0500 Subject: Colloquium Message-ID: <9401081653.AA24867@saguenay.IRO.UMontreal.CA> COLLOQUIUM to be held at the 62nd congress of ACFAS (Association Canadienne Francaise pour l'Avancement des Sciences) (French Canadian Association for the Advancement of Science) LEARNING AND ARTIFICIAL NEURAL NETWORKS The next ACFAS congress will be held in Montreal from May 16th to May 20th 1994. Colloquia in many areas of science are organized to allow researchers in a specific area to meet, exchange views and make their work better known. Talks are given in FRENCH (no posters). Abstracts of accepted contributions that were submitted before February 1st will be published in the proceedings of the congress. Submission procedure: ******************** A short French abstract (150-300 words) which could fit on a 10cm x 15cm rectangle should be submitted by February 1st. Electronic submissions (plain ascii or latex) are much preferred: cloutier at iro.umontreal.ca or bengioy at iro.umontreal.ca J. Cloutier or Y. Bengio Dept. I.R.O., Universite de Montreal, C.P. 6128 Succ. A, Montreal, Qc, Canada, H3C3J7 Topics of interest: ****************** LEARNING AND ARTIFICIAL NEURAL NETWORKS - Learning algorithms - Generalization - Applications of theoretical results to practical problems - How accelerate learning algorithms - Links between learning algorithms and generalization - Use of a-priori knowledge in the design of learning systems - Combinations of artificial neural networks with other techniques -- Yoshua Bengio E-mail: bengioy at iro.umontreal.ca Fax: (514) 343-5834 Tel: (514) 343-6804. Residence: (514) 738-6206 Y-219, Universite de Montreal, Dept. IRO, CP 6128, Succ. A, 2900 Edouard-Montpetit, Montreal, Quebec, Canada, H3T 1J4 From becker at cs.toronto.edu Sun Jan 9 13:58:54 1994 From: becker at cs.toronto.edu (Sue Becker) Date: Sun, 9 Jan 1994 13:58:54 -0500 Subject: 3 job advertisements, McMaster University Message-ID: <94Jan9.135902edt.191@neuron.ai.toronto.edu> The Department of Psychology at McMaster University is advertising tenure-track faculty positions in the following three areas: cognitive psychology, sensation/perception and behavioural neuroscience. The 3 ads are below. Applicants whose research combines one of these areas with connectionist modelling are strongly encouraged. Sue Becker Department of Psychology McMaster University Hamilton, Ontario Canada L8S 4K1 email: becker at hypatia.psychology.mcmaster.ca _________________________________________________________________________ McMaster University's Psychology Department invites applications for a tenure-track position at the assistant or associate level commencing no earlier than July 1, 1994. This position is subject to final budgetary approval. We are seeking someone with an established record of independent research who will provide links between our group in cognitive psychology and other researchers in the department, such as an expert in computational models of dyslexia, the neuropsychology of attention, etc. McMaster University is committed to Employment Equity and encourages applications from all qualified candidates, including persons with disabilities, members of visible minorities and women. In accordance with Canadian immigration regulations, this advertisement is directed to Canadian citizens and landed immigrants in the first instance. To apply, send a CV, short statement of research interests, a publication list with selected reprints, and arrange to have three letters of reference sent to Dr. L. R. Brooks, Chair of the Search Committee, Department of Psychology, McMaster University, Hamilton, Ontario, Canada L8S 4K1. ________________________________________________________________________ McMaster University's Psychology Department seeks a faculty member for a tenure-track position at the senior assistant or associate level commencing no earlier than July 1, 1994, with interest in some aspect of sensation or perception. This position is subject to final budgetary approval. The candidate should have strong quantitative skills and an established record of independent research, which may involve anatomy, physiology, behaviour, or modelling. McMaster University is committed to Employment Equity and encourages applications from all qualified candidates, including persons with disabilities, members of visible minorities and women. In accordance with Canadian immigration regulations, this advertisement is directed to Canadian citizens and landed immigrants in the first instance. To apply, send a CV, short statement of research interests, a publication list with selected reprints, and arrange to have three letters of reference sent to Dr. J. R. Platt, Chair of the Search Committee, Department of Psychology, McMaster University, Hamilton, Ontario, Canada L8S 4K1. ________________________________________________________________________ The Department of Psychology at McMaster University invites applications for a tenure-track position at the assistant professor level in the area of behavioural neuroscience commencing no earlier than July 1, 1994. This position is subject to final budgetary approval. The long-term goal of the applicant's research must be an understanding of a behavioural problem preferably in, but not restricted to, the areas of learning/memory, motivation, or perception. Preference will be given to applicants who can exploit new and innovative biological methods (e.g., molecular or imaging techniques) or other methods not currently represented in the department. McMaster University is committed to Employment Equity and encourages applications from all qualified candidates, including persons with disabilities, members of visible minorities and women. In accordance with Canadian immigreation regulations, this advertisment is directed to Canadian citizens and landed immigrants in the fisrt instance. To apply, send a CV, short statement of research interests, a publication list with selected reprints, and arrange to have three letters of reference to Dr. R. Racine, Chair of the Search Committee, Department of Psychology, McMaster University, Hamilton, Ontario, Canada L8S 4K1. __________________________________________________________________________ From hzs at cns.brown.edu Mon Jan 10 15:25:07 1994 From: hzs at cns.brown.edu (Harel Z. Shouval) Date: Mon, 10 Jan 1994 15:25:07 -0500 (EST) Subject: paper available in neuroprose Message-ID: <9401102025.AA09095@cns.brown.edu> A non-text attachment was scrubbed... Name: not available Type: text Size: 2164 bytes Desc: not available Url : https://mailman.srv.cs.cmu.edu/mailman/private/connectionists/attachments/00000000/2059d051/attachment-0001.ksh From zemel at salk.edu Mon Jan 10 19:52:40 1994 From: zemel at salk.edu (Richard S. Zemel) Date: Mon, 10 Jan 94 16:52:40 PST Subject: Thesis on neuroprose Message-ID: <9401110052.AA18791@broca> **DO NOT FORWARD TO OTHER GROUPS** A postscript copy of my PhD thesis has been placed in the neuroprose archive. It prints on 138 pages. The abstract is given below, followed by retrieval instructions. Rich Zemel e-mail: zemel at salk.edu ----------------------------------------------------------------------------- A Minimum Description Length Framework for Unsupervised Learning ABSTRACT A fundamental problem in learning and reasoning about a set of information is finding the right representation. The primary goal of an unsupervised learning procedure is to optimize the quality of a system's internal representation. In this thesis, we present a general framework for describing unsupervised learning procedures based on the Minimum Description Length (MDL) principle. The MDL principle states that the best model is one that minimizes the summed description length of the model and the data with respect to the model. Applying this approach to the unsupervised learning problem makes explicit a key trade off between the accuracy of a representation (i.e., how concise a description of the input may be generated from it) and its succinctness (i.e., how compactly the representation itself can be described). Viewing existing unsupervised learning procedures in terms of the framework exposes their implicit assumptions about the type of structure assumed to underlie the data. While these existing algorithms typically minimize the data description using a fixed-length representation, we use the framework to derive a class of objective functions for training self-supervised neural networks, where the goal is to minimize the description length of the representation simultaneously with that of the data. Formulating a description of the representation forces assumptions about the structure of the data to be made explicit, which in turn leads to a particular network configuration as well as an objective function that can be used to optimize the network parameters. We describe three new learning algorithms derived in this manner from the MDL framework. Each algorithm embodies a different scheme for describing the internal representation, and is therefore suited to a range of datasets based on the structure underlying the data. Simulations demonstrate the applicability of these algorithms on some simple computational vision tasks. ----------------------------------------------------------------------------- I divided the thesis for retrieval purposes into 3 chunks. Also, it is in book style, so it will look better if you print it out on a double-sided printer if you have access to one. To retrieve from neuroprose: unix> ftp cheops.cis.ohio-state.edu Name (cheops.cis.ohio-state.edu:zemel): anonymous Password: (use your email address) ftp> cd pub/neuroprose ftp> get zemel.thesis1.ps.Z ftp> get zemel.thesis2.ps.Z ftp> get zemel.thesis3.ps.Z ftp> quit unix> uncompress zemel* unix> lpr zemel.thesis1.ps unix> lpr zemel.thesis2.ps unix> lpr zemel.thesis3.ps From hamps at richibucto.jpl.nasa.gov Mon Jan 10 20:37:05 1994 From: hamps at richibucto.jpl.nasa.gov (John B. Hampshire II) Date: Mon, 10 Jan 94 17:37:05 -0800 Subject: Efficient Learning Message-ID: <9401110137.AA09840@richibucto.jpl.nasa.gov> A Differential Theory of Learning for Efficient Statistical Pattern Recognition J. B. Hampshire II Jet Propulsion Laboratory, M/S 238-420 California Institute of Technology 4800 Oak Grove Drive Pasadena, CA 91109-8099 hamps at bvd.jpl.nasa.gov ABSTRACT -------- There is more to learning stochastic concepts for robust statistical pattern recognition than the learning itself: computational resources must be allocated and information must be obtained. Therein lies the key to a learning strategy that is efficient, requiring the fewest resources and the least information necessary to produce classifiers that generalize well. Probabilistic learning strategies currently used with connectionist (as well as most traditional) classifiers are often inefficient, requiring high classifier complexity and large training sample sizes to ensure good generalization. An asymptotically efficient **differential learning strategy** is set forth. It guarantees the best generalization allowed by the choice of classifier paradigm as long as the training sample size is large; this guarantee also holds for small training sample sizes when the classifier is an ``improper parametric model'' of the data (as it often is). Differential learning requires the classifier with the minimum functional complexity necessary --- under a broad range of accepted complexity measures --- for Bayesian (i.e., minimum probability-of-error) discrimination. The theory is demonstrated in several real-world machine learning/pattern recognition tasks associated with Fisher's Iris data, optical character recognition, medical diagnosis, and airborne remote sensing imagery interpretation. These applications focus on the implementation of differential learning and illustrate its advantages and limitations in a series of experiments that complement the theory. The experiments demonstrate that differentially-generated classifiers consistently generalize better than their probabilistically-generated counterparts across a wide range of real-world learning-and-classification tasks. The discrimination improvements range from moderate to significant, depending on the statistical nature of the learning task and its relationship to the functional basis of the classifier used. ============================================================ RETRIEVING DOCUMENTS: To obtain a list of the materials/documents that can be retrieved electronically, use anonymous ftp as follows (the IP address of speech1 is 128.2.254.145): > ftp speech1.cs.cmu.edu > user: anonymous > passwd: > cd /usr0/hamps/public > get README Read the file README and choose what you want to retrieve. All files are in /usr0/hamps/public and /usr0/hamps/public/thesis. I welcome your comments and constructive criticism. Happy reading. -JBH2 From tap at cs.toronto.edu Tue Jan 11 08:15:42 1994 From: tap at cs.toronto.edu (Tony Plate) Date: Tue, 11 Jan 1994 08:15:42 -0500 Subject: NIPS preprint available Message-ID: <94Jan11.081543edt.197@neuron.ai.toronto.edu> Preprint Available: To appear in J. D. Cowan, G. Tesauro, and J. Alspector, editors, {\it Advances in Neural Information Processing Systems - 6 - (NIPS*93)}, Morgan Kaufmann, San Mateo, CA Estimating analogical similarity by dot-products of Holographic Reduced Representations. Tony A. Plate Department of Computer Science University of Toronto Toronto, M5S 1A4 Canada tap at ai.utoronto.ca ABSTRACT Models of analog retrieval require a computationally cheap method of estimating similarity between a probe and the candidates in a large pool of memory items. The vector dot-product operation would be ideal for this purpose if it were possible to encode complex structures as vector representations in such a way that the superficial similarity of vector representations reflected underlying structural similarity. This paper describes how such an encoding is provided by Holographic Reduced Representations (HRRs), which are a method for encoding nested relational structures as fixed-width distributed representations. The conditions under which structural similarity is reflected in the dot-product rankings of HRRs are discussed. [This paper is possibly relevant to the recent discussion of the binding problem on this list. In HRRs, I use convolution (which can be thought of as a compressed conjunctive code) to bind roles and fillers, and build up distributed representations of hierarchical predicate structures. This representation preserves the natural similarity structure of predicates and objects.] - Obtain by ftp from archive.cis.ohio-state.edu in pub/neuroprose. - No hardcopy available. - Software to perform the simulations available. - FTP procedure: unix> ftp archive.cis.ohio-state.edu (or 128.146.8.52) Name: anonymous Password: neuron ftp> cd pub/neuroprose ftp> binary ftp> get plate.nips93.ps.Z ftp> quit unix> uncompress plate.nips93.ps.Z unix> lpr plate.nips93.ps (or however you print postscript) From kolen-j at cis.ohio-state.edu Tue Jan 11 18:02:04 1994 From: kolen-j at cis.ohio-state.edu (john kolen) Date: Tue, 11 Jan 1994 18:02:04 -0500 Subject: Reprint Announcement Message-ID: <199401112302.SAA27441@pons.cis.ohio-state.edu> This is an announcement of a newly available paper in neuroprose: Fool's Gold: Extracting Finite State Machines From Recurrent Network Dynamics John F. Kolen Laboratory for Artificial Intelligence Research Department of Computer and Information Science The Ohio State University Columbus, OH 43210 kolen-j at cis.ohio-state.edu Abstract Several recurrent networks have been proposed as representations for the task of formal language learning. After training a recurrent network rec- ognize a formal language or predict the next symbol of a sequence, the next logical step is to understand the information processing carried out by the network. Some researchers have begun to extracting finite state machines from the internal state trajectories of their recurrent networks. This paper describes how sensitivity to initial conditions and discrete measurements can trick these extraction methods to return illusory finite state descriptions. This paper will appear in NIPS 6. ************************ How to obtain a copy ************************ Via Anonymous FTP: unix> ftp archive.cis.ohio-state.edu Name: anonymous Password: (type your email address) ftp> cd pub/neuroprose ftp> binary ftp> get kolen.foolsgold.ps.Z ftp> quit unix> uncompress kolen.foolsgold.ps.Z unix> lpr kolen.foolsgold.ps (or what you normally do to print PostScript) From reza at ai.mit.edu Wed Jan 12 09:38:12 1994 From: reza at ai.mit.edu (Reza Shadmehr) Date: Wed, 12 Jan 94 09:38:12 EST Subject: Rerport available: Human Adaptive Control Message-ID: <9401121438.AA12470@corpus-callosum> The following report is available from neuropose: fTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/shadmehr.elements.ps.Z number of pages: 8 contact: reza at ai.mit.edu ---------------------------------------------------- Computational Elements of the Adaptive Controller of the Human Arm Reza Shadmehr and Ferdinando Mussa-Ivaldi Dept. of Brain and Cognitive Sciences M. I. T. We consider the problem of how the CNS learns to control dynamics of a mechanical system. By using a paradigm where a subject's hand interacts with a virtual mechanical environment, we show that learning control is via composition of a model of the imposed dynamics. Some properties of the computational elements with which the CNS composes this model are inferred through the generalization capabilities of the subject outside the training data. with best wishes, Reza Shadmehr reza at ai.mit.edu From singer at cs.huji.ac.il Wed Jan 12 11:32:35 1994 From: singer at cs.huji.ac.il (Yoram Singer) Date: Wed, 12 Jan 1994 18:32:35 +0200 Subject: Preprint announcement Message-ID: <199401121632.AA03417@minuet.cs.huji.ac.il> *************** PAPERS AVAILABLE **************** *** DO NOT FORWARD TO ANY OTHER LISTS *** ************************************************* The following papers have been placed in cs.huji.ac.il (132.65.16.10). The files are vmm.ps.Z and cursive.ps.Z . Ftp instructions follow the abstracts. These are preprints of the papers to appear in the NIPS 6 proceedings. ----------------------------------------------------- Decoding Cursive Scripts Yoram Singer and Naftali Tishby Institute of Computer Science and Center for Neural Computation Hebrew University, Jerusalem 91904, Israel ABSTRACT: Online cursive handwriting recognition is currently one of the most intriguing challenges in pattern recognition. This study presents a novel approach to this problem which is composed of two complementary phases. The first is dynamic encoding of the writing trajectory into a compact sequence of discrete motor control symbols. In this compact representation we largely remove the redundancy of the script, while preserving most of its intelligible components. In the second phase these control sequences are used to train adaptive probabilistic acyclic automata (PAA) for the important ingredients of the writing trajectories e.g. letters. We present a new and efficient learning algorithm for such stochastic a automata, and demonstrate its utility for spotting and segmentation of cursive scripts. Our experiments show that over 90% of the letters are correctly spotted and identified, prior to any higher level language model. Moreover, both the training and recognition algorithms are very efficient compared to other modeling methods and the models are `on-line' adaptable to other writers and styles. ----------------------------------------------------- The Power of Amnesia Dana Ron Yoram Singer Naftali Tishby Institute of Computer Science and Center for Neural Computation Hebrew University, Jerusalem 91904, Israel ABSTRACT: We propose a learning algorithm for a variable memory length Markov process. Human communication, whether given as text, handwriting, or speech, has multi characteristic time scales. On short scales it is characterized mostly by the dynamics that generate the process, whereas on large scales, more syntactic and semantic information is carried. For that reason the conventionally used fixed memory Markov models cannot capture effectively the complexity of such structures. On the other hand using long memory models uniformly is not practical even for as short memory as four. The algorithm we propose is based on minimizing the statistical prediction error by extending the memory, or state length, adaptively, until the total prediction error is sufficiently small. We demonstrate the algorithm by learning the structure of natural English text and applying the learned model to the correction of corrupted text. Using less than 3000 states the model's performance is far superior to that of fixed memory models with similar number of states. We also show how the algorithm can be applied to intergenic E.coli DNA base prediction with results comparable to HMM-based methods. ----------------------------------------------------- FTP INSTRUCTIONS unix> ftp cs.huji.ac.il (or 132.65.16.10) Name: anonymous Password: your_full_email_address ftp> cd singer ftp> binary ftp> get vmm.ps.Z ftp> get cursive.ps.Z ftp> quit unix> uncompress vmm.ps.Z cursive.ps.Z unix> lpr -P vmm.ps cursive.ps From soller at asylum.cs.utah.edu Wed Jan 12 17:03:29 1994 From: soller at asylum.cs.utah.edu (Jerome Soller) Date: Wed, 12 Jan 94 15:03:29 -0700 Subject: Neural Plasticity and Control Modelling Faculty Position in Bioengineering Message-ID: <9401122203.AA12335@asylum.cs.utah.edu> I was recently made aware of the following job announcement from the University of Utah Department of Bioengineering that was advertised nationally. They have two tenure track faculty openings, one of which may focus on models of neural plasticity and control. I repeat their annoucement verbatim below, and I apologize for the short notice (responses must be made by Jan. 15th). For further information, contact, Dr. Richard Normann of Bioengineering. Sincerely, Jerome Soller (soller at asylum.cs.utah.edu) U. of Utah Dept. of Computer Science and VA Geriatric, Research, Education and Clinical Center ------------------------------------------------------------- Tenure Track Faculty Positions Department of Bioengineering University of Utah Applications are invited for two tenure-track positions in the area of "Biobased Engineering." Candidates must have an earned doctorate and a strong physical science or engineering background, with a specific biological direction to their research. We are specifically seeking candidates with demonstrated expertise in one of the following three areas. 1. Micro/Nano Fabrication of Inorganic, Organic, and Biomolecular Materials: use of inorganic materials, compliant biomaterials, or composite inorganic/polymeric materials to fabricate microsensors and microactuators, and the application of microsystems to problems in the life sciences. 2. Cellular Bioengineering: cytoskeletal biomechanics, mechanisms of cell attachment, locomotion, and bioenergetics. 3. Neural Plasticity and Control: information processing and plasticity in higher neural centers, neural network architectures, and adaptive and hierarchical control systems. Appointees will be expected to develop significant research programs, to assist in the development of new teaching laboratories, and to reach graduate classes in their area of specialization. A complete CV, names of three references, and brief career goals/objectives statement should be sent to Dr. R. Normann, Chair, Department of Bioengineering, 2480 MEB, University of Utah, Salt Lake City, UT 84112 (phone 801-581-8528, FAX 801-585-5361) by January 15, 1994, or until qualified applicants applications are identified. The University is an AA/EO employer, encourages applications from women and minorities, and provides reasonable accomodation to the known disabilities of applicants and employees. From gherrity at io.nosc.mil Thu Jan 13 21:02:31 1994 From: gherrity at io.nosc.mil (Mike Gherrity) Date: Thu, 13 Jan 94 18:02:31 PST Subject: Thesis available on neuroprose Message-ID: <199401140202.SAA02329@io.nosc.mil> FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/gherrity.thesis.ps.Z A postscript copy of my PhD thesis has been placed in the neuroprose archive. It prints on 110 pages. The abstract is given below, followed by retrieval instructions. Mike Gherrity e-mail: gherrity at nosc.mil ----------------------------------------------------------------------------- A Game-Learning Machine ABSTRACT This disertation describes a program which learns good strategies for two-person, deterministic, zero-sum board games of perfect information. The program learns by simply playing the game against either a human or computer opponent. The results of the program's learning the games of tic-tac-toe, connect-four, and chess are reported. The program consists of a game-independent kernel and a game-specific move generator module. Only the move generator is modified to reflect the rules of the game to be played. The kernel remains unchanged for different games. The kernal uses a temporal difference procedure combined with a backpropagation neural network to learn good evaluation functions for the game being played. Central to the performance of the program is the consistency search procedure. This is a game-independent generalization of the capture tree search used in most successful chess playing programs. It is based on the idea of using search to correct errors in evaluations of positions. This procedure is described, analyzed, tested, and implemented in the game-learning program. Both the test results and the performance of the program confirm the results of the analysis which indicate that consistency search improves game playing performance for sufficiently accurate evaluation functions. ----------------------------------------------------------------------------- To retrieve from neuroprose: unix> ftp cheops.cis.ohio-state.edu Name (cheops.cis.ohio-state.edu:zemel): anonymous Password: (use your email address) ftp> cd pub/neuroprose ftp> get gherrity.thesis.ps.Z ftp> quit unix> uncompress gherrity.thesis.ps.Z unix> lpr gherrity.thesis.ps From D.Gorse at cs.ucl.ac.uk Fri Jan 14 11:08:47 1994 From: D.Gorse at cs.ucl.ac.uk (D.Gorse@cs.ucl.ac.uk) Date: Fri, 14 Jan 94 16:08:47 +0000 Subject: Preprint available - reinforcement learning for continuous functions Message-ID: FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/gorse.reinforce.ps.Z The file gorse.reinforce.ps.Z is now available for copying from the Neuroprose archive. This is a 6 page paper, submitted to WCNN '94 San Diego. A longer and more detailed paper describing this work is in preparation and will be available soon. --------------------------------------------------------------------------- A PULSE-BASED REINFORCEMENT ALGORITHM FOR LEARNING CONTINUOUS FUNCTIONS D Gorse Department of Computer Science University College, Gower Street, London WC1E 6BT, UK J G Taylor Department of Mathematics T G Clarkson Department of Electrical and Electronic Engineering King's College, Strand, London WC2R 2LS, UK ABSTRACT: An algorithm is presented which allows continuous functions to be learned by a neural network using spike-based reinforcement learning. Both the mean and the variance of the weights are changed during training; the latter is accomplished by manipulating the lengths of the spike trains used to represent real-valued quantities. The method is here applied to the probabilistic RAM (pRAM) model, but it may be adapted for use with any pulse-based stochastic model in which individual weights behave as random variables. Denise Gorse (D.Gorse at cs.ucl.ac.uk) ---------------------------------------------------------------------------- To obtain a copy: ftp archive.cis.ohio-state.edu login: anonymous password: cd pub/neuroprose binary get gorse.reinforce.ps.Z quit Then at your system: uncompress gorse.reinforce.ps.Z lpr -P gorse.reinforce.ps From lfausett at zach.fit.edu Fri Jan 14 11:28:59 1994 From: lfausett at zach.fit.edu ( Laurene V. Fausett) Date: Fri, 14 Jan 94 11:28:59 -0500 Subject: book announcement Message-ID: <9401141628.AA15672@zach.fit.edu> BOOK ANNOUNCEMENT Title: Fundamentals of Neural Networks: Architectures, Algorithms, and Applications Author: Laurene V. Fausett Publisher: Prentice Hall Ordering Information: Price $49.00 ISBN 0-13-334186-0 To order, call Prentice-Hall Customer Service at 1-800-922-0579 or your local Prentice-Hall representative. This book has also been published in paperback as a Prentice Hall International Edition with ISBN 0-13-042250-9 for distribution outside of the U.S.A., Canada, and Mexico. Brief Description: Written with the beginner in mind, this volume offers an exceptionally clear and thorough introduction to neural networks at an elementary level. Systematic discussion of all major neural nets features presentation of the architectures, detailed algorithms, and examples of simple applications - in many cases variations on a theme. Each chapter concludes with suggestions for further study, including numerous exercises and computer projects. An instructor's manual with solutions and sample software (in Fortran and C) will be available later this spring. Table of Contents Chapter 1 INTRODUCTION; 1.1 Why neural networks, and why now?; 1.2 What is a neural net?; 1.3 Where are neural nets being used?; 1.4 How are neural networks used?; 1.5 Who is developing neural networks?; 1.6 When neural nets began - the McCulloch-Pitts neuron. Chapter 2 SIMPLE NEURAL NETS FOR PATTERN CLASSIFICATION; 2.1 General discussion; 2.2 Hebb net; 2.3 Perceptron; 2.4 Adaline. Chapter 3 PATTERN ASSOCIATION; 3.1 Training algorithms for pattern association; 3.2 Heteroassociative memory neural network; 3.3 Autoassociative net; 3.4 Iterative autoassociative net; 3.5 Bidirectional associative memory (BAM). Chapter 4 NEURAL NETWORKS BASED ON COMPETITION; 4.1 Fixed-weight competitive nets; 4.2 Kohonen self-organizing maps; 4.3 Learning vector quantization; 4.4 Counterpropagation. Chapter 5 ADAPTIVE RESONANCE THEORY; 5.1 Introduction; 5.2 ART1; 5.3 ART2. Chapter 6 BACKPROPAGATION NEURAL NET; 6.1 Standard backpropagation; 6.2 Variations; 6.3 Theoretical results. Chapter 7 A SAMPLER OF OTHER NEURAL NETS; 7.1 Fixed weight nets for constrained optimization; 7.2 A few more nets that learn; 7.3 Adaptive architectures; 7.4 Neocognitron. Glossary; References; Index. From ilya at cheme.seas.upenn.edu Fri Jan 14 02:26:05 1994 From: ilya at cheme.seas.upenn.edu (Ilya Rybak) Date: Fri, 14 Jan 94 02:26:05 -0500 Subject: AHP conductance descriptions Message-ID: <9401140726.AA23250@cheme.seas.upenn.edu> Dear Connectionists, I am developing models of respiratory and baroreflex neural networks on the base of H-H style model of single neuron. I have a problem with description of AHP conductance, that play a very important role in my models. I would be very thankful to everybody for any help. The problem consists in the following. Let's consider the description of AHP conductance in the paper of Yamada et al."Multiple channels and calcium dynamics"(In Methods in oNeuronal Modeling.Eds. Koch and Segev, The MIT press, 1989,97-133.) In the page 132, you can see for AHP conductance that tau=1000/(f(Ca)+b) (1) m=f(Ca)/(f(Ca)+b) (2) In the page 133, you can see that f(Ca)=1.25*10^8*[Ca++]^2 and b=2.5 (3) ----- Ca is measured by mM. (4) In the page 132, you can also see that "the midpoint for m is 44.7 nM and for m^2 is 69.5 nM" (5) It is very simple to chech that (1)-(4) do not correspond to (5). I think, that it is mistake. To correct this mistake we have to change (3). There are two ways for this: 1) f(Ca) is the same as in (3), but b=0.25 (6) ------- 2) f(Ca)=1.25*10^9*[Ca++]^2 and b is the same as in (3) (7) -------- I do not know what is correct (3) or (6) or (7). But the behavior of the behavior of my model depends on this very strong. I have tried to compare the discription of AHP conductance in Huguenard's and McCormick's Manual for VClamp and CClamp. In the page 28 of the Manual the authors refered to the same paper and wrote: alfa=1.2*10^9*[Ca++]^2 and betta=0.001 (8) Taking into account that f(Ca)=alfa*1000 and b=betta*1000 (9) and that Ca is measured in M (10) we can get the following f(Ca)=1.2*10^6*[Ca++]^2 and b=1 (11) ------- This is absolutely different from (3) as well as from (6) as well as from (7). The expresions (3), (6), (7) and (11) are too different. Because of this the behavior of my neuron and network models is absolutaly differnt depending on which one AHP description I use. I cannot go ahead without finding out which one of descriptions is correct. I will be very tankful to everybody for any explaination of this mysterious. Sinceraly, Ilya Rybak Dept. of Neuroscience University of Pennsylvania and Neural Computation Group at DuPont Com. ilya at cheme.seas.upenn.edu From lyle at ai.mit.edu Sat Jan 15 12:05:04 1994 From: lyle at ai.mit.edu (Lyle J. Borg-Graham) Date: Sat, 15 Jan 94 12:05:04 EST Subject: AHP conductance descriptions In-Reply-To: Ilya Rybak's message of Fri, 14 Jan 94 02:26:05 -0500 <9401140726.AA23250@cheme.seas.upenn.edu> Message-ID: <9401151705.AA25287@peduncle> Ilya, I developed a model for I_ahp for hippocampus that is described in: Borg-Graham, L., {\it Modelling the Somatic Electrical Behavior of Hippocampal Pyramidal Neurons}. MIT AI Lab Technical Report 1161, 1989 (290pp). This can be obtained from the MIT AI Lab Publications office (richter at ai.mit.edu). A description of the general extended H-H channel model (including V and Ca dependencies) that I used is found in: Borg-Graham, L., {\it Modelling the Non-Linear Conductances of Excitable Membranes}. Chapter in {\it Cellular Neurobiology: A Practical Approach}, edited by J.\ Chad and H.\ Wheal, IRL Press at Oxford University Press, 1991. I have lifted some of the Latex material that I have on AHP and included it below - I hope it can be of some use! Note that the shell.1 and shell.2 [Ca] that are referred to below are from a proposal in TR1161 that $I_C$ channels are co-localized with Ca channels, and that $I_{AHP}$ channels are evenly distributed across the somatic membrane. The motivation for this inhomogeneous arrangement was to account for the apparent fast and transient Ca-dependence of $I_C$, in contrast with the slower, more intergrative Ca-dependence of $I_{AHP}$. Simulations with a simple 3-compartment model of Ca accumulation (shell.1 being a fraction of the submembrane space in the vicinity of the co-localized CA and C channels, shell.2 being the remainder of the submembrane space which in turn supplies [Ca] for the AHP channels, and a constant [Ca] core compartment) show that this arrangement reproduces the Ca-dependent behavior of the fAHP (from $I_C$) and the slow AHP (from $I_{AHP}$). The parameters listed for the Ca-dep $w$ particle result in a (increasing/decreasing) sigmoidal dependence of the (steady-state/time constant) on the log[Ca], with a half point at 0.004mM and (0.1/0.9) points on the curves at about (0.002/0.009) mM. The maximum tau is 100ms. Also, the specific parameter values referenced below should be used mainly as guidelines; they are currently under revision. **************************************************************** **************************************************************** **************************************************************** **************************************************************** **************************************************************** % for laTeX \documentstyle[11pt]{report} \begin{document} \noindent Summary of $I_{AHP}$, extracted from Chapter 7 of:\\ \noindent Borg-Graham, L., {\it Modelling the Somatic Electrical Behavior of Hippocampal Pyramidal Neurons}. MIT AI Lab Technical Report 1161, 1989 (290pp). . . . . In the case of $I_{C}$ and $I_{AHP}$, little voltage clamp data is available for either their steady state or temporal properties of any presumed activation/inactivation parameters. In addition, describing these currents is complicated by the fact that they are presumably mediated by intracellular $Ca^{2+}$. Little quantitative data is available on this interaction for either current, and there is at present no consensus among workers in this field as to the mechanisms involved. As introduced in the previous chapter and which shall be expanded upon later, I have made the simple assumption (like that used by other workers, e.g. Tra-Lli-79) that $I_{C}$ and $I_{AHP}$ are dependent on a power of the concentration of $Ca^{2+}$ either directly beneath the membrane or in a secondary ``compartment''. This is a highly speculative model, as discussed in the previous chapter. The parameters of this description are based primarily on heuristics, specifically the simulation of the fAHP and the AHP that is observed in HPC. Making the situation more difficult is the fact that there are no protocols to date in which $I_C$ or $I_{AHP}$ are activated without the concomitment presence of other currents, thereby inextricably tying the behavior of any set of estimated parameters for these currents to those of other currents. To a first approximation, the actions of $I_C$ and $I_{AHP}$ are independent of one another. $I_C$ is transient over a time span of a few milliseconds during the spike, and the evidence indicates that this a large current. On the other hand, $I_{AHP}$ activates more slowly, is small, and may last from 0.5 to several seconds. However, since both these currents are dependent on $Ca^{2+}$ entry, their estimation was tied to the description of $I_{Ca}$ and the mechanisms regulating $[Ca^{2+}]_{shell.1}$ and $[Ca^{2+}]_{shell.2}$. Therefore, while the behavior of the $I_C$ or $I_{AHP}$ descriptions could be evaluated independently, whenever the $Ca^{2+}$ mechanisms were modified to alter one of the current's action, the effect of the modification on the other current had to be checked. \section{$Ca^{2+}$-Mediation of $K^+$ Currents by $Ca^{2+}$~-~binding Gating Particle $w$} In order to cause $I_C$ and $I_{AHP}$ to be mediated by intracellular $Ca^{2+}$, I incorporated a $Ca^{2+}$-binding gating particle in the expressions for both of these currents. Several workers have postulated mechanisms for such an interaction between intracellular $Ca^{2+}$ and different ion channels, ranging from complex multi-state kinetic models based on experimental data to very simple descriptions for modelling studies (Tra-Lli-79). In light of the paucity of quantitative data on such mechanisms in HPC, my goals for the description of a putative, generic $Ca^{2+}$-binding gating particle were as follows: \begin{itemize} \item Relationship between $Ca^{2+}$ concentration and particle activation allowing for non-degenerate kinetics considering the range of $Ca^{2+}$ concentrations during various cell responses. \item Binding kinetics based on a simple but reasonable model. \item Kinetic description that could be easily modified to yield significantly different behavior, that is a description that could be modified to suit a wide range of desired behaviors. \end{itemize} To this end the following description for a $Ca^{2+}$ -binding gating particle, $w$, was used. Each $w$ particle can be in one of two states, open or closed, just as the case for the Hodgkin-Huxley-like voltage-dependent activation and inactivation gating particles. Each $w$ particle is assumed to have $n$ $Ca^{2+}$ binding sites, all of which must be bound in order for the particle to be in the open state. Binding is cooperative in a sense that reflects the two states available to a given particle, i.e. either a particle has no $Ca^{2+}$ ions bound to it, and therefore it is in the closed state, or all $n$ binding sites are filled, and the particle is in the open state. The state diagram for this reaction is as follows: $$ w_{closed} + n\, Ca^{2+}_{in} \buildrel \alpha, \beta \over \rightleftharpoons w_{open}^*$$ \noindent where the $*$ notation means that the particle is bound to all $n$ (intracellular) $Ca^{2+}$ ions. $\alpha$ and $\beta$ are the forward and backward rate constants, respectively. This scheme results in the following differential equation for $w$, where now $w$ is the fraction of particles in the open state, assuming that the concentration of $Ca^{2+}$ is large enough that the reaction does not significantly change the store of intracellular $Ca^{2+}$: $$ { {\rm d}w \over {\rm dt}} = (\alpha (1 - w)[Ca^{2+}]_{in})^n - \beta w$$ The steady state value for $w$ ( the fraction of particles in the open state) as a function of the intracellular $Ca^{2+}$ concentration is then: $$ w_{\infty} = {(\alpha [Ca^{2+}]_{in})^n \over (\alpha [Ca^{2+}]_{in})^n + \beta} $$ The time constant for the differential equation is: $$ \tau_w = ((\alpha [Ca^{2+}]_{in})^n + \beta)^{-1} $$ The order of the binding reaction,$n$, that is the number of $Ca^{2+}$ binding sites per $w$ particle, determines the steepness of the previous two expressions, as a function of $ [Ca^{2+}]_{in}$. Given the constraints on the range for $[Ca^{2+}]_{shell.1}$ and $[Ca^{2+}]_{shell.2}$ during single and repetitive firing, $n$ was set to three for both the $I_C$ $w$ particle and the $I_{AHP}$ $w$ particle. On the other hand, as shall be presented shortly, the range of $Ca^{2+}$ concentrations for which the $I_{AHP}$ $w$ particle is activated is set to about one order of magnitude lower than that for the $I_C$ $w$ particle, since $I_C$ was exposed to the larger $[Ca^{2+}]_{shell.1}$ . \section{AHP Potassium Current - $I_{AHP}$} $I_{AHP}$ is a slow, $Ca^{2+}$-mediated $K^+$ current that underlies the long afterhyperpolarization (AHP). Typically the AHP is about 1 to 2 millivolts and lasts from 0.5 -- 3 seconds after a single spike. Adding $Ca^{2+}$ blockers or noradrenaline to the extracellular medium eliminates the AHP, and likewise markedly reduces the cell's accommodation to tonic stimulus. Since most of the data on the proposed $I_{AHP}$ is derived from various current clamp protocols, the model description of this current is based on that used in other models (Koch and Adams, 1986) and from heuristics derived from the properties of other currents, in particular $I_{Ca}$ and $I_{DR}$. The important relationship between the $I_{AHP}$ and $I_{DR}$ parameters arose when I attempted to simulate both the mAHP (mediated by $I_{DR}$) and the AHP according to data from Storm (). In addition, since $I_{AHP}$ is dependent on $Ca^{2+}$ entry, the derivation of this current and the dynamics of $[Ca]_{shell.1}$ and $[Ca]_{shell.2}$ was done simultaneously. In fact, it was determined that in order for the activation of $I_{AHP}$ to be delayed from the onset of the spike, it was necessary to introduce the second intracellular space (shell) that was described in Chapter 6. Such a relationship between $Ca^{2+}$ influx and the subsequent delayed activation of $I_{AHP}$ has been suggested in the literature (Lan-Ada-86). \subsection{Results} I propose that the conductance underlying $I_{AHP}$ is dependent both on $Ca^{2+}$ and voltage. The $Ca^{2+}$ dependence of this current is clearly demonstrated since the AHP is removed when $Ca^{2+}$ blockers are added, and construction of a reasonable model of $Ca^{2+}$ dynamics such that $I_{AHP}$ may be dependent on this is possible. The mechanism that I use for $Ca^{2+}$-mediation of $I_{AHP}$ is similar to that for $I_C$, that is the $I_{AHP}$ channel includes a single $Ca^{2+}$-binding $w$ particle, with the same binding reaction as shown in Equation x. Voltage-clamp studies (Lan-Ada-86) indicate that there is no voltage-dependent activation of $I_{AHP}$, however. This puts a greater constraint on the $Ca^{2+}$-mediated mechanism for this current since the activation necessary to underly the long, small hyperpolarization after a single spike is significantly less than that required to squelch rapid spikes after some delay in response to tonic stimulus. In particular, these requirements provided rather restricted constraints on the buildup of $Ca^{2+}$ during each spike in region of the $I_{AHP}$ channels, $shell.2$, and likewise the dependence of the $I_{AHP}$ $w$ particle on this localized concentration of $Ca^{2+}$ . On the other hand I have included two inactivation gating particles, $y$ and $z$. The rationale for the $y$ particle is based on two pieces of evidence. First, it has been reported that $Ca^{2+}$ spikes are insensitive to noradrenaline in protocols where $I_{DR}$ and $I_A$ have been blocked by TEA and 4-AP, respectively (Segal and Barker). The fact that these spikes are unchanged with the addition of noradrenaline implies that under this protocol $I_{AHP}$ is inactivated by some other mechanism, since presumably $I_{AHP}$ has not been disabled. Since the protocol involves a long (approximately 30 milliseconds) depolarization of the cell before the $Ca^{2+}$ spike, it was possible to include an inactivation particle for $I_{AHP}$ that was (a) fast enough to disable $I_{AHP}$ under these conditions, but (b) was slow enough so that normal spiking did not cause the $y$ particle to change states. A second indication for the voltage-dependent inactivation particle $y$ is consistent with the previous evidence, that is the amplitude and rate of rise of action potentials singly or in trains appears independent of the presence $I_{AHP}$. In particular, the size of the $I_{AHP}$ conductance necessary to repress repetitive firing is large enough to significantly effect the spike once threshold is achieved if this conductance remained during the spike. Such a role for $I_{AHP}$ has not been demonstrated. $y$ therefore causes $I_{AHP}$ to shut off during an action potential so that this current does not reduce the amplitude of the spike. The second inactivation particle, $z$, was included to account for the delayed peak seen in the large afterhyperpolarization that occurs after a long (greater than 100 ms) stimulus (Madison and Nicoll, 1982 and others). At rest, $z$ is partially closed. With a large, lengthy hyperpolarization the $z$ particle becomes more open, thereby slowly increasing $I_{AHP}$ and the magnitude of the sAHP, until the $Ca^{2+}$ in $shell.2$ eventually drains down to its resting level and subsequently shutting off $w$. The time constant for $z$ was set very slow above rest so that it did not change appreciably during firing. Below about -75 mV, however, the time constant approaches 120 milliseconds so that the desired role of $z$ during the sAHP is obtained. No voltage-dependence for $I_{AHP}$ has been noted in the literature. However, the dependence of $I_{AHP}$ on $Ca^{2+}$ influx may have precluded voltage-clamp experiments which might verify the voltage-dependencies indicated by the simulations. With the present formulation for $I_{AHP}$, this current plays an important role during repetitive firing by shutting off the spike train after several hundred milliseconds. This occurs primarily through the dependence of $I_{AHP}$ on $[Ca]_{shell.2}$, which slowly increases during repetitive firing. Eventually the rise of $[Ca]_{shell.2}$ causes $I_{AHP}$ to provide sufficient outward rectification for counter-acting the stimulus current and thus stop the cell from firing (Figure~\ref{f:ahp-spks}). The fact that $I_{AHP}$ is strongly activated by this protocol is indicated by the long hyperpolarization at the end of the stimulus (Madison and Nicoll, 1982, and see simulation of their results in Figure~\ref{f:ahp-spks}). Madison and Nicoll, 1982 Mad-Nic-82 report that noradrenaline blocks accommodation by selectively blocking $I_{AHP}$. The characteristics demonstrated by the model $I_{AHP}$ are in qualitative agreement with many of the characteristics reported in the literature (e.g. Lan-Ada-86 , Seg-Bar-86),, including the increased activation of $I_{AHP}$ with increasing numbers of spikes in a single train, delayed activation from onset of $Ca^{2+}$ influx, the role of $I_{AHP}$ in modulating repetitive firing, time constant for inactivation/deactivation of greater than one second, the apparent voltage insensitivity (the transition of $y$ and $z$ with sub-threshold depolarization is slow, and once $x$ is activated deactivation takes several seconds. The equation for $I_{AHP}$ is - $$I_{AHP}= y_{AHP}^2 \, z_{AHP}\, w_{ahp}\, (V - E_K)$$ \noindent where $$\overline g_{AHP} = 0.4 \, \mu \rm S$$ \begin{table} \centering \begin{tabular}{|c||c|c|c|c|c|c|c|}\hline Gating Variable & $z$ & $\gamma$ & $\alpha_0$ & $V_{{1 \over 2}}\,$(mV) & $\tau_0\,$(ms) & $\alpha_{Ca}^*$ & $\beta_{Ca}^{**}$ \\ \hline\hline $y$ (inactivation)& -15 & 0.2 & 0.01 & -50.0 & 1.0 & - & - \\ \hline $z$ (inactivation)& -12 & 1.0 & 0.0002 & -72.0 & 100.0 & - & - \\ \hline $w$ ($Ca^{2+}$-activation) & - & - & - & - & - & $50$ & 0.01 \\ \hline \end{tabular}\caption[Parameters of $I_{AHP}$ Gating Variables] {Parameters of $I_{AHP}$ Gating Variables. $* = (ms^{-1/3}mM^{-1})$, $** = (ms^{-1})$}\label{t:ahp}\end{table} Table \ref{t:ahp} lists the parameters for the $I_{AHP}$ gating variables. These are the rate functions for the activation variable, $x$, of $I_{AHP}$- $$\alpha_{y,AHP} = 0.015 \exp \biggl({(V + 50) 0.8 \cdot -15 \cdot F \over R T} \biggr)$$ $$\beta_{y,AHP} = 0.015 \exp \biggl({(-50 - V) 0.2 \cdot -15 \cdot F \over R T} \biggr)$$ These are the rate functions for the activation variable, $y$, of $I_{AHP}$- $$\alpha_{z,AHP} = 0.0002 \, (\gamma = 0)$$ $$\beta_{z,AHP} = 0.0002 \exp \biggl({(-72 - V) \cdot -12 \cdot F \over R T} \biggr)$$ Again, each $w$ particle was assumed to have three non-competitive $Ca^{2+}$ binding sites, all of which were either empty (corresponding to the closed state) or filled (corresponding to the open state). \end{document} From ingber at alumni.cco.caltech.edu Sat Jan 15 13:26:51 1994 From: ingber at alumni.cco.caltech.edu (Lester Ingber) Date: Sat, 15 Jan 1994 10:26:51 -0800 Subject: Preprint: Path-integral evolution of short-term memory Message-ID: <199401151826.KAA18504@alumni.cco.caltech.edu> The following is an abstract from a paper accepted for publication in Physical Review E. The preprint may be retrieved via anonymous ftp from ftp.caltech.edu [131.215.48.151] in the pub/ingber directory using instructions given below. The file, smni94_stm.ps.gz, is about 0.9 MBytes. Statistical mechanics of neocortical interactions: Path-integral evolution of short-term memory Lester Ingber Lester Ingber Research, P.O. Box 857, McLean, VA 22101 ingber at alumni.caltech.edu Previous papers in this series of statistical mechanics of neocortical interactions (SMNI) have detailed a development from the relatively microscopic scales of neurons up to the macroscopic scales as recorded by electroencephalography (EEG), requiring an intermediate mesocolumnar scale to be developed at the scale of minicolumns (~10^2 neurons) and macrocolumns (~10^5 neurons). Opportunity was taken to view SMNI as sets of statistical constraints, not necessarily describing specific synaptic or neuronal mechanisms, on neuronal interactions, on some aspects of short-term memory (STM), e.g., its capacity, stability and duration. A recently developed C-language code, PATHINT, provides a non-Monte Carlo technique for calculating the dynamic evolution of arbitrary-dimension (subject to computer resources) nonlinear Lagrangians, such as derived for the two- variable SMNI problem. Here, PATHINT is used to explicitly detail the evolution of the SMNI constraints on STM. Interactively [brackets signify machine prompts]: [your_machine%] ftp ftp.caltech.edu [Name (...):] anonymous [Password:] your_e-mail_address [ftp>] cd pub/ingber [ftp>] binary [ftp>] ls [ftp>] get smni94_stm.ps.gz [ftp>] quit This directory also contains the Adaptive Simulated Annealing (ASA) code, now at version 2.8, in ASA-shar, ASA-shar.Z, ASA.tar.gz, and ASA.zip formats. The 00index file contains an index of the other (p)reprints and information on getting gzip and unshar for DOS, MAC, UNIX, and VMS systems. To get on or off the ASA_list e-mailings, just send an e-mail to asa-request at alumni.caltech.edu with your request. If you do not have ftp access, get information on the FTPmail service by: mail ftpmail at decwrl.dec.com, and send only the word "help" in the body of the message. If any of the above are not possible, and if your mailer can handle large files (please test this first), the code or papers you require can be sent as uuencoded compressed files via electronic mail. If you have gzip, resulting in smaller files, please state this. Sorry, I cannot assume the task of mailing out hardcopies of code or papers. Lester || Prof. Lester Ingber || || Lester Ingber Research || || P.O. Box 857 E-Mail: ingber at alumni.caltech.edu || || McLean, VA 22101 Archive: ftp.caltech.edu:/pub/ingber || From harnad at Princeton.EDU Sat Jan 15 20:22:50 1994 From: harnad at Princeton.EDU (Stevan Harnad) Date: Sat, 15 Jan 94 20:22:50 EST Subject: EEG Models: Chaotic and Linear: PSYCOLOQUY Call for Commentary Message-ID: <9401160122.AA26789@clarity.Princeton.EDU> Note: This is a PSYCOLOQUY Call for Commentators, *not* a BBS Call: You are invited to submit a formal commentary on the target article whose abstract appears below. It has just been published in the refereed electronic journal PSYCOLOQUY. Instructions for retrieving the full article and for preparing a PSYCOLOQUY commentary appear after the abstract. All commentaries are refereed. TARGET ARTICLE AUTHOR'S RATIONALE FOR SOLICITING COMMENTARY The target article attempts to reconcile attractor neural network (ANN) theory with certain current models for the generation of the EEG as a step toward integrating ANN theory with gross observations of brain function. Emphasis is placed on symmetry of cortical connections at a macroscopic level as compared to symmetry at a microscopic level. We hope to elicit commentary on (1) the methodology of the experiments and simulations on which the work is based, (2) any contradictory experimental findings, (3) quantitative methods in anatomy required for further development, (4) other critiques of ANN applicability to global brain function. psycoloquy.93.4.60.EEG-chaos.1.wright Thursday 23 December 1993 ISSN 1055-0143 (53 parags, 12 equations, 3 figs, 62 refs, 1092 lines) PSYCOLOQUY is sponsored by the American Psychological Association (APA) Copyright 1993 JJ Wright, RR Kydd & DTJ Liley EEG MODELS: CHAOTIC AND LINEAR J.J. Wright, R.R. Kydd, D.T.J. Liley Department of Psychiatry and Behavioural Science, School of Medicine, University of Auckland, Auckland, New Zealand jwright at ccu1.auckland.ac.nz jjw at brain.physics.swin.oz.au ABSTRACT: Two complementary EEG models are considered. The first (Freeman 1991) predicts 40+ Hz oscillation and chaotic local dynamics. The second (Wright 1990) predicts propagating EEG waves exhibiting linear superposition, nondispersive transmission, and near-equilibrium dynamics, on the millimetric scale. Anatomical considerations indicate that these models must apply, respectively, to cortical neurons which are very asymmetrically coupled and to symmetric average couplings. Aspects of both are reconciled in a simulation which explains wave velocities, EEG harmonics, the 1/f spectrum of desynchronised EEG, and frequency-wavenumber spectra. Local dynamics can be compared to the attractor model of Amit and Tsodyks (1990) applied in conditions of highly asymmetric coupling. Nonspecific cortical afferents may confer an adiabatic energy landscape to the large-scale dynamics of cortex. KEYWORDS: chaos, EEG simulation, electroencephalogram, linear dynamics, neocortex, network symmetry, neurodynamics, pyramidal cell, wave velocity. ------------------------------------------------------------------- INSTRUCTIONS FOR PSYCOLOQUY COMMENTATORS Accepted PSYCOLOQUY target articles have been judged by 5-8 referees to be appropriate for Open Peer Commentary, the special service provided by PSYCOLOQUY to investigators in psychology, neuroscience, behavioral biology, cognitive sciences and philosophy who wish to solicit multiple responses from an international group of fellow specialists within and across these disciplines to a particularly significant and controversial piece of work. If you feel that you can contribute substantive criticism, interpretation, elaboration or pertinent complementary or supplementary material on a PSYCOLOQUY target article, you are invited to submit a formal electronic commentary. Please note that although commentaries are solicited and most will appear, acceptance cannot, of course, be guaranteed. 1. Before preparing your commentary, please read carefully the Instructions for Authors and Commentators and examine recent numbers of PSYCOLOQUY. 2. Commentaries should be limited to 200 lines (1800 words, references included). PSYCOLOQUY reserves the right to edit commentaries for relevance and style. In the interest of speed, commentators will only be sent the edited draft for review when there have been major editorial changes. Where judged necessary by the Editor, commentaries will be formally refereed. 3. Please provide a title for your commentary. As many commentators will address the same general topic, your title should be a distinctive one that reflects the gist of your specific contribution and is suitable for the kind of keyword indexing used in modern bibliographic retrieval systems. Each commentary should have a brief (~50-60 word) abstract 4. All paragraphs should be numbered consecutively. Line length should not exceed 72 characters. The commentary should begin with the title, your name and full institutional address (including zip code) and email address. References must be prepared in accordance with the examples given in the Instructions. Please read the sections of the Instruction for Authors concerning style, preparation and editing. PSYCOLOQUY is a refereed electronic journal (ISSN 1055-0143) sponsored on an experimental basis by the American Psychological Association and currently estimated to reach a readership of 36,000. PSYCOLOQUY publishes brief reports of new ideas and findings on which the author wishes to solicit rapid peer feedback, international and interdisciplinary ("Scholarly Skywriting"), in all areas of psychology and its related fields (biobehavioral, cognitive, neural, social, etc.) All contributions are refereed by members of PSYCOLOQUY's Editorial Board. Target article length should normally not exceed 500 lines [c. 4500 words]. Commentaries and responses should not exceed 200 lines [c. 1800 words]. All target articles, commentaries and responses must have (1) a short abstract (up to 100 words for target articles, shorter for commentaries and responses), (2) an indexable title, (3) the authors' full name(s) and institutional address(es). In addition, for target articles only: (4) 6-8 indexable keywords, (5) a separate statement of the authors' rationale for soliciting commentary (e.g., why would commentary be useful and of interest to the field? what kind of commentary do you expect to elicit?) and (6) a list of potential commentators (with their email addresses). All paragraphs should be numbered in articles, commentaries and responses (see format of already published articles in the PSYCOLOQUY archive; line length should be < 80 characters, no hyphenation). It is strongly recommended that all figures be designed so as to be screen-readable ascii. If this is not possible, the provisional solution is the less desirable hybrid one of submitting them as postscript files (or in some other universally available format) to be printed out locally by readers to supplement the screen-readable text of the article. PSYCOLOQUY also publishes multiple reviews of books in any of the above fields; these should normally be the same length as commentaries, but longer reviews will be considered as well. Book authors should submit a 500-line self-contained Precis of their book, in the format of a target article; if accepted, this will be published in PSYCOLOQUY together with a formal Call for Reviews (of the book, not the Precis). The author's publisher must agree in advance to furnish review copies to the reviewers selected. Authors of accepted manuscripts assign to PSYCOLOQUY the right to publish and distribute their text electronically and to archive and make it permanently retrievable electronically, but they retain the copyright, and after it has appeared in PSYCOLOQUY authors may republish their text in any way they wish -- electronic or print -- as long as they clearly acknowledge PSYCOLOQUY as its original locus of publication. However, except in very special cases, agreed upon in advance, contributions that have already been published or are being considered for publication elsewhere are not eligible to be considered for publication in PSYCOLOQUY, Please submit all material to psyc at pucc.bitnet or psyc at pucc.princeton.edu Anonymous ftp archive is DIRECTORY pub/harnad/Psycoloquy HOST princeton.edu ------------------------------------------------------------- To retrieve the file by ftp from a Unix/Internet site, type either: ftp princeton.edu or ftp 128.112.128.1 When you are asked for your login, type: anonymous Enter password as per instructions (make sure to include the specified @), and then change directories with: cd /pub/harnad/Psycoloquy/1993.volume.4 To show the available files, type: ls Next, retrieve the file you want with (for example): get psyc.93.4.60.EEG-chaos.1.wright [or you can abbreviate with: mget *wright When you have the file(s) you want, type: quit In case of doubt or difficulty, consult your system manager. A more elaborate version of these instructions for the U.K. is available on request (thanks to Brian Josephson)> These files can also be retrieved using gopher, archie, veronica, etc. ---------- Where the above procedures are not available (e.g. from Bitnet or other networks), there are two fileservers: ftpmail at decwrl.dec.com and bitftp at pucc.bitnet that will do the transfer for you. To one or the other of them, send the following one line message: help for instructions (which will be similar to the above, but will be in the form of a series of lines in an email message that ftpmail or bitftp will then execute for you). From harnad at Princeton.EDU Sat Jan 15 21:15:02 1994 From: harnad at Princeton.EDU (Stevan Harnad) Date: Sat, 15 Jan 94 21:15:02 EST Subject: Beyond Modularity: BBS Call for Book Reviewers Message-ID: <9401160215.AA27062@clarity.Princeton.EDU> Below is the abstract of a book that will be accorded multiple book review in Behavioral and Brain Sciences (BBS), an international, interdisciplinary journal that provides Open Peer Commentary on important and controversial current research in the biobehavioral and cognitive sciences. Reviewers must be current BBS Associates or nominated by a current BBS Associate. To be considered as a reviewer for this book, to suggest other appropriate reviewers, or for information about how to become a BBS Associate, please send email to: harnad at clarity.princeton.edu or harnad at pucc.bitnet or write to: BBS, 20 Nassau Street, #240, Princeton NJ 08542 [tel: 609-921-7771] To help us put together a balanced list of reviewers, please give some indication of the aspects of the topic on which you would bring your areas of expertise to bear if you are selected as a reviewer. Please also indicate whether you already have a copy of the book or will need one if you are selected. The author's article-length precis of the book is available for inspection by anonymous ftp according to the instructions that follow after the abstract. ____________________________________________________________________ BBS Multiple Book Review of: BEYOND MODULARITY: A DEVELOPMENTAL PERSPECTIVE ON COGNITIVE SCIENCE Cambridge, MA: MIT Press 1992 (234 pp.) Annette Karmiloff-Smith Cognitive Development Unit, Medical Research Council, 4 Taviton Street, London WC1H 0BT, U.K. Electronic Mail: annette at cdu.ucl.ac.uk KEYWORDS: cognitive development, connectionism, constructivism, developmental stages, Fodor, modularity, nativism, Piaget, representational redescription, species differences. ABSTRACT: Beyond Modularity attempts a synthesis of Fodor's anti-constructivist nativism and Piaget's anti-nativist constructivism. Contra Fodor, I argue that: (1) the study of cognitive development is essential to cognitive science, (2) the module/central processing dichotomy is too rigid, and (3) the mind does not begin with prespecified modules, but that development involves a gradual process of modularization. Contra Piaget, I argue that: (1) development rarely involves stage-like domain-general change, and (2) domain-specific predispositions give development a small but significant kickstart by focusing the infant's attention on proprietary inputs. Development does not stop at efficient learning. A fundamental aspect of human development ("Representational Redescription") is the hypothesized process by which information that is IN a cognitive system becomes progressively explicit knowledge TO that system. Development thus involves two complementary processes of progressive modularization and rendering explicit. Empirical findings on the child as linguist, physicist, mathematician, psychologist and notator are discussed in support of the theoretical framework. Each chapter concentrates first on the initial state of the infant mind/brain and on subsequent domain-specific learning in infancy and early childhood. They then go on to explore data on older children's problem solving and theory building, with particular focus on evolving cognitive flexibility. Throughout the book there is an emphasis on the status of representations underlying different capacities and on the multiple levels at which knowledge is stored and accessible. Finally, consideration is given to the need for more formal developmental models, and the Representational Redescription framework is compared with connectionist simulations of development. The concluding sections consider what is special about human cognition and offer some speculations about the status of representations underlying the structure of behavior in other species. -------------------------------------------------------------- To help you decide whether you would be an appropriate reviewer for this book, an electronic precis is retrievable by anonymous ftp from princeton.edu according to the instructions below (the filename is bbs.karmsmith). Please let us know, after having inspected it, what relevant expertise you feel you would bring to bear on what aspect of the article. Note that only the book, not the Precis, is the object of the reviews. ------------------------------------------------------------- To retrieve a file by ftp from a Unix/Internet site, type either: ftp princeton.edu or ftp 128.112.128.1 When you are asked for your login, type: anonymous Enter password as per instructions (make sure to include the specified @), and then change directories with: cd /pub/harnad/BBS To show the available files, type: ls Next, retrieve the file you want with (for example): get bbs.karmsmith When you have the file(s) you want, type: quit In case of doubt or difficulty, consult your system manager. A more elaborate version of these instructions for the U.K. is available on request (thanks to Brian Josephson)> These files can also be retrieved using gopher, archie, veronica, etc. ---------- Where the above procedures are not available (e.g. from Bitnet or other networks), there are two fileservers: ftpmail at decwrl.dec.com and bitftp at pucc.bitnet that will do the transfer for you. To one or the other of them, send the following one line message: help for instructions (which will be similar to the above, but will be in the form of a series of lines in an email message that ftpmail or bitftp will then execute for you). From harnad at Princeton.EDU Sun Jan 16 22:43:24 1994 From: harnad at Princeton.EDU (Stevan Harnad) Date: Sun, 16 Jan 94 22:43:24 EST Subject: Pattern Recognition Nets: PSYC Call for Book Reviewers Message-ID: <9401170343.AA03667@clarity.Princeton.EDU> CALL FOR BOOK REVIEWERS Below is the Precis of NEURAL NETWORKS FOR PATTERN RECOGNITION, by Albert Niigrin. This book has been selected for multiple review in PSYCOLOQUY. If you wish to submit a formal book review (see Instructions following Precis) please write to psyc at pucc.bitnet indicating what expertise you would bring to bear on reviewing the book if you were selected to review it (if you have never reviewed for PSYCOLOQUY or Behavioral & Brain Sciences before, it would be helpful if you could also append a copy of your CV to your message). If you are selected as one of the reviewers, you will be sent a copy of the book directly by the publisher (please let us know if you have a copy already). Reviews may also be submitted without invitation, but all reviews will be refereed. The author will reply to all accepted reviews. ----------------------------------------------------------------------- psycoloquy.93.4.2.pattern-recognition.1.nigrin Sunday 16 January 1994 ISSN 1055-0143 (34 paragraphs, 1 appendix, 1 table, 6 refs, 468 lines) PSYCOLOQUY is sponsored by the American Psychological Association (APA) Copyright 1994 Albert Nigrin Precis of: NEURAL NETWORKS FOR PATTERN RECOGNITION Albert Nigrin (1993) 8 chapters, 413 pages, Cambridge MA: The MIT Press Albert Nigrin Department of Computer Science and Information Systems The American University 4400 Massachusetts Avenue NW Washington DC 20016-8116 (202) 885-3145 [fax (202) 885-3155] nigrin at american.edu ABSTRACT: This Precis provides an overview of the book "Neural Networks for Pattern Recognition." First, it presents a list of properties that the author believes autonomous pattern classifiers should achieve. (These thirteen properties are also briefly discussed at the end.) It then describes the evolution of a self-organizing neural network called SONNET that was designed to satisfy those properties. It details the organization of (1) tutorial chapters that describe previous work; (2) chapters that present working neural networks for the context sensitive recognition of both spatial and temporal patterns; and (3) chapters that reorganize the mechanisms for competition to allow future networks to deal with synonymous and homonymic patterns in a distributed fashion. KEYWORDS: context sensitivity, machine learning, neural networks, pattern recognition, self-organization, synonymy 1. This book presents a self-organizing neural network called SONNET that has been designed to perform real-time pattern recognition. The book attempts to discover, through gedanken experiments, the fundamental properties that any pattern classifier should satisfy (see Table 1 and Appendix A below). It then proceeds to construct, step by step, a new neural network framework to achieve these properties. Although the framework described has not yet been fully implemented, a prototype network called SONNET 1 does exist. Simulations show that SONNET 1 can be used as a general purpose pattern classifier that can learn to recognize arbitrary spatial patterns (static patterns as in a snapshot) and segment temporal patterns (changing patterns as in speech) in an unsupervised fashion. Furthermore, SONNET 1 can learn new patterns without degrading the representations of previously classified patterns, even when patterns are allowed to be embedded within larger patterns. 2. The book can be subdivided into three major sections. The first section provides an introduction to neural networks for a general audience and presents the previous work upon which SONNET is based. The second section describes the structure of SONNET 1 and presents simulations to illustrate the operation of the network. And the third section describes a reorganization of the competitive structure of SONNET 1 to create more powerful networks that will achieve additional important properties. 3. The first segment consists of Chapters 1 and 2. After presenting a simplified network to introduce the subject to novices, Chapter 1 presents one possible definition for neural networks and an approach to designing them. The chapter then describes many of the fundamental properties that a neural network should achieve when it is being used for pattern classification. These properties are listed in Table 1 (reproduced from Nigrin, 1993) and are each briefly discussed in Appendix A below. _________________________________________________________________________ | | | A classification system should be able to: | | | | 1) self-organize using unsupervised learning. | | 2) form stable category codes. | | 3) operate under the presence of noise. | | 4) operate in real-time. | | 5) perform fast and slow learning. | | 6) scale well to large problems. | | 7) use feedback expectancies to bias classifications. | | 8) create arbitrarily coarse or tight classifications | | that are distortion insensitive. | | 9) perform context-sensitive recognition. | | 10) process multiple patterns simultaneously. | | 11) combine existing representations to create categories | | for novel patterns. | | 12) perform synonym processing. | | 13) unlearn or modify categories when necessary. | | | | TABLE 1 | |_______________________________________________________________________| 4. I believe that before one can construct (or understand) autonomous agents that can operate in real-world environments, one must design classification networks that satisfy all of the properties in Table 1. It is not easy to see how any of these properties could be pushed off to other components in a system, regardless of whether the architecture is used to classify higher level structures such as sentences or visual scenes, or lower level structures such as phonemes or feature detectors. For example, consider the problem of modeling language acquisition and recognition. It is illuminating to attempt to push off any of the above properties to a subsystem other than the classifying system and still account for human behavior without resorting to a homunculus or to circular arguments. 5. With a description of the goals for the book in hand, Chapter 2 begins the process of describing neural network mechanisms for achieving them. Chapter 2 presents a tutorial overview of the foundations underlying the neural networks in the book. The book presents only those mechanisms that are essential to SONNET. Alternative approaches such as backpropagation, Hopfield networks, or Kohonen networks are not discussed. The discourse begins at the level of the building blocks and discusses basic components such as cells and weights. It then describes some essential properties that must be achieved in short term memory (STM) and long term memory (LTM) and presents architectures that achieve them. 6. Chapter 2 also discusses how to incorporate these architectures into different networks. The two major networks described in the chapter are the ART networks of Carpenter and Grossberg (1987a, 1987b) and the masking field networks of Cohen and Grossberg (1986, 1987). The ART networks completely or partially achieve many important properties. They can self-organize using unsupervised learning; form stable category codes; operate in noise; operate in real-time; perform fast or slow learning; use feedback; and create tight or coarse classifications. The masking field is also an important architecture. It achieves a framework for achieving properties such as context sensitive recognition and simultaneous classification of multiple patterns. 7. After presenting the necessary groundwork, the book begins the presentation of the real-time network called SONNET, which is its main focus. Due to its complexity, the complete network has not yet been fully implemented. Instead, the implemented network contains simplifications that allowed it to be slowly built up and analyzed. These simplifications were also useful to allow the network to be completed within a reasonable time frame. However, they had the drawback of preventing the satisfaction of some important properties that will be achievable by the full network. 8. Chapter 3 presents the basic version of the model called SONNET 1, as it pertains to spatial patterns. This network merged the properties of the ART networks with those of the masking field networks. SONNET 1 either partially or totally achieved all but four of the properties listed in Table 1. (It did not use feedback, form distributed categories, perform synonym processing or unlearn classifications.) After the network is described, simulations are presented that show its behavior. Furthermore, simple improvements are described that could increase network performance. 9. To allow SONNET 1 to achieve these properties, several novel features were incorporated into the network. These included (among others) the following: (1) The network used a non-linear summing rule to allow the classifying nodes to reach decisions in real-time. This non-linear rule was similar to those found in networks using sigma-pi units. (2) A learning rule was used to allow the inhibitory weights to self-organize so that classifying nodes only competed with other nodes that represented similar patterns. This allowed the network to classify multiple patterns simultaneously. (3) Each node encoded two independent values in its output signal. The first output value represented the activity of the cell while the second value represented a confidence value that indicated how well the cell represented the input. The use of two output values allowed the network to form stable categories, even when input patterns were embedded within larger patterns. 10. Chapter 4 incorporates SONNET 1 into a framework that allows it to process temporal patterns. This chapter has several aspects. First, it shows how to design input fields that convert temporal sequences of events into classifiable spatial patterns of activity. Then, it describes how the use of feedback expectancies can help segment the sequences into reasonable length lists, and allow arbitrarily long sequences of events to be processed. 11. After describing the network, Chapter 4 presents simulations that show its operation. One of the simulations consisted of presenting the following list to the network, where each number refers to a specific input line. The list was presented by activating each input line for a constant period of time upon the presentation of its item. After the last item in the list was presented, the first item was immediately presented again, with no breaks between any of the items. 0 1 2 3 4 5 24 25 26 6 7 8 9 0 1 2 10 11 12 13 24 25 26 14 15 16 0 1 2 17 18 19 24 25 26 20 21 22 23 12. In this list, items (0,1,2) and (24,25,26) appear in three different contexts. Because of this, the network learned to create categories for those lists and to segment them accordingly. Thus, it learned in a real-time environment. It was also clear that it performed classifications in real-time since each of the lists was classified approximately 2 items after it had been fully presented. For example, if the list 22 23 0 1 2 3 4 5 6 was presented, the list (0,1,2) would be classified while item 4 or 5 was being presented. Simulations have shown that the amount of equilibration time needed for classification would not increase significantly, even if multiple similar patterns were classified by the network. 13. Chapter 5 continues to discuss the classification of temporal patterns. (However, many elements in this chapter are also applicable to purely spatial patterns.) The chapter shows how to cascade multiple homologous layers to create a hierarchy of representations. It also shows how to use feedback to bias the network in favor of expected occurrences and how to use a nonspecific attention signal to increase the power of the network. As is the case with the networks in later chapters, these proposed modifications are presented but not simulated. 14. One major limitation of the networks presented in Chapters 4 and 5 is that items can be presented only once within a classified list. For example, the list $ABC$ can be classified by the network, but the list $ABA$ cannot, since the $A$ occurs repeatedly. This deficiency is due to the simplifications that were made in the construction of SONNET 1. To overcome this and other weaknesses, the simplifications needed to be removed. 15. This is accomplished in Chapter 6, which presents a gedanken experiment analyzing the way repeated items in a list could be properly represented and classified. The chapter begins by showing that multiple representations of the same item are needed to allow the network to unambiguously represent the repeated occurrence of an item. It then analyzes methods by which the classifying system could learn to classify lists composed of these different representations. 16. During this gedanken experiment, it quickly became clear that the problem of classifying repeated items in a list was actually a subproblem of a more general one, called the synonym problem: Often, different input representations actually refer to the same concept and should therefore be treated by classifying cells as equivalent. However, the problem is complicated by the fact that sometimes different patterns refer to the same concept while sometimes the same pattern may have multiple meanings (homonyms). 17. To address the synonym problem, Chapter 6 presents a way to radically alter the method of competition between categories. In SONNET 1 (as in most competitive networks), classifying nodes compete with each other for the right to classify signals on active input lines. Conversely, in the altered network, it is the input lines that will compete with each other, and they will do so for the right to activate their respective classifying nodes. The principles in Chapter 6 are far and away the most important new contribution in this book. 18. After showing how synonyms could be learned and represented, Chapter 6 also discusses general mechanisms for creating distributed representations. These mechanisms were designed to allow existing representations to combine in STM (short-term memory) to temporarily represent novel patterns. They were also designed to allow the novel categories to be permanently bound in LTM (long-term memory). 19. After establishing the new mechanisms and principles in Chapter 6, these mechanisms are used in Chapter 7 to create specific architectures that tackle previously unsolved problems. The first section discusses the first implementation of SONNET that uses competition between links rather than nodes; it and shows how multiple patterns could be learned simultaneously. To complement the discussion in the previous chapter, the discussion here is as specific as possible (given that the network was yet to be implemented). The second section discusses how the new formulation could allow networks to solve the twin problems of translation and size invariant recognition of objects. This shows how the new mechanisms could be used to solve an important previously unresolved issue. 20. Finally, Chapter 8 concludes the book. It describes which properties have already been satisfied by SONNET 1, which properties can be satisfied by simple extensions to SONNET 1, and which properties must wait until future versions of SONNET are implemented. This chapter gives the reader a good indication of the current state of the network and also indicates areas for future research. 21. The following briefly summarizes thirteen properties that SONNET is meant to satisfy. Although it is possible to find examples in many different areas to motivate each of the following properties, the examples are mainly chosen from the area of natural language processing. This is done because the problems in this area are the easiest to describe and are often the most compelling. However, the reader should keep in mind that equivalent properties also exist in other domains and that, at least initially, SONNET is meant to be used primarily for lower level classification problems. 22. The first property is that a neural network should self-organize using unsupervised learning. It should form its own categories in response to the invariances in the environment. This allows the network to operate in an autonomous fashion and is important because in many areas, such as lower level perception, no external teacher is available to guide the system. Furthermore, as shown in the ARTMAP network (Carpenter, Grossberg, and Reynolds, 1991), it is often the case that if a network can perform unsupervised learning then it can also be embedded in a framework that allows it to perform supervised learning (but not the reverse). 23. The second property is that a neural network should form stable category codes. Thus, a neural network should learn new categories without degrading previous categories it has established. Networks that achieve this property can operate using both fast and slow learning (see fifth property). Conversely, those that do not are restricted to using slow learning. In addition, networks that don't form stable category codes must shut off learning at some point in time to prevent the degradation of useful categories. 24. The third property is that neural networks should operate in the presence of noise. This is necessary to allow them to operate in real-world environments. Noise can occur in three different areas. It can be present within an object, within the background of an object, and within the components of the system. A network must handle noise in all of these areas. 25. The fourth property is that a neural network should operate in real-time. There are several aspects to this. The first and most often recognized is that a net must equilibrate at least as fast as the patterns appear. However, there are several additional aspects to this property. First, in many applications, such as speech recognition and motion detection, a network should not equilibrate too rapidly, but at a pace that matches the evolution of the patterns. Second, in real-world environments, events do not come pre-labeled with markers designating the beginnings and endings of the events. Instead, the networks themselves must determine the beginning and end to each event and act accordingly. 26. The fifth property is that a neural network should perform fast and slow learning. A network should perform fast learning to allow it to classify patterns as quickly as a single trial when it is clear exactly what should be learned and it is important that the network learn quickly. (For example, one should not have to touch a hot stove 500 times before learning one will be burnt.) Furthermore, a network should also perform slow learning to allow it to generalize over multiple different examples. 27. The sixth property is that a neural network should scale well to large problems. There are at least two aspects to this property. First, as the size of a problem grows, the size of the required network should not grow too quickly. (While modularity may help in this respect, it is not a panacea, because of problems with locality and simultaneous processing.) Second, as the number of different patterns in a training set increases, the number of required presentations for each pattern (to obtain successful classifications) should not increase too rapidly. 28. The seventh property is that a neural network should use feedback expectancies to bias classifications. This is necessary because it is often ambiguous how to bind features into a category unless there is some context with which to place the features. 29. The eighth property is that a neural network should create arbitrarily coarse or tight classifications that are distortion insensitive. Patterns in a category often differ from the prototype (average) of the category. A network should vary the acceptable distortion from the prototype in at least two ways. It should globally vary the acceptable overall error. It should also allow different amounts of variance at different dimensions of the input pattern (the different input lines). This would allow the network to create categories that are more complex than just the nearest neighbor variety. 30. The ninth property is that a neural network should perform context-sensitive recognition. Two aspects of this will be discussed here. First, a network should learn and detect patterns that are embedded within extraneous information. For example, if the patterns SEEITRUN, ITSAT, and MOVEIT are presented, a network should establish a category for IT and later recognize the pattern when it appears within extraneous information. The second aspect occurs when a smaller classified pattern is embedded within a larger classified pattern. Then, the category for the smaller pattern should be turned off when the larger pattern is classified. For example, if a network has a category for a larger word like ITALY, then the category for IT should be turned off when the larger word is presented. Otherwise the category for IT would lose much of its predictive power, because it would learn the contexts of many non-related words such as HIT, KIT, SPIT, FIT, LIT, SIT, etc. 31. The tenth property is that a neural network should process multiple patterns simultaneously. This is important, because objects in the real world do not appear in isolation. Instead, scenes are cluttered with multiple objects that often overlap. To have any hope of segmenting a scene in real time, multiple objects often need to be classified in parallel. Furthermore, the parallel classifications must interact with one another, since it is often true that the segmentation for an object can only be determined by defining it in relation to other objects in the field. (Thus, it is not sufficient to use multiple stand-alone systems that each attempt to classify a single object in some selected portion of the input field.) The easiest modality in which to observe this is continuous speech, which often has no clear breaks between any words. (However, analogous situations also occur in vision.) For example, when the phrase ALL TURN TO THE SPEAKER is spoken, there is usually no break in the speech signal between the words ALL and TURN. Still, those words are perceived, rather than the embedded word ALTER. This can only be done by processing multiple patterns simultaneously, since the word ALTER by itself would overshadow both ALL and TURN. 32. The eleventh property is that a neural network should combine existing representations to create categories for novel patterns. These types of representations are typically called distributed ones. A network must form temporary representations in short term memory (STM) and also permanent iones in long term memory (LTM). Distributed representations are useful because they can reduce hardware requirements and also allow novel patterns to be represented as a combination of constituent parts. 33. The twelfth property is that a neural network should perform synonym processing. This is true because patterns that have entirely different physical attributes often have the same meaning, while a single pattern may have multiple meanings (as in homonyms). This is especially recognized in natural language, where words like "mean" and "average" sometimes refer to the same concept, and sometimes do not. However, solving the synonym problem will also solve problems that occur in the processing of lists composed of repeated occurrences of the same symbol (consider the letters "a" and "n" in the word "banana"). This follows because the different storage locations of a symbol can be viewed as (exact) synonyms for each other and handled in exactly the same way as the general case. Synonym representation is also necessary in object recognition, manifesting itself in several different ways. First, it is possible for multiple versions of the same object to appear within a scene (similar to the problem of repeated letters in a word). Second, since an object may appear completely when viewed different from different perspectives, it is important to map the dissimilar representations of the object onto the same category. Finally, it is also possible for an object to appear in different portions of the visual field (translation-invariant recognition) or with different apparent sizes (size-invariant recognition). Despite the fact that in both cases the object will be represented by entirely different sets of cells, a network should still classify the object correctly. 34. The thirteenth property is that a neural network should unlearn or modify categories when necessary. It should modify its categories passively to allow it to track slow changes in the environment. A network should also quickly change the meanings for its categories when the environment changes and renders them either superfluous or wrong. This property is the one least that ius discussed in the book, because it is possible that much unlearning could take place under the guise of reinforcement learning. APPENDIX: Table of Contents 1 Introduction 2 Highlights of Adaptive Resonance Theory 3 Classifying Spatial Patterns 4 Classifying Temporal Patterns 5 Multilayer Networks and the Use of Attention 6 Representing Synonyms 7 Specific Architectures That Use Presynaptic Inhibition 8 Conclusion Appendices REFERENCES Carpenter, G. and Grossberg, S. 1987a. A Massively Parallel Architecture for a Self-organizing Neural Pattern Recognition Machine. Computer Vision, Graphics, and Image Processing, 37:54--115. Carpenter, G. and Grossberg, S. 1987b. ART 2: Self-organization of Stable Category Recognition Codes for Analog Input Patterns. Applied Optics, 26(23):4919--4930. Carpenter,G., Grossberg, S., and Reynolds, J. 1991. ARTMAP: Supervised Real-time Learning and Classification of Nonstationary Data by a Self-organizing Neural Network. Neural Networks, 4(5):565-588. Cohen, M. and Grossberg, S. 1986. Neural Dynamics of Speech and Language Coding: Developmental Programs, Perceptual Grouping, and Competition for Short-term Memory. Human Neurobiology, 5(1):1--22. Cohen, M. and Grossberg, S. 1987. Masking Fields: a Massively Parallel Neural Architecture for Learning, Recognizing, and Predicting Multiple Groupings of Data. Applied Optics, 26:1866--1891. Nigrin, A. 1993. Neural Networks for Pattern Recognition. The MIT Press, Cambridge MA. -------------------------------------------------------------------- PSYCOLOQUY Book Review Instructions The PSYCOLOQUY book review procedure is very similar to the commentary procedure except that it is the book itself, not a target article, that is under review. (The Precis summarizing the book is intended to permit PSYCOLOQUY readers who have not read the book to assess the exchange, but the reviews should address the book, not primarily the Precis.) Note that as multiple reviews will be co-appearing, you need only comment on the aspects of the book relevant to your own specialty and interests, not necessarily the book in its entirety. Any substantive comments and criticism -- including points calling for a detailed and substantive response from the author -- are appropriate. Hence, investigators who have already reviewed or intend to review this book elsewhere are still encouraged to submit a PSYCOLOQUY review specifically written with this specialized multilateral review-and-response feature in mind. 1. Before preparing your review, please read carefully the Instructions for Authors and Commentators and examine recent numbers of PSYCOLOQUY. 2. Reviews should not exceed 500 lines. Where judged necessary by the Editor, reviews will be formally refereed. 3. Please provide a title for your review. As many commentators will address the same general topic, your title should be a distinctive one that reflects the gist of your specific contribution and is suitable for the kind of keyword indexing used in modern bibliographic retrieval systems. Each review should also have a brief (~50-60 word) Abstract 4. All paragraphs should be numbered consecutively. Line length should not exceed 72 characters. The review should begin with the title, your name and full institutional address (including zip code) and email address. References must be prepared in accordance with the examples given in the Instructions. Please read the sections of the Instruction for Authors concerning style, preparation and editing. PSYCOLOQUY is a refereed electronic journal (ISSN 1055-0143) sponsored on an experimental basis by the American Psychological Association and currently estimated to reach a readership of 20,000. PSYCOLOQUY publishes brief reports of new ideas and findings on which the author wishes to solicit rapid peer feedback, international and interdisciplinary ("Scholarly Skywriting"), in all areas of psychology and its related fields (biobehavioral, cognitive, neural, social, etc.) All contributions are refereed by members of PSYCOLOQUY's Editorial Board. Target article length should normally not exceed 500 lines [c. 4500 words]. Commentaries and responses should not exceed 200 lines [c. 1800 words]. All target articles, commentaries and responses must have (1) a short abstract (up to 100 words for target articles, shorter for commentaries and responses), (2) an indexable title, (3) the authors' full name(s) and institutional address(es). In addition, for target articles only: (4) 6-8 indexable keywords, (5) a separate statement of the authors' rationale for soliciting commentary (e.g., why would commentary be useful and of interest to the field? what kind of commentary do you expect to elicit?) and (6) a list of potential commentators (with their email addresses). All paragraphs should be numbered in articles, commentaries and responses (see format of already published articles in the PSYCOLOQUY archive; line length should be < 80 characters, no hyphenation). It is strongly recommended that all figures be designed so as to be screen-readable ascii. If this is not possible, the provisional solution is the less desirable hybrid one of submitting them as postscript files (or in some other universally available format) to be printed out locally by readers to supplement the screen-readable text of the article. PSYCOLOQUY also publishes multiple reviews of books in any of the above fields; these should normally be the same length as commentaries, but longer reviews will be considered as well. Book authors should submit a 500-line self-contained Precis of their book, in the format of a target article; if accepted, this will be published in PSYCOLOQUY together with a formal Call for Reviews (of the book, not the Precis). The author's publisher must agree in advance to furnish review copies to the reviewers selected. Authors of accepted manuscripts assign to PSYCOLOQUY the right to publish and distribute their text electronically and to archive and make it permanently retrievable electronically, but they retain the copyright, and after it has appeared in PSYCOLOQUY authors may republish their text in any way they wish -- electronic or print -- as long as they clearly acknowledge PSYCOLOQUY as its original locus of publication. However, except in very special cases, agreed upon in advance, contributions that have already been published or are being considered for publication elsewhere are not eligible to be considered for publication in PSYCOLOQUY, Please submit all material to psyc at pucc.bitnet or psyc at pucc.princeton.edu Anonymous ftp archive is DIRECTORY pub/harnad/Psycoloquy HOST princeton.edu From qin at turtle.fisher.com Mon Jan 17 09:56:58 1994 From: qin at turtle.fisher.com (qin@turtle.fisher.com) Date: Mon, 17 Jan 94 09:56:58 CDT Subject: Call For Papers Message-ID: <00978AA7A931E520.68601F60@turtle.fisher.com> The IEEE International Conference on Neural Networks is to be held in Orlando, Florida, June 26 - July 2, 1994. This conference is part of the World Congress on Computational Intelligence. This Announcement is to call for papers for a Special Session on "Neural Networks for Control". Papers related to using neural networks for control, system identification, fault detection and diagnosis are welcome, but not limitted to these areas. The suggested paper length is four pages. The maximum paper length is 6 pages. The deadline for submitting papers is January 31, 1994. All papers should be submitted to one of the two session organizers: Dr. S. Joe Qin Fisher-Rosemount Systems, Inc. 1712 Centre Creek Drive Austin, TX 78754 Tel 512-832-3635 FAX 512-834-7200 qin at fisher.com Dr. Ching-Fang Lin, President American GNC Corporation 9131 Mason Avenue Chatsworth, CA 91311 Tel 818-407-0092 FAX 818-407-0093 american_gnc at cup.portal.com FAX submissions are acceptable. From rupa at dendrite.cs.colorado.edu Mon Jan 17 12:38:10 1994 From: rupa at dendrite.cs.colorado.edu (Sreerupa Das) Date: Mon, 17 Jan 1994 10:38:10 -0700 Subject: NIPS preprint available via neuroprose Message-ID: <199401171738.AA08039@pons.cs.Colorado.EDU> FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/das.dolce.ps.Z Number of pages: 8 The following paper is now available for copying from the Neuroprose archive. Only electronic version of this paper is available. This is a preprint of the paper to appear in J.D. Cowan, G. Tesauro, and J. Alspector (eds.) Advances in Neural Information Processing Systems 6, 1994. A Unified Gradient-Descent/Clustering Architecture for Finite State Machine Induction Sreerupa Das and Michael C. Mozer Department of Computer Science University of Colorado at Boulder CO 80309--0430 ABSTRACT Although recurrent neural nets have been moderately successful in learning to emulate finite-state machines (FSMs), the continuous internal state dynamics of a neural net are not well matched to the discrete behavior of an FSM. We describe an architecture, called DOLCE, that allows discrete states to evolve in a net as learning progresses. DOLCE consists of a standard recurrent neural net trained by gradient descent and an adaptive clustering technique that quantizes the state space. DOLCE is based on the assumption that a finite set of discrete internal states is required for the task, and that the actual network state belongs to this set but has been corrupted by noise due to inaccuracy in the weights. DOLCE learns to recover the discrete state with maximum a posteriori probability from the noisy state. Simulations show that DOLCE leads to a significant improvement in generalization performance over earlier neural net approaches to FSM induction. ====================================================================== FTP procedure: unix> ftp archive.cis.ohio-state.edu (or 128.146.8.52) Name: anonymous Password: your_email_address ftp> cd pub/neuroprose ftp> binary ftp> get das.dolce.ps.Z ftp> quit unix> uncompress das.dolce.ps.Z unix> lpr das.dolce.ps Thanks to Jordan Pollack for maintaining the archive! Sreerupa Das Department of Computer Science University of Colorado at Boulder CO 80309-0430 email: rupa at cs.colorado.edu From zemel at salk.edu Mon Jan 17 17:18:38 1994 From: zemel at salk.edu (Richard S. Zemel) Date: Mon, 17 Jan 94 14:18:38 PST Subject: 2 NIPS preprints on neuroprose Message-ID: <9401172218.AA29430@broca> **DO NOT FORWARD TO OTHER GROUPS** The following two papers have been placed in the neuroprose archive. The first prints on 9 pages, the second on 8. The abstracts are given below, followed by retrieval instructions. Only electronic versions of these papers are available. Both are to appear in J.D. Cowan, G. Tesauro, and J. Alspector (Eds.), Advances in Neural Information Processing Systems 6, San Mateo, CA: Morgan Kaufmann. Rich Zemel e-mail: zemel at salk.edu FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/hinton.autoencoders.ps.Z FTP-filename: /pub/neuroprose/zemel.pop-codes.ps.Z ----------------------------------------------------------------------------- Autoencoders, Minimum Description Length and Helmholtz Free Energy Geoffrey E. Hinton and Richard S. Zemel An autoencoder network uses a set of {\it recognition} weights to convert an input vector into a code vector. It then uses a set of {\it generative} weights to convert the code vector into an approximate reconstruction of the input vector. We derive an objective function for training autoencoders based on the Minimum Description Length (MDL) principle. The aim is to minimize the information required to describe both the code vector and the reconstruction error. We show that this information is minimized by choosing code vectors stochastically according to a Boltzmann distribution, where the generative weights define the energy of each possible code vector given the input vector. Unfortunately, if the code vectors use distributed representations, it is exponentially expensive to compute this Boltzmann distribution because it involves all possible code vectors. We show that the recognition weights of an autoencoder can be used to compute an approximation to the Boltzmann distribution and that this approximation gives an upper bound on the description length. Even when this bound is poor, it can be used as a Lyapunov function for learning both the generative and the recognition weights. We demonstrate that this approach can be used to learn factorial codes. ----------------------------------------------------------------------------- Developing Population Codes By Minimizing Description Length Richard S. Zemel and Geoffrey E. Hinton The Minimum Description Length principle (MDL) can be used to train the hidden units of a neural network to extract a representation that is cheap to describe but nonetheless allows the input to be reconstructed accurately. We show how MDL can be used to develop highly redundant population codes. Each hidden unit has a location in a low-dimensional {\em implicit} space. If the hidden unit activities form a bump of a standard shape in this space, they can be cheaply encoded by the center of this bump. So the weights from the input units to the hidden units in an autoencoder are trained to make the activities form a standard bump. The coordinates of the hidden units in the implicit space are also learned, thus allowing flexibility, as the network develops a discontinuous topography when presented with different input classes. Population-coding in a space other than the input enables a network to extract nonlinear higher-order properties of the inputs. ----------------------------------------------------------------------------- To retrieve from neuroprose: unix> ftp cheops.cis.ohio-state.edu Name (cheops.cis.ohio-state.edu:zemel): anonymous Password: (use your email address) ftp> cd pub/neuroprose ftp> get zemel.pop-codes.ps.Z ftp> get hinton.autoencoders.ps.Z ftp> quit unix> uncompress zemel.pop-codes.ps unix> uncompress hinton.autoencoders.ps unix> lpr zemel.pop-codes.ps unix> lpr hinton.autoencoders.ps From schraudo at salk.edu Mon Jan 17 23:53:20 1994 From: schraudo at salk.edu (Nici Schraudolph) Date: Mon, 17 Jan 94 20:53:20 PST Subject: NIPS preprint available Message-ID: <9401180453.AA15851@salk.edu> Temporal Difference Learning of Position Evaluation in the Game of Go --------------------------------------------------------------------- Nicol N. Schraudolph Peter Dayan Terrence J. Sejnowski Computational Neurobiology Laboratory The Salk Institute for Biological Studies San Diego, CA 92186-5800 Abstract: The game of Go has a high branching factor that defeats the tree search approach used in computer chess, and long-range spatiotemporal inter- actions that make position evaluation extremely difficult. Development of conventional Go programs is hampered by their knowledge-intensive nature. We demonstrate a viable alternative by training networks to evaluate Go positions via temporal difference (TD) learning. Our approach is based on network architectures that reflect the spatial organization of both input and reinforcement signals on the Go board, and training protocols that provide exposure to competent (though un- labelled) play. These techniques yield far better performance than undifferentiated networks trained by self-play alone. A network with less than 500 weights learned within 3,000 games of 9x9 Go a position evaluation function that enables a primitive one-ply search to defeat a commercial Go program at a low playing level. -------- A preprint of the above paper is available by anonymous ftp from salk.edu (192.31.153.101), file pub/schraudo/nips93.ps.Z. (If you do not have ftp access to the Internet, send the message "help" to ftpmail at decwrl.dec.com for information on ftp-by-email service.) From D.Gorse at cs.ucl.ac.uk Tue Jan 18 13:36:32 1994 From: D.Gorse at cs.ucl.ac.uk (D.Gorse@cs.ucl.ac.uk) Date: Tue, 18 Jan 94 18:36:32 +0000 Subject: New preprint in neuroprose - avoiding local minima by homotopy Message-ID: FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/gorse.homotopy.ps.Z The file gorse.homotopy.ps.Z is now available for copying from the Neuroprose archive. This is a 6 page paper, submitted to WCNN '94 San Diego. A longer and more detailed paper describing this work is in preparation and will be available soon. --------------------------------------------------------------------------- A CLASSICAL ALGORITHM FOR AVOIDING LOCAL MINIMA D Gorse and A Shepherd Department of Computer Science University College, Gower Street, London WC1E 6BT, UK J G Taylor Department of Mathematics King's College, Strand, London WC2R 2LS, UK ABSTRACT: Conventional methods of supervised learning are inevitably faced with the problem of local minima; evidence is presented that conjugate gradient and quasi-Newton techniques are particularly susceptible to being trapped in sub-optimal solutions. A new classical technique is presented which by the use of a homotopy on the range of the target outputs allows supervised learning methods to find a global minimum of the error function in almost every case. Denise Gorse (D.Gorse at cs.ucl.ac.uk) ---------------------------------------------------------------------------- To obtain a copy: ftp archive.cis.ohio-state.edu login: anonymous password: cd pub/neuroprose binary get gorse.homotopy.ps.Z quit Then at your system: uncompress gorse.homotopy.ps.Z lpr -P gorse.homotopy.ps From N.Sharkey at dcs.shef.ac.uk Tue Jan 18 07:14:03 1994 From: N.Sharkey at dcs.shef.ac.uk (N.Sharkey@dcs.shef.ac.uk) Date: Tue, 18 Jan 94 12:14:03 GMT Subject: ADDRESS CHANGE Message-ID: <9401181214.AA04945@entropy.dcs.shef.ac.uk> ********************************* * * * CONNECTION SCIENCE * * ADDRESS CHANGE * * * ********************************* Please note that the Headquarters of Journal: Connection Science has moved to Sheffield University. The Journal is now into Volume 6 and is still going strong thanks to all of the support from the connectionist community. Because of the move, Lyn Shakelton, who has served as assistant editor since the beginning, is no longer with us. She has been replaced by a new Editorial Assistant Julie Clarke. I am sorry for any delay in responding to correspondence or in dealing with manuscripts that the move has caused. Bear with us. SUBMISSIONS SHOULD NOW BE SENT TO: Julie Clarke Connection Science Department of Computer Science Regent Court University of Sheffield S1 4DP, Sheffield, UK j.clarke at dcs.shef.ac.uk (or username julie) VOLUNTEER REVIEWERS: We have had extensive help from a number of reviewers of the past 5 years and we have worn some of them down to the bone. We are now trying to update our review panel to give some of the others a bit of a rest. If you wish to volunteer please contact Julie at the above address. We will be eternally grateful for your assistance. For other queries please contact me ************************************ * * * Professor Noel Sharkey * * Department of Computer Science * * Regent Court * * University of Sheffield * * S1 4DP, Sheffield, UK * * * * N.Sharkey at dcs.shef.ac.uk * * * ************************************ From hzs at cns.brown.edu Tue Jan 18 16:51:48 1994 From: hzs at cns.brown.edu (Harel Z. Shouval) Date: Tue, 18 Jan 1994 16:51:48 -0500 (EST) Subject: Correction Message-ID: <9401182151.AA26962@cns.brown.edu> A non-text attachment was scrubbed... Name: not available Type: text Size: 709 bytes Desc: not available Url : https://mailman.srv.cs.cmu.edu/mailman/private/connectionists/attachments/00000000/f4256760/attachment-0001.ksh From kirk at FIZ.HUJI.AC.IL Wed Jan 19 05:34:00 1994 From: kirk at FIZ.HUJI.AC.IL (Scott Kirkpatrick) Date: Wed, 19 Jan 1994 12:34:00 +0200 Subject: preprints available on Neuroprose Message-ID: <199401191034.AA15823@binah.fiz.huji.ac.il> **DO NOT FORWARD TO OTHER GROUPS** FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/kirkpatrick.critical.ps.Z FTP-file: pub/neuroprose/kirkpatrick.nips93-statistical.ps.Z Only soft copy is available. The two above preprints are available for anonymous ftp in the Neuroprose archive. Full titles and authors are: Critical Behavior at the k-Satisfaction Threshold (preprint), by Scott Kirkpatrick and Bart Selman The Statistical Mechanics of k-Satisfaction, (NIPS-6 preprint) by S. Kirkpatrick, G. Gyorgyi, N. Tishby and L. Troyansky abstracts follow: {Critical Behavior at the $k$-Satisfiability Threshold} The satisfiability of random Boolean formulae with precisely $k$ variables per clause is a popular testbed for the performance of search algorithms in artificial intelligence and computer science. For $k = 2$, formulae are almost aways satisfiable when the ratio of clauses to variables is less than 1; for ratios larger than 1, the formulae are almost never satisfiable. We present data showing a similar threshold behavior for higher values of $k$. We also show how finite-size scaling, a method from statistical physics, can be used to characterize size dependent effects near the threshold. Finally, we commment on the relationship between thresholds and computational complexity. {The Statistical Mechanics of $k$-Satisfaction} The satisfiability of random CNF formulae with precisely $k$ variables per clause (``$k$-SAT'') is a popular testbed for the performance of search algorithms. Formulae have $M$ clauses from $N$ variables, randomly negated, keeping the ratio $\alpha = M/N$ fixed. For $k = 2$, this model has been proven to have a sharp threshold at $\alpha = 1$ between formulae which are almost aways satisfiable and formulae which are almost never satisfiable as $N \rightarrow \infty$. Computer experiments for $k$ = 2, 3, 4, 5 and 6, (carried out in collaboration with B. Selman of ATT Bell Labs) show similar threshold behavior for each value of $k$. Finite-size scaling, a theory of the critical point phenomena used in statistical physics, is shown to characterize the size dependence near the threshold. Annealed and replica-based mean field theories give a good account of the results. From hzs at cns.brown.edu Wed Jan 19 10:18:15 1994 From: hzs at cns.brown.edu (Harel Z. Shouval) Date: Wed, 19 Jan 1994 10:18:15 -0500 (EST) Subject: Correction to Correction Message-ID: <9401191518.AA29437@cns.brown.edu> A non-text attachment was scrubbed... Name: not available Type: text Size: 758 bytes Desc: not available Url : https://mailman.srv.cs.cmu.edu/mailman/private/connectionists/attachments/00000000/ae7ff432/attachment-0001.ksh From skalsky at aaai.org Fri Jan 21 10:17:27 1994 From: skalsky at aaai.org (Rick Skalsky) Date: Fri, 21 Jan 94 07:17:27 PST Subject: AAAI-94 Special Notice Message-ID: <9401211517.AA17186@aaai.org> ----------------------------------------------------------------------------- Due to the earthquake in Los Angeles, and the severe winter weather in the North-Eastern U.S. and Canada, the deadline for the receipt of AAAI papers is being extended to Friday, January 28, 1994, for those authors that are severely impacted by these events. In order to minimize disruption of the self-selection and review process, however, the title pages, including paper title, authors, addresses, content areas, and abstract, must still arrive at the AAAI office by Monday, January 24th. These should be sent electronically, to abstract at aaai.org, in the format specified in the call for papers. This extension applies only to those individuals whose ability to complete their papers on time was severely impacted by these events, and authors are requested and expected to be honest in their use of it. If an electronic title page is submitted, but a decision is later made not to submit the paper, please send a message to abstract at aaai.org informing us of this fact as soon as possible. Please forward this message to anyone you know who may have been impacted by the storms or earthquake. Thank you very much. Barbara Hayes-Roth (bhr at ksl.stanford.edu) and Richard Korf (korf at cs.ucla.edu) Program Co-Chairs, AAAI-94 From moody at chianti.cse.ogi.edu Sat Jan 22 17:29:49 1994 From: moody at chianti.cse.ogi.edu (John Moody) Date: Sat, 22 Jan 94 14:29:49 -0800 Subject: Call for Papers: NEURAL NETWORKS in the CAPITAL MARKETS Message-ID: <9401222229.AA04012@chianti.cse.ogi.edu> ******************************************************************* --- Preliminary Announcement and Call for Papers --- NNCM-94 Second International Workshop NEURAL NETWORKS in the CAPITAL MARKETS Thursday-Friday, November 17-18, 1994 The Ritz-Carlton Hotel, Pasadena, California, U.S.A. Sponsored by Caltech and London Business School Neural networks have now been applied to a number of live systems in the capital markets, and in many cases have demonstrated better performance than competing approaches. Because of the overwhelming interest in the first NNCM workshop held in London in November 1993, and after the success of this workshop, the second annual NNCM workshop is planned for November 17-18, 1994, in Pasadena, California. This is a research meeting where original, significant contributions to the field are presented and discussed. In addition, two introductory tutorials will be included to familiarize audiences of different backgrounds with the financial aspects, and the mathematical aspects, of the field. Areas of Interest: Bond and stock valuation and trading, asset allocation and risk management, foreign exchange rate predication, commodity price forecasting, portfolio management, univariate time series analysis, multivariate data analysis, classification and ranking, pattern recognition, and hybrid systems. Organizing Committee: Dr. Y. Abu-Mostafa, California Institute of Technology Dr. A. Atiya, Cairo University Dr. N. Biggs, London School of Economics Dr. D. Bunn, London Business School Dr. B. LeBaron, University of Wisconsin Dr. A. Lo, MIT Sloan School Dr. J. Moody, Oregon Graduate Institute Dr. A. Refenes, London Business School Dr. M. Steiner, Universitaet Munster Dr. A. Timermann, Brickbeck College, London Dr. A. Weigend, University of Colorado Dr. H. White, University of California, San Diego Submission of Papers: Original contributions representing new and significant research, development, and applications in the above areas of interest will be considered. Authors should send 5 copies of a 1000-word summary clearly stating their results to Dr. Y. Abu-Mostafa, Caltech 116-81, Pasadena, CA 91125, U.S.A. All submissions must be received before May 1, 1994. There will be a rigorous refereeing process to select the high-quality papers to be presented at the workshop. Location: The workshop will be held at the Ritz-Carlton Huntington Hotel in Pasadena, within two miles from the Caltech campus. The hotel is a 35-minute drive from Los Angeles International Airport (LAX) with nonstop flights from most major cities in North America, Europe, the Far East, Australia, and South America. Mailing List: If you wish to be added to the mailing list of NNCM-94, please send your postal address, e-mail address, and fax number to Dr. Y. Abu-Mostafa, Caltech 116-81, Pasadena, CA 91125, U.S.A. e-mail: yaser at caltech.edu , fax (818) 568-8437 ******************************************************************* From SCHNEIDER at vms.cis.pitt.edu Sun Jan 23 09:15:00 1994 From: SCHNEIDER at vms.cis.pitt.edu (SCHNEIDER@vms.cis.pitt.edu) Date: Sun, 23 Jan 1994 09:15 EST Subject: Pre & Postdocs in neural processes in cognition in Pittsburgh Message-ID: <01H80M7Q2HSG9UP6GJ@vms.cis.pitt.edu> Pre- and Postdoctoral Training in Neural Processes in Cognition at the University of Pittsburgh and Carnegie Mellon University The Pittsburgh Neural Processes in Cognition program, now in its fourth year, is providing interdisciplinary training in brain sciences. The National Science Foundation has established an innovative program for students investigating the neurobiology of cognition. The program's focus is the interpretation of cognitive functions in terms of neuroanatomical and neurophysiological data and computer simulations. Such functions include perceiving, attending, learning, planning, and remembering in humans and in animals. This is an interdisciplinary program that prepares each student to perform original research investigating cortical function at multiple levels of analysis. State of the art facilities include: computerized microscopy, human and animal electrophysiological instrumentation, behavioral assessment laboratories, fMRI and PET brain scanners, the Pittsburgh Supercomputing Center, and a regional medical center providing access to human clinical populations. This is a joint program between the University of Pittsburgh, its School of Medicine, and Carnegie Mellon University. Each student receives full financial support, travel allowances and workstation support. Applications are encouraged from students with interest in biology, psychology, engineering, physics, mathematics, or computer science. Last year's class included mathematicians, psychologists, and neuroscience researchers. Pittsburgh is one of America's most exciting and affordable cities, offering outstanding symphony, theater, professional sports, and outdoor recreation in the surrounding Allegheny mountains. More than ten thousand graduate students attend its universities. Core Faculty and interests and affiliation CARNEGIE MELLON UNIVERSITY Psychology- James McClelland, Marlene Behrmann, Jonathan Cohen, Mark Johnson Computer Science - David Touretzky UNIVERSITY OF PITTSBURGH Behavioral Neuroscience - German Barrinonuevo, Susan Sesack Biology - Teresa Chay Information Science - Paul Munro Mathematics - Bard Ermentrout, Xiao-Jing Wang Neurobiology - John Horn, Al Humphrey, Peter Land, Charles Scudder, Dan Simons Neurological Surgery - Don Krieger, Robert Sclabassi Neurology - Steven Small, Robert Stowe Otolaryngology & physiology - Robert Schor Psychiatry - David Lewis, Lisa Morrow, Stuart Steinhauer Psychology - Walter Schneider, Velma Dobson, Michael Pogue-Geile Physiology - Dan Simons Radiology - Mark Mintun Applications: To apply to the program contact the program office or one of the affiliated departments. Students are admitted jointly to a home department and the Neural Processes in Cognition Program. Postdoctoral applicants MUST HAVE HAVE A SPONSOR AMONG THE TRAINING FACULTY. Most of our funds are limited to United States residents although there may be an option to consider some non-residents. To receive full consideration applications SHOULD BE SUBMITTED BY FEBRUARY 15. For information contact: Professor Walter Schneider Program Director Neural Processes in Cognition University of Pittsburgh 3939 O'Hara St Pittsburgh, PA 15260 Or: call 412-624-7064 or Email to NEUROCOG at VMS.CIS.PITT.EDU In Email requests for application materials, please provide your address and an indication of which department(s) you might be interested in. We can Email the research interests of the faculty. From mm at santafe.edu Sun Jan 23 20:19:10 1994 From: mm at santafe.edu (Melanie Mitchell) Date: Sun, 23 Jan 94 18:19:10 MST Subject: paper available Message-ID: <9401240119.AA00700@wupatki> The final version of our paper "When Will a Genetic Algorithm Outperform Hill Climbing?" (to appear in NIPS 6) is now available in neuroprose: When Will a Genetic Algorithm Outperform Hill Climbing? Melanie Mitchell John H. Holland Stephanie Forrest Santa Fe Institute University of Michigan University of New Mexico Abstract We analyze a simple hill-climbing algorithm (RMHC) that was previously shown to outperform a genetic algorithm (GA) on a simple ``Royal Road'' function. We then analyze an ``idealized'' genetic algorithm (IGA) that is significantly faster than RMHC and that gives a lower bound for GA speed. We identify the features of the IGA that give rise to this speedup, and discuss how these features can be incorporated into a real GA. The paper is 9 pages. To obtain a copy: unix> ftp archive.cis.ohio-state.edu Name: anonymous Password: (type your email address) ftp> cd pub/neuroprose ftp> binary ftp> get mitchell.ga-hillclimb.ps.Z ftp> quit unix>uncompress mitchell.ga-hillclimb.ps.Z unix> lpr mitchell.ga-hillclimb.ps From PIURI at IPMEL1.POLIMI.IT Mon Jan 24 18:58:01 1994 From: PIURI at IPMEL1.POLIMI.IT (PIURI@IPMEL1.POLIMI.IT) Date: Mon, 24 Jan 1994 18:58:01 MET-DST Subject: call for papers Message-ID: <01H82KU3MIJ6935O0G@ICIL64.CILEA.IT> ***************************************************************************** 37th MIDWEST SYMPOSIUM ON CIRCUITS AND SYSTEMS Lafayette Hilton and Towers, Lafayette, Louisiana August 3-5, 1994 CALL FOR PAPERS FOR THE SPECIAL SESSION ON THE EVALUATION OF NEURAL NETWORKS VS. CONVENTIONAL SYSTEMS ***************************************************************************** The 1994 Midwest Symposium on Circuits and Systems is organized by the Center for Advanced Computer Studies, University of Southwestern Louisiana. The symposium is devoted to all aspects of theory, design, and applications of circuits and systems. Emphasis is on current and future challenges in these areas as well as their interdisciplinary impact. Topics may include but are not limited to: + Analog/Digital Circuit Design + Digital Signal Processing + Control Systems & Robotics + Microwave Circuits + Nonlinear Circuits & Systems + Analog/Digital VLSI Design + Power Electronics & Systems + Analog and Digital Filter Design + Neural Networks + Image Processing + Fuzzy Logic + Communication Circuits + Solid State Circuits + Computer Networks + Expert Systems + Fault Analysis + Computer Aided Design + Petri Nets + Biomedical Applications + Military Applications + Space Applications + Automotive Applications + Intelligent Systems + Manufacturing + System Integration & + Multimedia Prototyping The special session on the evaluation of neural networks vs. conventional computing system is mainly directed (but not limited): - to identify and discuss the figures of merit that can be used to evaluate cost and performances of the neural networks, - to explore the limits and the advantages of the neural computation with respect to the conventional algorithmic approach, - to provide criteria for choosing among neural systems and for selecting a neural system vs. a conventional computing architecture. Authors interested in this special session are invited to send a one-page summary (by fax or e-mail) to the Session Program Chair by February 28, 1994. An extended summary (at least 4 pages plus figures) or the full paper must be sent (fax and e-mail latex submissions are accepted) to the Session Program Chair by March 28, 1994. Notification of acceptance or rejection and author kits will be sent by April 15, 1994. Session Program Chair for the special session on the evaluation of neural networks vs. conventional system Vincenzo Piuri Department of Electronics and Information Politecnico di Milano piazza L. da Vinci 32 I-20133 Milano, Italy phone no. +39-2-2399-3606 secretariat no. +39-2-2399-3623 fax no. +39-2-2399-3411 e-mail piuri at ipmel2.elet.polimi.it ***************************************************************************** General Chair: Registration: Magdy A. Bayoumi Cathy Pomier The Center for Advanced The Center for Advanced Computer Studies, USL Computer Studies, USL email: mab at cacs.usl.edu email: cathy at cacs.usl.edu Technical Program Chair: Technical Program Co-Chair: W. Ken Jenkins Hussein Baher Coordinated Science Lab Electrical Engineering Dept. University of Illinois KFU of Petroleum and Minerals email: jenkins at uicsl.csl.uiuc.edu Special Sessions Chair: Proceedings: Dolores Etter Nian-Feng Tzeng Department of Electrical Engg. The Center for Advanced University of Colorado Computer Studies email: etter at boulder.colorado.edu email: tzeng at cacs.usl.edu ***************************************************************************** From KELLYFJ at vax1.tcd.ie Mon Jan 24 08:03:00 1994 From: KELLYFJ at vax1.tcd.ie (Frank Kelly) Date: Mon, 24 Jan 1994 13:03 GMT Subject: ART vs. Leader clustering algorithm? Message-ID: <01H828GOASG0001GFZ@vax1.tcd.ie> Hello, Recently I heard an argument against Gail Carpenter and Stephen Grossberg's ART(Adaptive resonance theory). The basic argument was that ART was simply the 'leader clustering algorithm' enclosed in a load of neural net terminology. I am not very familiar with the leader clustering algorithm and was wondering would anyone like to remark for/against this argument as I am very interested in ART. Does anyone know of any paper on this subject? (ART vs. Leader clustering, or even leader clustering on it's own?). Cheers, --Frank Kelly *********************************************************************** kellyfj at vax1.tcd.ie | Senior Sophister Computer Science kellyfj at unix1.tcd.ie | Trinity College Dublin. Ireland. ======================================================================= From yuhas at bellcore.com Mon Jan 24 16:59:56 1994 From: yuhas at bellcore.com (Ben Yuhas) Date: Mon, 24 Jan 1994 16:59:56 -0500 Subject: Book Announcement Message-ID: <199401242159.QAA00782@om.bellcore.com> Title: NEURAL NETWORKS IN TELECOMMUNICATIONS Editors: Ben Yuhas, Bellcore Nirwan Ansari, New Jersey Institute of Technology Publisher: Kluwer Academic Publishers 367 pp. To Order: Phone: 617-871-6600 Fax: 617-871-6528 email: kluwer at world.std.com ISBN 0-7923-9417-8 Price is $105, but Kluwer will extend a 20% discount to those on the Connectionist mailing list through the end of February. Tell them you saw the add here when ordering. NEURAL NETWORKS IN TELECOMMUNICATIONS consists of a tightly edited collection of chapters that provides an overview of a wide range of telecommunications tasks being addressed with neural networks. These tasks range from the design and control of the underlying transport network to the filtering, interpretation and manipulation of the transported media. The chapters focus on specific applications, describe specific solutions and demonstrate the benefits that neural networks can provide. By doing this, the authors have demonstrated why neural networks should be another tool in the telecommunications engineer's toolbox. The contents include: 1. Introduction/ B.Yuhas, N.Ansari 2. Neural Networks for Switching/ T.X. Brown 3. Routing in Random Multistage Interconnection Networks/ M.W.Goudreau, C.L. Giles 4. ATM Traffic Control using Neural Networks/ A. H. Hiramatsu 5. Learning from Rare Events: Dynamic Cell Scheduling for ATM Networks/ D.B. Schwartz 6. A Neural Network Model for Adaptive Congestion Control in Broadband ATM Networks/ X. Chen 7. Structure and Performance of Neural Networks in Broadband Admission Control/ P.Trans-Gia, OLiver Gropp 8. Neural Network Channel Equalization/ W.R.Kirkland, D.P.Taylor 9. Application of Neural Networks as Exciser for Spread Spectrum Communication Systems/ R.Bijjani, P. K. Das 10. Static and Dynamic Channel Assignment using Simulated Annealing/ M. Duque-Anton, D.Kunz, B.Ruber 11. Cellular Mobile Communication Design Using Self-organizing Feature Maps/ T.Fritsch 12. Automatic Language Identification using Telephone Speech/ Y.K.Muthusamy, R.A. Cole 13.Text-Independent Talker Verification using Cohort Normalized Scores/ D.Burr 14. Neural Network Applications in Character Recognition and Document Analysis/ L.D. Jackel et al. 15. Image Vector Quantization by Neural Networks/ R. Lancini 16. Managing the Infoglut: Information Filtering using Neural Networks/ T.John by Thomas John 17. Empirical Comparisons of Neural Networks and Statistical Methods for Classification and Regression/ D.Duffy, B.Yuhas, A.Jain, A.Buja 18. A Neurocomputing Approach to Optimizing the Performance of a Satellite Communication Network/N.Ansari INDEX From webber at signal.dra.hmg.gb Tue Jan 25 04:25:54 1994 From: webber at signal.dra.hmg.gb (Chris Webber) Date: Tue, 25 Jan 94 09:25:54 +0000 Subject: NeuroProse preprint announcement Message-ID: FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/webber.self-org.ps.Z The file "webber.self-org.ps.Z" is available for copying from the Neuroprose preprint archive: TITLE: Self-organization of transformation-invariant neural detectors for constituents which recur within different perceptual patterns AUTHOR: Chris J.S. Webber (Cambridge University) (21 pages, preprint of article submitted to "Network" journal.) ABSTRACT: A simple self-organizing dynamics for governing the adaptation of individual neural perception units to the statistics of their input patterns is presented. The dynamics has a single adjustable parameter associated with each neuron, which directly controls the proportion of the patterns experienced that can induce response in the neuron, and thereby controls the nature of the neuron's response-preferences after the convergence of its adaptation. Neurons are driven by this dynamics to develop into detectors for the various individual pattern-constituents that recur frequently within the different patterns experienced: the elementary building-blocks which, in various combinations, make up those patterns. A detector develops so as to respond invariantly to those patterns which contain its trigger constituent. The development of discriminating detectors for specific faces, through adaptation to many photo-montages of combinations of different faces, is demonstrated. The characteristic property observed in the convergent states of this dynamics is that a neuron's synaptic vector becomes aligned symmetrically between pattern-vectors to which the neuron responds, so that those patterns project equal lengths onto the synaptic vector. Consequently, the neuron's response becomes invariant under the transformations which relate those patterns to one another. Transformation invariances that can develop in multi-layered systems of neurons, adapting according to this dynamics, include shape tolerance and local position tolerance. This is demonstrated using a two-level hierarchy, adapted to montages of cartoon faces generated to exhibit variability in facial expression and shape: neurons at the higher level of this hierarchy can discriminate between different faces invariantly with respect to expression, shape deformation, and local shift in position. These tolerances develop so as to correspond to the variability experienced during adaptation: the development of transformation invariances is driven entirely by statistical associations within patterns from the environment, and is not enforced by any constraints imposed on the architecture of neural connections. From hinton at cs.toronto.edu Tue Jan 25 11:36:50 1994 From: hinton at cs.toronto.edu (Geoffrey Hinton) Date: Tue, 25 Jan 1994 11:36:50 -0500 Subject: limited term faculty jobs at Toronto Message-ID: <94Jan25.113652edt.228@neuron.ai.toronto.edu> LIMITED TERM FACULTY POSITIONS AVAILABLE AT TORONTO The Department of Computer Science at the University of Toronto has two or three limited term faculty positions available. Appointments will be for 2 or 3 years and will not be renewed as these are NOT tenure-track jobs. The teaching load is approximately 4 hours of lectures per week for both semesters. Applications are invited from all areas of Computer Science. The needs of the Department would be well fitted by an applicant who can teach numerical analysis and does research on neural networks, especially learning algorithms, time series prediction, or image interpretation. The neural networks group in the department currently consists of Geoff Hinton, Peter Dayan, Mike Revow, Drew van Camp and eight graduate students (Tony Plate, Radford Neal, Chris Williams, Evan Steeg, Sid Fels, Ed Rasmussen, Brendan Frey and Sageeve Oore). We have close ties with other researchers in computer vision, statistics, and psychology. We have our own Silicon Graphics multiprocessor containing four R4400 chips. Applicants from the area of neural networks should send their applications to Geoffrey Hinton Computer Science Department University of Toronto 6 Kings College Road Toronto, Ontario M5S 1A4 CANADA Please include a CV, the names and addresses of 3 references, an outline of your research interests and a description of your background in numerical analysis and your teaching experience. Applications should be received by Feb 10, 1994. In accordance with Canadian immigration requirements, this advertisement is directed to Canadian citizens and permanent residents of Canada, but if there is no suitable Canadian applicant it may be possible to appoint another applicant. In accordance with its Employment Equity Policy, the University of Toronto encourages applications from qualified women or men, members of visible minorities, aboriginal peoples, and persons with disabilities. From rsun at cs.ua.edu Tue Jan 25 15:01:14 1994 From: rsun at cs.ua.edu (Ron Sun) Date: Tue, 25 Jan 1994 14:01:14 -0600 Subject: No subject Message-ID: <9401252001.AA28757@athos.cs.ua.edu> A special issue of _Connection Science_ journal on "Integrating Neural and Symbolic Processes" is now available Guest Editors of the special issue are: Larry Bookman Sun Microsystem Lab. Chelmsford, MA 01824 Ron Sun The University of Alabama Tuscaloosa, AL 35487 ----------------------------------------- Table of Contents for Special Issue Editorial: Integrating Neural and Symbolic Processes by L.A. Bookman and R. Sun Reflexive Reasoning with Multiple Instantiation in a Connectionist Reasoning System with a Type Hierarchy by D.R. Mani \& L. Shastri A Scalable Architecture for Integrating Associative and Semantic Memory by L. A. Bookman A Connectionist Production System with a Partial Match and its Use for Approximate Reasoning by N.K. Kasabov \& S.I. Shishkov Extraction, Insertion, and Refinement of Symbolic Rules in Dynamically-Driven Recurrent Neural Networks by C.L. Giles \& W. Omlin Combining Connectionist and Symbolic Learning to Refine Certainty-Factor Rule-Bases by J.J. Mahoney \& R.J. Mooney Combining Prior Symbolic Knowledge and Constructive Neural Network Learning by J. Fletcher \& Z. Obradovic Integrating Neural and Symbolic Approaches: A Symbolic Learning Scheme for a Connectionist Associative Memory by J.P. Ueberla \& A. Jagota Linking Symbolic and Subsymbolic Computing by A. Wilson \& J. Hendler Published by Carfax Publishing Company P.O.Box 25, Arbingdon, Oxfordsshire OK143UE UK e-mail: Carfax at ibmpcug.co.uk Editor-in-chief of Connection Science: Noel Sharkey e-mail: N.Sharkey at dcs.shef.ac.uk From barb at ai.mit.edu Tue Jan 25 18:21:48 1994 From: barb at ai.mit.edu (Barbara K. Moore Bryant) Date: Tue, 25 Jan 94 18:21:48 EST Subject: ART 1 Message-ID: <9401252321.AA09513@billberry> I have a paper about ART 1 and pattern clustering and would be happy to send it to you if you give me your mailing address. Or, you can look it up in Moore, "ART 1 and Pattern Clustering," Proceedings of the 1988 Connectionist Summer School, Morgan-Kaufman publ., pp. 174-185. I'd love to hear what you think. In the paper I show that ART 1 does in fact implement the leader clustering algorithm. ART 1's "stability" and "plasticity" are a property of the clustering algorithm (and the fact that only binary strings are the input and stored pattern), not of the underlying "neural" components. A careful reading of the paper and perusal of the examples might suggest that a different choice of distance metric or clustering algorithm might make more sense in a particular application. In fact, the final clusters formed by ART 1 might be described by some as downright weird (see Fig. 6 in my paper). I show by an example that other choices can be implemented in a similar architecture: there is no algorithmic constraint embodied in the architectural components of ART 1. About stability: ART 1 is stable because stored binary patterns can only be changed in one "direction" (you can change 1's to 0's but not 0's to 1's). So you will never get a situation where a pattern cycles. Moreover, no two stored patterns can be the same in ART 1, so you can only have finitely many stored patterns (because they're binary), and after some number of presentations of the same training set, the patterns will be fixed. Note that ART 1 would *not* be stable for real-valued inputs! As with any incremental clustering algorithm, different orders of presentation of input vectors to ART 1 during learning can result in different clusters. It is not necessarily bad or a problem that ART 1 implements the leader clustering algorithm. It would be nice, however, if this were made clear by the architects in the somewhat complicated papers that have been written on the subject. It might actually be very interesting that such architectures can implement clustering algorithms. It might be interesting to see what happens when you relax the constraint that all the underlying dynamical systems reach equilibrium before presenting the next training input. A cleverly designed architecture might behave in a useful way, or a biological way. (Note: I am not the only one to have made these observations about ART 1, but the presentation in my paper is the clearest that I know of. The paper is written so that it can be understood by people who aren't familiar with clustering.) barb at ai.mit.edu Please cc me on responses. From mherrmann at informatik.uni-leipzig.d400.de Wed Jan 26 09:58:36 1994 From: mherrmann at informatik.uni-leipzig.d400.de (mherrmann@informatik.uni-leipzig.d400.de) Date: Wed, 26 Jan 1994 15:58:36 +0100 Subject: job announcement Message-ID: <940126155836*/S=mherrmann/OU=informatik/PRMD=UNI-LEIPZIG/ADMD=D400/C=DE/@MHS> |------------------------------------------------------------| | Open Research Post in the EC-Project | | | | "Principles of Cortical Computation" | | | | at Leipzig University | |------------------------------------------------------------| The project forms a part of the "Human Capital and Mobility" programme and is a cooperative network between the University of Stirling (W.A. Phillips, coordinator), NORDITA (J. Hertz), MPI for Brain Research Frankfurt (W. Singer), the Insitute of Neuroinformatics Bochum (C. von der Malsburg), and the University of Leipzig (R. Der). The central goal of this network is to advance the understanding of the basic principles upon which computation in the mammalian cortex is based. The work of the Leipzig group is devoted to the statistical mechanical theory and/or applications of learning and self- organizing systems, in particular the Kohonen feature map. We use self-organizing layered Kohonen maps and the neural gas algorithm for hierarchical feature classification, time series predictions, and modelling and control of nonlinear dynamical systems. Reinforcement and Q-learning algorithms are of particular interest for the control tasks. Recent activities focus on the use of neural networks for the control of chaotic systems and possible implications for modelling the dynamical storage in the brain. The project runs until September 1995. Duration of the employment is about ten months. Preferentially the beginning of the employment should be in the next three months. Salary is about 5000.- DM per month according to qualification. As a rule applicants should have a doctoral degree but qualified graduate students are also considered. The applicant has to be a citizen of an EC country except Germany. Applications should contain a curriculum vitae, names and addresses of two references, a list of publications, and a statement of interests and should be submitted as soon as possible. Dr. habil. R. Der Leipzig, January 1994 Universitaet Leipzig Institut fuer Informatik Augustusplatz 10 - 11 D-04109 Leipzig Tel +49-341-719 2214 Fax +49-341-719 2399 e-mail: DER at INFORMATIK.UNI-LEIPZIG.D400.DE ------------------------------ End of body part 2 From kuh at spectra.eng.hawaii.edu Wed Jan 26 10:17:14 1994 From: kuh at spectra.eng.hawaii.edu (Anthony Kuh) Date: Wed, 26 Jan 94 10:17:14 HST Subject: extra NOLTA proceedings Message-ID: <9401262017.AA27871@spectra.eng.hawaii.edu> From sloman at columbo.cog.brown.edu Wed Jan 26 17:27:48 1994 From: sloman at columbo.cog.brown.edu (Steven Sloman) Date: Wed, 26 Jan 94 17:27:48 EST Subject: temporary job announcement Message-ID: <9401262227.AA06487@columbo.cog.brown.edu> Brown University Department of Cognitive and Linguistic Sciences Two Visiting Faculty Positions The Brown University Department of Cognitive and Linguistic Sciences invites applications for two temporary visiting faculty positions for the academic year September, 1994 to June, 1995. Each position would be suited to either a senior sabbatical visitor who, in exchange for half-time salary support, would teach one or two courses at Brown or to a more junior applicant who would receive full salary support and teach three courses. All applicants must have received the Ph.D. degree or equivalent by the time of their application. Position 1, Vision: A candidate should have strong teaching and research interests in one or more of the following areas: visual perception, visual cognition, computational vision, or computational neuroscience related to vision. Position 2, Cognition: A candidate should have strong teaching and research interests in an area such as memory, attention, problem solving, judgment and decision making, or comparative cognition. Please send vitae, recent publications, three references, and a cover letter describing teaching and research interests and qualifications to: Search Committee or Search Committee Vision Cognition Department of Cognitive and Linguistic Sciences Box 1978 Brown University Providence, RI 02912 The initial deadline for applications is February 15, 1994, but applications will be accepted after that time until the temporary positions are filled. Brown is an Equal Opportunity/Affirmative Action employer. Women and minorities are especially encouraged to apply. From mjhealy at espresso.rt.cs.boeing.com Wed Jan 26 21:08:03 1994 From: mjhealy at espresso.rt.cs.boeing.com (Michael J. Healy 865-3123 (206)) Date: Wed, 26 Jan 94 18:08:03 PST Subject: ART 1 Message-ID: <9401270208.AA09839@espresso.rt.cs.boeing.com> > Recently I heard an argument against Gail Carpenter and Stephen > Grossberg's ART(Adaptive resonance theory). The basic argument was that ART > was simply the 'leader clustering algorithm' enclosed in a load of neural > net terminology. I am not very familiar with the leader clustering > algorithm and was wondering would anyone like to remark for/against this > argument as I am very interested in ART. Does anyone know of any paper on > this subject? (ART vs. Leader clustering, or even leader clustering on > it's own?). > I thought it would be informative to post my reply, since I have done some work with ART. I would like to make two points: First, it is incorrect to state that the binary pattern clustering algorithm implemented by ART1 is equivalent to the leader clustering algorithm (ART is much more general than the ART1 architecture. I assumed the reference was to ART1). There are two significant differences: 1. ART1 is meant to function as a real-time clustering algorithm. This means that it (1) accepts and clusters input patterns in sequence, as they would appear in an application requiring an online system that learns as it processes data, and (2) is capable of finding a representation of the inputs that is arguably general (see below). The leader clustering algorithm, as I understand it, is supposed to have all its inputs available at once so that it can scan the set globally to form clusters. Hardly a real-time algorithm in any sense of the word. 2. The leader clustering algorithm does not generalize about its inputs. To explain, the patterns that it uses to represent its clusters are simply the input patterns that initiate the clusters (the "leaders"). ART1, on the other hand, forms a synaptic (in the neurobiological sense of the word) memory consisting of patterns that are templates for the patterns in each of the (real-time, dynamic) clusters that it forms. It updates these templates as it processes its inputs. Each template is the bitwise AND of all the input patterns that have been assigned to the corresponding cluster at some time in the learning history of ART1. This bitwise AND is a consequence of the Hebbian-like (actually, Weber-Fechner law) learning at each synapse in the outstar of F2 ---> F1 feedback connections from the F2 node that represents the cluster. A corresponding change occurs in the F1 ---> F2 connections to that same node, which form an adaptive filter for screening the inputs that come in through the F1 layer. Whether an input pattern is adopted by a particular cluster or not depends upon two measures of input pattern/template similarity that the ART1 system computes. The first measure is a result of F2 layer competition through inhibitory interconnections (again, synaptic). The second is computed by F2 ---> F1 gain control and the vigilance mechanism. The F2 ---> F1 gain control and F1 ---> vigilance node inhibitory connections, input layer ---> vigilance node connections, and vigilance node ---> F2 connections (all synaptic) effect the computation. The result is (1) Generalization. In fact, if the F1 nodes are thought of as implementing predicates in a two-valued logic, it is possible to prove that the ART1 templates represent conjunctive generalizations about the objects or events represented by the input patterns that have been adopted by a cluster. That is, each ART1 cluster represents a concept class. Each template also corresponds to a formula about any future objects that might be recognized as members of its concept class. This is more complicated than a simple conjunction of F1 predicates, but can be broken down into component conjunctions. I have a technical report on this, but the following reference is more useful relative to ART1 and its algorithm: Healy, M. J., Caudell, T. P. and Smith, S. D. G., A Neural Architecture for Pattern Sequence Verification Through Inferencing, IEEE Transactions on Neural Networks, Vol 4, No. 1, 1993, pp. 9-20. Suppose it is important to stabilize the memory on a fixed set of training patterns. Suppose it is desirable to know how many cycles, repeatedly showing the set of patterns to the ART1 system, are necessary to accomplish this; that is, how many cycles until the templates do not change any more, and each input pattern is recognized consistently as corresponding to a single template? Further, can the patterns be presented in some randomized order each time, or do they have to be presented in a particular order? The answer is as follows: Suppose that the number of distinct sizes of patterns---size being the number of 1-bits in a binary pattern---is M (obviously, M <= N, where N is the number of training patterns). Then M cycles are required. Further, the order of presentation can be arbitrary, and can be different with each cycle. Reference: M. Georgiopoulos, G. L. Heileman, and J. Huang, Properties of Learning Related to Pattern Diversity in ART1, Neural Networks, Vol. 4, pp. 751-757, 1991. This does not mean that the FORM of the templates is independent of the order of presentation. In fact, learning in ART1 is order-dependent, as it is in all clustering algorithms. I'll bet that leader clustering, even though it views the training set all at once, is also order-dependent. The inputs still have to be processed in some order and then deleted from the training set on each cycle. You could redo the entire training process for all N! possible presentation orders, but you would still have to somehow find the "best" of all the N! clusterings. My second point addresses the relevance of the argument that ART (meaning ART1) is "simply the leader clustering algorithm enclosed in a load of neural net terminology": ART1 represents a neural network, complete with a dynamic system model. Watch for Heileman, G., A Dynamical Adaptive Resonance Architecture, IEEE Transactions on Neural Networks (soon to appear) Given the relevance of ART1 to neural systems, including those that may actually exist in the brain, and given the proven stability of the ART1 algorithm, it seems to me that any argument that ART1 is simply this, that or the other algorithm is a moot point. I hope this sheds some light on the relationship between ART1 and the leader clustering algorithm. My thanks to the author of the original posting. Mike Healy From jbower at smaug.bbb.caltech.edu Thu Jan 27 13:00:45 1994 From: jbower at smaug.bbb.caltech.edu (Jim Bower) Date: Thu, 27 Jan 94 10:00:45 PST Subject: ART@ Message-ID: <9401271800.AA20109@smaug.bbb.caltech.edu> > Given the relevance of ART1 to neural systems, including > those that may actually exist in the brain, and given the proven > stability of the ART1 algorithm, it seems to me that any argument that > ART1 is simply this, that or the other algorithm is a moot point. As a neurobiologist who works with anatomically and physiologically derived models of cerebral cortical circuits, and memory, there is very little in the posted description of ART@ that justifies this statement. Ultimately, proposed solutions to engineering problems must live and die on their usefulness, not on asserted similarities to computing devices we do not understand (the brain in this case). It seems to me that if there is any lesson from the last 10 years research in "neural networks", it is that a thorough investigation of related algorithms in other domains is useful and appropriate. After all, this is supposed to be at least partially an intellectual exercise, not completely a sales job. Jim Bower From brown at galab3.mh.ua.edu Thu Jan 27 13:00:18 1994 From: brown at galab3.mh.ua.edu (brown@galab3.mh.ua.edu) Date: Thu, 27 Jan 1994 12:00:18 -0600 (CST) Subject: Batch Backprop versus Incremental Message-ID: <9401271800.AA15191@galab3.mh.ua.edu> A non-text attachment was scrubbed... Name: not available Type: text Size: 1329 bytes Desc: not available Url : https://mailman.srv.cs.cmu.edu/mailman/private/connectionists/attachments/00000000/39eda707/attachment-0001.ksh From 70712.3265 at CompuServe.COM Thu Jan 27 17:37:07 1994 From: 70712.3265 at CompuServe.COM (Morgan Downey) Date: 27 Jan 94 17:37:07 EST Subject: World Congress on Neural Networks 1994 Message-ID: <940127223706_70712.3265_FHP116-1@CompuServe.COM> WORLD CONGRESS ON NEURAL NETWORKS 1994 (WCNN '94) Annual Meeting of the International Neural Network Society Town and Country Hotel - San Diego, California, USA - June 4-9, 1994 Revised Call for Papers Due Date: Tuesday, February 15, 1994 The International Neural Network Society is pleased to announce that it can accept post-deadline papers and is inviting INNS members and non-members to submit papers for WCNN '94 by Tuesday, February 15, 1994. Papers will be reviewed by the Organizing Committee for acceptance and presentation format and will be published in the proceedings. INNS members can designate one paper that they have authored for automatic acceptance and publication in the proceedings. Papers submitted utilizing this deadline extension will not be eligible for revision. Papers previously accepted by journals, or publicly accessible Tech reports may be submitted for poster presentations. Submit three copies of the paper with a one page abstract of the talk which clearly cites the paper well enough to permit easy access to it. Only the abstract will be published. Submission Procedures (These procedures supersede the previously published Call for Papers information in the brochure.): o Six (6) copies (1 original, (5) copies) Do not fold or staple originals. o Six page limit in English. $20 per page for papers exceeding (6) pages (do not number pages). Checks for over length charges should be made out to INNS and must be included with submitted paper. o Format: camera-ready 8 1/2" x 11" white paper, 1" margins all aides, one column format, single spaced, in Times or similar type style of 10 points or larger, one side of paper only. Faxed copies are not acceptable. o Center at the top of first page: Full title of paper, author names(s), affiliation(s), and mailing address(es), followed by blank space, and abstract (us to 15 lines), and text. o Cover letter to accompany paper must include: full title of paper, corresponding author(s), and presenting author name, address, telephone and fax numbers, 1st and 2nd choices of Technical Session (see session topics), and INNS membership number if applicable. o Author agrees to the transfer of copyright to INNS for the conference proceedings. All submitted papers become the property of INNS. _________________________________ SCHEDULE: Saturday, June 4, 1994 and Sunday, June 5, 1994 INNS UNIVERSITY SHORT COURSES Monday, June 6, 1994 NEURAL NETWORK INDUSTRIAL EXPOSITION RECEPTION OPENING CEREMONY Tuesday, June 7, 1994 PARALLEL SESSIONS EXHIBITS SPECIAL SESSION ON "BIOMEDICAL APPLICATIONS" SPECIAL SESSION ON "COMMERCIAL AND INDUSTRIAL APPLICATIONS" PLENARY 1: LOTFI ZADEH PLENARY 2: PER BAK Wednesday, June 8, 1994 PARALLEL SESSIONS EXHIBITS SPECIAL SESSION ON "FINANCIAL AND ECONOMIC APPLICATIONS" PLENARY 1: BERNARD WIDROW PLENARY 2: MELANIE MITCHELL SPECIAL INTEREST GROUP (SIGINNS) SESSIONS Thursday, June 9, 1994 PARALLEL SESSIONS EXHIBITS SPECIAL SESSION ON "NEURAL NETWORKS IN CHEMICAL ENGINEERING" SPECIAL SESSION ON "MIND, BRAIN, AND CONSCIOUSNESS" PLENARY 1: PAUL WERBOS PLENARY 2: JOHN TAYLOR Friday, June 10, 1994 and Saturday, June 11, 1994 SATELLITE MEETING: WNN/FNN 94 SAN DIEGO - NEURAL NETWORKS AND FUZZY LOGIC Sponsoring Society: NASA (National Aeronautics and Space Administration) Cooperating: INNS, Society for Computer Simulation, SPIE and all other interested societies. For more information contact: Mary Lou Padgett, Auburn University, 1165 Owens Road, Auburn AL 36830 ph: 205-821-2472 or 3488; fax: 205-844-1809; e-mail: mpadgett at eng.auburn.edu -- NASA Rep: Robert Savely, NASA/JSC __________________________________________________ WCNN 1994 ORGANIZING COMMITTEE: Paul Werbos, Chair Harold Szu Bernard Widrow Liaison to the European Neural Network Society: John G. Taylor Liaison to the Japanese Neural Network Society: Kunihiko Fukushima PROGRAM COMMITTEE: Daniel Alkon Shun-ichi Amari James A. Anderson Richard Andersen Kaveh Ashenayi Andrew Barto David Brown Horacio Bouzas Gail Carpenter David Casasent Ralph Castain Cihan Dagli Joel Davis Judith Dayhoff Guido DeBoeck David Fong Judith Franklin Walter Freeman Kunihiko Fukushima Michael Georgiopoulos Lee Giles Stephen Grossberg Dan Hammerstrom Robert Hecht-Nielsen Robert Jannarone Jari Kangas Christof Koch Teuvo Kohonen Bart Kosko Clifford Lau Soo-Young Lee George Lendaris Daniel Levine Alianna Maren Kenneth Marko Thomas McAvoy Thomas McKenna Larry Medsker Len Neiberg Erkki Oja Robert Pap Rich Peterson David Rumelhart Mohammed Sayeh Dejan Sobajic Harold Szu John Taylor Brian Telfer Shiro Usui John Weinstein Bernard Widrow Takeshi Yamakawa Lotfi Zadeh Mona Zaghloul COOPERATING SOCIETIES/INSTITUTIONS: American Association for Artificial Intelligence American Institute for Chemical Engineers American Physical Society Center for Devices and Radiological Health, US FDA Cognitive Science Society European Neural Network Society International Fuzzy Systems Association Japanese Neural Network Society Korean Neural Network Society US National Institute of Allergy and Infectious Diseases US Office of Naval Research Society for Manufacturing Engineers SPIE - The International Society for Optical Engineering Division of Cancer Treatment, US National Cancer Institute ____________________________________________________ SESSIONS AND CHAIRS: 1 Biological Vision ... S. Grossberg Invited Talk: Stephen Grossberg - Recent Results in Biological Vision 2 Machine Vision ... K. Fukushima, R. Hecht-Nielsen Invited Talk: Kunihiko Fukushima - Visual Pattern Recognition with Selective Attention Invited Talk: Robert Hecht-Nielsen - Foveal Active Vision: Methods, Results, and Prospects 3 Speech and Language ... D. Rumelhart, T. Peterson 4 Biological Neural Networks ... T. McKenna, J. Davis Session One: From Biological Networks to Silicon Invited Speakers: Frank Werblin, UC Berkeley Richard Granger, UC Irvine Theodore Berger, USC Session Two: Real Neurons in Networks Invited Speakers: Jim Schwaber, DuPont Misha Mahowald, Oxford University David Stenger, NRL Session Three: Networks for Motor Control and Audition Invited Speakers: Randy Beer, Case Western Reserve University Daniel Bullock, Boston University Shihab Shamma, University of Maryland Session Four: Learning and Cognition and Biological Networks Invited Speakers: Mark Gluck, Rutgers University Nestor Schmajuk, Northwestern University Michael Hasselmo, Harvard University 5 Neurocontrol and Robotics ... A. Barto, K. Ashenayi 6 Supervised Learning ... G. Lendaris, S-Y. Lee Invited Talk: George Lendaris - Apriori Knowledge and NN Architectures Invited Talk: Soo-Young Lee - Error Minimization, Generalization, and Hardware Implementability of Supervised Learning 7 Unsupervised Learning ... G. Carpenter, R. Jannarone Invited Talk: Gail Carpenter - Distributed Recognition Codes and Catastrophic Forgetting Invited Talk: Robert Jannarone - Current Trends of Learning Algorithms 8 Pattern Recognition ... T. Kohonen, B. Telfer Invited Talk: Teuvo Kohonen - Physiological Model for the Self-Organizing Map Invited Talk: Brian Telfer - Challenges in Automatic Object Recognition: Adaptivity, Wavelets, Confidence 9 Prediction and System Identification ... P. Werbos, G. Deboeck Invited Talk: Guido Deboeck - Neural, Genetic, and Fuzzy Systems for Trading Chaotic Financial Markets 10 Cognitive Neuroscience ... D. Alkon, D. Fong 11 Links to Cognitive Science and Artificial Intelligence ... J. Anderson, L. Medsker Invited Talk: Larry Medsker - Hybrid Intelligent Systems: Research and Development Issues 12 Neural Fuzzy Systems ... L. Zadeh, B. Kosko 13 Signal Processing ... B. Widrow, H. Bouzas Invited Talk: Bernard Widrow - Nonlinear Adaptive Signal Processing 14 Neurodynamics and Chaos ... H. Szu, M. Zaghloul Invited Talk: Walter Freeman - Biological Neural Network Chaos Invited Talk: Harold Szu - Artificial Neural Network Chaos 15 Hardware Implementations ... C. Lau, R. Castain, M. Sayeh Invited Talk: Clifford Lau - Challenges in Neurocomputers Invited Talk: Mark Holler - High Performance Classifier Chip 16 Associative Memory ... J. Taylor, S. Usui Invited Talk: John G. Taylor - Where is Associative Memory Going? Invited Talk: Shiro Usui - Review of Associative Memory 17 Applications ... D. Casasent, B. Pap, D. Sobajic Invited Talk: David Casasent - Optical Neural Networks and Applications Invited Talk: Yoh-Han Pao - Mathematical Basis for the Power of the Functional-Link Net Approach: Applications to Semiconductor Processing Invited Talk: Mohammed Sayeh - Advances in Optical Neurocomputers 18 Neuroprocessing and Virtual Reality ... L. Giles, H. Hawkins Invited Talk: Harold Hawkins - Opportunities for Virtual Environment and Neuroprocessing 19 Circuits and System Neuroscience ... J. Dayhoff, C. Koch Invited Talk: Judith Dayhoff - Temporal Processing for Neurobiological Signal Processing Invited Talk: Christof Koch - Temporal Analysis of Spike Patterns in Monkeys and Artificial Neural Networks 20 Mathematical Foundations ... S-I. Amari, D. Levine Invited Talk: Shun-ichi Amari - Manifolds of Neural Networks and EM Algorithms Additional session invited talks to be determined. Session invited talks will not be scheduled to run concurrently at WCNN 1994. *Invited INNS wishes to acknowledge the US Office of Naval Research for its generous support of the Biological Neural Networks Session at WCNN 1994 ___________________________________________________________ NEW IN '94! INNS UNIVERSITY SHORT COURSES INNS is proud to announce the establishment of the INNS University Short Course format to replace the former tutorial program. The new 2-day, 4-hour per course format provides twice the instruction with much greater depth and detail. There will be six parallel tracks offered in three segments (morning, afternoon, and evening each day). INNS reserves the right to cancel Short Courses and refund payment should registration not meet the minimum number of persons required per Short Course. [Dates and times are listed after each instructor; course descriptions are available by contacting INNS.] A. Teuvo Kohonen, Helsinki University of Technology - SATURDAY, JUNE 4. 1994, 6-10 PM Advances in the Theory and Applications of Self-Organizing Maps B. James A. Anderson, Brown University - SATURDAY, JUNE 4. 1994, 1-5 PM Neural Network Computation as Viewed by Cognitive Science and Neuroscience C. Christof Koch, California Institute of Technology - SUNDAY, JUNE 5. 1994, 6-10 PM Vision Chips: Implementing Vision Algorithms with Analog VLSI Circuits D. Kunihiko Fukushima, Osaka University - SATURDAY, JUNE 4. 1994, 1-5 PM Visual Pattern Recognition with Neural Networks E. John G. Taylor, King's College London - SUNDAY, JUNE 5. 1994, 1-5 PM Stochastic Neural Computing: From Living Neurons to Hardware F. Harold Szu, Naval Surface Warfare Center - SATURDAY, JUNE 4. 1994, 6-10 PM Spatiotemporal Information Processing by Means of McCollouch-Pitts and Chaotic Neurons G. Shun-ichi Amari, University of Tokyo - SUNDAY, JUNE 5. 1994, 8 AM-12 PM Learning Curves, Generalization Errors and Model Selection H. Walter J. Freeman, University of California, Berkeley - SUNDAY, JUNE 5. 1994, 8 AM-12 PM Review of Neurobiology: From Single Neurons to Chaotic Dynamics of Cerebral Cortex I. Judith Dayhoff, University of Maryland - SATURDAY, JUNE 4. 1994, 8 AM-12 PM Neurodynamics of Temporal Processing J. Richard A. Anderson, Massachusetts Institute of Technology - SATURDAY, JUNE 4. 1994, 6-10 PM Neurobiologically Plausible Neural Networks K. Paul Werbos, National Science Foundation - SUNDAY, JUNE 5. 1994, 1-5 PM From stolcke at ICSI.Berkeley.EDU Thu Jan 27 21:35:32 1994 From: stolcke at ICSI.Berkeley.EDU (Andreas Stolcke) Date: Thu, 27 Jan 1994 18:35:32 PST Subject: TR on HMM induction available Message-ID: <199401280235.SAA07317@icsib30.ICSI.Berkeley.EDU> The following technical report is now available from ICSI. FTP instructions appear at the end of this message. Note that this report is a greatly expanded and revised follow-up to our paper in last year's NIPS volume. It replaces report TR-93-003 mentioned in that paper, which was never released as we decided to include substantial new material instead. We regret any confusion or inconvenience this may have caused. Andreas Stolcke Stephen Omohundro -------------------------------------------------------------------------- Best-first Model Merging for Hidden Markov Model Induction Andreas Stolcke and Stephen M. Omohundro TR-94-003 January 1994 Abstract: This report describes a new technique for inducing the structure of Hidden Markov Models from data which is based on the general `model merging' strategy (Omohundro 1992). The process begins with a maximum likelihood HMM that directly encodes the training data. Successively more general models are produced by merging HMM states. A Bayesian posterior probability criterion is used to determine which states to merge and when to stop generalizing. The procedure may be considered a heuristic search for the HMM structure with the highest posterior probability. We discuss a variety of possible priors for HMMs, as well as a number of approximations which improve the computational efficiency of the algorithm. We studied three applications to evaluate the procedure. The first compares the merging algorithm with the standard Baum-Welch approach in inducing simple finite-state languages from small, positive-only training samples. We found that the merging procedure is more robust and accurate, particularly with a small amount of training data. The second application uses labelled speech data from the TIMIT database to build compact, multiple-pronunciation word models that can be used in speech recognition. Finally, we describe how the algorithm was incorporated in an operational speech understanding system, where it is combined with neural network acoustic likelihood estimators to improve performance over single-pronunciation word models. -------------------------------------------------------------------------- Instructions for retrieving ICSI technical reports via ftp Replace YEAR and tr-XX-YYY with the appropriate year and TR number. If your name server is ignorant about ftp.icsi.berkeley.edu, use 128.32.201.55 instead. unix% ftp ftp.icsi.berkeley.edu Name (ftp.icsi.berkeley.edu:): anonymous Password: your_name at your_machine ftp> cd /pub/techreports/YEAR ftp> binary ftp> get tr-XX-YYY.ps.Z ftp> quit unix% uncompress tr-XX-YYY.ps.Z unix% lpr -Pyour_printer tr-XX-YYY.ps All files in this archive can also be obtained through an e-mail interface in case direct ftp is not available. Send mail containing the line `send help' to ftpmail at ICSI.Berkeley.EDU for instructions. As a last resort, hardcopies may be ordered for a small fee. Send mail to info at ICSI.Berkeley.EDU for more information. From sef+ at cs.cmu.edu Fri Jan 28 00:06:43 1994 From: sef+ at cs.cmu.edu (Scott E. Fahlman) Date: Fri, 28 Jan 94 00:06:43 EST Subject: Batch Backprop versus Incremental In-Reply-To: Your message of Thu, 27 Jan 94 12:00:18 -0600. <9401271800.AA15191@galab3.mh.ua.edu> Message-ID: ...a gentleman from the U.K. suggested that Batch mode learning could possibly be unstable in the long term for backpropagation. I did not know the gentleman and when I asked for a reference he could not provide one. Does anyone have any kind of proof stating that one method is better than another? Or that possibly batch backprop is unstable in <> sense? Those U.K. guys get some funny ideas. I think it's the intoxicating fumes from wet sheep. :-) Batch mode backprop is actually more stable (other things being equal) than online (also known as "stochastic") updating. In batch mode, each weight update is made with respect to the true error gradient, computed over the whole training set. In online, each update is made with respect to a single sample, and a few atypical samples in a row can take you very far afield, especially if you use one of the fast training methods that adapts step-size. In addition, online training never settles down into a stable minimum, since the network continues to be buffeted by the individual training cases as they arrive. (You can, of course, reduce the learning rate once the net seems to have found a solution.) Perhaps the origin of this myth about batch learning is that you need to scale down the gradient values (or the learning rate parameter) as the number of training cases goes up. If you don't, the effective learning rate can become arbitrarily large and the learning will be unstable. This isn't to say that batch learning is necessarily better. As long as you take small steps, online updating will usually be stable, and it can be much faster than batch updating if the training set is large and redundant. A net trained by online update might converge before even a single epoch has been completed. -- Scott =========================================================================== Scott E. Fahlman Internet: sef+ at cs.cmu.edu Senior Research Scientist Phone: 412 268-2575 School of Computer Science Fax: 412 681-5739 Carnegie Mellon University Latitude: 40:26:33 N 5000 Forbes Avenue Longitude: 79:56:48 W Pittsburgh, PA 15213 =========================================================================== From dfisher at vuse.vanderbilt.edu Fri Jan 28 07:39:55 1994 From: dfisher at vuse.vanderbilt.edu (Douglas H. Fisher) Date: Fri, 28 Jan 94 06:39:55 CST Subject: AI and Stats Workshop Message-ID: <9401281239.AA05663@aim.vuse> Call For Papers Fifth International Workshop on Artificial Intelligence and Statistics January 4-7, 1995 Ft. Lauderdale, Florida PURPOSE: This is the fifth in a series of workshops which has brought together researchers in Artificial Intelligence and in Statistics to discuss problems of mutual interest. The exchange has broadened research in both fields and has strongly encouraged interdisciplinary work. This workshop will have as its primary theme: ``Learning from data'' Papers on other topics at the interface of AI & Statistics are *strongly* encouraged as well (see TOPICS below). FORMAT: To encourage interaction and a broad exchange of ideas, the presentations will be limited to about 20 discussion papers in single session meetings over three days (Jan. 5-7). Focussed poster sessions will provide the means for presenting and discussing the remaining research papers. Papers for poster sessions will be treated equally with papers for presentation in publications. Attendance at the workshop will *not* be limited. The three days of research presentations will be preceded by a day of tutorials (Jan. 4). These are intended to expose researchers in each field to the methodology used in the other field. The Tutorial Chair is Prakash Shenoy. Suggestions on tutorial topics can be sent to him at pshenoy at ukanvm.bitnet. LANGUAGE: The language will be English. TOPICS OF INTEREST: The fifth workshop has a primary theme of ``Learning from data'' At least one third of the workshop schedule will be set aside for papers with this theme. Other themes will be developed according to the strength of the papers in other areas, including but not limited to: - integrated man-machine modeling methods - empirical discovery and statistical methods for knowledge acquisition - probability and search - uncertainty propagation - combined statistical and qualitative reasoning - inferring causation - quantitative programming tools and integrated software for data analysis and modeling. - discovery in databases - meta data and design of statistical data bases - automated data analysis and knowledge representation for statistics - connectionist approaches - cluster analysis SUBMISSION REQUIREMENTS: Three copies of an extended abstract (up to four pages) should be sent to H. Lenz, Program Chair or D. Fisher, General Chair 5th Int'l Workshop on AI & Stats 5th Int'l Workshop on AI & Stats Free University of Berlin Box 1679, Station B Department of Economics Department of Computer Science Institute for Statistics Vanderbilt University and Econometrics Nashville, Tennessee 37235 14185 Berlin, Garystr 21 USA Germany or electronically (postscript or latex documents preferred) to ai-stats-95 at vuse.vanderbilt.edu Submissions for discussion papers (and poster presentations) will be considered if *postmarked* by June 30, 1994. If the submission is electronic (e-mail), then it must be *received* by midnight June 30, 1994. Abstracts postmarked after this date but *before* July 31, 1994, will be considered for poster presentation *only*. Please indicate which topic(s) your abstract addresses and include an electronic mail address for correspondence. Receipt of all submissions will be confirmed via electronic mail. Acceptance notices will be mailed by September 1, 1994. Preliminary papers (up to 20 pages) must be returned by November 1, 1994. These preliminary papers will be copied and distributed at the workshop. PROGRAM COMMITTEE: General Chair: D. Fisher Vanderbilt U., USA Program Chair: H. Lenz Free U. Berlin, Germany Members: W. Buntine NASA (Ames), USA J. Catlett AT&T Bell Labs, USA P. Cheeseman NASA (Ames), USA P. Cohen U. of Mass., USA D. Draper U. of Bath, UK Wm. Dumouchel Columbia U., USA A. Gammerman U. of London, UK D. J. Hand Open U., UK P. Hietala U. Tampere, Finland R. Kruse TU Braunschweig, Germany S. Lauritzen Aalborg U., Denmark W. Oldford U. of Waterloo, Canada J. Pearl UCLA, USA D. Pregibon AT&T Bell Labs, USA E. Roedel Humboldt U., Germany G. Shafer Rutgers U., USA P. Smyth JPL, USA MORE INFORMATION: For more information write dfisher at vuse.vanderbilt.edu or write to ai-stats-request at watstat.uwaterloo.ca to subscribe to the AI and Statistics mailing list. Traditionally, the Workshop has attracted many with an interest in connectionism, and we encourage even more for the 1995 Workshop. From qin at turtle.fisher.com Fri Jan 28 08:49:46 1994 From: qin at turtle.fisher.com (qin@turtle.fisher.com) Date: Fri, 28 Jan 94 08:49:46 CDT Subject: Batch vs. Pattern Backprop Message-ID: <009793431874AAA0.686054A3@turtle.fisher.com> From: UUCP%"ds2100!galab3.mh.ua.edu!brown" 28-JAN-1994 06:35:06.88 To: cs.cmu.edu!Connectionists CC: Subj: Batch Backprop versus Incremental >From: ds2100!galab3.mh.ua.edu!brown >Message-Id: <9401271800.AA15191 at galab3.mh.ua.edu> >Subject: Batch Backprop versus Incremental >To: cs.cmu.edu!Connectionists >Date: Thu, 27 Jan 1994 12:00:18 -0600 (CST) >X-Mailer: ELM [version 2.4 PL22] >Content-Type: text >Content-Length: 1329 >Dear Connectionists, > I attended the 5th International Conference on Genetic Algorithms >this summer, and in one of the sessions on combinations of genetic >algorithms (GAs) and Neural Nets(ANNs) a gentleman from the U.K. >suggested that Batch mode learning could possibly be unstable in >the long term for backpropagation. I did not know the gentleman >and when I asked for a reference he could not provide one. >Does anyone have any kind of proof stating that one method is better >than another? Or that possibly batch backprop is unstable in <> >sense? >Any and all response are thanked for in advance, >Brown Cribbs =============================================================== Brown, I suggest the following paper for reference: S.Z. Qin, et al. (1992). Comparison of four neural net learning methods for system identification. IEEE TRANSACTIONS ON NEURAL NETWORKS. vol3, no.1 pp122-130. It is proven in the paper that the batch and pattern/incremental learning methods are equivalent given small learning rates. There is no result showing that batch learning is more unstable. However, one simulation in the paper shows that batch learning has ripples for large learning rates in a particular simulation. But the pattern learning does not. In other words, batch learning error decreases, then increases, and then decreases. Initially batch learning was a bit faster than pattern learning, but it has this kind of ripples. My guess for this observation is due to the particular error surface. In summary, there is no significant difference for small learning rate, but there is difference for large learning rates. Though there is one simulation example showing pattern learning is more stable than batch learning, it may not be generally true. S.Joe Qin Fisher-Rosemount Systems, Inc. 1712 Centre Creek Drive Austin, TX 78754 512-832-3635 words, The From BATTITI at itnvax.science.unitn.it Fri Jan 28 13:10:13 1994 From: BATTITI at itnvax.science.unitn.it (BATTITI@itnvax.science.unitn.it) Date: 28 Jan 1994 18:10:13 +0000 Subject: MEMORY, LEARNING & OPTIMIZATION - comments and preprint - Message-ID: <01H884B3Q7KIMAVS1W@itnvax.science.unitn.it> MEMORY, LEARNING & OPTIMIZATION Are Monte Carlo techniques appropriate? A research stream in Neural Networks proposes to use NN for Combinatorial Optimization (CO), with mixed results. Other streams consider state of the art optimization techniques and apply them to machine learning and NN tasks. In reality the areas of optimization and learning are deeply interrelated. In particular, recent CO algorithms use MEMORY and LEARNING to generate efficient search trajectories from a set of local moves. Unfortunately Markov Chain Monte Carlo methods are memory-less (by definition, the distribution of the random variable X(t+1) in a Markov chain is mediated entirely by the value of X(t)). The past history does NOT influence the current move. Is the importance assigned to Markov chain sampling methods in the area of optimization and learning justified by the theoretical results (whose practical interest is dubious, see the clear discussion about sa in [1]) or by the results obtained on significant tasks? We doubt it. =========================================================================== Now we describe a homework that we made after reading the interesting paper placed in the neuroprose archive by M. Mitchell [2] about a week ago. (to spare e-mail bytes we assume knowledge of [2]) On the "Royal Road" function R1 (Table 1 in the cited reference), GA converges in 61,334 function evaluations (mean), RMHC (a zero-temperature Monte Carlo method used by Forrest and Mitchell) in 6,179. In RMHC, a given string is mutated at a randomly chosen single locus. If the mutation leads to an equal or higher fitness, the new string replaces the old string. Now, if the last mutation increased the fitness, it is reasonable that it is NOT considered again in the next step (otherwise the successful mutation can be undone). Similarly, if the last mutation was not accepted because the fitness would have decreased, it is pointless to consider it again in the next step. Let us generalize and modify RMHC: if mutation (i) at iteration (t) either decreases or increases the fitness, (i) is prohibited for the next T iterations (the corresponding bit in the string is "frozen"). Nothing happens if the fit- ness remains unchanged. Here are the homework results as a function of T (100 runs for each value): T mean f.evals st. dev. 0 6,450 (289) ----(confirms results of [2]) 50 3,728 (148) 100 3,122 (104) 200 2,805 (94) 300 2,575 (88) 400 2,364 (83) 500 2,605 (79) The use of this simple form of memory can accelerate the convergence (about 1/3 function evaluations are sufficient when T=400). Bye the way, we would not be surprised if some biological mechanism could be used for the same purpose (the above modification of RMHC resembles a sort of "refractory period": if a mutation "fires" it does not fire again for T steps). No doubt there is noise in our biological neurons..., but is our gray matter generating Markov chains? Comments are welcome. Roberto Battiti Giampietro Tecchiolli Dip. di Matematica Univ. di Trento IRST 38050 Povo (Trento) - ITALY 38050 Povo (Trento) - ITALY e-mail: battiti at itnvax.science.unitn.it e-mail: tec at irst.it ***************************************************************************** A preprint with a benchmark of the Reactive Tabu Scheme and other popular techniques (including GAs) is available by ftp from our local archive ("Local Search with Memory: Benchmarking RTS", R. Battiti and G. Tecchiolli) To get a copy: unix> ftp volterra.science.unitn.it (130.186.34.16) Name: anonymous Password: (type your email address) ftp> cd pub/rts-neuro ftp> binary ftp> get rts-benchmark.ps.Z ftp> quit unix>uncompress rts-benchmark.ps.Z unix> lpr rts-benchmark.ps (34 pages) A limited number of hardcopies are available if printing is impossible. Additional background references are abstracted in the file: pub/rts-neuro/ABSTRACTS-TN of the above archive. ***************************************************************************** ftp references cited: [1] "Probabilistic inference using Markov chain Monte Carlo methods" R. M. Neal, posted 23 Nov 1993. ftp: ftp.cs.toronto.edu file: pub/radford/review.ps.Z). [2] " When Will a Genetic Algorithm Outperform Hill Climbing?" Melanie Mitchell, John H. Holland, Stephanie Forrest, posted 23 Jan 1994. ftp: archive.cis.ohio-state.edu file: pub/neuroprose/mitchell.ga-hillclimb.ps.Z). From arantza at cogs.susx.ac.uk Fri Jan 28 13:34:31 1994 From: arantza at cogs.susx.ac.uk (Arantza Etxeberria) Date: Fri, 28 Jan 94 18:34:31 GMT Subject: ECAL95 First Announcement Message-ID: First Announcement 3rd. EUROPEAN CONFERENCE ON ARTIFICIAL LIFE ECAL95 Granada, Spain, 4-6 June, 1995 It is a pleasure to announce the forthcoming 3rd European Conference on Artificial Life (ECAL95). Despite its short life, Artificial Life (AL) is already a mature scientific field. In trying to discover the rules of life and extract its essence so that it can be implemented in different media, AL research has led us to a better understanding of a large set of interesting biology-related problems, such as self organization, emergence, origins of life, self-reproduction, computer viruses, learning, growth and development, animal behavior, ecosystems, autonomous agents, adaptive robotics, etc. The Conference will be organized into Scientific Sessions, Demonstrations, Videos and Comercial Exhibits. Scientific Sessions will consist of Lectures (invited), Oral Presentations, and Posters. The site of ECAL95 will be the city of Granada, located in the south of Spain, in the region of Andalucia. Granada was the last moors site in the Iberic Peninsula, and it has the inheritance of their culture with the legacy of marvelous constructions such as the Alhambra and the Gardens of Generalife. ECAL95 will be organized in collaboration with the International Workshop on Artificial Neural Networks (IWANN95) to be held at Malaga (Costa del Sol, Spain), June 7-9, 1995. These places are only one hour apart by car. Special inscription fees will be offered to those attending both meetings. Scientific Sessions and Topics 1. Foundations and Epistemology: Philosophical Issues. Emergence. Levels of organization. Evolution of Hierarchical Systems. Evolvability. Computation and Dynamics. Ethical Problems. 2. Evolution: Prebiotic Evolution. Origins of Life. Evolution of Metabolism. Fitness Landscapes. Ecosystem Evolution. Biodiversity. Evolution of Sex. Natural Selection and Sexual selection. Units of Selection. 3. Adaptive and Cognitive Systems: Reaction, Neural and Immune Networks. Growth and Differentiation. Self-organization. Pattern Formation. Multicellulars and Development. Natural and Artificial Morphogenesis. 4. Artificial Worlds: Simulation of Adaptive and Cognitive Systems. System-Environment Correlation. Sensor-Effector Coordination. Environment Design. 5. Robotics and Emulation of Animal Behavior: Sensory and Motor Activity. Mobile Agents. Adaptive Robots. Autonomous Robots. Evolutionary Robotics. Ethology. 6. Societies and Collective Behavior: Swarm Intelligence. Cooperation and Communication among Animals and Robots. Evolution of Social Behavior. Social Organizations. Division of Tasks. 7. Applications and Common Tools: Optimization. Problem Solving. Virtual Reality and Computer Graphics. Genetic Algorithms. Neural Networks. Fuzzy Logic. Evolutionary Computation. Genetic Programming. Inscription / Information Those interested please send (mail/fax/e-mail) the Intention Form to the Programme Secretary, Juan J. Merelo, at the following address: Dept. Electronica | Facultad de Ciencias | Phone: +34-58-243162 Campus Fuentenueva | Fax: +34-58-243230 18071 Granada, Spain | E-mail: ecal95 at ugr.es Organization Committee Federico Moran. UCM. Madrid (E) Chair Alvaro Moreno. UPV. San Sebastian (E) Chair Arantza Etxeberria Univ. Sussex (UK) Julio Fernandez. UPV. San Sebastian (E) Francisco Montero. UCM. Madrid (E) Alberto Prieto. UGr. Granada (E) Carme Torras. UPC. Barcelona (E) Programm Committee Francisco Varela. CNRS/CREA. Paris (F) Chair Juan J. Merelo. UGr. Granada (E) Secretary (Definitive list of this Committee will be completed and announced in the forthcoming Call-for-Papers) -------------------------------- cut here -------------------------------- INTENTION FORM 3rd. EUROPEAN CONFERENCE ON ARTIFICIAL LIFE ECAL95 Granada, Spain, 4-6 June, 1995 Family Name: First Name: Institution: Address: Phone No.: Fax No.: e-mail: Signature: Date: From finnoff at predict.com Fri Jan 28 14:21:01 1994 From: finnoff at predict.com (William Finnoff) Date: Fri, 28 Jan 94 12:21:01 MST Subject: Batch Backprop versus Incremental Message-ID: <9401281921.AA28545@predict.com> H Cribbs writes: >Dear Connectionists, > I attended the 5th International Conference on Genetic Algorithms >this summer, and in one of the sessions on combinations of genetic >algorithms (GAs) and Neural Nets(ANNs) a gentleman from the U.K. >suggested that Batch mode learning could possibly be unstable in >the long term for backpropagation. I did not know the gentleman >and when I asked for a reference he could not provide one. >Does anyone have any kind of proof stating that one method is better >than another? Or that possibly batch backprop is unstable in <> >sense? As has been known for quite some time, incremental backprop can be viewed as a type of stochastic algorithm, which in the small step size limit will be essentially equivalent to the batch algorithm. Since the batch version of backprop is a type of gradient descent, there are only two stability issues involved. The first is the possibility that the weights will become infinitely large, which is possible if there is no genuine minimum, (local or global) which can be reached by the training process from a given starting point. An example where this occurs is when the function which one is trying to approximate with the network contains a discontinuity (a step function, say). This sort of behavior is sometimes called "converging to infinity". One should note here, that the error will still always decrease during training (though it won't necessarily converge), unless one encounters problems of "numerical instability". Problems of numerical instability are due to the fact that one is using only a discrete version of a "true" gradient descent. That is, the training algorithm with a constant step size can be viewed as the Euler approximation of the solution to a differential equation. For the solution to this differential equation, the error will always decrease and will either converge to a (perhaps local) minimum of the error function, or to infinity as described above. The error for the training may not always decrease and the training process can "explode" if the step size is choosen too "large". The question of what is "large" depends essentially on the size of the eigenvalues of the matrix of second derivatives of the error function, in particular, the smallest one. If the ratio between the largest and smallest eigenvalue is "large" the differential equation is referred to as being "stiff" or poorly conditioned. The trade off that has to be achieved is between stability (achieved by having a small step size) and speed of convergence (achieved by having a larger step size). It should be noted that the conditioning of the problem will also effect the stability of the incremental version of backprop, since it is also only an approximation of the solution to the same differential equation. The problems of numerical stability can be reduced by using Newton or Quasi-Newton methods (sometimes a problem for neural networks, due to the dimension of typical problems, where hundreds or thousands of parameters may be involved) or by regularization, which modifies the error function to improve the conditioning of the Hessian. The simplest type of conditioning is to simply add a quadratic term in the weights to the error function, i.e., if E(W) is the original error funtion (viewed as a function of the vector of network weights W = (w_i)_{i=1,,,.n}) then add a (penalty) term P_{\lambda}(W) = \lambda \sum_{i=1,...,n} w_i^2, which leads to a "weight decay" term in the training algorithm. This modification of the error function also has the effect of preventing the traing process from converging to infinity and will often improve the generalization ability of the network trained using the modified error function. The disadvantage with this is that one then has another parameter to choose (the weighting factor \lambda) and that the penalty term tends to create additional local minima (particularly around zero) in which one will frequently get stuck while searching for a "good" solution to the minimization problem, which brings us back to the incremental version of backprop. The advantages with using the incremental versions of backprop (in my opinion) have nothing to do with stability issues. First, and most obvious, is that it can be implemented online. Second is the question of efficiency: Since weight updates are made more frequently, the reduction in error can be much faster than with the batch algorithm, although this will depend on the specific nature of the data being used. Finally, due to its stochastic nature, the incremental version of backprop has a "quasi-annealling" character which makes it less likely to get stuck in local minima than the batch training process; (this statement can be made fairly rigorous, consult the references given below). References: 1) Battiti, R. (1992). First- and second order methods for learning: Between steepest descent and Newton's method, {\it Neural Computation} 4, 141-166. 2) Benveniste, A., M\'etivier, M. and Priouret, P., { \it Adaptive Algorithms and Stochastic Approximations}, Springer Verlag (1987). 3) Bouton C., Approximation Gaussienne d'algorithmes stochastiques a dynamique Markovienne. Thesis, Paris VI, (in French), (1985). 4) Darken C. and Moody J., Note on learning rate schedules for stochastic optimization, in{\it Advances in Neural Information Processing Systems 3}, Lippmann, R. Moody, J., and Touretzky, D., ed., Morgan Kaufmann, San Mateo, (1991). 5) Finnoff, W., Asymptotics for the constant step size backpropagation algorithm and resistance to local minima, {\it Neural Computation}, 6, pp. 285-293, 1994. 6) Finnoff, W., The dynamics of generalization for first and second order parameter adaptation methods, submitted to {\it IEEE Trans.~on Information Theory}. 7) Hornik, K. and Kuan, C.M., Convergence of learning algorithms with constant learning rates, {\it IEEE Trans. on Neural Networks} 2, pp. 484-489, (1991). 8) Krasovskii, N., {\it The Stability of Motion}, Stanford University Press, Stanford, (1963). 9) Le Cun, Y., Kanter I. and Solla, S. (1990). Second order properties of error surfaces: Learning time and generalization, In R. Lippman, J. Moody and D. Touretzy (Eds.), {\it Advances in Neural Information Processing III} (pp.918-924). San Mateo: Morgan Kaufman. 10) White, H. 1989. Learning in artificial neural networks: A statistical perspective, {\it Neural Computation} 1: 425-464. 11) White, H., Some asymptotic results for learning in single hidden-layer feedforward network models, Jour. Amer. Stat. Ass. 84, no. 408, pp. 1003-1013, (1989)I. Cheers William %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% William Finnoff Prediction Co. 320 Aztec St., Suite B Santa Fe, NM, 87501, USA Tel.: (505)-984-3123 Fax: (505)-983-0571 e-mail: finnoff at predict.com From yves at netid.com Fri Jan 28 14:43:02 1994 From: yves at netid.com (Yves Chauvin) Date: Fri, 28 Jan 94 11:43:02 PST Subject: HMMs and Molecular Biology Message-ID: <9401281943.AA00678@netid.com> **DO NOT FORWARD TO OTHER GROUPS** The following papers, "Hidden Markov Models of Biological Primary Sequence Information", to be published in the Proceedings of the National Academy of Sciences (USA), vol. 91, February 94. and "Hidden Markov Models for Human Genes", to be published in the Proceedings of the 1993 NIPS conference, vol. 6. have been placed on ftp site. Further information and retrieval instructions are given below. Yves Chauvin yves at netid.com ___________________________________________________________________________ Hidden Markov Models of Biological Primary Sequence Information Pierre Baldi Jet Propulsion Laboratory and Division of Biology, California Institute of Technology Pasadena, CA 91109 Yves Chauvin Net-ID, Inc. Tim Hunkapiller University of Washington Marcella A. McClure University of Nevada Hidden Markov Model (HMM) techniques are used to model families of biological sequences. A smooth and convergent algorithm is introduced to iteratively adapt the transition and emission parameters of the models from the examples in a given family. The HMM approach is applied to three protein families: globins, immunoglobulins and kinases. In all cases, the models derived capture the important statistical characteristic of the family and can be used for a number of tasks including: multiple alignments, motif detection and classification. For $K$ sequences of average length $N$, this approach yields an effective multiple alignment algorithm which requires $O(KN^2)$ operations, linear in the number of sequences. ___________________________________________________________________________ Hidden Markov Models for Human Genes Pierre Baldi Jet Propulsion Laboratory and Division of Biology, California Institute of Technology Pasadena, CA 91109 Soren Brunak The Technical University of Denmark Yves Chauvin Net-ID, Inc. Jacob Engelbrecht The Technical University of Denmark Anders Krogh The Technical University of Denmark Human genes are not continuous but rather consist of short coding regions (exons) interspersed with highly variable non-coding regions (introns). We apply HMMs to the problem of modeling exons, introns and detecting splice sites in the human genome. Our most interesting result so far is the detection of particular oscillatory patterns, with a minimal period of roughly 10 nucleotides, that seem to be characteristic of exon regions and may have significant biological implications. ___________________________________________________________________________ Retrieval instructions: The papers are "baldi.bioprimseq.ps.z" and "baldi.humgenes.ps.z". To retrieve these files: % ftp netcom.com Connected to netcom.com. 220 netcom FTP server (Version 2.0WU(10) [...] ready. Name (netcom.com:yourname): anonymous 331 Guest login ok, send your complete e-mail address as password. Password: .. ftp> cd pub/netid/papers ftp> ls ftp> binary ftp> get ftp> close .. % gunzip From hilario at cui.unige.ch Sat Jan 29 10:48:03 1994 From: hilario at cui.unige.ch (Hilario Melanie) Date: Sat, 29 Jan 1994 16:48:03 +0100 Subject: CFP: ECAI-94 WS on Combining Symbolic and Connectionist Processing Message-ID: <165*/S=hilario/OU=cui/O=unige/PRMD=switch/ADMD=arcom/C=ch/@MHS> Dear People, Please distribute the following CFP via the connectionists mailing list. Many thanks, Melanie Hilario ---------------------------------------------------------------------------- Call for Papers COMBINING SYMBOLIC AND CONNECTIONIST PROCESSING Workshop held in conjunction with ECAI-94 August 9, 1994 - Amsterdam, The Netherlands Until a few years ago, the history of AI has been marked by two parallel, often antagonistic streams of development -- classical or symbolic AI and connectionist processing. A recent research trend, premissed on the complementarity of these two paradigms, strives to build hybrid systems which combine the advantages of both to overcome the limitations of each. For instance, attempts have been made to accomplish complex tasks by blending neural networks with rule-based or case-based reasoning. This workshop will be the first Europe-wide effort to bring together researchers active in the area in view of laying the groundwork for a theory and methodology of symbolic/connectionist integration (SCI). The workshop will focus on the following topics: o theoretical (cognitive and computational) foundations of SCI o techniques and mechanisms for combining symbolic and neural processing methods (e.g. ways of improving and going beyond state-of-the-art rule compilation and extraction techniques) o outstanding problems encountered and issues involved in SCI (e.g. Which symbolic or connectionist representation schemes are best adapted to SCI? The vector space used in neural nets and the symbolic space have fundamental mathematical differences; how will these differences impact SCI? Do we have the conceptual tools needed to cope with this representation problem?) o profiles of application domains in which SCI has been (or can be) shown to perform better than traditional approaches o description, analysis and comparison of implemented symbolic/connectionist systems SUBMISSION REQUIREMENTS Prospective participants should submit an extended abstract to the contact person below, either via email in postscript format or via regular mail, in which case 3 copies are required. Each submission should include a separate information page containing the title of the paper, author names and affiliations, and the complete address (including telephone, fax and email) of the first author. The paper itself should not exceed 12 pages. Submission deadline is April 1, 1994. Each paper will be reviewed by at least two members of the Program Committee. Notification of acceptance or rejection will be sent to first authors by May 1, 1994. Camera-ready copies of accepted papers are due on June 1st and will be reproduced for distribution at the workshop. Those who wish to participate without presenting a paper should send a request describing their research interests and/or previous work in the field of SCI. Since attendance will be limited to ensure effective interaction, these requests will be considered after screening of submitted papers. All workshop participants are required to register for the main conference. PROGRAM COMMITTEE Bernard Amy (LIFIA-IMAG, Grenoble, France) Patrick Gallinari (LAFORIA, University of Paris 6, France) Franz Kurfess (Dept. Neural Information Processing, University of Ulm, Germany) Christian Pellegrini (CUI, University of Geneva, Switzerland) Alessandro Sperduti (CSD, University of Pisa, Italy) IMPORTANT DATES Submission deadline April 1, 1994 Notification of acceptance/rejection May 1, 1994 Final papers due June 1, 1994 Date of the workshop August 9, 1994 CONTACT PERSON Melanie Hilario CUI - University of Geneva 24 rue General Dufour CH-1211 Geneva 4 Voice: +41 22/705 7791 Fax: +41 22/320 2927 Email: hilario at cui.unige.ch From N.Sharkey at dcs.shef.ac.uk Sat Jan 29 08:08:29 1994 From: N.Sharkey at dcs.shef.ac.uk (N.Sharkey@dcs.shef.ac.uk) Date: Sat, 29 Jan 94 13:08:29 GMT Subject: IEE Colloquia Message-ID: <9401291308.AA10480@entropy.dcs.shef.ac.uk> IEE COLLOQUIUM IN LONDON, UK SYMBOLIC AND NEURAL COGNITIVE ENGINEERING Colloquium organised by Professional Group C4 (Artificial intelligence) of the Institute of Electical Engineers (IEE) to be held at Savoy Place on Monday, 14 February 1994 PROVISIONAL PROGRAMME 9.30 am Registration and coffee Chairman: Professor I Aleksander (Imperial College) 10.00 Chairman's introduction 1 10.10 A review of cognitive symbolic engineering: Professor B Richards (Imperial College) 2 10.40 The interplay of symbolic and adaptive techniques: R Garigliano and D J Nettleton (University of Durham) 3 11.10 Engineering cognitive systems - some conceptual issues: R Paton (University of Liverpool) 4 11.40 Variable binding in a neural network using a distributed representation: A Browne and J Pilkington (South Bank University) 12.10 LUNCH 5 1.30 Connectionist advances in natural language processing: Professor N Sharkey (University of Sheffield) 6 2.00 Systematicity and generalisation in connectionist models: L F Niklasson (University of Exeter) 7 2.30 Hierarchical symbolic structures and knowledge chunking: B K Purhoit (BT Laboratories) and J F Boyce (King's College London) 3.00 TEA 8 3.15 Relational computing: Professor J Taylor (King's College London) 9 3.45 The research challenge for symbolic and neural approaches: Professor I Aleksander (Imperial College) 4.15 Discussion 4.45 CLOSE The IEE is not, as a body, responsible for the views or opinions expressed by individual authors or speakers. 151/36/38 ref: 94/038 MB OTHER EVENTS ORGANISED BY THE COMPUTING AND CONTROL DIVISION TO BE HELD FEBRUARY 1994 1 Tue Colloquium on High performance applications of parallel PG C2 architectures 7 Mon Colloquium on Intelligent front-ends for existing systems PG C4 10 Thur 11TH COMPUTING AND CONTROL LECTURE Out of control into systems engineering research? By Professor C J Harris (University of Southampton) 21 Mon Colloquium on Modelling of controlled natural energy PG C6 systems 23 Wed Colloquium on Vehicle diagnostics in Europe PG C12 24 Thur Colloquium on Implications of the new legislation on PG C5 work with display screen equipment 28 Mon Colloquium on Molecular bioinformatics PG C4 Joint meeting with PAPAGENA Further details of the above events can be obtained from the Secretary, LS(D)CA, IEE, Savoy Place, London WC2R 0BL or by telephoning 071 240 1871 Ext: 2206 From srx014 at cck.coventry.ac.uk Mon Jan 31 10:59:00 1994 From: srx014 at cck.coventry.ac.uk (CRReeves) Date: Mon, 31 Jan 94 10:59:00 WET Subject: Batch Backprop versus Incremental (fwd) Message-ID: <15509.9401311059@cck.coventry.ac.uk> On 27th January, Brown Cribbs wrote: > > Dear Connectionists, > I attended the 5th International Conference on Genetic Algorithms > this summer, and in one of the sessions on combinations of genetic > algorithms (GAs) and Neural Nets(ANNs) a gentleman from the U.K. > suggested that Batch mode learning could possibly be unstable in > the long term for backpropagation. I did not know the gentleman > and when I asked for a reference he could not provide one. > > Does anyone have any kind of proof stating that one method is better > than another? Or that possibly batch backprop is unstable in <> > sense? > I thought a contribution from the UK was necessary, particularly in view of Scott Fahlman's later, somewhat provocative, remarks! I've seen a preprint of a paper by Steve Ellacott (University of Brighton) in which he considers just this problem. This may have been the paper referred to in the original question. He considers the question of {\em numerical\/} stability of batch and case-by-case training, and shows that in this sense there are conditions under which the delta rule is unstable for batch updates. He then proceeds to look at the generalized delta rule, with similar results at least in the neighbourhood of a local minimum. Of course, the choice of learning rate will affect the conclusions in a particular case. I understand these strange ideas have been expanded into a chapter (called "The Numerical Analysis Approach") of a book just published (end '93): Mathematical Approaches to Neural Networks (J.G.Taylor - Ed.) Elsevier Science Publishers ISBN 0-444-81692-5 -- ___________________________________________ | Colin Reeves | | Division of Statistics and OR | | School of Mathematical and Information | | Sciences | | Coventry University | | Priory St | | Coventry CV1 5FB | | tel :+44 (0)203 838979 | | fax :+44 (0)203 838585 | | email: CRReeves at uk.ac.cov.cck | | (alternative email: srx014 at cck.cov.ac.uk) | |___________________________________________| From presnik at caesar.East.Sun.COM Mon Jan 31 00:30:07 1994 From: presnik at caesar.East.Sun.COM (Philip Resnik - Sun Microsystems Labs BOS) Date: Mon, 31 Jan 1994 10:30:07 +0500 Subject: CFP: Combining symbolic and statistical approaches to language Message-ID: <9401311530.AA10552@caesar.East.Sun.COM> Hello, Although the following workshop does not specifically concern connectionist approaches, one of our goals is to initiate new discussions with the wide variety of researchers who have been thinking about similar issues. That includes many members of this list, and I would like to encourage connectionist researchers to participate. Philip ---------------------------------------------------------------- ***** CALL FOR PAPERS ****** THE BALANCING ACT: Combining Symbolic and Statistical Approaches to Language 1 July 1994 New Mexico State University Las Cruces, New Mexico, USA A workshop in conjunction with the 32nd Annual Meeting of the Association for Computational Linguistics (27-30 June 1994) A renaissance of interest in corpus-based statistical methods has rekindled old controversies -- rationalist vs. empiricist philosophies, theory-driven vs. data-driven methodologies, symbolic vs. statistical techniques. The aim of this workshop is to set aside a priori biases and explore the balancing act that must take place when symbolic and statistical approaches are brought together. We plan to accept papers from authors having a wide range of perspectives, and to initiate a discussion that includes philosophical, theoretical, and practical issues. Submissions to the workshop must describe research in which both symbolic and statistical methods play a part. All research of this kind requires that the researcher make choices: What knowledge will be represented symbolically and how will it be obtained? What assumptions underlie the statistical model? What is the researcher gaining by combining approaches? Questions like these, and the metaphor of the balancing act, will provide a unifying theme to draw contributions from a wide spectrum of language researchers. ORGANIZERS: Judith Klavans, Columbia Univerisity Philip Resnik, Sun Microsystems Laboratories, Inc. REQUIREMENTS: Papers should describe original work; they should clearly emphasize the type of paper to be presented (e.g. implementation, philosophical, etc.) and the state of completion of the research. A paper accepted for presentation cannot be presented or have been presented at any other meeting. In addition to the workshop proceedings, plans for publication as a book require that papers not have been published in any other publicly available proceedings. Papers submitted to other conferences will be considered, as long as this fact is clearly indicated in the submission. FORMAT FOR SUBMISSION: Following guidelines for the ACL meeting, authors should submit preliminary versions of their papers, not to exceed 3200 words (exclusive of references). Papers outside the specified length and formatting requirements are subject to rejection without review. Papers should be headed by a title page containing the paper title, a short (5 line) summary and a specification of the subject area(s). If the author wishes reviewing to be blind, a separate page with author identification information must be submitted. SUBMISSION MEDIA: Papers may be submitted electronically or in hard copy to either organizer at the addresses given below. Electronic submissions should be either self-contained LaTeX source or plain text. LaTeX submissions must use the ACL submission style (aclsub.sty) retrievable from the ACL LISTSERV server (access to which is described below) and should not refer to any external files or styles except for the standard styles for TeX 3.14 and LaTeX 2.09. A model submission modelsub.tex is also provided in the archive, as well as a bibliography style acl.bst. Note that the bibliography for a submission cannot be submitted as separate .bib file; the actual bibliography entries must be inserted in the submitted LaTeX source file. Be sure that e-mail submissions have no lines longer than 80 characters to avoid mailer problems. Hard copy submissions should consist of four (4) copies of the paper. A plain text version of the identification page should be sent separately by electronic mail if possible, giving the following information: title, author(s), address(es), abstract, content areas, word count. Schedule: Papers must be received by 15 March 1994. Late papers will not be considered. Notification of receipt will be mailed to the first author (or designated author) soon after receipt. Authors will be notified of acceptance by 10 April 1994. Camera-ready copies of final papers prepared in a double-column format, preferably using a laser printer, must be received by 10 May 1994, along with a signed copyright release statement. The ACL LaTeX proceedings format is available through the ACL LISTSERV. REGISTRATION: Registration fees are $25 for participants who register by 15 May 1994. Late registrations will be $30. Registration includes a copy of the proceedings, lunch, and refreshments during the day. Payment in US$ checks payable to ACL or credit card payment (Visa/Mastercard) can be sent to Philip Resnik at the address below. Please submit the following information along with payment: name affiliation postal address email method of payment (check or credit card) credit card info (name, card number, expiration date) dietary requirements (vegetarian, kosher, etc) ACL INFORMATION: For other information on the ACL conference which precedes the workshop and on the ACL more generally, please use the ACL LISTSERV, described below. ACL LISTSERV: Listserv is a facility to allow access to an electronic document archive by electronic mail. The ACL LISTSERV has been set up at Columbia University's Department of Computer Science. Requests from the archive should be sent as e-mail messages to listserv at cs.columbia.edu with an empty subject field and the message body containing the request command. The most useful requests are "help" for general help on using LISTSERV, "index acl-l" for the current contents of the ACL archive and "get acl-l " to get a particular file named >from the archive. For example, to get an ACL membership form, a message with the following body should be sent: get acl-l membership-form.txt Answers to requests are returned by e-mail. Since the server may have many requests for different archives to process, requests are queued up and may take a while (say, overnight) to be fulfilled. The ACL archive can also be accessed by anonymous FTP. Here is an example of how to get the same file by FTP (user typein is underlined): $ ftp cs.columbia.edu ------------------- Name (cs.columbia.edu:pereira): anonymous --------- Password:pereira at research.att.com << not echoed ------------------------ ftp> cd acl-l -------- ftp> get membership-form.txt.Z ------------------------- ftp> quit ---- $ uncompress membership-form.txt.Z -------------------------------- This file is listed under acl-l/ACL94/Workshop_balancing_act.ascii.Z. SPONSORSHIP: This workshop is sponsored by the Association for Computational Linguistics (ACL). It is organized by: Judith L. Klavans Philip Resnik Columbia University Sun Microsystems Laboratories, Inc. Department of Computer Science Mailstop UCHL03-207 500 W 120th Street Two Elizabeth Drive New York, NY 10027, USA Chelmsford, MA 01824-4195 USA klavans at cs.columbia.edu philip.resnik at east.sun.com Phone: (212) 939-7120 Phone: (508) 442-0841 Fax: (914) 478-1802 Fax: (508) 250-5067 [94-01-27] From harnad at Princeton.EDU Mon Jan 31 20:52:35 1994 From: harnad at Princeton.EDU (Stevan Harnad) Date: Mon, 31 Jan 94 20:52:35 EST Subject: Human Memory: BBS Call for Commentators Message-ID: <9402010152.AA22714@clarity.Princeton.EDU> Below is the abstract of a forthcoming target article by: MS Humphreys, J Wiles & S Dennis on: TOWARD A THEORY OF HUMAN MEMORY: DATA STRUCTURES AND ACCESS PROCESSES This article has been accepted for publication in Behavioral and Brain Sciences (BBS), an international, interdisciplinary journal providing Open Peer Commentary on important and controversial current research in the biobehavioral and cognitive sciences. Commentators must be current BBS Associates or nominated by a current BBS Associate. To be considered as a commentator for this article, to suggest other appropriate commentators, or for information about how to become a BBS Associate, please send email to: harnad at clarity.princeton.edu or harnad at pucc.bitnet or write to: BBS, 20 Nassau Street, #240, Princeton NJ 08542 [tel: 609-921-7771] To help us put together a balanced list of commentators, please give some indication of the aspects of the topic on which you would bring your areas of expertise to bear if you were selected as a commentator. An electronic draft of the full text is available for inspection by anonymous ftp according to the instructions that follow after the abstract. ____________________________________________________________________ TOWARD A THEORY OF HUMAN MEMORY: DATA STRUCTURES AND ACCESS PROCESSES Michael S. Humphreys, Department of Psychology Janet Wiles, Departments of Psychology and Computer Science Simon Dennis, Department of Computer Science University of Queensland QLD 4072 Australia mh at psych.psy.uq.oz.au KEYWORDS: amnesia, binding, context, data structure, lexical decision, memory access, perceptual identification, recall, recognition, representation. ABSTRACT: A theory of the data structures and access processes of human memory is proposed and demonstrated on 10 tasks. The two starting points are Marr's (1982) ideas about the levels at which we can understand an information processing device and the standard laboratory paradigms which demonstrate the power and complexity of human memory. The theory suggests how to capture the functional characteristics of human memory (e.g., analogies, reasoning, etc.) without having to be concerned with implementational details. Ours is not a performance theory. We specify what is computed by the memory system with a multidimensional task classification which encompasses existing classifications (e.g., the distinction between implicit and explicit, data driven and conceptually driven, and simple associative (2-way bindings) and higher order tasks (3-way bindings). This provides a broad basis for new experimentation. Our formal language clarifies the binding problem in episodic memory, the role of input pathways in both episodic and semantic (lexical) memory, the importance of the input set in episodic memory, and the ubiquitous calculation of an intersection in theories of episodic and lexical access. -------------------------------------------------------------- To help you decide whether you would be an appropriate commentator for this article, an electronic draft is retrievable by anonymous ftp from princeton.edu according to the instructions below (the filename is bbs.humphreys). Please do not prepare a commentary on this draft. Just let us know, after having inspected it, what relevant expertise you feel you would bring to bear on what aspect of the article. The file is also retrievable using archie, gopher, veronica, etc. ------------------------------------------------------------- To retrieve a file by ftp from an Internet site, type either: ftp princeton.edu or ftp 128.112.128.1 When you are asked for your login, type: anonymous Enter password as queried (your password is your actual userid: yourlogin at yourhost.whatever.whatever - be sure to include the "@") cd /pub/harnad/BBS To show the available files, type: ls Next, retrieve the file you want with (for example): get bbs.humphreys When you have the file(s) you want, type: quit These files can also be retrieved using gopher, archie, veronica, etc. ---------- Where the above procedure is not available there are two fileservers: ftpmail at decwrl.dec.com and bitftp at pucc.bitnet that will do the transfer for you. To one or the other of them, send the following one line message: help for instructions (which will be similar to the above, but will be in the form of a series of lines in an email message that ftpmail or bitftp will then execute for you). JANET users without ftp can instead utilise the file transfer facilities at sites uk.ac.ft-relay or uk.ac.nsf.sun. Full details are available on request. -------------------------------------------------------------