From Connectionists-Request at cs.cmu.edu Thu Jul 1 00:05:14 1993 From: Connectionists-Request at cs.cmu.edu (Connectionists-Request@cs.cmu.edu) Date: Thu, 01 Jul 93 00:05:14 -0400 Subject: Bi-monthly Reminder Message-ID: <24313.741499514@B.GP.CS.CMU.EDU> *** DO NOT FORWARD TO ANY OTHER LISTS *** This note was last updated January 4, 1993. This is an automatically posted bi-monthly reminder about how the CONNECTIONISTS list works and how to access various online resources. CONNECTIONISTS is not an edited forum like the Neuron Digest, or a free-for-all newsgroup like comp.ai.neural-nets. It's somewhere in between, relying on the self-restraint of its subscribers. Membership in CONNECTIONISTS is restricted to persons actively involved in neural net research. The following posting guidelines are designed to reduce the amount of irrelevant messages sent to the list. Before you post, please remember that this list is distributed to over a thousand busy people who don't want their time wasted on trivia. Also, many subscribers pay cash for each kbyte; they shouldn't be forced to pay for junk mail. Happy hacking. -- Dave Touretzky & David Redish --------------------------------------------------------------------- What to post to CONNECTIONISTS ------------------------------ - The list is primarily intended to support the discussion of technical issues relating to neural computation. - We encourage people to post the abstracts of their latest papers and tech reports. - Conferences and workshops may be announced on this list AT MOST twice: once to send out a call for papers, and once to remind non-authors about the registration deadline. A flood of repetitive announcements about the same conference is not welcome here. - Requests for ADDITIONAL references. This has been a particularly sensitive subject lately. Please try to (a) demonstrate that you have already pursued the quick, obvious routes to finding the information you desire, and (b) give people something back in return for bothering them. The easiest way to do both these things is to FIRST do the library work to find the basic references, then POST these as part of your query. Here's an example: WRONG WAY: "Can someone please mail me all references to cascade correlation?" RIGHT WAY: "I'm looking for references to work on cascade correlation. I've already read Fahlman's paper in NIPS 2, his NIPS 3 abstract, and found the code in the nn-bench archive. Is anyone aware of additional work with this algorithm? I'll summarize and post results to the list." - Announcements of job openings related to neural computation. - Short reviews of new text books related to neural computation. To send mail to everyone on the list, address it to Connectionists at CS.CMU.EDU ------------------------------------------------------------------- What NOT to post to CONNECTIONISTS: ----------------------------------- - Requests for addition to the list, change of address and other administrative matters should be sent to: "Connectionists-Request at cs.cmu.edu" (note the exact spelling: many "connectionists", one "request"). If you mention our mailing list to someone who may apply to be added to it, please make sure they use the above and NOT "Connectionists at cs.cmu.edu". - Requests for e-mail addresses of people who are believed to subscribe to CONNECTIONISTS should be sent to postmaster at appropriate-site. If the site address is unknown, send your request to Connectionists-Request at cs.cmu.edu and we'll do our best to help. A phone call to the appropriate institution may sometimes be simpler and faster. - Note that in many mail programs a reply to a message is automatically "CC"-ed to all the addresses on the "To" and "CC" lines of the original message. If the mailer you use has this property, please make sure your personal response (request for a Tech Report etc.) is NOT broadcast over the net. - Do NOT tell a friend about Connectionists at cs.cmu.edu. Tell him or her only about Connectionists-Request at cs.cmu.edu. This will save your friend from public embarrassment if she/he tries to subscribe. ------------------------------------------------------------------------------- The CONNECTIONISTS Archive: --------------------------- All e-mail messages sent to "Connectionists at cs.cmu.edu" starting 27-Feb-88 are now available for public perusal. A separate file exists for each month. The files' names are: arch.yymm where yymm stand for the obvious thing. Thus the earliest available data are in the file: arch.8802 Files ending with .Z are compressed using the standard unix compress program. To browse through these files (as well as through other files, see below) you must FTP them to your local machine. ------------------------------------------------------------------------------- How to FTP Files from the CONNECTIONISTS Archive ------------------------------------------------ 1. Open an FTP connection to host B.GP.CS.CMU.EDU (Internet address 128.2.242.8). 2. Login as user anonymous with password your username. 3. 'cd' directly to one of the following directories: /usr/connect/connectionists/archives /usr/connect/connectionists/bibliographies 4. The archives and bibliographies directories are the ONLY ones you can access. You can't even find out whether any other directories exist. If you are using the 'cd' command you must cd DIRECTLY into one of these two directories. Access will be denied to any others, including their parent directory. 5. The archives subdirectory contains back issues of the mailing list. Some bibliographies are in the bibliographies subdirectory. Problems? - contact us at "Connectionists-Request at cs.cmu.edu". ------------------------------------------------------------------------------- How to FTP Files from the Neuroprose Archive -------------------------------------------- Anonymous FTP on archive.cis.ohio-state.edu (128.146.8.52) pub/neuroprose directory This directory contains technical reports as a public service to the connectionist and neural network scientific community which has an organized mailing list (for info: connectionists-request at cs.cmu.edu) Researchers may place electronic versions of their preprints or articles in this directory, announce availability, and other interested researchers can rapidly retrieve and print the postscripts. This saves copying, postage and handling, by having the interested reader supply the paper. (Along this line, single spaced versions, if possible, will help!) To place a file, put it in the Inbox subdirectory, and send mail to pollack at cis.ohio-state.edu. Within a couple of days, I will move and protect it, and suggest a different name if necessary. When you announce a paper, you should consider whether (A) you want it automatically forwarded to other groups, like NEURON-DIGEST, (which gets posted to comp.ai.neural-networks) and if you want to provide (B) free or (C) prepaid hard copies for those unable to use FTP. If you do offer hard copies, be prepared for an onslaught. One author reported that when they allowed combination AB, the rattling around of their "free paper offer" on the worldwide data net generated over 2000 hardcopy requests! Experience dictates the preferred paradigm is to announce an FTP only version with a prominent "**DO NOT FORWARD TO OTHER GROUPS**" at the top of your announcement to the connectionist mailing list. Current naming convention is author.title.filetype[.Z] where title is enough to discriminate among the files of the same author. The filetype is usually "ps" for postscript, our desired universal printing format, but may be tex, which requires more local software than a spooler. Very large files (e.g. over 200k) must be squashed (with either a sigmoid function :) or the standard unix "compress" utility, which results in the .Z affix. To place or retrieve .Z files, make sure to issue the FTP command "BINARY" before transfering files. After retrieval, call the standard unix "uncompress" utility, which removes the .Z affix. An example of placing a file is attached as an appendix, and a shell script called Getps in the directory can perform the necessary retrival operations. For further questions contact: Jordan Pollack Assistant Professor CIS Dept/OSU Laboratory for AI Research 2036 Neil Ave Email: pollack at cis.ohio-state.edu Columbus, OH 43210 Phone: (614) 292-4890 Here is an example of naming and placing a file: gvax> cp i-was-right.txt.ps rosenblatt.reborn.ps gvax> compress rosenblatt.reborn.ps gvax> ftp archive.cis.ohio-state.edu Connected to archive.cis.ohio-state.edu. 220 archive.cis.ohio-state.edu FTP server ready. Name: anonymous 331 Guest login ok, send ident as password. Password:neuron 230 Guest login ok, access restrictions apply. ftp> binary 200 Type set to I. ftp> cd pub/neuroprose/Inbox 250 CWD command successful. ftp> put rosenblatt.reborn.ps.Z 200 PORT command successful. 150 Opening BINARY mode data connection for rosenblatt.reborn.ps.Z 226 Transfer complete. 100000 bytes sent in 3.14159 seconds ftp> quit 221 Goodbye. gvax> mail pollack at cis.ohio-state.edu Subject: file in Inbox. Jordan, I just placed the file rosenblatt.reborn.ps.Z in the Inbox. The INDEX sentence is "Boastful statements by the deceased leader of the neurocomputing field." Please let me know when it is ready to announce to Connectionists at cmu. BTW, I enjoyed reading your review of the new edition of Perceptrons! Frank ------------------------------------------------------------------------ How to FTP Files from the NN-Bench Collection --------------------------------------------- 1. Create an FTP connection from wherever you are to machine "pt.cs.cmu.edu" (128.2.254.155). 2. Log in as user "anonymous" with password your username. 3. Change remote directory to "/afs/cs/project/connect/bench". Any subdirectories of this one should also be accessible. Parent directories should not be. 4. At this point FTP should be able to get a listing of files in this directory and fetch the ones you want. Problems? - contact us at "nn-bench-request at cs.cmu.edu". From maass at igi.tu-graz.ac.at Thu Jul 1 13:19:37 1993 From: maass at igi.tu-graz.ac.at (Wolfgang Maass) Date: Thu, 01 Jul 93 19:19:37 +0200 Subject: new paper in neuroprose Message-ID: <9307011719.AA17606@figids01.tu-graz.ac.at> FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/maass.agnostic.ps.Z The file maass.agnostic.ps.Z is now available for copying from the Neuroprose repository. This is a 22-page long paper. Hardcopies are not available. AGNOSTIC PAC-LEARNING OF FUNCTIONS ON ANALOG NEURAL NETS by Wolfgang Maass Institute for Theoretical Computer Science Technische Universitaet Graz, A-8010 Graz, Austria email: maass at igi.tu-graz.ac.at Abstract: This is a revised version of a preprint from May 93. In this new version we have added a parallelization of the learning algorithm LEARN that runs in polynomial time in ALL relevant parameters. We consider learning of real-valued functions on analog neural nets in Haussler's refinement of Valiant's model for probably approximately correct learning ("PAC-learning"). One commonly refers to this refined model as "agnostic PAC-learning", since it requires no a-priori assumptions about the structure of the learning target. The learning target need not even be a function, instead it may be any distribution of input/output pairs. In particular, arbitrary errors and imprecisions are permitted in the training data. Hence the setup of this model is well-suited for the analysis of real-world learning problems on neural Cnets. The goal of the learner in this model is to find a hypothesis whose true error (resp. loss) is within an epsilon of the true error of the best hypothesis from the considered hypothesis class. In our application to neural nets this hypothesis class is given by the class of all functions that can be computed on some fixed neural net N (with arbitrary real weights). We prove a positive result about agnostic PAC-learning on an arbitrary fixed analog neural net N (with arbitrary piecewise polynomial activation functions). We first construct for such N a somewhat larger neural net N' (the "learning network"). Then we exhibit a learning algorithm LEARN that computes from the training examples an assignment of rational numbers to the weights of N' such that with high probability the true error of the function that is computed by N' with these weights is within an epsilon of that of the best hypothesis that is computable on N (with arbitrary real weights). If the number of gates in N may be viewed as a constant, then the computation time of the learning algorithm LEARN is polynomial (in all other relevant parameters). In addition one can parallelize this learning algorithm in such a way that its computation time is polynomial in ALL relevant parameters, including the number of gates in N . It should be noted that in contrast to common learning algorithms such as backwards propagation, this learning algorithm LEARN is guaranteed to solve its learning task (with high probability). As a part of this positive learning result we show that the pseudo- dimension of neural nets with piecewise polynomial activation functions can be bounded by a polynomial in the number of weights of the neural net. It has previously been shown by Haussler that the pseudo-dimension is the appropriate generalization of the VC-dimension for learning real valued functions. With the help of our upper bound on the pseudo- dimension of neural nets with piecewise polynomial activation functions we can bound the number of training-examples that are needed for agnostic PAC-learning. In this way one can reduce the task of minimizing the true error of the neural net to the finite task of minimizing its apparent error (i.e. its error on the training examples). From jong at miata.postech.ac.kr Fri Jul 2 15:04:14 1993 From: jong at miata.postech.ac.kr (Jong-Hoon Oh) Date: Fri, 2 Jul 93 15:04:14 KDT Subject: preprint Message-ID: <9307020504.AA01718@miata.postech.ac.kr> FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/oh.generalization.ps.Z The following paper has been placed in the Neuroprose archive (see above for ftp-host) in file oh.generalization.ps.Z (8 pages of output) ----------------------------------------------------------------- Generalization in a two-layer neural network Kukjin Kang, Jong-Hoon Oh Department of Physics, Pohang Institute of Science and Technology, Pohang, Kyongbuk, Korea Chulan Kwon, Youngah Park Department of Physics, Myong Ji University, Yongin, Kyonggi, Korea Learning of a fully connected two-layer neural networks with $N$ input nodes, $M$ hidden nodes and a single output node is studied using the annealed approximation. We study the generalization curve, i.e. the average generalization error as a function of the number of the examples. When the number of examples is the order of $N$, the generalization error is rapidly decreasing and the system is in a permutation symmetric(PS) phase. As the number of examples $P$ grows to the order of $MN$ the generalization error converges to a constant value. Finally the system undergoes a first order phase transition to a perfect learning and the permutation symmetry breaks. The computer simulations show a good agreement with analytic results. PACS number(s): 87.10.+e, 05.50.+s, 64.60.Cn Jong-Hoon Oh jhoh at miata.postech.ac.kr ----------------------------------------------------------------- From spotter at darwin.bio.uci.edu Fri Jul 2 15:26:17 1993 From: spotter at darwin.bio.uci.edu (Steve Potter) Date: Fri, 2 Jul 1993 12:26:17 -0700 (PDT) Subject: Cultured Neural Nets Message-ID: Below I present a bibliography of all of the researchers I know of that are growing neurons in culture on multielectrode substrates. A belated thank-you is due to several connectionists who responded to my request posted a couple of years ago. This is a surprisingly small list. If you know of someone I have missed, please send me email (spotter at darwin.bio.uci.edu). I believe that approaches such as these are likely to close the gap between the engineering and biological camps of neural network research. With long-term, multi-site monitoring of real (though simple) networks, we may learn which aspects of real neural processors must be included in our simulations if we hope to emulate the accomplishments of Mother Nature. If you are involved in this research and I have not contacted you already, please email me; I am looking for a post-doctoral position. Steve Potter Psychobiology dept. UC Irvine Irvine, CA 92717 spotter at darwin.bio.uci.edu CULTURED NETS ON MULTI-ELECTRODE SUBSTRATES: (Recent or representative publications are listed) Steve Potter 7-2-93 spotter at darwin.bio.uci.edu Masuo Aizawa (Layman's article) Freedman, D.H. (1992). If he only had a brain. Discover : 54-60. Robert L. Dawes, Martingale Research (Texas) (Proposal--Never followed up?) Dawes, R.L. (1987). Biomasscomp: A procedure for mapping the architecture of a living neural network into a machine. IEEE ICNN proceedings 3: 215-225. Mitch D. Eggers, MIT (Any subsequent work with this device?) Eggers, M.D., Astolfi, D.K., Liu, S., Zeuli, H.E., Doeleman, S.S., McKay, R., Khuon, T.S., and Ehrlich, D.J. (1990). Electronically wired petri dish: A microfabricated interface to the biological neuronal network. J. Vac. Sci. Technol. B 8: 1392-1398. Peter Fromherz, Ulm University (Germany) Fromherz, P., Offenhausser, A., Vetter, T., and Weis, J. (1991). A neuron-silicon junction: a Retzius cell of the leech on an insulated-gate field-effect transistor. Science 252: 1290-3. Guenter W. Gross, U. of N. Texas Gross, G.W. and Kowalski, J. (1991) Experimental and theoretical analysis of random nerve cell network dynamics, in Neural Networks: Concepts, applications, and implementations (P. Antognetti and B Milutinovic, Eds.) Prentice-Hall: NJ. p. 47-110. Vera Janossy, Central Research Inst. for Physics (Hungary) Janossy, V., Toth, A., Bodocs, L., Imrik, P., Madarasz, E., and Gyevai, A. (1990). Multielectrode culture chamber: a device for long-term recording of bioelectric activities in vitro. Acta Biol Hung 41: 309-20. Akio Kawana, NTT (Japan) (News article) Koppel, T. (1993). Computer firms look to the brain. Science 260: 1075-1077. Jerome Pine, Caltech Regehr, W.G., Pine, J., Cohan, C.S., Mischke, M.D., and Tank, D.W. (1989). Sealing cultured invertebrate neurons to embedded dish electrodes facilitates long-term stimulation and recording. J Neurosci Methods 30: 91-106. David W. Tank, AT&T Bell Labs (Abstract) Tank, D.W. and Ahmed, Z. (1985). Multiple site monitoring of activity in cultured neurons. Biophys. J. 47: 476a. C. D. W. Wilkinson, U. of Glasgow (Scotland) Connolly, P., Clark, P., Curtis, A.S., Dow, J.A., and Wilkinson, C.D. (1990). An extracellular microelectrode array for monitoring electrogenic cells in culture. Biosens Bioelectron 5: 223-34. Curtis, A.S., Breckenridge, L., Connolly, P., Dow, J.A., Wilkinson, C.D., and Wilson, R. (1992). Making real neural nets: design criteria. Med Biol Eng Comput 30: CE33-6. ACUTE PREPS (NOT CULTURED): Bruce C. Wheeler, U. of Illinois (Hippocampal slice) Boppart, S.A., Wheeler, B.C., and Wallace, C.S. (1992). A flexible perforated microelectrode array for extended neural recordings. Ieee Trans Biomed Eng 39: 37-42. Novak, J.L. and Wheeler, B.C. (1986). Recording from the Aplysia abdominal ganglion with a planar microelectrode array. Ieee Trans Biomed Eng 33: 196-202. Markus Meister, Harvard Meister, M., Wong, R.O., Baylor, D.A., and Shatz, C.J. (1991). Synchronous bursts of action potentials in ganglion cells of the developing mammalian retina. Science 252: 939-43. Litke, A. and Meister, M. (1991). The retinal readout array. Nuclear Instruments and Methods in Physics Research A310: 389-394. From white at TEETOT.ACUSD.EDU Tue Jul 6 19:04:41 1993 From: white at TEETOT.ACUSD.EDU (Ray White) Date: Tue, 6 Jul 1993 16:04:41 -0700 Subject: paper in neuroprose: white.c-hebb-2.ps.Z Message-ID: <9307062304.AA06538@TEETOT.ACUSD.EDU> FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/white.c-hebb-2.ps.Z ** PLEASE DO NOT FORWARD TO OTHER GROUPS ** The paper "Competitive Hebbian Learning 2: an Introduction", (8 pages) a poster to be presented at WCNN '93 in Portland Monday morning, July 12, 1993, has just been placed in the Neuroprose directory at Ohio State. Thanks again, Jordan Pollack. ABSTRACT In this paper the Competitive Hebbian Learning 2, or CHL 2, learning rule is introduced. CHL 2 is an unsupervised learning rule with characteristics which are plausible as a simplified model of biological learning, as well as useful learning properties for artificial systems. It combines a modified Hebbian learning rule with a competitive learning, and shows promise for unsupervised-learning tasks such as feature detection. The usual instructions to FTP and to print the paper apply. Ray White Departments of Physics and Computer Science University of San Diego 5998 Alcala Park San Diego, CA 92110 619-260-4627 From sperduti at ICSI.Berkeley.EDU Tue Jul 6 21:00:40 1993 From: sperduti at ICSI.Berkeley.EDU (Alessandro Sperduti) Date: Tue, 6 Jul 93 18:00:40 PDT Subject: TRs on reduced representations Message-ID: <9307070100.AA10347@icsib57.ICSI.Berkeley.EDU> FTP-host: ftp.icsi.berkeley.edu (128.32.201.7) FTP-filename: pub/techreports/tr-93-029.ps.Z FTP-filename: pub/techreports/tr-93-031.ps.Z The following technical reports are available by public ftp from the International Computer Science Institute. For hardcopies there is a small charge to cover postage and handling for each report (info at icsi.berkeley.edu). Comments welcome. Alessandro Sperduti sperduti at icsi.berkeley.edu ____________________________________________________________________________ TR-93-029 (48 pages) Labeling RAAM Alessandro Sperduti International Computer Science Institute 1947 Center Street, Suite 600 Berkeley, California 94704 TR-93-029 Abstract In this report we propose an extension of the Recursive Auto-Associative Memory (RAAM) by Pollack. This extension, the Labeling RAAM (LRAAM), is able to encode labeled graphs with cycles by representing pointers explicitly. A theoretical analysis of the constraints imposed on the weights by the learning task under the hypothesis of perfect learning and linear output units is presented. Cycles and confluent pointers result to be particularly effective in imposing constraints on the weights. Some technical problems encountered in the RAAM, such as the termination problem in the learning and decoding processes, are solved more naturally in the LRAAM framework. The representations developed for the pointers seem to be robust to recurrent decoding along a cycle. Data encoded in a LRAAM can be accessed by pointer as well as by content. The direct access by content can be achieved by transforming the encoder network of the LRAAM in a Bidirectional Associative Memory (BAM). Different access procedures can be defined according to the access key. The access procedures are not wholly reliable, however they seem to have a high likelihood of success. A geometric interpretation of the decoding process is given and the representations developed in the pointer space of a two hidden units LRAAM are presented and discussed. In particular, the pointer space results to be partitioned in a fractal-like fashion. Some effects on the representations induced by the Hopfield-like dynamics of the pointer decoding process are discussed and an encoding scheme able to retain the richness of representation devised by the decoding function is outlined. The application of the LRAAM model to the control of the dynamics of recurrent high-order networks is briefly sketched as well. TR-93-031 (19 pages) On Some Stability Properties of the LRAAM Model Alessandro Sperduti International Computer Science Institute 1947 Center Street, Suite 600 Berkeley, California 94704 TR-93-031 Abstract In this report we discuss some mathematical properties of the LRAAM model. The LRAAM model is an extension of the RAAM model by Pollack. It allows one to obtain distributed reduced representations of labeled graphs. In particular, we give sufficient conditions on the asymptotical stability of the decoding process along a cycle of the encoded structure. Data encoded in an LRAAM can also be accessed by content by transforming the LRAAM in an analog Hopfield network with hidden units and asymmetric connection matrix (CA network.) Different access procedures can be defined according to the access key. Each access procedure corresponds to a particular constrained version of the CA network. We give sufficient conditions under which the property of asymptotical stability of a fixed point in one particular constrained version of the CA network can be extended to related fixed points of different constrained versions of the CA network. An example of encoding of a labeled graph on which the theoretical results are applied is given as well. To obtain electronic copies: ftp ftp.icsi.berkeley.edu login: anonymous password: cd pub/techreports binary get tr-93-029.ps.Z get tr-93-031.ps.Z bye Then at your system: uncompress tr-93-029.ps.Z uncompress tr-93-031.ps.Z lpr -P tr-93-029.ps tr-93-031.ps From liaw%dylink.usc.edu at usc.edu Wed Jul 7 13:36:42 1993 From: liaw%dylink.usc.edu at usc.edu (Jim Liaw) Date: Wed, 7 Jul 93 10:36:42 PDT Subject: Neural Architectures and Distributed AI Message-ID: <9307071736.AA09576@dylink.usc.edu> **** Call for Papers **** Neural Architectures and Distributed AI: From Schema Assemblages to Neural Networks October 19-20, 1993 The Center for Neural Engineering University of Southern California announces a Workshop on Neural Architectures and Distributed AI: From georgiou at silicon.csci.csusb.edu Wed Jul 7 17:36:49 1993 From: georgiou at silicon.csci.csusb.edu (George M. Georgiou) Date: Wed, 7 Jul 1993 14:36:49 -0700 Subject: CFP: 2nd Int'l Conference on Fuzzy Theory and Technology Message-ID: <9307072136.AA24292@silicon.csci.csusb.edu> I am in the process of organizing two sessions on Neural Networks at the 2nd Int'l Conference on Fuzzy Theory and Technology, which will take place in Durham, N.C., October 13--16, 1993. There are a few slots for non-invited papers, and anyone interested should submit an extended abstract by July 28, 1993, to the following address: Dr. George M. Georgiou Computer Science Department California State University 5500 University Pkwy San Bernardino, CA 92407, USA FAX: (909) 880-7004 E-mail: georgiou at wiley.csusb.edu The extended abstract (maximum 3 pages of single column and single-space text with figures and tables) may be submitted by e-mail as well (in ASCII, postscript, or TeX/LaTeX form). Notification for acceptance will be sent on August 6, 1993. The final version of the full length paper must be submitted by October 14, 1993 (time of conference). Four copies of the paper shall be prepared according to the ``Information For Authors'' appearing at the back cover of {\em Information Sciences, An International Journal}, (Elsevier Publishing Co.). A full paper shall not exceed 20 pages including figures and tables. All full papers will be reviewed by experts. Revised papers will be due on April 15, 1994. Accepted papers will appear in the hard-covered proceedings (book with uniform type-setting) OR in the {\em Information Sciences Journal}. Lotfi Zadeh "Best paper award": All papers submitted to FT&T'93 will be considered for this award, the prize of which includes $2,500 plus hotel accommodations (traveling expenses excluded) at the FT&T'94. The date of announcement of the the best paper is March 30, 1994. Oral presentation in person at the FT&T'93 is required, as well an acceptance speech at the FT&T'94. The evaluation committee consists of the following 10 members: Jack Aldridge, B. Bouchon-Meunier, Abe Kandel, George Klir, I.R. Goodman, John Mordeson, Sujeet Shenoi, H. Cris Tseng, H. Zimmerman, Frank Y. Shih. (Alternates: Akira Nakamura and Frank Y. Shih) ----------------------------------------------------------------------- Further Conference Info contact: Jerry C.Y. Tyan, e-mail: ctyan at ee.egr.duke.edu, tel: (919)660-5294 OR Jing Dai, e-mail: jdai at ee.egr.duke.edu, tel: (919)660-5228 ----------------------------------------------------------------------- Exhibit Information: Interested vendors should contact: Rhett George, E.E. Dept., Duke University Tel: (919)660-5242 Fax: (919)660-5293 ----------------------------------------------------------------------- ----------------------------------------------------------------------- From mafm at cs.uwa.edu.au Thu Jul 8 13:34:17 1993 From: mafm at cs.uwa.edu.au (Matthew McDonald) Date: Thu, 8 Jul 93 13:34:17 WST Subject: Reinforcement Learning Mailing List Message-ID: <9307080534.AA02405@cs.uwa.edu.au> This message is to announce an informal mailing list devoted to reinforcement learning. The list is intended to provide an informal, unmoderated, forum for discussing subjects relevant to research in reinforcement learning; in particular, discussion of problems, interesting papers and that sort of thing is welcome. Announcements and other information relevant to researchers in the field are also welcome. People are encouraged to post abstracts of recent papers or reports. The list is intended to be fairly informal and unmoderated. If you'd like to join the list, please send mail to `reinforce-request at cs.uwa.edu.au' Cheers, -- Matthew McDonald mafm at cs.uwa.oz.au Nothing is impossible for anyone impervious to reason. From vet at cs.utwente.nl Thu Jul 8 05:08:45 1993 From: vet at cs.utwente.nl (Paul van der Vet) Date: Thu, 8 Jul 93 11:08:45 +0200 Subject: ECAI94 Message-ID: <9307080908.AA26633@utis31.cs.utwente.nl> E C A I '94 A M S T E R D A M 11th European Conference on Artificial Intelligence Amsterdam RAI International Exhibition and Congress Centre The Netherlands August 8-12, 1994 Call for Papers Call for Workshop proposals Exhibition Call for Tutorial proposals Organized by the European Coordinating Committee for Artificial Intelligence (ECCAI) Hosted by the Dutch Association for Artificial Intelligence (NVKI) The European Conference on Artificial Intelligence (ECAI) is the European forum for scientific exchange and presentation of AI research. The aim of the conference is to cover all aspects of AI research and to bring together basic research and applied research. The Technical Programme will include paper presentations, invited talks, panels, workshops, and tutorials. The conference is designed to cover all subfields of AI, including non-symbolic methods. ECAIs are held in alternate years and are organized by the European Coordinating Committee for Artificial Intelligence (ECCAI). The 11th ECAI in 1994 will be hosted by the Dutch AI Society (NVKI). The conference will take place at the Amsterdam RAI, International Exhibition and Congress Centre. E X H I B I T I O N An industrial and academic exhibition will be organized from August 9 - 11, 1994. Detailed information will be provided in the second call for papers or can be obtained at the conference office (for the adress see elsewhere). S P O N S O R S (preliminary list) Bolesian B.V. Municipality of Amsterdam University of Amsterdam Vrije Universiteit Amsterdam University of Limburg C A L L F O R P A P E R S T O P I C S O F I N T E R E S T You are invited to submit an original research paper that represents a significant contribution to any aspect of AI, including the principles underlying cognition, perception, and action in humans and machines; the design, application, and evaluation of AI algorithms and intelligent systems; and the analysis of tasks and domains in which intelligent systems perform. Theoretical and experimental results are equally welcome. Papers describing innovative ideas are especially sought providing such papers include substantial analysis of the ideas, the technology needed to realize them, and their potential impact. Of special interest this year are papers which address applied AI. Two kinds of papers are sought. The first category is case studies of AI applications that address significant real-world problems and which are used outside the AI community itself; these papers must justify the use of the AI technique, explain how the AI technology contributed to the solution and was integrated with other components, and most importantly explain WHY the application was successful (or perhaps why it failed) -- these "lessons learned" will be the most important review criteria. The second category is for papers on novel AI techniques and principles that may enable more ambitious real-world applications. All the usual AI topics are appropriate. These papers must describe the importance of the approach from an applications context, in sufficient technical detail and clarity, and clearly and thoroughly differentiate the work from previous efforts. There will be special prizes for the best papers in both these areas. S U B M I S S I O N O F P A P E R S Authors are requested to submit to the Programme Chairperson 5 copies of papers written in English in hardcopy format (electronic and fax submissions will not be accepted). Each submitted paper must conform to the following specifications. Papers should be no longer than 5500 words including references. (Each full page of figures counts as 1100 words.) Papers longer than this limit risk being rejected without refereeing. A separate title page should include the title, the name(s) of the author(s), complete address(es), email, fax and telephone numbers, the specification of between one and four Content Areas, preferably chosen from the list below and an abstract (maximum 200 words). The title page should also contain a declaration that the paper is unpublished, original work, substantially different from papers currently under review and will not be submitted elsewhere before the notification date other than to workshops and similar specialized presentations with a very limited audience. Papers should be printed on A4 or 8.5"x11" sized paper in letter quality print, with 12 point type (10 chars/inch on a typewriter), single spaced. Double sided printing is preferred. Authors who wish to check that their submission will fit into the final CRC format will be able to obtain detailed instructions including a latex style file and example postscript pages after October 15 by anonymous FTP from agora.leeds.ac.uk, directory ECAI94, or by e-mailing ecai94-style at scs.leeds.ac.uk with a message body of "help". When submitting a paper an electronic mail message should also be sent to ecai94-title at scs.leeds.ac.uk giving information in the format specified below. If an intending author has no e-mail facilities then this requirement is waived. Papers should be sent to: Programme Chairperson: Dr Tony Cohn Division of Artificial Intelligence School of Computer Studies University of Leeds Leeds LS2 9JT United Kingdom Tel.: (+44)-532-33.54.82 Fax: (+44)-532-33.54.68 E-mail: ecai94 at scs.leeds.ac.uk TITLE: AUTHOR: <first author last name, first name> AFFILIATION: <first author affiliation> AUTHOR: <second author last name, first name> AFFILIATION: <second author affiliation> ..<repeat for all authors> CORRESPONDENCE ADDRESS: <give name, address, fax and telephone on successivelines> CORRESPONDENCE E-MAIL: <give correspondence e-mail address> CONTENT AREAS: <at most four content areas, separated by semi-colons> ABSTRACT: <text of the abstract> The content areas preferably should be drawn from the topics listed below. The text of the abstract field may include formatting commands, if desired, but these should be omitted from all other fields. Work described in an accepted paper may also be illustrated with a videotape or a demo. Special sessions will be scheduled for video presentations and demos. Authors wishing to show a videotape or a demo should specify the duration and the requirements of the videotape/demo when submitting their paper for review. Reviewing criteria do not apply to these tapes. Only the submitted papers will be peer-reviewed. Authors wishing to augment their paper presentation with a video should submit a tape only after their paper has been accepted. For details concerning tape format, see the video track description below. C O N T E N T A R E A S Abduction; AI and Creativity; Artificial Life; Automated Reasoning; Automatic Programming; Belief Revision; Case Studies of AI Applications; Case-Based Reasoning; Cognitive Modelling; Common Sense Reasoning; Communication and Cooperation; Complexity of Reasoning; Computational Theories in Psychology; Computer-Aided Education; Concept Formation; Connectionist and PDP Models for AI; Constraint-Based Reasoning; Corpus-Based Language Analysis; Deduction; Description Logics; Design; Diagnosis; Discourse Analysis; Discovery; Multi-Agent Systems; Distributed Problem Solving; Enabling Technology and Systems; Epistemological Foundations; Expert System Design; Generic Applications; Genetic Algorithms; Integrating AI and Conventional Systems; Integrating Several AI Components; Kinematics; Knowledge Acquisition; Knowledge Representation; Large Scale Knowledge Engineering; Logic Programming; Machine Architectures; Machine Learning; Machine Translation; Mathematical Foundations; Model Based Reasoning; Monitoring; Natural Language Front Ends; Natural Language Processing; Navigation; Neural Networks; Nonmonotonic Reasoning; Philosophical Foundations and Implications; Plan Recognition; Planning and Scheduling; Principles of AI Applications; Qualitative Reasoning; Reactivity; Reasoning About Action; Reasoning About Physical Systems; Reasoning With Uncertainty; Resource Allocation; Robotics; Robot Navigation; Search; Sensor Interpretation; Sensory Fusion/Fission; Simulation; Situated Cognition; Social Economic, Ethical and Legal Implications; Spatial Reasoning; Speech Recognition; Standardisation, Exchange and Reuse of Ontologies or Knowledge; Parsing; Semantic Interpretation; Pragmatics; System Architectures; Temporal and Causal Reasoning; Terminological Reasoning; Text Generation and Understanding; Theorem Proving; Truth Maintenance; Tutoring Systems; User Interfaces; User Models; Verification, Validation and Testing of Knowledge-Based Systems; Virtual Reality; Vision and Signal Understanding. T I M E T A B L E Papers must be received by the Programme Chairperson no later than January 8, 1994. Acceptance letters will be posted no later than March 12, 1994. Final camera-ready papers must be received by April 19, 1994. P A N E L S Proposals for panel discussions (up to 1000 words) should be sent to the Programme Chairperson by Februar 8, 1994. E-mail is preferred. P R I Z E S As in previous years, a prize for the best paper as determined by the Programme Committee will be awarded; the Digital Equipment Prize and a prize for the best paper from Eastern Europe will also be awarded. Additionally, this year there will be two new prizes which will be awarded for application papers in the two categories described above under "Case Studies of AI Applications" and "Principles of AI Applications". V I D E O S U B M I S S I O N S In addition to the possibility of video enhanced papers described above, videos unaccompanied by papers may be submitted for presentation in special video track sessions. The purpose of these videos should be to demonstrate the current levels of usefulness of AI tools, techniques and methods. Videos presenting research arising out of interesting real world applications are especially sought. Authors should submit one copy of a videotape of 10 minutes maximum duration accompanied by a submission letter that includes: * Title * Full names, postal addresses, phone numbers and e-mail addresses of all authors * Duration of tape in minutes * Three copies of an abstract of one to two pages in length, containing the title of the video, and full names and addresses of the authors * Author's permission to copy tape for review purposes The timetable and conditions for submission, notification of acceptance or rejection, and receipt of final version are the same as for the paper track. All videotape submissions must be made to the Programme Chair. Tapes cannot be returned; authors should retain extra copies for making revisions. All videos must be in VHS-PAL format. An e-mail message giving the title, author, address and abstract should be e-mailed to ecai94-video at scs.leeds.ac.uk (unless the submitter has no e-mail access in which case this condition is waived). Tapes will be reviewed and selected for presentation during the conference. The following criteria will guide the selection: Level of interest to the conference audience Clarity of goals, methods and results Presentation quality (including audio, video and pace). Preference will be given to applications that show a high level of maturity. Tapes that are deemed to be advertising commercial products, propaganda, purely expository materials, merely taped lectures or other material not of scientific or technical value will be rejec- ted. P R O G R A M M E C O M M I T T E E C. Baeckstroem, Sweden J.P. Barthes, France I. Bratko, Slovenia P. Brazdil, Portugal J. Breuker, The Netherlands F. Bry, Germany R. Casati, Switzerland C. Castelfranchi, Italy J. Cuena, Spain Y. Davidor, Israel L. Farinas del Cerro, France F. Fogelman Soulie, France J. Fox, United Kingdom G. Friedrich, Austria A. Frisch, United Kingdom C. Froidevaux, France A. Fuhrmann, Germany A. Galton, United Kingdom J. Ganascia, France M. Ghallab, France J. Goncalves, Italy G. Gottlob, Austria F. Giunchiglia, Italy E. Hajicova, Czech Republic P. Hill, United Kingdom S. Hoelldobler, Germany D. Hogg, United Kingdom G. Kelleher, United Kingdom G. Kempen, The Netherlands M. King, Switzerland A. Kobsa, Germany M. Lenzerini, Italy R. Lopez de Mantaras, Spain N. Mars, The Netherlands J. Martins, Portugal P. Meseguer, Spain R. Milne, United Kingdom B. Nebel, Germany R. Nossum, Norway H.J. Ohlbach, Germany E. Oja, Finland E. Oliveira, Portugal E. Plaza, Spain J. Rosenschein, Israel Ph. Smets, Belgium L. Spanpinato, Italy O. Stock, Italy P. Struss, Germany P. Torasso, Italy R. Trappl, Austria L. Trave-Massuyes, France W. van de Velde, Belgium W. Wahlster, Germany T. Wittig, Germany W O R K S H O P S A full workshop programme is planned for ECAI '94. These will take place in the two days immediately before the main technical conference, i.e., on August 8 and 9, 1994. Workshops may last for either 1 or 2 days. They will give participants the opportunity to discuss specific technical topics in a small, informal environment, which encourages interaction and exchange of ideas. Workshops may address any topic covered by the list of areas given above [i.e., in the general call for papers]. Workshops on applications and related issues are especially welcome. Workshop proposals should be in the form of a draft call for participation containing a brief description of the workshop and the technical issues to be addressed, the proposed format and the kind of contributions solicited, and the names and addresses (postal, phone, fax, e-mail) of the organizing committee of the workshop. Additionally, proposals should specify the number of expected participants and some names of some potential participants. Proposers are encouraged to send their draft proposal to potential participants for comments before submission. The organizers of accepted workshops are responsible for producing a call for participation, for reviewing requests to participate and for scheduling the workshop activities within the constraints set by the conference organizers. Workshop proposals should be sent to the Workshop Chairpersons as soon as possible, but not later than November 1, 1993. Electronic submission (plain ascii text) is highly preferred, but hard copy submission is also accepted in which case 5 copies should be submitted. Proposals should not exceed 2 sides of A4 (i.e., approximately 120 lines of text). The proposals will be reviewed by the Programme Committee and the organizers will be notified not later than December 31, 1993. Details of all accepted workshops will be available by anonymous FTP from cs.vu.nl, directory ECAI94 by January 31, 1994; alternatively send electronic mail to ecai94-workshops at cs.vu.nl. It should be noted that registration for the main conference will be required in order to attend an ECAI '94 workshop. Workshop Chairpersons: Prof.dr Jan Treur Dr Frances Brazier Vrije Universiteit Amsterdam Department of Computer Science De Boelelaan 1081 a 1081 HV Amsterdam, The Netherlands Tel.: (+31)-20-548.55.88 Fax: (+31)-20 -642.77.05 E-mail: ecai94-workshops at cs.vu.nl T U T O R I A L S The ECAI '94 Organizing Committee invites proposals for the Tutorial Programme for ECAI '94. The tutorials will take place on August 8 and 9, 1994. Anyone who is interested in presenting a tutorial, or who has suggestions concerning possible speakers or topics is invited to contact the Tutorial Chair. A list of suggested topics that may be covered by tutorials is given below, but the list is only a guide. Other topics, both related to these and quite different from them, will be considered: Model-based reasoning; Natural language processing; Real-time reasoning; AI & databases (deductive databases, integrity constraints); Distributed AI, multi-agent systems; AI in industry (banking, networking, engineering); Knowledge sharing and reuse; Machine learning; Neutral Networks; Probabilistic reasoning and uncertainty; Genetic algorithms; Case-based reasoning; KBS design and methodology (including knowledge acquisition); Planning and scheduling; Hypermedia/multi-media in AI We are interested in: Proposals for the tutorials to be presented at the ECAI '94 Suggestions for topics (either an expression of interest in any of the topics above or in other topics) Suggestions for possible speakers for the tutorials (who would you like to hear if you attend a tutorial?) Anyone interested in presenting a tutorial should submit a proposal containing the following information: * A brief description and outline of the tutorial * The necessary background and the potential target audience * A description of why the tutorial topic is of interest to the ECAI '94 audience * A brief resume of the presenters Each tutorial will last for four hours, and must be offered by a team of presenters (usually two people, possibly three). Those submitting a proposal should keep in mind that tutorials are intented to provide an overview of a field or practical training in an area. They should present reasonably well agreed upon information in a balanced way. Tutorials should not be used to advocate a single avenue of research, nor should they promote a product. Presenters of a tutorial will receive a remuneration based on the number of participants in the tutorial. Proposals and suggestions must be received by September 1, 1993. Decisions about the tutorial programme will be made by September 30, 1993. Speakers should be prepared to submit completed course materials by May 6, 1994. Proposals, suggestions and enquiries should be sent (preferably electronically) to: Tutorial Chairperson: Dr Frank van Harmelen SWI University of Amsterdam Roetersstraat 15 1018 WB Amsterdam Tel.: (+31)-20-525.61.21, or (+31)-20-525.67.89 Fax: (+31)-20-525.68.96 E-mail: ecai94-tutorials at swi.psy.uva.nl Details of all tutorials will be available by September 30, 1993 by anonymous FTP from swi.psy.uva.nl, directory ECAI94. I N F O R M A T I O N For more information please contact: Organizing Chairperson: Workshop Chairpersons: Prof.dr Jaap van den Herik Prof.dr Jan Treur Dutch Association for Artificial Dr Frances Brazier Intelligence (NVKI) Vrije Universiteit Amsterdam University of Limburg Department of Computer Science Department of Computer Science De Boelelaan 1081 a P.O. Box 616 1081 HV Amsterdam 6200 MD Maastricht The Netherlands The Netherlands Tel.: (+31)-20-548.55.88 Tel.: (+31)-43-88.34.77 Fax: (+31)-20-642.77.05 Fax: (+31)-43-25.23.92 E-mail: ecai94-workshops at cs.vu.nl E-mail: bosch at cs.rulimburg.nl Programme Chairperson: Tutorial Chairperson: Dr Tony Cohn Dr Frank van Harmelen Division of Artificial SWI Intelligence University of Amsterdam School of Computer Studies Roetersstraat 15 University of Leeds 1018 WB Amsterdam Leeds LS2 9JT The Netherlands United Kingdom Tel.: (+31)-20-525.61.21, or Tel.:(+44)-532-33.54.82 (+31)-20-525.67.89 Fax: (+44)-532-33.54.68 Fax: (+31)-20-525.68.96 E-mail: ecai94 at scs.leeds.ac.uk E-mail: ecai94-tutorials at swi.psy.uva.nl CONFERENCE OFFICE: Erasmus Forum c/o ECAI '94 Marcel van Marrewijk, Project Manager Mirjam de Leeuw, Conference Manager E C A I '94 Erasmus University Rotterdam AMSTERDAM P.O. Box 1738 3000 DR Rotterdam The Netherlands Tel.: (+31)-10-408.23.02 Fax: (+31)-10-453.07.84 E-mail: M.M.deLeeuw at apv.oos.eur.nl ECCAI EUROPEAN COORDINATING COMMITTEE FOR ARTIFICIAL INTELLIGENCE From rwp at eng.cam.ac.uk Fri Jul 9 11:38:20 1993 From: rwp at eng.cam.ac.uk (Richard Prager) Date: Fri, 9 Jul 1993 11:38:20 BST Subject: Cambridge Neural Nets Summer School Message-ID: <6612.9307091038@dsl.eng.cam.ac.uk> The Cambridge University Programme for Industry in Collaboration with the Cambridge University Engineering Department Announce their Third Annual Neural Networks Summer School. 3 1/2 day short course 13-16 September 1993 BOURLARD GEE HINTON JERVIS JORDAN KOHONEN NARENDRA NIRANJAN PECE PRAGER SUTTON TARRASENKO Outline and aim of the course The course will give a broad introduction to the application and design of neural networks and deal with both the theory and with specific applications. Survey material will be given, together with recent research results in architecture and training methods, and applications including signal processing, control, speech, robotics and human vision. Design methodologies for a number of common neural network architectures will be covered, together with the theory behind neural network algorithms. Participants will learn the strengths and weaknesses of the neural network approach, and how to assess the potential of the technology in respect of their own requirements. Lectures are being given by international experts in the field, and delegates will have the opportunity of learning first hand the technical and practical details of recent work in neural networks from those who are contributing to those developments. Who Should Attend The course is intended for engineers, software specialists and other scientists who need to assess the current potential of neural networks. The course will be of interest to senior technical staff who require an overview of the subject, and to younger professionals who have recently moved into the field, as well as to those who already have expertise in this area and who need to keep abreast of recent developments. Some, although not all, of the lectures will involve graduate level mathematical theory. PROGRAMME Introduction and overview: Connectionist computing: an introduction and overview Programming a neural network Parallel distributed processing perspective Theory and parallels with conventional algorithms Architectures: Pattern processing and generalisation Bayesian methods in neural networks Reinforcement learning neural networks Communities of expert networks Self organising neural networks Feedback networks for optimization Applications: Classification of time series Learning forward and inverse dynamical models Control of nonlinear dynamical systems using neural networks Artificial and biological vision systems Silicon VLSI neural networks Applications to diagnostic systems Shape recognition in neural networks Applications to speech recognition Applications to mobile robotics Financial system modelling Applications in medical diagnostics LECTURERS DR HERVE BOURLARD is with Lernout & Hauspie Speech Products in Brussels. He has made many contributions to the subject particularly in the area of speech recognition. MR ANDREW GEE is with the Speech, Vision and Robotics Group of the Cambridge University Engineering Department. He specialises in the use of neural networks for solving complex optimization problems. PROFESSOR GEOFFREY HINTON is in the Computer Science Department at the University of Toronto. He was a founding member of the PDP research group and is responsible for many advances in the subject including the classic back-propagation paper. MR TIMOTHY JERVIS is with Cambridge University Engineering Department. His interests lie in the field of neural networks and in the application of Bayesian statistical techniques to learning control. PROFESSOR MICHAEL JORDAN is in the Department of Brain & Cognitive Science at MIT. He was a founding member of the PDP research group and he made many contributions to the subject particularly in forward and inverse systems. PROFESSOR TEUVO KOHONEN is with the Academy of Finland and Laboratory of Computer and Information Science at Helsinki University of Technology. His specialities are in self-organising maps and their applications. PROFESSOR K S NARENDRA is with the Center for Systems Science in the Electrical Engineering Department at Yale University. His interests are in the control of complex systems using neural networks. DR MAHESAN NIRANJAN is with the Department of Engineering at Cambridge University. His specialities are in speech processing and pattern classification. DR ARTHUR PECE is in the Physiological laboratory at the University of Cambridge. His interests are in biological vision and especially neural network models of cortical vision. DR RICHARD PRAGER is with the Department of Engineering at Cambridge University. His specialities are in speech and vision processing using artificial neural systems. DR RICH SUTTON is with the Adaptive Systems Department of GTE Laboratories near Boston, USA. His specialities are in reinforcement learning, planning and animal learning behaviour. DR LIONEL TARRASENKO is with the Department of Engineering at the University of Oxford. His specialities are in robotics and the hardware implementation of neural computing. COURSE FEES AND ACCOMMODATION The course fee is 750 (UK pounds), payable in advance, and includes full course notes, a certificate of attendance, and lunch and day-time refreshments for the duration of the course. A number of heavily discounted places are available for academics; please contact Renee Taylor if you would like to be considered for one of these places. Accommodation can be arranged for delegates in college rooms with shared facilities at Wolfson College at 163 (UK pounds) for 4 nights to include bed and breakfast, dinner with wine and a Course Dinner. For more information contact: Renee Taylor, Course Development Manager Cambridge Programme for Industry, 1 Trumpington Street, Cambridge CB2 1QA, United Kingdom tel: +44 (0)223 332722 fax +44 (0)223 301122 email: rt10005 at uk.ac.cam.phx From piero at dist.dist.unige.it Fri Jul 9 16:27:16 1993 From: piero at dist.dist.unige.it (Piero Morasso) Date: Fri, 9 Jul 93 16:27:16 MET DST Subject: ICANN'94 First Call for Papers Message-ID: <9307091427.AA08938@dist.dist.unige.it> -------------------------------------------------------------------- | ************************************************ | | * * | | * EUROPEAN NEURAL NETWORK SOCIETY * | | *----------------------------------------------* | | * C A L L F O R P A P E R S * | | *----------------------------------------------* | | * I C A N N ' 94 - SORRENTO * | | * * | | ************************************************ | | | | ICANN'94 (INTERNATIONAL CONFERENCE ON ARTIFICIAL NEURAL NETWORKS)| | is the fourth Annual Conference of ENNS and it comes after | | ICANN'91(Helsinki), ICANN'92 (Brighton), ICANN'93 (Amsterdam). | | It is co-sponsored by INNS, IEEE-NC, JNNS. | | It will take place at the Sorrento Congress Center, near Naples, | | Italy, on May 26-29, 1994. | | | |------------------------------------------------------------------| | S U B M I S S I O N | |------------------------------------------------------------------| | Interested authors are cordially invited to present their work | | in one of the following "Scientific Areas" (A-Cognitive Science; | | B-Mathematical Models; C- Neurobiology; D-Fuzzy Systems; | | E-Neurocomputing), indicating also an "Application domain" | | (1-Motor Control;2-Speech;3-Vision;4-Natural Language; | | 5-Process Control;6-Robotics;7-Signal Processing; | | 8-Pattern Recognition;9-Hybrid Systems;10-Implementation). | | | | DEADLINE for CAMERA-READY COPIES: December 15, 1993. | | ---------------------------------------------------- | | Papers received after that date will be returned unopened. | | Papers will be reviewed by senior researchers in the field | | and the authors will be informed of their decision by the end | | of January 1994. Accepted papers will be included in the | | Proceedings only if the authors have registered in advance. | | Allocation of accepted papers to oral or poster sessions will | | not be performed as a function of technical merit but only with | | the aim of coherently clustering different contributions in | | related topics; for this reason there will be no overlap of | | oral and poster sessions with the same denomination. Conference | | proceedings, that include all the accepted (and regularly | | registered) papers, will be distributed at the Conference desk | | to all regular registrants. | | | | SIZE: 4 pages, including figures, tables, and references. | | LANGUAGE: English. | | COPIES: submit a camera-ready original and 3 copies. | | (Accepted papers cannot be edited.) | | ADDRESS where to send the papers: | | IIASS (Intl. Inst. Adv. Sci. Studies), ICANN'94, | | Via Pellegrino 19, Vietri sul Mare (Salerno), 84019 Italy. | | ADDRESS where to send correspondence (not papers): | | Prof. Roberto Tagliaferri, Dept. Informatics, Univ. Salerno, | | Fax +39 89 822275, email iiass at salerno.infn.it | | EMAIL where to get LaTeX files: listserv at dist.unige.it | | | | In an accompanying letter, the following should be included: | | (i) title of the paper, (ii) corresponding author, | | (iii) presenting author, (iv) scientific area and application | | domain (e.g. "B-7"), (vi) preferred presentation (oral/poster), | | (vii) audio-visual requirements. | | | |------------------------------------------------------------------| | F O R M A T | |------------------------------------------------------------------| | The 4 pages of the manuscripts should be prepared on A4 white | | paper with a typewriter or letter- quality printer in | | one-column format, single-spaced, justified on both sides and | | printed on one side of the page only, without page numbers | | or headers/footers. Printing area: 120 mm x 195 mm. | | | | Authors are encouraged to use LaTeX. For LaTeX users, the LaTeX | | style-file and an example-file can be obtained via email as | | follows: | | - send an email message to the address "listserv at dist.unige.it" | | - the first two lines of the message must be: | | get ICANN94 icann94.sty | | get ICANN94 icann94-example.tex | | If problems arise, please contact the conference co-chair below. | | Non LaTeX users can ask for a specimen of the paper layout, | | to be sent via fax. | | | |------------------------------------------------------------------| | P R O G R A M C O M M I T T E E | |------------------------------------------------------------------| | The preliminary program committee is as follows: | | | | I. Aleksander (UK), D. Amit (ISR), L. B. Almeida (P), | | S.I. Amari (J), E. Bizzi (USA), E. Caianiello (I), | | L. Cotterill (DK), R. De Mori (CAN), R. Eckmiller (D), | | F. Fogelman Soulie (F), S. Gielen (NL), S. Grossberg (USA), | | J. Herault (F), M. Jordan (USA), M. Kawato (J), T. Kohonen (SF), | | V. Lopez Martinez (E), R.J. Marks II (USA), P. Morasso (I), | | E. Oja (SF), T. Poggio (USA), H. Ritter (D), H. Szu (USA), | | L. Stark (USA), J. G. Taylor (UK), S. Usui (J), L. Zadeh (USA) | | | | Conference Chair: Prof. Eduardo R. Caianiello, Univ. Salerno, | | Italy, Dept. Theoretic Physics; email: iiass at salerno.infn.it | | | | Conference Co-Chair: Prof. Pietro G. Morasso, Univ. Genova, | | Italy, Dept. Informatics, Systems, Telecommunication; | | email: morasso at dist.unige.it; fax: +39 10 3532948 | | | |------------------------------------------------------------------| | T U T O R I A L S | |------------------------------------------------------------------| | The preliminary list of tutorials is as follows: | | 1) Introduction to neural networks (D. Gorse), 2) Advanced | | techniques in supervised learning (F. Fogelman Soulie`), | | 3) Advanced techniques for self-organizing maps (T. Kohonen) | | 4) Weightless neural nets (I. Aleksander), 5) Applications of | | neural networks (R. Hecht-Nielsen), 6) Neurobiological modelling | | (J.G. Taylor), 7) Information theory and neural networks | | (M. Plumbley). | | Tutorial Chair: Prof. John G. Taylor, King's College, London, UK | | fax: +44 71 873 2017 | | | |------------------------------------------------------------------| | T E C H N I C A L E X H I B I T I O N | |------------------------------------------------------------------| | A technical exhibition will be organized for presenting the | | literature on neural networks and related fields, neural networks| | design and simulation tools, electronic and optical | | implementation of neural computers, and application | | demonstration systems. Potential exhibitors are kindly requested | | to contact the industrial liaison chair. | | | | Industrial Liaison Chair: Dr. Roberto Serra, Ferruzzi | | Finanziaria, Ravenna, fax: +39 544 35692/32358 | | | |------------------------------------------------------------------| | S O C I A L P R O G R A M | |------------------------------------------------------------------| | Social activities will include a welcome party, a banquet, and | | post-conference tours to some of the many possible targets of | | the area (participants will also have no difficulty to | | self-organize a la carte). | -------------------------------------------------------------------- From mcauley at cs.indiana.edu Tue Jul 13 13:11:20 1993 From: mcauley at cs.indiana.edu (J. Devin McAuley) Date: Tue, 13 Jul 1993 12:11:20 -0500 Subject: TR announcement: Analysis of the Effects of Noise on a Model for the Neural Mechanism of Short-Term Active Memory. Message-ID: <mailman.622.1149540263.24850.connectionists@cs.cmu.edu> FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/mcauley.noise.ps.Z The file mcauley.noise.ps.Z is now available for copying from the Neuroprose repository: Analysis of the Effects of Noise on a Model for the Neural Mechanism of Short-Term Active Memory. (8 pages) J. Devin McAuley and Joseph Stampfli Indiana University ABSTRACT: Zipser (1991) showed that the hidden unit activity of a fully-recurrent neural network model, trained on a simple memory task, matched the temporal activity patterns of memory-associated neurons in monkeys performing delayed saccade or delayed match-to-sample tasks. When noise, simulating random fluctuations in neural firing rate, is added to the unit activations of this model, the effect on the memory dynamics is to slow the rate of information loss. In this paper, we show that the dynamics of the iterated sigmoid function, with gain and bias parameters, is qualitatively very similar to the "output" behavior of Zipser's multi-unit model. Analysis of the simpler system provides an explanation for the effect of noise that is missing from the description of the multi-unit model. ---------------------------------- J. Devin McAuley Artificial Intelligence Laboratory Computer Science Department Indiana University Bloomington, Indiana 47405 ---------------------------------- From rohwerrj at cs.aston.ac.uk Tue Jul 13 12:49:52 1993 From: rohwerrj at cs.aston.ac.uk (rohwerrj) Date: Tue, 13 Jul 93 12:49:52 BST Subject: Research Opportunities in Neural Networks Message-ID: <23020.9307131149@cs.aston.ac.uk> ***************************************************************************** RESEARCH OPPORTUNITIES in NEURAL NETWORKS Dept. of Computer Science and Applied Mathematics Aston University ***************************************************************************** Funding has recently become available for up to 6 PhD studentships and up to 3 postdoctoral fellowships in the Neural Computing Research Group at Aston University. This group is currently undergoing a major expansion with the recent appointments of Professor Chris Bishop (formerly head of the Applied Neurocomputing Centre at AEA Technology, Harwell Laboratory) and Professor David Lowe (formerly head of the neural network research group at DRA, Malvern), joining Professor David Bounds and lecturers Richard Rohwer and Alan Harget. In addition, substantial funds are being invested in new computer hardware and software and other resources, which will provide the Group with extensive research facilities. The research programme of the Group is focussed on the development of neural computing techniques from a sound statistical pattern processing perspective. Research topics span the complete range from developments of the theoretical foundations of neural computing, through to a wide range of application areas. The Group maintains close links with several industrial organisations, and is participating in a number of collaborative projects. For further information, please contact me at the address below: Richard Rohwer Dept. of Computer Science and Applied Mathematics Aston University Aston Triangle Birmingham B4 7ET ENGLAND Tel: (44 or 0) (21) 359-3611 x4688 FAX: (44 or 0) (21) 333-6215 rohwerrj at uk.ac.aston.cs From goodman at unr.edu Mon Jul 12 18:59:14 1993 From: goodman at unr.edu (Phil Goodman) Date: Mon, 12 Jul 93 22:59:14 GMT Subject: POSITION AVAILABLE - STATISTICIAN Message-ID: <9307130559.AA28993@equinox.unr.edu> ******************* Professional Position Announcement ****************** "STATISTICIAN for NEURAL NETWORK & REGRESSION DATABASE RESEARCH" .- - - - - - - - - - - - - - OVERVIEW - - - - - - - - - - - - - - - - -. | | | THE LOCATION: | | Nevada's Reno/Lake Tahoe region is an outstanding environment for | | living, working, and raising a family. Winter skiing is world-class,| | summer recreation includes many mountain and water sports, and | | historical exploration and cultural opportunities abound. | | | | THE PROJECT: | | The new CENTER FOR BIOMEDICAL MODELING RESEARCH recently received | | federal funding to refine and apply a variety of artificial neural | | network algorithms to large cardiovascular health care databases. | | | | THE CHALLENGE: | | The predictive performance of neural nets will be compared to | | advanced regression models. Other comparisons to be made include | | handling of missing and noisy data, and selection of important | | interactions among variables. | | | | THE JOB REQUIREMENT: | | Masters-level or equivalent statistician with working knowledge | | of the SAS statistical package and the UNIX operating system. | | | | THE SALARY : | | Approximate starting annual salary: $42,000 + full benefits . | | (actual salary will depend on experience and qualifications) | ._ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ . POSITION: Research Statistics Coordinator for NEURAL NETWORKS / HEALTH CARE DATABASE PROJECT LOCATION: Center for Biomedical Modeling Research Department of Internal Medicine University of Nevada School of Medicine Washoe Medical Center, Reno, Nevada START DATE: September 1, 1993 CLOSING DATE: Open until filled. DESCRIPTION: Duties include acquisition and translation of data from multiple external national sources; data management and archiving; performance of exploratory and advanced regression statistics; performance of artificial neural network processing; participation in scholarly research and publications. QUALIFICATIONS: (1) M.S., M.A., M.P.H. or equivalent training in statistics with experience in logistic and Cox regression analyses, (2) ability to program in the SAS statistical language, and (3) experience with UNIX computer operating systems. Desirable but not mandatory are the abilities to use (4) the S-PLUS data management system and (5) the C programming language. SALARY: Commensurate with qualifications and experience. (For example, with database experience, typical annual salary would be approximately $42,000 + full benefits.) APPLICATION: > Informal inquiry may be made to: Phil Goodman, Director, Center for Biomedical Modeling Research Internet: goodman at unr.edu Phone: 702-328-4867 > Formal consideration requires a letter of application, vita, and names of three references sent to: Philip Goodman, MD, MS Director, Center for Biomedical Modeling Research University of Nevada School of Medicine Washoe Medical Center, Room H1-166 77 Pringle Way, Reno, NV 89520 The University of Nevada is an Equal Opportunity/Affirmative Action employer and does not discriminate on the basis of race, color, religion, sex, age, national origin, veteran's status or handicap in any program it operates. University of Nevada employs only U.S. citizens and aliens lawfully authorized to work in the United States. ************************************************************************ From smieja at nathan.gmd.de Wed Jul 14 14:07:06 1993 From: smieja at nathan.gmd.de (Frank Smieja) Date: Wed, 14 Jul 1993 20:07:06 +0200 Subject: TR announcement: reflective agent teams Message-ID: <199307141807.AA19358@trillian.gmd.de> The file beyer.teams.ps.Z is available for copying from the Neuroprose repository: Learning from Examples, Agent Teams and the Concept of Reflection (22 pages) Uwe Beyer and Frank Smieja GMD, Germany ABSTRACT. Learning from examples has a number of distinct algebraic forms, depending on what is to be learned from which available information. One of these forms is $x \stackrel{G}{\rightarrow} y$, where the input--output tuple $(x,y)$ is the available information, and $G$ represents the process determining the mapping from $x$ to $y$. Various models, $y = f(x)$, of $G$ can be constructed using the information from the $(x,y)$ tuples. In general, and for real-world problems, it is not reasonable to expect the exact representation of $G$ to be found (i.e.\ a formula that is correct for all possible $(x,y)$). The modeling procedure involves finding a satisfactory set of basis functions, their combination, a coding for $(x,y)$ and then to adjust all free parameters in an approximation process, to construct a final model. The approximation process can bring the accuracy of the model to a certain level, after which it becomes increasingly expensive to improve further. Further improvement may be gained through constructing a number of agents $\{\alpha\}$, each of which develops its own model $f_\alpha$. These may then be combined in a second modeling phase to synthesize a {\it team\/} model. If each agent has the ability of internal {\it reflection\/} the combination in a team framework becomes more profitable. We describe reflection and the generation of a {\it confidence\/} function: the agent's estimate of the correctness of each of its predictions. The presence of reflective information is shown to increase significantly the performance of a team. -Frank Smieja Gesellschaft fuer Mathematik und Datenverarbeitung (GMD) GMD-FIT.KI.AS, Schloss Birlinghoven, 5205 St Augustin 1, Germany. Tel: +49 2241-142214 email: smieja at gmd.de From luttrell at signal.dra.hmg.gb Thu Jul 15 06:15:17 1993 From: luttrell at signal.dra.hmg.gb (luttrell@signal.dra.hmg.gb) Date: Thu, 15 Jul 93 11:15:17 +0100 Subject: TR announcement: Bayesian Self-Organisation Message-ID: AA08529@mentor.dra.hmg.gb FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/luttrell.bayes-selforg.ps.Z The file luttrell.bayes-selforg.ps.Z is now available for copying from the Neuroprose repository: A Bayesian Analysis of Self-Organising Maps (24 pages) Stephen P Luttrell Defence Research Agency, United Kingdom ABSTRACT: In this paper Bayesian methods are used to analyse some of the properties of a special type of Markov chain. The forward transitions through the chain are followed by inverse transitions (using Bayes' theorem) backwards through a copy of the same chain; this is called a folded Markov chain. If an appropriately defined Euclidean error (between the original input and its "reconstruction" via Bayes' theorem) is minimised in the space of Markov chain transition probabilities, then the familiar theories of both vector quantisers and self-organising maps emerge. This approach is also used to derive the theory of self-supervision, in which the higher layers of a multi-layer network supervise the lower layers, even though overall there is no external teacher. Steve Luttrell Adaptive Systems Theory Section DRA, St Andrews Road, Malvern, Worcestershire, WR14 3PS, UK email: luttrell at signal.dra.hmg.gb Tel: +44-684-894046 Fax: +44-684-894384 From jrf at psy.ox.ac.uk Mon Jul 19 10:20:06 1993 From: jrf at psy.ox.ac.uk (John Fletcher) Date: Mon, 19 Jul 1993 15:20:06 +0100 Subject: postdoctoral position Message-ID: <9307191420.AA13411@axpbb.cns.ox.ac.uk> UNIVERSITY OF OXFORD MRC RESEARCH CENTRE IN BRAIN AND BEHAVIOUR Scientist for Neural Network Applications Applications are invited for a postdoctoral position to work on the operation of neuronal networks in the brain, with special reference to cortical computation. The post available is for a theoretician to perform analytic and/or simulation work colla- boratively with experimental neuroscientists on biologically realistic models of computation in the brain, including systems involved in vision, memory and/or motor function. The appoint- ment will be for three years in the first instance. The salary is on the RSIA scale, GBP 12,638-20,140, and is funded by the Medical Research Council. Further particulars are attached. Applications, including the names of two referees, to The Administrator, Department of Exper- imental Psychology, South Parks Road, Oxford OX1 3UD. The University is an Equal Opportunities Employer. ---- UNIVERSITY OF OXFORD MRC RESEARCH CENTRE IN BRAIN AND BEHAVIOUR Neural Network Scientist (RSIA Scale) The purpose of this post is to enable a formal modeller of neu- ronal networks to work on a daily basis with neuroscientists in order to develop models of cortical function, and to provide the neuroscientists in the MRC Research Centre in Brain and Behaviour with advice and assistance in relation to the expertise required for analytic models of neuronal networks. The postholder should have a PhD or equivalent experience, to- gether with evidence that he/she can initiate a research pro- gramme. The postholder is expected to have expertise in the mathematical analyses of neuronal networks. He/she will provide mathematical and related computational expertise required for the modelling of real neuronal networks, the architecture of which will be based on anatomical information about neuronal connec- tions and physiological information about their modifiability. He/she will probably have the opportunity to supervise graduate students reading for the DPhil degree. In addition, the postholder should also have expertise useful in enabling him or her to analyse biologically relevant models of neural networks. This latter requirement implies either existing expertise in neuroscience, or an aptitude for working with neu- roscientists to ensure that biological constraints are correctly incorporated into network models. The postholder will be sufficiently senior to be able to initiate independent analyses of neural networks, and to ensure that results are brought to publication. Since an important part of the Research Centre is training, the postholder will be expected to take part in the organization of seminar series on neural networks. It is anticipated that these seminars will be held at least annually and will provide training for both doctoral students and for other more senior members of the University interested in research on neural networks. Other teaching may include participation in a summer school. Finally, the postholder, who will be responsible to the Directors of the Research Centre, will be expected to serve on a Working Group of the Research Centre interested in computational analyses of neuronal networks. Examples of neural network projects in the Research Centre in- clude the following: Dr J F Stein/Dr R C Miall, University Laboratory of Physiology: (a)To reconsider the Zipser & Anderson network model of head- centred encoding of target positions by visual cells in the pos- terior parietal cortex. Their model was very successful at demonstrating how the hidden units within a standard back- propagation neural network could end up with receptive field pro- perties similar to those recorded in the PPC in awake monkeys. However, as the responses (in both monkey and model) were complex and hard to classify, it remains unclear whether there was a true similarity between the two or whether the solutions were merely equally hard to interpret. We plan to re-examine this model us- ing more realistic constraints on the model connectivity and on the encoding of its inputs, and should be taking on a new post- graduate student this October to pursue these questions. (b)To develop models of learning at the cerebellar parallel-fibre to Purkinje cell synapse. We have proposed a model of the cere- bellum in which the cortex learns to form a forward model of the limb dynamics to assist in visually guided arm movement. We have proposed that there is a mechanism related to reinforcement learning, but the details remain unclear. It is important to see whether the biophysics of long-term depression and the ideas of reinforcement learning can be tied together in a neural simula- tion. References: Barto AG, Sutton RS & Anderson CW (1983). Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Sys Man Cyb., 13: 834-846. Ito M (1989). Long-term depression. Ann. Rev. Neurosci. 12:85- 102. Miall RC, Weir DJ, Wolpert DM & Stein JF (1993). Is the cerebel- lum a Smith Predictor? J.motor Behaviour (in press). Zipser D & Andersen RA (1988). A back-propagation programmed network that simulates response properties of a subset of poste- rior parietal neurons. Nature 331:679-684. Dr K Plunkett, Department of Experimental Psychology: Kim Plunkett's research group is involved in computational and experimental investigations of language acquisition and cognitive development. Modelling work has covered connectionist simula- tions of inflectional morphology, concept formation, vocabulary development and early syntax. Experimental work focuses on the processes of lexical segmentation in early language acquisition. References: Plunkett K & Marcham V (1991). U-shaped learning and frequency effects in a multi-layered perceptron: Implications for child language acquisition. Cognition, 38, 43-102. Plunkett K & Sinha CG (1992). Connectionism and developmental theory. British Journal of Developmental Psychology, 10, 209- 254. Plunkett K (1993). Lexical segmentation and vocabulary growth in early language acquisition. Journal of Child Language, 20, 43-60. Professor D Sherrington, Department of Theoretical Physics: Our interest is in understanding and quantifying the design, per- formance and training of neural network models, stimulated by their potential as cartoons of parts of the brain, as expert sys- tems and as complex cooperative systems. Our methodology in- volves the application of analytical and computational techniques from the theoretical physics of strongly interacting systems and centres largely around issues of statistical relevance, as op- posed to worst-case or special-case analyses. References: Recent reviews: *Sherrington D. Neural Networks: the spinglass approach. OUTP- 92-485. *Coolen T & Sherrington D. Dynamics of attractor neural net- works. OUTP-92-495. [The above are to be published in Mathematical Studies of Neural Networks (Elsevier); ed. JG Taylor.] Watkin TLH, Rau A & Biehl M. Statistical Mechanics of Learning a Rule. OUTP-92-45S published in Reviews of Modern Physics 65, pp 499-556 (1993). *Copies of these are available for interested candidates. Published research articles (selection) Wong KYM, Kahn PE & Sherrington D. A neural network model of working memory exhibiting primacy and recency. J. Phys. A24, 1119 (1991). Sherrington D, Wong M & Rau A. Good Memories. Phil Mag. 65, 1303 (1992). Sherrington D, Wong KYM & Coolen ACC. Noise and competition in neural networks. J.Phys I France 3, 331 (1993). O'Kane D & Sherrington D. A feature-retrieving attractor neural network. J.Phys A 26, 2333 (1993). Dr E T Rolls, Department of Experimental Psychology: (a) Learning invariant responses to the natural transformations of objects. The primate visual system builds representations of objects which are invariant with respect to transforms such as translation, size, and eventually view, in a series of hierarchi- cal cortical areas. To clarify how such a system might learn to recognise "naturally" transformed objects, we are investigating a model of cortical visual processing which incorporates a number of features of the primate visual system. The model has a series of layers with convergence from a limited region of the preceding layer, and mutual inhibition over a short range within a layer. The feedforward connections between layers provide the inputs to competitive networks, each utilising a modified Hebb-like learn- ing rule which incorporates a temporal trace of the preceding neuronal activity. The trace learning rule is aimed at enabling the neurons to learn transform invariant responses via experience of the real world, with its inherent spatio-temporal constraints. We are showing that the model can learn to produce translation invariant responses, and plan to develop this neural network model to investigate its performance in learning other types of invariant representation, and its capacity. Rolls E (1992). Neurophysiological mechanisms underlying face processing within and beyond the temporal cortical areas. Phil. Trans. Roy. Society London Ser B 335, 11-21. Wallis G, Rolls E & Foldiak P (1993). In: Proc Int Joint Conference on Neural Networks. (b) Neural Networks in the Hippocampus involved in Memory and Recall. We are developing a model based on hippocampal anatomy, physiology, and psychology, of how the neural networks in the hippocampus could operate in memory. A key hypothesis is that the hippocampal CA3 circuitry forms an autoassociation memory. We are developing a quantitative theory of how the CA3 could operate, and of how it would function in relation to other parts of the hippocampus. We are also starting to develop a theory of how the hippocampus could recall memories in the cerebral neocor- tex using backprojections in the cerebral cortex. Rolls ET (1989a). Functions of neuronal networks in the hippocampus and neocortex in memory. In Neural models of plasti- city: Experimental and theoretical approaches (ed. JH Byrne & WO Berry), 13, 240-265). San Diego: Academic Press. Rolls ET (1990a). Theoretical and neurophysiological analysis of the functions of the primate hippocampus in memory. Cold Spring Harbor Symposia in Quantitative Biology 55, 995-1006. Treves A & Rolls ET (1991). What determines the capacity of autoassociative memories in the brain? Network 2: 371-397. Treves A & Rolls ET (1992). Computational constraints sug- gest the need for two distinct input systems to the hippocampal CA3 network. Hippocampus 2: 189-199. Rolls ET & Treves A (1993). Neural Networks in the Brain involved in Memory and Recall. In: Proc. Int. Joint Conference on Neural Networks. We would welcome collaboration on either or both projects. From mel at cns.caltech.edu Mon Jul 19 13:08:41 1993 From: mel at cns.caltech.edu (Bartlett Mel) Date: Mon, 19 Jul 93 10:08:41 PDT Subject: Neural Simulation Demonstration Message-ID: <9307191708.AA00449@plato.cns.caltech.edu> ANNOUNCEMENT FOR NEURAL SIMULATION DEMONSTRATIONS AT THE SOCIETY FOR NEUROSCIENCE ANNUAL MEETING 1993 Sponsored by the National Science Foundation This announcement contains information on an opportunity to demonstrate state of the art computer simulation techniques in computational neuroscience at the annual Society For neuroscience meeting to be held in Washington DC during the second week of November. Please distribute this announcement to other groups which might be interested in demonstrating their software. Purpose The primary purpose of these demonstrations will be to expose large numbers of neuroscientists to the use of computer simulation techniques in neurobiology. In addition, we want to provide technical information on simulation methods, the range of simulation software currently available, and demonstrations of several aspects of new computer technology that are relevant to efforts to simulate and understand the brain. Format The Neural Simulation Technology Demonstration will consist of three separate components. First, a series of simulation software demonstrations while be organized to provide a balanced and as complete as possible view of available non commercial software. Second, machines will be made available to participants in the meeting whose regular poster or oral presentations involve simulations so that they can demonstrate the simulations on which their paper presentations are based. Finally, a separate demonstration will focus on state of the art simulation technology including inter-computer communication, concurrent computing, and graphical visualization. Simulation software demonstrations This announcement seeks applicants for the simulation software demonstrations. This demonstration will be organized very much like a regular poster session, except that computers will be available for actual demonstrations of the modeling software and each demonstration will remain in place throughout the meeting. The session itself will be set up within the exhibit space of the meeting. Each presenter will be required to construct a poster as well as demonstrate their software on a computer. The poster presentations are intended to provide those interested in the simulators an understanding of what they will see when they actually look at the software on the adjacent computer. As such, the poster should include descriptions of the simulator itself, what it is best used for, and how it is used. We will provide an overview of the demonstrated software packages as a handout. We would also like to provide some basis of comparison between the simulators. For this reason we are asking that groups demonstrating unix software for compartmental modeling present the quantitative results obtained from the first three standard simulations in the Rallpack standards (Bhalla et al., Trends in Neurosci. 15: 453-458, 1992), which can be obtained by anonymous ftp from mordor.cns.caltech.edu What to do If you are interested in being considered as a simulations software demonstrator, you should fill out the application attached to the end of this document and return it to erik at cns.caltech.edu. You should note that the National Science Foundation has stipulated that applications cannot be accepted from vendors of commercial software, but only from "not for profit" concerns. Applications will be reviewed as they are received, deadline is October 1, 1993. Invitations will then be send out immediately based on this review. Criteria for acceptance of demonstrations will include the apparent maturity of the effort as well as evidence that the simulation software is being or could be used outside of the laboratory in which it was developed. Some packages are well known and established (De Schutter, Trends in Neurosci. 15: 462-464, 1992) and will be accepted with minimal review (i.e. submit application by e-mail). Applications for newer packages should send additional documentation (like manuals, information sheets, papers,...) by mail to help in the evaluation. Arrangements In order to increase the likely success of the demonstrations, each accepted participant will be asked toship their own computer to the meeting. This will avoid difficulties with hardware incompatibilities, or novel system configurations. Our technical staff will aid with setting up the machines at each meeting. Support Accepted participants will be provided support on a reimbursement basis in the following forms: 1) computer shipping expenses 2) reimbursement of meeting registration costs for one person designated as responsible for the software demonstration. For further information contact: Jim Bower or Erik De Schutter Div. of Biology 216-76 Caltech Pasadena, CA 91125 jbower at cns.caltech.edu, erik at cns.caltech.edu FAX 1-818-7952088 -------------------------------------------------------------------------- APPLICATION FORM This form can be e-mailed to erik at cns.caltech.edu, or mailed to Erik De Schutter, Div. of Biology 216-76, Caltech, Pasadena, CA 91125 APPLICANT Name: Title: Address: E-mail: PERSON DEMONSTRATING THE SOFTWARE (if different from applicant) Name: Title: Address: E-mail: COMPUTER USED FOR THE DEMONSTRATION: Type: Operating system: SOFTWARE PACKAGE Purpose (80 characters or less): Author: Operating system(s): Hardware platforms on which code has been tested: Parallel implementation (if yes, which platform)? Programming language source code: Source code available? Current version number: When was this version released? Continued development/upgrades of the package? How many programmers involved in continued development? User expandable (yes only if explicitly supported by the code): Interface type (answer yes or no): all features in a single program? interactive model creation/editing? compartments/channels described by arbitrary codes/indexes? compartments/channels described by user selected names? script language? Mac type interface (dialogs, buttons, popup menus)? graphic interface (point to dendrite to select etc.)? object-oriented approach? supports grouping of objects for easy selection? File I/O for model description: reads morphology files? saves interactively created models? reads file formats other software packages (list names): outputs file formats other software packages (list names): I/O of results (answer yes or no): file output? graph versus time during simulation? phase plots? Sholl plots? color representation of cell ('schematic')? color representation of cell ('3-dim')? output multiple variables during same run? output any computed variable? incorporates analysis routines for results? Simulator features: integration method(s): are Rallpack data available? variable time step? automatic detection of model setup errors? storage of initial values for variables? model size limited only by available memory? Neuron modeling features (answer yes or no): integrate-and-fire model? compartmental model? detailed morphology of dendritic/axonal trees? standard Hodgkin-Huxley channels? other channel types using Hodgkin-Huxley type equations? other channel types using any equation? Ca2+-dependent (in)activation of channels? synaptic channels (alpha equation)? synaptic plasticity? simple concentration pools (Ca2+, exponential decay)? complex concentration pools (buffers, diffusion)? second messengers, enzymatic kinetics? Network modeling features (answer yes or no): small network (<= 100 cells)? big networks (> 100 cells) mixed cell model types possible? automatic/random generation of connections? network learning (e.g. backprop)? synaptic/axonal delays implemented? network growth during simulations? Manual: is the manual complete (i.e. describes all program features, commands)? number of pages in manual: does the manual contain screen dumps? reference manual only or also tutorials? Other documentation: On line help? Tutorials? Classes? Papers? Distribution: Compiled program on disk/tape? Compiled program by ftp? As source code by ftp? Fee (how much)? Free upgrades? Upgrades by ftp? Available to foreign users? User support: E-mail support? Telephone support? Newsletter? Users group? USERS How long has this package been available? How many upgrades have been released? How many scientists use this package in your group? How many other groups in the USA use this package? How many foreign groups use this package? Optional and confidential: list 3 users (including e-mail address or FAX) who currently use the software package and who have never worked in your laboratory. REFERENCES: List max 2 publications describing this software package. List max 5 publications describing simulation results obtained with this software package. CONTACT ADDRESS FOR PROSPECTIVE USERS: Preferred method of contact: Address: E-mail: FAX: From mel at cns.caltech.edu Mon Jul 19 15:44:51 1993 From: mel at cns.caltech.edu (Bartlett Mel) Date: Mon, 19 Jul 93 12:44:51 PDT Subject: PostScript NIPS*93 Brochure Message-ID: <9307191944.AA00683@plato.cns.caltech.edu> ************ PLEASE POST **************** An electronic copy of the 1993 NIPS registration brochure is available in PostScript format via anonymous ftp. Instructions for retrieving and printing the brochure are as follows: unix> ftp helper.systems.caltech.edu (or ftp 131.215.68.12) Name: anonymous Password: your-login-name ftp> cd /pub/nips ftp> binary ftp> get NIPS_93_brochure.ps.Z ftp> quit unix> uncompress NIPS_93_brochure.ps.Z unix> lpr NIPS_93_brochure.ps If you would like a hardcopy of the brochure or need other information, please send a request to nips93 at systems.caltech.edu or the following address: NIPS Foundation P.O. Box 60035 Pasadena, CA 91116-6035 From mwitten at hermes.chpc.utexas.edu Mon Jul 19 19:17:09 1993 From: mwitten at hermes.chpc.utexas.edu (mwitten@hermes.chpc.utexas.edu) Date: Mon, 19 Jul 93 18:17:09 CDT Subject: COMPUTATIONAL HEALTH (long) Message-ID: <9307192317.AA11053@morpheus.chpc.utexas.edu> Preliminary Announcement FIRST WORLD CONGRESS ON COMPUTATIONAL MEDICINE, PUBLIC HEALTH AND BIOTECHNOLOGY 24-28 April 1994 Hyatt Regency Hotel Austin, Texas ----- (Feel Free To Cross Post This Announcement) ---- 1.0 CONFERENCE OVERVIEW: With increasing frequency, computational sciences are being exploited as a means with which to investigate biomedical processes at all levels of complexity; from molecular to systemic to demographic. Computational instruments are now used, not only as exploratory tools but also as diagnostic and prognostic tools. The appearance of high performance computing environments has, to a great extent, removed the problem of increasing the biological reality of themathematical models. For the first time in the historyof the field, practical biological reality is finally within the grasp of the biomedical modeler. Mathematical complexity is no longer as serious an issue as speeds of computation are now of the order necessary to allow extremely large and complex computational models to be analyzed. Large memory machines are now routinely available. Additionally, high speed, efficient, highly optimized numerical algorithms are under constant development. As these algorithms are understood and improved upon, many of them are transferred from software implementation to an implementation in the hardware itself; thereby further enhancing the available computational speed of current hardware. The purpose of this congress is to bring together a transdisciplinary group of researchers in medicine, public health, computer science, mathematics, nursing, veterinary medicine, ecology, allied health, as well as numerous otherdisciplines, for the purposes of examining the grand challenge problems of the next decades. This will be a definitive meeting in that it will be the first World Congress of its type and will be held as a followup tothe very well received Workshop On High Performance Computing In The Life Sciences and Medicine held by the University of Texas System Center For High Performance Computing in 1990. Young scientists are encouraged to attend and to present their work in this increasingly interesting discipline. Funding is being solicited from NSF, NIH, DOE, Darpa, EPA, and private foundations, as well as other sources to assist in travel support and in the offsetting of expenses for those unable to attend otherwise. Papers, poster presentations, tutorials, focussed topic workshops, birds of a feather groups, demonstrations, and other suggestions are also solicited. 2.0 CONFERENCE SCOPE AND TOPIC AREAS: The Congress hasa broad scope. If you are not sure as to whether or not your subject fits the Congress scope, contact the conference organizers at one of the addresses below. Subject areas include but are not limited to: *Visualization/Sonification --- medical imaging --- molecular visualization as a clinical research tool --- simulation visualization --- microscopy --- visualization as applied to problems arising in computational molecular biology and genetics or other non-traditional disciplines *Computational Molecular Biology and Genetics --- computational ramifications of clinical needs in the Human Genome, Plant Genome, and Animal Genome Projects --- computational and grand challenge problems in molecular biology and genetics --- algorithms and methodologies --- issues of multiple datatype databases *Computational Pharmacology, Pharmacodynamics, Drug Design *Computational Chemistry as Applied to Clinical Issues *Computational Cell Biology, Physiology, and Metabolism --- Single cell metabolic models (red blood cell) --- Cancer models --- Transport models --- Single cell interaction with external factors models (laser, ultrasound, electrical stimulus) *Computational Physiology and Metabolism --- Renal System --- Cardiovascular dynamics --- Liver function --- Pulmonary dynamics --- Auditory function, coclear dynamics, hearing --- Reproductive modeling: ovarian dynamics, reproductive ecotoxicology, modeling the hormonal cycle --- Metabolic Databases and metabolic models *Computational Demography, Epidemiology, and Statistics/Biostatistics --- Classical demographic, epidemiologic, and biostatistical modeling --- Modeling of the role of culture, poverty, and other sociological issues as they impact healthcare *Computational Disease Modeling --- AIDS --- TB --- Influenza --- Statistical Population Genetics Of Disease Processes --- Other *Computational Biofluids --- Blood flow --- Sperm dynamics --- Modeling of arteriosclerosis and related processes *Computational Dentistry, Orthodontics, and Prosthetics *Computational Veterinary Medicine --- Computational issues in modeling non-human dynamics such as equine, feline, canine dynamics (physiological/biomechanical) *Computational Allied Health Sciences --- Physical Therapy --- Neuromusic Therapy --- Resiratory Therapy *Computational Radiology --- Dose modeling --- Treatment planning *Computational Surgery --- Simulation of surgical procedures in VR worlds --- Surgical simulation as a precursor to surgical intervention *Computational Cardiology *Computational Nursing *Computational Models In Chiropractice *Computational Neurobiology and Neurophysiology --- Brain modeling --- Single neuron models --- Neural nets and clinical applications --- Neurophysiological dynamics --- Neurotransmitter modeling --- Neurological disorder modeling (Alzheimers Disease, for example) *Computational Models of Psychiatric and Psychological Processes *Computational Biomechanics --- Bone Modeling --- Joint Modeling *Computational Models of Non-tradional Medicine --- Acupuncture --- Other *Computational Issues In Medical Instrumentation Design and Simulation --- Scanner Design --- Optical Instrumentation *Ethical issues arising in the use of computational technology in medical diagnosis and simulation *The role of alternate reality methodologies and high performance environments in the medical and public health disciplines *Issues in the use of high performance computing environments in the teaching of health science curricula *The role of high performance environments for the handling of large medical datasets (high performance storage environments, high performance networking, high performance medical records manipulation and management, metadata structures and definitions) *Federal and private support for transdisciplinary research in computational medicine and public health 3.0 CONFERENCE COMMITTEE *CONFERENCE CHAIR: Matthew Witten, UT System Center For High Performance Computing, Austin, Texas m.witten at chpc.utexas.edu *CONFERENCE DIRECTORATE: Regina Monaco, Mt. Sinai Medical Center * Dan Davison, University of Houston * Chris Johnson, University of Utah * Lisa Fauci, Tulane University * Daniel Zelterman, University of Minnesota Minneapolis * James Hyman, Los Alamos National Laboratory * Richard Hart, Tulane University * Dennis Duke, SCRI-Florida State University * Sharon Meintz, University of Nevada Los Vegas * Dean Sittig, Vanderbilt University * Dick Tsur, UT System CHPC * Dan Deerfield, Pittsburgh Supercomputing Center * Istvan Gyori, Szeged University School of Medicine Computing Center * Don Fussell, University of Texas at Austin * Ken Goodman, University Of Miami School of Medicine * Martin Hugh-Jones, Louisiana State University * Stuart Zimmerman, MD Anderson Cancer Research Center * John Wooley, DOE * Sylvia Spengler, University of California Berkeley, Robert Blystone, Trinity University Additional conference directorate members are being added and will be updated on the anonymous ftp list as they agree. 4.0 CONTACTING THE CONFERENCE COMMITTEE: To contact the congress organizers for any reason use any of the following pathways: ELECTRONIC MAIL - compmed94 at chpc.utexas.edu FAX (USA) - (512) 471-2445 PHONE (USA) - (512) 471-2472 GOPHER:log into the University of Texas System-CHPC select the Computational Medicine and Allied Health menu choice ANONYMOUS FTP: ftp.chpc.utexas.edu cd /pub/compmed94 POSTAL: Compmed 1994 University of Texas System CHPC Balcones Research Center, 1.154CMS 10100 Burnet Road Austin, Texas 78758-4497 5.0 SUBMISSION PROCEDURES: Authors must submit 5 copies of a single-page 50-100 word abstract clearly discussing the topic of their presentation. In addition, authors must clearly state their choice of poster, contributed paper, tutorial, exhibit, focussed workshop or birds of a feather group along with a discussion of their presentation. Abstracts will be published as part of the preliminary conference material. To notify the congress organizing committee that you would like to participate and to be put on the congress mailing list, please fill out and return the form that follows this announcement. You may use any of the contact methods above. If you wish to organize a contributed paper session, tutorial session,focussed workshop, or birds of a feather group, please contact the conference director at mwitten at chpc.utexas.edu *CONFERENCE DEADLINES: The following deadlines should be noted: 1 October 1993 - Notification of interest in participation and/or intent to organize a special session 1 November 1993 - Abstracts for talks/posters/ workshops/birds of a feather sessions/demonstrations 15 January 1994 - Notification of acceptance of abstract 15 February 1994 - Application for financial aid 6.0 CONFERENCE PRELIMINARY DETAILS AND ENVIRONMENT LOCATION: Hyatt Regency Hotel, Austin, Texas, USA DATES: 24-28 April 1994 The 1st World Congress On Computational Medicine, Public Health, and Biotechnology will be held at the Hyatt Regency Hotel, Austin, Texas located in downtown Austin. The hotel is approximately 15 minutes from Robert Meuller Airport. Austin, the state capital, is renouned for its natural hill-country beauty and an active cultural scence. Several hiking and jogging trails are within walking distance of the hotel, as well as opportunities for a variety of aquatic sports. Live bands perform in various nightclubs around the city and at night spots along Sixth Street, offering a range of jazz, blues, country/Western, reggae, swing, and rock music. Day temperatures will be in the 80-90(degree F) range and fairly humid. Exhibitor and vendor presentations are also being planned. 7.0 CONFERENCE ENDORSEMENTS AND SPONSORSHIPS: Numerous potential academic sponsors have been contacted. Currently negotiations are underway for sponsorship with SIAM, AMS, MAA, IEEE, FASEB, and IMACS. Additionally AMA and ANA continuing medical education support is beging sought. Information will be updated regularly on the anonymous ftp site for the conference (see above). ================== INTENT TO PARTICIPATE ============= First Name: Middle Initial (if available): Family Name: Your Professional Title: [ ]Dr. [ ]Professor [ ]Mr. [ ]Mrs. [ ]Ms. [ ]Other:__________________ Office Phone (desk): Office Phone (message): Home/Evening Phone (for emergency contact): Fax: Electronic Mail (Bitnet): Electronic Mail (Internet): Postal Address: Institution or Center: Building Code: Mail Stop: Street Address1: Street Address2: City: State: Country: Zip or Country Code: Please list your three major interest areas: Interest1: Interest2: Interest3: ===================================================== From wray at ptolemy.arc.nasa.gov Mon Jul 19 23:51:27 1993 From: wray at ptolemy.arc.nasa.gov (Wray Buntine) Date: Mon, 19 Jul 93 20:51:27 PDT Subject: committees, agent teams, redundancy, Monte Carlo, ... Message-ID: <9307200351.AA03081@ptolemy.arc.nasa.gov> Some recent postings on connectionists have spurred me to draw a few comparisons with earlier work. I'm always amazed at when a new discovery is made, about 10 people will rediscover variations of the same thing and often times be too busy (and admirably) exploring the frontiers of research to relate their work to the current pool of research. The following stuff may seem different when you get into the fine detail, but they look mighty similar from a distance. committees (Thodberg '93, MacKay '93, both in connectionists) agent teams (Beyer & Smieja in connectionists as beyer.teams.ps), stacked generalization (Wolpert in Neural networks 5(2) '92, and Breiman in a TR from UC Berkeley Stats, '93) model averaging (Buntine and Weigend, Complex Systems, 5(1), '91, and on learning decision trees see Buntine, Statistics and Computing, v2, '92 who got the idea from S. Wok and C. Carter in UAI-87) Monte Carlo methods (surely this stuff is related!! see recent papers by Radford Neal, e.g. NIPS5) error correcting codes (Dietterich and Bakiri, AAAI-91) redundant knowledge (M. Gams, 4th EWSL '89) + probably lots more e.g MacKay's version of committee's which got the energy prediction prise smells like Breiman's version of Wolpert's stacked generalization e.g. Thodberg's committee's is a clever & pragmatic implementation of the model averaging approach suggested in Buntine and Weigend which itself is a cheap description of a standard Bayesian trick e.g. in the statistical community i'm told "model uncertainty" is all the rage in some circles, i.e. you don't return a single network but several for comparison and early work goes back many decades I find this fascinating because a few years ago we were all rediscovering smoothing (weight decay, weight elimination, regularisation, MDL, early stopping, cost complexity tradeoffs, etc., etc. etc.) in its various shapes and forms. Now we all seem to be rediscovering the use of multiple models, i.e. the next step in sophisticated learning algorithms. NB. i use the term "rediscovering" because I wouldn't dare attribute the discovery to any one individual !!! Nice work !! What's next ? ------------ Wray Buntine NASA Ames Research Center phone: (415) 604 3389 Mail Stop 269-2 fax: (415) 604 3594 Moffett Field, CA, 94035-1000 email: wray at kronos.arc.nasa.gov From terry at helmholtz.sdsc.edu Tue Jul 20 03:15:56 1993 From: terry at helmholtz.sdsc.edu (Terry Sejnowski) Date: Tue, 20 Jul 93 00:15:56 PDT Subject: Neural Computation 5:4 Message-ID: <9307200715.AA06550@helmholtz.sdsc.edu> Neural Computation, Volume 5, Number 4, July 1993 Review: The Use of Neural Networks in High Energy Physics Bruce Denby Articles: Stimulus Dependent Synchronization of Neuronal Assemblies E. R. Grannan, D. Kleinfeld and H. Sompolinsky Letters: Dynamics of Populations of Integrate-and-fire Neurons, Partial Synchronization and Memory Marius Usher, Heinz Georg Schuster and Ernst Niebur The Effects of Cell Duplication and Noise in a Pattern Generating Network Catherine H. Ferrar and Thelma L. Williams Emergence of Position-independent Detectors of Sense of Rotation and Dilation with Hebbian Learning: An Analysis Kechen Zhang, Martin I. Sereno and Margaret E. Sereno Improving Generalisation for Temporal Difference Learning: The Successor Representation Peter Dayan Discovering Predictable Classifications Jurgen Schmidhuber and Daniel Prelinger A Kinetic Model of Short- and Long-Term Potentiation M. Migliore and G. F. Ayala Artificial Dendritic Trees John G. Elias ----- SUBSCRIPTIONS - VOLUME 5 - BIMONTHLY (6 issues) ______ $40 Student ______ $65 Individual ______ $156 Institution Add $22 for postage and handling outside USA (+7% GST for Canada). (Back issues from Volumes 1-4 are regularly available for $28 each to institutions and $14 each for individuals Add $5 for postage per issue outside USA (+7% GST for Canada) MIT Press Journals, 55 Hayward Street, Cambridge, MA 02142. Tel: (617) 253-2889 FAX: (617) 258-6779 e-mail: hiscox at mitvma.mit.edu ----- From tommi at psyche.mit.edu Wed Jul 21 18:26:38 1993 From: tommi at psyche.mit.edu (Tommi Jaakkola) Date: Wed, 21 Jul 93 18:26:38 EDT Subject: Tech report available Message-ID: <9307212226.AA04323@psyche.mit.edu> THe following paper is now available on the neuroprose archive as "jaakkola.convergence.ps.Z". On the Convergence of Stochastic Iterative Dynamic Programming Algorithms Tommi Jaakkola Michael I. Jordan Department of Brain and Cognitive Sciences Massachusetts Institute of Technology Satinder P. Singh Department of Computer Science University of Massachusetts at Amherst Recent developments in the area of reinforcement learning have yielded a number of new algorithms for the prediction and control of Markovian environments. These algorithms, including the TD($\lambda$) algorithm of Sutton (1988) and the Q-learning algorithm of Watkins (1989), can be motivated heuristically as approximations to dynamic programming (DP). In this paper we provide a rigorous proof of convergence of these DP-based learning algorithms by relating them to the powerful techniques of stochastic approximation theory via a new convergence theorem. The theorem establishes a general class of convergent algorithms to which both TD($\lambda$) and Q-learning belong. From elsberry at beta.tricity.wsu.edu Wed Jul 21 12:39:44 1993 From: elsberry at beta.tricity.wsu.edu (Wesley Elsberry) Date: Wed, 21 Jul 93 09:39:44 -0700 Subject: UCLA Short Course on Wavelets announcement (Aug. 9-11) Message-ID: <9307211639.AA11928@beta.tricity.wsu.edu> ANNOUNCEMENT UCLA Extension Short Course The Wavelet Transform: Techniques and Applications Overview For many years, the Fourier Transform (FT) has been used in a wide variety of application areas, including multimedia compression of wideband ISDN for telecommunications; lossless transform for fingerprint storage, identification, and retrieval; an increased signal to noise ratio (SNR) for target discrimination in oil prospect seismic imaging; in-scale and rotation-invariant pattern recognition in automatic target recognition; and in heart, tumor, and biomedical research. This course describes a new technique, the Wavelet Transform (WT), that is replacing the windowed FT in the applications mentioned above. The WT uses appropriately matched bandpass kernels, called 'mother' wavelets, thereby enabling improved representation and analysis of wideband, transient, and noisy signals. The principal advantages of the WT are 1) its localized nature, which accepts less noise and enhances the SNR, and 2) the new problem-solving paradigm it offers in the treatment of nonlinear problems. The course covers WT principles as well as adaptive techniques, describing how WT's mimic human ears and eyes by tuning up "best mothers" to spawn "daughter" wavelets that catch multi-resolution components to be fed the expansion coefficient through an artificial neural network, called a "wavenet". This, in turn, provides the useful automation required in multiple application areas, a powerful tool when the inputs are constrained by real time sparse data (for example, the "cocktail party" effect where you perceive a desired message from the cacophony of a noisy party). Another advancement discussed in the course is the theory and experiment for solving nonlinear dynamics for information processing; e.g., the environmental simulation as a non-real time virtual reality. In other words, real time virtual reality can be achieved by the wavelet compression technique, followed by an optical flow technique to acquire those wavelet transform coefficients, then applying the inverse WT to retrieve the virtual reality dynamical evolution. (For example, an ocean wave is analyzed by soliton envelope wavelets.) Finally, implementation techniques in optics and digital electronics are presented, including optical wavelet transforms and wavelet chips. Course Materials Course note and relevant software are distributed on the first day of the course. The notes are for participants only, and are not for sale. Coordinator and Lecturer Harold Szu, Ph.D. Research physicist, Washington, D.C. Dr. Szu's current research involves wavelet transforms, character recognition, and constrained optimization implementation on a superconducting optical neural network computer. He is also involved with the design of a sixth-generation computer based on the confluence of neural networks and optical data base machines. Dr. Szu is also a technical representative to DARPA and consultant to the Office of Naval Research on neural networks and related research, and has been engaged in plasma physics and optical engineering research for the past 16 years. He holds five patents, has published about 100 technical papers, plus two textbooks. Dr. Szu is an editor for the journal Neural Networks and currently serves as the President of the International Neural Network Society. Lecturer and UCLA Faculty Representative John D. Villasenor, Ph.D. Assistant Professor, Department of Electrical Engineering, University of California, Los Angeles. Dr. Villasenor has been instrumental in the development of a number of efficient algorithms for a wide range of signal and image processing tasks. His contributions include application-specific optimal compression techniques for tomographic medical images, temporal change measures using synthetic aperture radar, and motion estimation and image modeling for angiogram video compression. Prior to joining UCLA, Dr. Villasenor was with the Radar Science and Engineering section of the Jet Propulsion Laboratory where he applied synthetic aperture radar to interferometric mapping, classification, and temporal change measurement. He has also studied parallelization of spectral analysis algorithms and multidimensional data visualization strategies. Dr. Villasenor's research activities at UCLA include still-frame and video medical image compression, processing and interpretation of satellite remote sensing images, development of fast algorithms for one- and two-dimensional spectral analysis, and studies of JPEG-based hybrid video coding techniques. For more information, call the Short Course Program Office at (310) 825-3344; Facsimile (213) 206-2815. Date: August 9-11 (Monday through Wednesday) Time: 8am - 5pm (subject to adjustment after the first class meeting), plus optional evening sessions, times to be determined. Location: Room 211, UCLA Extension Building, 10995 Le Conte Avenue (adjacent to the UCLA campus), Los Angeles, California. Reg# E8086W Course No. Engineering 867.118 1.8 CEU (18 hours of instruction) Fee: $1195, includes course materials ============================================================================ Wesley R. Elsberry, elsberry at beta.tricity.wsu.edu Sysop, Central Neural System BBS, FidoNet 1:3407/2, 509-627-6267 From reza at ai.mit.edu Thu Jul 22 11:01:00 1993 From: reza at ai.mit.edu (Reza Shadmehr) Date: Thu, 22 Jul 93 11:01:00 EDT Subject: Tech Reports from CBCL at M.I.T. Message-ID: <9307221501.AA11646@corpus-callosum.ai.mit.edu> The Center for Biological and Computational Learning (CBCL) is a newly formed organization at the Dept. of Brain and Cognitive Sciences at M.I.T. The Center's aim is to pursue projects which look at learning from a systems perspective, linking the neurophysiology of learning with its computational, mathematical, and conceptual components in areas of motor control, vision, speech, and language. Some of the work of the members of the Center is now available in the form of technical reports. These reports are published in conjuction with the AI Memo series. You can get a copy of these reports via anonymous ftp (see the end of this message for details). Here is a list of titles currently available via ftp: -------------- :CBCL Paper #79/AI Memo #1390 :author Jose L. Marroquin and Federico Girosi :title Some Extensions of the K-Means Algorithm for Image Segmentation and Pattern Classification :date January 1993 :pages 21 :keywords K-means, clustering, vector quantization, segmentation, classification :abstract We present some extensions to the k-means algorithm for vector quantization that permit its efficient use in image segmentation and pattern classification tasks. We show that by introducing a certain set of state variables it is possible to find the representative centers of the lower dimensional manifolds that define the boundaries between classes; this permits one, for example, to find class boundaries directly from sparse data or to efficiently place centers for pattern classification. The same state variables can be used to determine adaptively the optimal number of centers for clouds of data with space-varying density. Some examples of the application of these extensions are also given. -------------- :CBCL Paper #82/AI Memo #1437 :author Reza Shadmehr and Ferdinando A. Mussa-Ivaldi :title Geometric Structure of the Adaptive Controller of the Human Arm :date July 1993 :pages 34 :keywords Motor learning, reaching movements, internal models, force fields, virtual environments, generalization, motor control. :abstract The objects with which the hand interacts with may significantly change the dynamics of the arm. How does the brain adapt control of arm movements to this new dynamics? We show that adaptation is via composition of a model of the task's dynamics. By exploring generalization capabilities of this adaptation we infer some of the properties of the computational elements with which the brain formed this model: the elements have broad receptive fields and encode the learned dynamics as a map structured in an intrinsic coordinate system closely related to the geometry of the skeletomusculature. The low--level nature of these elements suggests that they may represent a set of primitives with which movement are represented in the CNS. ============================ How to get a copy of above reports: The files are in compressed postscript format and are named by their AI memo number, e.g., the Shadmehr and Mussa-Ivaldi paper is named AIM-1437.ps.Z. They are put in a directory named as the year in which the paper was written. Here is the procedure for ftp-ing: unix> ftp ftp.ai.mit.edu (log-in as anonymous) ftp> cd ai-pubs/publications/1993 ftp> binary ftp> get AIM-number.ps.Z ftp> quit unix> uncompress AIM-number.ps.Z unix> lpr AIM-number.ps.Z I will periodically update the above list as new titles become available. Best wishes, Reza Shadmehr Center for Biological and Computational Learning M. I. T. Cambridge, MA 02139 From niranjan at eng.cam.ac.uk Thu Jul 22 09:38:15 1993 From: niranjan at eng.cam.ac.uk (Mahesan Niranjan) Date: Thu, 22 Jul 93 09:38:15 BST Subject: Multiple Models, Committee of nets etc... Message-ID: <16459.9307220838@dsl.eng.cam.ac.uk> >From: Wray Buntine <wray at ptolemy.arc.nasa.gov> >Subject: RE: committees, agent teams, redundancy, Monte Carlo, ... >Date: Wed, 21 Jul 1993 01:57:31 GMT > [...] >e.g MacKay's version of committee's which got the energy prediction > prise smells like Breiman's version of Wolpert's stacked > generalization [...] >Now we all seem to be rediscovering the use of multiple models, >i.e. the next step in sophisticated learning algorithms. [...] An interesting and somewhat easy to understand application of multiple models is in the area of target-tracking (e.g. Bar-Shalom & Fortmann, 'Tracking and Data Association', Academic Press 1988, ISBN 0-12-079760). They show how to run several models in parallel, recursively estimating them with Kalman filtering and use the innovation probabilities for model selection. Apart from terminology (like you dont see the term "evidence" there), a lot of the ideas are in that framework too; but the assumptions etc are much clearer (at least to me), and the language not so strong. We have used this method to track parametric models of highly nonstationary signals (e.g. Formants in speech). The committee of networks doing the energy prediction (committee members chosen by ranking models by performance on cross-validation set, and the average performance of these being better than the best member) is a somewhat surprising result to me. Surprising because, the average predictions are taken without weighting by the model probabilities (which are difficult to compute). In practice, even for linear models in Gaussian noise, I find probabilities tend to differ by large numbers, for models that look very similar. Hence if these are difficult to evaluate and are assumed equal, I would have expected the average performance to be worse than the best member. In real life too, committees tend to be less efficient than the good individual members (when you give the members equal say), but thats a different story :-) niranjan From xiru at Think.COM Fri Jul 23 11:27:12 1993 From: xiru at Think.COM (Xiru Zhang) Date: Fri, 23 Jul 93 11:27:12 EDT Subject: Multiple Models, Committee of nets etc... Message-ID: <9307231527.AA23862@yangtze.think.com> Just to add another reference to the list: We have used a HYBRID system of three "experts" (neural net, statistical model, memory-based reasoning) for protein secondary structure prediction, and obtained, I believe, the best prediction accuracy reported to date. For reference, see: @article{jmb92, author = {Xiru Zhang, Jill P. Mesirov and David L. Waltz}, title = "Hybrid System for Protein Secondary Structure Prediction", journal = {Journal of Molecular Biology}, year = {1992}, volume = {225} } - Xiru Zhang Thinking Machines Corp. 245 First St. Cambridge, MA 02142 From dhw at santafe.edu Fri Jul 23 11:39:22 1993 From: dhw at santafe.edu (David Wolpert) Date: Fri, 23 Jul 93 09:39:22 MDT Subject: combining generalizers' guesses Message-ID: <9307231539.AA26719@sfi.santafe.edu> Mahesan Niranjan writes: >>> The committee of networks doing the energy prediction (committee members chosen by ranking models by performance on cross-validation set, and the average performance of these being better than the best member) is a somewhat surprising result to me. Surprising because, the average predictions are taken without weighting by the model probabilities (which are difficult to compute). In practice, even for linear models in Gaussian noise, I find probabilities tend to differ by large numbers, for models that look very similar. Hence if these are difficult to evaluate and are assumed equal, I would have expected the average performance to be worse than the best member. >>> In general, when using stacking to combine guesses of separate generalizers (i.e., when combining guesses by examining validation set behavior), one doesn't simply perform an unweighted average, as MacKay did, but rather a weighted average. For example, in Leo Breiman's Stacked regression paper of last year, he combined guesses by means of a weighted average. The weights were set to minimize LMS error on the validation sets. (Sets plural because J-fold partitioning of the training data was used, like in cross-validation, rather than a single split into a training set and a validation set.) In literally hundreds of regression experiments, Leo found that this almost always beat cross-validation, and never (substantially) lost to it. In essence, in this scheme validation set behavior is being used to estimate the model "probabilities" Niranjan refers to. Also, in MacKay's defense, just because he "got the probabilities wrong" doesn't imply his average would be worse than just choosing the single best model. Just a few of the other factors to consider: 1) What is the relationship between mis-assignment of model probabilities, model's guess, and optimal guess? 2) How do estimation errors (due to finite validation sets, due to finite training sets) come into play? Also, it should be noted that there are other ways to perform stacking (either to combine generalizers or to improve a single one) which do not use techniques which are interprable in terms of "model probabilities". For example, rather than combining generalizers via the generalizer "find the hyperlane (w/ non-negative summing-to-1 coefficients) w/ the minimal LMS error on the data", which is what Leo did, one can instead use nearest neighbor algorithms, or even neural nets. In general though, one should use such a "second level" generalizer which has low variance, i.e., which doesn't bounce around a lot w/ the data. Otherwise you can easily run into the kinds of problems Niranjan worries about. David Wolpert References: Breiman, L, "Stacked regressions", TR 367, Dept. of Stat., Univ. of Cal. Berkeley (1992). Wolpert, D., "Stacked Generalization", Neural Networks, vol. 5, 241-259 (1992). (Aside from an early tech. report, the original public presentation of the idea was at Snowbird '91.) I also managed to convince Zhang, Mesirov and Waltz to try combining with stacking rather than with non-validation-set-based methods (like Qian and Sejnowski used), for the problem of predicting protein secondary structure. Their (encouraging) results appeared last year in JMB. From gmk at learning.siemens.com Fri Jul 23 10:20:17 1993 From: gmk at learning.siemens.com (Gary M. Kuhn) Date: Fri, 23 Jul 93 10:20:17 EDT Subject: Advance Program NNSP'93 Message-ID: <9307231420.AA01000@petral.siemens.com> ADVANCE PROGRAM 1993 IEEE Workshop on Neural Networks for Signal Processing September 6 - September 9, 1993 Maritime Institute of Technology and Graduate Studies Linthicum Heights, Maryland, USA Sponsored by IEEE Signal Processing Society (In Cooperation with IEEE Neural Networks Council) Co-sponsored by Siemens Corporate Research ARPA-MTO INVITATION TO PARTICIPATE The members of the Workshop Organizing Committee invite you to attend the 1993 IEEE Workshop on Neural Networks for Signal Processing. The 1993 Workshop is the third workshop organized by the Neural Network Technical Committee of the IEEE Signal Processing Society. The first took place in 1991 in Princeton, NJ, USA, the second in 1992 in Helsingor, Denmark. The purpose of the Workshop is to foster informal technical interaction on topics related to the application of neural networks to problems in signal processing. WORKSHOP LOCATION The 1993 Workshop will be held at the Maritime Institute of Technology Graduate Studies (MITAGS), 5700 Hammonds Ferry Road, Linthicum Heights, MD, 21090, USA, telephone +1 410 859 5700. MITAGS is a training facility of the International Organization of Masters, Mates & Pilots. TRANSPORTATION TO MITAGS MITAGS is located directly south of Baltimore, Maryland. For those arriving by air at the Baltimore - Washington International Airport, MITAGS is 5 miles away by taxi. For those arriving by private car, we provide the following directions: 1. From NORTH VIA I-95: South through Fort McHenry Tunnel, stay on I-95 South to I-695 South (Glen Burnie). Proceed as in #2 below. 2. FROM BELTWAY(I-695) COMING FROM THE NORTH OR WEST: Get off at Exit 8 (Hammonds Ferry-Nursery Road), turn left at end of exit, straight through the traffic light, over the bridge to the sign saying MASTERS, MATES & PILOTS. Follow the driveway (blue lines) to Day Visitor Lots A, B or C, or to the Overnight Lot. 3. FROM BALTIMORE VIA BALTIMORE-WASHINGTON PARKWAY (I-295): Go South to I-695 West (To Towson). Proceed as in #4 below. 4. FROM BELTWAY(I-695) COMING FROM THE EAST: Get off at exit 8 (Nursery Road), stay to the right until you face a sign saying Hammonds Ferry Road, turn right, go to a traffic light, turn left on Hammonds Ferry road, continue over the bridge, turn right at the sign saying MASTERS, MATES & PILOTS and proceed to Day Visitor Parking Lots A through C, or if you are staying overnight, to the Overnight Lot. 5. FROM SOUTH VIA BALTIMORE-WASHINGTON PARKWAY (I-295): Turn off at exit just before Baltimore Beltway (I-695). This exit says West Nursery Road. Stay to the right on the exit ramp. Go to first light (International Drive), turn left to bottom of hill, left onto Aero Drive. At gate, sign says MASTERS, MATES & PILOTS. Turn right and follow blue line to parking lots. 6. FROM SOUTH VIA I-95: Go to Baltimore Beltway (I-695) South towards Glen Burnie. Get off at Exit 8 as in #3 above. WORKSHOP REGISTRATION INFORMATION There is a Registration Form at the end of the Advanced Program. The registration fee "without room" covers attendance at all workshop sessions, one copy of the hard-bound proceedings, the Monday night reception, the coffee breaks, and all meals during the three days of the Workshop, including the banquet at the Baltimore Orioles' Baseball Stadium. For those potential participants whose funds are not permitted to be spent on a banquet, we point out that corporation co-sponsorship is paying for the banquet. The registration fee "with room" covers all of the above plus lodging on the campus at MITAGS. Lodging at MITAGS is by far the most convenient, and it is very reasonably priced. This is the registration that we recommend for all participants coming from a distance. For IEEE members before August 1, the registration fee is $375 without room and $575 with room. After August 1, the registration fee is $425 without room and $625 with room. Non-members, please add $50 to these fees. Students may apply for a limited number of partial travel and registration grants. See registration form below. EVENING EVENTS A Pre-Workshop Reception will be held at MITAGS on Monday evening, September 6, 1993, at 8:00 PM. On Tuesday evening, a panel on Dual-use Applications of Neural Network Technology will be led by Workshop Co-Chair Barbara Yoon. On Wednesday evening, busses will take participants to the banquet at the new Baltimore Orioles' Baseball Stadium in downtown Baltimore. Dinner will be served in the Orioles' 6th floor banquet facility. Reservations have been made for 120 participants. Each reservation includes a ticket to the party rooms reserved for the workshop down on the Stadium Club level. After the banquet, participants may either relax in the banquet facility, or move to the party rooms and adjacent outside seating to enjoy the scheduled evening baseball game with the Seattle Mariners. Busses will return everyone to MITAGS at the end of the game. At the close of the Workshop on Thursday afternoon, participants are invited to stay for a demonstration of MITAG's $50 million computer-based simulator of the bridge of a ship. This hydraulically-mounted simulator will be operated in a cinerama representation of the harbor of the City of New York. PROGRAM OVERVIEW Time Tuesday 7/9/93 Wednesday 8/9/93 Thursday 9/9/93 _______________________________________________________________ 8:15 AM Opening Remarks 8:30 AM Keynote Address Keynote Address Keynote Address 9:20 AM Image Processing Learning Theory (Lecture) (Lecture) (Lecture) 10:50 AM Break Break Break 11:20 AM Theory Applications 1 Applications 2 (Poster preview) (Poster preview) (Poster preview) 12:20 PM Lunch Lunch Lunch 1:30 PM Theory Applications 1 Applications 2 (Poster) (Poster) (Poster) 2:45 PM Break Break Break 3.15 PM Classification Speech Processing Applications (Lecture) (Lecture) (Lecture) Evening Panel Banquet at Simulator Discussion Orioles Stadium Demonstration Note: Session Chairs listed in the following Technical Program may change. TECHNICAL PROGRAM Tuesday, September 7, 1993 [8:15 AM: Opening Remarks:] Gary Kuhn, Barbara Yoon, General Chairs Rama Chellappa, Program Chair [8:30 AM: Opening Keynote:] Learning, function approximation, and images Tomaso Poggio, Massachusetts Institute of Technology, Massachusetts, USA. [9:20 AM: Image Processing (Lecture Session)] Chair: B.S. Manjunath, UCSB A Nonlinear Scale-Space Filter by Physical Computation, Yiu-Fai Wong, Lawrence Livermore National Lab, Livermore, CA, USA. A Common Framework for Snakes and Kohonen Networks, Arnaldo J. Abrantes and Jorge S. Marques, INESC, Lisboa, Portugal. Detection of Ocean Wakes in Synthetic Aperture Radar Images with Neural Networks, Gregg Wilensky, Narbik Manukian, Joe Neuhaus and John Kirkwood, Logicon/RDA, Los Angeles, CA, USA. Image Generation and Inversion Based on a Probabilistic Recurrent Neural Model, N. Sonehara, K. Nakane, Y.Tokunaga, NTT Human Interface Laboratories, Yokosuka, Kanagawa, Japan. [10:50 AM: Coffee break] [11:20 AM: Theory (Oral previews of the afternoon poster session)] Chair: To be announced Liapunov Functions for Additive Neural Networks and Nonlinear Integral Equations of Hammerstein Type, Alexander Jourjine, Wang Laboratories, Lowell, MA, USA. A Hybrid Learning Method for Multilayer Neural Networks, Xin Wang, Meide Zhao, Department of Radio Engineering, Harbin Institute of Technology, Harbin, P.R. China. LS-Based Training Algorithm for Neural Networks, E.D. Di Claudio, R. Parisi and G. Orlandi, INFOCOM Department, University of Roma ``La Sapienza", Roma - Italy. MAP Estimation and the Multilayer Perceptron, Q. Yu and M.T. Manry, Dept. of Electrical Engineering, University of Texas at Arlington, Arlington, Texas and S.J. Apollo, General Dynamics, Fort Worth, Texas, USA. Self-Organizing Feature Map with Position Information and Spatial Frequency Information, Toshio Nakagawa and Takayuki Ito, NHK Science and Technical Research Laboratories, Setagaya-ku, Tokyo, Japan. Competitive Learning and Winning-Weighted Competition for Optimal Vector Quantizer Design, Zhicheng Wang and John V. Hanson, Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Ontario, Canada. Hierarchical Wavelet Neural Networks, Sathyanarayan S. Rao and Ravikanth S. Pappu, Department of Electrical and Computer Engineering, Villanova University, Villanova, PA, USA. Nonlinear Multilayer Principal Component Type Subspace Learning Algorithms, Jyrki Joutsensalo and Juha Karhunen, Helsinki University of Technology, Laboratory of Computer and Information Sciences, Espoo, Finland. Designer Networks for Time Series Processing, Claus Svarer, Lars Kai Hansen, Jan Larsen and Carl Edward Rasmussen, Technical University of Denmark, Denmark, USA. A Class of Hopfield Decodable Codes, Niclas Wiberg, Dept. of Electrical Engineering, Linkoping University, Linkoping, Sweden. Modeling the Spectral Transition Selectivity in the Primary Auditory Cortex, Kuansan Wang and Shihab A. Shamma, Institute of Systems Research and Department of Electrical Engineering,University of Maryland, College Park, MD, USA. Signal to Noise Analysis of a Neural Network with Nonmonotonic Dynamics, Ioan Opris, Dept. of Physics, University of Bucharest, Bucharest-Magurele, Romania. [12:20 PM: Lunch] [1:30 PM: Theory (Poster Session)] [2:45 PM: Break] [3:15 PM: Classification (Lecture session)] Chair: Candace Kamm, Bellcore Ordered Vector Quantization for Neural Network Pattern Classification, Lane Owsley and Les Atlas, University of Washington, Seattle, WA, USA. Differentially Trained Neural Network Classifiers are Efficient, J.B.Hampshire II and B.V.K. Vijaya Kumar, Carnegie Mellon University, Pittsburg, PA, USA. Extensions of Unsupervised BCM Projection Pursuit: Recurrent and Differential Models for Time-Dependent Classification, Charles M. Bachmann and Dong Luong, Naval Research Laboratory, Washington, D.C., USA. Fuzzy Decision Neural Networks and Signal Recognition Applications, J.S.Taur and S.Y. Kung, Dept. of Electrical Engineering, Princeton University, Princeton, USA. Temporal Sequence Classification by Memory Neuron Networks, Pinaki Poddar and P.V.S. Rao Tata Institute of Fundamental Research, Bombay, India. [8:00 PM: Panel Discussion] Dual-use Applications of Neural Network Technology Moderator: Barbara Yoon, ARPA. Wednesday, September 8, 1993 [8:30 AM: Keynote Address:] Pattern matching in a rapidly self-organizing neural network. Christoph von der Malsburg, Institute for Neuroinformatics, Ruhr-University, Bochum, Germany, and Dept. of Computer Science, USC, Los Angeles, USA. [9:20 AM: Learning (Lecture Session)] Chair: Lee Giles, NEC Neural Networks for Localized Approximation of Real Functions, H.N. Mhaskar, Department of Mathematics, California State University, Los Angeles, CA, USA. Generalization and Maximum Likelihood from Small Data Sets, Bill Byrne, Institute for Systems Research and Dept. of Electrical Engineering, University of Maryland, College Park, MD, USA. Backpropagation Through Time with Fixed Memory Size Requirements, Jose C. Principe and Jyh-Ming Kuo, Computational NeuroEngineering Laboratory, University of Florida, Gainesville, FL, USA. Hierarchical Recurrent Networks for Learning Musical Structure, D.J. Burr, Bellcore, Morristown, NJ, USA and Y. Miyata, Chukyo University, Toyota, Japan. [10:50 AM: Coffee break] [11:20 AM: Applications 1 (Oral previews of afternoon poster session) ] Chair: To be announced VLSI Hamming Neural Net Showing Digital Decoding, Felipe Gomez-Castaneda and Jose A. Moreno-Cadenas, Electrical Engineering Department, Mexico City, D.F. Mexico. Target Recognition Using Multiple Sensors, Y.T. Zhou and R. Hecht-Nielsen, HNC, Inc. San Diego, CA, USA. Recognition of Earthquakes and Explosions Using a Data Compression Neural Network, Roy C. Hsu and Shelton S. Alexander, The Pennsylvania State University, University Park, PA, USA. Characterization of Network Responses to Known, Unknown, and Ambiguous Inputs, Benjamin Hellstrom and Jim Brinsley, Westinghouse Electronic Corporation, Baltimore, MD, USA. Neural Network-Based Helicopter Gearbox Health Monitoring System, Peter T. Kazlas, Peter T. Monsen and Michael J. LeBlanc, The Charles Stark Draper Laboratory, Cambridge, MA, USA. A Hybrid Neural-Fuzzy Approach to VHF Frequency Management, Nancy H. Millstrom, Allen R. Bonde, Jr., and Michael J. Grimaldi, GTE Goverment Systems, Needham Heights, MA, USA. Analysis of Coarse Parallel Architectures for Artificial Neural Processing, K.S. Gugel, J.C Principe and S. Venkumahanti, Computational NeuroEngineering Lab, University of Florida, Gainesville, Florida, USA. Fast VLSI Implementations for MRF and ANN Applications, Haralambos C. Karathanasis, INTRACOM S.A and Computer Engineering Dept., University of Patras and John A. Vlontzos, INTRACOM S.A, Peania Attika, Greece. A Growing and Splitting Elastic Network for Vector Quantization, Bernd Fritzke, International Computer Science Institute, Berkeley, CA, USA. Quantized, Piecewise Linear Filter Network, John Aasted Sorensen, Electrical Institute, Technical University of Denmark, Lyngby, Denmark. Application of ordered codebooks to Image Coding, S. Carrato, Giovanni, L. Sicuranza and L. Manzo, D.E.E.I., University of Trieste, 34127 Trieste, Italy. [12:20 PM: Lunch] [1:30 PM: Applications 1 (Poster Session)] [2:45 PM: Break] [3:15 PM: Speech Processing (Lecture session)] Chair: Fred Juang, AT&T Bell Laboratories A New Learning Algorithm for Minimizing Spotting Errors, Takashi Komori and Shigeru Katagiri, ATR Auditory and Visual Preception Research Laboratories, Kyoto, Japan. A Neural network for Phoneme Recognition Based on Multiresolution Ideas, Kamran Etemad, Institute for System Research and Dept. of Electrical Engineering, University of Maryland, College Park, MD, USA. Text-Dependent Speaker Verification Using Recurrent Time Delay Neural Networks for Feature Extraction, Xin Wang, Department of Radio Engineering, Harbin Institute of Technology, Harbin, P.R. China. A New Learning Approach Based on Equidistortion Principle for Optimal Vector Quantizer Design, Naonori Ueda, Ryohei Nakano, NTT Communication Science Laboratories, Kyoto, Japan. A feedforward neural network for the wavelet decomposition of discrete time signals, Sylvie Marcos, Messaoud Benidir, Laboratoire des Signaux et Systems, E.S.E, Gif-sur-Yvette, France. [5:00 PM: Busses leave for Orioles' Stadium ] [10:00 PM: Busses return from Orioles' Stadium ] Thursday, September 9, 1993 [8:30 AM: Keynote Address:] Evaluation of Neural Network Classifiers Charles L. Wilson, National Institute of Standards and Technology, Gaithersburg, MD, USA. [9:20 AM: Theory (Lecture Session)] Chair: To be announced A Novel Recursive Network for Signal Processing, Irina F. Gorodnitsky and Bhaskar D. Rao, Dept of Electrical and Computer Engineering, University of California, San Diego, USA. A Geometric View of Neural Networks Using Homotopy, Frans M. Coetzee and Virginia L. Stonick, Electrical and Computer Engineering Department, Carnegie Mellon University, Pittsburgh, PA, USA. Nonlinear Predictive Vector Quantisation with Recurrent Neural Nets, Lizhong Wu, Mahesan Niranjan and Frank Fallside, Engineering Dept., Cambridge University, Cambridge. UK. Further Development of Hamiltonian Dynamics of Neural Networks, Ulrich Ramacher, Siemens Corporation, Muenchen, Germany. Invited Talk. [10:50 AM: Coffee break] [11:20 AM: Applications 2] (Oral previews of afternoon poster sessions) Chair: David Burr, Bellcore A Modified Recurrent Cascade-Correlation Network for Radar Signal Pulse Detection, N. Karunanithi, Bellcore, Morristown, NJ., D. Whitley, Dept. of Computer Science, Colorado State University, Fort Collins, CO and D. Newman, Texas Instruments, Colorado Springs, CO, USA. A Technique for Adapting to Speech Rate, Mai Huong T. Nguyen and Garrison W. Cottrell, Institute for Neural Computation, University of California, San Diego, USA. Neurofuzzy Control of a Wheelchair Robotic Hand, Anya L. Tascillo and Victor A. Skormin, Binghamton University, Binghamton, NY, USA. Applying Neural Network Developments to Sign Language Translation, Elizabeth J. Wilson, and Gretel Anspach, Raytheon Company, Riverside, RI, USA. Discriminative Feature Extraction for Speech Recognition, Alain Biem, Shigeru Katagiri and Biing-Hwang Juang, ATR Auditory and Visual Perception Research Laboratories, Kyoto, Japan. Neural Network Image Analysis and Classification in Hybrid Lung Nodule Detection (HLND) System, Y.S Peter Chiou, Y.M. Fleming Lure, Caelum Research Corporation, Silver Spring, MD,USA. A Nonlinear Lattice Structure Based Higher Order Neuron, Muzaffar U. Khurram and Hassan M. Ahmed, Nonlinear Modelling Laboratory, Boston University, Boston, MA, USA. Multisensor Image Classification by Structured Neural Networks, F. Roli, S.B. Serpico and G.Vernazza, Dept. of Biophisical and Electronic Eng., University of Genoa, Italy. A Modular Neural Network Architecture for Pattern Classification, H. Elsherif, M. Hambaba, Intelligent Systems Laboratory, Stevens Institute of Technology, Hoboken, NJ, USA. Compressing Moving Pictures Using the APEX Neural Principal Component Extractor, K.I. Diamantaras, Siemens Corp. Research, S.Y. Kung, Dept. Electrical Eng., Princeton, NJ, USA. Printed Circuit Boards Inspection Using two New Algorithms of Dilatation and Connectivity Preserving Shrinking, Jelloul El Mesbahi, Hassan II University Casablanca and Mohamed Chaibi, Rabat Instituts, Rabat Morocco. Using Self-Organized and Supervised Learning Neural Networks in Parallel for Automatic Target Recognition, Magnus Snorrason, Alper K. Caglayan, Charles River Analytics Inc. Cambridge, MA and Bruce T. Buller, Department of Air force, FL, USA. A Neural Net Application to Signal Identification, Ronald Sverdlove, David Sarnoff Research Center, Princeton, NJ, USA. [12:20 PM: Lunch] [1:30 PM: Applications 2 (Poster Session)] [2:45 PM: Break] [3:15 PM: Applications (Lecture session)] Chair: Bastiaan Kleijn, AT&T Bell Laboratories A Neural Network Model for Adaptive, Non-Uniform A/D Conversion, Marc M.Van Hulle, Massachusetts Institute of Technology, Cambridge, MA, USA. Recurrent Radial Basis Function Networks for Optimal Blind Equalization, Jesus Cid-Sueiro and Anibal R. Figueiras-Vidal, ETSI Telecomunication-UV, Valladolid, Spain. Neural Networks for the Classification of Biomagnetic Maps, Martin F. Schlang, Ralph Neuneier, Siemens AG, Corporate Research and Development, Munchen, Klaus Abraham- Fuchs and Johann Uebler, Siemens AG, Medical Eng., Group, Erlangen, Germany. Hidden Markov Models and Neural Networks for Fault Detection in Dynamic System, Padhraic Smyth, Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA, USA. [4:30 PM: MITAGS Simulator demonstration] WORKSHOP COMMITTEE GENERAL CHAIRS Gary Kuhn Barbara Yoon Siemens Corporate Research ARPA-MTO Princeton, NJ 08540 Arlington, VA, 22209, USA email: gmk at learning.siemens.come e-mail: byoon at arpa.mil PROGRAM CHAIR Rama Chellappa Dept. of Electrical Engineering University of Maryland College Park, MD 20742, USA. email: chella at surya.eng.umd.edu PROCEEDINGS CHAIR Candace Kamm Bellcore, 445 South Street Morristown, NJ 07960, USA email: cak at thumper.bellcore.com FINANCE CHAIR Raymond Watrous Siemens Corporate Research Princeton, NJ 08540, USA email: watrous at learning.siemens.com PROGRAM COMMITTEE Joshua Alspector Yann Le Cun Les Atlas Richard Lippman Charles Bachmann John Makhoul David Burr Christoph von der Malsburg Rama Chellappa Richard Mammone Gerard Chollet B.S. Manjunath Frank Fallside Dragan Obradovic Emile Fiesler Tomaso Poggio Lee Giles Jose Principe Steve Hanson Ulrich Ramacher Yu Hen Hu Noboru Sonehara Jenq-Neng Hwang Eduardo Sontag B.H. Juang John Sorenson Candace Kamm Yoh'ichi Tohkura Juha Karhunen Kari Torkkola Shigeru Katagiri John Vlontzos Sun-Yan Kung Raymond Watrous Gary Kuhn Christian Wellekens REGISTRATION FORM: 1993 IEEE Workshop on Neural Networks for Signal Processing. September 6 - September 9, 1993. Please complete this form (type or print clearly) and mail with payment for fee to NNSP'93, c/o R.L. Watrous, Siemens Corporate Research, 755 College Road East, Princeton, NJ 08540, USA. Name __________________________________________________________________ Last First Middle Firm or University ____________________________________________________ Mailing address _______________________________________________________ _______________________________________________________________________ _______________________________________________________________________ Country Phone Fax Fee payment must be made by money order/check/bank draft drawn on a U.S. bank or U.S. branch of a foreign bank. Do not send cash. Make fee payable to IEEE NNSP'93. Registration fee with single room and meals Date IEEE member Non-member ____________________________________________ Before August 1 U.S. $575 U.S. $625 After August 1 U.S. $625 U.S. $675 Registration fee without room but still with meals Date IEEE member Non-member ____________________________________________ Before August 1 U.S. $375 U.S. $425 After August 1 U.S. $425 U.S. $475 From tgd at chert.CS.ORST.EDU Sat Jul 24 11:43:24 1993 From: tgd at chert.CS.ORST.EDU (Tom Dietterich) Date: Sat, 24 Jul 93 08:43:24 PDT Subject: combining generalizers' guesses Message-ID: <9307241543.AA11416@curie.CS.ORST.EDU> It seems to me that more attention needs to be paid to *which* generalizer's guesses we are combining. There are three basic components that determine generalization error: * inherent error in the data (which determines the bayes optimal error rate) * small sample size * approximation error (which prevents the algorithm from correctly expressing the bayes optimal hypothesis, even with infinite samples) Combining guesses can't do anything about the first problem. I also don't think it can have a very great effect on the second problem, because all of the guesses are based on the same data. I think that the real win comes from combining guesses that make very different approximation errors. In my error-correcting code work, we re-code the outputs (by imposing an error-correcting distributed representation) in such a way that (we believe) the approximation errors committed by the learning algorithm are nearly independent. Then the decoding process combines these guesses. Interestingly, the decoding process takes a linear combination of the guesses where the linear combination is unique for each output class. We are currently doing experiments to try to understand the relative role of these three sources of error in the performance of error-correcting output codes. This analysis predicts that using a committee of very diverse algorithms (i.e., having diverse approximation errors) would yield better performance (as long as the committee members are competent) than a committee made up of a single algorithm applied multiple times under slightly varying conditions. In the error-correcting code work, we compared a committee of decision trees to an error-correcting output procedure that also used decision trees. The members of the committee were generated by training on different subsamples of the data (as in stacking), but the combination method was simple voting. No matter how many trees we added to the committee, we could not come close to achieving the same performance on the nettalk task as with the error-correcting output coding procedure. So, it seems to me the key question is what are the best ways of creating a diverse "committee"? --Tom From gomes at ICSI.Berkeley.EDU Mon Jul 26 12:53:47 1993 From: gomes at ICSI.Berkeley.EDU (Benedict A. Gomes) Date: Mon, 26 Jul 93 09:53:47 PDT Subject: No subject Message-ID: <9307261653.AA25101@icsib6.ICSI.Berkeley.EDU> I am interested in references for the problem of automatically mapping neural nets onto parallel machines i.e. algorithms for net partitioning, especially for structured connectionist nets. I'm particularly interested in the CM-5 and toroid/mesh distributed memory MIMD machines. References to work on mapping neural nets to particular architectures follows. However the application of automatic partitioning and mapping methods to structured nets is not dealt with. If this question has been asked before on this list or in the newsgroup, my apologies and I would appreciate a copy of any existing bibliography. Otherwise, I will compile and post the references I receive. I am also interested in new developments so as to maintain an up-to-date bibliography. Thanks! ben (gomes at icsi.berkeley.edu) @TechReport{Bahr88, author = "Bahr, Casey S.", title = "ANNE: Another Neural Network Emulator", institution = "Oregon Graduate Center", year = 1988, number = "CS/E-88-028", month = "August" } @Article{Blel87, author = "Blelloch, G. and Rosenberg, C.", title = "Network Learning on the Connection Machine", journal = "Proc. 10th Joint Conference on Artificial Intelligence", year = 1987 } @Article{PN:chung92, author = "Chung, Sang-Hwa and Moldovan, D.I.", title = "Modeling Semantic Networks on the Connection Machine" pages = "152--163" journal = The Journal of Parallel and Distributed Computing, volume = 17, year = 1992 } @TechReport{Fanty86, author = "Fanty, M.", title = "A Connectionist Simulator for the BBN Butterfly Multiprocessor", institution = "University of Rochestor", year = 1986, month = January } @Article{PN:kim89, author = "Kim, Kichul and Kumar, V.K. Prasanna", title = "Efficient implementation of neural networks on hypercube SIMD arrays", journal = "International Joint Conference on Neural Networks", year = 1989 } @Article{PN:kim93, author = "Kim, J.T. and Moldovan, D.I.", title = "Classification and retrieval of knowledge on parallel marker-passing architecture", journal = "IEEE Transactions on Knowledge and Data Engineering", month = oct, year = 1993 } @Article{PN:pomerleau88, author = "Pomerleau, Dean A., Gusciora, George L., Touretsky, David S., and Kung, H.T.", title = "Neural network simulation at warp speed: How we go 17 million connections per second", journal = "IEEE International Conference on Neural Networks", year = 1988, pages = II:155-161 } @TechReport{PN:singer90, author = "Singer, Alexander", title = "Implementations of Artificial Neural Networks on the Connection Machine", institution = "Thinking Machines Corporation", year = 1990, number = "RL90-2" } From bap at learning.siemens.com Mon Jul 26 13:27:39 1993 From: bap at learning.siemens.com (Barak Pearlmutter) Date: Mon, 26 Jul 93 13:27:39 EDT Subject: combining generalizers' guesses In-Reply-To: Tom Dietterich's message of Sat, 24 Jul 93 08:43:24 PDT <9307241543.AA11416@curie.CS.ORST.EDU> Message-ID: <9307261727.AA22861@gull.siemens.com> To my mind, there is a fourth source of error, which is also addressed by the ensemble or committee approach. To your * noise in the data * sampling error * approximation error I would add * randomness in the classifier itself For instance, if you run backpropagation on the same data twice, with the same architecture and all the other parameters held the same, it will still typically come up with different answers. Eg due to differences in the random initial weights. Averaging out this effect is a guaranteed win. --Barak. From drucker at monmouth.edu Mon Jul 26 14:15:19 1993 From: drucker at monmouth.edu (Drucker Harris) Date: Mon, 26 Jul 93 14:15:19 EDT Subject: No subject Message-ID: <9307261815.AA01088@harris.monmouth.edu> Subject: Committee Machines: The best method to generate a committee of learning machines is given by Schapire's algorithm [1]. The boosting algorithm that constructs a committee of three machines is as follows: (1) Train a first learning machine using some training set. (2) A training set for a second committee machine is obtained in the following manner: (a) Toss a fair coin. If heads, pass NEW data through the first machine until the first machine misclassifies the data and add this misclassified data to the training set for the second machine. If the coin tossing is tails pass data through the first network until the first network classifies correctly and add this data to the training set for the second machine. Thus the training set for the second machine consists of data which if passed through the first machine would give a 50% error rate. This procedure is iterated until there is a large enough training set. Data classified correctly when the coin tossing is heads or classified incorrectly when the coin tossing is tails is not used. (b) train the second machine. (3) A training set for a third machine is obtained in the following manner: (a) Pass NEW data through the first two trained machines. If the two machines agree on the classification (whether correct or not), toss out the data. If they disagree, add this data to the training set for the third machine. Iterate until there is a large enough training set. (b) Train the third machine. (4) In the testing phase, a pattern is presented to all three machines. If the first two machines agree, use that labeling; otherwise use the labeling of the third machine. The only problem with this approach is generating enough data. For OCR recognition we have synthetically enlarged the database by deforming the original data [2]. Boosting dramatically improved error rates. We are publishing a new paper that has much more detail [3]. Harris Drucker References: 1. R.Schapire, "The Strength of weak learnability" Machine Learning 5, Number 2, (1990), p197-227 2. H.Drucker, R.Schapire, and P. Simard, "Improving Performance in Neural Networks Using a Boosting Algorithm" Neural Information Processing Systems 5, proceeding of the 1992 conference (published 1993), Eds: J.Hanson, J Cowan, C.L. Giles p. 42-49. 3.H.Drucker, R. Schapire, P. Simard, "Boosting Performance in Neural Networks", International Journal of Pattern Recognition and Artificial Intelligence, Vol 7, Number 4, (1993), to be published. From cohn at psyche.mit.edu Mon Jul 26 12:01:15 1993 From: cohn at psyche.mit.edu (David Cohn) Date: Mon, 26 Jul 93 12:01:15 EDT Subject: combining generalizers' guesses Message-ID: <9307261601.AA02163@psyche.mit.edu> Tom Dietterich <tgd at chert.cs.orst.edu> writes: > ... (good stuff deleted) ... > This analysis predicts that using a committee of very diverse > algorithms (i.e., having diverse approximation errors) would yield > better performance (as long as the committee members are competent) > than a committee made up of a single algorithm applied multiple times > under slightly varying conditions. > > ... > > So, it seems to me the key question is what are the best ways of > creating a diverse "committee"? > > --Tom One possible way of diversifying the committee (don't *I* sound PC!) is to make the inductive bias of the learning algorithm explicit, or as an approximation, add a new inductive bias that is strong enough to override biases inherent in the algorithm. This can be done a number of ways, by adding extra terms to the error equation or some other kludges. By then running the same algorithm with widely differing biases, one can approximate different algorithms. [warning: blatant self-promotion follows :-)] For example, a few years ago, we looked at something like this with a different end in mind. The selective sampling algorithm was used to identify potentially useful training examples by means of what has become known as the committee approach (with a twist). Two identical networks were trained on the same positive-negative classification problem with the same training data. We added two different inductive biases to the backprop training, though: One network (S) was trained to find the most *specific* concept consistent with the data. That is, it was to try to classify only the positive training examples as positive, and as much else of the domain as possible was to be classified as negative. The other network (G) was trained to find the most *general* concept consistent with the data, that is, to classify as much of the domain as positive as it could while accommodating the negative training examples. The purpose of these biases was to decide whether a potential training example was interesting. If the two networks disagreed on its classification, then then it lay in the architecture's version space, and should be queried/added. These, and other biases would suggest themselves as appropriate, though, for producing diverse committee members for voting on the classification/output of a network. For those interested in the details of the selective sampling algorithm, we have a paper which is to appear in Machine Learning. It is available by anonymous ftp to "psyche.mit.edu"; the paper is in "pub/cohn/selsampling.ps.Z". -David Cohn e-mail: cohn at psyche.mit.edu Dept. of Brain & Cognitive Science phone: (617) 253-8409 MIT, E10-243 Cambridge, MA 02139 From dhw at santafe.edu Mon Jul 26 17:51:04 1993 From: dhw at santafe.edu (dhw@santafe.edu) Date: Mon, 26 Jul 93 15:51:04 MDT Subject: No subject Message-ID: <9307262151.AA02456@zia> Tom Dietterich writes: >>> This analysis predicts that using a committee of very diverse algorithms (i.e., having diverse approximation errors) would yield better performance (as long as the committee members are competent) than a committee made up of a single algorithm applied multiple times under slightly varying conditions. >>> There is a good deal of heuristic and empirical evidence supporting this claim. In general, when using stacking to combine generalizers, one wants them to be as "orthogonal" as possible, as Tom maintains. Indeed, one might even want to choose constituent generalizers which behave poorly stand-alone, just so that they are sufficiently different from one another when one combines them. (This is somewhat similar to what's going on w/ error-correcting codes, if one considers each (output code bit)-learning algorithm to be a different generalizer, trying to correctly classify things.) In fact, consider the situation where one uses very different generalizers, and their errors are highly correlated on a particular data set, so that *as far as the data is concerned* those generalizers are identical. For such situations, stacking (or any other kind of combining) can not help - all the guesses will be the same. However in such a situation you have good reason to suspect that you are data-limited - there is simply nothing more to be milked from the data. (An example of this phenomenon occurs with the intron-exon prediction problem; for some data sets, ID3, backprop, and a simple memory-based algorithm don't only get the same error rate; they have highly correlated errors, making mistakes in the same situations.) >>> In the error-correcting code work, we compared a committee of decision trees to an error-correcting output procedure that also used decision trees. The members of the committee were generated by training on different subsamples of the data (as in stacking), but the combination method was simple voting. No matter how many trees we added to the committee, we could not come close to achieving the same performance on the nettalk task as with the error-correcting output coding procedure. >>> Well, if I understand this correctly, Tom's using simple voting to combine, w/o any regard to behavior on the validation set. This will rarely be the best way of doing things. It amounts to training many times on subsets of the training data and then voting, rather than training once on the whole data; as such, this scheme might even result in worse generalization generically (whether it improves or helps probably depends on the generalizer's learning curve for the problem in question). Moreover, if (as in this case) one is playing w/ algorithms which are essentially identical (the members of the "committee"), then one might as well go whole-hog, and use the formulation of stacking designed to improve a single algorithm. (In this formulation, one uses partitions of the training set to try to find correlations between the *signed* error of the learning algorithm, and factors like its guess, the input value, distance to training set, etc.). Alternatively, as Tom well points out, you could modify the algorithms to make them substantially different from one another. Of course, one can also have fun by combining error-correcting-code algorithms based on different generalizers (say by stacking). *** Tom also writes: >>> There are three basic components that determine generalization error: * inherent error in the data (which determines the bayes optimal error rate) * small sample size * approximation error (which prevents the algorithm from correctly expressing the bayes optimal hypothesis, even with infinite samples) >>> One has to be a bit careful here; there's a lot of other stuff going on besides these three "components". Since Tom seems to have a Bayesian context in mind, it's worth analyzing things a bit from that perspective. In general, Pr(generalization error E | data D) = a (non-Euclidean) inner product between Pr(hypothesis h | data D) (i.e., one's learning algorithm) and Pr("truth" f | data D)) (i.e., the posterior distribution). Tom's component 3, "approximation error", seems to refer to having a poor alignment (so to speak) between Pr(h | D) and Pr(f | D); he seems to have in mind a scenario in which the learning algorithm is not optimally designed for the posterior. The first two components seem to instead refer to "lack of sharpness" (over f) of Pr(f | D). More precisely, they seem to refer to having the likelihood, Pr(D | f), broad, when viewed as a function of f. If this is indeed the context Tom has in mind, there is another factor to consider as well: Since the posterior is proportional to the likelihood times the prior, one also has to consider "lack of sharpness" in Pr(f), and more generally how aligned the prior is with the likelihood. In other words, there are situations in which one has no "approximation error", no error in the data, and a large sample, but there is still a large chance of large generalization error. This occurs for example if the prior f-peak is broad, but far removed from the likelihood f-peak. (Generically such situations are unlikely - that's the essential logic behind confidence intervals, VC stuff, etc. - but there's no way to assure that one isn't in such a situation in a given learning problem. And if you're running a program in which you search for something, like low error on the training set, then most bets - even conventinal VC bets, despite dogma to the contrary - are off. This is because conventional VC results refer to the distribution Pr(|gen. error E - training set error S| > bound | training set size m), which is NOT the same as Pr(|E - S| > bound | S, m); to calculate Pr(|E - S| > bound | S, m) you need to know something about Pr(f). Indeed, it's trivial to construct examples in which one can find S = 0, but whenever that occurs, one knows, w/ 100% certainty, that E is near maximal.) Moreover, if one instead defines "generalization error" to be off-training set error, then in fact you'll *always* have large error, if Pr(f) is uniform over f. (This is proven in a number of different guises in the papers referenced below. Intuitively, it holds because if all f are equally likely, any one off-training set behavior of f is as likely as any other, and the training set tells you nothing.) This result is completely independent of what learning algorithm you use, what VC analysis says, and the like, and well-exemplifies the importance of the prior Pr(f). David Wolpert Ref.'s [1] Wolpert, D. On the connection between in-sample testing and generalization error. Complex Systems, vol. 6, 47-94. (1992). [2] Wolpert, D. On overfitting-avoidance as bias. Not yet published (but placed in the neuroprose archive a couple of months ago). A related article, "Overfitting avoidance as bias", was published by C. Schaffer in the February Machine Learning. From watrous at learning.siemens.com Tue Jul 27 08:24:25 1993 From: watrous at learning.siemens.com (Raymond L Watrous) Date: Tue, 27 Jul 93 08:24:25 EDT Subject: Extended Registration Deadline for NNSP'93 Message-ID: <9307271224.AA05615@tiercel.siemens.com> In view of an unexpected delay in the mailing of the advance program and in the recent posting of the announcement to the Connectionists mailing list, the deadline for early registration for the 1993 IEEE Workshop on Neural Networks for Signal Processing has been extended to August 14. Raymond Watrous, Financial Chair 1993 IEEE Workshop on Neural Networks for Signal Processing c/o Siemens Corporate Research 755 College Road East Princeton, NJ 08540 (609) 734-6596 (609) 734-6565 (FAX) From hamps at shannon.ECE.CMU.EDU Tue Jul 27 08:40:24 1993 From: hamps at shannon.ECE.CMU.EDU (John B. Hampshire II) Date: Tue, 27 Jul 93 08:40:24 EDT Subject: committees are bad Message-ID: <9307271240.AA06235@ shannon.ece.cmu.edu.ECE.CMU.EDU > Belief in committees is paradoxically based on the notion that each member of the committee is a biased estimator of the Bayes-optimal classifier --- I stress that I am restricting my comments to pattern classification; I'm not commenting on function approximation (e.g., regression). Regression (i.e., estimating probabilities) and classification are not the same thing. The idea behind committees is that the average of a bunch of biased estimators will constitute an unbiased estimator. This is a provably *bad* idea, absent a proof that the biases all cancel (I'll bet there is no such proof in any of the committee work). Nevertheless, committees are obviously popular because the classifiers we typically generate in the connectionist community are provably biased --- even with regularization, pruning, and all the other complexity reduction tricks. Put in more organic terms, committees of humans often comprise large numbers of unremarkably average, biased individuals: their purpose is to achieve what one remarkable, unbiased individual could do alone. By virtue of their number, they generally involve huge development and maintenance overhead. This is a waste of resources. Compensating for one biased committee member with another one that has a different bias generally gives us a committee with lots of bias rather than one with no bias. The United States Congress is a perfect illustrative example: consider each member as a biased estimator of the ideal politician, and consider how effective the average of their efforts is... Barak certainly makes a valid point re. the initial parameterization issue, although it is also not that important if your model is minimum (or near minimum) complexity --- this gets into the issue of estimation variance, versus that of estimation bias. I'll take a single *provably* unbiased classifier over a committee of biased ones any day. Vapnik is right: excessive complexity is anathema. So are Geman, Bienenstock, and Doursat: connectionists face a "bias/variance dilemma". Fortunately, there is a relatively simple way to generate unbiased, low-complexity, minimum-variance classifiers. For those who care, I am prepared to defend this post with supporting proofs. However, I won't do it over connectionists in deference to those who don't care. -John From dhw at santafe.edu Tue Jul 27 10:52:47 1993 From: dhw at santafe.edu (David Wolpert) Date: Tue, 27 Jul 93 08:52:47 MDT Subject: The "best" way to do learning Message-ID: <9307271452.AA19037@sfi.santafe.edu> Harris Drucker writes: >>> The best method to generate a committee of learning machines is given by Schapire's algorithm [1]. >>> Schapire's boosting algorithm is a very interesting technique, which has now garnered some empirical support. It should be noted that it's really more a means of improving a single learning machine than a means of combining separate ones. More to the point though: There is no such thing as an a priori "best method" to do *anything* in machine learning. Anyone who thinks otherwise is highly encouraged to read Cullen Schaffer's Machine Learning article from Feb. '93. *At most*, one can say that a method is "best" *given some assumptions*. This is made explicit in Bayesian analysis. To my knowledge, boosting has only been analyzed (and found in a certain sense "best") from the perspective of PAC, VC stuff, etc. Now those formalisms can lend many insights into the learning process. But not if one isn't aware of their (highly non-trivial) implicit assumptions. Unfortunately, one of more problematic aspects of those formalisms is that that they encourage people to gloss over those implicit assumptions, and make blanket statements about "optimal" algorithms. From bisant at samish.stanford.edu Tue Jul 27 14:37:18 1993 From: bisant at samish.stanford.edu (bisant@samish.stanford.edu) Date: Tue, 27 Jul 93 11:37:18 PDT Subject: combining generalizers guesses Message-ID: <9307271837.AA26492@samish.stanford.edu> >It seems to me that more attention needs to be paid to *which* >generalizer's guesses we are combining. There are three basic >components that determine generalization error: > > * inherent error in the data (which determines the bayes optimal error rate) > * small sample size > * approximation error (which prevents the algorithm from correctly > expressing the bayes optimal hypothesis, even with infinite samples) I also think there is another source of error in addition to those given above which can be removed by combining generalizers. This source is: * the lack of confidence in the prediction. Most neural networks and other classifiers produce a continuous output. Usually, during classification, a threshold or winner take all method is used to decide the classification. If you imagine a network which classifies inputs into one of 3 outputs and you see some classifications which appear as follows: a. 0.27 0.21 0.86 b. 0.41 0.48 0.53 it is obvious the third class is the winner, but it is also obvious classification "a" has much more confidence than "b". Whichever arbitration mechanism is used to combine the generalizers should take this information into account. > So, it seems to me the key question is what are the best ways of > creating a diverse "committee"? Most researchers who work in applying neural networks use a committee approach for the final decision. Some empirical research has been done over the last 4 years to find the best way. Waibel and Hampshire have presented some work in NIPS, IEEE, and IJCNN 3 years ago where they used different objective functions to create very diverse networks. I believe they used the following objective functions: 1 squared error 2 classification figure of merit (CFM) 3 cross entropy. The networks produced, especially by the CFM, were very different. As an arbitration mechanism, they found that a simple average worked better than other more complicated methods including a neural network. All the arbitration mechanisms they tried were able to take the confidence factor, mentioned above, into account. David Bisant Stanford PDP Group @article{ hampshire2, author="Hampshire II, J. B. and Waibel, A. H.", title="{A Novel Objective Function for Improved Phoneme Recognition Using Time-Delay Neural Networks}", journal="IEEE Transactions on Neural Networks", volume="1", number="2", year="1990", pages="216-228"} From ingber at alumni.cco.caltech.edu Tue Jul 27 20:02:10 1993 From: ingber at alumni.cco.caltech.edu (Lester Ingber) Date: Tue, 27 Jul 1993 17:02:10 -0700 Subject: Summary of contributed paper to Frontier Science in EEG Symposium Message-ID: <9307280002.AA00529@alumni.cco.caltech.edu> Summary of contributed paper to Frontier Science in EEG Symposium A PostScript summary of a presentation, Multiple Scales of EEG, to be made at Frontier Science in EEG Symposium: Proceedings, New Orleans, 9 Oct 1993, is available via anonymous ftp, as eeg_proc93.ps.Z. The announcement and registration forms for this meeting are in the file eeg_announce_proc93.Z. The Introduction describes this meeting: Electroencephalography (EEG) is the study of the electrical activity of the brain. The field of EEG includes the technology to record these electrical signals, the science to analyze them and the expertise to apply them to patient care. This symposium will explore the scientific frontiers related to EEG, presenting the latest research and thought with this year's topic being continuous waveform analysis. As advances in science and technology often involve collaboration among scientists from different fields, we are bringing together a diverse group of investigators, many from areas not conventionally associated with EEG, to actively encourage multidisciplinary research in EEG and foster new ideas. A set of plates from which this talk will be developed is the file eeg_talk.ps.Z; this file is only 558K, but due to the inclusion of scanned figures, it will expand to over 8M when uncompressing. A paper giving more technical background is eeg92.ps.Z. Also in this directory is the latest Adaptive Simulated Annealing (ASA) code, version 1.37. The INDEX file gives a bibliographic reference for the files in this directory. Instructions for retrieval: ftp ftp.caltech.edu [Name:] anonymous [Password:] your_email_address cd pub/ingber binary get file_of_interest quit If you do not have ftp access, get information on the FTPmail service by: mail ftpmail at decwrl.dec.com, and send only the word "help" in the body of the message. If any of the above are not convenient, and if your mailer can handle large files (please test this first), the code or papers you require can be sent as uuencoded compressed files via electronic mail. If you have gzip, resulting in smaller files, please state this. Sorry, but I cannot assume the task of mailing out hardcopies of code or papers. Lester || Prof. Lester Ingber 1-800-L-INGBER || || Lester Ingber Research Fax: [10ATT]0-700-L-INGBER || || P.O. Box 857 EMail: ingber at alumni.caltech.edu || || McLean, VA 22101 Archive: ftp.caltech.edu:/pub/ingber || From kolen-j at cis.ohio-state.edu Tue Jul 27 11:24:38 1993 From: kolen-j at cis.ohio-state.edu (john kolen) Date: Tue, 27 Jul 93 11:24:38 -0400 Subject: combining generalizers' guesses In-Reply-To: Barak Pearlmutter's message of Mon, 26 Jul 93 13:27:39 EDT <9307261727.AA22861@gull.siemens.com> Message-ID: <9307271524.AA05339@pons.cis.ohio-state.edu> >Barak Pearlmutter <bap at learning.siemens.com> >For instance, if you run backpropagation on the same data twice, with >the same architecture and all the other parameters held the same, it >will still typically come up with different answers. Eg due to >differences in the random initial weights. [Blatant self promotion follows, but I looking for a job so I need all the promotion I can get ;-] A very vivid example of this can found in John. F. Kolen and Jordan. B. Pollack, 1990. Backpropagation is Sensitive to Initial Conditions. _Complex Systems_. 4:3. pg 269-280. Available from neuroprose: kolen.bpsic.* or John. F. Kolen and Jordan. B. Pollack, 1991. Backpropagation is Sensitive to Initial Conditions. NIPS 3. pg 860-867. > Averaging out this effect is a guaranteed win. > --Barak. This statement seems rely on an underlying assumption of convexity, which does not necessarily hold when you try to combine different processing strategies (ie. networks). If you massage your output representations so that linear combinations will always give you reasonable results, that's great. But it's not always the case that you have the leeway to make such a representational commitment. John From drucker at monmouth.edu Wed Jul 28 17:14:17 1993 From: drucker at monmouth.edu (Drucker Harris) Date: Wed, 28 Jul 93 17:14:17 EDT Subject: No subject Message-ID: <9307282114.AA00408@harris.monmouth.edu> About committees and boosting: My previous communication used the word "best". That was a little puffery on my part and a reasonable self-imposed limitation of the use of this medium. For an explicit list of limitations, assumptions, etc as to when and where boosting applies send me your e-mail address and I will send you a troff file of a preprint of an article. I can also send hard copies if there are not too many requests. More to the point: if you are interested in classification and want to improve performance, boosting is a reasonable approach, Instead of struggling to build different classifiers and then figuring out the best way to combine them, boosting by filtering explicitly shows us how you filter the data so that each machine learns a different distribution of the training set. In our work in OCR using multilayer networks (single layer networks are not powerful enough) boosting has ALWAYS improved performance. Synthetically enlarging the database using deformations of the original data is essential. In one case, a network (circa 1990) which had an error rate on United State Postal Service digits of 4.9% and a reject rate of 11.5% (in order to achieve a 1% error rate on those not rejected) was boosted to give a 3.6% error rate and a 6.6% reject rate. Someone then invented a new network that had a 3.3% error rate and a 7.7% reject rate and this was boosted to give a 2.6% error rate and 4.0% reject rate. This is very close to the estimated human performance of 2.5%. Can someone find a better single network (using the original database) that is better than a boosted committee. Maybe. But good networks are hard to find and if you can find it, you can probably boost it. Can one improve performance by using the synthetically enlarged database and a "larger" single machine. Yes, but we have yet to find a single network that does better that a boosted committee. A final note: rather than straight voting, we have found that simply summing the respective outputs of the three neural networks gives MUCH better results (as quoted above). As pointed out by David Bisant, voting does not explicitly include the confidence. In neural networks, a measure of confidence is the difference between the two largest outputs. By simply voting, you ignore the fact that one of the members of the committee may be very confident about its results. By adding, networks with high confidence influence the results more and lower both the error rate and especially the reject rate.. Harris Drucker Bell Labs phone: 908-949-4860 Monmouth College phone: 908-571-3698 email: drucker at monmouth.edu (preferred) From t-chan at rsc.u-aizu.ac.jp Thu Jul 29 12:27:15 1993 From: t-chan at rsc.u-aizu.ac.jp (Tony Y. T. Chan) Date: Thu, 29 Jul 93 12:27:15 JST Subject: The "best" way to do learning Message-ID: <9307290327.AA26368@profsv.rsc.u-aizu.ac.jp> It is obvious that as David Wolpert said, ``There is no such thing as an a priori "best method" to do *anything* in machine learning.'' But I would like to raise the following question: Is there such a thing as the best method to learn a random idea? More precisely, given n random (machine) learning problems, is there such a thing as the best method for dealing with these problems that will give the best overall performance. There may be some best methods for dealing with some specific types of learning problems but is there one that would deal with any learning problem and give the best overall performance? Tony Chan From mpp at cns.brown.edu Thu Jul 29 02:43:58 1993 From: mpp at cns.brown.edu (Michael P. Perrone) Date: Thu, 29 Jul 93 02:43:58 EDT Subject: Multiple Models, Committee of nets etc... Message-ID: <9307290643.AA18402@cns.brown.edu> For those interested in the recent discussion of Multiple Models, Committees, etc., the following references may be of interest. The first three references deal exactly with the issues that have recently been discussed on Connectionists. The salient contributions from these papers are: 1) A very general result which proves that averaging ALWAYS improves optimization performance for a broad class of (convex) optimization problems including MSE, MLE, Maximum Entropy, Maximum Mutual Information, Splines, HMMs, etc. This is a result about the topology of the optimization measure and is independent of the underlying data distribution, learning algorithm or network architecture. 2) A closed form solution to the optimal weighted average of a set of regression estimates (Here, I regard density estimation and classification as special cases of regression) for a given cross-validation set and MSE optimization. It should be noted that the solution may suffer from over-fitting when the CV set is not representative of the true underlying distribution. However the solution is amenable to ridge regression and a wide variety of heuristic robustification techniques. 3) Experiments on real-world datasets (NIST OCR data, human face data and timeseries data) which demonstrate the improvement due to averaging. The improvement is so dramatic that in most cases the average estimator performs significantly better than the best individual estimator. (It is important to note that the CV performance of a network is not a guaranteed predictor for performance on an independent test set. So a network which has the best performance on the CV set may not have the best performance on the test set; however in practice, even when the CV performance is a good predictor for test set performance, the average estimator usually performs better.) 4) Numerous extensions including bootstrapped and jackknifed neural net generation; and averaging over "hyperparameters" such as architectures, priors and/or regularizers. 5) An interpretation of averaging in the case of MSE optimization, as a regularizer which performs smoothing by variance reduction. This implies that averaging is having no effect on the bias of the estimators. In fact, for a given population of estimators, the bias of the average estimator will be the same as the expected bias of any estimator in the population. 6) A very natural definition of the number of "distinct" estimators in a population which emphasizes two points: (a) Local minima are not necessarily a bad thing! We can actually USE LOCAL MINIMA TO IMPROVE PERFORMANCE; and (b) There is an important distinction between the number of local minima in parameter space and the number of local minima in function space. Function space is what we are really concerned with and empirically, averaging suggests that there are not that many "distinct" local minima in trained populations. Therefore one direction for the future is to devise ways of generating as many "distinct" estimators as possible. The other three references deal with what I consider to be the flip side of the same coin: On one side is the problem of combining networks, on the other is the the problem of generating networks. These three references explore neural net motivated divide and conquer heuristics within the CART framework. Enjoy! Michael -------------------------------------------------------------------------------- Michael P. Perrone Email: mpp at cns.brown.edu Institute for Brain and Neural Systems Tel: 401-863-3920 Brown University Fax: 401-863-3934 Providence, RI 02912 -------------------------------------------------------------------------------- @phdthesis{Perrone93, AUTHOR = {Michael P. Perrone}, TITLE = {Improving Regression Estimation: Averaging Methods for Variance Reduction with Extensions to General Convex Measure Optimization}, YEAR = {1993}, SCHOOL = {Brown University, Institute for Brain and Neural Systems; Dr. Leon N Cooper, Thesis Supervisor}, MONTH = {May} } @inproceedings{PerroneCooper93CAIP, AUTHOR = {Michael P. Perrone and Leon N Cooper}, TITLE = {When Networks Disagree: Ensemble Method for Neural Networks}, BOOKTITLE = {Neural Networks for Speech and Image processing}, YEAR = {1993}, PUBLISHER = {Chapman-Hall}, EDITOR = {R. J. Mammone}, NOTE = {[To Appear]}, where = {London} } @inproceedings{PerroneCooper93WCNN, AUTHOR = {Michael P. Perrone and Leon N Cooper}, TITLE = {Learning from What's Been Learned: Supervised Learning in Multi-Neural Network Systems}, BOOKTITLE = {Proceedings of the World Conference on Neural Networks}, YEAR = {1993}, PUBLISHER = {INNS} } --------------------- @inproceedings{Perrone91, AUTHOR = {M. P. Perrone}, TITLE = {A Novel Recursive Partitioning Criterion}, BOOKTITLE = {Proceedings of the International Joint Conference on Neural Networks}, YEAR = {1991}, PUBLISHER = {IEEE}, PAGES = {989}, volume = {II} } @inproceedings{Perrone92, AUTHOR = {M. P. Perrone}, TITLE = {A Soft-Competitive Splitting Rule for Adaptive Tree-Structured Neural Networks}, BOOKTITLE = {Proceedings of the International Joint Conference on Neural Networks}, YEAR = {1992}, PUBLISHER = {IEEE}, PAGES = {689-693}, volume = {IV} } @inproceedings{PerroneIntrator92, AUTHOR = {M. P. Perrone and N. Intrator}, TITLE = {Unsupervised Splitting Rules for Neural Tree Classifiers}, BOOKTITLE = {Proceedings of the International Joint Conference on Neural Networks}, YEAR = {1992}, ORGANIZATION = {IEEE}, PAGES = {820-825}, volume = {III} } From mpp at cns.brown.edu Thu Jul 29 03:27:21 1993 From: mpp at cns.brown.edu (Michael P. Perrone) Date: Thu, 29 Jul 93 03:27:21 EDT Subject: Multiple Models, Committee of nets etc... Message-ID: <9307290727.AA19084@cns.brown.edu> Tom Dietterich write: > This analysis predicts that using a committee of very diverse > algorithms (i.e., having diverse approximation errors) would yield > better performance (as long as the committee members are competent) > than a committee made up of a single algorithm applied multiple times > under slightly varying conditions. and David Wolpert writes: >There is a good deal of heuristic and empirical evidence supporting >this claim. In general, when using stacking to combine generalizers, >one wants them to be as "orthogonal" as possible, as Tom maintains. One minor result from my thesis shows that when the estimators are orthogonal in the sense that E[n_i(x)n_j(x)] = 0 for all i<>j where n_i(x) = f(x) - f_i(x), f(x) is the target function, f_i(x) is the i-th estimator and the expected value is over the underlying distribution; then the MSE of the average estimator goes like 1/N times the average of the MSE of the estimators where N is the number of estimators in the population. This is a shocking result because all we have to do to get arbitrarily good performance is to increase the size of our estimator population! Of course in practice, the nets are correlated and the result is no longer true. Michael -------------------------------------------------------------------------------- Michael P. Perrone Email: mpp at cns.brown.edu Institute for Brain and Neural Systems Tel: 401-863-3920 Brown University Fax: 401-863-3934 Providence, RI 02912 From mpp at cns.brown.edu Thu Jul 29 03:45:48 1993 From: mpp at cns.brown.edu (Michael P. Perrone) Date: Thu, 29 Jul 93 03:45:48 EDT Subject: Multiple Models, Committee of nets etc... Message-ID: <9307290745.AA19374@cns.brown.edu> INNS SIG on Hybrid Neural Systems --------------------------------- The INNS has recently started a special interest group for hybrid neural systems which provides another forum for people interested in methods for combining networks and algorithms for improved performance. If you are interested in joining and receiving a membership list, please send email to me or Larry Medsker. me ----> mpp at cns.brown.edu Larry -> medsker at american.edu Thanks, Michael -------------------------------------------------------------------------------- Michael P. Perrone Email: mpp at cns.brown.edu Institute for Brain and Neural Systems Tel: 401-863-3920 Brown University Fax: 401-863-3934 Providence, RI 02912 From farrar at cogsci.UCSD.EDU Thu Jul 29 16:21:34 1993 From: farrar at cogsci.UCSD.EDU (Scott Farrar) Date: Thu, 29 Jul 93 13:21:34 PDT Subject: committees Message-ID: <9307292021.AA09768@cogsci.UCSD.EDU> John Hampshire characterized a committee as a collection of biased estimators; the idea being that a collection of many different kinds of bias might constitute a unbiased estimator. I was wondering if anyone had any ideas about how this might be related to, supported by, or refuted by the Central Limit Theorem. Could experimental variances or confounds be likened to "biases", and if so, do these "average out" in a manner which can give us a useful mean or useful estimator? --Scott Farrar From mpp at cns.brown.edu Thu Jul 29 16:20:28 1993 From: mpp at cns.brown.edu (Michael P. Perrone) Date: Thu, 29 Jul 93 16:20:28 EDT Subject: Harris Drucker comments Message-ID: <9307292020.AA22330@cns.brown.edu> Harris Drucker writes: > In our work in OCR > using multilayer networks (single layer networks are not powerful enough) > boosting has ALWAYS improved performance. This is a direct result of averaging [1]. > Can someone find a better single network (using the original database) that is > better than a boosted committee. Maybe. But good networks are hard to find and > if you can find it, you can probably boost it. This is the important take-home message for all of these averaging techniques: If you can generate a good estimator, you can ALWAYS improve it using averaging. Of course, you will eventually reach the point of diminishing returns on your resource investment (e.g. averaging several different sets of averaged estimators and then averaging the average of the averages ad infinitum). > A final note: rather than straight voting, we have found that simply summing > the respective outputs of the three neural networks gives MUCH better results > (as quoted above). This result is due to the fact that averaging the outputs is guaranteed to improve performance for MSE whereas averaging the Winner Take All output (i.e. voting) corresponds to a different optimization measure and there is no guarantee that averaging in one topology will improve the performance in the other [2]. [1] Michael P. Perrone and Leon N Cooper, When Networks Disagree: Ensemble Method for Neural Networks, In _Neural Networks for Speech and Image Processing_, R. J. Mammone (ed.), Chapman-Hall, London: 1993). [2] Michael P. Perrone and Leon N Cooper, Learning from what's been learned: Supervised learning in multi-neural network systems, Proceedings of the World Conference on Neural Networks 1993, INNS. Michael -------------------------------------------------------------------------------- Michael P. Perrone Email: mpp at cns.brown.edu Institute for Brain and Neural Systems Tel: 401-863-3920 Brown University Fax: 401-863-3934 Providence, RI 02912 From mpp at cns.brown.edu Thu Jul 29 16:50:53 1993 From: mpp at cns.brown.edu (Michael P. Perrone) Date: Thu, 29 Jul 93 16:50:53 EDT Subject: combining generalizers' guesses Message-ID: <9307292050.AA22464@cns.brown.edu> Barak Pearlmutter writes: >For instance, if you run backpropagation on the same data twice, with >the same architecture and all the other parameters held the same, it >will still typically come up with different answers. Eg due to >differences in the random initial weights. .. > Averaging out this effect is a guaranteed win. > --Barak. I agree. I think that the surprising issue here is that the local minima that people have been trying like crazy to avoid for the passed few years can actually be used to improve performance! I think that one direction to take is be to stop trying to find the global optimum and instead try to find "complementary" or "orthogonal" local optima. Reilly's multi-resolution architectures [1], Schapire's Boosting algorithm [2] and Brieman's Stacked Regression [3] are good examples. Of course there are many other approaches that one could take some of which are proposed in my PhD thesis. I think that there is a lot of work to be done in this area. I'd be glad to hear from people experimenting with related algorithms or who are interested in discussing more details. Michael -------------------------------------------------------------------------------- Michael P. Perrone Email: mpp at cns.brown.edu Institute for Brain and Neural Systems Tel: 401-863-3920 Brown University Fax: 401-863-3934 Providence, RI 02912 [1] @incollection{ReillyEtAl87, AUTHOR = {R. L. Reilly and C. L. Scofield and C. Elbaum and L. N Cooper}, TITLE = {Learning System Architectures Composed of Multiple Learning Modules}, BOOKTITLE = {Proc. IEEE First Int. Conf. on Neural Networks}, YEAR = {1987}, PUBLISHER = {IEEE}, PAGES = {495-503}, volume = 2 } [2] @article{Schapire90, AUTHOR = {R. Schapire}, TITLE = {The strength of weak learnability}, JOURNAL = {Machine Learning}, YEAR = {1990}, NUMBER = {2}, PAGES = {197-227}, VOLUME = {5} } [3] @techreport{Breiman92, AUTHOR = {Leo Breiman}, TITLE = {Stacked regression}, YEAR = {1992}, INSTITUTION = {Department of Statistics, University of California, Berkeley}, MONTH = {August}, NUMBER = {{TR}-367}, TYPE = {Technical Report} } From wolf at planck.lanl.gov Thu Jul 29 20:01:54 1993 From: wolf at planck.lanl.gov (David R Wolf) Date: Thu, 29 Jul 93 18:01:54 -0600 Subject: Two papers available Message-ID: <9307300001.AA00296@planck.lanl.gov> FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/wolpert.entropy.tar.Z FTP-filename: /pub/neuroprose/wolf.mutual1.ps.Z FTP-filename: /pub/neuroprose/wolf.mutual1.ps.Z The following tech reports are now available in neuroprose. The papers have been submitted, and any comments are welcomed. david ======================================================= D.H.Wolpert and D.R. Wolf: Estimating Functions of Probability Distributions from a Finite Set of Samples, Part 1: Bayes Estimators and the Shannon Entropy. D.R. Wolf and D.H. Wolpert Estimating Functions of Distributions from A Finite Set of Samples, Part 2: Bayes Estimators for Mutual Information, Chi-Squared, Covariance and other Statistics. We present estimators for entropy and other functions of a discrete probability distribution when the data is a finite sample drawn from that probability distribution. In particular, for the case when the probability distribution is a joint distribution, we present finite sample estimators for the mutual information, covariance, and chi-squared functions of that probability distribution. ======================================================= Retrieval instructions: The papers are found in the neuroprose archive under wolpert.entropy.tar.Z 21 pages text, 6 figures, captions wolf.mutual1.ps.Z 25 pages wolf.mutual2.ps.Z 25 pages The INDEX entries are wolpert.entropy.tar.Z Small sample estimator for entropy. wolf.mutual1.ps.Z wolf.mutual2.ps.Z Small sample estimators for mutual information and other functions. To retrieve these files from the neuroprose archives: For simplicity, make a directory <newdirname> on your system, then unix> ftp cheops.cis.ohio-state.edu Name (cheops.cis.ohio-state.edu:becker): anonymous Password: (use your email address) ftp> cd pub/neuroprose ftp> binary ftp> lcd <newdirname> Local directory now <newdirname> ftp> get wolpert.entropy.tar.Z 200 PORT command successful. 150 Opening BINARY mode data connection for wolpert.entropy.tar.Z From mpp at cns.brown.edu Fri Jul 30 21:29:27 1993 From: mpp at cns.brown.edu (Michael P. Perrone) Date: Fri, 30 Jul 93 21:29:27 EDT Subject: Preprint available Message-ID: <9307310129.AA05190@cns.brown.edu> FTP-host: archive.cis.ohio-state.edu FTP-filename: perrone.MSE-averaging.ps.Z The following paper is now available in neuroprose. It was presented at the 1992 CAIP Conference at Rutger University. It will appear in Neural Networks for Speech and Image processing, R. J. Mammone (ed.), Chapman-Hall, 1993. The paper is relevant to the recent discussion on Connectionists about multiple neural network estimators. Enjoy! Michael -------------------------------------------------------------------------------- Michael P. Perrone Email: mpp at cns.brown.edu Institute for Brain and Neural Systems Tel: 401-863-3920 Brown University Fax: 401-863-3934 Providence, RI 02912 -------------------------------------------------------------------------------- When Networks Disagree: Ensemble Method for Neural Networks M. P. Perrone and L. N Cooper Abstract: This paper presents a general theoretical framework for ensemble methods of constructing significantly improved regression estimates. Given a population of regression estimators, we construct a hybrid estimator which is as good as or better than, in the MSE sense, any estimator in the population. We argue that the ensemble method presented has several properties: 1) It efficiently uses all the networks of a population - none of the networks need be discarded. 2) It efficiently uses all the available data for training without over- fitting. 3) It inherently performs regularization by smoothing in functional space which helps to avoid over-fitting. 4) It utilizes local minima to construct improved estimates whereas other neural network algorithms are hindered by local minima. 5) It is ideally suited for parallel computation. 6) It leads to a very useful and natural measure of the number of distinct estimators in a population. 7) The optimal parameters of the ensemble estimator are given in closed form. Experimental results are provided which show that the ensemble method dramatically improves neural network performance on difficult real-world optical character recognition tasks. -------------------------------------------------------------------------------- Retrieval instructions: The paper is found in the neuroprose archive under perrone.MSE-averaging.ps.Z 15 pages To retrieve these files from the neuroprose archives: unix> ftp cheops.cis.ohio-state.edu Name (cheops.cis.ohio-state.edu:username): anonymous Password: (use your email address) ftp> cd pub/neuroprose ftp> binary ftp> get perrone.MSE-averaging.ps.Z ftp> bye  From hicks at cs.titech.ac.jp Fri Jul 30 15:49:15 1993 From: hicks at cs.titech.ac.jp (hicks@cs.titech.ac.jp) Date: Fri, 30 Jul 93 15:49:15 JST Subject: Synthetically enlarging the database In-Reply-To: Drucker Harris's message of Wed, 28 Jul 93 17:14:17 EDT <9307282114.AA00408@harris.monmouth.edu> Message-ID: <9307300649.AA09200@hilbert> About synthetically enlarging the database: Drucker Harris writes: >In our work in OCR using multilayer networks (single >layer networks are not powerful enough) boosting has >ALWAYS improved performance. Synthetically enlarging >the database using deformations of the original data is >essential. (Point of view from outside the learning system) It seems to me that the cost of obtaining training data is an issue implicit in the above statement, and ought to be made explicit. As the number of data in the original training set increases, the benefits of synthetically created data will become less. Moreover, wouldn't it be correct to say that one could always do better by using N extra randomly selected training data than by using N extra synthetically created data? Nevertheless, the cost of obtaining training data is a real factor and synthetically created training data may be virtually free. (Point of view from inside the system) But what about the cost of learning any training data, synthetic or otherwise? Synthesis of training data may be cheaper than obtaining real training data, but it still has to be learned. Is it possible to have synthesis without extra learning cost? Consider that synthetically creating data has the effect of compressing the size of the input space (and thus enforcing smoothing) in same way as would a preprocessing front giving translational invariance. In both cases a single input is given to the system and the system learns many samples, explicitly in the case of synthetic creation, implicitly in the case of translational invariance. The former incurrs extra learning cost, the latter none. I know this is not a good example, because translational invariance is a trivial problem, and the difficult problems do require more learning. Synthetically creating data is one way to go about smoothing the area around a (non-synthetic) training sample, but aren't there others? For example, adding a penalty term for the complexity of the output function (or some internal rep. if there is no continuous output function) around the sample point. Craig Hicks hicks at cs.titech.ac.jp Ogawa Laboratory, Dept. of Computer Science Tokyo Institute of Technology, Tokyo, Japan lab: 03-3726-1111 ext. 2190 home: 03-3785-1974 fax: +81(3)3729-0685 (from abroad), 03-3729-0685 (from Japan)  From lars at eiffel.ei.dth.dk Fri Jul 30 04:33:31 1993 From: lars at eiffel.ei.dth.dk (Lars Kai Hansen) Date: Fri, 30 Jul 93 09:33:31 +0100 Subject: committee's Message-ID: <9307300833.AA01270@eiffel.ei.dth.dk> It is great that attention is focussed on the effective use of solution space samples for non-linear models. Allow me to promote our pre-historic work on network voting: NEURAL NETWORK ENSEMBLES by L.K. Hansen and P. Salamon IEEE Trans. Pattern Analysis and Machine Intell. {\bf 12}, 993-1001, (1990) Besides finding experimentally that the ensemble consensus often is 'better than the best'.... expressions were derived for the ensemble error rate based on different assumptions on error correlations. The key invention is to describe the ensemble by the 'difficulty distribution'. This description was inspired by earlier work on so called 'N-version programming' by Eckhardt and Lee: A THEORETICAL BASIS FOR THE ANALYSIS OF MULTIVERSION SOFTWARE SUBJECT TO COINCIDENT ERRORS by D.E. Eckhardt and L.D. Lee IEEE Trans. Software Eng. {\bf 11} 1511-1517 (1985) In a feasibility study on Handwritten digits the viability of voting among small ensembles was confirmed (the consensus outperformed the best individual by 25%) and the theoretical estimate of ensemble performance was found to fit well to the observed. Further, the work of Schwartz et al. [Neural Computation {\bf 2}, 371-382 (1990)] was applied to estimate the learning curve based on the distribution of generalizations of a small ensemble: ENSEMBLE METHODS FOR HANDWRITTEN DIGIT RECOGNITION by L.K. Hansen, Chr. Liisberg, and P. Salamon In proceedings of The Second IEEE Workshop on Neural Networks for Signal Processing: NNSP'92 Eds. S.Y. Kung et al., IEEE Service Center Piscataway, 333-342, (1992) While I refer to these methods as *ensemble* methods (to emphasize the statistical relation and to invoke associations to artistic ensembles), I note that theorists have reserved *committee machines* for a special, constrained, network architecture (see eg. Schwarze and Hertz [Euro.Phys.Lett. {\bf 20}, 375-380, (1992)]). In the theorist committee (TC) all weights from hiddens to output are fixed to unity during training. This is very different from voting among independently trained networks: while the TC explores the function space of a large set of parameters (hence needs very many training examples), a voting system based on independently trained nets only explores the function space of the individual network. The voting system can improve generalization by reducing 'random' errors due to training algorithms etc. --------------------- Lars Kai Hansen, Tel: (+45) 4593 1222 (tone) 3889 CONNECT, Electronics Institute B349 Fax: (+45) 4288 0117 Technical University of Denmark email: lars at eiffel.ei.dth.dk DK-2800 Lyngby DENMARK  From skalsky at aaai.org Fri Jul 30 16:02:00 1993 From: skalsky at aaai.org (Rick Skalsky) Date: Fri, 30 Jul 93 13:02:00 PDT Subject: AAAI-94 Call for Papers Message-ID: <9307302002.AA16439@aaai.org> Twelfth National Conference on Artificial Intelligence (AAAI-94) Seattle, Washington July, 31-August 4, 1994 Call for Papers AAAI-94 is the twelfth national conference on artificial intelligence (AI). The purpose of the conference is to promote research in AI and scientific interchange among AI researchers and practitioners. Papers may represent significant contributions to any aspects of AI: a) principles underlying cognition, perception, and action; b) design, application, and evaluation of AI algorithms and systems; c) architectures and frameworks for classes of AI systems; and d) analysis of tasks and domains in which intelligent systems perform. One of the most important functions served by the national conference is to provide a forum for information exchange and interaction among researchers working in different sub- disciplines, in different research paradigms, and in different stages of research. Based on discussions among program committee members during the past few years, we aim to expand active participation in this year's conference to include a larger cross-section of the AI community and a larger cross-section of the community's research activities. Accordingly, we encourage submission of papers that: describe theoretical, empirical, or experimental results; represent areas of AI that may have been under-represented in recent conferences; present promising new research concepts, techniques, or perspectives; or discuss issues that cross traditional sub-disciplinary boundaries. As outlined below, we have revised and expanded the paper review criteria to recognize this broader spectrum of research contributions. We intend to accept more of the papers that are submitted and to publish them in an expanded conference proceedings. Requirements for Submission Authors must submit six (6) complete printed copies of their papers to the AAAI office by January 24, 1994. Papers received after that date will be returned unopened. Notification of receipt will be mailed to the first author (or designated author) soon after receipt. All inquiries regarding lost papers must be made by February 7, 1994. Authors should also send their paper's title page in an electronic mail message to abstract at aaai.org by January 24, 1994. Notification of acceptance or rejection of submitted papers will be mailed to the first author (or designated author) by March 11, 1994. Camera-ready copy of accepted papers will be due about one month later. Paper Format for Review All six (6) copies of a submitted paper must be clearly legible. Neither computer files nor fax submissions are acceptable. Submissions must be printed on 8 1/2" x 11" or A4 paper using 12 point type (10 characters per inch for typewriters). Each page must have a maximum of 38 lines and an average of 75 characters per line (corresponding to the LaTeX article-style, 12 point). Double-sided printing is strongly encouraged. Length The body of submitted papers must be at most 12 pages, including title, abstract, figures, tables, and diagrams, but excluding the title page and bibliography. Papers exceeding the specified length and formatting requirements are subject to rejection without review. Blind Review Reviewing for AAAI-94 will be blind to the identities of the authors. This requires that authors exercise some care not to identify themselves in their papers. Each copy of the paper must have a title page, separate from the body of the paper, including the title of the paper, the names and addresses of all authors, a list of content areas (see below) and any acknowledgements. The second page should include the exact same title, a short abstract of less than 200 words, and the exact same content areas, but not the names nor affiliations of the authors. The references should include all published literature relevant to the paper, including previous works of the authors, but should not include unpublished works of the authors. When referring to one's own work, use the third person, rather than the first person. For example, say "Previously, Korf [17] has shown that...", rather than "In our previous work [17] we have shown that...". Try to avoid including any information in the body of the paper or references that would identify the authors or their institutions. Such information can be added to the final camera-ready version for publication. Please do not staple the title page to the body of the paper. Electronic Title Page A title page should also be sent via electronic mail to abstract at aaai.org, in plain ASCII text, without any formatting commands for LaTeX, Scribe, etc. Each section of the electronic title page should be preceded by the name of that section as follows: title: <title> author: <name of first author> address: <address of first author> author: <name of last author> address: <address of last author> abstract: <abstract> content areas: <first area>, ..., <last area> To facilitate the reviewing process, authors are requested to select 1-3 appropriate content areas from the list below. Authors are welcome to add additional content area descriptors as needed. AI architectures, artificial life, automated reasoning, control, belief revision, case-based reasoning, cognitive modeling, common sense reasoning, computational complexity, computer-aided education, constraint satisfaction, decision theory, design, diagnosis, distributed AI, expert systems, game playing, genetic algorithms, geometric reasoning, knowledge acquisition, knowledge representation, machine learning, machine translation, mathematical foundations, multimedia, natural language processing, neural networks, nonmonotonic reasoning, perception, philosophical foundations, planning, probabilistic reasoning, problem solving, qualitative reasoning, real-time systems, robotics, scheduling, scientific discovery, search, simulation, speech understanding, temporal reasoning, theorem proving, user interfaces, virtual reality, vision Submissions to Multiple Conferences Papers that are being submitted to other conferences, whether verbatim or in essence, must reflect this fact on the title page. If a paper appears at another conference (with the exception of specialized workshops), it must be withdrawn from AAAI-94. Papers that violate these requirements are subject to rejection without review. Review Process Program committee (PC) members will identify papers they are qualified to review based on each paper's title, content areas, and electronic abstract. This information, along with other considerations, will be used to assign each submitted paper to two PC members. Using the criteria given below, they will review the paper independently. If the two reviewers of a paper agree to accept or reject it, that recommendation will be followed. If they do not agree, a third reviewer will be assigned and the paper will be discussed by an appropriate sub-group of the PC during its meeting in March. Note that the entire review process will be blind to the identities of the authors and their institutions. In general, papers will be accepted if they receive at least two positive reviews or if they generate an interesting controversy among the reviewers. The final decisions on all papers will be made by the program chairs. Questions that will appear on the review form appear below. Authors are advised to bear these questions in mind while writing their papers. Reviewers will look for papers that meet at least some (though not necessarily all) of the criteria in each category. Significance How important is the problem studied? Does the approach offered advance the state of the art? Does the paper stimulate discussion of important issues or alternative points of view? Originality Are the problems and approaches new? Is this a novel combination of existing techniques? Does the paper point out differences from related research? Does it address a new problem or one that has not been studied in depth? Does it introduce an interesting research paradigm? Does the paper describe an innovative combination of AI techniques with techniques from other disciplines? Does it introduce an idea that appears promising or might stimulate others to develop promising alternatives? Quality Is the paper technically sound? Does it carefully evaluate the strengths and limitations of its contributions? Are its claims backed up? Does the paper offer a new form of evidence in support of or against a well-known technique? Does the paper back up a theoretical idea already in the literature with experimental evidence? Does it offer a theoretical analysis of prior experimental results? Clarity Is the paper clearly written? Does it motivate the research? Does it describe the inputs, outputs, and basic algorithms employed? Are the results described and evaluated? Is the paper organized in a logical fashion? Is the paper written in a manner that makes its content accessible to most AI researchers? Publication Accepted papers will be allocated six (6) pages in the conference proceedings. Up to two (2) additional pages may be used at a cost to the authors of $250 per page. Papers exceeding eight (8) pages and those violating the instructions to authors will not be included in the proceedings. Copyright Authors will be required to transfer copyright of their paper to AAAI. Paper Submissions & Inquiries Please send papers and conference registration inquiries to: AAAI-94 American Association for Artificial Intelligence 445 Burgess Drive Menlo Park, CA 94025-3496 Registration and call clarification inquiries (ONLY) may be sent to the Internet address: NCAI at aaai.org. Please send program suggestions and inquiries to: Barbara Hayes-Roth, Program Cochair Knowledge Systems Laboratory Stanford University 701 Welch Road, Building C Palo Alto, CA 94304 bhr at ksl.stanford.edu Richard Korf, Program Cochair Department of Computer Science University of California, Los Angeles Los Angeles, CA 90024 korf at cs.ucla.edu Howard Shrobe, Associate Program Chair Artificial Intelligence Laboratory Massachusetts Institute of Technology Cambridge, MA, 02139 hes at reagan.ai.mit.edu  From mpp at cns.brown.edu Fri Jul 30 21:45:20 1993 From: mpp at cns.brown.edu (Michael P. Perrone) Date: Fri, 30 Jul 93 21:45:20 EDT Subject: Thesis available Message-ID: <9307310145.AA05445@cns.brown.edu> FTP-host: archive.cis.ohio-state.edu FTP-filename: perrone.thesis.ps.Z A condensed version of my thesis is now available in neuroprose. Hardcopy versions can be obtained from UMI Dissertation Services (800-521-0600). Enjoy! Michael -------------------------------------------------------------------------------- Michael P. Perrone Email: mpp at cns.brown.edu Institute for Brain and Neural Systems Tel: 401-863-3920 Brown University Fax: 401-863-3934 Providence, RI 02912 -------------------------------------------------------------------------------- Improving Regression Estimation: Averaging Methods for Variance Reduction with Extensions to General Convex Measure Optimization M. P. Perrone -------------------------------------------------------------------------------- Retrieval instructions: The thesis is found in the neuroprose archive under perrone.thesis.ps.Z 83 pages To retrieve these files from the neuroprose archives: unix> ftp cheops.cis.ohio-state.edu Name (cheops.cis.ohio-state.edu:username): anonymous Password: (use your email address) ftp> cd pub/neuroprose ftp> binary ftp> get perrone.thesis.ps.Z ftp> bye  From dhw at santafe.edu Fri Jul 30 14:50:29 1993 From: dhw at santafe.edu (dhw@santafe.edu) Date: Fri, 30 Jul 93 12:50:29 MDT Subject: No subject Message-ID: <9307301850.AA07624@zia> Tony Chan writes: >>> I would like to raise the following question: Is there such a thing as the best method to learn a random idea? More precisely, given n random (machine) learning problems, is there such a thing as the best method for dealing with these problems that will give the best overall performance. There may be some best methods for dealing with some specific types of learning problems but is there one that would deal with any learning problem and give the best overall performance? >>> The answer depends on the precise way the problem is phrased. But in general, the answer is (provably) no, at least as far as off-training set error is concerned. For example, if the prior distribution over target functions is uniform, then all algorithms have the exact same average off-training set performance. Moreover, in a broad number of contexts, it is always true that if "things" (be they priors, training sets, or whatever) are such that algorithm 1 will outperform algorithm 2, then one can always set up those "things" differently, so that algorithm 2 outperforms algorithm 1, at least as far as off-training set behavior is concerned. Many of the results in the literature which appear to dispute this are simply due to use of an error function which is not restricted to being off-training set. In other words, there's always a "win" if you perform rationally on the training set (e.g., reproduce it exactly, when there's no noise), if your error function gives you points for performing rationally on the training set. In a certain sense, this is trivial, and what's really interesting is off-training set behavior. In any case, this automatic on-training set win is all those aforementioned results refer to; in particular, they imply essentially nothing concerning performance off of the training set.  From Connectionists-Request at cs.cmu.edu Thu Jul 1 00:05:14 1993 From: Connectionists-Request at cs.cmu.edu (Connectionists-Request@cs.cmu.edu) Date: Thu, 01 Jul 93 00:05:14 -0400 Subject: Bi-monthly Reminder Message-ID: <24313.741499514@B.GP.CS.CMU.EDU> *** DO NOT FORWARD TO ANY OTHER LISTS *** This note was last updated January 4, 1993. This is an automatically posted bi-monthly reminder about how the CONNECTIONISTS list works and how to access various online resources. CONNECTIONISTS is not an edited forum like the Neuron Digest, or a free-for-all newsgroup like comp.ai.neural-nets. It's somewhere in between, relying on the self-restraint of its subscribers. Membership in CONNECTIONISTS is restricted to persons actively involved in neural net research. The following posting guidelines are designed to reduce the amount of irrelevant messages sent to the list. Before you post, please remember that this list is distributed to over a thousand busy people who don't want their time wasted on trivia. Also, many subscribers pay cash for each kbyte; they shouldn't be forced to pay for junk mail. Happy hacking. -- Dave Touretzky & David Redish --------------------------------------------------------------------- What to post to CONNECTIONISTS ------------------------------ - The list is primarily intended to support the discussion of technical issues relating to neural computation. - We encourage people to post the abstracts of their latest papers and tech reports. - Conferences and workshops may be announced on this list AT MOST twice: once to send out a call for papers, and once to remind non-authors about the registration deadline. A flood of repetitive announcements about the same conference is not welcome here. - Requests for ADDITIONAL references. This has been a particularly sensitive subject lately. Please try to (a) demonstrate that you have already pursued the quick, obvious routes to finding the information you desire, and (b) give people something back in return for bothering them. The easiest way to do both these things is to FIRST do the library work to find the basic references, then POST these as part of your query. Here's an example: WRONG WAY: "Can someone please mail me all references to cascade correlation?" RIGHT WAY: "I'm looking for references to work on cascade correlation. I've already read Fahlman's paper in NIPS 2, his NIPS 3 abstract, and found the code in the nn-bench archive. Is anyone aware of additional work with this algorithm? I'll summarize and post results to the list." - Announcements of job openings related to neural computation. - Short reviews of new text books related to neural computation. To send mail to everyone on the list, address it to Connectionists at CS.CMU.EDU ------------------------------------------------------------------- What NOT to post to CONNECTIONISTS: ----------------------------------- - Requests for addition to the list, change of address and other administrative matters should be sent to: "Connectionists-Request at cs.cmu.edu" (note the exact spelling: many "connectionists", one "request"). If you mention our mailing list to someone who may apply to be added to it, please make sure they use the above and NOT "Connectionists at cs.cmu.edu". - Requests for e-mail addresses of people who are believed to subscribe to CONNECTIONISTS should be sent to postmaster at appropriate-site. If the site address is unknown, send your request to Connectionists-Request at cs.cmu.edu and we'll do our best to help. A phone call to the appropriate institution may sometimes be simpler and faster. - Note that in many mail programs a reply to a message is automatically "CC"-ed to all the addresses on the "To" and "CC" lines of the original message. If the mailer you use has this property, please make sure your personal response (request for a Tech Report etc.) is NOT broadcast over the net. - Do NOT tell a friend about Connectionists at cs.cmu.edu. Tell him or her only about Connectionists-Request at cs.cmu.edu. This will save your friend from public embarrassment if she/he tries to subscribe. ------------------------------------------------------------------------------- The CONNECTIONISTS Archive: --------------------------- All e-mail messages sent to "Connectionists at cs.cmu.edu" starting 27-Feb-88 are now available for public perusal. A separate file exists for each month. The files' names are: arch.yymm where yymm stand for the obvious thing. Thus the earliest available data are in the file: arch.8802 Files ending with .Z are compressed using the standard unix compress program. To browse through these files (as well as through other files, see below) you must FTP them to your local machine. ------------------------------------------------------------------------------- How to FTP Files from the CONNECTIONISTS Archive ------------------------------------------------ 1. Open an FTP connection to host B.GP.CS.CMU.EDU (Internet address 128.2.242.8). 2. Login as user anonymous with password your username. 3. 'cd' directly to one of the following directories: /usr/connect/connectionists/archives /usr/connect/connectionists/bibliographies 4. The archives and bibliographies directories are the ONLY ones you can access. You can't even find out whether any other directories exist. If you are using the 'cd' command you must cd DIRECTLY into one of these two directories. Access will be denied to any others, including their parent directory. 5. The archives subdirectory contains back issues of the mailing list. Some bibliographies are in the bibliographies subdirectory. Problems? - contact us at "Connectionists-Request at cs.cmu.edu". ------------------------------------------------------------------------------- How to FTP Files from the Neuroprose Archive -------------------------------------------- Anonymous FTP on archive.cis.ohio-state.edu (128.146.8.52) pub/neuroprose directory This directory contains technical reports as a public service to the connectionist and neural network scientific community which has an organized mailing list (for info: connectionists-request at cs.cmu.edu) Researchers may place electronic versions of their preprints or articles in this directory, announce availability, and other interested researchers can rapidly retrieve and print the postscripts. This saves copying, postage and handling, by having the interested reader supply the paper. (Along this line, single spaced versions, if possible, will help!) To place a file, put it in the Inbox subdirectory, and send mail to pollack at cis.ohio-state.edu. Within a couple of days, I will move and protect it, and suggest a different name if necessary. When you announce a paper, you should consider whether (A) you want it automatically forwarded to other groups, like NEURON-DIGEST, (which gets posted to comp.ai.neural-networks) and if you want to provide (B) free or (C) prepaid hard copies for those unable to use FTP. If you do offer hard copies, be prepared for an onslaught. One author reported that when they allowed combination AB, the rattling around of their "free paper offer" on the worldwide data net generated over 2000 hardcopy requests! Experience dictates the preferred paradigm is to announce an FTP only version with a prominent "**DO NOT FORWARD TO OTHER GROUPS**" at the top of your announcement to the connectionist mailing list. Current naming convention is author.title.filetype[.Z] where title is enough to discriminate among the files of the same author. The filetype is usually "ps" for postscript, our desired universal printing format, but may be tex, which requires more local software than a spooler. Very large files (e.g. over 200k) must be squashed (with either a sigmoid function :) or the standard unix "compress" utility, which results in the .Z affix. To place or retrieve .Z files, make sure to issue the FTP command "BINARY" before transfering files. After retrieval, call the standard unix "uncompress" utility, which removes the .Z affix. An example of placing a file is attached as an appendix, and a shell script called Getps in the directory can perform the necessary retrival operations. For further questions contact: Jordan Pollack Assistant Professor CIS Dept/OSU Laboratory for AI Research 2036 Neil Ave Email: pollack at cis.ohio-state.edu Columbus, OH 43210 Phone: (614) 292-4890 Here is an example of naming and placing a file: gvax> cp i-was-right.txt.ps rosenblatt.reborn.ps gvax> compress rosenblatt.reborn.ps gvax> ftp archive.cis.ohio-state.edu Connected to archive.cis.ohio-state.edu. 220 archive.cis.ohio-state.edu FTP server ready. Name: anonymous 331 Guest login ok, send ident as password. Password:neuron 230 Guest login ok, access restrictions apply. ftp> binary 200 Type set to I. ftp> cd pub/neuroprose/Inbox 250 CWD command successful. ftp> put rosenblatt.reborn.ps.Z 200 PORT command successful. 150 Opening BINARY mode data connection for rosenblatt.reborn.ps.Z 226 Transfer complete. 100000 bytes sent in 3.14159 seconds ftp> quit 221 Goodbye. gvax> mail pollack at cis.ohio-state.edu Subject: file in Inbox. Jordan, I just placed the file rosenblatt.reborn.ps.Z in the Inbox. The INDEX sentence is "Boastful statements by the deceased leader of the neurocomputing field." Please let me know when it is ready to announce to Connectionists at cmu. BTW, I enjoyed reading your review of the new edition of Perceptrons! Frank ------------------------------------------------------------------------ How to FTP Files from the NN-Bench Collection --------------------------------------------- 1. Create an FTP connection from wherever you are to machine "pt.cs.cmu.edu" (128.2.254.155). 2. Log in as user "anonymous" with password your username. 3. Change remote directory to "/afs/cs/project/connect/bench". Any subdirectories of this one should also be accessible. Parent directories should not be. 4. At this point FTP should be able to get a listing of files in this directory and fetch the ones you want. Problems? - contact us at "nn-bench-request at cs.cmu.edu". From maass at igi.tu-graz.ac.at Thu Jul 1 13:19:37 1993 From: maass at igi.tu-graz.ac.at (Wolfgang Maass) Date: Thu, 01 Jul 93 19:19:37 +0200 Subject: new paper in neuroprose Message-ID: <9307011719.AA17606@figids01.tu-graz.ac.at> FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/maass.agnostic.ps.Z The file maass.agnostic.ps.Z is now available for copying from the Neuroprose repository. This is a 22-page long paper. Hardcopies are not available. AGNOSTIC PAC-LEARNING OF FUNCTIONS ON ANALOG NEURAL NETS by Wolfgang Maass Institute for Theoretical Computer Science Technische Universitaet Graz, A-8010 Graz, Austria email: maass at igi.tu-graz.ac.at Abstract: This is a revised version of a preprint from May 93. In this new version we have added a parallelization of the learning algorithm LEARN that runs in polynomial time in ALL relevant parameters. We consider learning of real-valued functions on analog neural nets in Haussler's refinement of Valiant's model for probably approximately correct learning ("PAC-learning"). One commonly refers to this refined model as "agnostic PAC-learning", since it requires no a-priori assumptions about the structure of the learning target. The learning target need not even be a function, instead it may be any distribution of input/output pairs. In particular, arbitrary errors and imprecisions are permitted in the training data. Hence the setup of this model is well-suited for the analysis of real-world learning problems on neural Cnets. The goal of the learner in this model is to find a hypothesis whose true error (resp. loss) is within an epsilon of the true error of the best hypothesis from the considered hypothesis class. In our application to neural nets this hypothesis class is given by the class of all functions that can be computed on some fixed neural net N (with arbitrary real weights). We prove a positive result about agnostic PAC-learning on an arbitrary fixed analog neural net N (with arbitrary piecewise polynomial activation functions). We first construct for such N a somewhat larger neural net N' (the "learning network"). Then we exhibit a learning algorithm LEARN that computes from the training examples an assignment of rational numbers to the weights of N' such that with high probability the true error of the function that is computed by N' with these weights is within an epsilon of that of the best hypothesis that is computable on N (with arbitrary real weights). If the number of gates in N may be viewed as a constant, then the computation time of the learning algorithm LEARN is polynomial (in all other relevant parameters). In addition one can parallelize this learning algorithm in such a way that its computation time is polynomial in ALL relevant parameters, including the number of gates in N . It should be noted that in contrast to common learning algorithms such as backwards propagation, this learning algorithm LEARN is guaranteed to solve its learning task (with high probability). As a part of this positive learning result we show that the pseudo- dimension of neural nets with piecewise polynomial activation functions can be bounded by a polynomial in the number of weights of the neural net. It has previously been shown by Haussler that the pseudo-dimension is the appropriate generalization of the VC-dimension for learning real valued functions. With the help of our upper bound on the pseudo- dimension of neural nets with piecewise polynomial activation functions we can bound the number of training-examples that are needed for agnostic PAC-learning. In this way one can reduce the task of minimizing the true error of the neural net to the finite task of minimizing its apparent error (i.e. its error on the training examples). From jong at miata.postech.ac.kr Fri Jul 2 15:04:14 1993 From: jong at miata.postech.ac.kr (Jong-Hoon Oh) Date: Fri, 2 Jul 93 15:04:14 KDT Subject: preprint Message-ID: <9307020504.AA01718@miata.postech.ac.kr> FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/oh.generalization.ps.Z The following paper has been placed in the Neuroprose archive (see above for ftp-host) in file oh.generalization.ps.Z (8 pages of output) ----------------------------------------------------------------- Generalization in a two-layer neural network Kukjin Kang, Jong-Hoon Oh Department of Physics, Pohang Institute of Science and Technology, Pohang, Kyongbuk, Korea Chulan Kwon, Youngah Park Department of Physics, Myong Ji University, Yongin, Kyonggi, Korea Learning of a fully connected two-layer neural networks with $N$ input nodes, $M$ hidden nodes and a single output node is studied using the annealed approximation. We study the generalization curve, i.e. the average generalization error as a function of the number of the examples. When the number of examples is the order of $N$, the generalization error is rapidly decreasing and the system is in a permutation symmetric(PS) phase. As the number of examples $P$ grows to the order of $MN$ the generalization error converges to a constant value. Finally the system undergoes a first order phase transition to a perfect learning and the permutation symmetry breaks. The computer simulations show a good agreement with analytic results. PACS number(s): 87.10.+e, 05.50.+s, 64.60.Cn Jong-Hoon Oh jhoh at miata.postech.ac.kr ----------------------------------------------------------------- From spotter at darwin.bio.uci.edu Fri Jul 2 15:26:17 1993 From: spotter at darwin.bio.uci.edu (Steve Potter) Date: Fri, 2 Jul 1993 12:26:17 -0700 (PDT) Subject: Cultured Neural Nets Message-ID: <Pine.3.05.9307021216.A8654-d100000@mendel.bio.uci.edu> Below I present a bibliography of all of the researchers I know of that are growing neurons in culture on multielectrode substrates. A belated thank-you is due to several connectionists who responded to my request posted a couple of years ago. This is a surprisingly small list. If you know of someone I have missed, please send me email (spotter at darwin.bio.uci.edu). I believe that approaches such as these are likely to close the gap between the engineering and biological camps of neural network research. With long-term, multi-site monitoring of real (though simple) networks, we may learn which aspects of real neural processors must be included in our simulations if we hope to emulate the accomplishments of Mother Nature. If you are involved in this research and I have not contacted you already, please email me; I am looking for a post-doctoral position. Steve Potter Psychobiology dept. UC Irvine Irvine, CA 92717 spotter at darwin.bio.uci.edu CULTURED NETS ON MULTI-ELECTRODE SUBSTRATES: (Recent or representative publications are listed) Steve Potter 7-2-93 spotter at darwin.bio.uci.edu Masuo Aizawa (Layman's article) Freedman, D.H. (1992). If he only had a brain. Discover : 54-60. Robert L. Dawes, Martingale Research (Texas) (Proposal--Never followed up?) Dawes, R.L. (1987). Biomasscomp: A procedure for mapping the architecture of a living neural network into a machine. IEEE ICNN proceedings 3: 215-225. Mitch D. Eggers, MIT (Any subsequent work with this device?) Eggers, M.D., Astolfi, D.K., Liu, S., Zeuli, H.E., Doeleman, S.S., McKay, R., Khuon, T.S., and Ehrlich, D.J. (1990). Electronically wired petri dish: A microfabricated interface to the biological neuronal network. J. Vac. Sci. Technol. B 8: 1392-1398. Peter Fromherz, Ulm University (Germany) Fromherz, P., Offenhausser, A., Vetter, T., and Weis, J. (1991). A neuron-silicon junction: a Retzius cell of the leech on an insulated-gate field-effect transistor. Science 252: 1290-3. Guenter W. Gross, U. of N. Texas Gross, G.W. and Kowalski, J. (1991) Experimental and theoretical analysis of random nerve cell network dynamics, in Neural Networks: Concepts, applications, and implementations (P. Antognetti and B Milutinovic, Eds.) Prentice-Hall: NJ. p. 47-110. Vera Janossy, Central Research Inst. for Physics (Hungary) Janossy, V., Toth, A., Bodocs, L., Imrik, P., Madarasz, E., and Gyevai, A. (1990). Multielectrode culture chamber: a device for long-term recording of bioelectric activities in vitro. Acta Biol Hung 41: 309-20. Akio Kawana, NTT (Japan) (News article) Koppel, T. (1993). Computer firms look to the brain. Science 260: 1075-1077. Jerome Pine, Caltech Regehr, W.G., Pine, J., Cohan, C.S., Mischke, M.D., and Tank, D.W. (1989). Sealing cultured invertebrate neurons to embedded dish electrodes facilitates long-term stimulation and recording. J Neurosci Methods 30: 91-106. David W. Tank, AT&T Bell Labs (Abstract) Tank, D.W. and Ahmed, Z. (1985). Multiple site monitoring of activity in cultured neurons. Biophys. J. 47: 476a. C. D. W. Wilkinson, U. of Glasgow (Scotland) Connolly, P., Clark, P., Curtis, A.S., Dow, J.A., and Wilkinson, C.D. (1990). An extracellular microelectrode array for monitoring electrogenic cells in culture. Biosens Bioelectron 5: 223-34. Curtis, A.S., Breckenridge, L., Connolly, P., Dow, J.A., Wilkinson, C.D., and Wilson, R. (1992). Making real neural nets: design criteria. Med Biol Eng Comput 30: CE33-6. ACUTE PREPS (NOT CULTURED): Bruce C. Wheeler, U. of Illinois (Hippocampal slice) Boppart, S.A., Wheeler, B.C., and Wallace, C.S. (1992). A flexible perforated microelectrode array for extended neural recordings. Ieee Trans Biomed Eng 39: 37-42. Novak, J.L. and Wheeler, B.C. (1986). Recording from the Aplysia abdominal ganglion with a planar microelectrode array. Ieee Trans Biomed Eng 33: 196-202. Markus Meister, Harvard Meister, M., Wong, R.O., Baylor, D.A., and Shatz, C.J. (1991). Synchronous bursts of action potentials in ganglion cells of the developing mammalian retina. Science 252: 939-43. Litke, A. and Meister, M. (1991). The retinal readout array. Nuclear Instruments and Methods in Physics Research A310: 389-394. From white at TEETOT.ACUSD.EDU Tue Jul 6 19:04:41 1993 From: white at TEETOT.ACUSD.EDU (Ray White) Date: Tue, 6 Jul 1993 16:04:41 -0700 Subject: paper in neuroprose: white.c-hebb-2.ps.Z Message-ID: <9307062304.AA06538@TEETOT.ACUSD.EDU> FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/white.c-hebb-2.ps.Z ** PLEASE DO NOT FORWARD TO OTHER GROUPS ** The paper "Competitive Hebbian Learning 2: an Introduction", (8 pages) a poster to be presented at WCNN '93 in Portland Monday morning, July 12, 1993, has just been placed in the Neuroprose directory at Ohio State. Thanks again, Jordan Pollack. ABSTRACT In this paper the Competitive Hebbian Learning 2, or CHL 2, learning rule is introduced. CHL 2 is an unsupervised learning rule with characteristics which are plausible as a simplified model of biological learning, as well as useful learning properties for artificial systems. It combines a modified Hebbian learning rule with a competitive learning, and shows promise for unsupervised-learning tasks such as feature detection. The usual instructions to FTP and to print the paper apply. Ray White <white at teetot.acusd.edu> Departments of Physics and Computer Science University of San Diego 5998 Alcala Park San Diego, CA 92110 619-260-4627 From sperduti at ICSI.Berkeley.EDU Tue Jul 6 21:00:40 1993 From: sperduti at ICSI.Berkeley.EDU (Alessandro Sperduti) Date: Tue, 6 Jul 93 18:00:40 PDT Subject: TRs on reduced representations Message-ID: <9307070100.AA10347@icsib57.ICSI.Berkeley.EDU> FTP-host: ftp.icsi.berkeley.edu (128.32.201.7) FTP-filename: pub/techreports/tr-93-029.ps.Z FTP-filename: pub/techreports/tr-93-031.ps.Z The following technical reports are available by public ftp from the International Computer Science Institute. For hardcopies there is a small charge to cover postage and handling for each report (info at icsi.berkeley.edu). Comments welcome. Alessandro Sperduti sperduti at icsi.berkeley.edu ____________________________________________________________________________ TR-93-029 (48 pages) Labeling RAAM Alessandro Sperduti International Computer Science Institute 1947 Center Street, Suite 600 Berkeley, California 94704 TR-93-029 Abstract In this report we propose an extension of the Recursive Auto-Associative Memory (RAAM) by Pollack. This extension, the Labeling RAAM (LRAAM), is able to encode labeled graphs with cycles by representing pointers explicitly. A theoretical analysis of the constraints imposed on the weights by the learning task under the hypothesis of perfect learning and linear output units is presented. Cycles and confluent pointers result to be particularly effective in imposing constraints on the weights. Some technical problems encountered in the RAAM, such as the termination problem in the learning and decoding processes, are solved more naturally in the LRAAM framework. The representations developed for the pointers seem to be robust to recurrent decoding along a cycle. Data encoded in a LRAAM can be accessed by pointer as well as by content. The direct access by content can be achieved by transforming the encoder network of the LRAAM in a Bidirectional Associative Memory (BAM). Different access procedures can be defined according to the access key. The access procedures are not wholly reliable, however they seem to have a high likelihood of success. A geometric interpretation of the decoding process is given and the representations developed in the pointer space of a two hidden units LRAAM are presented and discussed. In particular, the pointer space results to be partitioned in a fractal-like fashion. Some effects on the representations induced by the Hopfield-like dynamics of the pointer decoding process are discussed and an encoding scheme able to retain the richness of representation devised by the decoding function is outlined. The application of the LRAAM model to the control of the dynamics of recurrent high-order networks is briefly sketched as well. TR-93-031 (19 pages) On Some Stability Properties of the LRAAM Model Alessandro Sperduti International Computer Science Institute 1947 Center Street, Suite 600 Berkeley, California 94704 TR-93-031 Abstract In this report we discuss some mathematical properties of the LRAAM model. The LRAAM model is an extension of the RAAM model by Pollack. It allows one to obtain distributed reduced representations of labeled graphs. In particular, we give sufficient conditions on the asymptotical stability of the decoding process along a cycle of the encoded structure. Data encoded in an LRAAM can also be accessed by content by transforming the LRAAM in an analog Hopfield network with hidden units and asymmetric connection matrix (CA network.) Different access procedures can be defined according to the access key. Each access procedure corresponds to a particular constrained version of the CA network. We give sufficient conditions under which the property of asymptotical stability of a fixed point in one particular constrained version of the CA network can be extended to related fixed points of different constrained versions of the CA network. An example of encoding of a labeled graph on which the theoretical results are applied is given as well. To obtain electronic copies: ftp ftp.icsi.berkeley.edu login: anonymous password: <your email address> cd pub/techreports binary get tr-93-029.ps.Z get tr-93-031.ps.Z bye Then at your system: uncompress tr-93-029.ps.Z uncompress tr-93-031.ps.Z lpr -P<printer-name> tr-93-029.ps tr-93-031.ps From liaw%dylink.usc.edu at usc.edu Wed Jul 7 13:36:42 1993 From: liaw%dylink.usc.edu at usc.edu (Jim Liaw) Date: Wed, 7 Jul 93 10:36:42 PDT Subject: Neural Architectures and Distributed AI Message-ID: <9307071736.AA09576@dylink.usc.edu> **** Call for Papers **** Neural Architectures and Distributed AI: From Schema Assemblages to Neural Networks October 19-20, 1993 The Center for Neural Engineering University of Southern California announces a Workshop on Neural Architectures and Distributed AI: From georgiou at silicon.csci.csusb.edu Wed Jul 7 17:36:49 1993 From: georgiou at silicon.csci.csusb.edu (George M. Georgiou) Date: Wed, 7 Jul 1993 14:36:49 -0700 Subject: CFP: 2nd Int'l Conference on Fuzzy Theory and Technology Message-ID: <9307072136.AA24292@silicon.csci.csusb.edu> I am in the process of organizing two sessions on Neural Networks at the 2nd Int'l Conference on Fuzzy Theory and Technology, which will take place in Durham, N.C., October 13--16, 1993. There are a few slots for non-invited papers, and anyone interested should submit an extended abstract by July 28, 1993, to the following address: Dr. George M. Georgiou Computer Science Department California State University 5500 University Pkwy San Bernardino, CA 92407, USA FAX: (909) 880-7004 E-mail: georgiou at wiley.csusb.edu The extended abstract (maximum 3 pages of single column and single-space text with figures and tables) may be submitted by e-mail as well (in ASCII, postscript, or TeX/LaTeX form). Notification for acceptance will be sent on August 6, 1993. The final version of the full length paper must be submitted by October 14, 1993 (time of conference). Four copies of the paper shall be prepared according to the ``Information For Authors'' appearing at the back cover of {\em Information Sciences, An International Journal}, (Elsevier Publishing Co.). A full paper shall not exceed 20 pages including figures and tables. All full papers will be reviewed by experts. Revised papers will be due on April 15, 1994. Accepted papers will appear in the hard-covered proceedings (book with uniform type-setting) OR in the {\em Information Sciences Journal}. Lotfi Zadeh "Best paper award": All papers submitted to FT&T'93 will be considered for this award, the prize of which includes $2,500 plus hotel accommodations (traveling expenses excluded) at the FT&T'94. The date of announcement of the the best paper is March 30, 1994. Oral presentation in person at the FT&T'93 is required, as well an acceptance speech at the FT&T'94. The evaluation committee consists of the following 10 members: Jack Aldridge, B. Bouchon-Meunier, Abe Kandel, George Klir, I.R. Goodman, John Mordeson, Sujeet Shenoi, H. Cris Tseng, H. Zimmerman, Frank Y. Shih. (Alternates: Akira Nakamura and Frank Y. Shih) ----------------------------------------------------------------------- Further Conference Info contact: Jerry C.Y. Tyan, e-mail: ctyan at ee.egr.duke.edu, tel: (919)660-5294 OR Jing Dai, e-mail: jdai at ee.egr.duke.edu, tel: (919)660-5228 ----------------------------------------------------------------------- Exhibit Information: Interested vendors should contact: Rhett George, E.E. Dept., Duke University Tel: (919)660-5242 Fax: (919)660-5293 ----------------------------------------------------------------------- ----------------------------------------------------------------------- From mafm at cs.uwa.edu.au Thu Jul 8 13:34:17 1993 From: mafm at cs.uwa.edu.au (Matthew McDonald) Date: Thu, 8 Jul 93 13:34:17 WST Subject: Reinforcement Learning Mailing List Message-ID: <9307080534.AA02405@cs.uwa.edu.au> This message is to announce an informal mailing list devoted to reinforcement learning. The list is intended to provide an informal, unmoderated, forum for discussing subjects relevant to research in reinforcement learning; in particular, discussion of problems, interesting papers and that sort of thing is welcome. Announcements and other information relevant to researchers in the field are also welcome. People are encouraged to post abstracts of recent papers or reports. The list is intended to be fairly informal and unmoderated. If you'd like to join the list, please send mail to `reinforce-request at cs.uwa.edu.au' Cheers, -- Matthew McDonald mafm at cs.uwa.oz.au Nothing is impossible for anyone impervious to reason. From vet at cs.utwente.nl Thu Jul 8 05:08:45 1993 From: vet at cs.utwente.nl (Paul van der Vet) Date: Thu, 8 Jul 93 11:08:45 +0200 Subject: ECAI94 Message-ID: <9307080908.AA26633@utis31.cs.utwente.nl> E C A I '94 A M S T E R D A M 11th European Conference on Artificial Intelligence Amsterdam RAI International Exhibition and Congress Centre The Netherlands August 8-12, 1994 Call for Papers Call for Workshop proposals Exhibition Call for Tutorial proposals Organized by the European Coordinating Committee for Artificial Intelligence (ECCAI) Hosted by the Dutch Association for Artificial Intelligence (NVKI) The European Conference on Artificial Intelligence (ECAI) is the European forum for scientific exchange and presentation of AI research. The aim of the conference is to cover all aspects of AI research and to bring together basic research and applied research. The Technical Programme will include paper presentations, invited talks, panels, workshops, and tutorials. The conference is designed to cover all subfields of AI, including non-symbolic methods. ECAIs are held in alternate years and are organized by the European Coordinating Committee for Artificial Intelligence (ECCAI). The 11th ECAI in 1994 will be hosted by the Dutch AI Society (NVKI). The conference will take place at the Amsterdam RAI, International Exhibition and Congress Centre. E X H I B I T I O N An industrial and academic exhibition will be organized from August 9 - 11, 1994. Detailed information will be provided in the second call for papers or can be obtained at the conference office (for the adress see elsewhere). S P O N S O R S (preliminary list) Bolesian B.V. Municipality of Amsterdam University of Amsterdam Vrije Universiteit Amsterdam University of Limburg C A L L F O R P A P E R S T O P I C S O F I N T E R E S T You are invited to submit an original research paper that represents a significant contribution to any aspect of AI, including the principles underlying cognition, perception, and action in humans and machines; the design, application, and evaluation of AI algorithms and intelligent systems; and the analysis of tasks and domains in which intelligent systems perform. Theoretical and experimental results are equally welcome. Papers describing innovative ideas are especially sought providing such papers include substantial analysis of the ideas, the technology needed to realize them, and their potential impact. Of special interest this year are papers which address applied AI. Two kinds of papers are sought. The first category is case studies of AI applications that address significant real-world problems and which are used outside the AI community itself; these papers must justify the use of the AI technique, explain how the AI technology contributed to the solution and was integrated with other components, and most importantly explain WHY the application was successful (or perhaps why it failed) -- these "lessons learned" will be the most important review criteria. The second category is for papers on novel AI techniques and principles that may enable more ambitious real-world applications. All the usual AI topics are appropriate. These papers must describe the importance of the approach from an applications context, in sufficient technical detail and clarity, and clearly and thoroughly differentiate the work from previous efforts. There will be special prizes for the best papers in both these areas. S U B M I S S I O N O F P A P E R S Authors are requested to submit to the Programme Chairperson 5 copies of papers written in English in hardcopy format (electronic and fax submissions will not be accepted). Each submitted paper must conform to the following specifications. Papers should be no longer than 5500 words including references. (Each full page of figures counts as 1100 words.) Papers longer than this limit risk being rejected without refereeing. A separate title page should include the title, the name(s) of the author(s), complete address(es), email, fax and telephone numbers, the specification of between one and four Content Areas, preferably chosen from the list below and an abstract (maximum 200 words). The title page should also contain a declaration that the paper is unpublished, original work, substantially different from papers currently under review and will not be submitted elsewhere before the notification date other than to workshops and similar specialized presentations with a very limited audience. Papers should be printed on A4 or 8.5"x11" sized paper in letter quality print, with 12 point type (10 chars/inch on a typewriter), single spaced. Double sided printing is preferred. Authors who wish to check that their submission will fit into the final CRC format will be able to obtain detailed instructions including a latex style file and example postscript pages after October 15 by anonymous FTP from agora.leeds.ac.uk, directory ECAI94, or by e-mailing ecai94-style at scs.leeds.ac.uk with a message body of "help". When submitting a paper an electronic mail message should also be sent to ecai94-title at scs.leeds.ac.uk giving information in the format specified below. If an intending author has no e-mail facilities then this requirement is waived. Papers should be sent to: Programme Chairperson: Dr Tony Cohn Division of Artificial Intelligence School of Computer Studies University of Leeds Leeds LS2 9JT United Kingdom Tel.: (+44)-532-33.54.82 Fax: (+44)-532-33.54.68 E-mail: ecai94 at scs.leeds.ac.uk TITLE: <title of paper> AUTHOR: <first author last name, first name> AFFILIATION: <first author affiliation> AUTHOR: <second author last name, first name> AFFILIATION: <second author affiliation> ..<repeat for all authors> CORRESPONDENCE ADDRESS: <give name, address, fax and telephone on successivelines> CORRESPONDENCE E-MAIL: <give correspondence e-mail address> CONTENT AREAS: <at most four content areas, separated by semi-colons> ABSTRACT: <text of the abstract> The content areas preferably should be drawn from the topics listed below. The text of the abstract field may include formatting commands, if desired, but these should be omitted from all other fields. Work described in an accepted paper may also be illustrated with a videotape or a demo. Special sessions will be scheduled for video presentations and demos. Authors wishing to show a videotape or a demo should specify the duration and the requirements of the videotape/demo when submitting their paper for review. Reviewing criteria do not apply to these tapes. Only the submitted papers will be peer-reviewed. Authors wishing to augment their paper presentation with a video should submit a tape only after their paper has been accepted. For details concerning tape format, see the video track description below. C O N T E N T A R E A S Abduction; AI and Creativity; Artificial Life; Automated Reasoning; Automatic Programming; Belief Revision; Case Studies of AI Applications; Case-Based Reasoning; Cognitive Modelling; Common Sense Reasoning; Communication and Cooperation; Complexity of Reasoning; Computational Theories in Psychology; Computer-Aided Education; Concept Formation; Connectionist and PDP Models for AI; Constraint-Based Reasoning; Corpus-Based Language Analysis; Deduction; Description Logics; Design; Diagnosis; Discourse Analysis; Discovery; Multi-Agent Systems; Distributed Problem Solving; Enabling Technology and Systems; Epistemological Foundations; Expert System Design; Generic Applications; Genetic Algorithms; Integrating AI and Conventional Systems; Integrating Several AI Components; Kinematics; Knowledge Acquisition; Knowledge Representation; Large Scale Knowledge Engineering; Logic Programming; Machine Architectures; Machine Learning; Machine Translation; Mathematical Foundations; Model Based Reasoning; Monitoring; Natural Language Front Ends; Natural Language Processing; Navigation; Neural Networks; Nonmonotonic Reasoning; Philosophical Foundations and Implications; Plan Recognition; Planning and Scheduling; Principles of AI Applications; Qualitative Reasoning; Reactivity; Reasoning About Action; Reasoning About Physical Systems; Reasoning With Uncertainty; Resource Allocation; Robotics; Robot Navigation; Search; Sensor Interpretation; Sensory Fusion/Fission; Simulation; Situated Cognition; Social Economic, Ethical and Legal Implications; Spatial Reasoning; Speech Recognition; Standardisation, Exchange and Reuse of Ontologies or Knowledge; Parsing; Semantic Interpretation; Pragmatics; System Architectures; Temporal and Causal Reasoning; Terminological Reasoning; Text Generation and Understanding; Theorem Proving; Truth Maintenance; Tutoring Systems; User Interfaces; User Models; Verification, Validation and Testing of Knowledge-Based Systems; Virtual Reality; Vision and Signal Understanding. T I M E T A B L E Papers must be received by the Programme Chairperson no later than January 8, 1994. Acceptance letters will be posted no later than March 12, 1994. Final camera-ready papers must be received by April 19, 1994. P A N E L S Proposals for panel discussions (up to 1000 words) should be sent to the Programme Chairperson by Februar 8, 1994. E-mail is preferred. P R I Z E S As in previous years, a prize for the best paper as determined by the Programme Committee will be awarded; the Digital Equipment Prize and a prize for the best paper from Eastern Europe will also be awarded. Additionally, this year there will be two new prizes which will be awarded for application papers in the two categories described above under "Case Studies of AI Applications" and "Principles of AI Applications". V I D E O S U B M I S S I O N S In addition to the possibility of video enhanced papers described above, videos unaccompanied by papers may be submitted for presentation in special video track sessions. The purpose of these videos should be to demonstrate the current levels of usefulness of AI tools, techniques and methods. Videos presenting research arising out of interesting real world applications are especially sought. Authors should submit one copy of a videotape of 10 minutes maximum duration accompanied by a submission letter that includes: * Title * Full names, postal addresses, phone numbers and e-mail addresses of all authors * Duration of tape in minutes * Three copies of an abstract of one to two pages in length, containing the title of the video, and full names and addresses of the authors * Author's permission to copy tape for review purposes The timetable and conditions for submission, notification of acceptance or rejection, and receipt of final version are the same as for the paper track. All videotape submissions must be made to the Programme Chair. Tapes cannot be returned; authors should retain extra copies for making revisions. All videos must be in VHS-PAL format. An e-mail message giving the title, author, address and abstract should be e-mailed to ecai94-video at scs.leeds.ac.uk (unless the submitter has no e-mail access in which case this condition is waived). Tapes will be reviewed and selected for presentation during the conference. The following criteria will guide the selection: Level of interest to the conference audience Clarity of goals, methods and results Presentation quality (including audio, video and pace). Preference will be given to applications that show a high level of maturity. Tapes that are deemed to be advertising commercial products, propaganda, purely expository materials, merely taped lectures or other material not of scientific or technical value will be rejec- ted. P R O G R A M M E C O M M I T T E E C. Baeckstroem, Sweden J.P. Barthes, France I. Bratko, Slovenia P. Brazdil, Portugal J. Breuker, The Netherlands F. Bry, Germany R. Casati, Switzerland C. Castelfranchi, Italy J. Cuena, Spain Y. Davidor, Israel L. Farinas del Cerro, France F. Fogelman Soulie, France J. Fox, United Kingdom G. Friedrich, Austria A. Frisch, United Kingdom C. Froidevaux, France A. Fuhrmann, Germany A. Galton, United Kingdom J. Ganascia, France M. Ghallab, France J. Goncalves, Italy G. Gottlob, Austria F. Giunchiglia, Italy E. Hajicova, Czech Republic P. Hill, United Kingdom S. Hoelldobler, Germany D. Hogg, United Kingdom G. Kelleher, United Kingdom G. Kempen, The Netherlands M. King, Switzerland A. Kobsa, Germany M. Lenzerini, Italy R. Lopez de Mantaras, Spain N. Mars, The Netherlands J. Martins, Portugal P. Meseguer, Spain R. Milne, United Kingdom B. Nebel, Germany R. Nossum, Norway H.J. Ohlbach, Germany E. Oja, Finland E. Oliveira, Portugal E. Plaza, Spain J. Rosenschein, Israel Ph. Smets, Belgium L. Spanpinato, Italy O. Stock, Italy P. Struss, Germany P. Torasso, Italy R. Trappl, Austria L. Trave-Massuyes, France W. van de Velde, Belgium W. Wahlster, Germany T. Wittig, Germany W O R K S H O P S A full workshop programme is planned for ECAI '94. These will take place in the two days immediately before the main technical conference, i.e., on August 8 and 9, 1994. Workshops may last for either 1 or 2 days. They will give participants the opportunity to discuss specific technical topics in a small, informal environment, which encourages interaction and exchange of ideas. Workshops may address any topic covered by the list of areas given above [i.e., in the general call for papers]. Workshops on applications and related issues are especially welcome. Workshop proposals should be in the form of a draft call for participation containing a brief description of the workshop and the technical issues to be addressed, the proposed format and the kind of contributions solicited, and the names and addresses (postal, phone, fax, e-mail) of the organizing committee of the workshop. Additionally, proposals should specify the number of expected participants and some names of some potential participants. Proposers are encouraged to send their draft proposal to potential participants for comments before submission. The organizers of accepted workshops are responsible for producing a call for participation, for reviewing requests to participate and for scheduling the workshop activities within the constraints set by the conference organizers. Workshop proposals should be sent to the Workshop Chairpersons as soon as possible, but not later than November 1, 1993. Electronic submission (plain ascii text) is highly preferred, but hard copy submission is also accepted in which case 5 copies should be submitted. Proposals should not exceed 2 sides of A4 (i.e., approximately 120 lines of text). The proposals will be reviewed by the Programme Committee and the organizers will be notified not later than December 31, 1993. Details of all accepted workshops will be available by anonymous FTP from cs.vu.nl, directory ECAI94 by January 31, 1994; alternatively send electronic mail to ecai94-workshops at cs.vu.nl. It should be noted that registration for the main conference will be required in order to attend an ECAI '94 workshop. Workshop Chairpersons: Prof.dr Jan Treur Dr Frances Brazier Vrije Universiteit Amsterdam Department of Computer Science De Boelelaan 1081 a 1081 HV Amsterdam, The Netherlands Tel.: (+31)-20-548.55.88 Fax: (+31)-20 -642.77.05 E-mail: ecai94-workshops at cs.vu.nl T U T O R I A L S The ECAI '94 Organizing Committee invites proposals for the Tutorial Programme for ECAI '94. The tutorials will take place on August 8 and 9, 1994. Anyone who is interested in presenting a tutorial, or who has suggestions concerning possible speakers or topics is invited to contact the Tutorial Chair. A list of suggested topics that may be covered by tutorials is given below, but the list is only a guide. Other topics, both related to these and quite different from them, will be considered: Model-based reasoning; Natural language processing; Real-time reasoning; AI & databases (deductive databases, integrity constraints); Distributed AI, multi-agent systems; AI in industry (banking, networking, engineering); Knowledge sharing and reuse; Machine learning; Neutral Networks; Probabilistic reasoning and uncertainty; Genetic algorithms; Case-based reasoning; KBS design and methodology (including knowledge acquisition); Planning and scheduling; Hypermedia/multi-media in AI We are interested in: Proposals for the tutorials to be presented at the ECAI '94 Suggestions for topics (either an expression of interest in any of the topics above or in other topics) Suggestions for possible speakers for the tutorials (who would you like to hear if you attend a tutorial?) Anyone interested in presenting a tutorial should submit a proposal containing the following information: * A brief description and outline of the tutorial * The necessary background and the potential target audience * A description of why the tutorial topic is of interest to the ECAI '94 audience * A brief resume of the presenters Each tutorial will last for four hours, and must be offered by a team of presenters (usually two people, possibly three). Those submitting a proposal should keep in mind that tutorials are intented to provide an overview of a field or practical training in an area. They should present reasonably well agreed upon information in a balanced way. Tutorials should not be used to advocate a single avenue of research, nor should they promote a product. Presenters of a tutorial will receive a remuneration based on the number of participants in the tutorial. Proposals and suggestions must be received by September 1, 1993. Decisions about the tutorial programme will be made by September 30, 1993. Speakers should be prepared to submit completed course materials by May 6, 1994. Proposals, suggestions and enquiries should be sent (preferably electronically) to: Tutorial Chairperson: Dr Frank van Harmelen SWI University of Amsterdam Roetersstraat 15 1018 WB Amsterdam Tel.: (+31)-20-525.61.21, or (+31)-20-525.67.89 Fax: (+31)-20-525.68.96 E-mail: ecai94-tutorials at swi.psy.uva.nl Details of all tutorials will be available by September 30, 1993 by anonymous FTP from swi.psy.uva.nl, directory ECAI94. I N F O R M A T I O N For more information please contact: Organizing Chairperson: Workshop Chairpersons: Prof.dr Jaap van den Herik Prof.dr Jan Treur Dutch Association for Artificial Dr Frances Brazier Intelligence (NVKI) Vrije Universiteit Amsterdam University of Limburg Department of Computer Science Department of Computer Science De Boelelaan 1081 a P.O. Box 616 1081 HV Amsterdam 6200 MD Maastricht The Netherlands The Netherlands Tel.: (+31)-20-548.55.88 Tel.: (+31)-43-88.34.77 Fax: (+31)-20-642.77.05 Fax: (+31)-43-25.23.92 E-mail: ecai94-workshops at cs.vu.nl E-mail: bosch at cs.rulimburg.nl Programme Chairperson: Tutorial Chairperson: Dr Tony Cohn Dr Frank van Harmelen Division of Artificial SWI Intelligence University of Amsterdam School of Computer Studies Roetersstraat 15 University of Leeds 1018 WB Amsterdam Leeds LS2 9JT The Netherlands United Kingdom Tel.: (+31)-20-525.61.21, or Tel.:(+44)-532-33.54.82 (+31)-20-525.67.89 Fax: (+44)-532-33.54.68 Fax: (+31)-20-525.68.96 E-mail: ecai94 at scs.leeds.ac.uk E-mail: ecai94-tutorials at swi.psy.uva.nl CONFERENCE OFFICE: Erasmus Forum c/o ECAI '94 Marcel van Marrewijk, Project Manager Mirjam de Leeuw, Conference Manager E C A I '94 Erasmus University Rotterdam AMSTERDAM P.O. Box 1738 3000 DR Rotterdam The Netherlands Tel.: (+31)-10-408.23.02 Fax: (+31)-10-453.07.84 E-mail: M.M.deLeeuw at apv.oos.eur.nl ECCAI EUROPEAN COORDINATING COMMITTEE FOR ARTIFICIAL INTELLIGENCE From rwp at eng.cam.ac.uk Fri Jul 9 11:38:20 1993 From: rwp at eng.cam.ac.uk (Richard Prager) Date: Fri, 9 Jul 1993 11:38:20 BST Subject: Cambridge Neural Nets Summer School Message-ID: <6612.9307091038@dsl.eng.cam.ac.uk> The Cambridge University Programme for Industry in Collaboration with the Cambridge University Engineering Department Announce their Third Annual Neural Networks Summer School. 3 1/2 day short course 13-16 September 1993 BOURLARD GEE HINTON JERVIS JORDAN KOHONEN NARENDRA NIRANJAN PECE PRAGER SUTTON TARRASENKO Outline and aim of the course The course will give a broad introduction to the application and design of neural networks and deal with both the theory and with specific applications. Survey material will be given, together with recent research results in architecture and training methods, and applications including signal processing, control, speech, robotics and human vision. Design methodologies for a number of common neural network architectures will be covered, together with the theory behind neural network algorithms. Participants will learn the strengths and weaknesses of the neural network approach, and how to assess the potential of the technology in respect of their own requirements. Lectures are being given by international experts in the field, and delegates will have the opportunity of learning first hand the technical and practical details of recent work in neural networks from those who are contributing to those developments. Who Should Attend The course is intended for engineers, software specialists and other scientists who need to assess the current potential of neural networks. The course will be of interest to senior technical staff who require an overview of the subject, and to younger professionals who have recently moved into the field, as well as to those who already have expertise in this area and who need to keep abreast of recent developments. Some, although not all, of the lectures will involve graduate level mathematical theory. PROGRAMME Introduction and overview: Connectionist computing: an introduction and overview Programming a neural network Parallel distributed processing perspective Theory and parallels with conventional algorithms Architectures: Pattern processing and generalisation Bayesian methods in neural networks Reinforcement learning neural networks Communities of expert networks Self organising neural networks Feedback networks for optimization Applications: Classification of time series Learning forward and inverse dynamical models Control of nonlinear dynamical systems using neural networks Artificial and biological vision systems Silicon VLSI neural networks Applications to diagnostic systems Shape recognition in neural networks Applications to speech recognition Applications to mobile robotics Financial system modelling Applications in medical diagnostics LECTURERS DR HERVE BOURLARD is with Lernout & Hauspie Speech Products in Brussels. He has made many contributions to the subject particularly in the area of speech recognition. MR ANDREW GEE is with the Speech, Vision and Robotics Group of the Cambridge University Engineering Department. He specialises in the use of neural networks for solving complex optimization problems. PROFESSOR GEOFFREY HINTON is in the Computer Science Department at the University of Toronto. He was a founding member of the PDP research group and is responsible for many advances in the subject including the classic back-propagation paper. MR TIMOTHY JERVIS is with Cambridge University Engineering Department. His interests lie in the field of neural networks and in the application of Bayesian statistical techniques to learning control. PROFESSOR MICHAEL JORDAN is in the Department of Brain & Cognitive Science at MIT. He was a founding member of the PDP research group and he made many contributions to the subject particularly in forward and inverse systems. PROFESSOR TEUVO KOHONEN is with the Academy of Finland and Laboratory of Computer and Information Science at Helsinki University of Technology. His specialities are in self-organising maps and their applications. PROFESSOR K S NARENDRA is with the Center for Systems Science in the Electrical Engineering Department at Yale University. His interests are in the control of complex systems using neural networks. DR MAHESAN NIRANJAN is with the Department of Engineering at Cambridge University. His specialities are in speech processing and pattern classification. DR ARTHUR PECE is in the Physiological laboratory at the University of Cambridge. His interests are in biological vision and especially neural network models of cortical vision. DR RICHARD PRAGER is with the Department of Engineering at Cambridge University. His specialities are in speech and vision processing using artificial neural systems. DR RICH SUTTON is with the Adaptive Systems Department of GTE Laboratories near Boston, USA. His specialities are in reinforcement learning, planning and animal learning behaviour. DR LIONEL TARRASENKO is with the Department of Engineering at the University of Oxford. His specialities are in robotics and the hardware implementation of neural computing. COURSE FEES AND ACCOMMODATION The course fee is 750 (UK pounds), payable in advance, and includes full course notes, a certificate of attendance, and lunch and day-time refreshments for the duration of the course. A number of heavily discounted places are available for academics; please contact Renee Taylor if you would like to be considered for one of these places. Accommodation can be arranged for delegates in college rooms with shared facilities at Wolfson College at 163 (UK pounds) for 4 nights to include bed and breakfast, dinner with wine and a Course Dinner. For more information contact: Renee Taylor, Course Development Manager Cambridge Programme for Industry, 1 Trumpington Street, Cambridge CB2 1QA, United Kingdom tel: +44 (0)223 332722 fax +44 (0)223 301122 email: rt10005 at uk.ac.cam.phx From piero at dist.dist.unige.it Fri Jul 9 16:27:16 1993 From: piero at dist.dist.unige.it (Piero Morasso) Date: Fri, 9 Jul 93 16:27:16 MET DST Subject: ICANN'94 First Call for Papers Message-ID: <9307091427.AA08938@dist.dist.unige.it> -------------------------------------------------------------------- | ************************************************ | | * * | | * EUROPEAN NEURAL NETWORK SOCIETY * | | *----------------------------------------------* | | * C A L L F O R P A P E R S * | | *----------------------------------------------* | | * I C A N N ' 94 - SORRENTO * | | * * | | ************************************************ | | | | ICANN'94 (INTERNATIONAL CONFERENCE ON ARTIFICIAL NEURAL NETWORKS)| | is the fourth Annual Conference of ENNS and it comes after | | ICANN'91(Helsinki), ICANN'92 (Brighton), ICANN'93 (Amsterdam). | | It is co-sponsored by INNS, IEEE-NC, JNNS. | | It will take place at the Sorrento Congress Center, near Naples, | | Italy, on May 26-29, 1994. | | | |------------------------------------------------------------------| | S U B M I S S I O N | |------------------------------------------------------------------| | Interested authors are cordially invited to present their work | | in one of the following "Scientific Areas" (A-Cognitive Science; | | B-Mathematical Models; C- Neurobiology; D-Fuzzy Systems; | | E-Neurocomputing), indicating also an "Application domain" | | (1-Motor Control;2-Speech;3-Vision;4-Natural Language; | | 5-Process Control;6-Robotics;7-Signal Processing; | | 8-Pattern Recognition;9-Hybrid Systems;10-Implementation). | | | | DEADLINE for CAMERA-READY COPIES: December 15, 1993. | | ---------------------------------------------------- | | Papers received after that date will be returned unopened. | | Papers will be reviewed by senior researchers in the field | | and the authors will be informed of their decision by the end | | of January 1994. Accepted papers will be included in the | | Proceedings only if the authors have registered in advance. | | Allocation of accepted papers to oral or poster sessions will | | not be performed as a function of technical merit but only with | | the aim of coherently clustering different contributions in | | related topics; for this reason there will be no overlap of | | oral and poster sessions with the same denomination. Conference | | proceedings, that include all the accepted (and regularly | | registered) papers, will be distributed at the Conference desk | | to all regular registrants. | | | | SIZE: 4 pages, including figures, tables, and references. | | LANGUAGE: English. | | COPIES: submit a camera-ready original and 3 copies. | | (Accepted papers cannot be edited.) | | ADDRESS where to send the papers: | | IIASS (Intl. Inst. Adv. Sci. Studies), ICANN'94, | | Via Pellegrino 19, Vietri sul Mare (Salerno), 84019 Italy. | | ADDRESS where to send correspondence (not papers): | | Prof. Roberto Tagliaferri, Dept. Informatics, Univ. Salerno, | | Fax +39 89 822275, email iiass at salerno.infn.it | | EMAIL where to get LaTeX files: listserv at dist.unige.it | | | | In an accompanying letter, the following should be included: | | (i) title of the paper, (ii) corresponding author, | | (iii) presenting author, (iv) scientific area and application | | domain (e.g. "B-7"), (vi) preferred presentation (oral/poster), | | (vii) audio-visual requirements. | | | |------------------------------------------------------------------| | F O R M A T | |------------------------------------------------------------------| | The 4 pages of the manuscripts should be prepared on A4 white | | paper with a typewriter or letter- quality printer in | | one-column format, single-spaced, justified on both sides and | | printed on one side of the page only, without page numbers | | or headers/footers. Printing area: 120 mm x 195 mm. | | | | Authors are encouraged to use LaTeX. For LaTeX users, the LaTeX | | style-file and an example-file can be obtained via email as | | follows: | | - send an email message to the address "listserv at dist.unige.it" | | - the first two lines of the message must be: | | get ICANN94 icann94.sty | | get ICANN94 icann94-example.tex | | If problems arise, please contact the conference co-chair below. | | Non LaTeX users can ask for a specimen of the paper layout, | | to be sent via fax. | | | |------------------------------------------------------------------| | P R O G R A M C O M M I T T E E | |------------------------------------------------------------------| | The preliminary program committee is as follows: | | | | I. Aleksander (UK), D. Amit (ISR), L. B. Almeida (P), | | S.I. Amari (J), E. Bizzi (USA), E. Caianiello (I), | | L. Cotterill (DK), R. De Mori (CAN), R. Eckmiller (D), | | F. Fogelman Soulie (F), S. Gielen (NL), S. Grossberg (USA), | | J. Herault (F), M. Jordan (USA), M. Kawato (J), T. Kohonen (SF), | | V. Lopez Martinez (E), R.J. Marks II (USA), P. Morasso (I), | | E. Oja (SF), T. Poggio (USA), H. Ritter (D), H. Szu (USA), | | L. Stark (USA), J. G. Taylor (UK), S. Usui (J), L. Zadeh (USA) | | | | Conference Chair: Prof. Eduardo R. Caianiello, Univ. Salerno, | | Italy, Dept. Theoretic Physics; email: iiass at salerno.infn.it | | | | Conference Co-Chair: Prof. Pietro G. Morasso, Univ. Genova, | | Italy, Dept. Informatics, Systems, Telecommunication; | | email: morasso at dist.unige.it; fax: +39 10 3532948 | | | |------------------------------------------------------------------| | T U T O R I A L S | |------------------------------------------------------------------| | The preliminary list of tutorials is as follows: | | 1) Introduction to neural networks (D. Gorse), 2) Advanced | | techniques in supervised learning (F. Fogelman Soulie`), | | 3) Advanced techniques for self-organizing maps (T. Kohonen) | | 4) Weightless neural nets (I. Aleksander), 5) Applications of | | neural networks (R. Hecht-Nielsen), 6) Neurobiological modelling | | (J.G. Taylor), 7) Information theory and neural networks | | (M. Plumbley). | | Tutorial Chair: Prof. John G. Taylor, King's College, London, UK | | fax: +44 71 873 2017 | | | |------------------------------------------------------------------| | T E C H N I C A L E X H I B I T I O N | |------------------------------------------------------------------| | A technical exhibition will be organized for presenting the | | literature on neural networks and related fields, neural networks| | design and simulation tools, electronic and optical | | implementation of neural computers, and application | | demonstration systems. Potential exhibitors are kindly requested | | to contact the industrial liaison chair. | | | | Industrial Liaison Chair: Dr. Roberto Serra, Ferruzzi | | Finanziaria, Ravenna, fax: +39 544 35692/32358 | | | |------------------------------------------------------------------| | S O C I A L P R O G R A M | |------------------------------------------------------------------| | Social activities will include a welcome party, a banquet, and | | post-conference tours to some of the many possible targets of | | the area (participants will also have no difficulty to | | self-organize a la carte). | -------------------------------------------------------------------- From mcauley at cs.indiana.edu Tue Jul 13 13:11:20 1993 From: mcauley at cs.indiana.edu (J. Devin McAuley) Date: Tue, 13 Jul 1993 12:11:20 -0500 Subject: TR announcement: Analysis of the Effects of Noise on a Model for the Neural Mechanism of Short-Term Active Memory. Message-ID: <mailman.622.1149591284.29955.connectionists@cs.cmu.edu> FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/mcauley.noise.ps.Z The file mcauley.noise.ps.Z is now available for copying from the Neuroprose repository: Analysis of the Effects of Noise on a Model for the Neural Mechanism of Short-Term Active Memory. (8 pages) J. Devin McAuley and Joseph Stampfli Indiana University ABSTRACT: Zipser (1991) showed that the hidden unit activity of a fully-recurrent neural network model, trained on a simple memory task, matched the temporal activity patterns of memory-associated neurons in monkeys performing delayed saccade or delayed match-to-sample tasks. When noise, simulating random fluctuations in neural firing rate, is added to the unit activations of this model, the effect on the memory dynamics is to slow the rate of information loss. In this paper, we show that the dynamics of the iterated sigmoid function, with gain and bias parameters, is qualitatively very similar to the "output" behavior of Zipser's multi-unit model. Analysis of the simpler system provides an explanation for the effect of noise that is missing from the description of the multi-unit model. ---------------------------------- J. Devin McAuley Artificial Intelligence Laboratory Computer Science Department Indiana University Bloomington, Indiana 47405 ---------------------------------- From rohwerrj at cs.aston.ac.uk Tue Jul 13 12:49:52 1993 From: rohwerrj at cs.aston.ac.uk (rohwerrj) Date: Tue, 13 Jul 93 12:49:52 BST Subject: Research Opportunities in Neural Networks Message-ID: <23020.9307131149@cs.aston.ac.uk> ***************************************************************************** RESEARCH OPPORTUNITIES in NEURAL NETWORKS Dept. of Computer Science and Applied Mathematics Aston University ***************************************************************************** Funding has recently become available for up to 6 PhD studentships and up to 3 postdoctoral fellowships in the Neural Computing Research Group at Aston University. This group is currently undergoing a major expansion with the recent appointments of Professor Chris Bishop (formerly head of the Applied Neurocomputing Centre at AEA Technology, Harwell Laboratory) and Professor David Lowe (formerly head of the neural network research group at DRA, Malvern), joining Professor David Bounds and lecturers Richard Rohwer and Alan Harget. In addition, substantial funds are being invested in new computer hardware and software and other resources, which will provide the Group with extensive research facilities. The research programme of the Group is focussed on the development of neural computing techniques from a sound statistical pattern processing perspective. Research topics span the complete range from developments of the theoretical foundations of neural computing, through to a wide range of application areas. The Group maintains close links with several industrial organisations, and is participating in a number of collaborative projects. For further information, please contact me at the address below: Richard Rohwer Dept. of Computer Science and Applied Mathematics Aston University Aston Triangle Birmingham B4 7ET ENGLAND Tel: (44 or 0) (21) 359-3611 x4688 FAX: (44 or 0) (21) 333-6215 rohwerrj at uk.ac.aston.cs From goodman at unr.edu Mon Jul 12 18:59:14 1993 From: goodman at unr.edu (Phil Goodman) Date: Mon, 12 Jul 93 22:59:14 GMT Subject: POSITION AVAILABLE - STATISTICIAN Message-ID: <9307130559.AA28993@equinox.unr.edu> ******************* Professional Position Announcement ****************** "STATISTICIAN for NEURAL NETWORK & REGRESSION DATABASE RESEARCH" .- - - - - - - - - - - - - - OVERVIEW - - - - - - - - - - - - - - - - -. | | | THE LOCATION: | | Nevada's Reno/Lake Tahoe region is an outstanding environment for | | living, working, and raising a family. Winter skiing is world-class,| | summer recreation includes many mountain and water sports, and | | historical exploration and cultural opportunities abound. | | | | THE PROJECT: | | The new CENTER FOR BIOMEDICAL MODELING RESEARCH recently received | | federal funding to refine and apply a variety of artificial neural | | network algorithms to large cardiovascular health care databases. | | | | THE CHALLENGE: | | The predictive performance of neural nets will be compared to | | advanced regression models. Other comparisons to be made include | | handling of missing and noisy data, and selection of important | | interactions among variables. | | | | THE JOB REQUIREMENT: | | Masters-level or equivalent statistician with working knowledge | | of the SAS statistical package and the UNIX operating system. | | | | THE SALARY : | | Approximate starting annual salary: $42,000 + full benefits . | | (actual salary will depend on experience and qualifications) | ._ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ . POSITION: Research Statistics Coordinator for NEURAL NETWORKS / HEALTH CARE DATABASE PROJECT LOCATION: Center for Biomedical Modeling Research Department of Internal Medicine University of Nevada School of Medicine Washoe Medical Center, Reno, Nevada START DATE: September 1, 1993 CLOSING DATE: Open until filled. DESCRIPTION: Duties include acquisition and translation of data from multiple external national sources; data management and archiving; performance of exploratory and advanced regression statistics; performance of artificial neural network processing; participation in scholarly research and publications. QUALIFICATIONS: (1) M.S., M.A., M.P.H. or equivalent training in statistics with experience in logistic and Cox regression analyses, (2) ability to program in the SAS statistical language, and (3) experience with UNIX computer operating systems. Desirable but not mandatory are the abilities to use (4) the S-PLUS data management system and (5) the C programming language. SALARY: Commensurate with qualifications and experience. (For example, with database experience, typical annual salary would be approximately $42,000 + full benefits.) APPLICATION: > Informal inquiry may be made to: Phil Goodman, Director, Center for Biomedical Modeling Research Internet: goodman at unr.edu Phone: 702-328-4867 > Formal consideration requires a letter of application, vita, and names of three references sent to: Philip Goodman, MD, MS Director, Center for Biomedical Modeling Research University of Nevada School of Medicine Washoe Medical Center, Room H1-166 77 Pringle Way, Reno, NV 89520 The University of Nevada is an Equal Opportunity/Affirmative Action employer and does not discriminate on the basis of race, color, religion, sex, age, national origin, veteran's status or handicap in any program it operates. University of Nevada employs only U.S. citizens and aliens lawfully authorized to work in the United States. ************************************************************************ From smieja at nathan.gmd.de Wed Jul 14 14:07:06 1993 From: smieja at nathan.gmd.de (Frank Smieja) Date: Wed, 14 Jul 1993 20:07:06 +0200 Subject: TR announcement: reflective agent teams Message-ID: <199307141807.AA19358@trillian.gmd.de> The file beyer.teams.ps.Z is available for copying from the Neuroprose repository: Learning from Examples, Agent Teams and the Concept of Reflection (22 pages) Uwe Beyer and Frank Smieja GMD, Germany ABSTRACT. Learning from examples has a number of distinct algebraic forms, depending on what is to be learned from which available information. One of these forms is $x \stackrel{G}{\rightarrow} y$, where the input--output tuple $(x,y)$ is the available information, and $G$ represents the process determining the mapping from $x$ to $y$. Various models, $y = f(x)$, of $G$ can be constructed using the information from the $(x,y)$ tuples. In general, and for real-world problems, it is not reasonable to expect the exact representation of $G$ to be found (i.e.\ a formula that is correct for all possible $(x,y)$). The modeling procedure involves finding a satisfactory set of basis functions, their combination, a coding for $(x,y)$ and then to adjust all free parameters in an approximation process, to construct a final model. The approximation process can bring the accuracy of the model to a certain level, after which it becomes increasingly expensive to improve further. Further improvement may be gained through constructing a number of agents $\{\alpha\}$, each of which develops its own model $f_\alpha$. These may then be combined in a second modeling phase to synthesize a {\it team\/} model. If each agent has the ability of internal {\it reflection\/} the combination in a team framework becomes more profitable. We describe reflection and the generation of a {\it confidence\/} function: the agent's estimate of the correctness of each of its predictions. The presence of reflective information is shown to increase significantly the performance of a team. -Frank Smieja Gesellschaft fuer Mathematik und Datenverarbeitung (GMD) GMD-FIT.KI.AS, Schloss Birlinghoven, 5205 St Augustin 1, Germany. Tel: +49 2241-142214 email: smieja at gmd.de From luttrell at signal.dra.hmg.gb Thu Jul 15 06:15:17 1993 From: luttrell at signal.dra.hmg.gb (luttrell@signal.dra.hmg.gb) Date: Thu, 15 Jul 93 11:15:17 +0100 Subject: TR announcement: Bayesian Self-Organisation Message-ID: AA08529@mentor.dra.hmg.gb FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/luttrell.bayes-selforg.ps.Z The file luttrell.bayes-selforg.ps.Z is now available for copying from the Neuroprose repository: A Bayesian Analysis of Self-Organising Maps (24 pages) Stephen P Luttrell Defence Research Agency, United Kingdom ABSTRACT: In this paper Bayesian methods are used to analyse some of the properties of a special type of Markov chain. The forward transitions through the chain are followed by inverse transitions (using Bayes' theorem) backwards through a copy of the same chain; this is called a folded Markov chain. If an appropriately defined Euclidean error (between the original input and its "reconstruction" via Bayes' theorem) is minimised in the space of Markov chain transition probabilities, then the familiar theories of both vector quantisers and self-organising maps emerge. This approach is also used to derive the theory of self-supervision, in which the higher layers of a multi-layer network supervise the lower layers, even though overall there is no external teacher. Steve Luttrell Adaptive Systems Theory Section DRA, St Andrews Road, Malvern, Worcestershire, WR14 3PS, UK email: luttrell at signal.dra.hmg.gb Tel: +44-684-894046 Fax: +44-684-894384 From jrf at psy.ox.ac.uk Mon Jul 19 10:20:06 1993 From: jrf at psy.ox.ac.uk (John Fletcher) Date: Mon, 19 Jul 1993 15:20:06 +0100 Subject: postdoctoral position Message-ID: <9307191420.AA13411@axpbb.cns.ox.ac.uk> UNIVERSITY OF OXFORD MRC RESEARCH CENTRE IN BRAIN AND BEHAVIOUR Scientist for Neural Network Applications Applications are invited for a postdoctoral position to work on the operation of neuronal networks in the brain, with special reference to cortical computation. The post available is for a theoretician to perform analytic and/or simulation work colla- boratively with experimental neuroscientists on biologically realistic models of computation in the brain, including systems involved in vision, memory and/or motor function. The appoint- ment will be for three years in the first instance. The salary is on the RSIA scale, GBP 12,638-20,140, and is funded by the Medical Research Council. Further particulars are attached. Applications, including the names of two referees, to The Administrator, Department of Exper- imental Psychology, South Parks Road, Oxford OX1 3UD. The University is an Equal Opportunities Employer. ---- UNIVERSITY OF OXFORD MRC RESEARCH CENTRE IN BRAIN AND BEHAVIOUR Neural Network Scientist (RSIA Scale) The purpose of this post is to enable a formal modeller of neu- ronal networks to work on a daily basis with neuroscientists in order to develop models of cortical function, and to provide the neuroscientists in the MRC Research Centre in Brain and Behaviour with advice and assistance in relation to the expertise required for analytic models of neuronal networks. The postholder should have a PhD or equivalent experience, to- gether with evidence that he/she can initiate a research pro- gramme. The postholder is expected to have expertise in the mathematical analyses of neuronal networks. He/she will provide mathematical and related computational expertise required for the modelling of real neuronal networks, the architecture of which will be based on anatomical information about neuronal connec- tions and physiological information about their modifiability. He/she will probably have the opportunity to supervise graduate students reading for the DPhil degree. In addition, the postholder should also have expertise useful in enabling him or her to analyse biologically relevant models of neural networks. This latter requirement implies either existing expertise in neuroscience, or an aptitude for working with neu- roscientists to ensure that biological constraints are correctly incorporated into network models. The postholder will be sufficiently senior to be able to initiate independent analyses of neural networks, and to ensure that results are brought to publication. Since an important part of the Research Centre is training, the postholder will be expected to take part in the organization of seminar series on neural networks. It is anticipated that these seminars will be held at least annually and will provide training for both doctoral students and for other more senior members of the University interested in research on neural networks. Other teaching may include participation in a summer school. Finally, the postholder, who will be responsible to the Directors of the Research Centre, will be expected to serve on a Working Group of the Research Centre interested in computational analyses of neuronal networks. Examples of neural network projects in the Research Centre in- clude the following: Dr J F Stein/Dr R C Miall, University Laboratory of Physiology: (a)To reconsider the Zipser & Anderson network model of head- centred encoding of target positions by visual cells in the pos- terior parietal cortex. Their model was very successful at demonstrating how the hidden units within a standard back- propagation neural network could end up with receptive field pro- perties similar to those recorded in the PPC in awake monkeys. However, as the responses (in both monkey and model) were complex and hard to classify, it remains unclear whether there was a true similarity between the two or whether the solutions were merely equally hard to interpret. We plan to re-examine this model us- ing more realistic constraints on the model connectivity and on the encoding of its inputs, and should be taking on a new post- graduate student this October to pursue these questions. (b)To develop models of learning at the cerebellar parallel-fibre to Purkinje cell synapse. We have proposed a model of the cere- bellum in which the cortex learns to form a forward model of the limb dynamics to assist in visually guided arm movement. We have proposed that there is a mechanism related to reinforcement learning, but the details remain unclear. It is important to see whether the biophysics of long-term depression and the ideas of reinforcement learning can be tied together in a neural simula- tion. References: Barto AG, Sutton RS & Anderson CW (1983). Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Sys Man Cyb., 13: 834-846. Ito M (1989). Long-term depression. Ann. Rev. Neurosci. 12:85- 102. Miall RC, Weir DJ, Wolpert DM & Stein JF (1993). Is the cerebel- lum a Smith Predictor? J.motor Behaviour (in press). Zipser D & Andersen RA (1988). A back-propagation programmed network that simulates response properties of a subset of poste- rior parietal neurons. Nature 331:679-684. Dr K Plunkett, Department of Experimental Psychology: Kim Plunkett's research group is involved in computational and experimental investigations of language acquisition and cognitive development. Modelling work has covered connectionist simula- tions of inflectional morphology, concept formation, vocabulary development and early syntax. Experimental work focuses on the processes of lexical segmentation in early language acquisition. References: Plunkett K & Marcham V (1991). U-shaped learning and frequency effects in a multi-layered perceptron: Implications for child language acquisition. Cognition, 38, 43-102. Plunkett K & Sinha CG (1992). Connectionism and developmental theory. British Journal of Developmental Psychology, 10, 209- 254. Plunkett K (1993). Lexical segmentation and vocabulary growth in early language acquisition. Journal of Child Language, 20, 43-60. Professor D Sherrington, Department of Theoretical Physics: Our interest is in understanding and quantifying the design, per- formance and training of neural network models, stimulated by their potential as cartoons of parts of the brain, as expert sys- tems and as complex cooperative systems. Our methodology in- volves the application of analytical and computational techniques from the theoretical physics of strongly interacting systems and centres largely around issues of statistical relevance, as op- posed to worst-case or special-case analyses. References: Recent reviews: *Sherrington D. Neural Networks: the spinglass approach. OUTP- 92-485. *Coolen T & Sherrington D. Dynamics of attractor neural net- works. OUTP-92-495. [The above are to be published in Mathematical Studies of Neural Networks (Elsevier); ed. JG Taylor.] Watkin TLH, Rau A & Biehl M. Statistical Mechanics of Learning a Rule. OUTP-92-45S published in Reviews of Modern Physics 65, pp 499-556 (1993). *Copies of these are available for interested candidates. Published research articles (selection) Wong KYM, Kahn PE & Sherrington D. A neural network model of working memory exhibiting primacy and recency. J. Phys. A24, 1119 (1991). Sherrington D, Wong M & Rau A. Good Memories. Phil Mag. 65, 1303 (1992). Sherrington D, Wong KYM & Coolen ACC. Noise and competition in neural networks. J.Phys I France 3, 331 (1993). O'Kane D & Sherrington D. A feature-retrieving attractor neural network. J.Phys A 26, 2333 (1993). Dr E T Rolls, Department of Experimental Psychology: (a) Learning invariant responses to the natural transformations of objects. The primate visual system builds representations of objects which are invariant with respect to transforms such as translation, size, and eventually view, in a series of hierarchi- cal cortical areas. To clarify how such a system might learn to recognise "naturally" transformed objects, we are investigating a model of cortical visual processing which incorporates a number of features of the primate visual system. The model has a series of layers with convergence from a limited region of the preceding layer, and mutual inhibition over a short range within a layer. The feedforward connections between layers provide the inputs to competitive networks, each utilising a modified Hebb-like learn- ing rule which incorporates a temporal trace of the preceding neuronal activity. The trace learning rule is aimed at enabling the neurons to learn transform invariant responses via experience of the real world, with its inherent spatio-temporal constraints. We are showing that the model can learn to produce translation invariant responses, and plan to develop this neural network model to investigate its performance in learning other types of invariant representation, and its capacity. Rolls E (1992). Neurophysiological mechanisms underlying face processing within and beyond the temporal cortical areas. Phil. Trans. Roy. Society London Ser B 335, 11-21. Wallis G, Rolls E & Foldiak P (1993). In: Proc Int Joint Conference on Neural Networks. (b) Neural Networks in the Hippocampus involved in Memory and Recall. We are developing a model based on hippocampal anatomy, physiology, and psychology, of how the neural networks in the hippocampus could operate in memory. A key hypothesis is that the hippocampal CA3 circuitry forms an autoassociation memory. We are developing a quantitative theory of how the CA3 could operate, and of how it would function in relation to other parts of the hippocampus. We are also starting to develop a theory of how the hippocampus could recall memories in the cerebral neocor- tex using backprojections in the cerebral cortex. Rolls ET (1989a). Functions of neuronal networks in the hippocampus and neocortex in memory. In Neural models of plasti- city: Experimental and theoretical approaches (ed. JH Byrne & WO Berry), 13, 240-265). San Diego: Academic Press. Rolls ET (1990a). Theoretical and neurophysiological analysis of the functions of the primate hippocampus in memory. Cold Spring Harbor Symposia in Quantitative Biology 55, 995-1006. Treves A & Rolls ET (1991). What determines the capacity of autoassociative memories in the brain? Network 2: 371-397. Treves A & Rolls ET (1992). Computational constraints sug- gest the need for two distinct input systems to the hippocampal CA3 network. Hippocampus 2: 189-199. Rolls ET & Treves A (1993). Neural Networks in the Brain involved in Memory and Recall. In: Proc. Int. Joint Conference on Neural Networks. We would welcome collaboration on either or both projects. From mel at cns.caltech.edu Mon Jul 19 13:08:41 1993 From: mel at cns.caltech.edu (Bartlett Mel) Date: Mon, 19 Jul 93 10:08:41 PDT Subject: Neural Simulation Demonstration Message-ID: <9307191708.AA00449@plato.cns.caltech.edu> ANNOUNCEMENT FOR NEURAL SIMULATION DEMONSTRATIONS AT THE SOCIETY FOR NEUROSCIENCE ANNUAL MEETING 1993 Sponsored by the National Science Foundation This announcement contains information on an opportunity to demonstrate state of the art computer simulation techniques in computational neuroscience at the annual Society For neuroscience meeting to be held in Washington DC during the second week of November. Please distribute this announcement to other groups which might be interested in demonstrating their software. Purpose The primary purpose of these demonstrations will be to expose large numbers of neuroscientists to the use of computer simulation techniques in neurobiology. In addition, we want to provide technical information on simulation methods, the range of simulation software currently available, and demonstrations of several aspects of new computer technology that are relevant to efforts to simulate and understand the brain. Format The Neural Simulation Technology Demonstration will consist of three separate components. First, a series of simulation software demonstrations while be organized to provide a balanced and as complete as possible view of available non commercial software. Second, machines will be made available to participants in the meeting whose regular poster or oral presentations involve simulations so that they can demonstrate the simulations on which their paper presentations are based. Finally, a separate demonstration will focus on state of the art simulation technology including inter-computer communication, concurrent computing, and graphical visualization. Simulation software demonstrations This announcement seeks applicants for the simulation software demonstrations. This demonstration will be organized very much like a regular poster session, except that computers will be available for actual demonstrations of the modeling software and each demonstration will remain in place throughout the meeting. The session itself will be set up within the exhibit space of the meeting. Each presenter will be required to construct a poster as well as demonstrate their software on a computer. The poster presentations are intended to provide those interested in the simulators an understanding of what they will see when they actually look at the software on the adjacent computer. As such, the poster should include descriptions of the simulator itself, what it is best used for, and how it is used. We will provide an overview of the demonstrated software packages as a handout. We would also like to provide some basis of comparison between the simulators. For this reason we are asking that groups demonstrating unix software for compartmental modeling present the quantitative results obtained from the first three standard simulations in the Rallpack standards (Bhalla et al., Trends in Neurosci. 15: 453-458, 1992), which can be obtained by anonymous ftp from mordor.cns.caltech.edu What to do If you are interested in being considered as a simulations software demonstrator, you should fill out the application attached to the end of this document and return it to erik at cns.caltech.edu. You should note that the National Science Foundation has stipulated that applications cannot be accepted from vendors of commercial software, but only from "not for profit" concerns. Applications will be reviewed as they are received, deadline is October 1, 1993. Invitations will then be send out immediately based on this review. Criteria for acceptance of demonstrations will include the apparent maturity of the effort as well as evidence that the simulation software is being or could be used outside of the laboratory in which it was developed. Some packages are well known and established (De Schutter, Trends in Neurosci. 15: 462-464, 1992) and will be accepted with minimal review (i.e. submit application by e-mail). Applications for newer packages should send additional documentation (like manuals, information sheets, papers,...) by mail to help in the evaluation. Arrangements In order to increase the likely success of the demonstrations, each accepted participant will be asked toship their own computer to the meeting. This will avoid difficulties with hardware incompatibilities, or novel system configurations. Our technical staff will aid with setting up the machines at each meeting. Support Accepted participants will be provided support on a reimbursement basis in the following forms: 1) computer shipping expenses 2) reimbursement of meeting registration costs for one person designated as responsible for the software demonstration. For further information contact: Jim Bower or Erik De Schutter Div. of Biology 216-76 Caltech Pasadena, CA 91125 jbower at cns.caltech.edu, erik at cns.caltech.edu FAX 1-818-7952088 -------------------------------------------------------------------------- APPLICATION FORM This form can be e-mailed to erik at cns.caltech.edu, or mailed to Erik De Schutter, Div. of Biology 216-76, Caltech, Pasadena, CA 91125 APPLICANT Name: Title: Address: E-mail: PERSON DEMONSTRATING THE SOFTWARE (if different from applicant) Name: Title: Address: E-mail: COMPUTER USED FOR THE DEMONSTRATION: Type: Operating system: SOFTWARE PACKAGE Purpose (80 characters or less): Author: Operating system(s): Hardware platforms on which code has been tested: Parallel implementation (if yes, which platform)? Programming language source code: Source code available? Current version number: When was this version released? Continued development/upgrades of the package? How many programmers involved in continued development? User expandable (yes only if explicitly supported by the code): Interface type (answer yes or no): all features in a single program? interactive model creation/editing? compartments/channels described by arbitrary codes/indexes? compartments/channels described by user selected names? script language? Mac type interface (dialogs, buttons, popup menus)? graphic interface (point to dendrite to select etc.)? object-oriented approach? supports grouping of objects for easy selection? File I/O for model description: reads morphology files? saves interactively created models? reads file formats other software packages (list names): outputs file formats other software packages (list names): I/O of results (answer yes or no): file output? graph versus time during simulation? phase plots? Sholl plots? color representation of cell ('schematic')? color representation of cell ('3-dim')? output multiple variables during same run? output any computed variable? incorporates analysis routines for results? Simulator features: integration method(s): are Rallpack data available? variable time step? automatic detection of model setup errors? storage of initial values for variables? model size limited only by available memory? Neuron modeling features (answer yes or no): integrate-and-fire model? compartmental model? detailed morphology of dendritic/axonal trees? standard Hodgkin-Huxley channels? other channel types using Hodgkin-Huxley type equations? other channel types using any equation? Ca2+-dependent (in)activation of channels? synaptic channels (alpha equation)? synaptic plasticity? simple concentration pools (Ca2+, exponential decay)? complex concentration pools (buffers, diffusion)? second messengers, enzymatic kinetics? Network modeling features (answer yes or no): small network (<= 100 cells)? big networks (> 100 cells) mixed cell model types possible? automatic/random generation of connections? network learning (e.g. backprop)? synaptic/axonal delays implemented? network growth during simulations? Manual: is the manual complete (i.e. describes all program features, commands)? number of pages in manual: does the manual contain screen dumps? reference manual only or also tutorials? Other documentation: On line help? Tutorials? Classes? Papers? Distribution: Compiled program on disk/tape? Compiled program by ftp? As source code by ftp? Fee (how much)? Free upgrades? Upgrades by ftp? Available to foreign users? User support: E-mail support? Telephone support? Newsletter? Users group? USERS How long has this package been available? How many upgrades have been released? How many scientists use this package in your group? How many other groups in the USA use this package? How many foreign groups use this package? Optional and confidential: list 3 users (including e-mail address or FAX) who currently use the software package and who have never worked in your laboratory. REFERENCES: List max 2 publications describing this software package. List max 5 publications describing simulation results obtained with this software package. CONTACT ADDRESS FOR PROSPECTIVE USERS: Preferred method of contact: Address: E-mail: FAX: From mel at cns.caltech.edu Mon Jul 19 15:44:51 1993 From: mel at cns.caltech.edu (Bartlett Mel) Date: Mon, 19 Jul 93 12:44:51 PDT Subject: PostScript NIPS*93 Brochure Message-ID: <9307191944.AA00683@plato.cns.caltech.edu> ************ PLEASE POST **************** An electronic copy of the 1993 NIPS registration brochure is available in PostScript format via anonymous ftp. Instructions for retrieving and printing the brochure are as follows: unix> ftp helper.systems.caltech.edu (or ftp 131.215.68.12) Name: anonymous Password: your-login-name ftp> cd /pub/nips ftp> binary ftp> get NIPS_93_brochure.ps.Z ftp> quit unix> uncompress NIPS_93_brochure.ps.Z unix> lpr NIPS_93_brochure.ps If you would like a hardcopy of the brochure or need other information, please send a request to nips93 at systems.caltech.edu or the following address: NIPS Foundation P.O. Box 60035 Pasadena, CA 91116-6035 From mwitten at hermes.chpc.utexas.edu Mon Jul 19 19:17:09 1993 From: mwitten at hermes.chpc.utexas.edu (mwitten@hermes.chpc.utexas.edu) Date: Mon, 19 Jul 93 18:17:09 CDT Subject: COMPUTATIONAL HEALTH (long) Message-ID: <9307192317.AA11053@morpheus.chpc.utexas.edu> Preliminary Announcement FIRST WORLD CONGRESS ON COMPUTATIONAL MEDICINE, PUBLIC HEALTH AND BIOTECHNOLOGY 24-28 April 1994 Hyatt Regency Hotel Austin, Texas ----- (Feel Free To Cross Post This Announcement) ---- 1.0 CONFERENCE OVERVIEW: With increasing frequency, computational sciences are being exploited as a means with which to investigate biomedical processes at all levels of complexity; from molecular to systemic to demographic. Computational instruments are now used, not only as exploratory tools but also as diagnostic and prognostic tools. The appearance of high performance computing environments has, to a great extent, removed the problem of increasing the biological reality of themathematical models. For the first time in the historyof the field, practical biological reality is finally within the grasp of the biomedical modeler. Mathematical complexity is no longer as serious an issue as speeds of computation are now of the order necessary to allow extremely large and complex computational models to be analyzed. Large memory machines are now routinely available. Additionally, high speed, efficient, highly optimized numerical algorithms are under constant development. As these algorithms are understood and improved upon, many of them are transferred from software implementation to an implementation in the hardware itself; thereby further enhancing the available computational speed of current hardware. The purpose of this congress is to bring together a transdisciplinary group of researchers in medicine, public health, computer science, mathematics, nursing, veterinary medicine, ecology, allied health, as well as numerous otherdisciplines, for the purposes of examining the grand challenge problems of the next decades. This will be a definitive meeting in that it will be the first World Congress of its type and will be held as a followup tothe very well received Workshop On High Performance Computing In The Life Sciences and Medicine held by the University of Texas System Center For High Performance Computing in 1990. Young scientists are encouraged to attend and to present their work in this increasingly interesting discipline. Funding is being solicited from NSF, NIH, DOE, Darpa, EPA, and private foundations, as well as other sources to assist in travel support and in the offsetting of expenses for those unable to attend otherwise. Papers, poster presentations, tutorials, focussed topic workshops, birds of a feather groups, demonstrations, and other suggestions are also solicited. 2.0 CONFERENCE SCOPE AND TOPIC AREAS: The Congress hasa broad scope. If you are not sure as to whether or not your subject fits the Congress scope, contact the conference organizers at one of the addresses below. Subject areas include but are not limited to: *Visualization/Sonification --- medical imaging --- molecular visualization as a clinical research tool --- simulation visualization --- microscopy --- visualization as applied to problems arising in computational molecular biology and genetics or other non-traditional disciplines *Computational Molecular Biology and Genetics --- computational ramifications of clinical needs in the Human Genome, Plant Genome, and Animal Genome Projects --- computational and grand challenge problems in molecular biology and genetics --- algorithms and methodologies --- issues of multiple datatype databases *Computational Pharmacology, Pharmacodynamics, Drug Design *Computational Chemistry as Applied to Clinical Issues *Computational Cell Biology, Physiology, and Metabolism --- Single cell metabolic models (red blood cell) --- Cancer models --- Transport models --- Single cell interaction with external factors models (laser, ultrasound, electrical stimulus) *Computational Physiology and Metabolism --- Renal System --- Cardiovascular dynamics --- Liver function --- Pulmonary dynamics --- Auditory function, coclear dynamics, hearing --- Reproductive modeling: ovarian dynamics, reproductive ecotoxicology, modeling the hormonal cycle --- Metabolic Databases and metabolic models *Computational Demography, Epidemiology, and Statistics/Biostatistics --- Classical demographic, epidemiologic, and biostatistical modeling --- Modeling of the role of culture, poverty, and other sociological issues as they impact healthcare *Computational Disease Modeling --- AIDS --- TB --- Influenza --- Statistical Population Genetics Of Disease Processes --- Other *Computational Biofluids --- Blood flow --- Sperm dynamics --- Modeling of arteriosclerosis and related processes *Computational Dentistry, Orthodontics, and Prosthetics *Computational Veterinary Medicine --- Computational issues in modeling non-human dynamics such as equine, feline, canine dynamics (physiological/biomechanical) *Computational Allied Health Sciences --- Physical Therapy --- Neuromusic Therapy --- Resiratory Therapy *Computational Radiology --- Dose modeling --- Treatment planning *Computational Surgery --- Simulation of surgical procedures in VR worlds --- Surgical simulation as a precursor to surgical intervention *Computational Cardiology *Computational Nursing *Computational Models In Chiropractice *Computational Neurobiology and Neurophysiology --- Brain modeling --- Single neuron models --- Neural nets and clinical applications --- Neurophysiological dynamics --- Neurotransmitter modeling --- Neurological disorder modeling (Alzheimers Disease, for example) *Computational Models of Psychiatric and Psychological Processes *Computational Biomechanics --- Bone Modeling --- Joint Modeling *Computational Models of Non-tradional Medicine --- Acupuncture --- Other *Computational Issues In Medical Instrumentation Design and Simulation --- Scanner Design --- Optical Instrumentation *Ethical issues arising in the use of computational technology in medical diagnosis and simulation *The role of alternate reality methodologies and high performance environments in the medical and public health disciplines *Issues in the use of high performance computing environments in the teaching of health science curricula *The role of high performance environments for the handling of large medical datasets (high performance storage environments, high performance networking, high performance medical records manipulation and management, metadata structures and definitions) *Federal and private support for transdisciplinary research in computational medicine and public health 3.0 CONFERENCE COMMITTEE *CONFERENCE CHAIR: Matthew Witten, UT System Center For High Performance Computing, Austin, Texas m.witten at chpc.utexas.edu *CONFERENCE DIRECTORATE: Regina Monaco, Mt. Sinai Medical Center * Dan Davison, University of Houston * Chris Johnson, University of Utah * Lisa Fauci, Tulane University * Daniel Zelterman, University of Minnesota Minneapolis * James Hyman, Los Alamos National Laboratory * Richard Hart, Tulane University * Dennis Duke, SCRI-Florida State University * Sharon Meintz, University of Nevada Los Vegas * Dean Sittig, Vanderbilt University * Dick Tsur, UT System CHPC * Dan Deerfield, Pittsburgh Supercomputing Center * Istvan Gyori, Szeged University School of Medicine Computing Center * Don Fussell, University of Texas at Austin * Ken Goodman, University Of Miami School of Medicine * Martin Hugh-Jones, Louisiana State University * Stuart Zimmerman, MD Anderson Cancer Research Center * John Wooley, DOE * Sylvia Spengler, University of California Berkeley, Robert Blystone, Trinity University Additional conference directorate members are being added and will be updated on the anonymous ftp list as they agree. 4.0 CONTACTING THE CONFERENCE COMMITTEE: To contact the congress organizers for any reason use any of the following pathways: ELECTRONIC MAIL - compmed94 at chpc.utexas.edu FAX (USA) - (512) 471-2445 PHONE (USA) - (512) 471-2472 GOPHER:log into the University of Texas System-CHPC select the Computational Medicine and Allied Health menu choice ANONYMOUS FTP: ftp.chpc.utexas.edu cd /pub/compmed94 POSTAL: Compmed 1994 University of Texas System CHPC Balcones Research Center, 1.154CMS 10100 Burnet Road Austin, Texas 78758-4497 5.0 SUBMISSION PROCEDURES: Authors must submit 5 copies of a single-page 50-100 word abstract clearly discussing the topic of their presentation. In addition, authors must clearly state their choice of poster, contributed paper, tutorial, exhibit, focussed workshop or birds of a feather group along with a discussion of their presentation. Abstracts will be published as part of the preliminary conference material. To notify the congress organizing committee that you would like to participate and to be put on the congress mailing list, please fill out and return the form that follows this announcement. You may use any of the contact methods above. If you wish to organize a contributed paper session, tutorial session,focussed workshop, or birds of a feather group, please contact the conference director at mwitten at chpc.utexas.edu *CONFERENCE DEADLINES: The following deadlines should be noted: 1 October 1993 - Notification of interest in participation and/or intent to organize a special session 1 November 1993 - Abstracts for talks/posters/ workshops/birds of a feather sessions/demonstrations 15 January 1994 - Notification of acceptance of abstract 15 February 1994 - Application for financial aid 6.0 CONFERENCE PRELIMINARY DETAILS AND ENVIRONMENT LOCATION: Hyatt Regency Hotel, Austin, Texas, USA DATES: 24-28 April 1994 The 1st World Congress On Computational Medicine, Public Health, and Biotechnology will be held at the Hyatt Regency Hotel, Austin, Texas located in downtown Austin. The hotel is approximately 15 minutes from Robert Meuller Airport. Austin, the state capital, is renouned for its natural hill-country beauty and an active cultural scence. Several hiking and jogging trails are within walking distance of the hotel, as well as opportunities for a variety of aquatic sports. Live bands perform in various nightclubs around the city and at night spots along Sixth Street, offering a range of jazz, blues, country/Western, reggae, swing, and rock music. Day temperatures will be in the 80-90(degree F) range and fairly humid. Exhibitor and vendor presentations are also being planned. 7.0 CONFERENCE ENDORSEMENTS AND SPONSORSHIPS: Numerous potential academic sponsors have been contacted. Currently negotiations are underway for sponsorship with SIAM, AMS, MAA, IEEE, FASEB, and IMACS. Additionally AMA and ANA continuing medical education support is beging sought. Information will be updated regularly on the anonymous ftp site for the conference (see above). ================== INTENT TO PARTICIPATE ============= First Name: Middle Initial (if available): Family Name: Your Professional Title: [ ]Dr. [ ]Professor [ ]Mr. [ ]Mrs. [ ]Ms. [ ]Other:__________________ Office Phone (desk): Office Phone (message): Home/Evening Phone (for emergency contact): Fax: Electronic Mail (Bitnet): Electronic Mail (Internet): Postal Address: Institution or Center: Building Code: Mail Stop: Street Address1: Street Address2: City: State: Country: Zip or Country Code: Please list your three major interest areas: Interest1: Interest2: Interest3: ===================================================== From wray at ptolemy.arc.nasa.gov Mon Jul 19 23:51:27 1993 From: wray at ptolemy.arc.nasa.gov (Wray Buntine) Date: Mon, 19 Jul 93 20:51:27 PDT Subject: committees, agent teams, redundancy, Monte Carlo, ... Message-ID: <9307200351.AA03081@ptolemy.arc.nasa.gov> Some recent postings on connectionists have spurred me to draw a few comparisons with earlier work. I'm always amazed at when a new discovery is made, about 10 people will rediscover variations of the same thing and often times be too busy (and admirably) exploring the frontiers of research to relate their work to the current pool of research. The following stuff may seem different when you get into the fine detail, but they look mighty similar from a distance. committees (Thodberg '93, MacKay '93, both in connectionists) agent teams (Beyer & Smieja in connectionists as beyer.teams.ps), stacked generalization (Wolpert in Neural networks 5(2) '92, and Breiman in a TR from UC Berkeley Stats, '93) model averaging (Buntine and Weigend, Complex Systems, 5(1), '91, and on learning decision trees see Buntine, Statistics and Computing, v2, '92 who got the idea from S. Wok and C. Carter in UAI-87) Monte Carlo methods (surely this stuff is related!! see recent papers by Radford Neal, e.g. NIPS5) error correcting codes (Dietterich and Bakiri, AAAI-91) redundant knowledge (M. Gams, 4th EWSL '89) + probably lots more e.g MacKay's version of committee's which got the energy prediction prise smells like Breiman's version of Wolpert's stacked generalization e.g. Thodberg's committee's is a clever & pragmatic implementation of the model averaging approach suggested in Buntine and Weigend which itself is a cheap description of a standard Bayesian trick e.g. in the statistical community i'm told "model uncertainty" is all the rage in some circles, i.e. you don't return a single network but several for comparison and early work goes back many decades I find this fascinating because a few years ago we were all rediscovering smoothing (weight decay, weight elimination, regularisation, MDL, early stopping, cost complexity tradeoffs, etc., etc. etc.) in its various shapes and forms. Now we all seem to be rediscovering the use of multiple models, i.e. the next step in sophisticated learning algorithms. NB. i use the term "rediscovering" because I wouldn't dare attribute the discovery to any one individual !!! Nice work !! What's next ? ------------ Wray Buntine NASA Ames Research Center phone: (415) 604 3389 Mail Stop 269-2 fax: (415) 604 3594 Moffett Field, CA, 94035-1000 email: wray at kronos.arc.nasa.gov From terry at helmholtz.sdsc.edu Tue Jul 20 03:15:56 1993 From: terry at helmholtz.sdsc.edu (Terry Sejnowski) Date: Tue, 20 Jul 93 00:15:56 PDT Subject: Neural Computation 5:4 Message-ID: <9307200715.AA06550@helmholtz.sdsc.edu> Neural Computation, Volume 5, Number 4, July 1993 Review: The Use of Neural Networks in High Energy Physics Bruce Denby Articles: Stimulus Dependent Synchronization of Neuronal Assemblies E. R. Grannan, D. Kleinfeld and H. Sompolinsky Letters: Dynamics of Populations of Integrate-and-fire Neurons, Partial Synchronization and Memory Marius Usher, Heinz Georg Schuster and Ernst Niebur The Effects of Cell Duplication and Noise in a Pattern Generating Network Catherine H. Ferrar and Thelma L. Williams Emergence of Position-independent Detectors of Sense of Rotation and Dilation with Hebbian Learning: An Analysis Kechen Zhang, Martin I. Sereno and Margaret E. Sereno Improving Generalisation for Temporal Difference Learning: The Successor Representation Peter Dayan Discovering Predictable Classifications Jurgen Schmidhuber and Daniel Prelinger A Kinetic Model of Short- and Long-Term Potentiation M. Migliore and G. F. Ayala Artificial Dendritic Trees John G. Elias ----- SUBSCRIPTIONS - VOLUME 5 - BIMONTHLY (6 issues) ______ $40 Student ______ $65 Individual ______ $156 Institution Add $22 for postage and handling outside USA (+7% GST for Canada). (Back issues from Volumes 1-4 are regularly available for $28 each to institutions and $14 each for individuals Add $5 for postage per issue outside USA (+7% GST for Canada) MIT Press Journals, 55 Hayward Street, Cambridge, MA 02142. Tel: (617) 253-2889 FAX: (617) 258-6779 e-mail: hiscox at mitvma.mit.edu ----- From tommi at psyche.mit.edu Wed Jul 21 18:26:38 1993 From: tommi at psyche.mit.edu (Tommi Jaakkola) Date: Wed, 21 Jul 93 18:26:38 EDT Subject: Tech report available Message-ID: <9307212226.AA04323@psyche.mit.edu> THe following paper is now available on the neuroprose archive as "jaakkola.convergence.ps.Z". On the Convergence of Stochastic Iterative Dynamic Programming Algorithms Tommi Jaakkola Michael I. Jordan Department of Brain and Cognitive Sciences Massachusetts Institute of Technology Satinder P. Singh Department of Computer Science University of Massachusetts at Amherst Recent developments in the area of reinforcement learning have yielded a number of new algorithms for the prediction and control of Markovian environments. These algorithms, including the TD($\lambda$) algorithm of Sutton (1988) and the Q-learning algorithm of Watkins (1989), can be motivated heuristically as approximations to dynamic programming (DP). In this paper we provide a rigorous proof of convergence of these DP-based learning algorithms by relating them to the powerful techniques of stochastic approximation theory via a new convergence theorem. The theorem establishes a general class of convergent algorithms to which both TD($\lambda$) and Q-learning belong. From elsberry at beta.tricity.wsu.edu Wed Jul 21 12:39:44 1993 From: elsberry at beta.tricity.wsu.edu (Wesley Elsberry) Date: Wed, 21 Jul 93 09:39:44 -0700 Subject: UCLA Short Course on Wavelets announcement (Aug. 9-11) Message-ID: <9307211639.AA11928@beta.tricity.wsu.edu> ANNOUNCEMENT UCLA Extension Short Course The Wavelet Transform: Techniques and Applications Overview For many years, the Fourier Transform (FT) has been used in a wide variety of application areas, including multimedia compression of wideband ISDN for telecommunications; lossless transform for fingerprint storage, identification, and retrieval; an increased signal to noise ratio (SNR) for target discrimination in oil prospect seismic imaging; in-scale and rotation-invariant pattern recognition in automatic target recognition; and in heart, tumor, and biomedical research. This course describes a new technique, the Wavelet Transform (WT), that is replacing the windowed FT in the applications mentioned above. The WT uses appropriately matched bandpass kernels, called 'mother' wavelets, thereby enabling improved representation and analysis of wideband, transient, and noisy signals. The principal advantages of the WT are 1) its localized nature, which accepts less noise and enhances the SNR, and 2) the new problem-solving paradigm it offers in the treatment of nonlinear problems. The course covers WT principles as well as adaptive techniques, describing how WT's mimic human ears and eyes by tuning up "best mothers" to spawn "daughter" wavelets that catch multi-resolution components to be fed the expansion coefficient through an artificial neural network, called a "wavenet". This, in turn, provides the useful automation required in multiple application areas, a powerful tool when the inputs are constrained by real time sparse data (for example, the "cocktail party" effect where you perceive a desired message from the cacophony of a noisy party). Another advancement discussed in the course is the theory and experiment for solving nonlinear dynamics for information processing; e.g., the environmental simulation as a non-real time virtual reality. In other words, real time virtual reality can be achieved by the wavelet compression technique, followed by an optical flow technique to acquire those wavelet transform coefficients, then applying the inverse WT to retrieve the virtual reality dynamical evolution. (For example, an ocean wave is analyzed by soliton envelope wavelets.) Finally, implementation techniques in optics and digital electronics are presented, including optical wavelet transforms and wavelet chips. Course Materials Course note and relevant software are distributed on the first day of the course. The notes are for participants only, and are not for sale. Coordinator and Lecturer Harold Szu, Ph.D. Research physicist, Washington, D.C. Dr. Szu's current research involves wavelet transforms, character recognition, and constrained optimization implementation on a superconducting optical neural network computer. He is also involved with the design of a sixth-generation computer based on the confluence of neural networks and optical data base machines. Dr. Szu is also a technical representative to DARPA and consultant to the Office of Naval Research on neural networks and related research, and has been engaged in plasma physics and optical engineering research for the past 16 years. He holds five patents, has published about 100 technical papers, plus two textbooks. Dr. Szu is an editor for the journal Neural Networks and currently serves as the President of the International Neural Network Society. Lecturer and UCLA Faculty Representative John D. Villasenor, Ph.D. Assistant Professor, Department of Electrical Engineering, University of California, Los Angeles. Dr. Villasenor has been instrumental in the development of a number of efficient algorithms for a wide range of signal and image processing tasks. His contributions include application-specific optimal compression techniques for tomographic medical images, temporal change measures using synthetic aperture radar, and motion estimation and image modeling for angiogram video compression. Prior to joining UCLA, Dr. Villasenor was with the Radar Science and Engineering section of the Jet Propulsion Laboratory where he applied synthetic aperture radar to interferometric mapping, classification, and temporal change measurement. He has also studied parallelization of spectral analysis algorithms and multidimensional data visualization strategies. Dr. Villasenor's research activities at UCLA include still-frame and video medical image compression, processing and interpretation of satellite remote sensing images, development of fast algorithms for one- and two-dimensional spectral analysis, and studies of JPEG-based hybrid video coding techniques. For more information, call the Short Course Program Office at (310) 825-3344; Facsimile (213) 206-2815. Date: August 9-11 (Monday through Wednesday) Time: 8am - 5pm (subject to adjustment after the first class meeting), plus optional evening sessions, times to be determined. Location: Room 211, UCLA Extension Building, 10995 Le Conte Avenue (adjacent to the UCLA campus), Los Angeles, California. Reg# E8086W Course No. Engineering 867.118 1.8 CEU (18 hours of instruction) Fee: $1195, includes course materials ============================================================================ Wesley R. Elsberry, elsberry at beta.tricity.wsu.edu Sysop, Central Neural System BBS, FidoNet 1:3407/2, 509-627-6267 From reza at ai.mit.edu Thu Jul 22 11:01:00 1993 From: reza at ai.mit.edu (Reza Shadmehr) Date: Thu, 22 Jul 93 11:01:00 EDT Subject: Tech Reports from CBCL at M.I.T. Message-ID: <9307221501.AA11646@corpus-callosum.ai.mit.edu> The Center for Biological and Computational Learning (CBCL) is a newly formed organization at the Dept. of Brain and Cognitive Sciences at M.I.T. The Center's aim is to pursue projects which look at learning from a systems perspective, linking the neurophysiology of learning with its computational, mathematical, and conceptual components in areas of motor control, vision, speech, and language. Some of the work of the members of the Center is now available in the form of technical reports. These reports are published in conjuction with the AI Memo series. You can get a copy of these reports via anonymous ftp (see the end of this message for details). Here is a list of titles currently available via ftp: -------------- :CBCL Paper #79/AI Memo #1390 :author Jose L. Marroquin and Federico Girosi :title Some Extensions of the K-Means Algorithm for Image Segmentation and Pattern Classification :date January 1993 :pages 21 :keywords K-means, clustering, vector quantization, segmentation, classification :abstract We present some extensions to the k-means algorithm for vector quantization that permit its efficient use in image segmentation and pattern classification tasks. We show that by introducing a certain set of state variables it is possible to find the representative centers of the lower dimensional manifolds that define the boundaries between classes; this permits one, for example, to find class boundaries directly from sparse data or to efficiently place centers for pattern classification. The same state variables can be used to determine adaptively the optimal number of centers for clouds of data with space-varying density. Some examples of the application of these extensions are also given. -------------- :CBCL Paper #82/AI Memo #1437 :author Reza Shadmehr and Ferdinando A. Mussa-Ivaldi :title Geometric Structure of the Adaptive Controller of the Human Arm :date July 1993 :pages 34 :keywords Motor learning, reaching movements, internal models, force fields, virtual environments, generalization, motor control. :abstract The objects with which the hand interacts with may significantly change the dynamics of the arm. How does the brain adapt control of arm movements to this new dynamics? We show that adaptation is via composition of a model of the task's dynamics. By exploring generalization capabilities of this adaptation we infer some of the properties of the computational elements with which the brain formed this model: the elements have broad receptive fields and encode the learned dynamics as a map structured in an intrinsic coordinate system closely related to the geometry of the skeletomusculature. The low--level nature of these elements suggests that they may represent a set of primitives with which movement are represented in the CNS. ============================ How to get a copy of above reports: The files are in compressed postscript format and are named by their AI memo number, e.g., the Shadmehr and Mussa-Ivaldi paper is named AIM-1437.ps.Z. They are put in a directory named as the year in which the paper was written. Here is the procedure for ftp-ing: unix> ftp ftp.ai.mit.edu (log-in as anonymous) ftp> cd ai-pubs/publications/1993 ftp> binary ftp> get AIM-number.ps.Z ftp> quit unix> uncompress AIM-number.ps.Z unix> lpr AIM-number.ps.Z I will periodically update the above list as new titles become available. Best wishes, Reza Shadmehr Center for Biological and Computational Learning M. I. T. Cambridge, MA 02139 From niranjan at eng.cam.ac.uk Thu Jul 22 09:38:15 1993 From: niranjan at eng.cam.ac.uk (Mahesan Niranjan) Date: Thu, 22 Jul 93 09:38:15 BST Subject: Multiple Models, Committee of nets etc... Message-ID: <16459.9307220838@dsl.eng.cam.ac.uk> >From: Wray Buntine <wray at ptolemy.arc.nasa.gov> >Subject: RE: committees, agent teams, redundancy, Monte Carlo, ... >Date: Wed, 21 Jul 1993 01:57:31 GMT > [...] >e.g MacKay's version of committee's which got the energy prediction > prise smells like Breiman's version of Wolpert's stacked > generalization [...] >Now we all seem to be rediscovering the use of multiple models, >i.e. the next step in sophisticated learning algorithms. [...] An interesting and somewhat easy to understand application of multiple models is in the area of target-tracking (e.g. Bar-Shalom & Fortmann, 'Tracking and Data Association', Academic Press 1988, ISBN 0-12-079760). They show how to run several models in parallel, recursively estimating them with Kalman filtering and use the innovation probabilities for model selection. Apart from terminology (like you dont see the term "evidence" there), a lot of the ideas are in that framework too; but the assumptions etc are much clearer (at least to me), and the language not so strong. We have used this method to track parametric models of highly nonstationary signals (e.g. Formants in speech). The committee of networks doing the energy prediction (committee members chosen by ranking models by performance on cross-validation set, and the average performance of these being better than the best member) is a somewhat surprising result to me. Surprising because, the average predictions are taken without weighting by the model probabilities (which are difficult to compute). In practice, even for linear models in Gaussian noise, I find probabilities tend to differ by large numbers, for models that look very similar. Hence if these are difficult to evaluate and are assumed equal, I would have expected the average performance to be worse than the best member. In real life too, committees tend to be less efficient than the good individual members (when you give the members equal say), but thats a different story :-) niranjan From xiru at Think.COM Fri Jul 23 11:27:12 1993 From: xiru at Think.COM (Xiru Zhang) Date: Fri, 23 Jul 93 11:27:12 EDT Subject: Multiple Models, Committee of nets etc... Message-ID: <9307231527.AA23862@yangtze.think.com> Just to add another reference to the list: We have used a HYBRID system of three "experts" (neural net, statistical model, memory-based reasoning) for protein secondary structure prediction, and obtained, I believe, the best prediction accuracy reported to date. For reference, see: @article{jmb92, author = {Xiru Zhang, Jill P. Mesirov and David L. Waltz}, title = "Hybrid System for Protein Secondary Structure Prediction", journal = {Journal of Molecular Biology}, year = {1992}, volume = {225} } - Xiru Zhang Thinking Machines Corp. 245 First St. Cambridge, MA 02142 From dhw at santafe.edu Fri Jul 23 11:39:22 1993 From: dhw at santafe.edu (David Wolpert) Date: Fri, 23 Jul 93 09:39:22 MDT Subject: combining generalizers' guesses Message-ID: <9307231539.AA26719@sfi.santafe.edu> Mahesan Niranjan writes: >>> The committee of networks doing the energy prediction (committee members chosen by ranking models by performance on cross-validation set, and the average performance of these being better than the best member) is a somewhat surprising result to me. Surprising because, the average predictions are taken without weighting by the model probabilities (which are difficult to compute). In practice, even for linear models in Gaussian noise, I find probabilities tend to differ by large numbers, for models that look very similar. Hence if these are difficult to evaluate and are assumed equal, I would have expected the average performance to be worse than the best member. >>> In general, when using stacking to combine guesses of separate generalizers (i.e., when combining guesses by examining validation set behavior), one doesn't simply perform an unweighted average, as MacKay did, but rather a weighted average. For example, in Leo Breiman's Stacked regression paper of last year, he combined guesses by means of a weighted average. The weights were set to minimize LMS error on the validation sets. (Sets plural because J-fold partitioning of the training data was used, like in cross-validation, rather than a single split into a training set and a validation set.) In literally hundreds of regression experiments, Leo found that this almost always beat cross-validation, and never (substantially) lost to it. In essence, in this scheme validation set behavior is being used to estimate the model "probabilities" Niranjan refers to. Also, in MacKay's defense, just because he "got the probabilities wrong" doesn't imply his average would be worse than just choosing the single best model. Just a few of the other factors to consider: 1) What is the relationship between mis-assignment of model probabilities, model's guess, and optimal guess? 2) How do estimation errors (due to finite validation sets, due to finite training sets) come into play? Also, it should be noted that there are other ways to perform stacking (either to combine generalizers or to improve a single one) which do not use techniques which are interprable in terms of "model probabilities". For example, rather than combining generalizers via the generalizer "find the hyperlane (w/ non-negative summing-to-1 coefficients) w/ the minimal LMS error on the data", which is what Leo did, one can instead use nearest neighbor algorithms, or even neural nets. In general though, one should use such a "second level" generalizer which has low variance, i.e., which doesn't bounce around a lot w/ the data. Otherwise you can easily run into the kinds of problems Niranjan worries about. David Wolpert References: Breiman, L, "Stacked regressions", TR 367, Dept. of Stat., Univ. of Cal. Berkeley (1992). Wolpert, D., "Stacked Generalization", Neural Networks, vol. 5, 241-259 (1992). (Aside from an early tech. report, the original public presentation of the idea was at Snowbird '91.) I also managed to convince Zhang, Mesirov and Waltz to try combining with stacking rather than with non-validation-set-based methods (like Qian and Sejnowski used), for the problem of predicting protein secondary structure. Their (encouraging) results appeared last year in JMB. From gmk at learning.siemens.com Fri Jul 23 10:20:17 1993 From: gmk at learning.siemens.com (Gary M. Kuhn) Date: Fri, 23 Jul 93 10:20:17 EDT Subject: Advance Program NNSP'93 Message-ID: <9307231420.AA01000@petral.siemens.com> ADVANCE PROGRAM 1993 IEEE Workshop on Neural Networks for Signal Processing September 6 - September 9, 1993 Maritime Institute of Technology and Graduate Studies Linthicum Heights, Maryland, USA Sponsored by IEEE Signal Processing Society (In Cooperation with IEEE Neural Networks Council) Co-sponsored by Siemens Corporate Research ARPA-MTO INVITATION TO PARTICIPATE The members of the Workshop Organizing Committee invite you to attend the 1993 IEEE Workshop on Neural Networks for Signal Processing. The 1993 Workshop is the third workshop organized by the Neural Network Technical Committee of the IEEE Signal Processing Society. The first took place in 1991 in Princeton, NJ, USA, the second in 1992 in Helsingor, Denmark. The purpose of the Workshop is to foster informal technical interaction on topics related to the application of neural networks to problems in signal processing. WORKSHOP LOCATION The 1993 Workshop will be held at the Maritime Institute of Technology Graduate Studies (MITAGS), 5700 Hammonds Ferry Road, Linthicum Heights, MD, 21090, USA, telephone +1 410 859 5700. MITAGS is a training facility of the International Organization of Masters, Mates & Pilots. TRANSPORTATION TO MITAGS MITAGS is located directly south of Baltimore, Maryland. For those arriving by air at the Baltimore - Washington International Airport, MITAGS is 5 miles away by taxi. For those arriving by private car, we provide the following directions: 1. From NORTH VIA I-95: South through Fort McHenry Tunnel, stay on I-95 South to I-695 South (Glen Burnie). Proceed as in #2 below. 2. FROM BELTWAY(I-695) COMING FROM THE NORTH OR WEST: Get off at Exit 8 (Hammonds Ferry-Nursery Road), turn left at end of exit, straight through the traffic light, over the bridge to the sign saying MASTERS, MATES & PILOTS. Follow the driveway (blue lines) to Day Visitor Lots A, B or C, or to the Overnight Lot. 3. FROM BALTIMORE VIA BALTIMORE-WASHINGTON PARKWAY (I-295): Go South to I-695 West (To Towson). Proceed as in #4 below. 4. FROM BELTWAY(I-695) COMING FROM THE EAST: Get off at exit 8 (Nursery Road), stay to the right until you face a sign saying Hammonds Ferry Road, turn right, go to a traffic light, turn left on Hammonds Ferry road, continue over the bridge, turn right at the sign saying MASTERS, MATES & PILOTS and proceed to Day Visitor Parking Lots A through C, or if you are staying overnight, to the Overnight Lot. 5. FROM SOUTH VIA BALTIMORE-WASHINGTON PARKWAY (I-295): Turn off at exit just before Baltimore Beltway (I-695). This exit says West Nursery Road. Stay to the right on the exit ramp. Go to first light (International Drive), turn left to bottom of hill, left onto Aero Drive. At gate, sign says MASTERS, MATES & PILOTS. Turn right and follow blue line to parking lots. 6. FROM SOUTH VIA I-95: Go to Baltimore Beltway (I-695) South towards Glen Burnie. Get off at Exit 8 as in #3 above. WORKSHOP REGISTRATION INFORMATION There is a Registration Form at the end of the Advanced Program. The registration fee "without room" covers attendance at all workshop sessions, one copy of the hard-bound proceedings, the Monday night reception, the coffee breaks, and all meals during the three days of the Workshop, including the banquet at the Baltimore Orioles' Baseball Stadium. For those potential participants whose funds are not permitted to be spent on a banquet, we point out that corporation co-sponsorship is paying for the banquet. The registration fee "with room" covers all of the above plus lodging on the campus at MITAGS. Lodging at MITAGS is by far the most convenient, and it is very reasonably priced. This is the registration that we recommend for all participants coming from a distance. For IEEE members before August 1, the registration fee is $375 without room and $575 with room. After August 1, the registration fee is $425 without room and $625 with room. Non-members, please add $50 to these fees. Students may apply for a limited number of partial travel and registration grants. See registration form below. EVENING EVENTS A Pre-Workshop Reception will be held at MITAGS on Monday evening, September 6, 1993, at 8:00 PM. On Tuesday evening, a panel on Dual-use Applications of Neural Network Technology will be led by Workshop Co-Chair Barbara Yoon. On Wednesday evening, busses will take participants to the banquet at the new Baltimore Orioles' Baseball Stadium in downtown Baltimore. Dinner will be served in the Orioles' 6th floor banquet facility. Reservations have been made for 120 participants. Each reservation includes a ticket to the party rooms reserved for the workshop down on the Stadium Club level. After the banquet, participants may either relax in the banquet facility, or move to the party rooms and adjacent outside seating to enjoy the scheduled evening baseball game with the Seattle Mariners. Busses will return everyone to MITAGS at the end of the game. At the close of the Workshop on Thursday afternoon, participants are invited to stay for a demonstration of MITAG's $50 million computer-based simulator of the bridge of a ship. This hydraulically-mounted simulator will be operated in a cinerama representation of the harbor of the City of New York. PROGRAM OVERVIEW Time Tuesday 7/9/93 Wednesday 8/9/93 Thursday 9/9/93 _______________________________________________________________ 8:15 AM Opening Remarks 8:30 AM Keynote Address Keynote Address Keynote Address 9:20 AM Image Processing Learning Theory (Lecture) (Lecture) (Lecture) 10:50 AM Break Break Break 11:20 AM Theory Applications 1 Applications 2 (Poster preview) (Poster preview) (Poster preview) 12:20 PM Lunch Lunch Lunch 1:30 PM Theory Applications 1 Applications 2 (Poster) (Poster) (Poster) 2:45 PM Break Break Break 3.15 PM Classification Speech Processing Applications (Lecture) (Lecture) (Lecture) Evening Panel Banquet at Simulator Discussion Orioles Stadium Demonstration Note: Session Chairs listed in the following Technical Program may change. TECHNICAL PROGRAM Tuesday, September 7, 1993 [8:15 AM: Opening Remarks:] Gary Kuhn, Barbara Yoon, General Chairs Rama Chellappa, Program Chair [8:30 AM: Opening Keynote:] Learning, function approximation, and images Tomaso Poggio, Massachusetts Institute of Technology, Massachusetts, USA. [9:20 AM: Image Processing (Lecture Session)] Chair: B.S. Manjunath, UCSB A Nonlinear Scale-Space Filter by Physical Computation, Yiu-Fai Wong, Lawrence Livermore National Lab, Livermore, CA, USA. A Common Framework for Snakes and Kohonen Networks, Arnaldo J. Abrantes and Jorge S. Marques, INESC, Lisboa, Portugal. Detection of Ocean Wakes in Synthetic Aperture Radar Images with Neural Networks, Gregg Wilensky, Narbik Manukian, Joe Neuhaus and John Kirkwood, Logicon/RDA, Los Angeles, CA, USA. Image Generation and Inversion Based on a Probabilistic Recurrent Neural Model, N. Sonehara, K. Nakane, Y.Tokunaga, NTT Human Interface Laboratories, Yokosuka, Kanagawa, Japan. [10:50 AM: Coffee break] [11:20 AM: Theory (Oral previews of the afternoon poster session)] Chair: To be announced Liapunov Functions for Additive Neural Networks and Nonlinear Integral Equations of Hammerstein Type, Alexander Jourjine, Wang Laboratories, Lowell, MA, USA. A Hybrid Learning Method for Multilayer Neural Networks, Xin Wang, Meide Zhao, Department of Radio Engineering, Harbin Institute of Technology, Harbin, P.R. China. LS-Based Training Algorithm for Neural Networks, E.D. Di Claudio, R. Parisi and G. Orlandi, INFOCOM Department, University of Roma ``La Sapienza", Roma - Italy. MAP Estimation and the Multilayer Perceptron, Q. Yu and M.T. Manry, Dept. of Electrical Engineering, University of Texas at Arlington, Arlington, Texas and S.J. Apollo, General Dynamics, Fort Worth, Texas, USA. Self-Organizing Feature Map with Position Information and Spatial Frequency Information, Toshio Nakagawa and Takayuki Ito, NHK Science and Technical Research Laboratories, Setagaya-ku, Tokyo, Japan. Competitive Learning and Winning-Weighted Competition for Optimal Vector Quantizer Design, Zhicheng Wang and John V. Hanson, Department of Electrical and Computer Engineering, University of Waterloo, Waterloo, Ontario, Canada. Hierarchical Wavelet Neural Networks, Sathyanarayan S. Rao and Ravikanth S. Pappu, Department of Electrical and Computer Engineering, Villanova University, Villanova, PA, USA. Nonlinear Multilayer Principal Component Type Subspace Learning Algorithms, Jyrki Joutsensalo and Juha Karhunen, Helsinki University of Technology, Laboratory of Computer and Information Sciences, Espoo, Finland. Designer Networks for Time Series Processing, Claus Svarer, Lars Kai Hansen, Jan Larsen and Carl Edward Rasmussen, Technical University of Denmark, Denmark, USA. A Class of Hopfield Decodable Codes, Niclas Wiberg, Dept. of Electrical Engineering, Linkoping University, Linkoping, Sweden. Modeling the Spectral Transition Selectivity in the Primary Auditory Cortex, Kuansan Wang and Shihab A. Shamma, Institute of Systems Research and Department of Electrical Engineering,University of Maryland, College Park, MD, USA. Signal to Noise Analysis of a Neural Network with Nonmonotonic Dynamics, Ioan Opris, Dept. of Physics, University of Bucharest, Bucharest-Magurele, Romania. [12:20 PM: Lunch] [1:30 PM: Theory (Poster Session)] [2:45 PM: Break] [3:15 PM: Classification (Lecture session)] Chair: Candace Kamm, Bellcore Ordered Vector Quantization for Neural Network Pattern Classification, Lane Owsley and Les Atlas, University of Washington, Seattle, WA, USA. Differentially Trained Neural Network Classifiers are Efficient, J.B.Hampshire II and B.V.K. Vijaya Kumar, Carnegie Mellon University, Pittsburg, PA, USA. Extensions of Unsupervised BCM Projection Pursuit: Recurrent and Differential Models for Time-Dependent Classification, Charles M. Bachmann and Dong Luong, Naval Research Laboratory, Washington, D.C., USA. Fuzzy Decision Neural Networks and Signal Recognition Applications, J.S.Taur and S.Y. Kung, Dept. of Electrical Engineering, Princeton University, Princeton, USA. Temporal Sequence Classification by Memory Neuron Networks, Pinaki Poddar and P.V.S. Rao Tata Institute of Fundamental Research, Bombay, India. [8:00 PM: Panel Discussion] Dual-use Applications of Neural Network Technology Moderator: Barbara Yoon, ARPA. Wednesday, September 8, 1993 [8:30 AM: Keynote Address:] Pattern matching in a rapidly self-organizing neural network. Christoph von der Malsburg, Institute for Neuroinformatics, Ruhr-University, Bochum, Germany, and Dept. of Computer Science, USC, Los Angeles, USA. [9:20 AM: Learning (Lecture Session)] Chair: Lee Giles, NEC Neural Networks for Localized Approximation of Real Functions, H.N. Mhaskar, Department of Mathematics, California State University, Los Angeles, CA, USA. Generalization and Maximum Likelihood from Small Data Sets, Bill Byrne, Institute for Systems Research and Dept. of Electrical Engineering, University of Maryland, College Park, MD, USA. Backpropagation Through Time with Fixed Memory Size Requirements, Jose C. Principe and Jyh-Ming Kuo, Computational NeuroEngineering Laboratory, University of Florida, Gainesville, FL, USA. Hierarchical Recurrent Networks for Learning Musical Structure, D.J. Burr, Bellcore, Morristown, NJ, USA and Y. Miyata, Chukyo University, Toyota, Japan. [10:50 AM: Coffee break] [11:20 AM: Applications 1 (Oral previews of afternoon poster session) ] Chair: To be announced VLSI Hamming Neural Net Showing Digital Decoding, Felipe Gomez-Castaneda and Jose A. Moreno-Cadenas, Electrical Engineering Department, Mexico City, D.F. Mexico. Target Recognition Using Multiple Sensors, Y.T. Zhou and R. Hecht-Nielsen, HNC, Inc. San Diego, CA, USA. Recognition of Earthquakes and Explosions Using a Data Compression Neural Network, Roy C. Hsu and Shelton S. Alexander, The Pennsylvania State University, University Park, PA, USA. Characterization of Network Responses to Known, Unknown, and Ambiguous Inputs, Benjamin Hellstrom and Jim Brinsley, Westinghouse Electronic Corporation, Baltimore, MD, USA. Neural Network-Based Helicopter Gearbox Health Monitoring System, Peter T. Kazlas, Peter T. Monsen and Michael J. LeBlanc, The Charles Stark Draper Laboratory, Cambridge, MA, USA. A Hybrid Neural-Fuzzy Approach to VHF Frequency Management, Nancy H. Millstrom, Allen R. Bonde, Jr., and Michael J. Grimaldi, GTE Goverment Systems, Needham Heights, MA, USA. Analysis of Coarse Parallel Architectures for Artificial Neural Processing, K.S. Gugel, J.C Principe and S. Venkumahanti, Computational NeuroEngineering Lab, University of Florida, Gainesville, Florida, USA. Fast VLSI Implementations for MRF and ANN Applications, Haralambos C. Karathanasis, INTRACOM S.A and Computer Engineering Dept., University of Patras and John A. Vlontzos, INTRACOM S.A, Peania Attika, Greece. A Growing and Splitting Elastic Network for Vector Quantization, Bernd Fritzke, International Computer Science Institute, Berkeley, CA, USA. Quantized, Piecewise Linear Filter Network, John Aasted Sorensen, Electrical Institute, Technical University of Denmark, Lyngby, Denmark. Application of ordered codebooks to Image Coding, S. Carrato, Giovanni, L. Sicuranza and L. Manzo, D.E.E.I., University of Trieste, 34127 Trieste, Italy. [12:20 PM: Lunch] [1:30 PM: Applications 1 (Poster Session)] [2:45 PM: Break] [3:15 PM: Speech Processing (Lecture session)] Chair: Fred Juang, AT&T Bell Laboratories A New Learning Algorithm for Minimizing Spotting Errors, Takashi Komori and Shigeru Katagiri, ATR Auditory and Visual Preception Research Laboratories, Kyoto, Japan. A Neural network for Phoneme Recognition Based on Multiresolution Ideas, Kamran Etemad, Institute for System Research and Dept. of Electrical Engineering, University of Maryland, College Park, MD, USA. Text-Dependent Speaker Verification Using Recurrent Time Delay Neural Networks for Feature Extraction, Xin Wang, Department of Radio Engineering, Harbin Institute of Technology, Harbin, P.R. China. A New Learning Approach Based on Equidistortion Principle for Optimal Vector Quantizer Design, Naonori Ueda, Ryohei Nakano, NTT Communication Science Laboratories, Kyoto, Japan. A feedforward neural network for the wavelet decomposition of discrete time signals, Sylvie Marcos, Messaoud Benidir, Laboratoire des Signaux et Systems, E.S.E, Gif-sur-Yvette, France. [5:00 PM: Busses leave for Orioles' Stadium ] [10:00 PM: Busses return from Orioles' Stadium ] Thursday, September 9, 1993 [8:30 AM: Keynote Address:] Evaluation of Neural Network Classifiers Charles L. Wilson, National Institute of Standards and Technology, Gaithersburg, MD, USA. [9:20 AM: Theory (Lecture Session)] Chair: To be announced A Novel Recursive Network for Signal Processing, Irina F. Gorodnitsky and Bhaskar D. Rao, Dept of Electrical and Computer Engineering, University of California, San Diego, USA. A Geometric View of Neural Networks Using Homotopy, Frans M. Coetzee and Virginia L. Stonick, Electrical and Computer Engineering Department, Carnegie Mellon University, Pittsburgh, PA, USA. Nonlinear Predictive Vector Quantisation with Recurrent Neural Nets, Lizhong Wu, Mahesan Niranjan and Frank Fallside, Engineering Dept., Cambridge University, Cambridge. UK. Further Development of Hamiltonian Dynamics of Neural Networks, Ulrich Ramacher, Siemens Corporation, Muenchen, Germany. Invited Talk. [10:50 AM: Coffee break] [11:20 AM: Applications 2] (Oral previews of afternoon poster sessions) Chair: David Burr, Bellcore A Modified Recurrent Cascade-Correlation Network for Radar Signal Pulse Detection, N. Karunanithi, Bellcore, Morristown, NJ., D. Whitley, Dept. of Computer Science, Colorado State University, Fort Collins, CO and D. Newman, Texas Instruments, Colorado Springs, CO, USA. A Technique for Adapting to Speech Rate, Mai Huong T. Nguyen and Garrison W. Cottrell, Institute for Neural Computation, University of California, San Diego, USA. Neurofuzzy Control of a Wheelchair Robotic Hand, Anya L. Tascillo and Victor A. Skormin, Binghamton University, Binghamton, NY, USA. Applying Neural Network Developments to Sign Language Translation, Elizabeth J. Wilson, and Gretel Anspach, Raytheon Company, Riverside, RI, USA. Discriminative Feature Extraction for Speech Recognition, Alain Biem, Shigeru Katagiri and Biing-Hwang Juang, ATR Auditory and Visual Perception Research Laboratories, Kyoto, Japan. Neural Network Image Analysis and Classification in Hybrid Lung Nodule Detection (HLND) System, Y.S Peter Chiou, Y.M. Fleming Lure, Caelum Research Corporation, Silver Spring, MD,USA. A Nonlinear Lattice Structure Based Higher Order Neuron, Muzaffar U. Khurram and Hassan M. Ahmed, Nonlinear Modelling Laboratory, Boston University, Boston, MA, USA. Multisensor Image Classification by Structured Neural Networks, F. Roli, S.B. Serpico and G.Vernazza, Dept. of Biophisical and Electronic Eng., University of Genoa, Italy. A Modular Neural Network Architecture for Pattern Classification, H. Elsherif, M. Hambaba, Intelligent Systems Laboratory, Stevens Institute of Technology, Hoboken, NJ, USA. Compressing Moving Pictures Using the APEX Neural Principal Component Extractor, K.I. Diamantaras, Siemens Corp. Research, S.Y. Kung, Dept. Electrical Eng., Princeton, NJ, USA. Printed Circuit Boards Inspection Using two New Algorithms of Dilatation and Connectivity Preserving Shrinking, Jelloul El Mesbahi, Hassan II University Casablanca and Mohamed Chaibi, Rabat Instituts, Rabat Morocco. Using Self-Organized and Supervised Learning Neural Networks in Parallel for Automatic Target Recognition, Magnus Snorrason, Alper K. Caglayan, Charles River Analytics Inc. Cambridge, MA and Bruce T. Buller, Department of Air force, FL, USA. A Neural Net Application to Signal Identification, Ronald Sverdlove, David Sarnoff Research Center, Princeton, NJ, USA. [12:20 PM: Lunch] [1:30 PM: Applications 2 (Poster Session)] [2:45 PM: Break] [3:15 PM: Applications (Lecture session)] Chair: Bastiaan Kleijn, AT&T Bell Laboratories A Neural Network Model for Adaptive, Non-Uniform A/D Conversion, Marc M.Van Hulle, Massachusetts Institute of Technology, Cambridge, MA, USA. Recurrent Radial Basis Function Networks for Optimal Blind Equalization, Jesus Cid-Sueiro and Anibal R. Figueiras-Vidal, ETSI Telecomunication-UV, Valladolid, Spain. Neural Networks for the Classification of Biomagnetic Maps, Martin F. Schlang, Ralph Neuneier, Siemens AG, Corporate Research and Development, Munchen, Klaus Abraham- Fuchs and Johann Uebler, Siemens AG, Medical Eng., Group, Erlangen, Germany. Hidden Markov Models and Neural Networks for Fault Detection in Dynamic System, Padhraic Smyth, Jet Propulsion Laboratory, California Institute of Technology, Pasadena, CA, USA. [4:30 PM: MITAGS Simulator demonstration] WORKSHOP COMMITTEE GENERAL CHAIRS Gary Kuhn Barbara Yoon Siemens Corporate Research ARPA-MTO Princeton, NJ 08540 Arlington, VA, 22209, USA email: gmk at learning.siemens.come e-mail: byoon at arpa.mil PROGRAM CHAIR Rama Chellappa Dept. of Electrical Engineering University of Maryland College Park, MD 20742, USA. email: chella at surya.eng.umd.edu PROCEEDINGS CHAIR Candace Kamm Bellcore, 445 South Street Morristown, NJ 07960, USA email: cak at thumper.bellcore.com FINANCE CHAIR Raymond Watrous Siemens Corporate Research Princeton, NJ 08540, USA email: watrous at learning.siemens.com PROGRAM COMMITTEE Joshua Alspector Yann Le Cun Les Atlas Richard Lippman Charles Bachmann John Makhoul David Burr Christoph von der Malsburg Rama Chellappa Richard Mammone Gerard Chollet B.S. Manjunath Frank Fallside Dragan Obradovic Emile Fiesler Tomaso Poggio Lee Giles Jose Principe Steve Hanson Ulrich Ramacher Yu Hen Hu Noboru Sonehara Jenq-Neng Hwang Eduardo Sontag B.H. Juang John Sorenson Candace Kamm Yoh'ichi Tohkura Juha Karhunen Kari Torkkola Shigeru Katagiri John Vlontzos Sun-Yan Kung Raymond Watrous Gary Kuhn Christian Wellekens REGISTRATION FORM: 1993 IEEE Workshop on Neural Networks for Signal Processing. September 6 - September 9, 1993. Please complete this form (type or print clearly) and mail with payment for fee to NNSP'93, c/o R.L. Watrous, Siemens Corporate Research, 755 College Road East, Princeton, NJ 08540, USA. Name __________________________________________________________________ Last First Middle Firm or University ____________________________________________________ Mailing address _______________________________________________________ _______________________________________________________________________ _______________________________________________________________________ Country Phone Fax Fee payment must be made by money order/check/bank draft drawn on a U.S. bank or U.S. branch of a foreign bank. Do not send cash. Make fee payable to IEEE NNSP'93. Registration fee with single room and meals Date IEEE member Non-member ____________________________________________ Before August 1 U.S. $575 U.S. $625 After August 1 U.S. $625 U.S. $675 Registration fee without room but still with meals Date IEEE member Non-member ____________________________________________ Before August 1 U.S. $375 U.S. $425 After August 1 U.S. $425 U.S. $475 From tgd at chert.CS.ORST.EDU Sat Jul 24 11:43:24 1993 From: tgd at chert.CS.ORST.EDU (Tom Dietterich) Date: Sat, 24 Jul 93 08:43:24 PDT Subject: combining generalizers' guesses Message-ID: <9307241543.AA11416@curie.CS.ORST.EDU> It seems to me that more attention needs to be paid to *which* generalizer's guesses we are combining. There are three basic components that determine generalization error: * inherent error in the data (which determines the bayes optimal error rate) * small sample size * approximation error (which prevents the algorithm from correctly expressing the bayes optimal hypothesis, even with infinite samples) Combining guesses can't do anything about the first problem. I also don't think it can have a very great effect on the second problem, because all of the guesses are based on the same data. I think that the real win comes from combining guesses that make very different approximation errors. In my error-correcting code work, we re-code the outputs (by imposing an error-correcting distributed representation) in such a way that (we believe) the approximation errors committed by the learning algorithm are nearly independent. Then the decoding process combines these guesses. Interestingly, the decoding process takes a linear combination of the guesses where the linear combination is unique for each output class. We are currently doing experiments to try to understand the relative role of these three sources of error in the performance of error-correcting output codes. This analysis predicts that using a committee of very diverse algorithms (i.e., having diverse approximation errors) would yield better performance (as long as the committee members are competent) than a committee made up of a single algorithm applied multiple times under slightly varying conditions. In the error-correcting code work, we compared a committee of decision trees to an error-correcting output procedure that also used decision trees. The members of the committee were generated by training on different subsamples of the data (as in stacking), but the combination method was simple voting. No matter how many trees we added to the committee, we could not come close to achieving the same performance on the nettalk task as with the error-correcting output coding procedure. So, it seems to me the key question is what are the best ways of creating a diverse "committee"? --Tom From gomes at ICSI.Berkeley.EDU Mon Jul 26 12:53:47 1993 From: gomes at ICSI.Berkeley.EDU (Benedict A. Gomes) Date: Mon, 26 Jul 93 09:53:47 PDT Subject: No subject Message-ID: <9307261653.AA25101@icsib6.ICSI.Berkeley.EDU> I am interested in references for the problem of automatically mapping neural nets onto parallel machines i.e. algorithms for net partitioning, especially for structured connectionist nets. I'm particularly interested in the CM-5 and toroid/mesh distributed memory MIMD machines. References to work on mapping neural nets to particular architectures follows. However the application of automatic partitioning and mapping methods to structured nets is not dealt with. If this question has been asked before on this list or in the newsgroup, my apologies and I would appreciate a copy of any existing bibliography. Otherwise, I will compile and post the references I receive. I am also interested in new developments so as to maintain an up-to-date bibliography. Thanks! ben (gomes at icsi.berkeley.edu) @TechReport{Bahr88, author = "Bahr, Casey S.", title = "ANNE: Another Neural Network Emulator", institution = "Oregon Graduate Center", year = 1988, number = "CS/E-88-028", month = "August" } @Article{Blel87, author = "Blelloch, G. and Rosenberg, C.", title = "Network Learning on the Connection Machine", journal = "Proc. 10th Joint Conference on Artificial Intelligence", year = 1987 } @Article{PN:chung92, author = "Chung, Sang-Hwa and Moldovan, D.I.", title = "Modeling Semantic Networks on the Connection Machine" pages = "152--163" journal = The Journal of Parallel and Distributed Computing, volume = 17, year = 1992 } @TechReport{Fanty86, author = "Fanty, M.", title = "A Connectionist Simulator for the BBN Butterfly Multiprocessor", institution = "University of Rochestor", year = 1986, month = January } @Article{PN:kim89, author = "Kim, Kichul and Kumar, V.K. Prasanna", title = "Efficient implementation of neural networks on hypercube SIMD arrays", journal = "International Joint Conference on Neural Networks", year = 1989 } @Article{PN:kim93, author = "Kim, J.T. and Moldovan, D.I.", title = "Classification and retrieval of knowledge on parallel marker-passing architecture", journal = "IEEE Transactions on Knowledge and Data Engineering", month = oct, year = 1993 } @Article{PN:pomerleau88, author = "Pomerleau, Dean A., Gusciora, George L., Touretsky, David S., and Kung, H.T.", title = "Neural network simulation at warp speed: How we go 17 million connections per second", journal = "IEEE International Conference on Neural Networks", year = 1988, pages = II:155-161 } @TechReport{PN:singer90, author = "Singer, Alexander", title = "Implementations of Artificial Neural Networks on the Connection Machine", institution = "Thinking Machines Corporation", year = 1990, number = "RL90-2" } From bap at learning.siemens.com Mon Jul 26 13:27:39 1993 From: bap at learning.siemens.com (Barak Pearlmutter) Date: Mon, 26 Jul 93 13:27:39 EDT Subject: combining generalizers' guesses In-Reply-To: Tom Dietterich's message of Sat, 24 Jul 93 08:43:24 PDT <9307241543.AA11416@curie.CS.ORST.EDU> Message-ID: <9307261727.AA22861@gull.siemens.com> To my mind, there is a fourth source of error, which is also addressed by the ensemble or committee approach. To your * noise in the data * sampling error * approximation error I would add * randomness in the classifier itself For instance, if you run backpropagation on the same data twice, with the same architecture and all the other parameters held the same, it will still typically come up with different answers. Eg due to differences in the random initial weights. Averaging out this effect is a guaranteed win. --Barak. From drucker at monmouth.edu Mon Jul 26 14:15:19 1993 From: drucker at monmouth.edu (Drucker Harris) Date: Mon, 26 Jul 93 14:15:19 EDT Subject: No subject Message-ID: <9307261815.AA01088@harris.monmouth.edu> Subject: Committee Machines: The best method to generate a committee of learning machines is given by Schapire's algorithm [1]. The boosting algorithm that constructs a committee of three machines is as follows: (1) Train a first learning machine using some training set. (2) A training set for a second committee machine is obtained in the following manner: (a) Toss a fair coin. If heads, pass NEW data through the first machine until the first machine misclassifies the data and add this misclassified data to the training set for the second machine. If the coin tossing is tails pass data through the first network until the first network classifies correctly and add this data to the training set for the second machine. Thus the training set for the second machine consists of data which if passed through the first machine would give a 50% error rate. This procedure is iterated until there is a large enough training set. Data classified correctly when the coin tossing is heads or classified incorrectly when the coin tossing is tails is not used. (b) train the second machine. (3) A training set for a third machine is obtained in the following manner: (a) Pass NEW data through the first two trained machines. If the two machines agree on the classification (whether correct or not), toss out the data. If they disagree, add this data to the training set for the third machine. Iterate until there is a large enough training set. (b) Train the third machine. (4) In the testing phase, a pattern is presented to all three machines. If the first two machines agree, use that labeling; otherwise use the labeling of the third machine. The only problem with this approach is generating enough data. For OCR recognition we have synthetically enlarged the database by deforming the original data [2]. Boosting dramatically improved error rates. We are publishing a new paper that has much more detail [3]. Harris Drucker References: 1. R.Schapire, "The Strength of weak learnability" Machine Learning 5, Number 2, (1990), p197-227 2. H.Drucker, R.Schapire, and P. Simard, "Improving Performance in Neural Networks Using a Boosting Algorithm" Neural Information Processing Systems 5, proceeding of the 1992 conference (published 1993), Eds: J.Hanson, J Cowan, C.L. Giles p. 42-49. 3.H.Drucker, R. Schapire, P. Simard, "Boosting Performance in Neural Networks", International Journal of Pattern Recognition and Artificial Intelligence, Vol 7, Number 4, (1993), to be published. From cohn at psyche.mit.edu Mon Jul 26 12:01:15 1993 From: cohn at psyche.mit.edu (David Cohn) Date: Mon, 26 Jul 93 12:01:15 EDT Subject: combining generalizers' guesses Message-ID: <9307261601.AA02163@psyche.mit.edu> Tom Dietterich <tgd at chert.cs.orst.edu> writes: > ... (good stuff deleted) ... > This analysis predicts that using a committee of very diverse > algorithms (i.e., having diverse approximation errors) would yield > better performance (as long as the committee members are competent) > than a committee made up of a single algorithm applied multiple times > under slightly varying conditions. > > ... > > So, it seems to me the key question is what are the best ways of > creating a diverse "committee"? > > --Tom One possible way of diversifying the committee (don't *I* sound PC!) is to make the inductive bias of the learning algorithm explicit, or as an approximation, add a new inductive bias that is strong enough to override biases inherent in the algorithm. This can be done a number of ways, by adding extra terms to the error equation or some other kludges. By then running the same algorithm with widely differing biases, one can approximate different algorithms. [warning: blatant self-promotion follows :-)] For example, a few years ago, we looked at something like this with a different end in mind. The selective sampling algorithm was used to identify potentially useful training examples by means of what has become known as the committee approach (with a twist). Two identical networks were trained on the same positive-negative classification problem with the same training data. We added two different inductive biases to the backprop training, though: One network (S) was trained to find the most *specific* concept consistent with the data. That is, it was to try to classify only the positive training examples as positive, and as much else of the domain as possible was to be classified as negative. The other network (G) was trained to find the most *general* concept consistent with the data, that is, to classify as much of the domain as positive as it could while accommodating the negative training examples. The purpose of these biases was to decide whether a potential training example was interesting. If the two networks disagreed on its classification, then then it lay in the architecture's version space, and should be queried/added. These, and other biases would suggest themselves as appropriate, though, for producing diverse committee members for voting on the classification/output of a network. For those interested in the details of the selective sampling algorithm, we have a paper which is to appear in Machine Learning. It is available by anonymous ftp to "psyche.mit.edu"; the paper is in "pub/cohn/selsampling.ps.Z". -David Cohn e-mail: cohn at psyche.mit.edu Dept. of Brain & Cognitive Science phone: (617) 253-8409 MIT, E10-243 Cambridge, MA 02139 From dhw at santafe.edu Mon Jul 26 17:51:04 1993 From: dhw at santafe.edu (dhw@santafe.edu) Date: Mon, 26 Jul 93 15:51:04 MDT Subject: No subject Message-ID: <9307262151.AA02456@zia> Tom Dietterich writes: >>> This analysis predicts that using a committee of very diverse algorithms (i.e., having diverse approximation errors) would yield better performance (as long as the committee members are competent) than a committee made up of a single algorithm applied multiple times under slightly varying conditions. >>> There is a good deal of heuristic and empirical evidence supporting this claim. In general, when using stacking to combine generalizers, one wants them to be as "orthogonal" as possible, as Tom maintains. Indeed, one might even want to choose constituent generalizers which behave poorly stand-alone, just so that they are sufficiently different from one another when one combines them. (This is somewhat similar to what's going on w/ error-correcting codes, if one considers each (output code bit)-learning algorithm to be a different generalizer, trying to correctly classify things.) In fact, consider the situation where one uses very different generalizers, and their errors are highly correlated on a particular data set, so that *as far as the data is concerned* those generalizers are identical. For such situations, stacking (or any other kind of combining) can not help - all the guesses will be the same. However in such a situation you have good reason to suspect that you are data-limited - there is simply nothing more to be milked from the data. (An example of this phenomenon occurs with the intron-exon prediction problem; for some data sets, ID3, backprop, and a simple memory-based algorithm don't only get the same error rate; they have highly correlated errors, making mistakes in the same situations.) >>> In the error-correcting code work, we compared a committee of decision trees to an error-correcting output procedure that also used decision trees. The members of the committee were generated by training on different subsamples of the data (as in stacking), but the combination method was simple voting. No matter how many trees we added to the committee, we could not come close to achieving the same performance on the nettalk task as with the error-correcting output coding procedure. >>> Well, if I understand this correctly, Tom's using simple voting to combine, w/o any regard to behavior on the validation set. This will rarely be the best way of doing things. It amounts to training many times on subsets of the training data and then voting, rather than training once on the whole data; as such, this scheme might even result in worse generalization generically (whether it improves or helps probably depends on the generalizer's learning curve for the problem in question). Moreover, if (as in this case) one is playing w/ algorithms which are essentially identical (the members of the "committee"), then one might as well go whole-hog, and use the formulation of stacking designed to improve a single algorithm. (In this formulation, one uses partitions of the training set to try to find correlations between the *signed* error of the learning algorithm, and factors like its guess, the input value, distance to training set, etc.). Alternatively, as Tom well points out, you could modify the algorithms to make them substantially different from one another. Of course, one can also have fun by combining error-correcting-code algorithms based on different generalizers (say by stacking). *** Tom also writes: >>> There are three basic components that determine generalization error: * inherent error in the data (which determines the bayes optimal error rate) * small sample size * approximation error (which prevents the algorithm from correctly expressing the bayes optimal hypothesis, even with infinite samples) >>> One has to be a bit careful here; there's a lot of other stuff going on besides these three "components". Since Tom seems to have a Bayesian context in mind, it's worth analyzing things a bit from that perspective. In general, Pr(generalization error E | data D) = a (non-Euclidean) inner product between Pr(hypothesis h | data D) (i.e., one's learning algorithm) and Pr("truth" f | data D)) (i.e., the posterior distribution). Tom's component 3, "approximation error", seems to refer to having a poor alignment (so to speak) between Pr(h | D) and Pr(f | D); he seems to have in mind a scenario in which the learning algorithm is not optimally designed for the posterior. The first two components seem to instead refer to "lack of sharpness" (over f) of Pr(f | D). More precisely, they seem to refer to having the likelihood, Pr(D | f), broad, when viewed as a function of f. If this is indeed the context Tom has in mind, there is another factor to consider as well: Since the posterior is proportional to the likelihood times the prior, one also has to consider "lack of sharpness" in Pr(f), and more generally how aligned the prior is with the likelihood. In other words, there are situations in which one has no "approximation error", no error in the data, and a large sample, but there is still a large chance of large generalization error. This occurs for example if the prior f-peak is broad, but far removed from the likelihood f-peak. (Generically such situations are unlikely - that's the essential logic behind confidence intervals, VC stuff, etc. - but there's no way to assure that one isn't in such a situation in a given learning problem. And if you're running a program in which you search for something, like low error on the training set, then most bets - even conventinal VC bets, despite dogma to the contrary - are off. This is because conventional VC results refer to the distribution Pr(|gen. error E - training set error S| > bound | training set size m), which is NOT the same as Pr(|E - S| > bound | S, m); to calculate Pr(|E - S| > bound | S, m) you need to know something about Pr(f). Indeed, it's trivial to construct examples in which one can find S = 0, but whenever that occurs, one knows, w/ 100% certainty, that E is near maximal.) Moreover, if one instead defines "generalization error" to be off-training set error, then in fact you'll *always* have large error, if Pr(f) is uniform over f. (This is proven in a number of different guises in the papers referenced below. Intuitively, it holds because if all f are equally likely, any one off-training set behavior of f is as likely as any other, and the training set tells you nothing.) This result is completely independent of what learning algorithm you use, what VC analysis says, and the like, and well-exemplifies the importance of the prior Pr(f). David Wolpert Ref.'s [1] Wolpert, D. On the connection between in-sample testing and generalization error. Complex Systems, vol. 6, 47-94. (1992). [2] Wolpert, D. On overfitting-avoidance as bias. Not yet published (but placed in the neuroprose archive a couple of months ago). A related article, "Overfitting avoidance as bias", was published by C. Schaffer in the February Machine Learning. From watrous at learning.siemens.com Tue Jul 27 08:24:25 1993 From: watrous at learning.siemens.com (Raymond L Watrous) Date: Tue, 27 Jul 93 08:24:25 EDT Subject: Extended Registration Deadline for NNSP'93 Message-ID: <9307271224.AA05615@tiercel.siemens.com> In view of an unexpected delay in the mailing of the advance program and in the recent posting of the announcement to the Connectionists mailing list, the deadline for early registration for the 1993 IEEE Workshop on Neural Networks for Signal Processing has been extended to August 14. Raymond Watrous, Financial Chair 1993 IEEE Workshop on Neural Networks for Signal Processing c/o Siemens Corporate Research 755 College Road East Princeton, NJ 08540 (609) 734-6596 (609) 734-6565 (FAX) From hamps at shannon.ECE.CMU.EDU Tue Jul 27 08:40:24 1993 From: hamps at shannon.ECE.CMU.EDU (John B. Hampshire II) Date: Tue, 27 Jul 93 08:40:24 EDT Subject: committees are bad Message-ID: <9307271240.AA06235@ shannon.ece.cmu.edu.ECE.CMU.EDU > Belief in committees is paradoxically based on the notion that each member of the committee is a biased estimator of the Bayes-optimal classifier --- I stress that I am restricting my comments to pattern classification; I'm not commenting on function approximation (e.g., regression). Regression (i.e., estimating probabilities) and classification are not the same thing. The idea behind committees is that the average of a bunch of biased estimators will constitute an unbiased estimator. This is a provably *bad* idea, absent a proof that the biases all cancel (I'll bet there is no such proof in any of the committee work). Nevertheless, committees are obviously popular because the classifiers we typically generate in the connectionist community are provably biased --- even with regularization, pruning, and all the other complexity reduction tricks. Put in more organic terms, committees of humans often comprise large numbers of unremarkably average, biased individuals: their purpose is to achieve what one remarkable, unbiased individual could do alone. By virtue of their number, they generally involve huge development and maintenance overhead. This is a waste of resources. Compensating for one biased committee member with another one that has a different bias generally gives us a committee with lots of bias rather than one with no bias. The United States Congress is a perfect illustrative example: consider each member as a biased estimator of the ideal politician, and consider how effective the average of their efforts is... Barak certainly makes a valid point re. the initial parameterization issue, although it is also not that important if your model is minimum (or near minimum) complexity --- this gets into the issue of estimation variance, versus that of estimation bias. I'll take a single *provably* unbiased classifier over a committee of biased ones any day. Vapnik is right: excessive complexity is anathema. So are Geman, Bienenstock, and Doursat: connectionists face a "bias/variance dilemma". Fortunately, there is a relatively simple way to generate unbiased, low-complexity, minimum-variance classifiers. For those who care, I am prepared to defend this post with supporting proofs. However, I won't do it over connectionists in deference to those who don't care. -John From dhw at santafe.edu Tue Jul 27 10:52:47 1993 From: dhw at santafe.edu (David Wolpert) Date: Tue, 27 Jul 93 08:52:47 MDT Subject: The "best" way to do learning Message-ID: <9307271452.AA19037@sfi.santafe.edu> Harris Drucker writes: >>> The best method to generate a committee of learning machines is given by Schapire's algorithm [1]. >>> Schapire's boosting algorithm is a very interesting technique, which has now garnered some empirical support. It should be noted that it's really more a means of improving a single learning machine than a means of combining separate ones. More to the point though: There is no such thing as an a priori "best method" to do *anything* in machine learning. Anyone who thinks otherwise is highly encouraged to read Cullen Schaffer's Machine Learning article from Feb. '93. *At most*, one can say that a method is "best" *given some assumptions*. This is made explicit in Bayesian analysis. To my knowledge, boosting has only been analyzed (and found in a certain sense "best") from the perspective of PAC, VC stuff, etc. Now those formalisms can lend many insights into the learning process. But not if one isn't aware of their (highly non-trivial) implicit assumptions. Unfortunately, one of more problematic aspects of those formalisms is that that they encourage people to gloss over those implicit assumptions, and make blanket statements about "optimal" algorithms. From bisant at samish.stanford.edu Tue Jul 27 14:37:18 1993 From: bisant at samish.stanford.edu (bisant@samish.stanford.edu) Date: Tue, 27 Jul 93 11:37:18 PDT Subject: combining generalizers guesses Message-ID: <9307271837.AA26492@samish.stanford.edu> >It seems to me that more attention needs to be paid to *which* >generalizer's guesses we are combining. There are three basic >components that determine generalization error: > > * inherent error in the data (which determines the bayes optimal error rate) > * small sample size > * approximation error (which prevents the algorithm from correctly > expressing the bayes optimal hypothesis, even with infinite samples) I also think there is another source of error in addition to those given above which can be removed by combining generalizers. This source is: * the lack of confidence in the prediction. Most neural networks and other classifiers produce a continuous output. Usually, during classification, a threshold or winner take all method is used to decide the classification. If you imagine a network which classifies inputs into one of 3 outputs and you see some classifications which appear as follows: a. 0.27 0.21 0.86 b. 0.41 0.48 0.53 it is obvious the third class is the winner, but it is also obvious classification "a" has much more confidence than "b". Whichever arbitration mechanism is used to combine the generalizers should take this information into account. > So, it seems to me the key question is what are the best ways of > creating a diverse "committee"? Most researchers who work in applying neural networks use a committee approach for the final decision. Some empirical research has been done over the last 4 years to find the best way. Waibel and Hampshire have presented some work in NIPS, IEEE, and IJCNN 3 years ago where they used different objective functions to create very diverse networks. I believe they used the following objective functions: 1 squared error 2 classification figure of merit (CFM) 3 cross entropy. The networks produced, especially by the CFM, were very different. As an arbitration mechanism, they found that a simple average worked better than other more complicated methods including a neural network. All the arbitration mechanisms they tried were able to take the confidence factor, mentioned above, into account. David Bisant Stanford PDP Group @article{ hampshire2, author="Hampshire II, J. B. and Waibel, A. H.", title="{A Novel Objective Function for Improved Phoneme Recognition Using Time-Delay Neural Networks}", journal="IEEE Transactions on Neural Networks", volume="1", number="2", year="1990", pages="216-228"} From ingber at alumni.cco.caltech.edu Tue Jul 27 20:02:10 1993 From: ingber at alumni.cco.caltech.edu (Lester Ingber) Date: Tue, 27 Jul 1993 17:02:10 -0700 Subject: Summary of contributed paper to Frontier Science in EEG Symposium Message-ID: <9307280002.AA00529@alumni.cco.caltech.edu> Summary of contributed paper to Frontier Science in EEG Symposium A PostScript summary of a presentation, Multiple Scales of EEG, to be made at Frontier Science in EEG Symposium: Proceedings, New Orleans, 9 Oct 1993, is available via anonymous ftp, as eeg_proc93.ps.Z. The announcement and registration forms for this meeting are in the file eeg_announce_proc93.Z. The Introduction describes this meeting: Electroencephalography (EEG) is the study of the electrical activity of the brain. The field of EEG includes the technology to record these electrical signals, the science to analyze them and the expertise to apply them to patient care. This symposium will explore the scientific frontiers related to EEG, presenting the latest research and thought with this year's topic being continuous waveform analysis. As advances in science and technology often involve collaboration among scientists from different fields, we are bringing together a diverse group of investigators, many from areas not conventionally associated with EEG, to actively encourage multidisciplinary research in EEG and foster new ideas. A set of plates from which this talk will be developed is the file eeg_talk.ps.Z; this file is only 558K, but due to the inclusion of scanned figures, it will expand to over 8M when uncompressing. A paper giving more technical background is eeg92.ps.Z. Also in this directory is the latest Adaptive Simulated Annealing (ASA) code, version 1.37. The INDEX file gives a bibliographic reference for the files in this directory. Instructions for retrieval: ftp ftp.caltech.edu [Name:] anonymous [Password:] your_email_address cd pub/ingber binary get file_of_interest quit If you do not have ftp access, get information on the FTPmail service by: mail ftpmail at decwrl.dec.com, and send only the word "help" in the body of the message. If any of the above are not convenient, and if your mailer can handle large files (please test this first), the code or papers you require can be sent as uuencoded compressed files via electronic mail. If you have gzip, resulting in smaller files, please state this. Sorry, but I cannot assume the task of mailing out hardcopies of code or papers. Lester || Prof. Lester Ingber 1-800-L-INGBER || || Lester Ingber Research Fax: [10ATT]0-700-L-INGBER || || P.O. Box 857 EMail: ingber at alumni.caltech.edu || || McLean, VA 22101 Archive: ftp.caltech.edu:/pub/ingber || From kolen-j at cis.ohio-state.edu Tue Jul 27 11:24:38 1993 From: kolen-j at cis.ohio-state.edu (john kolen) Date: Tue, 27 Jul 93 11:24:38 -0400 Subject: combining generalizers' guesses In-Reply-To: Barak Pearlmutter's message of Mon, 26 Jul 93 13:27:39 EDT <9307261727.AA22861@gull.siemens.com> Message-ID: <9307271524.AA05339@pons.cis.ohio-state.edu> >Barak Pearlmutter <bap at learning.siemens.com> >For instance, if you run backpropagation on the same data twice, with >the same architecture and all the other parameters held the same, it >will still typically come up with different answers. Eg due to >differences in the random initial weights. [Blatant self promotion follows, but I looking for a job so I need all the promotion I can get ;-] A very vivid example of this can found in John. F. Kolen and Jordan. B. Pollack, 1990. Backpropagation is Sensitive to Initial Conditions. _Complex Systems_. 4:3. pg 269-280. Available from neuroprose: kolen.bpsic.* or John. F. Kolen and Jordan. B. Pollack, 1991. Backpropagation is Sensitive to Initial Conditions. NIPS 3. pg 860-867. > Averaging out this effect is a guaranteed win. > --Barak. This statement seems rely on an underlying assumption of convexity, which does not necessarily hold when you try to combine different processing strategies (ie. networks). If you massage your output representations so that linear combinations will always give you reasonable results, that's great. But it's not always the case that you have the leeway to make such a representational commitment. John From drucker at monmouth.edu Wed Jul 28 17:14:17 1993 From: drucker at monmouth.edu (Drucker Harris) Date: Wed, 28 Jul 93 17:14:17 EDT Subject: No subject Message-ID: <9307282114.AA00408@harris.monmouth.edu> About committees and boosting: My previous communication used the word "best". That was a little puffery on my part and a reasonable self-imposed limitation of the use of this medium. For an explicit list of limitations, assumptions, etc as to when and where boosting applies send me your e-mail address and I will send you a troff file of a preprint of an article. I can also send hard copies if there are not too many requests. More to the point: if you are interested in classification and want to improve performance, boosting is a reasonable approach, Instead of struggling to build different classifiers and then figuring out the best way to combine them, boosting by filtering explicitly shows us how you filter the data so that each machine learns a different distribution of the training set. In our work in OCR using multilayer networks (single layer networks are not powerful enough) boosting has ALWAYS improved performance. Synthetically enlarging the database using deformations of the original data is essential. In one case, a network (circa 1990) which had an error rate on United State Postal Service digits of 4.9% and a reject rate of 11.5% (in order to achieve a 1% error rate on those not rejected) was boosted to give a 3.6% error rate and a 6.6% reject rate. Someone then invented a new network that had a 3.3% error rate and a 7.7% reject rate and this was boosted to give a 2.6% error rate and 4.0% reject rate. This is very close to the estimated human performance of 2.5%. Can someone find a better single network (using the original database) that is better than a boosted committee. Maybe. But good networks are hard to find and if you can find it, you can probably boost it. Can one improve performance by using the synthetically enlarged database and a "larger" single machine. Yes, but we have yet to find a single network that does better that a boosted committee. A final note: rather than straight voting, we have found that simply summing the respective outputs of the three neural networks gives MUCH better results (as quoted above). As pointed out by David Bisant, voting does not explicitly include the confidence. In neural networks, a measure of confidence is the difference between the two largest outputs. By simply voting, you ignore the fact that one of the members of the committee may be very confident about its results. By adding, networks with high confidence influence the results more and lower both the error rate and especially the reject rate.. Harris Drucker Bell Labs phone: 908-949-4860 Monmouth College phone: 908-571-3698 email: drucker at monmouth.edu (preferred) From t-chan at rsc.u-aizu.ac.jp Thu Jul 29 12:27:15 1993 From: t-chan at rsc.u-aizu.ac.jp (Tony Y. T. Chan) Date: Thu, 29 Jul 93 12:27:15 JST Subject: The "best" way to do learning Message-ID: <9307290327.AA26368@profsv.rsc.u-aizu.ac.jp> It is obvious that as David Wolpert said, ``There is no such thing as an a priori "best method" to do *anything* in machine learning.'' But I would like to raise the following question: Is there such a thing as the best method to learn a random idea? More precisely, given n random (machine) learning problems, is there such a thing as the best method for dealing with these problems that will give the best overall performance. There may be some best methods for dealing with some specific types of learning problems but is there one that would deal with any learning problem and give the best overall performance? Tony Chan From mpp at cns.brown.edu Thu Jul 29 02:43:58 1993 From: mpp at cns.brown.edu (Michael P. Perrone) Date: Thu, 29 Jul 93 02:43:58 EDT Subject: Multiple Models, Committee of nets etc... Message-ID: <9307290643.AA18402@cns.brown.edu> For those interested in the recent discussion of Multiple Models, Committees, etc., the following references may be of interest. The first three references deal exactly with the issues that have recently been discussed on Connectionists. The salient contributions from these papers are: 1) A very general result which proves that averaging ALWAYS improves optimization performance for a broad class of (convex) optimization problems including MSE, MLE, Maximum Entropy, Maximum Mutual Information, Splines, HMMs, etc. This is a result about the topology of the optimization measure and is independent of the underlying data distribution, learning algorithm or network architecture. 2) A closed form solution to the optimal weighted average of a set of regression estimates (Here, I regard density estimation and classification as special cases of regression) for a given cross-validation set and MSE optimization. It should be noted that the solution may suffer from over-fitting when the CV set is not representative of the true underlying distribution. However the solution is amenable to ridge regression and a wide variety of heuristic robustification techniques. 3) Experiments on real-world datasets (NIST OCR data, human face data and timeseries data) which demonstrate the improvement due to averaging. The improvement is so dramatic that in most cases the average estimator performs significantly better than the best individual estimator. (It is important to note that the CV performance of a network is not a guaranteed predictor for performance on an independent test set. So a network which has the best performance on the CV set may not have the best performance on the test set; however in practice, even when the CV performance is a good predictor for test set performance, the average estimator usually performs better.) 4) Numerous extensions including bootstrapped and jackknifed neural net generation; and averaging over "hyperparameters" such as architectures, priors and/or regularizers. 5) An interpretation of averaging in the case of MSE optimization, as a regularizer which performs smoothing by variance reduction. This implies that averaging is having no effect on the bias of the estimators. In fact, for a given population of estimators, the bias of the average estimator will be the same as the expected bias of any estimator in the population. 6) A very natural definition of the number of "distinct" estimators in a population which emphasizes two points: (a) Local minima are not necessarily a bad thing! We can actually USE LOCAL MINIMA TO IMPROVE PERFORMANCE; and (b) There is an important distinction between the number of local minima in parameter space and the number of local minima in function space. Function space is what we are really concerned with and empirically, averaging suggests that there are not that many "distinct" local minima in trained populations. Therefore one direction for the future is to devise ways of generating as many "distinct" estimators as possible. The other three references deal with what I consider to be the flip side of the same coin: On one side is the problem of combining networks, on the other is the the problem of generating networks. These three references explore neural net motivated divide and conquer heuristics within the CART framework. Enjoy! Michael -------------------------------------------------------------------------------- Michael P. Perrone Email: mpp at cns.brown.edu Institute for Brain and Neural Systems Tel: 401-863-3920 Brown University Fax: 401-863-3934 Providence, RI 02912 -------------------------------------------------------------------------------- @phdthesis{Perrone93, AUTHOR = {Michael P. Perrone}, TITLE = {Improving Regression Estimation: Averaging Methods for Variance Reduction with Extensions to General Convex Measure Optimization}, YEAR = {1993}, SCHOOL = {Brown University, Institute for Brain and Neural Systems; Dr. Leon N Cooper, Thesis Supervisor}, MONTH = {May} } @inproceedings{PerroneCooper93CAIP, AUTHOR = {Michael P. Perrone and Leon N Cooper}, TITLE = {When Networks Disagree: Ensemble Method for Neural Networks}, BOOKTITLE = {Neural Networks for Speech and Image processing}, YEAR = {1993}, PUBLISHER = {Chapman-Hall}, EDITOR = {R. J. Mammone}, NOTE = {[To Appear]}, where = {London} } @inproceedings{PerroneCooper93WCNN, AUTHOR = {Michael P. Perrone and Leon N Cooper}, TITLE = {Learning from What's Been Learned: Supervised Learning in Multi-Neural Network Systems}, BOOKTITLE = {Proceedings of the World Conference on Neural Networks}, YEAR = {1993}, PUBLISHER = {INNS} } --------------------- @inproceedings{Perrone91, AUTHOR = {M. P. Perrone}, TITLE = {A Novel Recursive Partitioning Criterion}, BOOKTITLE = {Proceedings of the International Joint Conference on Neural Networks}, YEAR = {1991}, PUBLISHER = {IEEE}, PAGES = {989}, volume = {II} } @inproceedings{Perrone92, AUTHOR = {M. P. Perrone}, TITLE = {A Soft-Competitive Splitting Rule for Adaptive Tree-Structured Neural Networks}, BOOKTITLE = {Proceedings of the International Joint Conference on Neural Networks}, YEAR = {1992}, PUBLISHER = {IEEE}, PAGES = {689-693}, volume = {IV} } @inproceedings{PerroneIntrator92, AUTHOR = {M. P. Perrone and N. Intrator}, TITLE = {Unsupervised Splitting Rules for Neural Tree Classifiers}, BOOKTITLE = {Proceedings of the International Joint Conference on Neural Networks}, YEAR = {1992}, ORGANIZATION = {IEEE}, PAGES = {820-825}, volume = {III} } From mpp at cns.brown.edu Thu Jul 29 03:27:21 1993 From: mpp at cns.brown.edu (Michael P. Perrone) Date: Thu, 29 Jul 93 03:27:21 EDT Subject: Multiple Models, Committee of nets etc... Message-ID: <9307290727.AA19084@cns.brown.edu> Tom Dietterich write: > This analysis predicts that using a committee of very diverse > algorithms (i.e., having diverse approximation errors) would yield > better performance (as long as the committee members are competent) > than a committee made up of a single algorithm applied multiple times > under slightly varying conditions. and David Wolpert writes: >There is a good deal of heuristic and empirical evidence supporting >this claim. In general, when using stacking to combine generalizers, >one wants them to be as "orthogonal" as possible, as Tom maintains. One minor result from my thesis shows that when the estimators are orthogonal in the sense that E[n_i(x)n_j(x)] = 0 for all i<>j where n_i(x) = f(x) - f_i(x), f(x) is the target function, f_i(x) is the i-th estimator and the expected value is over the underlying distribution; then the MSE of the average estimator goes like 1/N times the average of the MSE of the estimators where N is the number of estimators in the population. This is a shocking result because all we have to do to get arbitrarily good performance is to increase the size of our estimator population! Of course in practice, the nets are correlated and the result is no longer true. Michael -------------------------------------------------------------------------------- Michael P. Perrone Email: mpp at cns.brown.edu Institute for Brain and Neural Systems Tel: 401-863-3920 Brown University Fax: 401-863-3934 Providence, RI 02912 From mpp at cns.brown.edu Thu Jul 29 03:45:48 1993 From: mpp at cns.brown.edu (Michael P. Perrone) Date: Thu, 29 Jul 93 03:45:48 EDT Subject: Multiple Models, Committee of nets etc... Message-ID: <9307290745.AA19374@cns.brown.edu> INNS SIG on Hybrid Neural Systems --------------------------------- The INNS has recently started a special interest group for hybrid neural systems which provides another forum for people interested in methods for combining networks and algorithms for improved performance. If you are interested in joining and receiving a membership list, please send email to me or Larry Medsker. me ----> mpp at cns.brown.edu Larry -> medsker at american.edu Thanks, Michael -------------------------------------------------------------------------------- Michael P. Perrone Email: mpp at cns.brown.edu Institute for Brain and Neural Systems Tel: 401-863-3920 Brown University Fax: 401-863-3934 Providence, RI 02912 From farrar at cogsci.UCSD.EDU Thu Jul 29 16:21:34 1993 From: farrar at cogsci.UCSD.EDU (Scott Farrar) Date: Thu, 29 Jul 93 13:21:34 PDT Subject: committees Message-ID: <9307292021.AA09768@cogsci.UCSD.EDU> John Hampshire characterized a committee as a collection of biased estimators; the idea being that a collection of many different kinds of bias might constitute a unbiased estimator. I was wondering if anyone had any ideas about how this might be related to, supported by, or refuted by the Central Limit Theorem. Could experimental variances or confounds be likened to "biases", and if so, do these "average out" in a manner which can give us a useful mean or useful estimator? --Scott Farrar From mpp at cns.brown.edu Thu Jul 29 16:20:28 1993 From: mpp at cns.brown.edu (Michael P. Perrone) Date: Thu, 29 Jul 93 16:20:28 EDT Subject: Harris Drucker comments Message-ID: <9307292020.AA22330@cns.brown.edu> Harris Drucker writes: > In our work in OCR > using multilayer networks (single layer networks are not powerful enough) > boosting has ALWAYS improved performance. This is a direct result of averaging [1]. > Can someone find a better single network (using the original database) that is > better than a boosted committee. Maybe. But good networks are hard to find and > if you can find it, you can probably boost it. This is the important take-home message for all of these averaging techniques: If you can generate a good estimator, you can ALWAYS improve it using averaging. Of course, you will eventually reach the point of diminishing returns on your resource investment (e.g. averaging several different sets of averaged estimators and then averaging the average of the averages ad infinitum). > A final note: rather than straight voting, we have found that simply summing > the respective outputs of the three neural networks gives MUCH better results > (as quoted above). This result is due to the fact that averaging the outputs is guaranteed to improve performance for MSE whereas averaging the Winner Take All output (i.e. voting) corresponds to a different optimization measure and there is no guarantee that averaging in one topology will improve the performance in the other [2]. [1] Michael P. Perrone and Leon N Cooper, When Networks Disagree: Ensemble Method for Neural Networks, In _Neural Networks for Speech and Image Processing_, R. J. Mammone (ed.), Chapman-Hall, London: 1993). [2] Michael P. Perrone and Leon N Cooper, Learning from what's been learned: Supervised learning in multi-neural network systems, Proceedings of the World Conference on Neural Networks 1993, INNS. Michael -------------------------------------------------------------------------------- Michael P. Perrone Email: mpp at cns.brown.edu Institute for Brain and Neural Systems Tel: 401-863-3920 Brown University Fax: 401-863-3934 Providence, RI 02912 From mpp at cns.brown.edu Thu Jul 29 16:50:53 1993 From: mpp at cns.brown.edu (Michael P. Perrone) Date: Thu, 29 Jul 93 16:50:53 EDT Subject: combining generalizers' guesses Message-ID: <9307292050.AA22464@cns.brown.edu> Barak Pearlmutter writes: >For instance, if you run backpropagation on the same data twice, with >the same architecture and all the other parameters held the same, it >will still typically come up with different answers. Eg due to >differences in the random initial weights. .. > Averaging out this effect is a guaranteed win. > --Barak. I agree. I think that the surprising issue here is that the local minima that people have been trying like crazy to avoid for the passed few years can actually be used to improve performance! I think that one direction to take is be to stop trying to find the global optimum and instead try to find "complementary" or "orthogonal" local optima. Reilly's multi-resolution architectures [1], Schapire's Boosting algorithm [2] and Brieman's Stacked Regression [3] are good examples. Of course there are many other approaches that one could take some of which are proposed in my PhD thesis. I think that there is a lot of work to be done in this area. I'd be glad to hear from people experimenting with related algorithms or who are interested in discussing more details. Michael -------------------------------------------------------------------------------- Michael P. Perrone Email: mpp at cns.brown.edu Institute for Brain and Neural Systems Tel: 401-863-3920 Brown University Fax: 401-863-3934 Providence, RI 02912 [1] @incollection{ReillyEtAl87, AUTHOR = {R. L. Reilly and C. L. Scofield and C. Elbaum and L. N Cooper}, TITLE = {Learning System Architectures Composed of Multiple Learning Modules}, BOOKTITLE = {Proc. IEEE First Int. Conf. on Neural Networks}, YEAR = {1987}, PUBLISHER = {IEEE}, PAGES = {495-503}, volume = 2 } [2] @article{Schapire90, AUTHOR = {R. Schapire}, TITLE = {The strength of weak learnability}, JOURNAL = {Machine Learning}, YEAR = {1990}, NUMBER = {2}, PAGES = {197-227}, VOLUME = {5} } [3] @techreport{Breiman92, AUTHOR = {Leo Breiman}, TITLE = {Stacked regression}, YEAR = {1992}, INSTITUTION = {Department of Statistics, University of California, Berkeley}, MONTH = {August}, NUMBER = {{TR}-367}, TYPE = {Technical Report} } From wolf at planck.lanl.gov Thu Jul 29 20:01:54 1993 From: wolf at planck.lanl.gov (David R Wolf) Date: Thu, 29 Jul 93 18:01:54 -0600 Subject: Two papers available Message-ID: <9307300001.AA00296@planck.lanl.gov> FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/wolpert.entropy.tar.Z FTP-filename: /pub/neuroprose/wolf.mutual1.ps.Z FTP-filename: /pub/neuroprose/wolf.mutual1.ps.Z The following tech reports are now available in neuroprose. The papers have been submitted, and any comments are welcomed. david ======================================================= D.H.Wolpert and D.R. Wolf: Estimating Functions of Probability Distributions from a Finite Set of Samples, Part 1: Bayes Estimators and the Shannon Entropy. D.R. Wolf and D.H. Wolpert Estimating Functions of Distributions from A Finite Set of Samples, Part 2: Bayes Estimators for Mutual Information, Chi-Squared, Covariance and other Statistics. We present estimators for entropy and other functions of a discrete probability distribution when the data is a finite sample drawn from that probability distribution. In particular, for the case when the probability distribution is a joint distribution, we present finite sample estimators for the mutual information, covariance, and chi-squared functions of that probability distribution. ======================================================= Retrieval instructions: The papers are found in the neuroprose archive under wolpert.entropy.tar.Z 21 pages text, 6 figures, captions wolf.mutual1.ps.Z 25 pages wolf.mutual2.ps.Z 25 pages The INDEX entries are wolpert.entropy.tar.Z Small sample estimator for entropy. wolf.mutual1.ps.Z wolf.mutual2.ps.Z Small sample estimators for mutual information and other functions. To retrieve these files from the neuroprose archives: For simplicity, make a directory <newdirname> on your system, then unix> ftp cheops.cis.ohio-state.edu Name (cheops.cis.ohio-state.edu:becker): anonymous Password: (use your email address) ftp> cd pub/neuroprose ftp> binary ftp> lcd <newdirname> Local directory now <newdirname> ftp> get wolpert.entropy.tar.Z 200 PORT command successful. 150 Opening BINARY mode data connection for wolpert.entropy.tar.Z From mpp at cns.brown.edu Fri Jul 30 21:29:27 1993 From: mpp at cns.brown.edu (Michael P. Perrone) Date: Fri, 30 Jul 93 21:29:27 EDT Subject: Preprint available Message-ID: <9307310129.AA05190@cns.brown.edu> FTP-host: archive.cis.ohio-state.edu FTP-filename: perrone.MSE-averaging.ps.Z The following paper is now available in neuroprose. It was presented at the 1992 CAIP Conference at Rutger University. It will appear in Neural Networks for Speech and Image processing, R. J. Mammone (ed.), Chapman-Hall, 1993. The paper is relevant to the recent discussion on Connectionists about multiple neural network estimators. Enjoy! Michael -------------------------------------------------------------------------------- Michael P. Perrone Email: mpp at cns.brown.edu Institute for Brain and Neural Systems Tel: 401-863-3920 Brown University Fax: 401-863-3934 Providence, RI 02912 -------------------------------------------------------------------------------- When Networks Disagree: Ensemble Method for Neural Networks M. P. Perrone and L. N Cooper Abstract: This paper presents a general theoretical framework for ensemble methods of constructing significantly improved regression estimates. Given a population of regression estimators, we construct a hybrid estimator which is as good as or better than, in the MSE sense, any estimator in the population. We argue that the ensemble method presented has several properties: 1) It efficiently uses all the networks of a population - none of the networks need be discarded. 2) It efficiently uses all the available data for training without over- fitting. 3) It inherently performs regularization by smoothing in functional space which helps to avoid over-fitting. 4) It utilizes local minima to construct improved estimates whereas other neural network algorithms are hindered by local minima. 5) It is ideally suited for parallel computation. 6) It leads to a very useful and natural measure of the number of distinct estimators in a population. 7) The optimal parameters of the ensemble estimator are given in closed form. Experimental results are provided which show that the ensemble method dramatically improves neural network performance on difficult real-world optical character recognition tasks. -------------------------------------------------------------------------------- Retrieval instructions: The paper is found in the neuroprose archive under perrone.MSE-averaging.ps.Z 15 pages To retrieve these files from the neuroprose archives: unix> ftp cheops.cis.ohio-state.edu Name (cheops.cis.ohio-state.edu:username): anonymous Password: (use your email address) ftp> cd pub/neuroprose ftp> binary ftp> get perrone.MSE-averaging.ps.Z ftp> bye  From hicks at cs.titech.ac.jp Fri Jul 30 15:49:15 1993 From: hicks at cs.titech.ac.jp (hicks@cs.titech.ac.jp) Date: Fri, 30 Jul 93 15:49:15 JST Subject: Synthetically enlarging the database In-Reply-To: Drucker Harris's message of Wed, 28 Jul 93 17:14:17 EDT <9307282114.AA00408@harris.monmouth.edu> Message-ID: <9307300649.AA09200@hilbert> About synthetically enlarging the database: Drucker Harris writes: >In our work in OCR using multilayer networks (single >layer networks are not powerful enough) boosting has >ALWAYS improved performance. Synthetically enlarging >the database using deformations of the original data is >essential. (Point of view from outside the learning system) It seems to me that the cost of obtaining training data is an issue implicit in the above statement, and ought to be made explicit. As the number of data in the original training set increases, the benefits of synthetically created data will become less. Moreover, wouldn't it be correct to say that one could always do better by using N extra randomly selected training data than by using N extra synthetically created data? Nevertheless, the cost of obtaining training data is a real factor and synthetically created training data may be virtually free. (Point of view from inside the system) But what about the cost of learning any training data, synthetic or otherwise? Synthesis of training data may be cheaper than obtaining real training data, but it still has to be learned. Is it possible to have synthesis without extra learning cost? Consider that synthetically creating data has the effect of compressing the size of the input space (and thus enforcing smoothing) in same way as would a preprocessing front giving translational invariance. In both cases a single input is given to the system and the system learns many samples, explicitly in the case of synthetic creation, implicitly in the case of translational invariance. The former incurrs extra learning cost, the latter none. I know this is not a good example, because translational invariance is a trivial problem, and the difficult problems do require more learning. Synthetically creating data is one way to go about smoothing the area around a (non-synthetic) training sample, but aren't there others? For example, adding a penalty term for the complexity of the output function (or some internal rep. if there is no continuous output function) around the sample point. Craig Hicks hicks at cs.titech.ac.jp Ogawa Laboratory, Dept. of Computer Science Tokyo Institute of Technology, Tokyo, Japan lab: 03-3726-1111 ext. 2190 home: 03-3785-1974 fax: +81(3)3729-0685 (from abroad), 03-3729-0685 (from Japan)  From lars at eiffel.ei.dth.dk Fri Jul 30 04:33:31 1993 From: lars at eiffel.ei.dth.dk (Lars Kai Hansen) Date: Fri, 30 Jul 93 09:33:31 +0100 Subject: committee's Message-ID: <9307300833.AA01270@eiffel.ei.dth.dk> It is great that attention is focussed on the effective use of solution space samples for non-linear models. Allow me to promote our pre-historic work on network voting: NEURAL NETWORK ENSEMBLES by L.K. Hansen and P. Salamon IEEE Trans. Pattern Analysis and Machine Intell. {\bf 12}, 993-1001, (1990) Besides finding experimentally that the ensemble consensus often is 'better than the best'.... expressions were derived for the ensemble error rate based on different assumptions on error correlations. The key invention is to describe the ensemble by the 'difficulty distribution'. This description was inspired by earlier work on so called 'N-version programming' by Eckhardt and Lee: A THEORETICAL BASIS FOR THE ANALYSIS OF MULTIVERSION SOFTWARE SUBJECT TO COINCIDENT ERRORS by D.E. Eckhardt and L.D. Lee IEEE Trans. Software Eng. {\bf 11} 1511-1517 (1985) In a feasibility study on Handwritten digits the viability of voting among small ensembles was confirmed (the consensus outperformed the best individual by 25%) and the theoretical estimate of ensemble performance was found to fit well to the observed. Further, the work of Schwartz et al. [Neural Computation {\bf 2}, 371-382 (1990)] was applied to estimate the learning curve based on the distribution of generalizations of a small ensemble: ENSEMBLE METHODS FOR HANDWRITTEN DIGIT RECOGNITION by L.K. Hansen, Chr. Liisberg, and P. Salamon In proceedings of The Second IEEE Workshop on Neural Networks for Signal Processing: NNSP'92 Eds. S.Y. Kung et al., IEEE Service Center Piscataway, 333-342, (1992) While I refer to these methods as *ensemble* methods (to emphasize the statistical relation and to invoke associations to artistic ensembles), I note that theorists have reserved *committee machines* for a special, constrained, network architecture (see eg. Schwarze and Hertz [Euro.Phys.Lett. {\bf 20}, 375-380, (1992)]). In the theorist committee (TC) all weights from hiddens to output are fixed to unity during training. This is very different from voting among independently trained networks: while the TC explores the function space of a large set of parameters (hence needs very many training examples), a voting system based on independently trained nets only explores the function space of the individual network. The voting system can improve generalization by reducing 'random' errors due to training algorithms etc. --------------------- Lars Kai Hansen, Tel: (+45) 4593 1222 (tone) 3889 CONNECT, Electronics Institute B349 Fax: (+45) 4288 0117 Technical University of Denmark email: lars at eiffel.ei.dth.dk DK-2800 Lyngby DENMARK  From skalsky at aaai.org Fri Jul 30 16:02:00 1993 From: skalsky at aaai.org (Rick Skalsky) Date: Fri, 30 Jul 93 13:02:00 PDT Subject: AAAI-94 Call for Papers Message-ID: <9307302002.AA16439@aaai.org> Twelfth National Conference on Artificial Intelligence (AAAI-94) Seattle, Washington July, 31-August 4, 1994 Call for Papers AAAI-94 is the twelfth national conference on artificial intelligence (AI). The purpose of the conference is to promote research in AI and scientific interchange among AI researchers and practitioners. Papers may represent significant contributions to any aspects of AI: a) principles underlying cognition, perception, and action; b) design, application, and evaluation of AI algorithms and systems; c) architectures and frameworks for classes of AI systems; and d) analysis of tasks and domains in which intelligent systems perform. One of the most important functions served by the national conference is to provide a forum for information exchange and interaction among researchers working in different sub- disciplines, in different research paradigms, and in different stages of research. Based on discussions among program committee members during the past few years, we aim to expand active participation in this year's conference to include a larger cross-section of the AI community and a larger cross-section of the community's research activities. Accordingly, we encourage submission of papers that: describe theoretical, empirical, or experimental results; represent areas of AI that may have been under-represented in recent conferences; present promising new research concepts, techniques, or perspectives; or discuss issues that cross traditional sub-disciplinary boundaries. As outlined below, we have revised and expanded the paper review criteria to recognize this broader spectrum of research contributions. We intend to accept more of the papers that are submitted and to publish them in an expanded conference proceedings. Requirements for Submission Authors must submit six (6) complete printed copies of their papers to the AAAI office by January 24, 1994. Papers received after that date will be returned unopened. Notification of receipt will be mailed to the first author (or designated author) soon after receipt. All inquiries regarding lost papers must be made by February 7, 1994. Authors should also send their paper's title page in an electronic mail message to abstract at aaai.org by January 24, 1994. Notification of acceptance or rejection of submitted papers will be mailed to the first author (or designated author) by March 11, 1994. Camera-ready copy of accepted papers will be due about one month later. Paper Format for Review All six (6) copies of a submitted paper must be clearly legible. Neither computer files nor fax submissions are acceptable. Submissions must be printed on 8 1/2" x 11" or A4 paper using 12 point type (10 characters per inch for typewriters). Each page must have a maximum of 38 lines and an average of 75 characters per line (corresponding to the LaTeX article-style, 12 point). Double-sided printing is strongly encouraged. Length The body of submitted papers must be at most 12 pages, including title, abstract, figures, tables, and diagrams, but excluding the title page and bibliography. Papers exceeding the specified length and formatting requirements are subject to rejection without review. Blind Review Reviewing for AAAI-94 will be blind to the identities of the authors. This requires that authors exercise some care not to identify themselves in their papers. Each copy of the paper must have a title page, separate from the body of the paper, including the title of the paper, the names and addresses of all authors, a list of content areas (see below) and any acknowledgements. The second page should include the exact same title, a short abstract of less than 200 words, and the exact same content areas, but not the names nor affiliations of the authors. The references should include all published literature relevant to the paper, including previous works of the authors, but should not include unpublished works of the authors. When referring to one's own work, use the third person, rather than the first person. For example, say "Previously, Korf [17] has shown that...", rather than "In our previous work [17] we have shown that...". Try to avoid including any information in the body of the paper or references that would identify the authors or their institutions. Such information can be added to the final camera-ready version for publication. Please do not staple the title page to the body of the paper. Electronic Title Page A title page should also be sent via electronic mail to abstract at aaai.org, in plain ASCII text, without any formatting commands for LaTeX, Scribe, etc. Each section of the electronic title page should be preceded by the name of that section as follows: title: <title> author: <name of first author> address: <address of first author> author: <name of last author> address: <address of last author> abstract: <abstract> content areas: <first area>, ..., <last area> To facilitate the reviewing process, authors are requested to select 1-3 appropriate content areas from the list below. Authors are welcome to add additional content area descriptors as needed. AI architectures, artificial life, automated reasoning, control, belief revision, case-based reasoning, cognitive modeling, common sense reasoning, computational complexity, computer-aided education, constraint satisfaction, decision theory, design, diagnosis, distributed AI, expert systems, game playing, genetic algorithms, geometric reasoning, knowledge acquisition, knowledge representation, machine learning, machine translation, mathematical foundations, multimedia, natural language processing, neural networks, nonmonotonic reasoning, perception, philosophical foundations, planning, probabilistic reasoning, problem solving, qualitative reasoning, real-time systems, robotics, scheduling, scientific discovery, search, simulation, speech understanding, temporal reasoning, theorem proving, user interfaces, virtual reality, vision Submissions to Multiple Conferences Papers that are being submitted to other conferences, whether verbatim or in essence, must reflect this fact on the title page. If a paper appears at another conference (with the exception of specialized workshops), it must be withdrawn from AAAI-94. Papers that violate these requirements are subject to rejection without review. Review Process Program committee (PC) members will identify papers they are qualified to review based on each paper's title, content areas, and electronic abstract. This information, along with other considerations, will be used to assign each submitted paper to two PC members. Using the criteria given below, they will review the paper independently. If the two reviewers of a paper agree to accept or reject it, that recommendation will be followed. If they do not agree, a third reviewer will be assigned and the paper will be discussed by an appropriate sub-group of the PC during its meeting in March. Note that the entire review process will be blind to the identities of the authors and their institutions. In general, papers will be accepted if they receive at least two positive reviews or if they generate an interesting controversy among the reviewers. The final decisions on all papers will be made by the program chairs. Questions that will appear on the review form appear below. Authors are advised to bear these questions in mind while writing their papers. Reviewers will look for papers that meet at least some (though not necessarily all) of the criteria in each category. Significance How important is the problem studied? Does the approach offered advance the state of the art? Does the paper stimulate discussion of important issues or alternative points of view? Originality Are the problems and approaches new? Is this a novel combination of existing techniques? Does the paper point out differences from related research? Does it address a new problem or one that has not been studied in depth? Does it introduce an interesting research paradigm? Does the paper describe an innovative combination of AI techniques with techniques from other disciplines? Does it introduce an idea that appears promising or might stimulate others to develop promising alternatives? Quality Is the paper technically sound? Does it carefully evaluate the strengths and limitations of its contributions? Are its claims backed up? Does the paper offer a new form of evidence in support of or against a well-known technique? Does the paper back up a theoretical idea already in the literature with experimental evidence? Does it offer a theoretical analysis of prior experimental results? Clarity Is the paper clearly written? Does it motivate the research? Does it describe the inputs, outputs, and basic algorithms employed? Are the results described and evaluated? Is the paper organized in a logical fashion? Is the paper written in a manner that makes its content accessible to most AI researchers? Publication Accepted papers will be allocated six (6) pages in the conference proceedings. Up to two (2) additional pages may be used at a cost to the authors of $250 per page. Papers exceeding eight (8) pages and those violating the instructions to authors will not be included in the proceedings. Copyright Authors will be required to transfer copyright of their paper to AAAI. Paper Submissions & Inquiries Please send papers and conference registration inquiries to: AAAI-94 American Association for Artificial Intelligence 445 Burgess Drive Menlo Park, CA 94025-3496 Registration and call clarification inquiries (ONLY) may be sent to the Internet address: NCAI at aaai.org. Please send program suggestions and inquiries to: Barbara Hayes-Roth, Program Cochair Knowledge Systems Laboratory Stanford University 701 Welch Road, Building C Palo Alto, CA 94304 bhr at ksl.stanford.edu Richard Korf, Program Cochair Department of Computer Science University of California, Los Angeles Los Angeles, CA 90024 korf at cs.ucla.edu Howard Shrobe, Associate Program Chair Artificial Intelligence Laboratory Massachusetts Institute of Technology Cambridge, MA, 02139 hes at reagan.ai.mit.edu  From mpp at cns.brown.edu Fri Jul 30 21:45:20 1993 From: mpp at cns.brown.edu (Michael P. Perrone) Date: Fri, 30 Jul 93 21:45:20 EDT Subject: Thesis available Message-ID: <9307310145.AA05445@cns.brown.edu> FTP-host: archive.cis.ohio-state.edu FTP-filename: perrone.thesis.ps.Z A condensed version of my thesis is now available in neuroprose. Hardcopy versions can be obtained from UMI Dissertation Services (800-521-0600). Enjoy! Michael -------------------------------------------------------------------------------- Michael P. Perrone Email: mpp at cns.brown.edu Institute for Brain and Neural Systems Tel: 401-863-3920 Brown University Fax: 401-863-3934 Providence, RI 02912 -------------------------------------------------------------------------------- Improving Regression Estimation: Averaging Methods for Variance Reduction with Extensions to General Convex Measure Optimization M. P. Perrone -------------------------------------------------------------------------------- Retrieval instructions: The thesis is found in the neuroprose archive under perrone.thesis.ps.Z 83 pages To retrieve these files from the neuroprose archives: unix> ftp cheops.cis.ohio-state.edu Name (cheops.cis.ohio-state.edu:username): anonymous Password: (use your email address) ftp> cd pub/neuroprose ftp> binary ftp> get perrone.thesis.ps.Z ftp> bye  From dhw at santafe.edu Fri Jul 30 14:50:29 1993 From: dhw at santafe.edu (dhw@santafe.edu) Date: Fri, 30 Jul 93 12:50:29 MDT Subject: No subject Message-ID: <9307301850.AA07624@zia> Tony Chan writes: >>> I would like to raise the following question: Is there such a thing as the best method to learn a random idea? More precisely, given n random (machine) learning problems, is there such a thing as the best method for dealing with these problems that will give the best overall performance. There may be some best methods for dealing with some specific types of learning problems but is there one that would deal with any learning problem and give the best overall performance? >>> The answer depends on the precise way the problem is phrased. But in general, the answer is (provably) no, at least as far as off-training set error is concerned. For example, if the prior distribution over target functions is uniform, then all algorithms have the exact same average off-training set performance. Moreover, in a broad number of contexts, it is always true that if "things" (be they priors, training sets, or whatever) are such that algorithm 1 will outperform algorithm 2, then one can always set up those "things" differently, so that algorithm 2 outperforms algorithm 1, at least as far as off-training set behavior is concerned. Many of the results in the literature which appear to dispute this are simply due to use of an error function which is not restricted to being off-training set. In other words, there's always a "win" if you perform rationally on the training set (e.g., reproduce it exactly, when there's no noise), if your error function gives you points for performing rationally on the training set. In a certain sense, this is trivial, and what's really interesting is off-training set behavior. In any case, this automatic on-training set win is all those aforementioned results refer to; in particular, they imply essentially nothing concerning performance off of the training set.