From Connectionists-Request at CS.CMU.EDU Sun Sep 1 00:05:14 1991 From: Connectionists-Request at CS.CMU.EDU (Connectionists-Request@CS.CMU.EDU) Date: Sun, 01 Sep 91 00:05:14 -0400 Subject: Bi-monthly Reminder Message-ID: <8717.683697914@B.GP.CS.CMU.EDU> *** DO NOT FORWARD TO ANY OTHER LISTS *** This is an automatically posted bi-monthly reminder about how the CONNECTIONISTS list works and how to access various online resources. CONNECTIONISTS is not an edited forum like the Neuron Digest, or a free-for-all newsgroup like comp.ai.neural-nets. It's somewhere in between, relying on the self-restraint of its subscribers. Membership in CONNECTIONISTS is restricted to persons actively involved in neural net research. The following posting guidelines are designed to reduce the amount of irrelevant messages sent to the list. Before you post, please remember that this list is distributed to over a thousand busy people who don't want their time wasted on trivia. Also, many subscribers pay cash for each kbyte; they shouldn't be forced to pay for junk mail. Happy hacking. -- Dave Touretzky & Scott Crowder --------------------------------------------------------------------- What to post to CONNECTIONISTS ------------------------------ - The list is primarily intended to support the discussion of technical issues relating to neural computation. - We encourage people to post the abstracts of their latest papers and tech reports. - Conferences and workshops may be announced on this list AT MOST twice: once to send out a call for papers, and once to remind non-authors about the registration deadline. A flood of repetitive announcements about the same conference is not welcome here. - Requests for ADDITIONAL references. This has been a particularly sensitive subject lately. Please try to (a) demonstrate that you have already pursued the quick, obvious routes to finding the information you desire, and (b) give people something back in return for bothering them. The easiest way to do both these things is to FIRST do the library work to find the basic references, then POST these as part of your query. Here's an example: WRONG WAY: "Can someone please mail me all references to cascade correlation?" RIGHT WAY: "I'm looking for references to work on cascade correlation. I've already read Fahlman's paper in NIPS 2, his NIPS 3 abstract, and found the code in the nn-bench archive. Is anyone aware of additional work with this algorithm? I'll summarize and post results to the list." - Announcements of job openings related to neural computation. - Short reviews of new text books related to neural computation. To send mail to everyone on the list, address it to Connectionists at CS.CMU.EDU ------------------------------------------------------------------- What NOT to post to CONNECTIONISTS: ----------------------------------- - Requests for addition to the list, change of address and other administrative matters should be sent to: "Connectionists-Request at cs.cmu.edu" (note the exact spelling: many "connectionists", one "request"). If you mention our mailing list to someone who may apply to be added to it, please make sure they use the above and NOT "Connectionists at cs.cmu.edu". - Requests for e-mail addresses of people who are believed to subscribe to CONNECTIONISTS should be sent to postmaster at appropriate-site. If the site address is unknown, send your request to Connectionists-Request at cs.cmu.edu and we'll do our best to help. A phone call to the appropriate institution may sometimes be simpler and faster. - Note that in many mail programs a reply to a message is automatically "CC"-ed to all the addresses on the "To" and "CC" lines of the original message. If the mailer you use has this property, please make sure your personal response (request for a Tech Report etc.) is NOT broadcast over the net. - Do NOT tell a friend about Connectionists at cs.cmu.edu. Tell him or her only about Connectionists-Request at cs.cmu.edu. This will save your friend from public embarrassment if she/he tries to subscribe. - Limericks should not be posted here. ------------------------------------------------------------------------------- The CONNECTIONISTS Archive: --------------------------- All e-mail messages sent to "Connectionists at cs.cmu.edu" starting 27-Feb-88 are now available for public perusal. A separate file exists for each month. The files' names are: arch.yymm where yymm stand for the obvious thing. Thus the earliest available data are in the file: arch.8802 Files ending with .Z are compressed using the standard unix compress program. To browse through these files (as well as through other files, see below) you must FTP them to your local machine. ------------------------------------------------------------------------------- How to FTP Files from the CONNECTIONISTS Archive ------------------------------------------------ 1. Open an FTP connection to host B.GP.CS.CMU.EDU (Internet address 128.2.242.8). 2. Login as user anonymous with password your username. 3. 'cd' directly to one of the following directories: /usr/connect/connectionists/archives /usr/connect/connectionists/bibliographies 4. The archives and bibliographies directories are the ONLY ones you can access. You can't even find out whether any other directories exist. If you are using the 'cd' command you must cd DIRECTLY into one of these two directories. Access will be denied to any others, including their parent directory. 5. The archives subdirectory contains back issues of the mailing list. Some bibliographies are in the bibliographies subdirectory. Problems? - contact us at "Connectionists-Request at cs.cmu.edu". ------------------------------------------------------------------------------- How to FTP Files from the Neuroprose Archive -------------------------------------------- Anonymous FTP on cheops.cis.ohio-state.edu (128.146.8.62) pub/neuroprose directory This directory contains technical reports as a public service to the connectionist and neural network scientific community. Researchers may place electronic versions of their preprints or articles in this directory, announce availability, and other interested researchers can rapidly retrieve and print the postscripts. This saves copying, postage and handling, by having the interested reader supply the paper. To place a file, put it in the Inbox subdirectory, and send mail to pollack at cis.ohio-state.edu. Within a couple of days, I will move and protect it, and suggest a different name if necessary. Current naming convention is author.title.filetype[.Z] where title is enough to discriminate among the files of the same author. The filetype is usually "ps" for postscript, our desired universal printing format, but may be tex, which requires more local software than a spooler. Very large files (e.g. over 200k) must be squashed (with either a sigmoid function :) or the standard unix "compress" utility, which results in the .Z affix. To place or retrieve .Z files, make sure to issue the FTP command "BINARY" before transfering files. After retrieval, call the standard unix "uncompress" utility, which removes the .Z affix. An example of placing a file is attached as an appendix, and a shell script called Getps in the directory can perform the necessary retrival operations. For further questions contact: Jordan Pollack Email: pollack at cis.ohio-state.edu Here is an example of naming and placing a file: gvax> cp i-was-right.txt.ps rosenblatt.reborn.ps gvax> compress rosenblatt.reborn.ps gvax> ftp cheops.cis.ohio-state.edu Connected to cheops.cis.ohio-state.edu. 220 cheops.cis.ohio-state.edu FTP server ready. Name: anonymous 331 Guest login ok, send ident as password. Password:neuron 230 Guest login ok, access restrictions apply. ftp> binary 200 Type set to I. ftp> cd pub/neuroprose/Inbox 250 CWD command successful. ftp> put rosenblatt.reborn.ps.Z 200 PORT command successful. 150 Opening BINARY mode data connection for rosenblatt.reborn.ps.Z 226 Transfer complete. 100000 bytes sent in 3.14159 seconds ftp> quit 221 Goodbye. gvax> mail pollack at cis.ohio-state.edu Subject: file in Inbox. Jordan, I just placed the file rosenblatt.reborn.ps.Z in the Inbox. The INDEX sentence is "Boastful statements by the deceased leader of the neurocomputing field." Please let me know when it is ready to announce to Connectionists at cmu. BTW, I enjoyed reading your review of the new edition of Perceptrons! Frank ------------------------------------------------------------------------ How to FTP Files from the NN-Bench Collection --------------------------------------------- 1. Create an FTP connection from wherever you are to machine "pt.cs.cmu.edu" (128.2.254.155). 2. Log in as user "anonymous" with password your username. 3. Change remote directory to "/afs/cs/project/connect/bench". Any subdirectories of this one should also be accessible. Parent directories should not be. 4. At this point FTP should be able to get a listing of files in this directory and fetch the ones you want. Problems? - contact us at "nn-bench-request at cs.cmu.edu". From Scott_Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU Sun Sep 1 21:34:27 1991 From: Scott_Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU (Scott_Fahlman@SEF-PMAX.SLISP.CS.CMU.EDU) Date: Sun, 01 Sep 91 21:34:27 -0400 Subject: Processing of auditory sequences In-Reply-To: Your message of Sun, 01 Sep 91 19:23:00 -0500. <683767402/xdh@SPEECH2.CS.CMU.EDU> Message-ID: For the purpose of speech-compression, current technology using vector quantization can compress speech to 2k bits/s without much fidelity loss. Even lower rate (at 200-800 bits/s) also has acceptable intelligibility. They can be found in many commercial applications. I think the original question was not asking about data-compression in the usual sense, but about devices that take a normal speech signal and try to play it back as fast as possible without loss of intelligibility -- sort of speed-reading for the blind. If you just speed up the tape, all the frequencies go up and the speech is hard to understand. So these machines try to keep the frequencies the same while speeding up everything else. -- Scott From issnnet at park.bu.edu Sun Sep 1 11:04:24 1991 From: issnnet at park.bu.edu (issnnet@park.bu.edu) Date: Sun, 1 Sep 91 11:04:24 -0400 Subject: we want your messages! Message-ID: <9109011504.AA01159@copley.bu.edu> *** WE WANT YOUR POSTINGS !!!! *** This is a reminder that the new USENET newsgroup comp.org.issnnet was created earlier in August. The newsgroup is part of the International Student Society for Neural Networks (ISSNNet) effort to establish a centralized organization for students in Neural Networks. Many of the messages posted to this group would be ideally suited for posting to comp.org.issnnet. For example any requests for housing at a conference site, or sharing rides, or inquiring about academic programs, job offers, or even requests for references. If you have no access to comp.org.issnnet you can still take advantage of this newsgroup by e-mailing your message to "issnnet at park.bu.edu". Please be sure to indicate you wish your message to be relayed to the newsgroup, and include your name and e-mail address, and any other information that may be needed for people to contact you. Please take advantage of this new opportunity, and become part of this useful new endeavour! ISSNNet From kddlab!hcrlgw.crl.hitachi.co.jp!nitin at uunet.UU.NET Mon Sep 2 09:04:01 1991 From: kddlab!hcrlgw.crl.hitachi.co.jp!nitin at uunet.UU.NET (Nitin Indurkhya) Date: Mon, 2 Sep 91 09:04:01 JST Subject: cascade-correlation Message-ID: <9109020004.AA13200@hcrlgw.crl.hitachi.co.jp> > > Experiments with the Cascade-Correlation Algorithm > Technical report # 91-16 (July 1991) > Jihoon Yang & Vasant Honavar > Department of Computer Science > Iowa State University > folks interested in this might also want to look at the following: "Heuristic Configuration of Hidden Units in Neural Network Classifers" Technical report DCS-TR 279 (June 1991) Nitin Indurkhya and Sholom Weiss Department of Computer Science Rutgers University, New Brunswick, NJ 08903 An abstract can also be found in the IJCNN-91-Seattle proceedings. in this report, the cascade-correlation architecture is evaluated on some well-known real-world datasets. copies of the tech report can be most easily obtained by sending mail to weiss at cs.rutgers.edu --nitin From rosauer at ira.uka.de Mon Sep 2 11:23:05 1991 From: rosauer at ira.uka.de (Bernd Rosauer) Date: Mon, 2 Sep 91 11:23:05 MET DST Subject: connectionist navigation Message-ID: Inspired by the recent summary on chemotaxis by Shawn Lockery I think it would be interesting -- at least to some people -- to compile a bibliography on connectionist approaches to locomotion and navigation. Since I started collecting references on that topic some time ago I will take on the job. The bibliography should include references to work on modeling spatial orientation, cognitive mapping, piloting, and navigation; alife simulations of animat navigation and applications of neural networks to mobile robot obstacle avoidance and path planning. Further suggestions are welcome. In order to bound my own efforts in surveying the references I would like to encourage (a) those people who know that I have some references to their work to send me a complete list, and (b) even to send me older references you are aware of. Although I do not promise to finish the bibliography this month I expect that it could be done this year. So feel free to send me your references. (I know that similar requests have come up in several mailing lists and news groups now and then but I for myself have never seen a summary, even on direct request.) Thanks in advance, Bernd Rosauer Research Center of Computer Science at the University of Karlsruhe, FRG From HJBORTOL at BRFUEM.BITNET Mon Sep 2 10:41:00 1991 From: HJBORTOL at BRFUEM.BITNET (Humberto Jose' Bortolossi) Date: Mon, 02 Sep 91 10:41:00 C Subject: Bibliography to a novice ... Message-ID: Greetings ... I just started to study neural-nets and related topics. But, I have no idea of basic bibliography. Please, could you suggest me some good bo- oks? Is there something available from FTP (remember, I'm a novice in neural nets|)??? Any help will be welcome| Thanks a lot in advance. Humberto Jose' Bortolossi Department of Mathematics From koch at CitIago.Bitnet Mon Sep 2 12:54:04 1991 From: koch at CitIago.Bitnet (Christof Koch) Date: Mon, 2 Sep 91 09:54:04 PDT Subject: Synchronization Binding? Freq. Locking? Bursting? In-Reply-To: Your message <9108211921.AA04522@avw.umd.edu> dated 21-Aug-1991 Message-ID: <910902095401.204025b5@Iago.Caltech.Edu> Sorry for not replying earlier to Thomas Edwards RFC on oscillations and sychnronization. The issue of oscillations is primarily an experimental one. Do single cells in the awake monkey show an oscillatory component in their firing activity and is their firing correlated with the firing of other cells? Well, the only strong oscillations appear to be those found by E. Fetz and V. Murphy in Seattle in motorcortex. If the monkey does a repetitive handmovements these oscillations have a small amplitude. If the monkey has to get a raisin from a Kluever box, their amplitude increases a lot. Crick and I would interpret this to mean that if the money has to attend, oscillations increase. However, with the exceptions of some tantalizing hints of some power at 40 Hz in the spectrum of single cells from some labs, nothing much has been found in the monkey visual cortex, although maybe somebody out there can correct me. Of course, Llinas does see 40 Hz waves between thalamus and cortex in the awake humans using a 37 channel MEG machine. That's why I find synchronized firing, in the absence of oscillations, an attractive possibility, in particular since phase-locked firing will lead to a much bigger response at the single cell level than a temporally smeared out signal (due to the low-pass nature of the neuronal membrane). Christof From CADEPS at BBRNSF11.BITNET Mon Sep 2 13:18:27 1991 From: CADEPS at BBRNSF11.BITNET (JANSSEN Jacques) Date: Mon, 02 Sep 91 19:18:27 +0200 Subject: Evolvability of Recurrent Nets Message-ID: Dear Connectionists, Has anyone out there done any work on, or know about people who have done work on evolvability criteria for time dependent recurrent network behaviors? There's a growing literature now on using the Genetic Algorithm to evolve dynamic behaviors in neural networks, but these attempts sometimes fail. Why? To give a simple (static) example - take the multiplier problem. The aim is to evolve a network which takes two real numbered inputs and returns the product as its output. If the inputs can be of mixed sign, the network does not evolve. If the inputs are always positive, the network evolves. This is just the beginning. I have several examples of desired dynamic behaviors which failed to evolve. Kaufmann has talked about criteria for evolvability of his (simple) Boolean networks. Has anyone seen work on extending these ideas to recurrent nets? Or anything which might do the job? Cheers, Hugo de Garis, Univ of Brussels, & George Mason Univ, VA. From spotter at sanger Mon Sep 2 16:07:45 1991 From: spotter at sanger (Steve Potter) Date: Mon, 2 Sep 91 13:07:45 PDT Subject: Neuroprose index? Message-ID: <9109022007.AA06160@sanger.bio.uci.edu> Browsing through the neuroprose archive directory, I am overwhelmed by the number of interesting topics covered. Is there a list of titles or of brief summaries of the articles that have been archived? The filenames are usually a bit obscure. Steve Potter U of Cal Irvine Psychobiology dept. (714)856-4723 spotter at darwin.bio.uci.edu From dhw at t13.Lanl.GOV Mon Sep 2 17:20:26 1991 From: dhw at t13.Lanl.GOV (David Wolpert) Date: Mon, 2 Sep 91 15:20:26 MDT Subject: No subject Message-ID: <9109022120.AA00432@t13.lanl.gov> John Kruschke writes: "One motive for using RBFs has been the promise of better interpolation between training examples (i.e., better generalization). " Whether or not RBFs result in "better interpolation" is one issue. Whether or not they result in "better generalization" is another. For some situations the two issues are intimately related, and sometimes even identical. For other situations they are not. David Wolpert (dhw at tweety.lanl.gov) From nowlan at helmholtz.sdsc.edu Mon Sep 2 21:21:50 1991 From: nowlan at helmholtz.sdsc.edu (Steven J. Nowlan) Date: Mon, 02 Sep 91 18:21:50 MST Subject: another RBF ref. Message-ID: <9109030121.AA17099@bose> A slightly different slant on the approach of Moody and Darken to training RBF networks, and its relation to fitting statistical mixtures, may be found in: S.J. Nowlan "Maximum Likelihood Competitive Learning" In: D.S. Touretzky (ed.) Advances in Neural Information Processing Systems 2, pp. 574-582, 1990. From terry at jeeves.UCSD.EDU Mon Sep 2 23:20:56 1991 From: terry at jeeves.UCSD.EDU (Terry Sejnowski) Date: Mon, 2 Sep 91 20:20:56 PDT Subject: Neural Computation: 3.3 Message-ID: <9109030320.AA18059@jeeves.UCSD.EDU> Neural Computation Contents 3:3 View: A Practical Approach for Representing Context and for Performing Word Sense Disambiguation Using Neural Networks. Stephen I. Gallant Notes: A Modified Quickprop Algorithm Alistair Craig Veitch and Geoffrey Holmes Letters: Removing Time Variation with the Anti-Hebbian Differential Synapse Graeme Mitchison Simulations of a Reconstructed Cerebellar Purkinje Cell Based on Simplified Channel Kinetics Paul Bush and Terrence Sejnowski On the Mechanisms Underlying Directional Selectivity H. Ogmen 2-Degrees-of-Freedom Robot Path Planning using Cooperative Neural Fields Michael Lemmon Parameter Sensitivity of the Elastic Net Approach to the Traveling Salesman Problem Martin W. Simmen FIR and IIR Synapses, a New Neural Network Architecture for Time Series Modeling A. D. Back and A. C. Tsoi The Transition to Perfect Generalization in Perceptrons Eric B. Baum and Yuh-Dauh Lyuu Learning by Asymmetric Parallel Boltzmann Machines Bruno Apolloni, Diego de Falco Generalization Effects of K-Neighbor Interpolation Training Takeshi Kawabata Including Hints in Training Neural Nets Khalid A. Al-Mashouq and Irving S. Reed On the Characteristics of the Auto-Associative Memory with Nonzero- Diagonal Terms in the Memory Matrix Jung-Hua Wang, Tai-Lang Jong, Thomas F. Krile, and John F. Walkup Handwritten Digit Recognition Using K-Nearest Neighbor, Radial-Basis Function, and Backpropagation Neural Networks Yuchun Lee A Matrix Method for Optimizing a Neural Network Simon A. Barton ----- SUBSCRIPTIONS - VOLUME 3 ______ $35 Student ______ $55 Individual ______ $110 Institution Add $18. for postage and handling outside USA (Back issues from volumes 1 and 2 are available for $28 each.) MIT Press Journals, 55 Hayward Street, Cambridge, MA 02142. (617) 253-2889. ----- From thildebr at athos.csee.lehigh.edu Tue Sep 3 10:42:49 1991 From: thildebr at athos.csee.lehigh.edu (Thomas H. Hildebrandt ) Date: Tue, 3 Sep 91 10:42:49 -0400 Subject: RBFs and Generalization In-Reply-To: David Wolpert's message of Mon, 2 Sep 91 15:20:26 MDT <9109022120.AA00432@t13.lanl.gov> Message-ID: <9109031442.AA07136@athos.csee.lehigh.edu> Begin Wolpert Quote ---------- Date: Mon, 2 Sep 91 15:20:26 MDT From: David Wolpert John Kruschke writes: "One motive for using RBFs has been the promise of better interpolation between training examples (i.e., better generalization). " Whether or not RBFs result in "better interpolation" is one issue. Whether or not they result in "better generalization" is another. For some situations the two issues are intimately related, and sometimes even identical. For other situations they are not. David Wolpert (dhw at tweety.lanl.gov) End Wolpert Quote ---------- I have come to treat interpolation and generalization as the same animal, since obtaining good generalization is a matter of interpolating in the right metric space (i.e. the one that best models the underlying process). If one obtains good results using RBFs, he may assume that the underlying metric space is well represented by some combination of hyperspheres. If he obtains good results using sigmoidally scaled linear functionals, he may assume that the underlying metric space is well represented by some combination of sigmoidal sheets. If the form of the underlying metric space is unknown, then it is a toss-up whether sigmoidal sheets, RBFs, piece-wise hyperplanar, or any number of other basis functions will work best. Thomas H. Hildebrandt Visiting Researcher CSEE Dept. Lehigh University From dhw at t13.Lanl.GOV Tue Sep 3 11:24:48 1991 From: dhw at t13.Lanl.GOV (David Wolpert) Date: Tue, 3 Sep 91 09:24:48 MDT Subject: No subject Message-ID: <9109031524.AA00731@t13.lanl.gov> Thomas Hildebrandt writes: "I have come to treat interpolation and generalization as the same animal, since obtaining good generalization is a matter of interpolating in the right metric space (i.e. the one that best models the underlying process)." Certainly I am sympathetic to this point of view. Simple versions of nearest neighbor interpolation (i.e., memory-based reasoners) do very well in many circumstances. (In fact, I've published a couple of papers making just that point.) However it is trivial to construct problems where the target function is extremely volatile and non-smooth in any "reasonable" metric; who are we to say that Nature should not be allowed to have such target functions? Moreover, for a number of discrete, symbolic problems, the notion of a "metric" is ill-defined, to put it mildly. I am not claiming that metric-based generalizers will necessarily do poorly for these kinds of problems. Rather I'm simply saying that it is a bit empty to state that "If the form of the underlying metric space is unknown, then it is a toss-up whether sigmoidal sheets, RBFs, piece-wise hyperplanar, or any number of other basis functions will work best." That's like saying that if the underlying target function is unknown, then it is a toss-up what hypothesis function will work best. Loosely speaking, "interpolation" is something you do once you've decided on the metric. In addition to such interpolation, "generalization" also involves the preceding step of performing the "toss up" between metrics in a (hopefully) rational manner. David Wolpert (dhw at tweety.lanl.gov) From yair at siren.arc.nasa.gov Tue Sep 3 10:58:23 1991 From: yair at siren.arc.nasa.gov (Yair Barniv) Date: Tue, 3 Sep 91 07:58:23 PDT Subject: No subject Message-ID: <9109031458.AA02472@siren.arc.nasa.gov.> Hello I would appreciate if you could send me a copy of: @techreport{Barto-88a, author="Barto, A. G.", title="From Chemotaxis to Cooperativity: {A}bstract Exercises in Neuronal Learning Strategies", institution="University of Massachusetts", address="Amherst, MA", number="88-65", year=1988, note="To appear in {\it The Computing Neurone}, R. Durbin and R. Maill and G. Mitchison (eds.), Addison-Wesley"}. in a .tex or .ps form Thanks a lot, Yair Barniv From lwyse at park.bu.edu Tue Sep 3 19:09:03 1991 From: lwyse at park.bu.edu (lwyse@park.bu.edu) Date: Tue, 3 Sep 91 19:09:03 -0400 Subject: AI/IS/CS Career Newsletter In-Reply-To: connectionists@c.cs.cmu.edu's message of 3 Sep 91 08:58:35 GM Message-ID: <9109032309.AA03565@copley.bu.edu> What are the rates? - lonce XXX XXX Lonce Wyse | X X Center for Adaptive Systems \ | / X X Boston University \ / 111 Cummington St. Boston,MA 02215 ---- ---- X X X X "The best things in life / \ XXX XXX are emergent." / | \ | From rba at vintage.bellcore.com Wed Sep 4 07:54:11 1991 From: rba at vintage.bellcore.com (Bob Allen) Date: Wed, 4 Sep 91 07:54:11 -0400 Subject: Neuroprose index? Message-ID: <9109041154.AA13516@vintage.bellcore.com> I suggest that the community explore installing neuroprose under WAIS (Wide Area Information Server). This allows keyword searching of documents and a fairly nice X frontend. For information, contact brewster at think.com. If nobody else is interested and I could get the texts from OSU, I'd be happy to do the installation. From thildebr at athos.csee.lehigh.edu Wed Sep 4 12:02:42 1991 From: thildebr at athos.csee.lehigh.edu (Thomas H. Hildebrandt ) Date: Wed, 4 Sep 91 12:02:42 -0400 Subject: Generalization vs. Interpolation In-Reply-To: David Wolpert's message of Tue, 3 Sep 91 09:24:48 MDT <9109031524.AA00731@t13.lanl.gov> Message-ID: <9109041602.AA07514@athos.csee.lehigh.edu> Hildebrandt -- "I have come to treat interpolation and generalization as the same animal, since obtaining good generalization is a matter of interpolating in the right metric space (i.e. the one that best models the underlying process)." Wolpert -- Certainly I am sympathetic to this point of view. Simple versions of nearest neighbor interpolation (i.e., memory-based reasoners) do very well in many circumstances. (In fact, I've published a couple of papers making just that point.) However it is trivial to construct problems where the target function is extremely volatile and non-smooth in any "reasonable" metric; who are we to say that Nature should not be allowed to have such target functions? Moreover, for a number of discrete, symbolic problems, the notion of a "metric" is ill-defined, to put it mildly. I do not presume to tell Nature what to do. We may consider problems for which there is no simple transformation from the input (sensor) space into a (linear) metric space to be "hard" problems, in a sense. Discrete problems, which naturally inhibit interpolation, must be handled by table look-up, i.e. each case treated separately. However, table look-up can be considered to be an extreme case of interpolation -- the transition between one recorded data point and a neighboring one being governed by a Heaviside (threshold) function rather than a straight line. Wolpert -- I am not claiming that metric-based generalizers will necessarily do poorly for these kinds of problems. Rather I'm simply saying that it is a bit empty to state that Hildebrandt -- "If the form of the underlying metric space is unknown, then it is a toss-up whether sigmoidal sheets, RBFs, piece-wise hyperplanar, or any number of other basis functions will work best." Wolpert -- That's like saying that if the underlying target function is unknown, then it is a toss-up what hypothesis function will work best. Loosely speaking, "interpolation" is something you do once you've decided on the metric. In addition to such interpolation, "generalization" also involves the preceding step of performing the "toss up" between metrics in a (hopefully) rational manner. It would be splitting hairs to suggest that the process of choosing an appropriate set of basis functions be called "learning to generalize" rather than "generalization". I could not agree with you more in thinking that the search for an appropriate basis set is one of the important open problems in connectionist research. If nothing is known about the process to be modelled, is there any more efficient way to select a basis than trial-and-error? Are some sets of basis functions more likely to efficiently describe a randomly selected process? Aside from compactness, what other properties can be ascribed to a desirable basis? Given a particular set of basis functions, what criteria must be met by the underlying process in order for the bases to generalize well? Can these criteria be tested easily? These are just a few of the questions that come to mind. I'll be interested in any thoughts you have in this area. Thomas H. Hildebrandt From COUSEIN at lem.rug.ac.be Thu Sep 5 14:43:00 1991 From: COUSEIN at lem.rug.ac.be (COUSEIN@lem.rug.ac.be) Date: Thu, 5 Sep 91 14:43 N Subject: can't find article. Message-ID: <01GA85PPTJ9C000MXM@BGERUG51.BITNET> Dear connectionist, recently I browsed thru the unpublished abstracts of the 1990 Paris ICNN, and came across an abstract that looked interesting, i.e. Generalised Hopfield Networks for Robot Planning. by P. Morasso, V. sanguineti, G. Vercelli, R. Zaccaria, Computer Science, Genova, Italy. However, there was no article. I would like to know if anyone has seen the article published somewhere, or seen an internal tech report or anything of the kind. Where could I obtain a copy? Thanks for your help, best regards, Alexis Cousein, Ghent University Electronics Lab, Belgium. cousein at lem.rug.ac.be p.s. greetings to Martin Dudziak. Long time no see/hear. From GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca Wed Sep 4 15:48:35 1991 From: GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca (GOLDFARB%unb.ca@UNBMVS1.csd.unb.ca) Date: Wed, 04 Sep 91 16:48:35 ADT Subject: Evolvability of Recurrent Nets In-Reply-To: Message of Wed, 04 Sep 91 01:48:54 ADT from Message-ID: On Wed, 04 Sep 91 01:48:54 ADT JANSSEN Jacques writes: > Has anyone seen work on extending these ideas to > recurrent nets? Or anything which might do the job? > > Cheers, Hugo de Garis, > > Univ of Brussels, & > George Mason Univ, VA. I have developed a new model - evolving transformation systems, or reconfigurable learning machines - which is a far-reaching generalization of the NN models and which is, for the first time, accommodates the (structurally) evolving nature of the learning process, i.e., in the language of NN, new nodes that are compositions of some basic units can be introduced during learning. For more information see my previous postings on this mailing list, as well as L. Goldfarb, What is distance and why do we need the metric model for pattern learning?, to appear in Pattern Recognition. L. Goldfarb, Verifiable characterization of an intelligent process, Proc. of the 4th UNB Artificial Intelligence Symposium, UNB, Fredericton, Sept.20-21, 1991. --Lev Goldfarb From GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca Wed Sep 4 15:50:19 1991 From: GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca (GOLDFARB%unb.ca@UNBMVS1.csd.unb.ca) Date: Wed, 04 Sep 91 16:50:19 ADT Subject: Evolvability of Recurrent Nets In-Reply-To: Message of Wed, 04 Sep 91 01:48:54 ADT from Message-ID: On Wed, 04 Sep 91 01:48:54 ADT JANSSEN Jacques writes: > Has anyone seen work on extending these ideas to > recurrent nets? Or anything which might do the job? > > Cheers, Hugo de Garis, > > Univ of Brussels, & > George Mason Univ, VA. I have developed a new model - evolving transformation systems, or reconfigurable learning machines - which is a far-reaching generalization of the NN models and which is, for the first time, accommodates the (structurally) evolving nature of the learning process, i.e., in the language of NN, new nodes that are compositions of some basic units can be introduced during learning. For more information see my previous postings on this mailing list, as well as L. Goldfarb, What is distance and why do we need the metric model for pattern learning?, to appear in Pattern Recognition. L. Goldfarb, Verifiable characterization of an intelligent process, Proc. of the 4th UNB Artificial Intelligence Symposium, UNB, Fredericton, Sept.20-21, 1991. --Lev Goldfarb From PAR at DM0MPI11.BITNET Thu Sep 5 12:30:37 1991 From: PAR at DM0MPI11.BITNET (Pal Ribarics) Date: Thu, 05 Sep 91 16:30:37 GMT Subject: NN Workshop (You can have it already, we have problems with some mailer) Message-ID: From TPPEREIR at BRFUEM.BITNET Thu Sep 5 21:22:07 1991 From: TPPEREIR at BRFUEM.BITNET (TPPEREIR@BRFUEM.BITNET) Date: Thu, 05 Sep 91 21:22:07 C Subject: how can I adder this list? Message-ID: <7DD204880A800067@BITNET.CC.CMU.EDU> include Tarcisio Praciano Pereira in this list, please. ------------------------------------------------------------------------ DR. PRACIANO-PEREIRA, | E-MAIL: BITNET= TPPEREIR at BRFUEM TARCISIO | ANSP= TPPEREIR at NPDVM1.FUEM.ANPR.BR - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - DEP. DE MATEMATICA - UNIVERSIDADE ESTADUAL DE MARINGA AGENCIA POSTAL UEM 87 020 - MARINGA - PR - BRAZIL TELEFAX: (0442) 22-2754 FONE (0442) 26 27 27 ------------------------------------------------------------------------ From PAR at DM0MPI11.BITNET Fri Sep 6 09:21:16 1991 From: PAR at DM0MPI11.BITNET (Pal Ribarics) Date: Fri, 06 Sep 91 13:21:16 GMT Subject: NN Workshop Message-ID: Note: Our previous message had a long header list containing a large number of addresses. This could not be managed by some intermediate mailer so you could get criptic messages. Now the mail is sent individually to avoid this problem. We beg your pardon for that inconvenience. ******************************************************************************* Dear Colleague , You are receiving this mail either - because you showed interest in applying NN methods in the trigger of HEP experiments and responded to our recent survey or - because you have participated in the Elba Workshop Our original intention was to organize a workshop only for trigger applications. We now have accepted the kind invitation of the organizers of the Second International Workshop on Software Engineering, Artificial Intelligence and Expert Systems for High Energy and Nuclear Physics and we think that the 2 days topical workshop (number 4 in the following Bulletin) would be a good start to bring interested people together. This - in case of strong interest - could be followed by a dedicated workshop in the Munich area next fall or later. So we encourage you to send abstracts to this workshop in order to have a succesfull meeting. ******************************************************************************* SECOND INTERNATIONAL WORKSHOP ON SOFTWARE ENGINEERING, ARTIFICIAL INTELLIGENCE AND EXPERT SYSTEMS FOR HIGH ENERGY AND NUCLEAR PHYSICS January 13 - 18 1992 L'AGELONDE FRANCE-TELECOM LA LONDE-LES-MAURES BP 64 F-83250 FRANCE ORGANIZING COMMITTEE G. Auger GANIL Caen, F K. H. Becks Bergische Univ. Wuppertal, D R. Brun CERN CN Geneva, CH S. Cittolin CERN ECP Geneva, CH B. Denby FNAL Chicago, USA F. Etienne CPPM Marseille, F V. Frigo CERN DG Geneva, CH R. Gatto Geneva univ. Geneva, CH G. Gonnet ETHZ Zurich CH M. Green Royal Holloway Col. Egham Surray, GB F. Hautin CNET Lannion, F A. Kataev INR Moscow, USSR C. Kiesling MPI Munich, D P. Kunz SLAC Stanford, USA J. Lefoll DPHPE Saclay Palaisseau, F E. Malandain CERN PS Geneva, CH V. Matveev INR Moscow, USSR M. Metcalf CERN CN Geneva, CH T. Nash FNAL Chicago, USA D. Perret-Gallix LAPP Annecy, F C. Peterson Lund Univ. Lund, S P. Ribarics MPI Munich, D M. Sendall CERN ECP Geneva, CH Y. Shimizu KEK Tsukuba, JP D. Shirkov JINR Dubna, USSR M. Veltman Univ. Michigan Ann Arbor, USA J. Vermaseren NIKHEF-H Amsterdam, NL C. Vogel CISI Paris, F W. Wojcik CCIN2P3 Lyon, F Dear Colleague, Some of you may remember the first workshop organized in Lyon in March 1990. The enthusiasm and interest of the participants (200) showed clearly that this type of meeting was filling a real need for discussion and information. Two years later, we think it is time for another gettogether with a perspective of the large High Energy and Nuclear Physics experiments. We foresee a true type of workshop where in addition to scheduled presentations, plenty of time will be available for informal discussion. Our main objectives are: * To create a bridge, a real link, between computer experts and others. Modern computing techniques are developing rapidly and the gap between physics and computer science activites is getting larger and larger. This workshop is intended to bring to a physicist an understanding of them to a level that real work can begin. ----> Tutorials * Physics research is at the mercy of the industry in many aspects. We have to create a forum between research and industry. The problems encountered in large experiments are close to industrial level and both sides can profit from such collaboration ----> Specialized Workshops, Products Showroom * There will not be HEP experiments at SSC/LHC without integrating new computing technologies, like going to the moon without the transistor. "How we will do it ?" is a question that should be addressed right now. Experiments must have these techniques built-in in order to perform at nominal value. ----> ASTEC project The workshop will take place in a nice resort, in the France-Telecom site "L'AGELONDE", on the French riviera (cote d'Azur, 500 m from the beach). We will have access to an Ethernet network and video-conferencing can be set up. The full six-day Workshop will be organized as follows: Monday 13-14 January Tutorials and, in parallel, Topical Workshops TUTORIALS: Monday 13 (1) 9h-12h30 CASE and Graphical User Interfaces (2) 14h-18h C++ and Object Oriented Programing Tuesday 14 (3) 9h-12h30 Expert systems and Artificial Intelligence (4) 14h-18h Neural Networks in trigger and data analysis TOPICAL WORKSHOP (Monday 13-Tuesday 14) (1) Symbolic Manipulation Techniques "Problems and Results of Symbolic Computations in High Energy and Quantum field Theory" (2) Networks and distributed computing (3) Software development for "Big Sciences" (4) Methods and techniques of artificial intelligence Wednesday 15 January Round-table, discussions, ASTEC project organization Thursday 15 - Saturday 17 PLENARY SESSIONS The ASTEC Project One session will be devoted to the organization of long standing working groups, not directly depending on experimental projects or accelerators (LEP200, LHC, UNK, SSC, RHIC, HERA, KEK, CEBAF, B-Factories, Tau/charm-Factories, ...) but with this future in mind. The importance of these techniques has been largely demonstrated at LEP. The purpose of the ASTEC project is to prepare reports to be used as references or user-guides for developing more dedicated applications. (Similar to the CERN yellow report for Physics at LEP or LHC) This workshop will launch the project by precisely surveying the items to be studied and naming various responsibilities for a one to two year program. The outcome of this work will be presented at a later workshop and published as a book. The large task facing us will include evaluation of commercial products, collaboration with companies to improve selected items, definition and running of benchmarks, building pilot projects and making proposals for standardization of tools, data description, languages and systems. REMEMBER: "Higher luminosities are also achieved through a low down-time and DAQ dead-time, more accurate triggers and better event reconstruction programs. " "ASTEC project: an Energy Saver Enterprise" Three main groups will be organized: (A) Group: Software Engineering (1) Subgroup: Languages and Systems - Conventional languages, Fortran 90, C, ... - Object Oriented Languages, C++, Objective-C, ... - Mixed languages environment - Operating systems HEPIX, HEPVM, ... - Network Wide Application software maintenance - Porting packages between languages and OS. - DataBase maintenance (updating, access protection) - Data description and representation (2) Subgroup: CASE Tools for developing, maintaining and designing software projects. - Intelligent Editors - Maintenance of multi-version application: CMZ, Historian, ... - On-line Documentation - Symbolic debuggers - Data representation - Software design and simulation - System simulations for real-time application - (3) Subgroup: Interactive Analysis - Event Server - Workstation <-> Mainframe cooperation - Graphical User Interface - Interactive Analysis packages PAW, Reason, IDAL, ... - (B) Group: A.I. (1) Subgroup: Languages, Systems - Prolog II, Prolog III, ... - Mixing Prolog, OOL and Conventional languages in applications - Expert system development management. - (2) Subgroup: Expert systems - Off-line support - Hardware testing and maintenance - On-line assistance - Real-time expert systems - Electronic log-book - Testing expert systems: Validation and verification - Embedding E.S. support in detectors or systems (3) Subgroup: Pattern recognition methods - Track and shower recognition - Event selection - Fuzzy-logic in pattern recognition - Genetic Algorithms - (4) Subgroup: Neural Networks - Algorithms for off-line pattern recognition - Algorithms for fast triggering - Test of Silicon Neural Network prototypes - Neural Network training (C) Group: Symbolic Manipulation Techniques (1) Subgroup: Languages, Systems - Schoonschip, Form, Reduce, Mathematica, Scratchpad II, GAL, Maple, ... a critical review. - Graphics for representation of diagrams and for display of results - Database for intermediate computations and results, integrals, sub-diagrams, ... - (2) Subgroup: Feynman Diagrams - Diagram generation - Symbolic diagram computation - Symbolic/numeric integral computation - Feynman Diagram Symbolic Manipulation Collaboration - (3) Subgroup: Quantum Field Theory and Super-Algebra - methods and algorithms of higher order calculations - symbolic manipulation for N-loop calculations - numerical methods for N-loop calculations - calculations in supersymmetry theories (SUSY, SG, Strings) - applications in Quantum Field Theory - ------------------------------------------------------------------------ Talks will be selected by the Organizing Committee on the basis of a detailed abstract to be submitted before: 15 October, 1991. A poster session will be organized. ======================================================================== SECOND INTERNATIONAL WORKSHOP ON SOFTWARE ENGINEERING, ARTIFICIAL INTELLIGENCE AND EXPERT SYSTEMS FOR HIGH ENERGY AND NUCLEAR PHYSICS 1992 January 13 - 18 L'AGELONDE FRANCE-TELECOM LA LONDE LES MAURES BP 64 F-83250 REGISTRATION NAME: FIRSTNAME: LABORATORY: COUNTRY ADDRESS: TEL: FAX: TELEX: E-MAIL: HOTEL RESERVATION (Number of persons): In the following you are expected to answer with the corresponding number or character from the list above. However if your interest is not mentioned in the list give a full description. WOULD YOU BE INTERESTED TO JOIN A WORKING GROUP OF THE ASTEC PROJECT ? YES/NO GROUP: SUBGROUP: WOULD YOU LIKE TO ATTEND TOPICAL WORKSHOPS OR TUTORIALS ? WORKSHOPS: TUTORIALS: WOULD YOU LIKE TO PRESENT A TALK ? YES/NO TALK TITLE: To be considered by the organizing committee, send an extended abstract before Oct. 15, 1991 to: Michele Jouhet Marie-claude Fert CERN L.A.P.P. - IN2P3 PPE-ADM B.P. 110 CH-1211 Geneve 23 F-74941 Annecy-Le-Vieux SWITZERLAND FRANCE Tel: (41) 22 767 21 23 Tel: (33) 50 23 32 45 Fax: (41) 22 767 65 55 Fax: (33) 50 27 94 95 Telex: 419 000 Telex: 385 180 F E-mail: jouhet at CERNVM Workshop fee : 700 FFr. Student : 500 FFr. Accommodation : 2000 FFr. Accompagning Person: +1200 FFr. To be paid by check: Title: International Workshop CREDIT LYONNAIS/Agence Internationale Bank: 30002 Guichet: 1000 Account: 909154 V Address: LYON REPUBLIQUE The accommodation includes: hotel-room, breakfast, lunch and dinner for 6 days. Tennis, mountain bike and other activities will be available. Denis Perret-Gallix Tel: (41) 22 767 62 93 E-mail: Perretg at CERNVM Fax: (41) 22 782 89 23 From dhw at t13.Lanl.GOV Fri Sep 6 11:20:11 1991 From: dhw at t13.Lanl.GOV (David Wolpert) Date: Fri, 6 Sep 91 09:20:11 MDT Subject: interpolation vs. generalization Message-ID: <9109061520.AA04395@t13.lanl.gov> Thomas Hildebrandt and I probably agree on all important points, but simply emphasize different aspects of the problem. However, 1) One must be very careful about talking about "basis sets". There are many generalizers (e.g., nearest neighbor generalizers) which can not be expressed as forming a linear combination of basis functions in the usual Taylor decomposition manner. In fact, one can prove the following: Assume that we want a generalizer which obeys all the invariances of Euclidean space. For example, if the entire training set is translated in input space, then the guessed hypothesis function should be translated in the same manner. (We also want scaling invariance, rotation invariance, etc.) There are many such generalizers. However there does not exist a basis set of functions such that a generalizer which works by fitting a linear combination of the basis functions to the elements of the training set obeys those invariances. There is an exception to this theorem: if the input space is one-dimensional, then there *is* such a basis set of functions, but only one. Those functions are just the monomials. 2) Hildebrandt says: "We may consider problems for which there is no simple transformation from the input (sensor) space into a (linear) metric space to be "hard" problems, in a sense." Well, yes and no. For some generalizers, namely those which assume such a metric space, such problems are hard. For other generalizers they are not. A simple example is fan generalizers, which can be viewed as the multi-dimensional generalization of the non-linear time-series technique of "embedding" a time series in a delay space. For such generalizers, even target functions which are extremely volatile can be generalized exceedingly well. 3) Hildebrandt also says that "Discrete problems, which naturally inhibit interpolation, must be handled by table look-up, i.e. each case treated separately. However, table look-up can be considered to be an extreme case of interpolation -- the transition between one recorded data point and a neighboring one being governed by a Heaviside (threshold) function rather than a straight line. Yes, you can view the behavior of any generalizer as "interpolation", if you stretch the meaning of the term sufficiently. My point is that viewing things that way for some problems (not all) amounts to a not-very-informative tautology. For some problems, the appropriate response to such a view is "well, yes, it is, technically speaking, interpolation, but so what? What does such a perspective gain you? Again, I *am* very sympathetic to the interpolation model. There are many advantages of memory-based reasoning over conventional neural nets, and (to my mind at least) not many advantages of neural nets over memory-based reasoning. But that doesn't mean that "interpolation" is the end of the story and we can all go home now. 4) "If nothing is known about the process to be modelled, is there any more efficient way to select a basis than trial-and-error?" I'm not sure what you mean by "trial-and-error". Do you mean cross- validation? If so, then the answer is yes, there are techniques better than cross-validation. Cross-validation is a winner-takes-all strategy, in which one picks a single generalizer and uses it. One can instead use stacked generalization, in which one *combines* generalizers non-linearly. For any who are interested, I can mail reprints of mine which discuss some of the points above. David Wolpert (dhw at tweety.lanl.gov) From zoubin at learning.siemens.com Fri Sep 6 14:33:06 1991 From: zoubin at learning.siemens.com (Zoubin Ghahramani) Date: Fri, 6 Sep 91 14:33:06 EDT Subject: interpolation vs. generalization In-Reply-To: David Wolpert's message of Fri, 6 Sep 91 09:20:11 MDT <9109061520.AA04395@t13.lanl.gov> Message-ID: <9109061833.AA22287@learning.siemens.com.siemens.com> Just to throw a monkey wrench in: How does one interpret generalization as interpolation in a problem like n-bit parity? For any given data point, the n nearest neighbours in input space would all predict an incorrect classification. However, I wouldn't say that a problem like parity is ungeneralizable. Zoubin Ghahramani Dept. of Brain & Cognitive Sciences MIT From russ at oceanus.MITRE.ORG Fri Sep 6 14:53:15 1991 From: russ at oceanus.MITRE.ORG (Russell Leighton) Date: Fri, 6 Sep 91 14:53:15 EDT Subject: Beta Test Sites For Aspirin v5.0 Message-ID: <9109061853.AA05585@oceanus> Attention users of the MITRE Neural Network Simulator Aspirin/MIGRAINES Version 4.0 Aspirin/MIGRAINES Version 5.0 is currently in beta test. I am seeking to expand the list of sites before the general release in mid October. Version 5.0 is *much* more portable than version 4.0 and has new graphics using libraries from the apE2.1 visualization package. Supported platforms include: Sun4, Sun3 Silicon Graphics Iris IBM RS/6000 Intel 486/386 (Unix System V) NeXT DecStation Cray YMP Convex Coprocessors: Mercury i860 (40MHz) Coprocessors Meiko Computing Surface w/i860 (40MHz) Nodes iWarp Cells I would like to expand this list to other Unix platforms (e.g. HP snakes, MIPS, etc.). This should be very easy. If you are currently using Aspirin and would like to be part of this preliminary release AND... 1. You have email on the Internet 2. You have ftp access on the Internet ...then please reply to this message. In particular, I would like to find users on: 1. DecStations 2. NeXT 3. IBMRS6000 4. Other Unix platforms. Thanks. Russ INTERNET: russ at dash.mitre.org Russell Leighton MITRE Signal Processing Lab 7525 Colshire Dr. McLean, Va. 22102 USA From GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca Sat Sep 7 16:38:07 1991 From: GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca (GOLDFARB%unb.ca@UNBMVS1.csd.unb.ca) Date: Sat, 07 Sep 91 17:38:07 ADT Subject: Choice of the basis functions In-Reply-To: Message of Fri, 06 Sep 91 02:09:17 ADT from Message-ID: On Fri, 06 Sep 91 02:09:17 ADT "Thomas H. Hildebrandt " writes: > I could not agree with you more in > thinking that the search for an appropriate basis set is one of the > important open problems in connectionist research. > > If nothing is known about the process to be modelled, is there any > more efficient way to select a basis than trial-and-error? > > Are some sets of basis functions more likely to efficiently describe a > randomly selected process? Aside from compactness, what other > properties can be ascribed to a desirable basis? > > Given a particular set of basis functions, what criteria must be met > by the underlying process in order for the bases to generalize well? > Can these criteria be tested easily? > > These are just a few of the questions that come to mind. I'll be > interested in any thoughts you have in this area. > > Thomas H. Hildebrandt Within the metric model proposed in L.Goldfarb, A New Approach to Pattern Recognition, in Progress in Pattern Recognition 2, L.N.Kanal and A.Rosenfeld, eds., North- Holland, Amsterdam,1985 the question of choice of the "right" basis can be resolved quite naturally: the finite training metric set is first isometrically embedded in the "appropriate" Minkowski vector space, and then a subset best representing the principal axes of the constructed vector representation is chosen as the basis. -- Lev Goldfarb From GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca Sun Sep 8 16:30:24 1991 From: GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca (GOLDFARB%unb.ca@UNBMVS1.csd.unb.ca) Date: Sun, 08 Sep 91 17:30:24 ADT Subject: interpolation vs. generalization In-Reply-To: Message of Sun, 08 Sep 91 13:36:50 ADT from Message-ID: On Sun, 08 Sep 91 13:36:50 ADT Zoubin Ghahramani writes: > > Just to throw a monkey wrench in: > > How does one interpret generalization as interpolation in a problem > like n-bit parity? For any given data point, the n nearest neighbours > in input space would all predict an incorrect classification. However, > I wouldn't say that a problem like parity is ungeneralizable. > > > Zoubin Ghahramani > Dept. of Brain & Cognitive Sciences > MIT Dear Zoubin, I am happy to inform you that the above is no "monkey wrench". Here is the solution to your parity problem within the transformation systems model. To simplify the exposition, I will ignore the positions of 1's and delete all 0's. Let C+ (the positive learning set) be as follows: ||||, ||, ||||||||; and let C- (the negative learning set) be as follows: |||||, |, |||. For this problem the initial transformation system T0 is S0 = {s1} the initial set of operations consists of a single operation s1 - deletion-insertion of |, CR = {R1} the set of composition rules consists of a single rule R1 - concatenation of two strings of |'s. The competing family of distance functions is defined by means of the shortest weighted path between two strings of |'s - the smallest weighted number of current operations necessary to transform one string into the other. Since we require that the sum of the operation weights must be 1, and S0 consists of a single operation, the initial distance between two patterns is just the difference between the number of |'s. The following function is maximized during learning F1 (w) F(w) = -----------, F2 (w) + c where w is the weight vector w = (w^1, w^2, . . ., w^m) , m is the number of current operations, sum of w^i's is 1, F1 is the (shortest) distance between C+ and C-, F2 is the average distance in C+, and c is a very small positive constant (to prevent the overflow). For T0 the maximum of F is 1/4. Applying rule R1 to the only existing operation, we obtain new operation s2 - deletion-insertion of ||. Thus, in the evolved transformation system T1 set S1 = {s1, s2}, and the optimization is over the 1-d simplex w^2 | 1| . | . | . |__________1______ w^1 For T1 the maximum of F is a very large number 1/c and it is achieved at w* = (1, 0). Moreover, F1 (w*) = 1 and F2 (w*) = 0. The learning stage is completed. The recognition stage and the propositional class description readily follow from the above. -- Lev Goldfarb From eric at mcc.com Sun Sep 8 20:58:54 1991 From: eric at mcc.com (Eric Hartman) Date: Sun, 8 Sep 91 19:58:54 CDT Subject: more RBF refs Message-ID: <9109090058.AA10679@bird.aca.mcc.com> Here are a couple more RBF refs: Hartman, Keeler & Kowalski Layered Neural Networks With Gaussian Hidden Units as Universal Approximators Neural Computation, 2 (2), 210-215 (1990) Hartman & Keeler Semi-local Units for Prediction IJCNN-91-Seattle, II-561-566 The latter describes some limitations of RBFs and a new activation function that is less localized than an RBF but more localized than a sigmoid. The function is based on gaussians; backpropagation networks using these basis functions learn the Mackey-Glass time series nearly as fast as the standard RBF network training algorithm. The full paper will appear in Neural Computation. Eric Hartman and Jim Keeler From dhw at t13.Lanl.GOV Mon Sep 9 11:41:03 1991 From: dhw at t13.Lanl.GOV (David Wolpert) Date: Mon, 9 Sep 91 09:41:03 MDT Subject: No subject Message-ID: <9109091541.AA07866@t13.lanl.gov> Zoubin Ghahramani writes "How does one interpret generalization as interpolation in a problem like n-bit parity? For any given data point, the n nearest neighbours in input space would all predict an incorrect classification. However, I wouldn't say that a problem like parity is ungeneralizable. " A very good example. In fact, any generalizer which acts in a somewhat local manner (i.e., looks mostly at nearby elements in the learning set) has the nice property that for the parity problem, the larger the training set the *worse* the generalization off of that training set, for precisely the reason Dr. Ghahramani gives. Interestingly, for small (number of bits small) versions of the parity problem, backprop has exactly this property; as the learning set grows, so does its error rate off of the learning set. (Dave Rumelhart has told me that this property goes away in big versions of parity however.) David Wolpert From dsomers at park.bu.edu Mon Sep 9 12:23:11 1991 From: dsomers at park.bu.edu (David C. Somers) Date: Mon, 9 Sep 91 12:23:11 -0400 Subject: Synchronization Binding? Freq. Locking? Bursting? Message-ID: <9109091623.AA06807@park.bu.edu> This is in reply to a recent message from the connectionist news group >====================================================================== >From: connectionists at c.cs.cmu.edu Newsgroups: bu.mail.connectionists >Subject: Synchronization Binding? Freq. Locking? Bursting? Date: 22 >Aug 91 06:03:07 GMT > >>From: Thomas VLSI Edwards >I have just read "Synchronized Oscillations During Cooperative >Feature Linking in a Cortical Model of Visual Perception" >(Grossberg, Somers, Neural Networks Vol. 4 pp 453-466). >It describes some models of phase-locking (supposedly neuromorphic) >relaxation oscillators, including a cooperative bipole coupling which >appears similar to the Kammen comparator model, and fits into BCS >theory. Cooperative Bipole Coupling is significantly different from the comparator model used by Kammen, Holmes and Koch. The Bipole mechanism is a sort of "statistical AND-gate" which becomes active (and thus provides feedback) only when both of its spatially independent receptive flanks are sufficiently active. Feedback then passes to a cell or cells which lie intermediate to the two flanking regions. In the full Boundary Contour System, this feedback is also passed only to cells whose receptive field properties (e.g., orientation) are similar to those cells which activate the particular bipole cell. This mechanism was proposed (By Grossberg and Mingolla) to handle the process of emergent segmentation in the visual system such as occurs in the perception of occluding contours, textural boundaries, and illusory or broken contours. As Grossberg and I noted in our paper this mechanism has received both neuroanatomical and neurophysiological support. In the context of synchronized oscillations, we used the bipole mechanism to not only synchronize activity along and over two regions of oscillatory activity, but also to induce and synchronize oscillatory activity within a slit region in between the two oscillating regions. The bipole mechanism accomplished this "perceptual boundary completion" without inducing a spreading of oscillatory activity to the outlying regions. The comparator model cannot robustly achieve this effect since it does not distinguish between the completion of a boundary over a break between two regions and the outward spreading of activity from the end of a line segment. That is, the comparator is not sensitive to the spatial distribution of its inputs but rather only to the total input. The Adaptive Filter mechanism that we use in our simulations reduces to the comparator mechanism when the fan-in of the Adaptive Filter equals its Fan-out. The Adaptive Filter achieved synchronization without achieving boundary completion. These results taken together suggest that different architectures be used to achieve different results. We interpret the Bipole cell results as a pre-attentive boundary completion, while the adaptive filter results correspond to an attentive resonant state as may occur during recall or learning (cf. Adaptive Resonance Theory). >I am curious at this date what readers of connectionists think about >the theory that syncrhonous oscillations reflect the binding of local >feature detectors to form coherent groups. I am also curious as to >whether or not phase-locking of oscillators is a reasonable model >of the phenomena going on, or whether synchronized bursting, yet >not frequency-locked oscillation, is a more biologically acceptable >answer. Charlie Gray's data seems to indicate that it is actually bursting, not single spikes that are being synchronized. This is consistent with our oscillations of average firing rate. Note that bursting can still be viewed as an oscillatory phenomena. Also note that Gray's data indicates that synchrony occurs very rapidly--within one or two cycles. Our simulations demonstrate this effect, although many other researchers have had great difficulty in achieving rapid synchrony. Although the architecture of connections between the oscillators is important, so is the form of the individual oscillators. Nancy Kopell and I have a series of results that show the advantages of using neural relaxation oscillators rather than boring old sinusoids (papers in preparation). As far as the meaning of the synchronized oscillations, I think we really need a lot more data to really be able to tell. Right now we've got a lot of modellers chasing a little bit of data. Having said that, there is still something compelling about synchronized oscillations. My hunch is that the oscillations represent a form of multiplexing where the average firing rate over several burst cycles indicates the information local to the classical receptive field while the synchronization of the bursts represents some form of global grouping of information. Anything less than this multiplexing would not seem to serve any purpose (at least in vision) -- Why give up coding by average firing rate in order to code by phase relationship? This is just trading one dimension for another (with a seemingly small dynamic range) I suggest that the visual system may be making use of both coding dimensions. This kind of multiplexing would also allow for very rapid computations, since global extraction would be performed while local information accumulates rather than performing these operations sequentially. David Somers (dsomers at park.bu.edu) Center for Adaptive Systems 111 Cummington St. Boston, MA 02215 From barto at envy.cs.umass.edu Mon Sep 9 13:17:05 1991 From: barto at envy.cs.umass.edu (Andy Barto) Date: Mon, 9 Sep 91 13:17:05 -0400 Subject: Technical Report Available Message-ID: <9109091717.AA17616@envy.cs.umass.edu> The following technical report is available: Real-Time Learning and Control using Asynchronous Dynamic Programming Andrew G. Barto, Steven J. Bradtke, Satinder P. Singh Department of Computer Science University of Massachusetts, Amherst MA 01003 Technical Report 91-57 Abstract---Learning methods based on dynamic programming (DP) are receiving increasing attention in artificial intelligence. Researchers have argued that DP provides the appropriate basis for compiling planning results into reactive strategies for real-time control, as well as for learning such strategies when the system being controlled is incompletely known. We extend the existing theory of DP-based learning algorithms by bringing to bear on their analysis a collection of relevant mathematical results from the theory of asynchronous DP. We present convergence results for a class of DP-based algorithms for real-time learning and control which generalizes Korf's Learning-Real-Time-A* (LRTA*) algorithm to problems involving uncertainty. We also discuss Watkins' Q-Learning algorithm in light of asynchronous DP, as well as some of the methods included in Sutton's Dyna architecture. We provide an account that is more complete than currently available of what is formally known, and what is not formally known, about the behavior of DP-based learning algorithms. A secondary aim is to provide a bridge between AI research on real-time planning and learning and relevant concepts and algorithms from control theory. -------------------------------------------------------------------- This TR has been placed in the Neuroprose directory (courtesy Jordan Pollack) in compressed form. The file name is "barto.realtime-dp.ps.Z". The instructions for retreiving this document from that archive are given below. WARNING: This paper is SIXTY EIGHT pages long. If you are unable to retreive/print it and therefore wish to receive a hardcopy please send mail to the following address: Connie Smith Department of Computer Science University of Massachusetts Amherst, MA 01003 Smith at cs.umass.edu ****PLEASE DO NOT REPLY TO THIS MESSAGE***** NOTE: This is the paper on which my talk at the Machine Learning Workshop, July 1991, was based. If you requested a copy at that time, it is already in the mail. Thanks, Andy Barto -------------------------------------------------------------------- Here is how to ftp this paper: unix> ftp cheops.cis.ohio-state.edu (or 128.146.8.62) Name: anonymous Password: neuron ftp> cd pub/neuroprose ftp> binary ftp> get barto.realtime-dp.ps.Z ftp> quit unix> uncompress barto.realtime-dp.ps.Z unix> lpr barto.realtimedp.ps From cherwig at eng.clemson.edu Mon Sep 9 15:28:16 1991 From: cherwig at eng.clemson.edu (christoph bruno herwig) Date: Mon, 9 Sep 91 15:28:16 EDT Subject: morphology Message-ID: <9109091928.AA08621@eng.clemson.edu> Dear all, I am interested in having neural networks learn morphological operators like DILATION, EROSION, CLOSING, OPENING, SKELETONIZING. Initial attempt is an input-layer/output-layer feedforward network and backprop learning algorithm. I posted in 'comp.ai.vision' before and received the appended replies. I would appreciate it, if anyone of you can add references to my list. Thank you very much in advance! ++++++++++++++++++ From sch at ee.UManitoba.CA Mon Sep 9 16:24:23 1991 From: sch at ee.UManitoba.CA (sch@ee.UManitoba.CA) Date: Mon, 9 Sep 91 15:24:23 CDT Subject: unsubscribe Message-ID: <9109092024.AA20880@ic12.ee.umanitoba.ca> Please remove me from the connectionist mailing list. From tp-temp at ai.mit.edu Mon Sep 9 20:58:53 1991 From: tp-temp at ai.mit.edu (Tomaso Poggio) Date: Mon, 9 Sep 91 20:58:53 EDT Subject: more RBF refs In-Reply-To: Eric Hartman's message of Sun, 8 Sep 91 19:58:54 CDT <9109090058.AA10679@bird.aca.mcc.com> Message-ID: <9109100058.AA00733@erice> Why call gradient descent backpropagation? From eric at mcc.com Mon Sep 9 21:07:31 1991 From: eric at mcc.com (Eric Hartman) Date: Mon, 9 Sep 91 20:07:31 CDT Subject: more RBF refs Message-ID: <9109100107.AA12412@bird.aca.mcc.com> From tp-temp at ai.mit.edu Mon Sep 9 19:59:14 1991 Posted-Date: Mon, 9 Sep 91 20:58:53 EDT Received-Date: Mon, 9 Sep 91 19:59:13 CDT Received: from MCC.COM by bird.aca.mcc.com (4.0/ACAv4.1i) id AA12394; Mon, 9 Sep 91 19:59:13 CDT Received: from life.ai.mit.edu by MCC.COM with TCP; Mon 9 Sep 91 19:59:13-CDT Received: from erice (erice.ai.mit.edu) by life.ai.mit.edu (4.1/AI-4.10) id AA14982; Mon, 9 Sep 91 20:58:38 EDT From: tp-temp at ai.mit.edu (Tomaso Poggio) Received: by erice (4.1/AI-4.10) id AA00733; Mon, 9 Sep 91 20:58:53 EDT Date: Mon, 9 Sep 91 20:58:53 EDT Message-Id: <9109100058.AA00733 at erice> To: eric at mcc.com Cc: connectionists at cs.cmu.edu In-Reply-To: Eric Hartman's message of Sun, 8 Sep 91 19:58:54 CDT <9109090058.AA10679 at bird.aca.mcc.com> Subject: more RBF refs Status: R Why call gradient descent backpropagation? ---------------- I didn't mean to. Careless phrasing. Backprop implements gradient descent. Eric Hartman From jose at tractatus.siemens.com Tue Sep 10 07:33:25 1991 From: jose at tractatus.siemens.com (Steve Hanson) Date: Tue, 10 Sep 1991 07:33:25 -0400 (EDT) Subject: more RBF refs In-Reply-To: <9109090058.AA10679@bird.aca.mcc.com> References: <9109090058.AA10679@bird.aca.mcc.com> Message-ID: <0cn_q5y1GEMn0lOzIS@tractatus.siemens.com> Another paper on RBFs ("spherical units") also using a basis function (cauchy) less localized than the gaussian but more localized than the linear-logistic ("sigmoid") is to be found in: Hanson, S. J. & Gluck, M. A. Spherical Units as Dynamic Consequential Regions: Implications for Attention, Competition and Categorization. Advances in Neural Information Processing 3, R. Lippmann, J. Moody & D. Touretzky (Eds.), Morgan Kaufman, pp. 656-665, (1991). Stephen J. Hanson Learning Systems Department SIEMENS Research 755 College Rd. East Princeton, NJ 08540 From N.E.Sharkey at dcs.exeter.ac.uk Tue Sep 10 13:31:09 1991 From: N.E.Sharkey at dcs.exeter.ac.uk (Noel Sharkey) Date: Tue, 10 Sep 91 13:31:09 BST Subject: CONNECTION SCIENCE REDUCTIONS Message-ID: <8443.9109101231@propus.dcs.exeter.ac.uk> A VERY SPECIAL DEAL FOR MEMBERS OF THE CONNECTIONISTS MAILING. Thanks to persistent representations from members of the editorial board of Connection Science (especially Gary Cottrell) and to the dramatic increase in sales this year, a massive discount is being offered to recipients of this mailing list for personal subscriptions to the journal. Prices for members of this list will now be: North America 44 US Dollars (reduced from 126 dollars) Elsewhere and U.K. 22 pounds sterling. (Sterling checks must be drawn on a UK bank) These rates start from 1st January 1992 (volume 4). Conditions: 1. Personal use only (i.e. non-institutional). 2. Must subscribe from your private address. You can receive a subscription form by emailing direct to the publisher: email: carfax at ibmpcug.co.uk Say for the attention of David Green and say CONNECTIONISTS MAILING LIST. noel From riffraff at mentor.cc.purdue.edu Tue Sep 10 10:56:55 1991 From: riffraff at mentor.cc.purdue.edu (bertrands clarke) Date: Tue, 10 Sep 91 09:56:55 -0500 Subject: unsubscribe Message-ID: <9109101456.AA17157@mentor.cc.purdue.edu> Please remove me from the connectionist mailing list. From hinton at ai.toronto.edu Tue Sep 10 13:06:26 1991 From: hinton at ai.toronto.edu (Geoffrey Hinton) Date: Tue, 10 Sep 1991 13:06:26 -0400 Subject: No subject In-Reply-To: Your message of Mon, 09 Sep 91 11:41:03 -0400. Message-ID: <91Sep10.130641edt.262@neuron.ai.toronto.edu> David Wolpert writes "A very good example. In fact, any generalizer which acts in a somewhat local manner (i.e., looks mostly at nearby elements in the learning set) has the nice property that for the parity problem, the larger the training set the *worse* the generalization off of that training set, for precisely the reason Dr. Ghahramani gives." His definition of "somewhat local" would seem to include K nearest neighbors. Curiously, this can do a good job of generalizing parity from randomly distributed examples IFF we use generalization on a validation set to determine the optimal value of K. Each n-bit input vector has n neighbors that differ on 1 bit, but order n^2 neighbors that differ on two bits (and have the same parity). So the generalization to a validation set will be best for K small and even. He also writes "Interestingly, for small (number of bits small) versions of the parity problem, backprop has exactly this property; as the learning set grows, so does its error rate off of the learning set." In fact, backprop generalizes parity rather well using n hidden units, provided the number of connections (n+1)^2 is considerably smaller than the number of training examples. For small n, (n+1)^2 is smaller than 2^n, so its not surprising it doesnt generalize. For n=10, it generalises very well from 512 training examples to the rest (only 2 or 3 errors). Geoff From dhw at t13.Lanl.GOV Tue Sep 10 13:53:35 1991 From: dhw at t13.Lanl.GOV (David Wolpert) Date: Tue, 10 Sep 91 11:53:35 MDT Subject: yet more interp. vs. gen. Message-ID: <9109101753.AA09444@t13.lanl.gov> Geoff Hinton writes, concerning using generalizers on the parity problem: "Curiously, K nearest neighbors can do a good job of generalizing parity from randomly distributed examples IFF we use generalization on a validation set to determine the optimal value of K. Each n-bit input vector has n neighbors that differ on 1 bit, but order n^2 neighbors that differ on two bits (and have the same parity). So the generalization to a validation set will be best for K small and even." 1) I don't understand the conclusion; for K small (e.g., <~ n), as Geoff points out, the nearest neighbors differ by 1 bit, have the wrong parity, and you run into the problem of increasingly bad generalization. In fact, 2) I've done tests with K <= n on parity, using a weighted (by Hamming distance) average of those K neighbors, and generalization error off of the training set definitely gets worse as the training set size increase, asymptoting at 100% error. Indeed, if K <= n, I don't care what kind of weighting one uses, or what the actual value of K is; 100% error will occur for training sets consisting of samples from the entire space except for the single off-training set question. 3) I don't doubt that if one uses cross-validation to set K one can do better than if one doesn't. (In fact, this is exactly the tack I took in my 1990 Neural Networks article to beat NETtalk using a weighted average generalizer.) However I should point out that if we're broadening the discussion to allow the technique of cross-validation, then one should consider non-linear-time-series-type techniques, like fan generalizers. On the parity problem it turns out that the guessing of such generalizers is more accurate than either backprop or nearest neighbors. For example, I've recently run a test on the parity problem with n = 24, with a training set consisting solely of the 301 sample points with 2 or fewer of the 24 bits on; fan generalizers have *perfect* generalization to each of the remaining 16 million + sample points. Not a single error. (With a bit of work, I've managed to prove why this behavior occurs.) For some other sets of sample points (e.g., for most randomly distributed training sets) the kind of fan generalizers I've been experimenting with can only make guesses for a subset of the questions off of the training set; for those points where they can't make a guess one must use some other technique. Nonetheless, for those questions where they *can* make a guess, in the thousand or so experiments I've run on the parity problem they NEVER make an error. I should point out that fan generalizers obviously can't have behavior superior to backprop on *all* target functions. Nonetheless, fan generalizers *do* have superior behavior for 8 out of the 9 simple target functions that have been tested so far; even for those target functions where they don't do generalize perfectly, they have fewer errors than backprop. David From lakoff at cogsci.Berkeley.EDU Wed Sep 11 04:13:12 1991 From: lakoff at cogsci.Berkeley.EDU (George Lakoff) Date: Wed, 11 Sep 91 01:13:12 -0700 Subject: unsubscribe Message-ID: <9109110813.AA05097@cogsci.Berkeley.EDU> UNSUBSCRIBE From Alexis_Manaster_Ramer at MTS.cc.Wayne.edu Wed Sep 11 08:08:02 1991 From: Alexis_Manaster_Ramer at MTS.cc.Wayne.edu (Alexis_Manaster_Ramer@MTS.cc.Wayne.edu) Date: Wed, 11 Sep 91 08:08:02 EDT Subject: No subject Message-ID: <359394@MTS.cc.Wayne.edu> Please take me off the list. From koch at CitIago.Bitnet Tue Sep 10 16:12:27 1991 From: koch at CitIago.Bitnet (Christof Koch) Date: Tue, 10 Sep 91 13:12:27 PDT Subject: Conference announcement Message-ID: <910910131204.20404837@Iago.Caltech.Edu> CALL FOR PAPERS ********************************************************* Intelligent Vehicles `92 July I and 2, 1992, Radisson on the Lake Hotel near Detroit, USA ********************************************************* Organized by: IEEE/IES Intelligent Vehicle Subcommittee Cooperation with: American Society of Mechanical Engineers IEEE Vehicular Technology Society IEEE Neural Nets Council Japan Society for Fuzzy Theory and Systems Robotics Society of Japan Society of Automotive Engineers, Intemational Society of Automotive Engineers, Japan Society of Instrument and Control Engineers (Some of them listed above are in the application process and cooperations are not approved yet.) The IEEE/IES Intelligent Vehicle Subcommittee is organizing international meetings once every year. In 1991, for example, an international meeting will be held on "Fuzzy and Neural Systems, and Vehicle Applications" on November 8 and 9, 1991 in Tokyo. The meeting in 1990 was on "Vision- Based Vehicle Guidance". For 1992, we are planning to have multiple sessions. We will consider publishing a book, in addition to the proceedings, by selecting good papers presented in the special session as is the tradition of this workshop. This workshop will be held in conjunction with IROS '92 (International Conference on Intelligent Robots and Systems) which will be held in North Carolina from July 7, 1992. Topics: Real-Time Traffic Control (Special Session) Fuzzy Logic & Neural Nets for Vehicles Vision-Based Vehicle Guidance Other Related Issues including: Navigation Microwave Radar & Laser Radar Communication Architectures Advanced Electronics for Vehicles Deadlines: December 1, 1991, for one-page abstracts February 1, 1992, for acceptance notices April 1, 1992, for camera-ready papers If you would like to have your name on our mailing list, please write "Intelligent Vehicle" and/or "IROS" on the back of your business card (or a card with your address, phone, fax, and e-mail), and mail it to: Ichiro Masaki, Computer Science Department General Motors Research Laboratories 30500 Mound Road, Warren, Michigan, 48090-9055, USA Phone: (USA) 313-986-1466, FAX: (USA) 313-986-9356 CSNET:MASAKI at GMR.COM From jvillarreal at nasamail.nasa.gov Tue Sep 10 15:33:00 1991 From: jvillarreal at nasamail.nasa.gov (JAMES A. VILLARREAL) Date: Tue, 10 Sep 91 12:33 PDT Subject: unsubscribe Message-ID: Please remove me from the connectionist mailing list. From hinton at ai.toronto.edu Tue Sep 10 18:04:27 1991 From: hinton at ai.toronto.edu (Geoffrey Hinton) Date: Tue, 10 Sep 1991 18:04:27 -0400 Subject: whoops Message-ID: <91Sep10.180442edt.423@neuron.ai.toronto.edu> I said that K nearest neighbors does well at generalizing parity from a randomly sampled training set when K is small and even. This was a mistake. What I meant was that K nearest neighbors does well when K is set so that the modal hamming distance to the K nearest neighbors is small and even. For example, if the training set contains a fraction, p, of all possible input vectors of length n, then if we set K to be: np + n(n-1)p/2 we can expect about np neighbors one bit away and about n(n-1)p/2 neighbors two bits away. If n is not very small, there will be far more neighbors two bits away than one bit away, so generalization will be correct. The use of a validation set to fix K should therefore give good generalization for n not too small. Geoff From CADEPS at BBRNSF11.BITNET Wed Sep 11 08:31:35 1991 From: CADEPS at BBRNSF11.BITNET (JANSSEN Jacques) Date: Wed, 11 Sep 91 14:31:35 +0200 Subject: Neural evolution group Message-ID: <9B98FC01C4400068@BITNET.CC.CMU.EDU> Dear ConXnists, I hear there's a neural evolution email interest group. Could somebody please broadcast its email address and what its aims are. Cheers, Hugo de Garis. From lehman at pinkpanther.llnl.gov Wed Sep 11 10:39:09 1991 From: lehman at pinkpanther.llnl.gov (Sean Lehman) Date: Wed, 11 Sep 91 07:39:09 PDT Subject: more RBF refs In-Reply-To: Tomaso Poggio's message of Mon, 9 Sep 91 20:58:53 EDT <9109100058.AA00733@erice> Message-ID: <9109111439.AA01026@pinkpanther.llnl.gov> -> From: Tomaso Poggio -> Date: Mon, 9 Sep 91 20:58:53 EDT -> -> Why call gradient descent backpropagation? -> -> I think you are confusing gradient descent, a mathematical method for finding a local mininum, with backpropagation, a learning algorithm for artificial neural networks. -->skl (Sean K. Lehman) LEHMAN2 at llnl.gov lehman at tweety.llnl.gov (128.115.53.23) ("I tot I taw a puddy tat") From mre1 at it-research-institute.brighton.ac.uk Wed Sep 11 15:26:14 1991 From: mre1 at it-research-institute.brighton.ac.uk (Mark Evans) Date: Wed, 11 Sep 91 15:26:14 BST Subject: Literature Databases Message-ID: <17583.9109111426@itri.bton.ac.uk> A couple of times when people have requested information about a certain paper or book over the connectionists network, I have noted people producing the details for the article in the form a database entry. I would like to know if there are any databases for neural network literature (or general literature databases) that users from external sites can access. Thank you, Mark Evans ################################################# # # # Mark Evans mre1 at itri.bton.ac.uk # # Research Assistant mre1 at itri.uucp # # # # ITRI, # # Brighton Polytechnic, # # Lewes Road, # # BRIGHTON, # # E. Sussex, # # BN2 4AT. # # # # Tel: +44 273 642915/642900 # # Fax: +44 273 606653 # # # ################################################# From D.M.Peterson at computer-science.birmingham.ac.uk Wed Sep 11 17:01:49 1991 From: D.M.Peterson at computer-science.birmingham.ac.uk (D.M.Peterson@computer-science.birmingham.ac.uk) Date: Wed, 11 Sep 91 17:01:49 BST Subject: No subject Message-ID: I'm interested in any possible connection between 'spontaneity' in the philosophical theory of judgement and connectionism. The idea in the theory of judgement is that people engage in reasoning, inference, discussion etc. about some problem, and then a decision or solution emerges which is good, but is not an inevitable consequence of what preceeded it. The preceding reasoning prepares the way for the decision, but does not necessitate it. The decision or solution is a *result* of the preceeding thought processes etc., but is not strictly a logical consequence of them. So if we take law-governed logical deduction (or perhaps law-governed deterministic causation) as our model, it becomes hard to explain this everyday phenomenon. That, briefly, is the idea, and I'd be very grateful for any leads on any perspective on this or analagous cases to be found in connectionism. Please send replies to: D.M.Peterson at cs.bham.ac.uk From p-mehra at uiuc.edu Wed Sep 11 16:57:53 1991 From: p-mehra at uiuc.edu (Pankaj Mehra) Date: Wed, 11 Sep 91 16:57:53 EDT Subject: Exploiting duality to analyze ANNs Message-ID: <9109112157.AA26067@rhea> I had been waiting a long time to see a paper that will relate the geometry of function spaces with the statistical theory of approximation and estimation. I finally got what I was looking for in a recent paper in the journal Neural Networks (vol. 4, pp. 443-451, 1991) titled ``Dualistic Geometry of the Manifold of Higher-Order Neurons,'' by Amari. I thought I will begin searching for additional references by sharing a few pointers: 1. ``Applied Regression Analysis,'' (2nd ed.) by Draper and Smith pp 491, Chapter 10, An Intro to Nonlinear Estimation. The idea of sample spaces is introduced and the concepts of approximation and estimation errors explained in geometric terms. 2. ``Principled Constructive Induction,'' by [yours truly], Rendell, & Wah, Proc. IJCAI-89. (Extended abstract in Machine Learning Workshop, 1989.) Abstract ideas of Satosi Watanabe on object-predicate duality are given a concrete interpretation for learning systems. This paper introduces inverted spaces similar to sample spaces (somewhat customized for 2-class discrimination). A preliminary result relating the geometry and statistics of feature construction is proved. 3. ``Generalizing the PAC Model ...,'' by Haussler, in FOCS'89. The concept of combinatorial dimension, which measures the ability of a function class to cover the combinatorially many orthants of the sample space, is used for extending PAC learning ideas to analysis of continuous maps. [well, this is the way I interpret it] Amari's work presents (in my opinion) an elegant treatment of approximation theory. His proofs are limited to HONNs transforming bipolar (-1,+1) inputs. But he mentions technical reports describing extensions to Boltzmann machines. (Can someone at Univ. of Tokyo send me a copy?) (Also, can someone help me understand how eqn 3.2 of Amari's paper follows from eqn 3.1?) I'd like to hear about other approaches that exploit duality between feature space and function space to characterize the behavior of neural networks. -Pankaj Mehra Univ. Illinois From tp-temp at ai.mit.edu Wed Sep 11 23:30:09 1991 From: tp-temp at ai.mit.edu (Tomaso Poggio) Date: Wed, 11 Sep 91 23:30:09 EDT Subject: more RBF refs In-Reply-To: Sean Lehman's message of Wed, 11 Sep 91 07:39:09 PDT <9109111439.AA01026@pinkpanther.llnl.gov> Message-ID: <9109120330.AA01122@erice> I am not the one responsible for the confusion (i.e. why finding new names for old and perfectly well known things like gradient descent and chain rule?) From port at iuvax.cs.indiana.edu Thu Sep 12 03:02:49 1991 From: port at iuvax.cs.indiana.edu (Robert Port) Date: Thu, 12 Sep 91 02:02:49 -0500 Subject: conference on dynamic models Message-ID: DYNAMIC REPRESENTATION IN COGNITION November 14-17, 1991 (Thurs eve til Sun noon) Indiana University - Bloomington, Indiana INVITED SPEAKERS James Crutchfield (UC Berkeley, Mathematics) Jeffrey Elman (UC San Diego, Cognitive Science) Walter Freeman (UC Berkeley, Physiology-Anatomy) Paul van Geert (Groningen University, Psychology) Jordan Pollack (Ohio State, Computer Science) Jean Petitot (CNRS Paris, Mathematics) Elliot Saltzman (Haskins Laboratories) James Townsend (IU Bloomington, Psychology) Michael T. Turvey (University of Connecticut, Psychology) Cognition is a dynamic phenomenon. Cognitive processes are in continuous adaptive interaction with the changing environment. The cognizing system must deal in real time with a constantly changing environment. Many crucial features of the environment, such as the escape path of prey or an utterance in a natural language, have an extended temporal structure. Further, in development and learning, the system itself undergoes change. Yet cognitive science has traditionally tried to abstract away from the dynamic nature of cognition, using various strategies -- such as dividing time into discrete segments or taking cognitive processing to be the sequential manipulation of static representational structures. Increasingly, this approach is being challenged by researchers in a wide variety of fields who are building dynamics directly into their theories and models of cognitive processes. These include many who now believe that dynamical systems theory is a more appropriate mathematical framework for the study of cognition than symbolic computation. A radical new conception of mental representation is gradually emerging: representations might hemselves be dynamic structures such as trajectories in a system state space. There are now many concrete examples of dynamical models of cognitive phenomena in areas such as motor control, olfaction and language processing. The aim of this workshop-style conference is to bring together many key researchers, to share perspectives from diverse areas of cognitive modeling, and to provide plenty of time to discuss the foundational issues in genuinely dynamical conceptions of cognition. ORGANIZING COMMITTEE Robert Port (Linguistics and Computer Science), co-chair port at cs.indiana.edu ph:(812)-855-9217 Timothy van Gelder (Philosophy), co-chair tgelder at ucs.indiana.edu ph:(812)-855-7088 Geoffery Bingham (Psychology), Linda Smith (Psychology), Esther Thelen (Psychology), James Townsend (Psychology) POSTER SESSION There will be a poster session Friday evening for work related to these issues. Posters will remain on display throughout the conference. Please submit your poster abstract before October 15, 1991. REGISTRATION FEE = $50 ($20 for students) CONFERENCE LIMIT = 120 persons FOR FURTHER INFORMATION US MAIL: Conference Registrar | EMAIL: MMACKENZ at UCS.INDIANA.EDU IU Conference Bureau, IMU Room 677 | PHONE: (812)-855-4661 Bloomington, IN 47406 | FAX: (812)-855-8077 SPONSORED BY: Office of Naval Research, IU Institute of the Study of Human Capabilities, Departments of Philosophy and Linguistics, and the Cognitive Science Program. From lo at vaxserv.sarnoff.com Thu Sep 12 09:19:49 1991 From: lo at vaxserv.sarnoff.com (Leonid Oliker x2419) Date: Thu, 12 Sep 91 09:19:49 EDT Subject: No subject Message-ID: <9109121319.AA17121@sarnoff.sarnoff.com> Please remove me from the connectionist mailing list. From sch at ee.UManitoba.CA Thu Sep 12 11:10:17 1991 From: sch at ee.UManitoba.CA (sch@ee.UManitoba.CA) Date: Thu, 12 Sep 91 10:10:17 CDT Subject: unsubscribe Message-ID: <9109121510.AA00183@ic14.ee.umanitoba.ca> Please take me off the list. From reznik at cs.wisc.edu Thu Sep 12 13:14:58 1991 From: reznik at cs.wisc.edu (Dan S. Reznik) Date: Thu, 12 Sep 91 12:14:58 -0500 Subject: request for removal Message-ID: <9109121714.AA01681@tao.cs.wisc.edu> please remove me from this list. thanks. dan reznik From ross at psych.psy.uq.oz.au Thu Sep 12 17:43:25 1991 From: ross at psych.psy.uq.oz.au (Ross Gayler) Date: Fri, 13 Sep 1991 07:43:25 +1000 Subject: interpolation vs generalisation Message-ID: <9109122143.AA03888@psych.psy.uq.oz.au> The people following this thread might want to consider where analogical inference fits in. Analogical inference is a form of generalisation that is performed on the basis of structural or relational similarity rather than literal similarity. It is generalisation, because it involves the application of knowledge from previously encountered situations to a novel situation. However, the interpolation does not occur in the space defined by the input patterns, instead it occurs in the space describing the structural relationships of the input tokens. The structural relationships between any set of inputs is not necessarily fixed by those inputs, but generated dynamically as an 'interpretation' that ties the inputs to a context. There is an argument that analogical inference is the basic mode of retrieval from memory, but most connectionist research has focused on the degenerate case where the structural mapping is an identity mapping - so the interest is focused on interpolation in the input space instead of the structural representation space. In brief: Generalisation can occur without interpolation in a data space that you can observe, but it may involve interpolation in some other space that is constructed internally and dynamically. Ross Gayler ross at psych.psy.uq.oz.au From rudnick at ogicse Thu Sep 12 20:03:57 1991 From: rudnick at ogicse (Mike Rudnick) Date: Thu, 12 Sep 1991 17:03:57 PDT Subject: Neural evolution group In-Reply-To: JANSSEN Jacques "Neural evolution group" (Sep 12, 1:21) Message-ID: <9109130003.AA09392@cse.ogi.edu> Neuro-evolution is a forum for discussion of technical issues relating to using genetic algorithms (GAs) and evolutionary approachs for the design of artificial neural networks. Other GA/ANN topics are also welcome. Postings of abstracts, notices of availability of tech reports and papers, references, general discussion, and the like are welcome. Send requests to have your name added to the distribution list to neuro-evolution-request at cse.ogi.edu. Mike Rudnick From meyer at FRULM63.BITNET Fri Sep 13 10:49:54 1991 From: meyer at FRULM63.BITNET (meyer) Date: Fri, 13 Sep 91 16:49:54 +0200 Subject: A new journal Message-ID: <9109131449.AA16028@wotan.ens.fr> ============================= Call for papers ============================== A D A P T I V E B E H A V I O R An international journal devoted to experimental and theoretical research on adaptive behavior in animals and in autonomous artificial systems, with emphasis on mechanisms, organizational principles, and architectures that can be expressed in computational, physical, or mathematical models. Broadly, behavior is adaptive if it deals successfully with changed circumstances. The adapting entities may be individuals or populations, over short or long time scales. The journal will publish articles, reviews, and short communications that treat the following topics, among others, from the perspective of adaptive behavior. Perception and motor control Ontogeny, learning and evolution Motivation and emotion Action selection and behavioral sequences Internal world models and cognitive processes Architectures, organizational principles, and functional approaches Collective behavior Characterization of environments Among its scientific objectives, the Journal aims to emphasize an approach complementary to traditional AI, in which basic abilities that allow animals to survive, or robots to perform their mission in unpredictable environments, will be studied in preference to more elaborated and human-specific abilities. The Journal also aims to investigate which new insights into intelligence or cognition can be achieved by explicitly taking into account the environmental feedback --mediated by behavior--that an animal or a robot receives, instead of studying components of intelligence in isolation. The journal will be published quarterly, beginning with the Summer issue of 1992. EDITOR-IN-CHIEF Jean-Arcady Meyer (Ecole Normale Superieure, France) email: meyer at wotan.ens.fr meyer at frulm63.bitnet tel: (1) 43 29 12 25 ext 3623 fax: (1) 43 29 70 85 ASSOCIATE EDITORS Randall Beer (Case Western Reserve Univ., USA) Lashon Booker (MITRE Corp., USA) Jean-Louis Deneubourg (Univ. of Bruxelles, Belgium) Janet Halperin (Univ. of Toronto, Canada) Pattie Maes (MIT Media Lab., USA) Herbert Roitblat (Univ. of Hawaii, USA) Ronald Williams (Northeastern University, USA) Stewart Wilson (The Rowland Institute for Science, USA). EDITORIAL BOARD David Ackley (Bellcore, USA) Michael Arbib (Univ. South. California, USA) Andrew Barto (Univ. of Massachusetts, USA) Richard Belew (Univ. of California, USA) Rodney Brooks (MIT AI Lab., USA) Patrick Colgan (Canadian Museum of Nature, Canada) Holk Cruse (Univ. Bielefeld, Germany) Daniel Dennett (Tufts Univ., USA) Jorg-Peter Ewert (Univ. Kassel, Germany) Nicolas Franceschini (Univ. Marseille, France) David Goldberg (Univ. of Illinois, USA) John Greffenstette (Naval Research Lab., USA) Patrick Greussay (Univ. Paris 8, France) Stephen Grossberg (Center for Adaptive Systems, USA) John Holland (Univ. Michigan, USA) Keith Holyoak (Univ. California, USA) Christopher Langton (Los Alamos National Lab., USA) David McFarland (Univ. of Oxford, UK) Thomas Miller (Univ. of New Hampshire, USA) Norman Packard (Univ. of Illinois, USA) Tim Smithers (Edinburgh Univ., UK) Luc Steels (VUB AI Lab., Belgium) Richard Sutton (GTE Labs., USA) Frederick Toates (The Open University, UK) David Waltz (Thinking Machines Corp., USA) To be published, an article should report substantive new results that significantly advance understanding of adaptive behavior. Critical reviews of existing work will also be considered. Contributions will originate from a range of disciplines including robotics, artificial intelligence, connectionism, classifier systems and genetic algorithms, psychology and cognitive science, behavioral ecology, and ethology among others. Ideally, an article will suggest implications for both natural and artificial systems. Authors should aim to make their results, and the results' significance, clear and understandable to the Journal's multi- disciplinary readership. Very general, speculative, or narrowly specialized papers, papers with substantially incomplete conceptual, experimental, or computational results, or papers irrelevant to the subject of adaptive behavior may be returned to authors without formal review. Submissions should be sent to: Dr. Jean-Arcady Meyer, Editor Adaptive Behavior Groupe de BioInformatique Ecole Normale Superieure 46 rue d'Ulm 75230 Paris Cedex05 FRANCE Please send five (5) copies of all materials. Manuscripts must be in English, with American spelling preferred. Please briefly define terms that may not be familiar outside your specialty. Avoid jargon and non-standard abbreviations. Make every attempt to employ technical terms that are already in use before making up new ones. The following guidelines should be adhered to, or papers may be returned for reformatting prior to review. Double-space all materials. Manuscripts should be typed (or laser printed) on 8 1/2 x 11 inch or A4 paper, one side only, with one-inch margins all around. Every page should be numbered in the upper right hand corner starting with the title page. Manuscript length should not normally exceed the equivalent of twenty journal pages. The title page (page 1) should have: - the paper's title, preferably not too long - the names, affiliations, and complete addresses of the authors, including electronic mail addresses if available - a daytime telephone number for the author with whom the editors should correspond. The second page should contain an abstract of 200 words or less, a list of six or fewer key words, and a shortened title for use as a running head. Begin the text of the article on page 3. Aid the reader by dividing the text into logical sections and subsections. Footnotes may be used sparingly. Follow the text with acknowledgements on a separate page. Begin the reference list on a new page following the acknowledgements page. References and citations should conform to the APA Publication Manual except: (1) do not cite page numbers of any book; (2) use the same format for unpublished references as for published ones. Please carefully check citations and references to be sure thay are correct and consistent. Note that the names of all authors of a publication should be given in the reference list and the first time it is cited in the text; after that "et al." may be used in citations. If a publication has 3 or more authors, "et al." may also be used in the first citation unless ambiguity would result. Include figures and tables at the end of the manuscript. Number them consecutively using Arabic numerals. Include a brief title above each table and a caption below each figure. Indicate in the text an approximate position for each figure and table. Besides graphical material, figures consisting of high quality black and white photographs are acceptable. Submit only clear reproductions of artwork. Authors should retain original artwork until the final version of the manuscript has been accepted. No page charges will be levied. Authors may order reprints when corrected proofs are returned. For subscription information, please contact: MIT Press Journals Circulation Department 55 Hayward Street Cambridge, Ma 02142 tel: 617-253-2889 fax: 617-258-6779 From sussmann at hamilton.rutgers.edu Fri Sep 13 13:02:11 1991 From: sussmann at hamilton.rutgers.edu (sussmann@hamilton.rutgers.edu) Date: Fri, 13 Sep 91 13:02:11 EDT Subject: more RBF refs Message-ID: <9109131702.AA08973@hamilton.rutgers.edu> > Date: Wed, 11 Sep 91 07:39:09 PDT > From: Sean Lehman > > I think you are confusing gradient descent, a mathematical method for > finding a local mininum, with backpropagation, a learning algorithm > for artificial neural networks. I don't quite understand the distinction. Backpropagation is of course "a learning algorithm for artificial neural networks," but it consists of using gradient descent to look for a local minimum of a function. (And, yeah, to compute the gradient one uses the chain rule.) You seem to be saying that, because backpropagation is not gradient descent in general, but gradient descent in a special case, then it's not gradient descent. Or am I missing something? ---Hector Sussmann From watrous at cortex.siemens.com Fri Sep 13 17:14:26 1991 From: watrous at cortex.siemens.com (Ray Watrous) Date: Fri, 13 Sep 91 17:14:26 EDT Subject: Poor Taste Message-ID: <9109132114.AA12637@cortex.siemens.com.noname> I find offensive this practice of unsubscribing to the connectionists list with a petulant note to the whole readership. If people have lost interest, why don't they leave quietly via Connectionists-Request? If they are dissatisfied, and want everyone to know it, they should grow up and act more constructively. Ray Watrous From GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca Sat Sep 14 23:33:57 1991 From: GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca (GOLDFARB%unb.ca@UNBMVS1.csd.unb.ca) Date: Sun, 15 Sep 91 00:33:57 ADT Subject: No subject Message-ID: From mozer at dendrite.cs.colorado.edu Sat Sep 14 15:45:17 1991 From: mozer at dendrite.cs.colorado.edu (Mike Mozer) Date: Sat, 14 Sep 91 13:45:17 -0600 Subject: tech report announcement Message-ID: <199109141945.AA20028@neuron.cs.colorado.edu> Sorry to disappoint you, but this is not another request to be removed from the mailing list. Please do not forward this announcement to other boards. Thank you. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- LEARNING TO SEGMENT IMAGES USING DYNAMIC FEATURE BINDING Michael C. Mozer, Richard S. Zemel, and Marlene Behrmann Despite the fact that complex visual scenes contain multiple, overlapping objects, people perform object recognition with ease and accuracy. One operation that facilitates recognition is an early segmentation process in which features of objects are grouped and labeled according to which object they belong. Current computational systems that perform this operation are based on predefined grouping heuristics. We describe a system called MAGIC that _learns_ how to group features based on a set of presegmented examples. In many cases, MAGIC discovers grouping heuristics similar to those previously proposed, but it also has the capability of finding nonintuitive structural regularities in images. Grouping is performed by a relaxation network that attempts to dynamically bind related features. Features transmit a complex-valued signal (amplitude and phase) to one another; binding can thus be represented by phase locking related features. MAGIC's training procedure is a generalization of back propagation to complex-valued units. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- This TR has been placed in the Neuroprose archive at Ohio State. Instructions for its retrieval are given below. If you are unable to retrieve and print the TR and therefore wish to receive a hardcopy, please send mail to conn_tech_report at cs.colorado.edu. Please do not reply to this message. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- FTP INSTRUCTIONS NOTE CHANGE OF HOSTNAME FROM cheops TO archive ---------------------------------------------- unix> ftp archive.cis.ohio-state.edu (or 128.146.8.52) Name: anonymous Password: neuron ftp> cd pub/neuroprose ftp> binary ftp> get mozer.segment.ps.Z ftp> quit unix> zcat mozer.segment.ps.Z | lpr From PHARMY at SUMMER.CHEM.SU.OZ.AU Sun Sep 15 20:32:15 1991 From: PHARMY at SUMMER.CHEM.SU.OZ.AU (PHARMY@SUMMER.CHEM.SU.OZ.AU) Date: Mon, 16 Sep 1991 10:32:15 +1000 (EST) Subject: morphology Message-ID: <910916103215.2720065a@SUMMER.CHEM.SU.OZ.AU> hi I am trying to locate a program that does fractal analysis of cell images which is compatible with a IBM or IBM clone. I'd also be greatful for any replies relating to mailing lists dealing with information processing of single neurons in relation to the observed morphology. thank you herbert Jelinek From jim at hydra.maths.unsw.OZ.AU Sun Sep 15 20:15:52 1991 From: jim at hydra.maths.unsw.OZ.AU (jim@hydra.maths.unsw.OZ.AU) Date: Mon, 16 Sep 91 10:15:52 +1000 Subject: interpolation vs generalisation Message-ID: <9109160015.AA23675@hydra.maths.unsw.OZ.AU> What is the `argument that analogical inference is the basic mode of retrieval from memory'? Jim Franklin From marek at iuvax.cs.indiana.edu Mon Sep 16 09:41:47 1991 From: marek at iuvax.cs.indiana.edu (Marek W Lugowski) Date: Mon, 16 Sep 91 08:41:47 -0500 Subject: interpolation vs generalisation Message-ID: would Pentti Kanerva's model of associative memory fit in with your definition of analogical inference? It seems that the input of successive binary vectors to his distributed memory creats a structure in Hamming space, often creating never-before seen resident vectors... which are then evoked. -- Marek P.s. On the other hand, Jim Keeler shows that Kanerva memory and Hopfield nets can have a common notation, so I am confused. After all Hoopfield nets are connectionist. From tackett at ipla00.hac.com Sun Sep 15 11:17:57 1991 From: tackett at ipla00.hac.com (Walter Tackett) Date: Sun, 15 Sep 91 11:17:57 EDT Subject: more RBF refs Message-ID: <9109151817.AA23567@ipla00.ipl.hac.com> uh.... you got the wrong guy. excuse me, person. My posting regarded how to build a many-class classifier, since bp seems to break down rather severely as the number of classes increases. btw, i have been severely dissapointed at the lack of responses. none, to be specific. the mail i rec'd from you regarded rbf's & gradient descent. -thanks, anyway.... walter hughes aircraft and the university of southern california base all policy decisions solely on my opinion. watch out. From hanna at cs.uq.oz.au Sun Sep 15 20:10:33 1991 From: hanna at cs.uq.oz.au (hanna@cs.uq.oz.au) Date: Mon, 16 Sep 91 10:10:33 +1000 Subject: unsubscribe Message-ID: <9109160010.AA08192@client> Please UNSUBSCRIBE From tedwards at wam.umd.edu Mon Sep 16 12:06:18 1991 From: tedwards at wam.umd.edu (Thomas VLSI Edwards) Date: Mon, 16 Sep 91 12:06:18 EDT Subject: backprop vs. gradient descent Message-ID: <9109161606.AA03424@next07pg2.wam.umd.edu> -> From: Tomaso Poggio -> Date: Mon, 9 Sep 91 20:58:53 EDT -> -> Why call gradient descent backpropagation? -> -> -->skl (Sean K. Lehman) LEHMAN2 at llnl.gov lehman at tweety.llnl.gov (128.115.53.23) ("I tot I taw a puddy tat") says.. >I think you are confusing gradient descent, a mathematical >method for finding a local mininum, with backpropagation, a learning >algorithm for artificial neural networks. Probably the best way to deal with this is to consider backpropagation as a manner of obtaining the error gradient of a neural net with multiplicative weights and a typically non-linear transfer function (as you must propagate the error back through the net to obtain the gradient, even if you use it for something like conjugate gradient). Extending the definition of backpropagation to include delta weight=learning rate times gradient confuses people, although it should be mentioned that this simple gradient descent method was often used in early backpropagation applications. -Thomas Edwards From p-mehra at uiuc.edu Mon Sep 16 11:49:05 1991 From: p-mehra at uiuc.edu (Pankaj Mehra) Date: Mon, 16 Sep 91 11:49:05 EDT Subject: Exploiting duality to analyze ANNs Message-ID: <9109161649.AA14738@manip> Several people requested a PostScript copy of the following citation from my previous note: - - ------ 2. ``Principled Constructive Induction,'' by Mehra, Rendell, & Wah, Proc. IJCAI-89. (Extended abstract in Machine Learning Workshop, 1989.) Abstract ideas of Satosi Watanabe on object-predicate duality are given a concrete interpretation for learning systems. This paper introduces inverted spaces similar to sample spaces (somewhat customized for 2-class discrimination). A preliminary result relating the geometry and statistics of feature construction is proved. - - ------ A REVISED version of this paper is now available from the neuroprose archive. The IJCAI paper contained a few oversights; these have been fixed in the recent revision. - - -Pankaj +++++++ HERE'S HOW TO GET THE PAPER +++++++ ftp archive.cis.ohio-state.edu (128.146.8.52) Connected to 128.146.8.52. 220 archive FTP server (SunOS 4.1) ready. Name (128.146.8.52:you): anonymous 331 Guest login ok, send ident as password. Password: neuron 230 Guest login ok, access restrictions apply. ftp> cd pub/neuroprose 250 CWD command successful. ftp> binary 200 Type set to I. ftp> get mehra.duality.ps.Z 200 PORT command successful. 150 Binary data connection for mehra.duality.ps.Z (128.174.31.18,1296) (48242 by tes). 226 Binary Transfer complete. local: mehra.duality.ps.Z remote: mehra.duality.ps.Z 48242 bytes received in 6.4 seconds (7.3 Kbytes/s) ftp> quit 221 Goodbye. uncompress mehra.duality.ps lpr -P mehra.duality.ps From qin at eng.umd.edu Mon Sep 16 15:45:20 1991 From: qin at eng.umd.edu (Si-Zhao Qin) Date: Mon, 16 Sep 91 15:45:20 -0400 Subject: Authors Kuhn & Herzberg Message-ID: <9109161945.AA21518@cm14.eng.umd.edu> I need the authors' initials for the following paper: Kuhn & Herzberg, "Variations on Training of Recurrent Networks", from 24th Conference on Information Sciences and Systems, Princeton, N.J., 3/21/90. I would appreciate if anybody could find the authors' full names. Thanks. Joe at qin at eng.umd.edu From karit at spine.hut.fi Tue Sep 17 11:06:45 1991 From: karit at spine.hut.fi (Kari Torkkola) Date: Tue, 17 Sep 91 11:06:45 DST Subject: RESEARCH POSITIONS IN SPEECH PROCESSING IN SWITZERLAND Message-ID: <9109170806.AA01412@spine.hut.fi.hut.fi> RESEARCH POSITIONS AVAILABLE IN SPEECH PROCESSING The newly created "Institut Dalle Molle d'Intelligence Artificielle Perceptive" (IDIAP) in Martigny, Switzerland seeks to hire qualified researchers in the area of automatic speech recognition. Candidates should be able to conduct independent research in a UNIX environment on the basis of solid theoretical and applied knowledge. Salaries will be aligned with those offered by the Swiss government for equivalent positions. Researchers are expected to begin activity in the beginning of 1992. IDIAP is supported by the Dalle Molle Foundation along with public-sector partners at the local and federal levels (in Switzerland). IDIAP is the third institute of artificial intelligence supported by the Dalle Molle Foundation, the others being ISSCO (attached to the University of Geneva) and IDSIA (situated in Lugano). The new institute maintains close contact with these latter centers as well as with the Polytechnical School of Lausanne and the University of Geneva. Applications for a research position at IDIAP should include the following elements: - a curriculum vitae - sample publications or technical reports - a brief description of the research programme that the candidate wishes to pursue - a list of personal references. Applications are due by December 1, 1991 and may be sent to the address below: Daniel Osherson IDIAP Case Postale 609 CH-1920 Martigny SWITZERLAND For further information by e-mail, contact: osherson at idiap.ch (Daniel Osherson, director) or karit at idiap.ch (Kari Torkkola, researcher) Please use the latter email address only for inquiries concerning speech recognition research. From cutrer at bend.UCSD.EDU Tue Sep 17 11:42:11 1991 From: cutrer at bend.UCSD.EDU (Michelle Cutrer) Date: Tue, 17 Sep 91 08:42:11 PDT Subject: unsubscribe Message-ID: <9109171542.AA26240@bend.UCSD.EDU> unsubscribe From ai-vie!georg at relay.EU.net Tue Sep 17 13:29:58 1991 From: ai-vie!georg at relay.EU.net (Georg Dorffner) Date: Tue, 17 Sep 91 19:29:58 +0200 Subject: meeting: connectionism and cognition Message-ID: <9109171729.AA05125@ai-vie.uucp> ! ! ! Call for Papers ! ! ! Symposium on CONNECTIONISM AND COGNITIVE PROCESSING as part of the Eleventh European Meeting on Cybernetics and Systems Research (EMCSR) April 21 - 24, 1992 Vienna, Austria Chairs: Noel Sharkey (Univ. of Exeter) Georg Dorffner (Univ. of Vienna) After the successes of two sessions on Parallel Distributed Processing at the previous EMCSRs, this symposium is designed to keep abreast with the increasing importance of connectionism (neural networks) in artificial intelligence, cognitive science, as well as in neuroscience and the philosophy and psychology of the mind. Therefore, original papers from all these areas are invited. Descriptions of implemented models are as welcome as theoretical or application-oriented contributions. Papers must not exceed 7 single-spaced A4 pages (max. 50 lines, final size will be 8.5 x 6 inch) and be written in English. They must contain the final text to be submitted, including graphs and figures (these need not be of reducible quality). Please send t h r e e copies of each submission. The deadline for submissions is Oct 15, 1991 (postmarked). However, if a brief note of intent to submit a paper (containing the tentative title) is emailed to georg at ai-vie.uucp (alternatively to georg%ai-vie.uucp at relay.eu.net) by the above date, papers can be sent until Nov 1, 1991 (postmarked) Authors will then be notified about acceptance within three to four weeks. Authors of accepted papers will be provided detailed instructions for the final format for the proceedings to be published by the time of the conference. Send all submissions - marked with the letter 'N' for the connectionist symposium - to EMCSR conference secretariat Oesterreichische Studiengesellschaft fuer Kybernetik Schottengasse 3 A-1010 Vienna, Austria or inquire about more details at the same address, at tel: +43 1 535 32 810 or at email: georg at ai-vie.uucp ------------------------------------------------------ other symposia at the EMCSR will be: A: General Systems Methodology B: Mathematical Systems Theory C: Computer Aided Process Interpretation D: Fuzzy Sets, Approximate Reasoning and Knowledge-Based Systems E: Designing and Systems F: Humanity, Architecture and Conceptualization G: Biocybernetics and Mathematical Biology H: Systems and Ecology I: Cybernetics in Medicine J: Cybernetics of Socio-Economic Systems K: Systems, Management and Organization L: Cybernetics of National Development M: Communication and Computers O: Intelligent Autonomous Systems P: Telepresence, Virtual Environments, and Interactive Fantasy Q: Impacts of Artificial Intelligence R: History of Cybernetics and Systems Research Submissions to these symposia can also be sent to the secretariat no later than Oct 15, 1991 (the above extended deadline only applies to the connectionist meeting). From jfj at m53.limsi.fr Wed Sep 18 04:17:59 1991 From: jfj at m53.limsi.fr (Jean-Francois Jadouin) Date: Wed, 18 Sep 91 10:17:59 +0200 Subject: TIme-unfolding Message-ID: <9109180817.AA16735@m53.limsi.fr> Dear connectionists, I've been doing a little work with Time-Unfolding Networks (first mentioned, I think, in PDP. Paul Werbos has an article out on the technique in the proceedings of the IEEE 1990). I've been getting pretty terrible results. My intuition is that I've misunderstood the model. Does anyone out there use this model ? If so, would you be prepared to exchange benchmark results (or even better, software) and compare notes ? A little discouraged, jfj From jfj at m53.limsi.fr Wed Sep 18 04:57:13 1991 From: jfj at m53.limsi.fr (Jean-Francois Jadouin) Date: Wed, 18 Sep 91 10:57:13 +0200 Subject: Bibliography to a novice ... Message-ID: <9109180857.AA16836@m53.limsi.fr> The most-often cited reference I know of, and one you should definitely read is: J. McClelland, D. Rumelhart, Parallel Distributed Processing vols 1 & 2, MIT press, 1986. The following is a pretty complete collection of the most influential papers in the field, Anderson, Rosenfeld, Neurocomputing, foundations of research, MIT press, 1989. Both these references are from memory and may be a little off - my apologies for this. Oh yes: Artificial Intelligence (# 40, I think), had a special issue on learning, where Hinton wrote a good introduction on connectionist learning procedures. Good luck ! jfj From pollack at cis.ohio-state.edu Wed Sep 18 11:48:22 1991 From: pollack at cis.ohio-state.edu (Jordan B Pollack) Date: Wed, 18 Sep 91 11:48:22 -0400 Subject: Neuroprose Message-ID: <9109181548.AA01754@dendrite.cis.ohio-state.edu> ***Do not forward to other bboards*** As many of you are aware by now, the cis facility at OSU has been unstable the last week of August and first week of September. One of the changes which occured in the spring was that the cheops pyramid computer was replaced with an alias to a new sun server called: archive.cis.ohio-state.edu 128.146.8.52 (Cheops was ..62) One of the changes which recently happened is that the alias mechanism has become unreliable, so transparent access through the "cheops" name has repeatedly failed for people. Please change your Getps and neuroprose scripts accordingly. Also, I'd like to remind the LIST that Neuroprose is a convenience mechanism for the low-cost distribution of preprints and reprints to the MAILING LIST. Neuroprose is NOT a vanity press -- placing a report or book in neuroprose does not constitute publication. Many of the papers in neuroprose are reprints of journal articles and book chapters, THEREFORE NEUROPROSE IS NOT PUBLIC DOMAIN SOFTWARE. While I can appreciate some people wanting to share their results with the highest possible readership, the forwarding of FTP announcements to neuron-digest and/or comp.ai.neural networks will eventually cause major legal problems for somebody. So, when you announce a paper, please include "**do not forward**"'s in the message. Finally, please halt suggestions that the papers be copied all over the world, distributed on floppy disk, or accessed through various public servers. "Don't call me Kinko's" Pollack Assistant Professor CIS Dept/OSU Laboratory for AI Research 2036 Neil Ave Email: pollack at cis.ohio-state.edu Columbus, OH 43210 Phone: (614)292-4890 (then * to fax) ***Do not forward to other ML's or Newsgroups*** From kddlab!hcrlgw.crl.hitachi.co.jp!nitin at uunet.UU.NET Thu Sep 19 14:36:35 1991 From: kddlab!hcrlgw.crl.hitachi.co.jp!nitin at uunet.UU.NET (Nitin Indurkhya) Date: Thu, 19 Sep 91 14:36:35 JST Subject: DP-matching with NN-hardware Message-ID: <9109190536.AA01532@hcrlgw.crl.hitachi.co.jp> one of the problems i'm facing in analyzing time-series data with DP-matching methods is that when the number of templates is very large, the run-time performance is quite poor (i.e. it is definitely not possible to do it in real-time). i recall reading somewhere about DP-matching with NN-hardware. can someone give me a reference on that. also, any pointers to more recent work in this area would be appreciated. i am using the one-stage dp method discussed in the excellent article by hermann ney. thanks in advance, --nitin indurkhya (nitin at crl.hitachi.co.jp) From tackett at ipla00.hac.com Thu Sep 19 08:56:55 1991 From: tackett at ipla00.hac.com (Walter Tackett) Date: Thu, 19 Sep 91 08:56:55 EDT Subject: Bibliography to a novice ... Message-ID: <9109191556.AA07873@ipla00.ipl.hac.com> i would add to jfj's list that IEEE computer had a real good special issue g that contained overview articles concerning ART, BP, Neocog, BAM, Hopfield, etc. And some kohonen, as i recall, by the original researchers. this was in late 88 or early 89. -walter From Renate_Crowley at unixgw.siemens.com Thu Sep 19 17:35:48 1991 From: Renate_Crowley at unixgw.siemens.com (Renate Crowley) Date: 19 Sep 91 17:35:48 Subject: NIPS 1991 submissions Message-ID: <9109192139.AA11973@siemens.siemens.com> Subject: Time:17:42 OFFICE MEMO NIPS 1991 submissions Date:9/19/91 For your information notifications regarding acceptance/rejection have been mailed to all first authors or corresponding authors last week in August. If you have not received a letter please contact me. Renate Crowley Tel 609 734 3311 Fax 609 734 6565 email: renate at siemens.siemens.com From kddlab!hcrlgw.crl.hitachi.co.jp!nitin at uunet.UU.NET Fri Sep 20 10:15:57 1991 From: kddlab!hcrlgw.crl.hitachi.co.jp!nitin at uunet.UU.NET (Nitin Indurkhya) Date: Fri, 20 Sep 91 10:15:57 JST Subject: Ney reference Message-ID: <9109200115.AA14595@hcrlgw.crl.hitachi.co.jp> i've rec'd many requests for a reference to the Ney paper: Hermann Ney, "The use of a one-stage dynamic programming algorithm for connected word recognition", ieee trans. assp 32(2):263-271, apr 1984 --nitin (nitin at crl.hitachi.co.jp) From 7923509%TWNCTU01.BITNET at BITNET.CC.CMU.EDU Fri Sep 20 13:45:00 1991 From: 7923509%TWNCTU01.BITNET at BITNET.CC.CMU.EDU (7923509%TWNCTU01.BITNET@BITNET.CC.CMU.EDU) Date: Fri, 20 Sep 91 13:45 U Subject: Could some one tell me ... ? Message-ID: <01GASD3QJRA8D7PO30@BITNET.CC.CMU.EDU> Hi: Could anyone tell me the following question ? 1.What is the reason that the states of the k th unit Sk = 1 with probability 1 Pk = ------------------- 1+exp(-delta Ek/T) where delta Ek is the energy gap between the 1 and 0 states of the k th unit, T is a parameter which acts like the temperature 2.Why this local decision rule ensures that in thermal equilibrium the relative probability of two global states is determined by their energy difference,and follows a Boltzmann distribution: Pa ---- = exp(-(Ea - Eb))/T Pb where Pa is the probability of being in the a th global state and Ea is the energy of that state. and how do we know that whether the system reaches thermal equilibrium or not? reply should be send to 7923509 at twnctu01 thank's a lot! From david at cns.edinburgh.ac.uk Fri Sep 20 11:53:09 1991 From: david at cns.edinburgh.ac.uk (David Willshaw) Date: Fri, 20 Sep 91 11:53:09 BST Subject: NETWORK Message-ID: <3796.9109201053@subnode.cns.ed.ac.uk> CONTENTS OF NETWORK - COMPUTATION IN NEURAL SYSTEMS Volume 2 Number 3 August 1991 LETTER TO THE EDITOR On the capacity of a neuron with a non-monotone output function. K Kobayashi PAPERS Realistic synaptic inputs for model neural networks. L F Abbott Quantitative study of attractor neural network retrieving at low spike rates: I. Substrate-spikes, rates and neuronal gain. D J Amit and M V Tsodyks Quantitative study of attractor neural network retrieving at low spike rates: II. Low-rate retrieval in symmetric networks. D J Amit and M V Tsodyks Dynamics of an auto-associative neural network model with arbitrary connectivity and noise in the threshold. H F Yanai, Y Sawada and S Yoshizawa A high storage capacity neural network content-addressable memory. E Hartman ABSTRACTS SECTION BOOK REVIEWS --------------------------------------------------------------------- Network welcomes research Papers and Letters where the findings have demonstrable relevance across traditional disciplinary boundaries. Research Papers can be of any length, if that length can be justified by content. Rarely, however, is it expected that a length in excess of 10,000 words will be justified. 2,500 words is the expected limit for research Letters. Articles can be published from authors' TeX source codes. Macros can be supplied to produce papers in the form suitable for refereeing and for IOP house style. For more details contact the Editorial Services Manager at IOP Publishing, Techno House, Redcliffe Way, Bristol BS1 6NX, UK. Telephone: 0272 297481 Fax: 0272 294318 Telex: 449149 INSTP G Email Janet: IOPPL at UK.AC.RL.GB Subscription Information Frequency: quarterly Subscription rates: Institution 125.00 pounds (US$220.00) Individual (UK) 17.30 pounds (Overseas) 20.50 pounds (US$37.90) A microfiche edition is also available at 75.00 pounds (US$132.00) From ross at psych.psy.uq.oz.au Fri Sep 20 08:31:54 1991 From: ross at psych.psy.uq.oz.au (Ross Gayler) Date: Fri, 20 Sep 1991 22:31:54 +1000 Subject: analogy, generalisation & interpolation (LONG!) Message-ID: <9109201231.AA20896@psych.psy.uq.oz.au> One week ago I made a posting that attempted to place analogical inference into the debate on interpolation and generalisation. I have received a number of replies (some direct and some via the mailing list) with sufficient overlap to justify a mass reply via the mailing list - so here it is. - References to my own work in this area Sadly, there are none. Designing a connectionist architecture to support dynamic analogical inference and retrieval is my night job, so I lack the time and resources to produce papers. Most of my work has been in scribbled notes and thought-experiments. My day job is mundane stuff in the computer industry, but I am searching for a new job right now and would love to convert my night job to a day one - so all offers will be considered. - References to other connectionist work on analogical inference The classic is: Holyoak, K.J., & Thagard, P. (1989). Analogical mapping by constraint satisfaction. Cognitive Science, 13(3), 295-355. They produced a program (ACME) that performs analogical inference by a connectionist constraint satisfaction network. The program takes two *symbolic* structures (base and target) and attempts to find a consistent mapping from base to target to fill some gaps in the target structure. For example, the base structure might describe a planetary system and the target structure describe a Rutherford atom. The program tries to map objects and predicates from the base into the target. The fun starts when the structures are not isomorphic, so there is ambiguity as to what is the 'best' mapping. ACME uses a symbolic program to parse the symbolic inputs and create a connectionist net to solve the constraint satisfaction problem. The net uses a localist representation (each unit corresponds to a partial mapping) and the weight between the units encode constraints on combining the partial mappings. After the net has been constructed it is set in action and the 'best' mapping read from the settled state. The Holyoak and Thagard paper is connectionist only in the sense that it uses a connectionist technique to solve the constraint satisafction problem. The theoretical problem they attacked was how to incorporate pragmatic constraints into the mapping. There was no intent for the specific mechanism to be plausible or useful. My aim is to produce a practically useful analogical retrieval mechanism. From jfj at m53.limsi.fr Fri Sep 20 07:03:58 1991 From: jfj at m53.limsi.fr (Jean-Francois Jadouin) Date: Fri, 20 Sep 91 13:03:58 +0200 Subject: Bibliography to a novice ... Message-ID: <9109201103.AA18473@m53.limsi.fr> The references for that IEEE special issue (actually, there are two) : Proceedings of the IEEE, Vol 78 no 9 (sept) & 10 (nov). The first issue addresses more basic models (among which the article by Kohonen tackett speaks of). The second is more appolications oriented. jfj From shawnd at ee.ubc.ca Fri Sep 20 18:19:00 1991 From: shawnd at ee.ubc.ca (shawnd@ee.ubc.ca) Date: Fri, 20 Sep 91 15:19:00 PDT Subject: Preprint Announcement Message-ID: <9109202220.AA01077@fridge.ee.ubc.ca> ** Please do not forward to other mailing lists ** The following preprint is available by ftp from the neuroprose archive at archive.cis.ohio-state.edu: Continuous-Time Temporal Back-Propagation with Adaptable Time Delays Shawn P. Day Michael R. Davenport Departments of Electrical Engineering and Physics University of British Columbia Vancouver, B.C., Canada ABSTRACT We present a generalization of back-propagation for training multilayer feed-forward networks in which all connections have time delays as well as weights. The technique assumes that the network inputs and outputs are continuous time-varying multidimensional signals. Both the weights and the time delays adapt using gradient descent, either in ``epochs'' where they change after each presentation of a training signal, or ``on-line'', where they change continuously. Adaptable time delays allow the network to discover simpler and more accurate mappings than can be achieved with fixed delays. The resulting networks can be used for temporal and spatio-temporal pattern recognition, signal prediction, and signal production. We present simulation results for networks that were trained on-line to predict future values of a chaotic signal using its present value as an input. For a chaotic signal generated by the Mackey-Glass differential-delay equation, networks with adaptable delays typically had less than half the prediction error of networks with fixed delays. Here's how to get the preprint from neuroprose: unix> ftp archive.cis.ohio-state.edu (or 128.146.8.52) Name: anonymous Password: neuron ftp> cd pub/neuroprose ftp> binary ftp> get day.temporal.ps.Z ftp> quit unix> uncompress day.temporal.ps.Z unix> lpr day.temporal.ps (or however you print postscript) Any questions or comments can be addressed to me at: Shawn Day Department of Electrical Engineering 2356 Main Mall University of British Columbia Vancouver, B.C., Canada V6T 1Z4 phone: (604) 264-0024 email: shawnd at ee.ubc.ca From fsegovia at batman.fi.upm.es Fri Sep 20 12:36:40 1991 From: fsegovia at batman.fi.upm.es (fsegovia@batman.fi.upm.es) Date: Fri, 20 Sep 1991 18:36:40 +0200 Subject: Bibliography to a novice Message-ID: <"<9109201636.AA18631@batman.fi.upm.es>*"@MHS> The following articles treat the problem of recurrent NN and their learning procedures. They also include modifications to the original BP applied to recurrent networks (the Rumelhart's trick of unfold in time the network's operation) which make the learning phase more practical (less computations, less time): Williams, R.J., and Zipser, D. 1990. "Gradient-Based learning algo rithms for recurrent connectionist networks". Tech. Rep. NU-CCS-90-9. Northeastern University, College of Computer Science, Boston. Williams, R.J., and Peng, J. 1990. "An Efficient Gradient-Based Algo rithm for On-Line Training of Recurrent Network Trajectories". Neural Computation, Vol. 2, Num 4, 490-501. For an extension to continuous time see: Pearlmutter, B.A. 1989. "Learning state space trjectories in recurrent neural neural networks". Neural Computation, Vol 1, Num 2, 263-269. Sejnowski, T.J., and Fang, Y. 1990. "Faster learning for dynamic recurrent backpropagation". Neural Computation, Vol 2, Num 3, 270-273. The papers mentioned above include experiments and good references for related works. Javier Segovia From jmyers at casbah.acns.nwu.edu Sun Sep 22 21:51:23 1991 From: jmyers at casbah.acns.nwu.edu (J Myers) Date: Sun, 22 Sep 91 20:51:23 CDT Subject: Add Message-ID: <9109230151.AA17154@casbah.acns.nwu.edu> Please add me to the mailing list. Thank you. From mdp at eng.cam.ac.uk Mon Sep 23 10:58:48 1991 From: mdp at eng.cam.ac.uk (Mark Plumbley) Date: Mon, 23 Sep 91 10:58:48 BST Subject: NCM'91: One Day Conference on Applications of NNs Message-ID: <12702.9109230958@dsl.eng.cam.ac.uk> One Day Conference Announcement: NCM'91: APPLICATIONS OF NEURAL NETWORKS October 1, 1991 AT CENTRE FOR NEURAL NETWORKS KING'S COLLEGE LONDON, UK Outline Programme 10.00: Coffee 10.30-12.50 Talks 1) N Biggs, "Learning Algorithms - theory and practice" 2) Dr EPK Tsang and Dr CJ Wang, "A Generic Neural Network Approach for Contraint Satisfaction Problems" 3) D Gorse, "Temporal Processing in Probabilistic RAM nets" 4) GJ Chappell (with J Lee and JG Taylor), "A Review of Medical Diagnostic Applications of Neural networks" 5) DL Toulson, JF Boyce and C Hinton, "Data Representation and Generalisation in an application of a Feedforward Net" 6) IW Ricketts, AY Cairns, S Dickson, M Hudson, K Hussein, M Nimmo, PE Preece, AJ Thompson and C Walker, "Artificial Neural Networks Applied to the Inspection of Medical Images" 12.30-1.45 Buffet Lunch (Provided) 1.45-2.00 AGM of BNNS 2.00-4.00 Talks 7) D Anthony, E Hines, D Hutchins and T Mottram, "Ultrasound Tomography Imaging of Defects using Neural Networks" 8) K Kodaira, H Nakata and M Takamura, "An Apple Sorting System Using Neural Network-Based on Image Processing" 9) EC Mertzanis, "Quadtrees for Neural Network based Position Invariant Pattern Recognition" 10) NA Jalel, AR Mirzai and JR Leigh, "Application of Neural Networks in Process Control" 11) CM Bishop, PM Cox, PS Haynes, CM Roach, TN Todd and DL Trotman, "A Neural Network Approach to Tokamak Equilibrium Control" 12) D Shumsheruddin, "Neural Network Control of Robot Arm Tracking Movements" 4.00-4.30 Tea 4.30-5.30 Talks 13) TG Clarkson, "The pRAM as a hardware-realisable neuron" 14) S Hancock, "A Neural Instruction Set Processor (NISP) and Development System (NDEV)" 15) RE Wright, "The Cognitive Modalities ('CM') System of Knowledge Representation The 'DNA' of Neural Networks?" The talks are each 20 minutes in length (including discussions). ------------------------%<------CUT HERE -----%<------------------------- REGISTRATION SLIP FOR NCM'91 I wish to attend the one day conference on APPLICATIONS OF NEURAL NETWORKS: NAME: ............................................................................... ADDRESS: ............................................................................... ............................................................................... (Please make cheque for 30 pounds sterling payable to `Centre for Neural Networks', and address it to: Prof J. G. Taylor, Centre for Neural Networks, King's College, Strand, London WC2R 2LS, UK) From jose at tractatus.siemens.com Mon Sep 23 18:45:08 1991 From: jose at tractatus.siemens.com (Steve Hanson) Date: Mon, 23 Sep 1991 18:45:08 -0400 (EDT) Subject: NIPS*91 UPDATE Message-ID: NIPS*91 FINAL POSTER PROGRAM: SESSIONS I & II TUESDAY EVENING: SESSION I From jose at tractatus.siemens.com Mon Sep 23 18:39:27 1991 From: jose at tractatus.siemens.com (Steve Hanson) Date: Mon, 23 Sep 1991 18:39:27 -0400 (EDT) Subject: NIPS*91 UPDATE Message-ID: NIPS*91 FINAL ORAL PROGRAM MONDAY DECEMBER 2 After Dinner Talk: Allan Hobson, Harvard Medical School "Models Wanted: Must Fit Dimensions of Sleep and Dreaming" TUESDAY DECEMBER 3 ORAL 1: LEARNING and GENERALIZATION I O.1.1 V. Vapnik, Institute of Control Sciences, Academy of Sciences "Principles of Risk Minimization for Learning Theory" (INVITED TALK) O.1.2 D. MacKay, Caltech "A Practical Bayesian Framework for Backprop Networks" 0.1.3 J .Moody, Yale Computer Science " Generalization, Weight Decay, and Architecture Selection for Nonlinear Learning Systems" O.1.4 D. Haussler UC Santa Cruz M. Kearns, International Computer Science Institute M. Opper, Institute fuer Theoretische Physik R. Schapire, Harvard University "Estimating Average-Case Learning Curves Using Bayesian, Statistical Physics and VC Dimension Methods" ORAL 2: TEMPORAL PROCESSING O.2.1 S. P. Singh, University of Massachusetts "The Efficient Learning of Multiple Task Sequences" O.2.2 G. Tesauro, IBM "Practical Issues in Temporal Difference Learning" O.2.3 H. Hild, Universitat Karlsruhe W. Menzel, Universitat Karlsruhe J. Feulner, Universitat Karlsruhe "A Neural Net For Harmonizing Chorals in the Style of J.S. Bach" O.2.4 M.A. Jabri, Sydney University S. Pickard, Sydney University P. Leong, Sydney University Z. Chi, Sydney University B. Flower, Sydney University "Architectures and Implementation of Right Ventricular Apex Signal Classifiers for Pacemakers" SPOTLIGHT I: TEMPORAL PROCESSING ORAL 3: VISUAL PROCESSING O.3.1 D. A. Robinson, Johns Hopkins University School of Medicine "Information Processing to Create Eye-Movements" (INVITED TALK) O.3.2 S. Becker, University of Toronto G. E. Hinton, University of Toronto "Learning to Make Coherent Predictions in Domains with Discontinuities" O.3.3 K. A. Boahen, Caltech A. G. Andreou, John Hopkins "A Contrast Sensitive Silicon Retina with Reciprocal Synapses" O.3.4 P.A. Viola, MIT S.G. Lisberger, UC San Francisco T.J. Sejnowski, Salk Institute for Biological Science, "Recurrent Eye Tracking Network Using a Distributed Representation of Image Motion" ORAL 4: OPTICAL CHARACTER RECOGNITION O.4.1 F. Faggin, Synaptics "Neural Network Analog VLSI Implementations" (INVITED TALK) O.4.2 J. D. Keeler, MCC D. E. Rumelhart, Stanford University "Self-Organizing Segmentation and Recognition Neural Network" O.4.3 I. Guyon, ATt&T Bell Laboratories V.N. Vapnik, ATt&T Bell Laboratories B.E. Boser, ATt&T Bell Laboratories L.Y. Bottou, ATt&T Bell Laboratories S.A. Solla, ATt&T Bell Laboratories "Structural Risk Minimization for Character Recognition" SPOTLIGHT II: VISUAL PROCESSING AND OCR SPOTLIGHT III: APPLICATIONS and PERFORMANCE COMPARISONS WEDNESDAY DECEMBER 4 ORAL 5: LEARNING and GENERALIZATION II O.5.1 J. S. Judd, Siemens Research "Constant-Time Loading of Shallow 1-Dimensional Networks" O.5.2 J. Alspector, Bellcore A. Jayakumar,Bellcore S. Luna, University of California "Experimental Evaluation of Learning in a Neural Microsystem" O.5.3 C. McMillan, University of Colorado M. C. Mozer, University of Colorado P. Smolensky, University of Colorado "Rule Induction Through A Combination of Symbolic and Subsymbolic Processing" O.5.4 G. Towell, University of Wisconsin-Madison J. W. Shavlik, University of Wisconsin-Madison "Interpretation of Artificial Neural Networks: Mapping Knowledge-Based NN into Rules" SPOTLIGHT IV: LEARNING & ARCHITECTURES ORAL 6: LOCOMOTION, PLANNING & CONTROL O.6.1 A. W. Moore, MIT "Fast, Robust Adaptive Control by Learning only Forward Models" O.6.2 S. B. Thrun, German National Research Center for Computer Science K. Moeller, University of Bonn "Active Exploration in Dynamic Environments" O.6.3 J. Buchanan, Marquette University "Locomotion In A Lower Vertebrate: Studies of The Cellular Basis Of Rhythmogenesis and Oscillator Coupling" O.6.4 M. Lemmon, University of Notre Dame "Oscillatory Neural Networks for Globally Optimal Path Planning" SPOTLIGHT V: NEURAL CONTROL ORAL 7: SELF ORGANIZATION, ARCHITECTURES and LEARNING O.7.1 M. I. Jordan, MIT R. A. Jacobs , MIT "Hierarchies of Adaptive Experts" O.7.3 S. J. Nowlan, University of Toronto G. E. Hinton, University of Toronto "Adaptive Soft Weight Tying using Gaussian Mixtures" O.7.4 D. Rogers, Research Institute for Advanced Computer Science "Friedman's Multivariate Adaptive Regression Splines (MARS)Algorithm with Holland's Genetic Algorithm" O.7.5 D. Wettschereck, Oregon State University T. Dietterich, Oregon State University "Improving the Performance of Radial Basis Function Networks by Learning Center Locations" SPOTLIGHT VI: SPEECH ORAL 8: VISUAL SYSTEM O.8.1 A. Bonds, Vanderbilt University "Dual Inhibitory Mechanisms for Definition of Receptive Field Characteristics in Cat Striate Cortex" O.8.2 K. Obermayer, University of Illinois K. Schulten, University of Illinois G.G. Blasdel, Harvard Medical School "Comparison Between a Neural Network Model for the Formation of Brain Maps and Experimental Data" O.8.3 R. Kessing, Ricoh California Research Center D. Stork, Ricoh California Research Center C.J. Schatz, Stanford University School of Medicine "Retinogeniculate Development: The Role of Competition and Correlated Retinal Activity" SPOTLIGHT VII: LEARNING & ARCHITECTURES THURSDAY DECEMBER 5 ORAL 9: APPLICATIONS O.9.1 M. R. Rutenburg, NeuroMedical Systems, Inc, "PapNET: A Neural Net Based Cytological Screening System" (INVITED TALK) O.9.2 M. Roscheisen Munich Technical University R. Hofmann, Munich Technical University V. Tresp, Siemens AG "Incorporating Prior Knowledge in Parsimonious Networks of Locally-Tuned Units" O.9.3 P. Smyth, JPL J. Mellstrom, JPL "Fault Diagnosis of Antenna Pointing Systems using Hybrid Neural Network and Signal Processing Techniques" O.9.4 C. F. Neugebauer, Caltech A. Yariv, Caltech " A Parallel Analog CCD/CMOS Neural Network IC with Digital I/O" BREAK ORAL 10: SPEECH & SIGNAL PROCESSING O.10.1 K. Church, AT&T Bell Labs "Part of Speech Tagging" (INVITED TALK) O.10.2 A. R. Bulsara, Naval Ocean Systems Center F. E. Moss, Univ. of Missouri "Single Neuron Dynamics: Noise-Enhanced Signal Processing" O.10.3 D. Warland, University of California, Berkeley F. Rieke, NEC Research Institute W. Bialek, NEC Research Institute "Efficient Coding in Sensory Systems" O.10.4 J. Platt, Synaptics F. Faggin, Synaptics "A Network for the Separation of Sources That Are Superimposed And Delayed" O.10.5 A. Waibel, CMU A. N. Jain, CMU A. E. McNair, CMU J. Tebelskis, CMU A. Hauptmann, CMU H. Saito, CMU "JANUS: Speech-to-Speech Translation Using Connectionist and Non-Connectionist Techniques" SPOTLIGHT TALKS (4 Minute Talks) SPOTLIGHT I: TEMPORAL PROCESSING J. Connor, University of Washington L.E. Atlas, University of Washington D. Martin, University of Washington "Advantages of Recurrency and Innovations in Time Series Prediction" C. Brody, ILMAS, UNAM "Fast Learning with Predictive Forward Models" C.G. Atkeson, MIT "Robot Juggling: A Real-Time Implementation of Learning Based on Locally Weighted Regression" SPOTLIGHT II: OCR and VISUAL PROCESSING G.L. Martin, MCC & Eastman Kodak "Centered-Object Integrated Segmentation and Recognition for Visual Character Recognition" G.E. Hinton, University of Toronto C.K.I. Williams, University of Toronto "Adaptive Elastic Models for Character Recognition" S. Ahmad, ICSI "Modelling Visual Attention and Visual Search" D.Z. Anderson, University of Colorado C. Benkert, University of Colorado V. Hebler, University of Colorado J.S. Jang, University of Colorado D. D. Montgomery, University of Colorado M. Saffman, University of Colorado "Optical Implementation of a self-organizing feature extractor" SPOTLIGHT III: APPLICATIONS AND PERFORMANCE COMPARISONS M.W. Goudreau, NEC Research Institute, Inc. C.L. Giles, NEC Research Institute, Inc. "Neural Network Routing for Random Multistage Interconnection Networks" P. Stolorz, Los Alamos National Laboratory A. Lapedes, Los Alamos National Laboratory R. Farber, Los Alamos National Laboratory D. Wolf, Los Alamos National Laboratory Y. Xia, Los Alamos National Laboratory J. Bryngelson, Los Alamos National Laboratory "Prediction of Protein Structure Using Neural Nets and Information Theory" P. Kohn, ICSI J. Bilmes, ICSI N. Morgan, ICSI J. Beck, ICSI "Software for ANN Training on a Ring Array Processor" J. Bernasconi, Asea Brown Boveri Corp. K. Gustafson, University of Colorado "Human and Machine 'Quick Modeling'" L.G.C. Hamey, Macquarie University "Benchmarking feed-forward Neural Networks: models and measures" SPOTLIGHT IV: LEARNING & ARCHITECTURES P. Koistinen, Rolf Nevanlinna Institute L. Holmstroem, Rolf Nevanlinna Institute "Kernel Regression and Backpropagation Training With Noise" K.Y. Siu, Stanford University J. Bruck, IBM Research Division "Neural Computing With Small Weights" SPOTLIGHT V: NEURAL CONTROL T. Anastasio, USC "Learning In the Vestibular System: Simulations Of Vestibular Compensation Using Recurrent Back-Propagation" P. Dean, University of Sheffield E.W. Mayhew, University of Sheffield "A Neural Net Model For Adaptive Control of Saccadic Accuracy By Primate Cerebellum and Brainstem" SPOTLIGHT VI: SPEECH R. Cole, Oregon Graduate Institute of Science and Technology K. Roginski, Oregon Graduate Institute of Science and Technology M. Fanty, Oregon Graduate Institute of Science and Technology "English Alphabet Recognition With Telephone Speech" Y. Bengio, McGill University R. DeMori, McGill University G. Flammia, McGill University R. Kompe, McGill University "Global Optimization of Neural Network - Hidden Markov Model Hybrid" SPOTLIGHT VII: LEARNING AND ARCHITECTURES J. Bridle, R.S.R.E. D.J.C. MacKay, Caltech "Unsupervised Classifiers, Mutual Information And 'Phantom Targets'" P. Simard, AT&T Bell Laboratories B. Victorri, Universite de Caen Y. Le Cun, AT&T Bell Laboratories John Denker, AT&T Bell Laboratories "Tangent Prop - A formalism for specifying selected invariances in an adaptive network" D. Montana, Bolt Beranek and Newman, Inc. "A weighted probabilistic Neural Net" T. Lange, University of California "Dynamically-Adaptive Winner-Take-All Networks" S. Omohundro, International Computer Science Institute "Model-Merging for Improved Generalization" From jose at tractatus.siemens.com Mon Sep 23 18:46:11 1991 From: jose at tractatus.siemens.com (Steve Hanson) Date: Mon, 23 Sep 1991 18:46:11 -0400 (EDT) Subject: NIPS*91 UPDATE Message-ID: <8craunS1GEMn41K2Ih@tractatus.siemens.com> NIPS*91 FINAL POSTER PROGRAM: SESSIONS I & II TUESDAY EVENING: SESSION I From jose at tractatus.siemens.com Mon Sep 23 18:51:11 1991 From: jose at tractatus.siemens.com (Steve Hanson) Date: Mon, 23 Sep 1991 18:51:11 -0400 (EDT) Subject: NIPS*91 UPDATE Message-ID: <0crazT21GEMnA1K3Rn@tractatus.siemens.com> NIPS*91 FINAL POSTER PROGRAM: SESSIONS I & II TUESDAY EVENING: SESSION I LEARNING METHODS AND GENERALIZATION I M. R. Sydorenko, The Johns Hopkins School of Medicine E. D. Young The Johns Hopkins School of Medicine "Analysis of Stationarity of the Strength of Synaptic Coupling Between Pairs of Neurons with Non-Stationary Discharge Properties" P. Dayan, University of Edinburgh G. Goodhill, University of Sussex "Perturbing Hebbian Rules" D. Geiger, Siemens Research R. Pereira, Universita di Trento "Selecting Minimal Surface Data" J. Shawe-Taylor, University of London "Threshold Network Learning in the Presence of Equivalences" B. Pearlmutter, Yale "Asymptotic Convergence of Gradient Descent with Second Order Momentum" P. Munro, University of Pittsburgh "Repeat until bored: a pattern selection strategy" Y. Freund University of California, Santa Cruz D. Haussler, University of California, Santa Cruz "A fast and exact learning rule for a restricted class of Boltzmann machines" IMPLEMENTATION I R.G. Benson, Caltech "Silicon ON-Cell Adaptive Photorecepetors" E. Sackinger, AT&T Bell Laboratories B.E. Boser, AT&T Bell Laboratories L.D. Jackel, AT&T Bell Laboratories "A Neurocomputer Board Based on the ANNA Neural Network Chip" P. Kohn, ICSI J. Bilmes, ICSI N. Morgan, ICSI J. Beck, ICSI "Software for ANN Training on a Ring Array Processor" VISUAL PROCESSING I G.L. Martin, MCC & Eastman Kodak "Centered-Object Integrated Segmentation and Recognition for Visual Character Recognition" S. Ahmad, ICSI "Modelling Visual Attention and Visual Search" E. Mjolsness,Yale University "Visual Grammars and their Neural Nets" M.C. Mozer, University of Colorado R.S. Zemel, University of Toronto M. Behrmann, University of Toronto "Learning to Segment Images using Dynamic Feature Binding" H. Greenspan, Caltech R.M. Goodman, Caltech R. Chellappa, University of Maryland "A Combined Neural Network and Rule Based Framework for Probabilistic Pattern Recognition and Discovery" R. Basri MIT S. Ullman MIT "Linear Operator for Object Recognition" G.E. Hinton, University of Toronto C.K.I. Williams, University of Toronto "Adaptive Elastic Models for Character Recognition" O. Matan, AT&T Bell Laboratories C.J.C. Burges, AT&TBell Laboratories Y. Le Cun, AT&TBell Laboratories J.S. Denker, AT&TBell Laboratories "Multi-Digit Recognition Using a Space-Delay Neural Network" N. Intrator, Brown University J.I. Gold, Brown University H.H. Buelthoff, Brown University S. Edelman, Weizmann Institute of Science Three-Dimensional Object Recognition Using an "Unsupervised Neural Network: Understanding the Distinguishing Feature" ARCHITECTURES AND APPROXIMATION P. Koistinen, Rolf Nevanlinna Institute L. Holmstroem, Rolf Nevanlinna Institute "Kernel Regression and Backpropagation Training With Noise" R. Williamson, Australian National University P. Bartlett, University of Queensland "Piecewise Linear Feedforward Neural Networks" C. Ji, Caltech D. PsaltisCaltech "Storage Capacity and Generalization of Two-Layer Networks With Binary Weights The VC-Dimension vs. The Statistical Capacity" Y. Zhao, MIT C.G. Atkeson, MIT "Some Approximation Properties of Projection Pursuit Learning Networks" J. Moody, Yale N. Yarvin, Yale "Networks with learned unit response functions" N.J. Redding, Electronics Research Laboratory T. Downs, University of Queensland "Networks with Nonsmooth Functions" T.D. Sanger, Massachusetts Institute of Technology R.S. Sutton GTE Laboratories Inc. C.J.Matheus, GTE Laboratories Inc. "Iterative Construction of Sparse Polynomial Approximations" M. Wynne-Jones, RSRE "Node splitting: a constructive algorithm for feed-forward NN" S. Ramachandran, Rutgers L.Y. Pratt, Rutgers "Information Measure Based Skeletonisation" TEMPORAL PROCESSING C. Koch, Caltech H. Schuster, Universitaet Kiel "A Simple Network Showing Burst Synchronization without Frequency-Locking" C. Brody, ILMAS, UNAM "Fast Learning with Predictive Forward Models" C.G. Atkeson, MIT "Robot Juggling: A Real-Time Implementation of Learning Based on Locally Weighted Regression" J.P. Sutton, Harvard Medical School A.N. Mamelak, Harvard Medical School A. Hobson, Harvard Medical School "Network Model of State-Dependent Sequencing" M.C. Mozer, University of Colorado "Connectionist Music Composition and the Induction of Multiscale Temporal Structure" J. Schmidhuber, University of Colorado "History Compression for Adaptive Sequence Chunking" PERFORMANCE COMPARISONS J. Bernasconi,Asea Brown Boveri Corp. K. Gustafson, University of Colorado "Human and Machine 'Quick Modeling'" M. Maechler, University of Washington R. D. Martin, University of Washington J. Schimert, University of Washington J.N. Hwang, University of Washington "A Comparison of Projection Pursuit and NN Regression Modeling" L.G.C. Hamey, Macquarie University "Benchmarking feed-forward Neural Networks: models and measures" APPLICATIONS I P. Stolorz, Los Alamos National Laboratory A. Lapedes, Los Alamos National Laboratory R. Farber, Los Alamos National Laboratory D. Wolf, Los Alamos National Laboratory Y. Xia, Los Alamos National Laboratory J. Bryngelson,Los Alamos National Laboratory "Prediction of Protein Structure Using Neural Nets and Information Theory" J. Connor, University of Washington L.E. Atlas, University of Washington D. Martin, University of Washington "Advantages of Recurrency and Innovations in Time Series Prediction" M.W.Goudreau, NEC Research Institute, Inc. C.L. Giles, NEC Research Institute, Inc. "Neural Network Routing for Random Multistage Interconnection Networks" J. Moody, Yale University J. Utans, Yale University "Selecting Neural Network Architectures via the Prediction Risk: Application to Corporate Bond Rating Prediction" R. Venturini, The Salk Institute W.W. Lytton, The Salk Institute T.J. Sejnowski, The Salk Institute "Neural Network Analysis of Event Related Potentials Predicts Vigilance" SIGNAL PROCESSING D. B. Schwartz, GTE Laboratories, Inc. "Making Precise Measurements with Sloppy Sensors" J. Lazzaro, University of Colorado "Temporal Adaptation in a Silicon Auditory Nerve" M.H. Cohen, The Johns Hopkins University P.O. Pouliquen, The Johns Hopkins University A.G. Andreou, The Johns Hopkins University "Analog VLSI Implementation of an Auto-Adaptive Network for Real-Time Separation of Independent Signals" R. de Ruyter van Steveninck, University Hospital Groningen W. Bialek, NEC Research Institute "Statistical Reliability of a Movement-Sensitive Neuron" B. De Vries, University of Florida J.C. Principe, University of Florida P.Guedes de Olivierra, Universidade de Aviero "Neural Signal Procesing with an Adaptive Dispersive Tapped Delay Line" PATTERN RECOGNITION A.M. Chiang, MIT Lincoln Laboratory M.L. Chuang, MIT Lincoln Laboratory J.R. LaFranchise, MIT Lincoln Laboratory "CCD Neural Network Processors for Pattern Recognition" E.N. Eskandar, National Institute of Mental Health B.J. Richmond, National Institute of Mental Health J.A. Hertz, Nordita L.M. Optican, National Eye Institute "Decoding of Neuronal Signals in Visual Pattern Recognition" B.W. Mel, Caltech "NMDA-Based Pattern Discrimination in a Modeled Cortical Neuron" O. Bernander, Caltech R. Douglas, Oxford, UK & Caltech K. Martin, Oxford, UK C. Koch, Caltech "Synaptic Background Activity Determines Spatio-Temporal Integration in Single Cells" SELF ORGANIZATION A. Zador, Yale University B.J. Clairborne, University of Texas T.H. Brown, Yale University "Nonlinear Processing in Single Hippocampal Neurons with Dendritic Hot and Cold Spots" D.Z. Anderson, University of Colorado C. Benkert, University of Colorado V. Hebler, University of Colorado J. Jang, University of Colorado D.D. Montgomery, University of Colorado "Optical Implementation of a self-organizing feature extractor" T. Bell, Vrije Universiteit Brussel "Self-Organisation in Real Neurons: Gradient Descent In Channel Space?" H.U. Bauer, Universitaet Frankfurt K. Pawelzik, Universitaet Frankfurt F. Wolf, Universitaet Frankfurt T. Geisel, Universitaet Frankfurt "A Topographic Product for the Optimization of Self-Organizing Feature Maps" WEDNESDAY EVENING: SESSION II VISUAL PROCESSING II H.P. Graf, AT&T Bell Laboratories C. Nohl, AT&T Bell Laboratories J. Ben, AT&T Bell Laboratories "Image Segmentation with Networks of Variable Scale" T. Darrell, MIT A. Pentland, MIT "The Multi-Layer Support Process" P. Cooper, Northwestern University P. Prokopowicz, Northwestern University "M. Random Fields can Bridge Levels of Abstraction" A. Shashua, M.I.T. "Illumination and 3D Object Recognition" A. Pouget, The Salk Institute S.A. Fisher, The Salk Institute T.J. Sejnowski,The Salk Institute "Hierarchical Transformation of Space in The Visual System" LANGUAGE A. Jain, CMU "Generalization Performance in PARSEC- A Structured Connectionist Parsing Architecture" G. Pinkas, Washington University "Syntactic Proofs Using Connectionist Constraint Satisfaction" P. Gupta, CMU D.S. Touretzky, CMU "A Connectionist Learning Approach to Analyzing Linguistic Stress" R.A. Sumida, University of California M.G. Dyer, University of California "Propagation Filters in PDS Networks for Sequencing and Ambiguity Resolution Ambiguity Resolution" Y. Muthusamy, Oregon Graduate Institute of Science and Technolgy R. A. Cole, Oregon Graduate Institute of Science and Technolgy "Segment-Based Automatic Language Identification System" R.L. Watrous, Siemens Research C.L. Giles, NEC Research Institute C.B. Miller NEC Research Institute G.M. Kuhn, IDA D. Chen, University of Maryland H.H. Chen, University of Maryland G.Z. Sun, University of Maryland Y.C. Lee, University of Maryland "Induction of Finite State Automata from Discrete-time Recurrent Neural Networks" SPEECH PROCESSING M. Hirayama, ATR E.V. Bateson, ATR M. Kawato, ATR M.I. Jordan MIT "Forward Dynamics Modeling of Speech Motor Control Using Physiological Data" E. Levin, AT&T Bell Laboratories R. Pieraccini, AT&T Bell Laboratories E. Bocchieri, AT&T Bell Laboratories "Time Warping Network: A Hybrid Framework for Speech Recognition" R. Cole, Oregon Graduate Institute of Science and Technology K. Roginski, Oregon Graduate Institute of Science and Technology M. Fanty, Oregon Graduate Institute of Science and Technology "English Alphabet Recognition With Telephone Speech" P. Haffner, Centre National d'Etudes des Telecommunications A. Waibel, CMU "Multi-State Time Delay Neural Networks for Continous Speech Recognition" HYBRID MODELS E. Singer, MIT Lincoln Laboratory R. P. Lippmann, MIT Lincoln Laboratory "Improved Hidden Markov Model Speech Recognition Using Radial Basis Function Networks" S. Renals, ICSI N. Morgan, ICSI H. Bourlard, L&H Speechproducts H. Franco, SRI International Mike Cohen, SRI International "Connectionist Optimisation of Tied Mixture Hidden Markov Models" Y. Bengio, McGill University R. DeMori, McGill University G. Flammia, McGill University R. Kompe, McGill University "Global Optimization of Neural Network - Hidden Markov Model Hybrid" P. Stolorz, Los Alamos National Laboratory "Merging Constrained Optimization with Deterministic Annealing to "Solve"combinatorially hard problems" CONTROL AND PLANNING T. Prescott, University of Sheffield J. Mayhew, University of Sheffield "Obstacle Avoidance through Reinforcement Learning" K. Doya, University of Tokyo S. Yoshizawa, University of Tokyo "Learning of Locomotion Patterns by Recurrent Neural Networks: How can a central pattern generator adapt itself to the physical world?" H. Gomi, ATR M. Kawato, ATR "Learning Closed-Loop Control and Recognition of Manipulated Objects" G.M. Scott, University of Wisconsin J.W. Shavlik, University of Wisconsin W. H. Ray, University of Wisconsin "Refining PID Controllers Using Neural Networks" P. Dean, University of Sheffield E.W. Mayhew, University of Sheffield "A Neural Net Model For Adaptive Control of Saccadic Accuracy By Primate Cerebellum and Brainstem" T. Anastasio, USC "Learning In the Vestibular System: Simulations Of Vestibular Compensation Using Recurrent Back-Propagation" APPLICATIONS II S. Thiria, Universite de Paris Sud C. Mejia, Universite de Paris Sud F. Badran, Universite de Paris Sud M. Creponi, LODYC "Multimodular Neural Architecture for Remote Sensing Operations" A. Manduca, Mayo Foundation P. Christy, Mayo Foundation R. Ehman, Mayo Foundation "Neural Network Diagnosis of Avascular Necrosis From Magnetic Resonance Images" S.L. Gish, IBM M. Blaum, IBM "Adaptive Development of Connectionist Decoders for Complex Error-Correcting Codes" A. C. Tsoi, University of Queensland "Application of Neural Network Methodology to the Modelling of the Yield Strength in a Steel Rolling Plate Mill" D.T. Freeman, University of Pittsburgh School of Medicine "Computer Recognition of Wave Location in Graphical Data by a Neural Network" H. Tunley, University of Sussex "A Neural Network for Motion Detection of Drift-Based Stimuli" MEMORY D. Horn, Tel Aviv University M. Usher, Weizman Institute "Oscillatory Model of Short Term Memory" K-Y. Siu, Stanford University J. Bruck, IBM Research Division "Neural Computing With Small Weights" IMPLEMENTATION D. Kirk, Caltech K. Fleischer, Caltech A. Barr, Caltech "Constrained Optimization Applied to the Parameter Setting Problem for Analog Circuits" J.G. Harris,Caltech "Segmentation Circuits Using Contrained Optimization" KINEMATICS & TRAJECTORIES M. Dornay, ATR Y. Uno,University of Tokyo M. Kawato, ATR R. Suzuki, University of Tokyo "Simulation of Optimal Movements Using the Minimum Muscle-Tension-Change Model" P. Simard, AT&T Bell Laboratories Y. LeCun, AT&T Bell Laboratories "Reverse TDNN: An Architecture for Trajectory Generation" E. Henis, The Weizmann Institute of Science T. Flash, The Weizmann Institute of Science "A Computational Mechanism to Account for Averaged Modified Hand Trajectories" D. DeMers, UC San Diego K. Kreutz-Delgado, UC San Diego "Learning Global Direct Inverse Kinematics" N. E. Berthier,University of Massachusetts S.P. Singh, University of Massachusetts A.G. Barto, University of Massachusetts J.C. Houk, Northwestern University Medical School "A Cortico-Cerebellar Model That Learns To Generate Distributed Motor Commands to Control A Kinematic Arm" ARCHITECTURES AND GENERALIZATION J. Bridle, R.S.R.E. D.J.C. MacKay, Caltech "Unsupervised Classifiers, Mutual Information And 'Phantom Targets'" T. Lange, University of California "Dynamically-Adaptive Winner-Take-All Networks" A. Krogh, The Niels Bohr Institute J.A. Hertz, Nordita "A Simple Weight Decay Can Improve Generalization" M.S. Glassman, Siemens Research "A Network of Localized Linear Discriminants" D. Montana, Bolt Beranek and Newman, Inc. "A weighted probabilistic Neural Net" S. Omohundro, International Computer Science Institute "Model-Merging for Improved Generalization" I. Grebert, Ricoh California Research Center D.G. Stork, Ricoh California Research Center R. Keesing, Ricoh California Research Center S.Mims, Stanford University "Network generalization for production: learning and producing styled letterforms" J.B. Hampshire II, Carnegie Mellon University D.A. Pomerleau, Carnegie Mellon University B.V.K. Vijaya Kumar Carnegie Mellon University "Automatic Feature Selection and Functional Capacity Modulation in MLP Classifiers via Learned Synaptic Correlations" LEARNING METHODS AND GENERALIZATION II P. Simard, AT&T Bell Laboratories B. Victorri, Universite de Caen Y. Le Cun, AT&T Bell Laboratories J. Denker, AT&T Bell Laboratories "Tangent Prop - A formalism for specifying selected invariances in an adaptive networks" G.Z. Sun, University of Maryland H.H. Chen, University of Maryland Y.C. Lee, University of Maryland "Green's Function Method for Fast On-line learning Algorithm of recurrent Method for Fast On-line learning Algorithm of recurrent NN" C. Darken, Yale University J. Moody, Yale University "Faster Stochastic Gradient Search" N.N. Schraudolph, UCSD T. Sejnowski, The Salk Institute "Competitive Anti-Hebbian Learning of Invariants" J. Wiles, University of Queensland A. Bloesch, University of Queensland "Operators and curried functions: training and analysis of simple recurrent networks" A. Bertoni, Universita degli Studi Di Milano P. Campadelli, Universita degli Studi Di Milano A. Morpurgo, Universita degli Studi Di Milano S. Panizza, Universita degli Studi Di Milano "Polynomial Uniform Convergence of Relative Frequencies to Probabilities" T. Kuh, University of Hawaii T. Petsche, Siemens Research R. Rivest, MIT "Mistake Bounds of Incremental Learners when Concepts Drift with Applications to Feedforward Networks" From jai at sol.boeing.com Tue Sep 24 18:21:21 1991 From: jai at sol.boeing.com (Jai Choi) Date: Tue, 24 Sep 91 15:21:21 PDT Subject: TR's available Message-ID: <9109242221.AA06689@sol.boeing.com> The following technical report is available. This's been submitted for FUZZ-IEEE '92, San Diego. A copy will be available by request only to Jai J. Choi Boeing Computer Services P.O.Box 24346, MS 6C-04 Seattle, WA 98124, USA or drop your surface address to "jai at sol.boeing.com". . We propose a real-time diagnostic system using a combination of neural networks and fuzzy logic. This neuro-fuzzy hybrid system utilizes real-time processing, prediction and data fusion. A layer of n trained neural networks processes $n$ independent time series (channels) which can be contaminated with environmental noise. Each network is trained to predict the future behavior of one time series. The prediction error and its rate of change from each channel are computed and sent to a fuzzy logic decision output stage, which contains $n+1$ modules. The (n+1)st final-output module performs data fusion by combining n individual fuzzy decisions that are tuned to match the domain expert's need. From uh311ae at sunmanager.lrz-muenchen.de Wed Sep 25 15:09:31 1991 From: uh311ae at sunmanager.lrz-muenchen.de (Henrik Klagges) Date: 25 Sep 91 21:09:31+0200 Subject: logarithmic/linear weights Message-ID: <9109251909.AA07867@sunmanager.lrz-muenchen.de> Where do I loose more accuracy or information, if I store the weights linear or logarithmic. So to speak, do I need 'high end' resolution or 'low end' resolution ? Cheers, Henrik From harnad at Princeton.EDU Wed Sep 25 17:27:41 1991 From: harnad at Princeton.EDU (Stevan Harnad) Date: Wed, 25 Sep 91 17:27:41 EDT Subject: Connectionits Models of Reading: Call for Commentators Message-ID: <9109252127.AA16450@clarity.Princeton.EDU> The following paper has just appeared in PSYCOLOQUY. Commentaries may be directed to psyc at pucc.bitnet or to psyc at pucc.princeton.edu All contributions will be refereed. --Stevan Harnad From harnad at clarity.princeton.edu Wed Sep 25 11:43:38 1991 From: harnad at clarity.princeton.edu (Stevan Harnad) Date: Wed, 25 Sep 91 11:43:38 EDT Subject: PSYCOLOQUY V2 #8 (2.8.4 Paper: Connectionism/Reading/Skoyles: 288 l) Message-ID: ---------------------------------------------------------------------- From: ucjtprs at ucl.ac.uk Subject: 2.8.4 Paper:Connectionism, Reading and the Limits of Cognition/Skoyles Rationale for Inviting PSYCOLOQUY Commentary: (a) The literature on reading acquisition is becoming fragmented: Here is a theory which can bring together the modellers and those collecting data. (b) I was brought up upon Thomas Kuhn, so the first things I look for are anomalies. I think I found two. What better way to bring them to other people's attention than to show that they are different aspects of the same phenomena? (c) Many literate but phonologically disabled readers exist who were delayed, often to the point of failure, in learning to read. Yet they eventually learn to read, though often only in their teens. This suggests that the processes underlying their reading competence are functionally intact but that something related to their phonological handicap stops their normal development. At present that link between phonological disability and reading failure is only understood in terms of a correlation. By proposing a mechanism I hope to persuade other researchers if not to adopt my theory then propose other causal mechanisms. This is important: Only with causal theories can scientifically based interventions to ameliorate dyslexia develop. Here are some issues and related questions which might be addressed in the PSYCOLOQUY Commentary. (Note the length of my own comments does not indicate importance.) (1) The nature of connectionist training: Much has been written about the training rules used to adjust networks )back-propagation, etc., and effects of the numbers of hidden units). Little (actually I can find none) has been written about the provision of error-correction feedback. Some questions: What happens if error-correction feedback is degraded -- if it is only sometimes correct: How does this affect learning? What happens if this feedback is delayed (one reason why it may be degraded) or its acquisition interferes with the other processes involved in network training? These questions are important because networks learn and function in the real world where error correction feedback may not be synchronised with network training. Or feedback may only be obtainable by sacrificing the efficiency of other processes, including those training the network. (2) Humphries and Evett in a target article in Behavioral and Brain Sciences argued that adult reading was nonphonetic. That was in 1985. Are they and their commentators of the same opinion now as then? (3) What is phonetic reading? It used to be defined in terms of a person's ability to read nonwords. How could nonwords be read except by sequential grapheme-phoneme translation? Connectionism shows that nonword reading can be done purely by processes trained on real words without the use of special grapheme-phoneme translation processes. I defined phonetic reading in terms of word recognition processes which depend for their recognition of words in some way upon a reader's oral vocabulary. Some commentators may want to challenge this. (4) Phonology and success at reading is a greatly debated subject. I have left the exact nature of phonetic reading unstated -- partly because I believe that the mechanisms involved may vary over time and between individuals (although all share a common dependence upon access to oral vocabulary). This may be controversial, however. (5) Dyslexic and phonology -- there is a link, but its nature is still unresolved. Some commentators might want to question this link (e.g., Bruno Breitmeyer and William Lovegrove). (6) Reading research has had little effect on educators. There is the "whole word method" movement, which does not ascribe any importance to phonology. Representatives of various educational methods might want to give their views. (7) Cognitive development rarely discusses the role of error-correction feedback upon cognitive development (although it would hardly be an original idea to suggest it is important). In reading development we have an example of how important it is. I suggest it is not unique. Commentators might wish to suggest other examples (both involving endogenous and external sources of feedback). (8) Dyslexics might like to contribute. I am (or rather was) dyslexic. I would be interested to contact other dyslexic psychologists. Connectionism, Reading and the Limits of Cognition John R. Skoyles, Department of Psychology, University College London, London WC1E 6BT, UK. Abstract You read these written words without using your knowledge of how they sound. A children's ability to do this predicts their success in learning to read. Connectionist (PDP) models of cognitive processes using networks can explain why this is so. Connectionism successfully simulates "nonphonetic" reading skills. Networks, however, learn by error-correction feedback -- but where does the learner-reader get this feedback? I suggest that it is from the phonetic identification of words. This conjecture might help explain (1) the pattern of reading development in children and (2) dyslexia as well as (3) raising questions for research aimed at making learning to read easier. In addition, I suggest that error-correction feedback may have an important and general role as a "bottle-neck" in the development of cognition. Keywords: dyslexia, connectionism, development, error correction, reading. How did you learn to read these words? Recent reading research has thrown up two anomalies in the acquisition of word learning. I propose a theory which links them. It provides a new way of looking at the development of reading and cognition. Anomaly 1: You read these words nonphonetically, that is, you identify them without using your oral knowledge of how words sound (Humphrey and Evett, 1985). But the best predictors of children's success in learning to read are language related skills such as initial phonetic awareness (Stanovitch, Cunningham, & Feeman, 1984) and phonetic instruction (Bradley and Bryant, 1983) - skills which are needed for phonetic reading. Why should skills needed for phonetic reading predict later success in nonphonetic reading? Anomaly 2: Connectionist (PDP) neural network simulations of reading successfully explain many experimental facts found about word recognition (Seidenberg & McClelland, 1989). Like adult reading, these simulations are nonphonetic -- they lack any connection with our oral knowledge about how words sound. But there is a problem with their success: although they are advocated as models of word learning, they can, paradoxically, only learn to recognise words if the word learner can already read. The problem originates in the need for networks to be tutored with error correction feedback. Reading networks learn by having their inner nodes adjusted after the network has read a word. This adjustment (also called training) depends upon whether the network has read a word correctly or not: The network's nodes are adjusted differently depending on whether or not it correctly identifies a word. This error correction, however, puts an element of circularity at the heart of network word learning, for it makes successful word learning depend on the learner's already being a good reader (otherwise the processes training the network cannot know whether or not the network has read a word correctly). This condition in which successful word learning depends on pre-existing reading does correctly describe learning readers, however -- the children who learn to read most easily are those who are good readers already -- if we appreciate that the phonetic identification of words is a form of reading. The first anomaly raises the question of why phonetic reading predicts success in nonphonetic reading. The second anomaly answers it: I conjecture that initial phonetic reading skills are needed to train nonphonetic skills. The better their phonetic skills, the easier learning readers find it to provide error correction feedback (by recognising words phonetically). They are accordingly in a better position to train their developing reading networks and thereby learn to read nonphonetically. I suggest that reading development is a double process, involving a trade off between the different advantages and disadvantages of phonetic reading and nonphonetic reading. The underlying story, I propose, is this: The phonetic identification of words is inefficient for normal reading, both in terms of speed and its use of mental attention (because it depends upon the use of oral vocabulary). In contrast, the nonphonetic recognition of words is suited for the demands of reading as it is quick and undemanding cognitively (an important requirement if the mind is to focus upon what it has read rather than recognising words). However, nonphonetic reading is difficult to acquire because it needs to be trained with error correction feedback. The reading development overcomes this limitation by the use of the less efficient phonetic identification of words to provide this feedback; afterwards, the phonetic identification of words is dispensed with. This suggestion explains other facets of the acquisition of reading skills. It fits the pattern of normal child reading development: Children go through a period of phonetic reading before progressing to nonphonetic reading (Frith, 1985). I suggest that this is due to the advantage mentioned above for the development of nonphonetic reading derived from prior skills in recognising words phonetically. This suggestion may also explain the link between difficulties in recognising words phonetically and dyslexia (Snowling, 1987). The majority of dyslexics have impaired phonological skills (particularly, but not exclusively, phonologically segmenting words), but they seem to have intact nonphonological processes. Indeed, given time, some of them (Campbell & Butterworth, 1985) persevere at learning words and become normal readers -- presumably because they use less efficient means than phonology to training their nonphonetic reading abilities. The notion that phonology may play a role in word learning is not new (e.g., see Jorm and Share, 1983). What is novel, however, is the link between phonology and the network training. This is unexpected. It also allows some important questions to be raised. There must be an optimal time window for error correction to aid network training. How long is this? There must also be a limit on how cognitively demanding obtaining error correction feedback can be before it interferes with training and the reading process, but how demanding? It may be that the importance of phonetic reading is not just that it provides error correction feedback but that it provides this information effectively and nondisruptively when word training processes need it. If these questions can be answered then this may help in the development of better methods for helping people with word learning problems. This model of word learning has wider importance. Word learning in reading may not be unique. Error correction feedback -- the need for learners to know whether or not they have performed the skill they are learning correctly -- may be a general bottle-neck limiting cognitive growth. No one has looked at cognitive development from this perspective, so its importance is largely unknown, but it could explain why cognitive development tends to go in stages: Spurts of cognitive growth could be due to the development of new strategies and means for overcoming problems in obtaining this information. References. Bradley, L., & Bryant, P. E. (1983). Categorisation of sounds and learning to read: A causal connection. Nature 301: 419-421. Campbell, R. & Butterworth, B. (1985). Phonological dyslexia and dysgraphia in a highly literate subject: A development case with associated deficits of phonemic processing and awareness, Quarterly Journal of Experiment Psychology 37A: 435 - 475. Frith, U. (1985). Beneath the surface of developmental dyslexia. In K. E. Patterson, J. C. Marshall, & M. Coltheart (Eds.), Surface dyslexia. London: Routledge and Kegan Paul. Humphries, G. W., & Evett, L. L. (1985). Are there independent lexical and nonlexical routes in word processing? An evaluation of the dual-route model of reading. Behavioral and Brain Sciences 8: 689-740. Joam, A. F. & Share, D. L. (1983). Phonological recoding and reading acquisition. Applied Psycholinguistics 4: 103- 147. Seidenberg, M. S. and McClelland, J. I. (1989). A distributed, developmental model of word recognition and naming. Psychological Review 96: 523-568. Skoyles, J. R. (1988). Training the brain using neural-network models. Nature 333: 401. Snowling, M. (1987). Dyslexia: A cognitive developmental perspective. Oxford: Basil Blackwell. Stanovitch, K. E., Cunningham, A. F., & Feeman, D. J. (1984). Intelligence, cognitive skills and early reading progress. Reading Research Quarterly 19: 278-303. ------------------------------ PSYCOLOQUY is sponsored by the Science Directorate of the American Psychological Association (202) 955-7653 Co-Editors: (scientific discussion) (professional/clinical discussion) Stevan Harnad Perry London, Dean, Cary Cherniss (Assoc Ed.) Psychology Department Graduate School of Applied Graduate School of Applied Princeton University and Professional Psychology and Professional Psychology Rutgers University Rutgers University Assistant Editor: Malcolm Bauer Psychology Department Princeton University End of PSYCOLOQUY Digest ****************************** From kak at max.ee.lsu.edu Wed Sep 25 12:43:11 1991 From: kak at max.ee.lsu.edu (Dr. S. Kak) Date: Wed, 25 Sep 91 11:43:11 CDT Subject: No subject Message-ID: <9109251643.AA03185@max.ee.lsu.edu> Call for Papers Special Issue of INFORMATION SCIENCES Topic: Neural Networks and Artificial Intelligence The motivation for this Special Issue, which will be published in 1992, is to evaluate the advances in the field of neural networks in perspective. Critics have charged that engineering neural computing has turned out to be no more than a new technique of clustering data. From A.G.Hofland at newcastle.ac.uk Wed Sep 25 18:04:31 1991 From: A.G.Hofland at newcastle.ac.uk (Anton Hofland) Date: Wed, 25 Sep 91 18:04:31 BST Subject: Announcement Follow Up Message-ID: Announcement Follow Up Forgot the extensions of Prof. Morris and Dr. Montague. They are on ext. 7342 and ext 7265 respectively. Anton Hofland. From karunani at CS.ColoState.EDU Thu Sep 26 01:09:05 1991 From: karunani at CS.ColoState.EDU (n karunanithi) Date: Wed, 25 Sep 91 23:09:05 MDT Subject: No subject Message-ID: <9109260509.AA01806@zappa> To those who use connectionist networks for sequential prediction ------------------------------------------------------------------ applications. ------------ Background: ----------- I have been using neural network models (both Feed-Forward Nets and Recurrent Nets) in a prediction application and I am getting pretty good results. In fact neural networks approach outperformed many well known analytic models. Similar results have been reported by many researchers in (chaotic) time series predictions. Suppose that X is the independent variable and Y is the dependent variable. Let (x(i),y(i)) represent a sequence of actual input/output values observed at time i = 0,1,2,..,t of a temporal process. Let further that both the input and the output variables are single dimensional variable and can take on a sequence of +ve integers up to a maximum of 2000. Once we train a network with the history of the system up to time "t" we can use the network to predict outputs y(t+h), h=1,..,n for any future input x(t+h). In my application I already have the complete sequence and hence I know what is the maximum value for x and y. Using these maximum I normalized both X and Y over a 0.1 to 0.9 range. (Here I call such normalization as "scaled representation".) Since I have the complete sequence it is possible for me to evaluate how good the networks' predictions are. Now some basic issues: --------------------- 1) How to represent these variables if we don't know in advance what the maximum values are? Scaled representation presupposes the existence of a maximum value. Some may suggest that a linear units can be used at the output layer to get rid of scaling. If so how do I represent the input variable? The standard sigmoidal unit(with temp = 1.0) gets saturated(or railed to 1.0) when the sum is >= 14. However one may suggest that changing the output range of the sigmoidal can help to get rid of saturation effect. Is it a correct approach? 2) In such prediction application, people (including me) compare the predictive accuracy of neural networks with that of parametric models(that are based on analytical reasons). But one main advantage with the parametric models is that their parameters can be calculated using any of the following parameter estimation techniques: least square, maximum likelyhood, Bayesian, Genetic Algorithms or any other method. These parameter estimation techniques do not require any scaling, and hence there is no need for preguessing of the maximum values. However with the scaled representation in neural networks one can not proceed without making guesses about the maximum(or a future) input and/or output. In many real life situations such guesses are infeasible or dangerous. How do we address this situation? ____________________________________________________________________________ N. KARUNANITHI E-Mail: karunani at handel.CS.ColoState.EDU Computer Science Dept, Colorado State University, Collins, CO 80523. ____________________________________________________________________________ From A.G.Hofland at newcastle.ac.uk Wed Sep 25 17:59:07 1991 From: A.G.Hofland at newcastle.ac.uk (Anton Hofland) Date: Wed, 25 Sep 91 17:59:07 BST Subject: Symposium Announcement Message-ID: NEURAL NETWORKS AND ENGINEERING APPLICATIONS International Symposium 23/24 October 1991 Background The annual symposium on aspects of advanced process control, organised in collaboration with major engineering companies, has in the past been highly successful. This year the fifth meeting will continue this tradition when it takes as its theme Neural Networks and Engineering Applications. Learning systems of all kinds are presently being studied for potential application to a wide variety of industrial situations. Artificial Neural Networks, in particular, are one of the fastest growing paradigms. The reason for the significant industrial interest in neural networks is that they hold great promise for solving problems that have proved to be extremely difficult for standard techniques. In fact some industries, especially in the USA and Japan, have already transferred the technology through to successful plant applications. In many industrial situations, mathematical models of the processes are either too complex for companies to develop or not accurate enough to be used for control or optimisation. Neural networks can enable rapid and accurate model development and thus play a significant role in improving the operation and control of a wide range of strategically important processes. Indeed there is a potential for their application in virtually every operation in industry. The underlying theme of the Symposium, organised by the University of Newcastle and the International Neural Networks Industrial Club, will be to look at the methodology of artificial neural networks and their applications. Industrial studies will be presented with the aim of identifying opportunities for cross- fertilisation of ideas and technology between industrial sectors and academic research. The speakers are technical authorities in their area and are drawn from international industry and university research centres. They will all address issues associated with the technology and the potential applications. Poster and demonstration sessions will also be mounted. Ample time will be provided for discussions adding to the spectrum of experience of the delegates. In order to facilitate fruitful discussion periods the number of delegates will be restricted to 75. The symposium will be held at the Imperial Hotel in Newcastle upon Tyne, UK. It will start at 1 pm. on the 23rd October with registration from 12 noon. The Lord Mayor of Newcastle will host a civic reception and dinner on the evening of the 23rd starting at 7.30 pm. The meeting will re-convene on the 24th at 8.30 am and finish at 4.30 pm. PROGRAMME 1) Artificial Neural Networks for Process Control. Prof. Julian Morris, University of Newcastle UK. 2) Network Training Paradigms. Dr. Mark Willis, University of Newcastle, UK. 3) Neural Networks in Process Control. Dr. Tom McAvoy, University of Maryland, USA. 4) Autoassociative and Radial Basis Function Networks. Dr. Mark Kramer, MIT, USA. 5) Fault Detection via Artificial Neural Networks. Dr. David Himmelblau, University of Texas, USA. 6) Neural Networks in Industrial Modelling and Control. Dr. Mike Piovoso, DuPont, USA. 7) Neural Networks in Process Engineering. Dr. Gary Montague, University of Newcastle, UK. 8) Time Series Forecasting using Neural Networks. Mr. Chris Gent, SD Scicon, UK. 9) Neural Network Tools. Dr. Alan Hall, Scientific Computers Ltd., UK. REGISTRATION Registration will close at the 17th October. More information and/or registration forms can be obtained by contacting : Prof. A. Julian Morris Department of Chemical and Process Engineering Merz Court Claremont Road Newcastle upon Tyne NE1 7RU Telephone : +44-91-2226000 ext. ???? Fax : +44-91-2611182 e-mail : julian.morris at newcastle.ac.uk or Dr. Gary Montague Department of Chemical and Process Engineering Merz Court Claremont Road Newcastle upon Tyne NE1 7RU Telephone : +44-91-2226000 ext. ???? Fax : +44-91-2611182 e-mail : gary.montague at newcastle.ac.uk From yoshio at tuna.cis.upenn.edu Fri Sep 27 00:59:19 1991 From: yoshio at tuna.cis.upenn.edu (Yamamoto Yoshio) Date: Fri, 27 Sep 91 00:59:19 EDT Subject: No subject Message-ID: <9109270459.AA14476@tuna.cis.upenn.edu> I am looking for information about Kawato's recent works. Recently I've read his articles about the applications in robot control, one from Neural Networks (1988), another from a book, "Neural Networks for Control", and a couple of others from Biological Cybernetics (1987,1988). Therefore I am particularly interested in his works after 1988 to present. Any pointer will be greatly appreciated. I am also interested in getting information about other people's works on similar subjects beside Grossberg and Kuperstein's work and Miller et. al.'s CMAC type approach. Thanks. - Yoshio Yamamoto GRASP Lab (General Robotics And Sensory Perception Laboratory) University of Pennsilvania From bernard at arti1.vub.ac.be Thu Sep 26 11:30:18 1991 From: bernard at arti1.vub.ac.be (Bernard Manderick) Date: Thu, 26 Sep 91 17:30:18 +0200 Subject: PPSN-92: Call for Papers Message-ID: <9109261530.AA21742@arti1.vub.ac.be> Dear moderator, Can you put this Call for the PPSN-conference in the next number of your electronic magazine? This call is followed by the Latex version so that people may print it out if they want. I apologize for any inconvenience. Many thanks in advance, Bernard Manderick AI Lab VUB Pleinlaan 2 B-1050 Brussel tel.: +32/2/641.35.75 email: bernard at arti.vub.ac.be ------------------------------------------------------------------------------- CUT HERE FOR THE CALL ------------------------------------------------------------------------------- Call for Papers PPSN 92 Parallel Problem Solving from Nature Free University Brussels, Belgium 28-30 September 1992 The unifying theme of the PPSN-conference is ``natural computation'', i.e. the design, the theoretical and empirical understanding, and the comparison of algorithms gleaned from nature as well as their application to real-world problems in science, technology, etc. Characteristic for natural computation is the metaphorical use of concepts, principles, and mechanisms explaining natural systems. Examples are genetic algorithms, evolution strategies, algorithms based on neural networks, immune networks, and so on. A first focus of the conference is on problem solving in general, and learning and adaptiveness in particular. Since natural systems usually operate in a massively parallel way, a second focus is on parallel algorithms and their implementations. The conference scope includes but is not limited to the following topics: Physical metaphors such as simulated annealing, Biological metaphors such as evolution strategies, genetic algorithms, immune networks, classifier systems and neural networks insofar problem solving, learning and adaptability are concerned, and Transfer of other natural metaphors to artificial problem solving. Objectives of this conference are 1) to bring together scientists and practitioners working with these algorithms, 2) to discuss theoretical and empirical results, 3) to compare these algorithms, 4) to discuss various implementations on different parallel computer architectures, 5) to discuss applications in science, technology, administration, etc., and 6) to summarize the state of the art. For practical reasons, there will be both oral and poster presentations. The way of presentation of a paper does not say anything about its quality. Conference Chair: B. Manderick (VUB, Belgium) and H. Bersini (ULB, Belgium) Conference Address: PPSN - p/a D. Roggen - Dienst WEIN - Vrije Universiteit Brussel - Pleinlaan 2 - B-1050 Brussels - Belgium tel. +32/2/641.35.75 fax +32/2/641.28.70 email ppsn at arti.vub.ac.be Organizing Committee: D. Keymeulen, D. Roggen, P. Spiessens, J. Toreele (all VUB) Program Co-chairs: Y. Davidor (Israel) and H.-P. Schwefel (Germany) Program Committee: E.M.L. Aarts (The Netherlands) R.K. Belew (USA) K.A. de Jong (USA) J. Decuyper (Belgium) M. Dorigo (Italy) D.E. Goldberg (USA) M. Gorges-Schleuter (Germany) J.J. Grefenstette (USA) A.W.J. Kolen (The Netherlands) R. Maenner (Germany) W. Ebeling (Germany) J.-A. Meyer (France) H. Muehlenbein (Germany) F. Varela (France) H.-M. Voigt (Germany) Important Dates: April 1, 1992: Submission of papers (four copies) not exceeding 5000 words to be sent to the conference address. May 15, 1992: Notification of acceptance or rejection. June 15, 1992: Camera ready revised versions due. Sept. 28-30, 1992: PPSN-Conference. The proceedings will be published by Elsevier Publishing Company and will be available at the time of the conference. %----------------------------------------------------------------------------- % CUT HERE TO PRINT OUT THE CALL FOR PAPERS %------------------------------------------------------------------------------ % % call.tex Ma, 05.Sep.'91 LaTeX 2.09 % % call-for-papers to PPSN II in 1992 in Brussels, Belgium % \documentstyle{article} \pagestyle{empty} \makeatletter \def\deadline#1#2{% \par\@hangfrom{\hbox to 9em{\it #1\/\hfil}}#2\par } \def\committee#1#2{% \par\@hangfrom{\hbox to 15.5em{\bf #1\hfil}}#2\par } \makeatother \iffalse \topmargin -2.5cm \textwidth 15cm \textheight 25.5cm \else % % dinA4.sty Ho, 14.Dec.'89 LaTeX 2.09 % % LaTeX style modifications to change the paper size to DIN A4 \setlength{\textheight}{8.9in} % 22.6cm \setlength{\textwidth}{6.5in} % 16.5cm \setlength{\headheight}{12pt} % max.possible line heigth \setlength{\headsep}{25pt} \setlength{\footheight}{12pt} \setlength{\footskip}{25pt} \setlength{\oddsidemargin}{0.10in} % + 1in \setlength{\evensidemargin}{0.10in} % + 1in \setlength{\marginparwidth}{0.08in} \setlength{\marginparsep}{0.001in} % 0.1in + 0.08in + 0.001in = 3.0cm \setlength{\marginparpush}{0.4\parindent} \setlength{\topmargin}{-0.54cm} % 1in - 0.54cm = 2.0cm \setlength{\columnsep}{10pt} \setlength{\columnseprule}{0pt} % EOF dinA4.sty \fi \makeatletter % % myList.sty Ho, 14.Feb.'91 LaTeX 2.09 % % private variants of standard LaTeX lists %{ % % \begin{myList}{listname}{topsep}{parsep}{itemsep} % % \item ... % % \end{myList} % % base construct to realize re-defined standard lists % the seperation measures may be also given as \default. % \newenvironment{myList}[4]{% \edef\@myList{#1}% % \edef\@@myList{#2}% \ifx\@@myList\relax \else \setlength{\topsep}{\@@myList} \fi % \edef\@@myList{#3}% \ifx\@@myList\relax \else \setlength{\parsep}{\@@myList} \fi % \edef\@@myList{#4}% \ifx\@@myList\relax \else \setlength{\itemsep}{\@@myList} \fi % \begin{\@myList} }% {% \end{\@myList} }% \newcommand{\default}{} % % \begin{Itemize} % % \item ... % % \end{Itemize} % % single spaced, itemized list % \newenvironment{Itemize}{% \begin{myList}{itemize}{\parskip}{0pt}{0pt} }% {% \end{myList} }% % % \begin{Enumerate} % % \item ... % % \end{Enumerate} % % single spaced, enumerated list % \newenvironment{Enumerate}{% \begin{myList}{enumerate}{\parskip}{0pt}{0pt} }% {% \end{myList} }% % % \begin{enumerate} % % \item ... % % \end{enumerate} % % own enumeration style (1) ... (2) ... % which is a modification of LaTeX's enumerate. % \def\enumerate{% \def\labelEnumi{(\theenumi)}% \def\labelEnumii{(\theenumii)}% \def\labelEnumiii{(\theenumiii)}% \def\labelEnumiv{(\theenumiv)}% % \ifnum \@enumdepth > 3 \@toodeep \else \advance\@enumdepth \@ne \edef\@enumctr{\romannumeral\the\@enumdepth}% \list{% \csname labelEnum\@enumctr\endcsname }{% \usecounter{enum\@enumctr}% \def\makelabel##1{\hss\llap{##1}}% }% \fi } \def\endenumerate{% \endlist \@ignoretrue } % EOF myList.sty %} \makeatother \parskip=0.5\baselineskip \parindent=0pt \begin{document} {\large\em\centerline{Call for Papers}} \bigskip {\large\bf\centerline{PPSN~92}} {\large\bf\centerline{Parallel Problem Solving from Nature}} {\large\bf\centerline{Free University Brussels, Belgium}} {\large\bf\centerline{28--30 September 1992}} \bigskip \normalsize The unifying theme of the PPSN-conference is ``natural computation'', i.e. the design, the theoretical and empirical understanding, and the comparison of algorithms gleaned from nature as well as their application to real-world problems in science, technology, etc. Characteristic for natural computation is the metaphorical use of concepts, principles, and mechanisms explaining natural systems. Examples are genetic algorithms, evolution strategies, algorithms based on neural networks, immune networks, and so on. A first focus of the conference is on problem solving in general, and learning and adaptiveness in particular. Since natural systems usually operate in a massively parallel way, a second focus is on parallel algorithms and their implementations. The conference {\em scope\/} includes but is not limited to the following topics: \begin{Itemize} \item Physical metaphors such as simulated annealing, \item Biological metaphors such as evolution strategies, genetic algorithms, immune networks, classifier systems and neural networks insofar problem solving, learning and adaptability are concerned, and \item Transfer of other natural metaphors to artificial problem solving. \end{Itemize} {\em Objectives\/} of this conference are 1)~to bring together scientists and practitioners working with these algorithms, 2)~to discuss theoretical and empirical results, 3)~to compare these algorithms, 4)~to discuss various implementations on different parallel computer architectures, 5)~to discuss applications in science, technology, administration, etc., and 6)~to summarize the state of the art. For practical reasons, there will be both oral and poster presentations. The way of presentation of a paper does not say anything about its quality. \medskip \committee{Conference Chair:}% {B. Manderick (VUB, Belgium) and H. Bersini (ULB, Belgium)} \committee{Conference Address:}% {PPSN - p/a D. Roggen - Dienst WEIN - Vrije Universiteit Brussel - Pleinlaan 2 -- B-1050 Brussels -- Belgium -- {\bf tel.} +32/2/641.35.75 -- {\bf fax} +32/2/641.28.70 -- {\bf email} ppsn at arti.vub.ac.be} \committee{Organizing Committee:}% {D. Keymeulen, D. Roggen, P. Spiessens, J. Toreele (all VUB)} \committee{Program Co-chairpersons:}% {Y. Davidor (Israel) and H.-P. Schwefel (Germany)} {\bf Program Committee:} \\ \begin{tabular}{@{}lll} E.M.L. Aarts (The Netherlands) & R.K. Belew (USA) & K.A. de Jong (USA) \\ J. Decuyper (Belgium) & M. Dorigo (Italy) & D.E. Goldberg (USA) \\ M. Gorges-Schleuter (Germany) & J.J. Grefenstette (USA) & A.W.J. Kolen (The Netherlands) \\ R. M\"{a}nner (Germany) & W. Ebeling (Germany) & J.-A. Meyer (France) \\ H. M\"{u}hlenbein (Germany) & F. Varela (France) & H.-M. Voigt (Germany) \\ \end{tabular} \medskip \begingroup \deadline{\bf Important Dates:}{} \parskip=0pt \deadline{April 1, 1992:}{Submission of papers (four copies) not exceeding 5000 words to be sent to the conference address.} \deadline{May 15, 1992:}{Notification of acceptance or rejection.} \deadline{June 15, 1992:}{Camera ready revised versions due.} \deadline{Sept.~28-30, 1992:}{PPSN-Conference.} \endgroup The proceedings will be published by Elsevier Publishing Company and will be available at the time of the conference. \end{document} From ashley at spectrum.cs.unsw.oz.au Fri Sep 27 12:51:58 1991 From: ashley at spectrum.cs.unsw.oz.au (Ashley Aitken) Date: Fri, 27 Sep 91 11:51:58 EST Subject: Biological Threshold Function Message-ID: <9109270152.2118@munnari.oz.au> G'day, In browsing literature to help construct a simple information processing model of (biological) neurons I have seen little discussion of the threshold function - that is, how the threshold (membrane potential) of a neuron changes. I understand that there is a refractory period after each particular action potential pulse (which effectively represents a change in the threshold value, and hence limits the burst frequency) and their is a period of super-excitability after this. However, I am interested in changes on a larger timescale. Is there local change in the threshold value (c.f. the biases in artificial neural networks) on a larger timescale ? Is there any global modulation of the threshold values of groups of neurons (say in an area of the cortex) ? If anyone has any comments or references relevant to these questions I would be most grateful if they could email them to me - I will summarize if there is interest and a good response. Regards, Ashley Aitken. -- E-MAIL : ashley at spectrum.cs.unsw.oz.au AARNet SNAIL : University of New South Wales PO Box 1 Kensington NSW 2033 Australia "... any day above ground is a good day ..." J. Reyne? From btan at bluering.cowan.edu.au Fri Sep 27 03:34:56 1991 From: btan at bluering.cowan.edu.au (btan@bluering.cowan.edu.au) Date: Fri, 27 Sep 91 15:34:56 +0800 Subject: Add me messages Message-ID: <9109270734.AA12508@bluering.cowan.edu.au> Dear Sir/Madam My name is Stanley Tan. I am interested to be part of your connection. I am calling from Cowan University. My e_mail address is btan at bluering@cowan.edu.au Please add me messages, many thanks With regards Stanley Tah Australia From smagt at fwi.uva.nl Fri Sep 27 10:12:10 1991 From: smagt at fwi.uva.nl (Patrick van der Smagt) Date: Fri, 27 Sep 91 16:12:10 +0200 Subject: neural robotics motor control Message-ID: <9109271412.AA12259@fwi.uva.nl> Yamamoto Yoshio writes: >I am also interested in getting >information about other people's works on similar subjects beside Grossberg >and Kuperstein's work and Miller et. al.'s CMAC type approach. I have recently published an article on neural robotic control: P. P. van der Smagt & B. J. A. Kr\"ose, `A real-time learning neural robot controller,' Proceedings of the 1991 International Conference on Artificial Neural Networks, Espoo, Finland, 24-28 June, 1991, pp. 351--356. It describes a fast-learning self-supervised robot controller which remains adaptive during operation. Patrick van der Smagt From koch%CitJulie.Bitnet at BITNET.CC.CMU.EDU Fri Sep 27 17:27:32 1991 From: koch%CitJulie.Bitnet at BITNET.CC.CMU.EDU (koch%CitJulie.Bitnet@BITNET.CC.CMU.EDU) Date: Fri, 27 Sep 91 14:27:32 PDT Subject: Threshold changes Message-ID: <910927142732.2080c452@Juliet.Caltech.Edu> Yes, there exists good evidence in the PNS and CNS that various neuro- modulatotion--- can have similar effects. Since individual axons from neurons .quit From tackett at ipla00.dnet.hac.com Mon Sep 30 09:14:16 1991 From: tackett at ipla00.dnet.hac.com (Walter Tackett) Date: Mon, 30 Sep 91 09:14:16 EDT Subject: No subject Message-ID: <9109301614.AA27054@ipla00.ipl.hac.com> Does anyone have a reasonably complete list of papers on conjugate gradient methods for training multilayer perceptrons? If so, i'd appreciate it. From attou at ICSI.Berkeley.EDU Mon Sep 30 13:53:56 1991 From: attou at ICSI.Berkeley.EDU (abdelghani attou) Date: Mon, 30 Sep 91 11:53:56 MDT Subject: neural robotics motor control In-Reply-To: Your message of "Fri, 27 Sep 91 16:12:10 +0100." <9109271412.AA12259@fwi.uva.nl> Message-ID: <9109301853.AA06596@icsib45.berkeley.edu.Berkeley.EDU> hello well i got your email regarding the paper on neural robotic control , is it possible that you can send me a copy . thanks From jb%s1.gov at CARNEGIE.BITNET Fri Sep 20 19:11:50 1991 From: jb%s1.gov at CARNEGIE.BITNET (jb%s1.gov@CARNEGIE.BITNET) Date: Fri, 20 Sep 91 16:11:50 PDT Subject: Could some one tell me ... ? Message-ID: <9109202311.AA08365@perseus.s1.gov> The property (2) is called detailed balance resulting in a Gibbs distribution for the probability to find the system in a particular state. The rule (1) is an update procedure for the spin Sk which ensure detailed balance provided that E is an energy. Both principles are fundamental facts of statistical mechanics of neural networks (or if you prefer result from an maximum entropy analysis of neural nets). The book by Hertz Krogh and Palmer summerizes all that in a nice way. The book title is "Introduction to Neural Computation". Also consult D. Amit's book "Modelling brain functions" from Cambridge University Press. Joachim M. Buhmann University of California Lawrence Livermore National Laboratory 7000 East Ave., P.O.Box 808, L-270 Livermore, California 94550 email address: jb at s1.gov From ray%cs.su.oz.au at CARNEGIE.BITNET Mon Sep 23 07:13:11 1991 From: ray%cs.su.oz.au at CARNEGIE.BITNET (ray%cs.su.oz.au@CARNEGIE.BITNET) Date: Mon, 23 Sep 1991 21:13:11 +1000 Subject: Could some one tell me ... ? Message-ID: <01GAWVDE399CD7PXGF@BITNET.CC.CMU.EDU> I'll try to give a brief, intuitive, answer. For what might be a more formal and satisfying answer, try: van Laarhoven, P and Aarts, E "Simulated Annealing: Theory and Applications" Reidel Publishing (and Kluwer), 1987. Aarts, E, and Korst, J "Simulated Annealing and Boltzmann Machines", Wiley, 1989. > 1.What is the reason that the states of the k th unit Sk = 1 with > probability > 1 > Pk = ------------------- > 1+exp(-delta Ek/T) > > where delta Ek is the energy gap between the 1 and 0 states of the > k th unit, T is a parameter which acts like the temperature Its really only normal simulated annealing, in a slightly different form. Suppose we explicitly used traditional simulated annealing instead to determine the state of a unit. The energy of the s=0 state is E0=0, and the energy of the s=1 state is E1=Ek. Suppose also, for simplicity, that Ek>0. Now, since we must be in one state or the other: P(s=0,n) + P(s=1,n) = 1 Where s=x indicates the state of the unit, and n is the iteration number. At thermal equilibrium: P(s=0,n) = P(s=0,n-1) P(s=1,n) = P(s=1,n-1) So we will drop the second component to the function, and simply write P(s=0) and P(s=1). You can only transfer from s=0 to s=1, and vice versa. Let the probabilty of those transitions (given that you are already in the initial state) be designated P(0->1) and P(1->0) respectively. Since Ek>0, if we find ourself in the s=1 state, we will always immediately drop back to the s=0 state i.e. P(1->0) = 1 If we're in the s=0 state, then: P(0->1) = exp(-Ek/T) i.e. the normal simulated annealing function. At equilibrium, P(s=1) equals the probability of being in state s=0 at the previous iteration , and then making a successful transition to s=1 i.e.: P(s=1) = P(s=0) * P(0->1) = (1 - P(s=1)) * exp(-delta Ek/T) Rearranging the above equation gives - I think! - the Pk expression normally used for the Boltzmann Machine. > 2.Why this local decision rule ensures that in thermal equilibrium the > relative probability of two global states is determined by their > energy difference,and follows a Boltzmann distribution: > > Pa > ---- = exp(-(Ea - Eb))/T > Pb > > where Pa is the probability of being in the a th global state and > Ea is the energy of that state. It follows from noting that exp(x) * exp(y) = exp(x+y). Or, putting it another way P(A->B)*P(B->C) = P(A->C). > and how do we know that whether the system reaches thermal equilibrium > or not? There are formal ways of showing it. In practise, they're no help. The answer is "Run the machine for a very long time." In a lot of cases, "a very long time" is approximately equal to infinity, and this accounts for the mediocre performance of the Boltzmann Machine. From Connectionists-Request at CS.CMU.EDU Sun Sep 1 00:05:14 1991 From: Connectionists-Request at CS.CMU.EDU (Connectionists-Request@CS.CMU.EDU) Date: Sun, 01 Sep 91 00:05:14 -0400 Subject: Bi-monthly Reminder Message-ID: <8717.683697914@B.GP.CS.CMU.EDU> *** DO NOT FORWARD TO ANY OTHER LISTS *** This is an automatically posted bi-monthly reminder about how the CONNECTIONISTS list works and how to access various online resources. CONNECTIONISTS is not an edited forum like the Neuron Digest, or a free-for-all newsgroup like comp.ai.neural-nets. It's somewhere in between, relying on the self-restraint of its subscribers. Membership in CONNECTIONISTS is restricted to persons actively involved in neural net research. The following posting guidelines are designed to reduce the amount of irrelevant messages sent to the list. Before you post, please remember that this list is distributed to over a thousand busy people who don't want their time wasted on trivia. Also, many subscribers pay cash for each kbyte; they shouldn't be forced to pay for junk mail. Happy hacking. -- Dave Touretzky & Scott Crowder --------------------------------------------------------------------- What to post to CONNECTIONISTS ------------------------------ - The list is primarily intended to support the discussion of technical issues relating to neural computation. - We encourage people to post the abstracts of their latest papers and tech reports. - Conferences and workshops may be announced on this list AT MOST twice: once to send out a call for papers, and once to remind non-authors about the registration deadline. A flood of repetitive announcements about the same conference is not welcome here. - Requests for ADDITIONAL references. This has been a particularly sensitive subject lately. Please try to (a) demonstrate that you have already pursued the quick, obvious routes to finding the information you desire, and (b) give people something back in return for bothering them. The easiest way to do both these things is to FIRST do the library work to find the basic references, then POST these as part of your query. Here's an example: WRONG WAY: "Can someone please mail me all references to cascade correlation?" RIGHT WAY: "I'm looking for references to work on cascade correlation. I've already read Fahlman's paper in NIPS 2, his NIPS 3 abstract, and found the code in the nn-bench archive. Is anyone aware of additional work with this algorithm? I'll summarize and post results to the list." - Announcements of job openings related to neural computation. - Short reviews of new text books related to neural computation. To send mail to everyone on the list, address it to Connectionists at CS.CMU.EDU ------------------------------------------------------------------- What NOT to post to CONNECTIONISTS: ----------------------------------- - Requests for addition to the list, change of address and other administrative matters should be sent to: "Connectionists-Request at cs.cmu.edu" (note the exact spelling: many "connectionists", one "request"). If you mention our mailing list to someone who may apply to be added to it, please make sure they use the above and NOT "Connectionists at cs.cmu.edu". - Requests for e-mail addresses of people who are believed to subscribe to CONNECTIONISTS should be sent to postmaster at appropriate-site. If the site address is unknown, send your request to Connectionists-Request at cs.cmu.edu and we'll do our best to help. A phone call to the appropriate institution may sometimes be simpler and faster. - Note that in many mail programs a reply to a message is automatically "CC"-ed to all the addresses on the "To" and "CC" lines of the original message. If the mailer you use has this property, please make sure your personal response (request for a Tech Report etc.) is NOT broadcast over the net. - Do NOT tell a friend about Connectionists at cs.cmu.edu. Tell him or her only about Connectionists-Request at cs.cmu.edu. This will save your friend from public embarrassment if she/he tries to subscribe. - Limericks should not be posted here. ------------------------------------------------------------------------------- The CONNECTIONISTS Archive: --------------------------- All e-mail messages sent to "Connectionists at cs.cmu.edu" starting 27-Feb-88 are now available for public perusal. A separate file exists for each month. The files' names are: arch.yymm where yymm stand for the obvious thing. Thus the earliest available data are in the file: arch.8802 Files ending with .Z are compressed using the standard unix compress program. To browse through these files (as well as through other files, see below) you must FTP them to your local machine. ------------------------------------------------------------------------------- How to FTP Files from the CONNECTIONISTS Archive ------------------------------------------------ 1. Open an FTP connection to host B.GP.CS.CMU.EDU (Internet address 128.2.242.8). 2. Login as user anonymous with password your username. 3. 'cd' directly to one of the following directories: /usr/connect/connectionists/archives /usr/connect/connectionists/bibliographies 4. The archives and bibliographies directories are the ONLY ones you can access. You can't even find out whether any other directories exist. If you are using the 'cd' command you must cd DIRECTLY into one of these two directories. Access will be denied to any others, including their parent directory. 5. The archives subdirectory contains back issues of the mailing list. Some bibliographies are in the bibliographies subdirectory. Problems? - contact us at "Connectionists-Request at cs.cmu.edu". ------------------------------------------------------------------------------- How to FTP Files from the Neuroprose Archive -------------------------------------------- Anonymous FTP on cheops.cis.ohio-state.edu (128.146.8.62) pub/neuroprose directory This directory contains technical reports as a public service to the connectionist and neural network scientific community. Researchers may place electronic versions of their preprints or articles in this directory, announce availability, and other interested researchers can rapidly retrieve and print the postscripts. This saves copying, postage and handling, by having the interested reader supply the paper. To place a file, put it in the Inbox subdirectory, and send mail to pollack at cis.ohio-state.edu. Within a couple of days, I will move and protect it, and suggest a different name if necessary. Current naming convention is author.title.filetype[.Z] where title is enough to discriminate among the files of the same author. The filetype is usually "ps" for postscript, our desired universal printing format, but may be tex, which requires more local software than a spooler. Very large files (e.g. over 200k) must be squashed (with either a sigmoid function :) or the standard unix "compress" utility, which results in the .Z affix. To place or retrieve .Z files, make sure to issue the FTP command "BINARY" before transfering files. After retrieval, call the standard unix "uncompress" utility, which removes the .Z affix. An example of placing a file is attached as an appendix, and a shell script called Getps in the directory can perform the necessary retrival operations. For further questions contact: Jordan Pollack Email: pollack at cis.ohio-state.edu Here is an example of naming and placing a file: gvax> cp i-was-right.txt.ps rosenblatt.reborn.ps gvax> compress rosenblatt.reborn.ps gvax> ftp cheops.cis.ohio-state.edu Connected to cheops.cis.ohio-state.edu. 220 cheops.cis.ohio-state.edu FTP server ready. Name: anonymous 331 Guest login ok, send ident as password. Password:neuron 230 Guest login ok, access restrictions apply. ftp> binary 200 Type set to I. ftp> cd pub/neuroprose/Inbox 250 CWD command successful. ftp> put rosenblatt.reborn.ps.Z 200 PORT command successful. 150 Opening BINARY mode data connection for rosenblatt.reborn.ps.Z 226 Transfer complete. 100000 bytes sent in 3.14159 seconds ftp> quit 221 Goodbye. gvax> mail pollack at cis.ohio-state.edu Subject: file in Inbox. Jordan, I just placed the file rosenblatt.reborn.ps.Z in the Inbox. The INDEX sentence is "Boastful statements by the deceased leader of the neurocomputing field." Please let me know when it is ready to announce to Connectionists at cmu. BTW, I enjoyed reading your review of the new edition of Perceptrons! Frank ------------------------------------------------------------------------ How to FTP Files from the NN-Bench Collection --------------------------------------------- 1. Create an FTP connection from wherever you are to machine "pt.cs.cmu.edu" (128.2.254.155). 2. Log in as user "anonymous" with password your username. 3. Change remote directory to "/afs/cs/project/connect/bench". Any subdirectories of this one should also be accessible. Parent directories should not be. 4. At this point FTP should be able to get a listing of files in this directory and fetch the ones you want. Problems? - contact us at "nn-bench-request at cs.cmu.edu". From Scott_Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU Sun Sep 1 21:34:27 1991 From: Scott_Fahlman at SEF-PMAX.SLISP.CS.CMU.EDU (Scott_Fahlman@SEF-PMAX.SLISP.CS.CMU.EDU) Date: Sun, 01 Sep 91 21:34:27 -0400 Subject: Processing of auditory sequences In-Reply-To: Your message of Sun, 01 Sep 91 19:23:00 -0500. <683767402/xdh@SPEECH2.CS.CMU.EDU> Message-ID: For the purpose of speech-compression, current technology using vector quantization can compress speech to 2k bits/s without much fidelity loss. Even lower rate (at 200-800 bits/s) also has acceptable intelligibility. They can be found in many commercial applications. I think the original question was not asking about data-compression in the usual sense, but about devices that take a normal speech signal and try to play it back as fast as possible without loss of intelligibility -- sort of speed-reading for the blind. If you just speed up the tape, all the frequencies go up and the speech is hard to understand. So these machines try to keep the frequencies the same while speeding up everything else. -- Scott From issnnet at park.bu.edu Sun Sep 1 11:04:24 1991 From: issnnet at park.bu.edu (issnnet@park.bu.edu) Date: Sun, 1 Sep 91 11:04:24 -0400 Subject: we want your messages! Message-ID: <9109011504.AA01159@copley.bu.edu> *** WE WANT YOUR POSTINGS !!!! *** This is a reminder that the new USENET newsgroup comp.org.issnnet was created earlier in August. The newsgroup is part of the International Student Society for Neural Networks (ISSNNet) effort to establish a centralized organization for students in Neural Networks. Many of the messages posted to this group would be ideally suited for posting to comp.org.issnnet. For example any requests for housing at a conference site, or sharing rides, or inquiring about academic programs, job offers, or even requests for references. If you have no access to comp.org.issnnet you can still take advantage of this newsgroup by e-mailing your message to "issnnet at park.bu.edu". Please be sure to indicate you wish your message to be relayed to the newsgroup, and include your name and e-mail address, and any other information that may be needed for people to contact you. Please take advantage of this new opportunity, and become part of this useful new endeavour! ISSNNet From kddlab!hcrlgw.crl.hitachi.co.jp!nitin at uunet.UU.NET Mon Sep 2 09:04:01 1991 From: kddlab!hcrlgw.crl.hitachi.co.jp!nitin at uunet.UU.NET (Nitin Indurkhya) Date: Mon, 2 Sep 91 09:04:01 JST Subject: cascade-correlation Message-ID: <9109020004.AA13200@hcrlgw.crl.hitachi.co.jp> > > Experiments with the Cascade-Correlation Algorithm > Technical report # 91-16 (July 1991) > Jihoon Yang & Vasant Honavar > Department of Computer Science > Iowa State University > folks interested in this might also want to look at the following: "Heuristic Configuration of Hidden Units in Neural Network Classifers" Technical report DCS-TR 279 (June 1991) Nitin Indurkhya and Sholom Weiss Department of Computer Science Rutgers University, New Brunswick, NJ 08903 An abstract can also be found in the IJCNN-91-Seattle proceedings. in this report, the cascade-correlation architecture is evaluated on some well-known real-world datasets. copies of the tech report can be most easily obtained by sending mail to weiss at cs.rutgers.edu --nitin From rosauer at ira.uka.de Mon Sep 2 11:23:05 1991 From: rosauer at ira.uka.de (Bernd Rosauer) Date: Mon, 2 Sep 91 11:23:05 MET DST Subject: connectionist navigation Message-ID: Inspired by the recent summary on chemotaxis by Shawn Lockery I think it would be interesting -- at least to some people -- to compile a bibliography on connectionist approaches to locomotion and navigation. Since I started collecting references on that topic some time ago I will take on the job. The bibliography should include references to work on modeling spatial orientation, cognitive mapping, piloting, and navigation; alife simulations of animat navigation and applications of neural networks to mobile robot obstacle avoidance and path planning. Further suggestions are welcome. In order to bound my own efforts in surveying the references I would like to encourage (a) those people who know that I have some references to their work to send me a complete list, and (b) even to send me older references you are aware of. Although I do not promise to finish the bibliography this month I expect that it could be done this year. So feel free to send me your references. (I know that similar requests have come up in several mailing lists and news groups now and then but I for myself have never seen a summary, even on direct request.) Thanks in advance, Bernd Rosauer Research Center of Computer Science at the University of Karlsruhe, FRG From HJBORTOL at BRFUEM.BITNET Mon Sep 2 10:41:00 1991 From: HJBORTOL at BRFUEM.BITNET (Humberto Jose' Bortolossi) Date: Mon, 02 Sep 91 10:41:00 C Subject: Bibliography to a novice ... Message-ID: Greetings ... I just started to study neural-nets and related topics. But, I have no idea of basic bibliography. Please, could you suggest me some good bo- oks? Is there something available from FTP (remember, I'm a novice in neural nets|)??? Any help will be welcome| Thanks a lot in advance. Humberto Jose' Bortolossi Department of Mathematics From koch at CitIago.Bitnet Mon Sep 2 12:54:04 1991 From: koch at CitIago.Bitnet (Christof Koch) Date: Mon, 2 Sep 91 09:54:04 PDT Subject: Synchronization Binding? Freq. Locking? Bursting? In-Reply-To: Your message <9108211921.AA04522@avw.umd.edu> dated 21-Aug-1991 Message-ID: <910902095401.204025b5@Iago.Caltech.Edu> Sorry for not replying earlier to Thomas Edwards RFC on oscillations and sychnronization. The issue of oscillations is primarily an experimental one. Do single cells in the awake monkey show an oscillatory component in their firing activity and is their firing correlated with the firing of other cells? Well, the only strong oscillations appear to be those found by E. Fetz and V. Murphy in Seattle in motorcortex. If the monkey does a repetitive handmovements these oscillations have a small amplitude. If the monkey has to get a raisin from a Kluever box, their amplitude increases a lot. Crick and I would interpret this to mean that if the money has to attend, oscillations increase. However, with the exceptions of some tantalizing hints of some power at 40 Hz in the spectrum of single cells from some labs, nothing much has been found in the monkey visual cortex, although maybe somebody out there can correct me. Of course, Llinas does see 40 Hz waves between thalamus and cortex in the awake humans using a 37 channel MEG machine. That's why I find synchronized firing, in the absence of oscillations, an attractive possibility, in particular since phase-locked firing will lead to a much bigger response at the single cell level than a temporally smeared out signal (due to the low-pass nature of the neuronal membrane). Christof From CADEPS at BBRNSF11.BITNET Mon Sep 2 13:18:27 1991 From: CADEPS at BBRNSF11.BITNET (JANSSEN Jacques) Date: Mon, 02 Sep 91 19:18:27 +0200 Subject: Evolvability of Recurrent Nets Message-ID: Dear Connectionists, Has anyone out there done any work on, or know about people who have done work on evolvability criteria for time dependent recurrent network behaviors? There's a growing literature now on using the Genetic Algorithm to evolve dynamic behaviors in neural networks, but these attempts sometimes fail. Why? To give a simple (static) example - take the multiplier problem. The aim is to evolve a network which takes two real numbered inputs and returns the product as its output. If the inputs can be of mixed sign, the network does not evolve. If the inputs are always positive, the network evolves. This is just the beginning. I have several examples of desired dynamic behaviors which failed to evolve. Kaufmann has talked about criteria for evolvability of his (simple) Boolean networks. Has anyone seen work on extending these ideas to recurrent nets? Or anything which might do the job? Cheers, Hugo de Garis, Univ of Brussels, & George Mason Univ, VA. From spotter at sanger Mon Sep 2 16:07:45 1991 From: spotter at sanger (Steve Potter) Date: Mon, 2 Sep 91 13:07:45 PDT Subject: Neuroprose index? Message-ID: <9109022007.AA06160@sanger.bio.uci.edu> Browsing through the neuroprose archive directory, I am overwhelmed by the number of interesting topics covered. Is there a list of titles or of brief summaries of the articles that have been archived? The filenames are usually a bit obscure. Steve Potter U of Cal Irvine Psychobiology dept. (714)856-4723 spotter at darwin.bio.uci.edu From dhw at t13.Lanl.GOV Mon Sep 2 17:20:26 1991 From: dhw at t13.Lanl.GOV (David Wolpert) Date: Mon, 2 Sep 91 15:20:26 MDT Subject: No subject Message-ID: <9109022120.AA00432@t13.lanl.gov> John Kruschke writes: "One motive for using RBFs has been the promise of better interpolation between training examples (i.e., better generalization). " Whether or not RBFs result in "better interpolation" is one issue. Whether or not they result in "better generalization" is another. For some situations the two issues are intimately related, and sometimes even identical. For other situations they are not. David Wolpert (dhw at tweety.lanl.gov) From nowlan at helmholtz.sdsc.edu Mon Sep 2 21:21:50 1991 From: nowlan at helmholtz.sdsc.edu (Steven J. Nowlan) Date: Mon, 02 Sep 91 18:21:50 MST Subject: another RBF ref. Message-ID: <9109030121.AA17099@bose> A slightly different slant on the approach of Moody and Darken to training RBF networks, and its relation to fitting statistical mixtures, may be found in: S.J. Nowlan "Maximum Likelihood Competitive Learning" In: D.S. Touretzky (ed.) Advances in Neural Information Processing Systems 2, pp. 574-582, 1990. From terry at jeeves.UCSD.EDU Mon Sep 2 23:20:56 1991 From: terry at jeeves.UCSD.EDU (Terry Sejnowski) Date: Mon, 2 Sep 91 20:20:56 PDT Subject: Neural Computation: 3.3 Message-ID: <9109030320.AA18059@jeeves.UCSD.EDU> Neural Computation Contents 3:3 View: A Practical Approach for Representing Context and for Performing Word Sense Disambiguation Using Neural Networks. Stephen I. Gallant Notes: A Modified Quickprop Algorithm Alistair Craig Veitch and Geoffrey Holmes Letters: Removing Time Variation with the Anti-Hebbian Differential Synapse Graeme Mitchison Simulations of a Reconstructed Cerebellar Purkinje Cell Based on Simplified Channel Kinetics Paul Bush and Terrence Sejnowski On the Mechanisms Underlying Directional Selectivity H. Ogmen 2-Degrees-of-Freedom Robot Path Planning using Cooperative Neural Fields Michael Lemmon Parameter Sensitivity of the Elastic Net Approach to the Traveling Salesman Problem Martin W. Simmen FIR and IIR Synapses, a New Neural Network Architecture for Time Series Modeling A. D. Back and A. C. Tsoi The Transition to Perfect Generalization in Perceptrons Eric B. Baum and Yuh-Dauh Lyuu Learning by Asymmetric Parallel Boltzmann Machines Bruno Apolloni, Diego de Falco Generalization Effects of K-Neighbor Interpolation Training Takeshi Kawabata Including Hints in Training Neural Nets Khalid A. Al-Mashouq and Irving S. Reed On the Characteristics of the Auto-Associative Memory with Nonzero- Diagonal Terms in the Memory Matrix Jung-Hua Wang, Tai-Lang Jong, Thomas F. Krile, and John F. Walkup Handwritten Digit Recognition Using K-Nearest Neighbor, Radial-Basis Function, and Backpropagation Neural Networks Yuchun Lee A Matrix Method for Optimizing a Neural Network Simon A. Barton ----- SUBSCRIPTIONS - VOLUME 3 ______ $35 Student ______ $55 Individual ______ $110 Institution Add $18. for postage and handling outside USA (Back issues from volumes 1 and 2 are available for $28 each.) MIT Press Journals, 55 Hayward Street, Cambridge, MA 02142. (617) 253-2889. ----- From thildebr at athos.csee.lehigh.edu Tue Sep 3 10:42:49 1991 From: thildebr at athos.csee.lehigh.edu (Thomas H. Hildebrandt ) Date: Tue, 3 Sep 91 10:42:49 -0400 Subject: RBFs and Generalization In-Reply-To: David Wolpert's message of Mon, 2 Sep 91 15:20:26 MDT <9109022120.AA00432@t13.lanl.gov> Message-ID: <9109031442.AA07136@athos.csee.lehigh.edu> Begin Wolpert Quote ---------- Date: Mon, 2 Sep 91 15:20:26 MDT From: David Wolpert John Kruschke writes: "One motive for using RBFs has been the promise of better interpolation between training examples (i.e., better generalization). " Whether or not RBFs result in "better interpolation" is one issue. Whether or not they result in "better generalization" is another. For some situations the two issues are intimately related, and sometimes even identical. For other situations they are not. David Wolpert (dhw at tweety.lanl.gov) End Wolpert Quote ---------- I have come to treat interpolation and generalization as the same animal, since obtaining good generalization is a matter of interpolating in the right metric space (i.e. the one that best models the underlying process). If one obtains good results using RBFs, he may assume that the underlying metric space is well represented by some combination of hyperspheres. If he obtains good results using sigmoidally scaled linear functionals, he may assume that the underlying metric space is well represented by some combination of sigmoidal sheets. If the form of the underlying metric space is unknown, then it is a toss-up whether sigmoidal sheets, RBFs, piece-wise hyperplanar, or any number of other basis functions will work best. Thomas H. Hildebrandt Visiting Researcher CSEE Dept. Lehigh University From dhw at t13.Lanl.GOV Tue Sep 3 11:24:48 1991 From: dhw at t13.Lanl.GOV (David Wolpert) Date: Tue, 3 Sep 91 09:24:48 MDT Subject: No subject Message-ID: <9109031524.AA00731@t13.lanl.gov> Thomas Hildebrandt writes: "I have come to treat interpolation and generalization as the same animal, since obtaining good generalization is a matter of interpolating in the right metric space (i.e. the one that best models the underlying process)." Certainly I am sympathetic to this point of view. Simple versions of nearest neighbor interpolation (i.e., memory-based reasoners) do very well in many circumstances. (In fact, I've published a couple of papers making just that point.) However it is trivial to construct problems where the target function is extremely volatile and non-smooth in any "reasonable" metric; who are we to say that Nature should not be allowed to have such target functions? Moreover, for a number of discrete, symbolic problems, the notion of a "metric" is ill-defined, to put it mildly. I am not claiming that metric-based generalizers will necessarily do poorly for these kinds of problems. Rather I'm simply saying that it is a bit empty to state that "If the form of the underlying metric space is unknown, then it is a toss-up whether sigmoidal sheets, RBFs, piece-wise hyperplanar, or any number of other basis functions will work best." That's like saying that if the underlying target function is unknown, then it is a toss-up what hypothesis function will work best. Loosely speaking, "interpolation" is something you do once you've decided on the metric. In addition to such interpolation, "generalization" also involves the preceding step of performing the "toss up" between metrics in a (hopefully) rational manner. David Wolpert (dhw at tweety.lanl.gov) From yair at siren.arc.nasa.gov Tue Sep 3 10:58:23 1991 From: yair at siren.arc.nasa.gov (Yair Barniv) Date: Tue, 3 Sep 91 07:58:23 PDT Subject: No subject Message-ID: <9109031458.AA02472@siren.arc.nasa.gov.> Hello I would appreciate if you could send me a copy of: @techreport{Barto-88a, author="Barto, A. G.", title="From Chemotaxis to Cooperativity: {A}bstract Exercises in Neuronal Learning Strategies", institution="University of Massachusetts", address="Amherst, MA", number="88-65", year=1988, note="To appear in {\it The Computing Neurone}, R. Durbin and R. Maill and G. Mitchison (eds.), Addison-Wesley"}. in a .tex or .ps form Thanks a lot, Yair Barniv From lwyse at park.bu.edu Tue Sep 3 19:09:03 1991 From: lwyse at park.bu.edu (lwyse@park.bu.edu) Date: Tue, 3 Sep 91 19:09:03 -0400 Subject: AI/IS/CS Career Newsletter In-Reply-To: connectionists@c.cs.cmu.edu's message of 3 Sep 91 08:58:35 GM Message-ID: <9109032309.AA03565@copley.bu.edu> What are the rates? - lonce XXX XXX Lonce Wyse | X X Center for Adaptive Systems \ | / X X Boston University \ / 111 Cummington St. Boston,MA 02215 ---- ---- X X X X "The best things in life / \ XXX XXX are emergent." / | \ | From rba at vintage.bellcore.com Wed Sep 4 07:54:11 1991 From: rba at vintage.bellcore.com (Bob Allen) Date: Wed, 4 Sep 91 07:54:11 -0400 Subject: Neuroprose index? Message-ID: <9109041154.AA13516@vintage.bellcore.com> I suggest that the community explore installing neuroprose under WAIS (Wide Area Information Server). This allows keyword searching of documents and a fairly nice X frontend. For information, contact brewster at think.com. If nobody else is interested and I could get the texts from OSU, I'd be happy to do the installation. From thildebr at athos.csee.lehigh.edu Wed Sep 4 12:02:42 1991 From: thildebr at athos.csee.lehigh.edu (Thomas H. Hildebrandt ) Date: Wed, 4 Sep 91 12:02:42 -0400 Subject: Generalization vs. Interpolation In-Reply-To: David Wolpert's message of Tue, 3 Sep 91 09:24:48 MDT <9109031524.AA00731@t13.lanl.gov> Message-ID: <9109041602.AA07514@athos.csee.lehigh.edu> Hildebrandt -- "I have come to treat interpolation and generalization as the same animal, since obtaining good generalization is a matter of interpolating in the right metric space (i.e. the one that best models the underlying process)." Wolpert -- Certainly I am sympathetic to this point of view. Simple versions of nearest neighbor interpolation (i.e., memory-based reasoners) do very well in many circumstances. (In fact, I've published a couple of papers making just that point.) However it is trivial to construct problems where the target function is extremely volatile and non-smooth in any "reasonable" metric; who are we to say that Nature should not be allowed to have such target functions? Moreover, for a number of discrete, symbolic problems, the notion of a "metric" is ill-defined, to put it mildly. I do not presume to tell Nature what to do. We may consider problems for which there is no simple transformation from the input (sensor) space into a (linear) metric space to be "hard" problems, in a sense. Discrete problems, which naturally inhibit interpolation, must be handled by table look-up, i.e. each case treated separately. However, table look-up can be considered to be an extreme case of interpolation -- the transition between one recorded data point and a neighboring one being governed by a Heaviside (threshold) function rather than a straight line. Wolpert -- I am not claiming that metric-based generalizers will necessarily do poorly for these kinds of problems. Rather I'm simply saying that it is a bit empty to state that Hildebrandt -- "If the form of the underlying metric space is unknown, then it is a toss-up whether sigmoidal sheets, RBFs, piece-wise hyperplanar, or any number of other basis functions will work best." Wolpert -- That's like saying that if the underlying target function is unknown, then it is a toss-up what hypothesis function will work best. Loosely speaking, "interpolation" is something you do once you've decided on the metric. In addition to such interpolation, "generalization" also involves the preceding step of performing the "toss up" between metrics in a (hopefully) rational manner. It would be splitting hairs to suggest that the process of choosing an appropriate set of basis functions be called "learning to generalize" rather than "generalization". I could not agree with you more in thinking that the search for an appropriate basis set is one of the important open problems in connectionist research. If nothing is known about the process to be modelled, is there any more efficient way to select a basis than trial-and-error? Are some sets of basis functions more likely to efficiently describe a randomly selected process? Aside from compactness, what other properties can be ascribed to a desirable basis? Given a particular set of basis functions, what criteria must be met by the underlying process in order for the bases to generalize well? Can these criteria be tested easily? These are just a few of the questions that come to mind. I'll be interested in any thoughts you have in this area. Thomas H. Hildebrandt From COUSEIN at lem.rug.ac.be Thu Sep 5 14:43:00 1991 From: COUSEIN at lem.rug.ac.be (COUSEIN@lem.rug.ac.be) Date: Thu, 5 Sep 91 14:43 N Subject: can't find article. Message-ID: <01GA85PPTJ9C000MXM@BGERUG51.BITNET> Dear connectionist, recently I browsed thru the unpublished abstracts of the 1990 Paris ICNN, and came across an abstract that looked interesting, i.e. Generalised Hopfield Networks for Robot Planning. by P. Morasso, V. sanguineti, G. Vercelli, R. Zaccaria, Computer Science, Genova, Italy. However, there was no article. I would like to know if anyone has seen the article published somewhere, or seen an internal tech report or anything of the kind. Where could I obtain a copy? Thanks for your help, best regards, Alexis Cousein, Ghent University Electronics Lab, Belgium. cousein at lem.rug.ac.be p.s. greetings to Martin Dudziak. Long time no see/hear. From GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca Wed Sep 4 15:48:35 1991 From: GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca (GOLDFARB%unb.ca@UNBMVS1.csd.unb.ca) Date: Wed, 04 Sep 91 16:48:35 ADT Subject: Evolvability of Recurrent Nets In-Reply-To: Message of Wed, 04 Sep 91 01:48:54 ADT from Message-ID: On Wed, 04 Sep 91 01:48:54 ADT JANSSEN Jacques writes: > Has anyone seen work on extending these ideas to > recurrent nets? Or anything which might do the job? > > Cheers, Hugo de Garis, > > Univ of Brussels, & > George Mason Univ, VA. I have developed a new model - evolving transformation systems, or reconfigurable learning machines - which is a far-reaching generalization of the NN models and which is, for the first time, accommodates the (structurally) evolving nature of the learning process, i.e., in the language of NN, new nodes that are compositions of some basic units can be introduced during learning. For more information see my previous postings on this mailing list, as well as L. Goldfarb, What is distance and why do we need the metric model for pattern learning?, to appear in Pattern Recognition. L. Goldfarb, Verifiable characterization of an intelligent process, Proc. of the 4th UNB Artificial Intelligence Symposium, UNB, Fredericton, Sept.20-21, 1991. --Lev Goldfarb From GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca Wed Sep 4 15:50:19 1991 From: GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca (GOLDFARB%unb.ca@UNBMVS1.csd.unb.ca) Date: Wed, 04 Sep 91 16:50:19 ADT Subject: Evolvability of Recurrent Nets In-Reply-To: Message of Wed, 04 Sep 91 01:48:54 ADT from Message-ID: On Wed, 04 Sep 91 01:48:54 ADT JANSSEN Jacques writes: > Has anyone seen work on extending these ideas to > recurrent nets? Or anything which might do the job? > > Cheers, Hugo de Garis, > > Univ of Brussels, & > George Mason Univ, VA. I have developed a new model - evolving transformation systems, or reconfigurable learning machines - which is a far-reaching generalization of the NN models and which is, for the first time, accommodates the (structurally) evolving nature of the learning process, i.e., in the language of NN, new nodes that are compositions of some basic units can be introduced during learning. For more information see my previous postings on this mailing list, as well as L. Goldfarb, What is distance and why do we need the metric model for pattern learning?, to appear in Pattern Recognition. L. Goldfarb, Verifiable characterization of an intelligent process, Proc. of the 4th UNB Artificial Intelligence Symposium, UNB, Fredericton, Sept.20-21, 1991. --Lev Goldfarb From PAR at DM0MPI11.BITNET Thu Sep 5 12:30:37 1991 From: PAR at DM0MPI11.BITNET (Pal Ribarics) Date: Thu, 05 Sep 91 16:30:37 GMT Subject: NN Workshop (You can have it already, we have problems with some mailer) Message-ID: From TPPEREIR at BRFUEM.BITNET Thu Sep 5 21:22:07 1991 From: TPPEREIR at BRFUEM.BITNET (TPPEREIR@BRFUEM.BITNET) Date: Thu, 05 Sep 91 21:22:07 C Subject: how can I adder this list? Message-ID: <7DD204880A800067@BITNET.CC.CMU.EDU> include Tarcisio Praciano Pereira in this list, please. ------------------------------------------------------------------------ DR. PRACIANO-PEREIRA, | E-MAIL: BITNET= TPPEREIR at BRFUEM TARCISIO | ANSP= TPPEREIR at NPDVM1.FUEM.ANPR.BR - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - DEP. DE MATEMATICA - UNIVERSIDADE ESTADUAL DE MARINGA AGENCIA POSTAL UEM 87 020 - MARINGA - PR - BRAZIL TELEFAX: (0442) 22-2754 FONE (0442) 26 27 27 ------------------------------------------------------------------------ From PAR at DM0MPI11.BITNET Fri Sep 6 09:21:16 1991 From: PAR at DM0MPI11.BITNET (Pal Ribarics) Date: Fri, 06 Sep 91 13:21:16 GMT Subject: NN Workshop Message-ID: Note: Our previous message had a long header list containing a large number of addresses. This could not be managed by some intermediate mailer so you could get criptic messages. Now the mail is sent individually to avoid this problem. We beg your pardon for that inconvenience. ******************************************************************************* Dear Colleague , You are receiving this mail either - because you showed interest in applying NN methods in the trigger of HEP experiments and responded to our recent survey or - because you have participated in the Elba Workshop Our original intention was to organize a workshop only for trigger applications. We now have accepted the kind invitation of the organizers of the Second International Workshop on Software Engineering, Artificial Intelligence and Expert Systems for High Energy and Nuclear Physics and we think that the 2 days topical workshop (number 4 in the following Bulletin) would be a good start to bring interested people together. This - in case of strong interest - could be followed by a dedicated workshop in the Munich area next fall or later. So we encourage you to send abstracts to this workshop in order to have a succesfull meeting. ******************************************************************************* SECOND INTERNATIONAL WORKSHOP ON SOFTWARE ENGINEERING, ARTIFICIAL INTELLIGENCE AND EXPERT SYSTEMS FOR HIGH ENERGY AND NUCLEAR PHYSICS January 13 - 18 1992 L'AGELONDE FRANCE-TELECOM LA LONDE-LES-MAURES BP 64 F-83250 FRANCE ORGANIZING COMMITTEE G. Auger GANIL Caen, F K. H. Becks Bergische Univ. Wuppertal, D R. Brun CERN CN Geneva, CH S. Cittolin CERN ECP Geneva, CH B. Denby FNAL Chicago, USA F. Etienne CPPM Marseille, F V. Frigo CERN DG Geneva, CH R. Gatto Geneva univ. Geneva, CH G. Gonnet ETHZ Zurich CH M. Green Royal Holloway Col. Egham Surray, GB F. Hautin CNET Lannion, F A. Kataev INR Moscow, USSR C. Kiesling MPI Munich, D P. Kunz SLAC Stanford, USA J. Lefoll DPHPE Saclay Palaisseau, F E. Malandain CERN PS Geneva, CH V. Matveev INR Moscow, USSR M. Metcalf CERN CN Geneva, CH T. Nash FNAL Chicago, USA D. Perret-Gallix LAPP Annecy, F C. Peterson Lund Univ. Lund, S P. Ribarics MPI Munich, D M. Sendall CERN ECP Geneva, CH Y. Shimizu KEK Tsukuba, JP D. Shirkov JINR Dubna, USSR M. Veltman Univ. Michigan Ann Arbor, USA J. Vermaseren NIKHEF-H Amsterdam, NL C. Vogel CISI Paris, F W. Wojcik CCIN2P3 Lyon, F Dear Colleague, Some of you may remember the first workshop organized in Lyon in March 1990. The enthusiasm and interest of the participants (200) showed clearly that this type of meeting was filling a real need for discussion and information. Two years later, we think it is time for another gettogether with a perspective of the large High Energy and Nuclear Physics experiments. We foresee a true type of workshop where in addition to scheduled presentations, plenty of time will be available for informal discussion. Our main objectives are: * To create a bridge, a real link, between computer experts and others. Modern computing techniques are developing rapidly and the gap between physics and computer science activites is getting larger and larger. This workshop is intended to bring to a physicist an understanding of them to a level that real work can begin. ----> Tutorials * Physics research is at the mercy of the industry in many aspects. We have to create a forum between research and industry. The problems encountered in large experiments are close to industrial level and both sides can profit from such collaboration ----> Specialized Workshops, Products Showroom * There will not be HEP experiments at SSC/LHC without integrating new computing technologies, like going to the moon without the transistor. "How we will do it ?" is a question that should be addressed right now. Experiments must have these techniques built-in in order to perform at nominal value. ----> ASTEC project The workshop will take place in a nice resort, in the France-Telecom site "L'AGELONDE", on the French riviera (cote d'Azur, 500 m from the beach). We will have access to an Ethernet network and video-conferencing can be set up. The full six-day Workshop will be organized as follows: Monday 13-14 January Tutorials and, in parallel, Topical Workshops TUTORIALS: Monday 13 (1) 9h-12h30 CASE and Graphical User Interfaces (2) 14h-18h C++ and Object Oriented Programing Tuesday 14 (3) 9h-12h30 Expert systems and Artificial Intelligence (4) 14h-18h Neural Networks in trigger and data analysis TOPICAL WORKSHOP (Monday 13-Tuesday 14) (1) Symbolic Manipulation Techniques "Problems and Results of Symbolic Computations in High Energy and Quantum field Theory" (2) Networks and distributed computing (3) Software development for "Big Sciences" (4) Methods and techniques of artificial intelligence Wednesday 15 January Round-table, discussions, ASTEC project organization Thursday 15 - Saturday 17 PLENARY SESSIONS The ASTEC Project One session will be devoted to the organization of long standing working groups, not directly depending on experimental projects or accelerators (LEP200, LHC, UNK, SSC, RHIC, HERA, KEK, CEBAF, B-Factories, Tau/charm-Factories, ...) but with this future in mind. The importance of these techniques has been largely demonstrated at LEP. The purpose of the ASTEC project is to prepare reports to be used as references or user-guides for developing more dedicated applications. (Similar to the CERN yellow report for Physics at LEP or LHC) This workshop will launch the project by precisely surveying the items to be studied and naming various responsibilities for a one to two year program. The outcome of this work will be presented at a later workshop and published as a book. The large task facing us will include evaluation of commercial products, collaboration with companies to improve selected items, definition and running of benchmarks, building pilot projects and making proposals for standardization of tools, data description, languages and systems. REMEMBER: "Higher luminosities are also achieved through a low down-time and DAQ dead-time, more accurate triggers and better event reconstruction programs. " "ASTEC project: an Energy Saver Enterprise" Three main groups will be organized: (A) Group: Software Engineering (1) Subgroup: Languages and Systems - Conventional languages, Fortran 90, C, ... - Object Oriented Languages, C++, Objective-C, ... - Mixed languages environment - Operating systems HEPIX, HEPVM, ... - Network Wide Application software maintenance - Porting packages between languages and OS. - DataBase maintenance (updating, access protection) - Data description and representation (2) Subgroup: CASE Tools for developing, maintaining and designing software projects. - Intelligent Editors - Maintenance of multi-version application: CMZ, Historian, ... - On-line Documentation - Symbolic debuggers - Data representation - Software design and simulation - System simulations for real-time application - (3) Subgroup: Interactive Analysis - Event Server - Workstation <-> Mainframe cooperation - Graphical User Interface - Interactive Analysis packages PAW, Reason, IDAL, ... - (B) Group: A.I. (1) Subgroup: Languages, Systems - Prolog II, Prolog III, ... - Mixing Prolog, OOL and Conventional languages in applications - Expert system development management. - (2) Subgroup: Expert systems - Off-line support - Hardware testing and maintenance - On-line assistance - Real-time expert systems - Electronic log-book - Testing expert systems: Validation and verification - Embedding E.S. support in detectors or systems (3) Subgroup: Pattern recognition methods - Track and shower recognition - Event selection - Fuzzy-logic in pattern recognition - Genetic Algorithms - (4) Subgroup: Neural Networks - Algorithms for off-line pattern recognition - Algorithms for fast triggering - Test of Silicon Neural Network prototypes - Neural Network training (C) Group: Symbolic Manipulation Techniques (1) Subgroup: Languages, Systems - Schoonschip, Form, Reduce, Mathematica, Scratchpad II, GAL, Maple, ... a critical review. - Graphics for representation of diagrams and for display of results - Database for intermediate computations and results, integrals, sub-diagrams, ... - (2) Subgroup: Feynman Diagrams - Diagram generation - Symbolic diagram computation - Symbolic/numeric integral computation - Feynman Diagram Symbolic Manipulation Collaboration - (3) Subgroup: Quantum Field Theory and Super-Algebra - methods and algorithms of higher order calculations - symbolic manipulation for N-loop calculations - numerical methods for N-loop calculations - calculations in supersymmetry theories (SUSY, SG, Strings) - applications in Quantum Field Theory - ------------------------------------------------------------------------ Talks will be selected by the Organizing Committee on the basis of a detailed abstract to be submitted before: 15 October, 1991. A poster session will be organized. ======================================================================== SECOND INTERNATIONAL WORKSHOP ON SOFTWARE ENGINEERING, ARTIFICIAL INTELLIGENCE AND EXPERT SYSTEMS FOR HIGH ENERGY AND NUCLEAR PHYSICS 1992 January 13 - 18 L'AGELONDE FRANCE-TELECOM LA LONDE LES MAURES BP 64 F-83250 REGISTRATION NAME: FIRSTNAME: LABORATORY: COUNTRY ADDRESS: TEL: FAX: TELEX: E-MAIL: HOTEL RESERVATION (Number of persons): In the following you are expected to answer with the corresponding number or character from the list above. However if your interest is not mentioned in the list give a full description. WOULD YOU BE INTERESTED TO JOIN A WORKING GROUP OF THE ASTEC PROJECT ? YES/NO GROUP: SUBGROUP: WOULD YOU LIKE TO ATTEND TOPICAL WORKSHOPS OR TUTORIALS ? WORKSHOPS: TUTORIALS: WOULD YOU LIKE TO PRESENT A TALK ? YES/NO TALK TITLE: To be considered by the organizing committee, send an extended abstract before Oct. 15, 1991 to: Michele Jouhet Marie-claude Fert CERN L.A.P.P. - IN2P3 PPE-ADM B.P. 110 CH-1211 Geneve 23 F-74941 Annecy-Le-Vieux SWITZERLAND FRANCE Tel: (41) 22 767 21 23 Tel: (33) 50 23 32 45 Fax: (41) 22 767 65 55 Fax: (33) 50 27 94 95 Telex: 419 000 Telex: 385 180 F E-mail: jouhet at CERNVM Workshop fee : 700 FFr. Student : 500 FFr. Accommodation : 2000 FFr. Accompagning Person: +1200 FFr. To be paid by check: Title: International Workshop CREDIT LYONNAIS/Agence Internationale Bank: 30002 Guichet: 1000 Account: 909154 V Address: LYON REPUBLIQUE The accommodation includes: hotel-room, breakfast, lunch and dinner for 6 days. Tennis, mountain bike and other activities will be available. Denis Perret-Gallix Tel: (41) 22 767 62 93 E-mail: Perretg at CERNVM Fax: (41) 22 782 89 23 From dhw at t13.Lanl.GOV Fri Sep 6 11:20:11 1991 From: dhw at t13.Lanl.GOV (David Wolpert) Date: Fri, 6 Sep 91 09:20:11 MDT Subject: interpolation vs. generalization Message-ID: <9109061520.AA04395@t13.lanl.gov> Thomas Hildebrandt and I probably agree on all important points, but simply emphasize different aspects of the problem. However, 1) One must be very careful about talking about "basis sets". There are many generalizers (e.g., nearest neighbor generalizers) which can not be expressed as forming a linear combination of basis functions in the usual Taylor decomposition manner. In fact, one can prove the following: Assume that we want a generalizer which obeys all the invariances of Euclidean space. For example, if the entire training set is translated in input space, then the guessed hypothesis function should be translated in the same manner. (We also want scaling invariance, rotation invariance, etc.) There are many such generalizers. However there does not exist a basis set of functions such that a generalizer which works by fitting a linear combination of the basis functions to the elements of the training set obeys those invariances. There is an exception to this theorem: if the input space is one-dimensional, then there *is* such a basis set of functions, but only one. Those functions are just the monomials. 2) Hildebrandt says: "We may consider problems for which there is no simple transformation from the input (sensor) space into a (linear) metric space to be "hard" problems, in a sense." Well, yes and no. For some generalizers, namely those which assume such a metric space, such problems are hard. For other generalizers they are not. A simple example is fan generalizers, which can be viewed as the multi-dimensional generalization of the non-linear time-series technique of "embedding" a time series in a delay space. For such generalizers, even target functions which are extremely volatile can be generalized exceedingly well. 3) Hildebrandt also says that "Discrete problems, which naturally inhibit interpolation, must be handled by table look-up, i.e. each case treated separately. However, table look-up can be considered to be an extreme case of interpolation -- the transition between one recorded data point and a neighboring one being governed by a Heaviside (threshold) function rather than a straight line. Yes, you can view the behavior of any generalizer as "interpolation", if you stretch the meaning of the term sufficiently. My point is that viewing things that way for some problems (not all) amounts to a not-very-informative tautology. For some problems, the appropriate response to such a view is "well, yes, it is, technically speaking, interpolation, but so what? What does such a perspective gain you? Again, I *am* very sympathetic to the interpolation model. There are many advantages of memory-based reasoning over conventional neural nets, and (to my mind at least) not many advantages of neural nets over memory-based reasoning. But that doesn't mean that "interpolation" is the end of the story and we can all go home now. 4) "If nothing is known about the process to be modelled, is there any more efficient way to select a basis than trial-and-error?" I'm not sure what you mean by "trial-and-error". Do you mean cross- validation? If so, then the answer is yes, there are techniques better than cross-validation. Cross-validation is a winner-takes-all strategy, in which one picks a single generalizer and uses it. One can instead use stacked generalization, in which one *combines* generalizers non-linearly. For any who are interested, I can mail reprints of mine which discuss some of the points above. David Wolpert (dhw at tweety.lanl.gov) From zoubin at learning.siemens.com Fri Sep 6 14:33:06 1991 From: zoubin at learning.siemens.com (Zoubin Ghahramani) Date: Fri, 6 Sep 91 14:33:06 EDT Subject: interpolation vs. generalization In-Reply-To: David Wolpert's message of Fri, 6 Sep 91 09:20:11 MDT <9109061520.AA04395@t13.lanl.gov> Message-ID: <9109061833.AA22287@learning.siemens.com.siemens.com> Just to throw a monkey wrench in: How does one interpret generalization as interpolation in a problem like n-bit parity? For any given data point, the n nearest neighbours in input space would all predict an incorrect classification. However, I wouldn't say that a problem like parity is ungeneralizable. Zoubin Ghahramani Dept. of Brain & Cognitive Sciences MIT From russ at oceanus.MITRE.ORG Fri Sep 6 14:53:15 1991 From: russ at oceanus.MITRE.ORG (Russell Leighton) Date: Fri, 6 Sep 91 14:53:15 EDT Subject: Beta Test Sites For Aspirin v5.0 Message-ID: <9109061853.AA05585@oceanus> Attention users of the MITRE Neural Network Simulator Aspirin/MIGRAINES Version 4.0 Aspirin/MIGRAINES Version 5.0 is currently in beta test. I am seeking to expand the list of sites before the general release in mid October. Version 5.0 is *much* more portable than version 4.0 and has new graphics using libraries from the apE2.1 visualization package. Supported platforms include: Sun4, Sun3 Silicon Graphics Iris IBM RS/6000 Intel 486/386 (Unix System V) NeXT DecStation Cray YMP Convex Coprocessors: Mercury i860 (40MHz) Coprocessors Meiko Computing Surface w/i860 (40MHz) Nodes iWarp Cells I would like to expand this list to other Unix platforms (e.g. HP snakes, MIPS, etc.). This should be very easy. If you are currently using Aspirin and would like to be part of this preliminary release AND... 1. You have email on the Internet 2. You have ftp access on the Internet ...then please reply to this message. In particular, I would like to find users on: 1. DecStations 2. NeXT 3. IBMRS6000 4. Other Unix platforms. Thanks. Russ INTERNET: russ at dash.mitre.org Russell Leighton MITRE Signal Processing Lab 7525 Colshire Dr. McLean, Va. 22102 USA From GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca Sat Sep 7 16:38:07 1991 From: GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca (GOLDFARB%unb.ca@UNBMVS1.csd.unb.ca) Date: Sat, 07 Sep 91 17:38:07 ADT Subject: Choice of the basis functions In-Reply-To: Message of Fri, 06 Sep 91 02:09:17 ADT from Message-ID: On Fri, 06 Sep 91 02:09:17 ADT "Thomas H. Hildebrandt " writes: > I could not agree with you more in > thinking that the search for an appropriate basis set is one of the > important open problems in connectionist research. > > If nothing is known about the process to be modelled, is there any > more efficient way to select a basis than trial-and-error? > > Are some sets of basis functions more likely to efficiently describe a > randomly selected process? Aside from compactness, what other > properties can be ascribed to a desirable basis? > > Given a particular set of basis functions, what criteria must be met > by the underlying process in order for the bases to generalize well? > Can these criteria be tested easily? > > These are just a few of the questions that come to mind. I'll be > interested in any thoughts you have in this area. > > Thomas H. Hildebrandt Within the metric model proposed in L.Goldfarb, A New Approach to Pattern Recognition, in Progress in Pattern Recognition 2, L.N.Kanal and A.Rosenfeld, eds., North- Holland, Amsterdam,1985 the question of choice of the "right" basis can be resolved quite naturally: the finite training metric set is first isometrically embedded in the "appropriate" Minkowski vector space, and then a subset best representing the principal axes of the constructed vector representation is chosen as the basis. -- Lev Goldfarb From GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca Sun Sep 8 16:30:24 1991 From: GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca (GOLDFARB%unb.ca@UNBMVS1.csd.unb.ca) Date: Sun, 08 Sep 91 17:30:24 ADT Subject: interpolation vs. generalization In-Reply-To: Message of Sun, 08 Sep 91 13:36:50 ADT from Message-ID: On Sun, 08 Sep 91 13:36:50 ADT Zoubin Ghahramani writes: > > Just to throw a monkey wrench in: > > How does one interpret generalization as interpolation in a problem > like n-bit parity? For any given data point, the n nearest neighbours > in input space would all predict an incorrect classification. However, > I wouldn't say that a problem like parity is ungeneralizable. > > > Zoubin Ghahramani > Dept. of Brain & Cognitive Sciences > MIT Dear Zoubin, I am happy to inform you that the above is no "monkey wrench". Here is the solution to your parity problem within the transformation systems model. To simplify the exposition, I will ignore the positions of 1's and delete all 0's. Let C+ (the positive learning set) be as follows: ||||, ||, ||||||||; and let C- (the negative learning set) be as follows: |||||, |, |||. For this problem the initial transformation system T0 is S0 = {s1} the initial set of operations consists of a single operation s1 - deletion-insertion of |, CR = {R1} the set of composition rules consists of a single rule R1 - concatenation of two strings of |'s. The competing family of distance functions is defined by means of the shortest weighted path between two strings of |'s - the smallest weighted number of current operations necessary to transform one string into the other. Since we require that the sum of the operation weights must be 1, and S0 consists of a single operation, the initial distance between two patterns is just the difference between the number of |'s. The following function is maximized during learning F1 (w) F(w) = -----------, F2 (w) + c where w is the weight vector w = (w^1, w^2, . . ., w^m) , m is the number of current operations, sum of w^i's is 1, F1 is the (shortest) distance between C+ and C-, F2 is the average distance in C+, and c is a very small positive constant (to prevent the overflow). For T0 the maximum of F is 1/4. Applying rule R1 to the only existing operation, we obtain new operation s2 - deletion-insertion of ||. Thus, in the evolved transformation system T1 set S1 = {s1, s2}, and the optimization is over the 1-d simplex w^2 | 1| . | . | . |__________1______ w^1 For T1 the maximum of F is a very large number 1/c and it is achieved at w* = (1, 0). Moreover, F1 (w*) = 1 and F2 (w*) = 0. The learning stage is completed. The recognition stage and the propositional class description readily follow from the above. -- Lev Goldfarb From eric at mcc.com Sun Sep 8 20:58:54 1991 From: eric at mcc.com (Eric Hartman) Date: Sun, 8 Sep 91 19:58:54 CDT Subject: more RBF refs Message-ID: <9109090058.AA10679@bird.aca.mcc.com> Here are a couple more RBF refs: Hartman, Keeler & Kowalski Layered Neural Networks With Gaussian Hidden Units as Universal Approximators Neural Computation, 2 (2), 210-215 (1990) Hartman & Keeler Semi-local Units for Prediction IJCNN-91-Seattle, II-561-566 The latter describes some limitations of RBFs and a new activation function that is less localized than an RBF but more localized than a sigmoid. The function is based on gaussians; backpropagation networks using these basis functions learn the Mackey-Glass time series nearly as fast as the standard RBF network training algorithm. The full paper will appear in Neural Computation. Eric Hartman and Jim Keeler From dhw at t13.Lanl.GOV Mon Sep 9 11:41:03 1991 From: dhw at t13.Lanl.GOV (David Wolpert) Date: Mon, 9 Sep 91 09:41:03 MDT Subject: No subject Message-ID: <9109091541.AA07866@t13.lanl.gov> Zoubin Ghahramani writes "How does one interpret generalization as interpolation in a problem like n-bit parity? For any given data point, the n nearest neighbours in input space would all predict an incorrect classification. However, I wouldn't say that a problem like parity is ungeneralizable. " A very good example. In fact, any generalizer which acts in a somewhat local manner (i.e., looks mostly at nearby elements in the learning set) has the nice property that for the parity problem, the larger the training set the *worse* the generalization off of that training set, for precisely the reason Dr. Ghahramani gives. Interestingly, for small (number of bits small) versions of the parity problem, backprop has exactly this property; as the learning set grows, so does its error rate off of the learning set. (Dave Rumelhart has told me that this property goes away in big versions of parity however.) David Wolpert From dsomers at park.bu.edu Mon Sep 9 12:23:11 1991 From: dsomers at park.bu.edu (David C. Somers) Date: Mon, 9 Sep 91 12:23:11 -0400 Subject: Synchronization Binding? Freq. Locking? Bursting? Message-ID: <9109091623.AA06807@park.bu.edu> This is in reply to a recent message from the connectionist news group >====================================================================== >From: connectionists at c.cs.cmu.edu Newsgroups: bu.mail.connectionists >Subject: Synchronization Binding? Freq. Locking? Bursting? Date: 22 >Aug 91 06:03:07 GMT > >>From: Thomas VLSI Edwards >I have just read "Synchronized Oscillations During Cooperative >Feature Linking in a Cortical Model of Visual Perception" >(Grossberg, Somers, Neural Networks Vol. 4 pp 453-466). >It describes some models of phase-locking (supposedly neuromorphic) >relaxation oscillators, including a cooperative bipole coupling which >appears similar to the Kammen comparator model, and fits into BCS >theory. Cooperative Bipole Coupling is significantly different from the comparator model used by Kammen, Holmes and Koch. The Bipole mechanism is a sort of "statistical AND-gate" which becomes active (and thus provides feedback) only when both of its spatially independent receptive flanks are sufficiently active. Feedback then passes to a cell or cells which lie intermediate to the two flanking regions. In the full Boundary Contour System, this feedback is also passed only to cells whose receptive field properties (e.g., orientation) are similar to those cells which activate the particular bipole cell. This mechanism was proposed (By Grossberg and Mingolla) to handle the process of emergent segmentation in the visual system such as occurs in the perception of occluding contours, textural boundaries, and illusory or broken contours. As Grossberg and I noted in our paper this mechanism has received both neuroanatomical and neurophysiological support. In the context of synchronized oscillations, we used the bipole mechanism to not only synchronize activity along and over two regions of oscillatory activity, but also to induce and synchronize oscillatory activity within a slit region in between the two oscillating regions. The bipole mechanism accomplished this "perceptual boundary completion" without inducing a spreading of oscillatory activity to the outlying regions. The comparator model cannot robustly achieve this effect since it does not distinguish between the completion of a boundary over a break between two regions and the outward spreading of activity from the end of a line segment. That is, the comparator is not sensitive to the spatial distribution of its inputs but rather only to the total input. The Adaptive Filter mechanism that we use in our simulations reduces to the comparator mechanism when the fan-in of the Adaptive Filter equals its Fan-out. The Adaptive Filter achieved synchronization without achieving boundary completion. These results taken together suggest that different architectures be used to achieve different results. We interpret the Bipole cell results as a pre-attentive boundary completion, while the adaptive filter results correspond to an attentive resonant state as may occur during recall or learning (cf. Adaptive Resonance Theory). >I am curious at this date what readers of connectionists think about >the theory that syncrhonous oscillations reflect the binding of local >feature detectors to form coherent groups. I am also curious as to >whether or not phase-locking of oscillators is a reasonable model >of the phenomena going on, or whether synchronized bursting, yet >not frequency-locked oscillation, is a more biologically acceptable >answer. Charlie Gray's data seems to indicate that it is actually bursting, not single spikes that are being synchronized. This is consistent with our oscillations of average firing rate. Note that bursting can still be viewed as an oscillatory phenomena. Also note that Gray's data indicates that synchrony occurs very rapidly--within one or two cycles. Our simulations demonstrate this effect, although many other researchers have had great difficulty in achieving rapid synchrony. Although the architecture of connections between the oscillators is important, so is the form of the individual oscillators. Nancy Kopell and I have a series of results that show the advantages of using neural relaxation oscillators rather than boring old sinusoids (papers in preparation). As far as the meaning of the synchronized oscillations, I think we really need a lot more data to really be able to tell. Right now we've got a lot of modellers chasing a little bit of data. Having said that, there is still something compelling about synchronized oscillations. My hunch is that the oscillations represent a form of multiplexing where the average firing rate over several burst cycles indicates the information local to the classical receptive field while the synchronization of the bursts represents some form of global grouping of information. Anything less than this multiplexing would not seem to serve any purpose (at least in vision) -- Why give up coding by average firing rate in order to code by phase relationship? This is just trading one dimension for another (with a seemingly small dynamic range) I suggest that the visual system may be making use of both coding dimensions. This kind of multiplexing would also allow for very rapid computations, since global extraction would be performed while local information accumulates rather than performing these operations sequentially. David Somers (dsomers at park.bu.edu) Center for Adaptive Systems 111 Cummington St. Boston, MA 02215 From barto at envy.cs.umass.edu Mon Sep 9 13:17:05 1991 From: barto at envy.cs.umass.edu (Andy Barto) Date: Mon, 9 Sep 91 13:17:05 -0400 Subject: Technical Report Available Message-ID: <9109091717.AA17616@envy.cs.umass.edu> The following technical report is available: Real-Time Learning and Control using Asynchronous Dynamic Programming Andrew G. Barto, Steven J. Bradtke, Satinder P. Singh Department of Computer Science University of Massachusetts, Amherst MA 01003 Technical Report 91-57 Abstract---Learning methods based on dynamic programming (DP) are receiving increasing attention in artificial intelligence. Researchers have argued that DP provides the appropriate basis for compiling planning results into reactive strategies for real-time control, as well as for learning such strategies when the system being controlled is incompletely known. We extend the existing theory of DP-based learning algorithms by bringing to bear on their analysis a collection of relevant mathematical results from the theory of asynchronous DP. We present convergence results for a class of DP-based algorithms for real-time learning and control which generalizes Korf's Learning-Real-Time-A* (LRTA*) algorithm to problems involving uncertainty. We also discuss Watkins' Q-Learning algorithm in light of asynchronous DP, as well as some of the methods included in Sutton's Dyna architecture. We provide an account that is more complete than currently available of what is formally known, and what is not formally known, about the behavior of DP-based learning algorithms. A secondary aim is to provide a bridge between AI research on real-time planning and learning and relevant concepts and algorithms from control theory. -------------------------------------------------------------------- This TR has been placed in the Neuroprose directory (courtesy Jordan Pollack) in compressed form. The file name is "barto.realtime-dp.ps.Z". The instructions for retreiving this document from that archive are given below. WARNING: This paper is SIXTY EIGHT pages long. If you are unable to retreive/print it and therefore wish to receive a hardcopy please send mail to the following address: Connie Smith Department of Computer Science University of Massachusetts Amherst, MA 01003 Smith at cs.umass.edu ****PLEASE DO NOT REPLY TO THIS MESSAGE***** NOTE: This is the paper on which my talk at the Machine Learning Workshop, July 1991, was based. If you requested a copy at that time, it is already in the mail. Thanks, Andy Barto -------------------------------------------------------------------- Here is how to ftp this paper: unix> ftp cheops.cis.ohio-state.edu (or 128.146.8.62) Name: anonymous Password: neuron ftp> cd pub/neuroprose ftp> binary ftp> get barto.realtime-dp.ps.Z ftp> quit unix> uncompress barto.realtime-dp.ps.Z unix> lpr barto.realtimedp.ps From cherwig at eng.clemson.edu Mon Sep 9 15:28:16 1991 From: cherwig at eng.clemson.edu (christoph bruno herwig) Date: Mon, 9 Sep 91 15:28:16 EDT Subject: morphology Message-ID: <9109091928.AA08621@eng.clemson.edu> Dear all, I am interested in having neural networks learn morphological operators like DILATION, EROSION, CLOSING, OPENING, SKELETONIZING. Initial attempt is an input-layer/output-layer feedforward network and backprop learning algorithm. I posted in 'comp.ai.vision' before and received the appended replies. I would appreciate it, if anyone of you can add references to my list. Thank you very much in advance! ++++++++++++++++++ From sch at ee.UManitoba.CA Mon Sep 9 16:24:23 1991 From: sch at ee.UManitoba.CA (sch@ee.UManitoba.CA) Date: Mon, 9 Sep 91 15:24:23 CDT Subject: unsubscribe Message-ID: <9109092024.AA20880@ic12.ee.umanitoba.ca> Please remove me from the connectionist mailing list. From tp-temp at ai.mit.edu Mon Sep 9 20:58:53 1991 From: tp-temp at ai.mit.edu (Tomaso Poggio) Date: Mon, 9 Sep 91 20:58:53 EDT Subject: more RBF refs In-Reply-To: Eric Hartman's message of Sun, 8 Sep 91 19:58:54 CDT <9109090058.AA10679@bird.aca.mcc.com> Message-ID: <9109100058.AA00733@erice> Why call gradient descent backpropagation? From eric at mcc.com Mon Sep 9 21:07:31 1991 From: eric at mcc.com (Eric Hartman) Date: Mon, 9 Sep 91 20:07:31 CDT Subject: more RBF refs Message-ID: <9109100107.AA12412@bird.aca.mcc.com> From tp-temp at ai.mit.edu Mon Sep 9 19:59:14 1991 Posted-Date: Mon, 9 Sep 91 20:58:53 EDT Received-Date: Mon, 9 Sep 91 19:59:13 CDT Received: from MCC.COM by bird.aca.mcc.com (4.0/ACAv4.1i) id AA12394; Mon, 9 Sep 91 19:59:13 CDT Received: from life.ai.mit.edu by MCC.COM with TCP; Mon 9 Sep 91 19:59:13-CDT Received: from erice (erice.ai.mit.edu) by life.ai.mit.edu (4.1/AI-4.10) id AA14982; Mon, 9 Sep 91 20:58:38 EDT From: tp-temp at ai.mit.edu (Tomaso Poggio) Received: by erice (4.1/AI-4.10) id AA00733; Mon, 9 Sep 91 20:58:53 EDT Date: Mon, 9 Sep 91 20:58:53 EDT Message-Id: <9109100058.AA00733 at erice> To: eric at mcc.com Cc: connectionists at cs.cmu.edu In-Reply-To: Eric Hartman's message of Sun, 8 Sep 91 19:58:54 CDT <9109090058.AA10679 at bird.aca.mcc.com> Subject: more RBF refs Status: R Why call gradient descent backpropagation? ---------------- I didn't mean to. Careless phrasing. Backprop implements gradient descent. Eric Hartman From jose at tractatus.siemens.com Tue Sep 10 07:33:25 1991 From: jose at tractatus.siemens.com (Steve Hanson) Date: Tue, 10 Sep 1991 07:33:25 -0400 (EDT) Subject: more RBF refs In-Reply-To: <9109090058.AA10679@bird.aca.mcc.com> References: <9109090058.AA10679@bird.aca.mcc.com> Message-ID: <0cn_q5y1GEMn0lOzIS@tractatus.siemens.com> Another paper on RBFs ("spherical units") also using a basis function (cauchy) less localized than the gaussian but more localized than the linear-logistic ("sigmoid") is to be found in: Hanson, S. J. & Gluck, M. A. Spherical Units as Dynamic Consequential Regions: Implications for Attention, Competition and Categorization. Advances in Neural Information Processing 3, R. Lippmann, J. Moody & D. Touretzky (Eds.), Morgan Kaufman, pp. 656-665, (1991). Stephen J. Hanson Learning Systems Department SIEMENS Research 755 College Rd. East Princeton, NJ 08540 From N.E.Sharkey at dcs.exeter.ac.uk Tue Sep 10 13:31:09 1991 From: N.E.Sharkey at dcs.exeter.ac.uk (Noel Sharkey) Date: Tue, 10 Sep 91 13:31:09 BST Subject: CONNECTION SCIENCE REDUCTIONS Message-ID: <8443.9109101231@propus.dcs.exeter.ac.uk> A VERY SPECIAL DEAL FOR MEMBERS OF THE CONNECTIONISTS MAILING. Thanks to persistent representations from members of the editorial board of Connection Science (especially Gary Cottrell) and to the dramatic increase in sales this year, a massive discount is being offered to recipients of this mailing list for personal subscriptions to the journal. Prices for members of this list will now be: North America 44 US Dollars (reduced from 126 dollars) Elsewhere and U.K. 22 pounds sterling. (Sterling checks must be drawn on a UK bank) These rates start from 1st January 1992 (volume 4). Conditions: 1. Personal use only (i.e. non-institutional). 2. Must subscribe from your private address. You can receive a subscription form by emailing direct to the publisher: email: carfax at ibmpcug.co.uk Say for the attention of David Green and say CONNECTIONISTS MAILING LIST. noel From riffraff at mentor.cc.purdue.edu Tue Sep 10 10:56:55 1991 From: riffraff at mentor.cc.purdue.edu (bertrands clarke) Date: Tue, 10 Sep 91 09:56:55 -0500 Subject: unsubscribe Message-ID: <9109101456.AA17157@mentor.cc.purdue.edu> Please remove me from the connectionist mailing list. From hinton at ai.toronto.edu Tue Sep 10 13:06:26 1991 From: hinton at ai.toronto.edu (Geoffrey Hinton) Date: Tue, 10 Sep 1991 13:06:26 -0400 Subject: No subject In-Reply-To: Your message of Mon, 09 Sep 91 11:41:03 -0400. Message-ID: <91Sep10.130641edt.262@neuron.ai.toronto.edu> David Wolpert writes "A very good example. In fact, any generalizer which acts in a somewhat local manner (i.e., looks mostly at nearby elements in the learning set) has the nice property that for the parity problem, the larger the training set the *worse* the generalization off of that training set, for precisely the reason Dr. Ghahramani gives." His definition of "somewhat local" would seem to include K nearest neighbors. Curiously, this can do a good job of generalizing parity from randomly distributed examples IFF we use generalization on a validation set to determine the optimal value of K. Each n-bit input vector has n neighbors that differ on 1 bit, but order n^2 neighbors that differ on two bits (and have the same parity). So the generalization to a validation set will be best for K small and even. He also writes "Interestingly, for small (number of bits small) versions of the parity problem, backprop has exactly this property; as the learning set grows, so does its error rate off of the learning set." In fact, backprop generalizes parity rather well using n hidden units, provided the number of connections (n+1)^2 is considerably smaller than the number of training examples. For small n, (n+1)^2 is smaller than 2^n, so its not surprising it doesnt generalize. For n=10, it generalises very well from 512 training examples to the rest (only 2 or 3 errors). Geoff From dhw at t13.Lanl.GOV Tue Sep 10 13:53:35 1991 From: dhw at t13.Lanl.GOV (David Wolpert) Date: Tue, 10 Sep 91 11:53:35 MDT Subject: yet more interp. vs. gen. Message-ID: <9109101753.AA09444@t13.lanl.gov> Geoff Hinton writes, concerning using generalizers on the parity problem: "Curiously, K nearest neighbors can do a good job of generalizing parity from randomly distributed examples IFF we use generalization on a validation set to determine the optimal value of K. Each n-bit input vector has n neighbors that differ on 1 bit, but order n^2 neighbors that differ on two bits (and have the same parity). So the generalization to a validation set will be best for K small and even." 1) I don't understand the conclusion; for K small (e.g., <~ n), as Geoff points out, the nearest neighbors differ by 1 bit, have the wrong parity, and you run into the problem of increasingly bad generalization. In fact, 2) I've done tests with K <= n on parity, using a weighted (by Hamming distance) average of those K neighbors, and generalization error off of the training set definitely gets worse as the training set size increase, asymptoting at 100% error. Indeed, if K <= n, I don't care what kind of weighting one uses, or what the actual value of K is; 100% error will occur for training sets consisting of samples from the entire space except for the single off-training set question. 3) I don't doubt that if one uses cross-validation to set K one can do better than if one doesn't. (In fact, this is exactly the tack I took in my 1990 Neural Networks article to beat NETtalk using a weighted average generalizer.) However I should point out that if we're broadening the discussion to allow the technique of cross-validation, then one should consider non-linear-time-series-type techniques, like fan generalizers. On the parity problem it turns out that the guessing of such generalizers is more accurate than either backprop or nearest neighbors. For example, I've recently run a test on the parity problem with n = 24, with a training set consisting solely of the 301 sample points with 2 or fewer of the 24 bits on; fan generalizers have *perfect* generalization to each of the remaining 16 million + sample points. Not a single error. (With a bit of work, I've managed to prove why this behavior occurs.) For some other sets of sample points (e.g., for most randomly distributed training sets) the kind of fan generalizers I've been experimenting with can only make guesses for a subset of the questions off of the training set; for those points where they can't make a guess one must use some other technique. Nonetheless, for those questions where they *can* make a guess, in the thousand or so experiments I've run on the parity problem they NEVER make an error. I should point out that fan generalizers obviously can't have behavior superior to backprop on *all* target functions. Nonetheless, fan generalizers *do* have superior behavior for 8 out of the 9 simple target functions that have been tested so far; even for those target functions where they don't do generalize perfectly, they have fewer errors than backprop. David From lakoff at cogsci.Berkeley.EDU Wed Sep 11 04:13:12 1991 From: lakoff at cogsci.Berkeley.EDU (George Lakoff) Date: Wed, 11 Sep 91 01:13:12 -0700 Subject: unsubscribe Message-ID: <9109110813.AA05097@cogsci.Berkeley.EDU> UNSUBSCRIBE From Alexis_Manaster_Ramer at MTS.cc.Wayne.edu Wed Sep 11 08:08:02 1991 From: Alexis_Manaster_Ramer at MTS.cc.Wayne.edu (Alexis_Manaster_Ramer@MTS.cc.Wayne.edu) Date: Wed, 11 Sep 91 08:08:02 EDT Subject: No subject Message-ID: <359394@MTS.cc.Wayne.edu> Please take me off the list. From koch at CitIago.Bitnet Tue Sep 10 16:12:27 1991 From: koch at CitIago.Bitnet (Christof Koch) Date: Tue, 10 Sep 91 13:12:27 PDT Subject: Conference announcement Message-ID: <910910131204.20404837@Iago.Caltech.Edu> CALL FOR PAPERS ********************************************************* Intelligent Vehicles `92 July I and 2, 1992, Radisson on the Lake Hotel near Detroit, USA ********************************************************* Organized by: IEEE/IES Intelligent Vehicle Subcommittee Cooperation with: American Society of Mechanical Engineers IEEE Vehicular Technology Society IEEE Neural Nets Council Japan Society for Fuzzy Theory and Systems Robotics Society of Japan Society of Automotive Engineers, Intemational Society of Automotive Engineers, Japan Society of Instrument and Control Engineers (Some of them listed above are in the application process and cooperations are not approved yet.) The IEEE/IES Intelligent Vehicle Subcommittee is organizing international meetings once every year. In 1991, for example, an international meeting will be held on "Fuzzy and Neural Systems, and Vehicle Applications" on November 8 and 9, 1991 in Tokyo. The meeting in 1990 was on "Vision- Based Vehicle Guidance". For 1992, we are planning to have multiple sessions. We will consider publishing a book, in addition to the proceedings, by selecting good papers presented in the special session as is the tradition of this workshop. This workshop will be held in conjunction with IROS '92 (International Conference on Intelligent Robots and Systems) which will be held in North Carolina from July 7, 1992. Topics: Real-Time Traffic Control (Special Session) Fuzzy Logic & Neural Nets for Vehicles Vision-Based Vehicle Guidance Other Related Issues including: Navigation Microwave Radar & Laser Radar Communication Architectures Advanced Electronics for Vehicles Deadlines: December 1, 1991, for one-page abstracts February 1, 1992, for acceptance notices April 1, 1992, for camera-ready papers If you would like to have your name on our mailing list, please write "Intelligent Vehicle" and/or "IROS" on the back of your business card (or a card with your address, phone, fax, and e-mail), and mail it to: Ichiro Masaki, Computer Science Department General Motors Research Laboratories 30500 Mound Road, Warren, Michigan, 48090-9055, USA Phone: (USA) 313-986-1466, FAX: (USA) 313-986-9356 CSNET:MASAKI at GMR.COM From jvillarreal at nasamail.nasa.gov Tue Sep 10 15:33:00 1991 From: jvillarreal at nasamail.nasa.gov (JAMES A. VILLARREAL) Date: Tue, 10 Sep 91 12:33 PDT Subject: unsubscribe Message-ID: Please remove me from the connectionist mailing list. From hinton at ai.toronto.edu Tue Sep 10 18:04:27 1991 From: hinton at ai.toronto.edu (Geoffrey Hinton) Date: Tue, 10 Sep 1991 18:04:27 -0400 Subject: whoops Message-ID: <91Sep10.180442edt.423@neuron.ai.toronto.edu> I said that K nearest neighbors does well at generalizing parity from a randomly sampled training set when K is small and even. This was a mistake. What I meant was that K nearest neighbors does well when K is set so that the modal hamming distance to the K nearest neighbors is small and even. For example, if the training set contains a fraction, p, of all possible input vectors of length n, then if we set K to be: np + n(n-1)p/2 we can expect about np neighbors one bit away and about n(n-1)p/2 neighbors two bits away. If n is not very small, there will be far more neighbors two bits away than one bit away, so generalization will be correct. The use of a validation set to fix K should therefore give good generalization for n not too small. Geoff From CADEPS at BBRNSF11.BITNET Wed Sep 11 08:31:35 1991 From: CADEPS at BBRNSF11.BITNET (JANSSEN Jacques) Date: Wed, 11 Sep 91 14:31:35 +0200 Subject: Neural evolution group Message-ID: <9B98FC01C4400068@BITNET.CC.CMU.EDU> Dear ConXnists, I hear there's a neural evolution email interest group. Could somebody please broadcast its email address and what its aims are. Cheers, Hugo de Garis. From lehman at pinkpanther.llnl.gov Wed Sep 11 10:39:09 1991 From: lehman at pinkpanther.llnl.gov (Sean Lehman) Date: Wed, 11 Sep 91 07:39:09 PDT Subject: more RBF refs In-Reply-To: Tomaso Poggio's message of Mon, 9 Sep 91 20:58:53 EDT <9109100058.AA00733@erice> Message-ID: <9109111439.AA01026@pinkpanther.llnl.gov> -> From: Tomaso Poggio -> Date: Mon, 9 Sep 91 20:58:53 EDT -> -> Why call gradient descent backpropagation? -> -> I think you are confusing gradient descent, a mathematical method for finding a local mininum, with backpropagation, a learning algorithm for artificial neural networks. -->skl (Sean K. Lehman) LEHMAN2 at llnl.gov lehman at tweety.llnl.gov (128.115.53.23) ("I tot I taw a puddy tat") From mre1 at it-research-institute.brighton.ac.uk Wed Sep 11 15:26:14 1991 From: mre1 at it-research-institute.brighton.ac.uk (Mark Evans) Date: Wed, 11 Sep 91 15:26:14 BST Subject: Literature Databases Message-ID: <17583.9109111426@itri.bton.ac.uk> A couple of times when people have requested information about a certain paper or book over the connectionists network, I have noted people producing the details for the article in the form a database entry. I would like to know if there are any databases for neural network literature (or general literature databases) that users from external sites can access. Thank you, Mark Evans ################################################# # # # Mark Evans mre1 at itri.bton.ac.uk # # Research Assistant mre1 at itri.uucp # # # # ITRI, # # Brighton Polytechnic, # # Lewes Road, # # BRIGHTON, # # E. Sussex, # # BN2 4AT. # # # # Tel: +44 273 642915/642900 # # Fax: +44 273 606653 # # # ################################################# From D.M.Peterson at computer-science.birmingham.ac.uk Wed Sep 11 17:01:49 1991 From: D.M.Peterson at computer-science.birmingham.ac.uk (D.M.Peterson@computer-science.birmingham.ac.uk) Date: Wed, 11 Sep 91 17:01:49 BST Subject: No subject Message-ID: I'm interested in any possible connection between 'spontaneity' in the philosophical theory of judgement and connectionism. The idea in the theory of judgement is that people engage in reasoning, inference, discussion etc. about some problem, and then a decision or solution emerges which is good, but is not an inevitable consequence of what preceeded it. The preceding reasoning prepares the way for the decision, but does not necessitate it. The decision or solution is a *result* of the preceeding thought processes etc., but is not strictly a logical consequence of them. So if we take law-governed logical deduction (or perhaps law-governed deterministic causation) as our model, it becomes hard to explain this everyday phenomenon. That, briefly, is the idea, and I'd be very grateful for any leads on any perspective on this or analagous cases to be found in connectionism. Please send replies to: D.M.Peterson at cs.bham.ac.uk From p-mehra at uiuc.edu Wed Sep 11 16:57:53 1991 From: p-mehra at uiuc.edu (Pankaj Mehra) Date: Wed, 11 Sep 91 16:57:53 EDT Subject: Exploiting duality to analyze ANNs Message-ID: <9109112157.AA26067@rhea> I had been waiting a long time to see a paper that will relate the geometry of function spaces with the statistical theory of approximation and estimation. I finally got what I was looking for in a recent paper in the journal Neural Networks (vol. 4, pp. 443-451, 1991) titled ``Dualistic Geometry of the Manifold of Higher-Order Neurons,'' by Amari. I thought I will begin searching for additional references by sharing a few pointers: 1. ``Applied Regression Analysis,'' (2nd ed.) by Draper and Smith pp 491, Chapter 10, An Intro to Nonlinear Estimation. The idea of sample spaces is introduced and the concepts of approximation and estimation errors explained in geometric terms. 2. ``Principled Constructive Induction,'' by [yours truly], Rendell, & Wah, Proc. IJCAI-89. (Extended abstract in Machine Learning Workshop, 1989.) Abstract ideas of Satosi Watanabe on object-predicate duality are given a concrete interpretation for learning systems. This paper introduces inverted spaces similar to sample spaces (somewhat customized for 2-class discrimination). A preliminary result relating the geometry and statistics of feature construction is proved. 3. ``Generalizing the PAC Model ...,'' by Haussler, in FOCS'89. The concept of combinatorial dimension, which measures the ability of a function class to cover the combinatorially many orthants of the sample space, is used for extending PAC learning ideas to analysis of continuous maps. [well, this is the way I interpret it] Amari's work presents (in my opinion) an elegant treatment of approximation theory. His proofs are limited to HONNs transforming bipolar (-1,+1) inputs. But he mentions technical reports describing extensions to Boltzmann machines. (Can someone at Univ. of Tokyo send me a copy?) (Also, can someone help me understand how eqn 3.2 of Amari's paper follows from eqn 3.1?) I'd like to hear about other approaches that exploit duality between feature space and function space to characterize the behavior of neural networks. -Pankaj Mehra Univ. Illinois From tp-temp at ai.mit.edu Wed Sep 11 23:30:09 1991 From: tp-temp at ai.mit.edu (Tomaso Poggio) Date: Wed, 11 Sep 91 23:30:09 EDT Subject: more RBF refs In-Reply-To: Sean Lehman's message of Wed, 11 Sep 91 07:39:09 PDT <9109111439.AA01026@pinkpanther.llnl.gov> Message-ID: <9109120330.AA01122@erice> I am not the one responsible for the confusion (i.e. why finding new names for old and perfectly well known things like gradient descent and chain rule?) From port at iuvax.cs.indiana.edu Thu Sep 12 03:02:49 1991 From: port at iuvax.cs.indiana.edu (Robert Port) Date: Thu, 12 Sep 91 02:02:49 -0500 Subject: conference on dynamic models Message-ID: DYNAMIC REPRESENTATION IN COGNITION November 14-17, 1991 (Thurs eve til Sun noon) Indiana University - Bloomington, Indiana INVITED SPEAKERS James Crutchfield (UC Berkeley, Mathematics) Jeffrey Elman (UC San Diego, Cognitive Science) Walter Freeman (UC Berkeley, Physiology-Anatomy) Paul van Geert (Groningen University, Psychology) Jordan Pollack (Ohio State, Computer Science) Jean Petitot (CNRS Paris, Mathematics) Elliot Saltzman (Haskins Laboratories) James Townsend (IU Bloomington, Psychology) Michael T. Turvey (University of Connecticut, Psychology) Cognition is a dynamic phenomenon. Cognitive processes are in continuous adaptive interaction with the changing environment. The cognizing system must deal in real time with a constantly changing environment. Many crucial features of the environment, such as the escape path of prey or an utterance in a natural language, have an extended temporal structure. Further, in development and learning, the system itself undergoes change. Yet cognitive science has traditionally tried to abstract away from the dynamic nature of cognition, using various strategies -- such as dividing time into discrete segments or taking cognitive processing to be the sequential manipulation of static representational structures. Increasingly, this approach is being challenged by researchers in a wide variety of fields who are building dynamics directly into their theories and models of cognitive processes. These include many who now believe that dynamical systems theory is a more appropriate mathematical framework for the study of cognition than symbolic computation. A radical new conception of mental representation is gradually emerging: representations might hemselves be dynamic structures such as trajectories in a system state space. There are now many concrete examples of dynamical models of cognitive phenomena in areas such as motor control, olfaction and language processing. The aim of this workshop-style conference is to bring together many key researchers, to share perspectives from diverse areas of cognitive modeling, and to provide plenty of time to discuss the foundational issues in genuinely dynamical conceptions of cognition. ORGANIZING COMMITTEE Robert Port (Linguistics and Computer Science), co-chair port at cs.indiana.edu ph:(812)-855-9217 Timothy van Gelder (Philosophy), co-chair tgelder at ucs.indiana.edu ph:(812)-855-7088 Geoffery Bingham (Psychology), Linda Smith (Psychology), Esther Thelen (Psychology), James Townsend (Psychology) POSTER SESSION There will be a poster session Friday evening for work related to these issues. Posters will remain on display throughout the conference. Please submit your poster abstract before October 15, 1991. REGISTRATION FEE = $50 ($20 for students) CONFERENCE LIMIT = 120 persons FOR FURTHER INFORMATION US MAIL: Conference Registrar | EMAIL: MMACKENZ at UCS.INDIANA.EDU IU Conference Bureau, IMU Room 677 | PHONE: (812)-855-4661 Bloomington, IN 47406 | FAX: (812)-855-8077 SPONSORED BY: Office of Naval Research, IU Institute of the Study of Human Capabilities, Departments of Philosophy and Linguistics, and the Cognitive Science Program. From lo at vaxserv.sarnoff.com Thu Sep 12 09:19:49 1991 From: lo at vaxserv.sarnoff.com (Leonid Oliker x2419) Date: Thu, 12 Sep 91 09:19:49 EDT Subject: No subject Message-ID: <9109121319.AA17121@sarnoff.sarnoff.com> Please remove me from the connectionist mailing list. From sch at ee.UManitoba.CA Thu Sep 12 11:10:17 1991 From: sch at ee.UManitoba.CA (sch@ee.UManitoba.CA) Date: Thu, 12 Sep 91 10:10:17 CDT Subject: unsubscribe Message-ID: <9109121510.AA00183@ic14.ee.umanitoba.ca> Please take me off the list. From reznik at cs.wisc.edu Thu Sep 12 13:14:58 1991 From: reznik at cs.wisc.edu (Dan S. Reznik) Date: Thu, 12 Sep 91 12:14:58 -0500 Subject: request for removal Message-ID: <9109121714.AA01681@tao.cs.wisc.edu> please remove me from this list. thanks. dan reznik From ross at psych.psy.uq.oz.au Thu Sep 12 17:43:25 1991 From: ross at psych.psy.uq.oz.au (Ross Gayler) Date: Fri, 13 Sep 1991 07:43:25 +1000 Subject: interpolation vs generalisation Message-ID: <9109122143.AA03888@psych.psy.uq.oz.au> The people following this thread might want to consider where analogical inference fits in. Analogical inference is a form of generalisation that is performed on the basis of structural or relational similarity rather than literal similarity. It is generalisation, because it involves the application of knowledge from previously encountered situations to a novel situation. However, the interpolation does not occur in the space defined by the input patterns, instead it occurs in the space describing the structural relationships of the input tokens. The structural relationships between any set of inputs is not necessarily fixed by those inputs, but generated dynamically as an 'interpretation' that ties the inputs to a context. There is an argument that analogical inference is the basic mode of retrieval from memory, but most connectionist research has focused on the degenerate case where the structural mapping is an identity mapping - so the interest is focused on interpolation in the input space instead of the structural representation space. In brief: Generalisation can occur without interpolation in a data space that you can observe, but it may involve interpolation in some other space that is constructed internally and dynamically. Ross Gayler ross at psych.psy.uq.oz.au From rudnick at ogicse Thu Sep 12 20:03:57 1991 From: rudnick at ogicse (Mike Rudnick) Date: Thu, 12 Sep 1991 17:03:57 PDT Subject: Neural evolution group In-Reply-To: JANSSEN Jacques "Neural evolution group" (Sep 12, 1:21) Message-ID: <9109130003.AA09392@cse.ogi.edu> Neuro-evolution is a forum for discussion of technical issues relating to using genetic algorithms (GAs) and evolutionary approachs for the design of artificial neural networks. Other GA/ANN topics are also welcome. Postings of abstracts, notices of availability of tech reports and papers, references, general discussion, and the like are welcome. Send requests to have your name added to the distribution list to neuro-evolution-request at cse.ogi.edu. Mike Rudnick From meyer at FRULM63.BITNET Fri Sep 13 10:49:54 1991 From: meyer at FRULM63.BITNET (meyer) Date: Fri, 13 Sep 91 16:49:54 +0200 Subject: A new journal Message-ID: <9109131449.AA16028@wotan.ens.fr> ============================= Call for papers ============================== A D A P T I V E B E H A V I O R An international journal devoted to experimental and theoretical research on adaptive behavior in animals and in autonomous artificial systems, with emphasis on mechanisms, organizational principles, and architectures that can be expressed in computational, physical, or mathematical models. Broadly, behavior is adaptive if it deals successfully with changed circumstances. The adapting entities may be individuals or populations, over short or long time scales. The journal will publish articles, reviews, and short communications that treat the following topics, among others, from the perspective of adaptive behavior. Perception and motor control Ontogeny, learning and evolution Motivation and emotion Action selection and behavioral sequences Internal world models and cognitive processes Architectures, organizational principles, and functional approaches Collective behavior Characterization of environments Among its scientific objectives, the Journal aims to emphasize an approach complementary to traditional AI, in which basic abilities that allow animals to survive, or robots to perform their mission in unpredictable environments, will be studied in preference to more elaborated and human-specific abilities. The Journal also aims to investigate which new insights into intelligence or cognition can be achieved by explicitly taking into account the environmental feedback --mediated by behavior--that an animal or a robot receives, instead of studying components of intelligence in isolation. The journal will be published quarterly, beginning with the Summer issue of 1992. EDITOR-IN-CHIEF Jean-Arcady Meyer (Ecole Normale Superieure, France) email: meyer at wotan.ens.fr meyer at frulm63.bitnet tel: (1) 43 29 12 25 ext 3623 fax: (1) 43 29 70 85 ASSOCIATE EDITORS Randall Beer (Case Western Reserve Univ., USA) Lashon Booker (MITRE Corp., USA) Jean-Louis Deneubourg (Univ. of Bruxelles, Belgium) Janet Halperin (Univ. of Toronto, Canada) Pattie Maes (MIT Media Lab., USA) Herbert Roitblat (Univ. of Hawaii, USA) Ronald Williams (Northeastern University, USA) Stewart Wilson (The Rowland Institute for Science, USA). EDITORIAL BOARD David Ackley (Bellcore, USA) Michael Arbib (Univ. South. California, USA) Andrew Barto (Univ. of Massachusetts, USA) Richard Belew (Univ. of California, USA) Rodney Brooks (MIT AI Lab., USA) Patrick Colgan (Canadian Museum of Nature, Canada) Holk Cruse (Univ. Bielefeld, Germany) Daniel Dennett (Tufts Univ., USA) Jorg-Peter Ewert (Univ. Kassel, Germany) Nicolas Franceschini (Univ. Marseille, France) David Goldberg (Univ. of Illinois, USA) John Greffenstette (Naval Research Lab., USA) Patrick Greussay (Univ. Paris 8, France) Stephen Grossberg (Center for Adaptive Systems, USA) John Holland (Univ. Michigan, USA) Keith Holyoak (Univ. California, USA) Christopher Langton (Los Alamos National Lab., USA) David McFarland (Univ. of Oxford, UK) Thomas Miller (Univ. of New Hampshire, USA) Norman Packard (Univ. of Illinois, USA) Tim Smithers (Edinburgh Univ., UK) Luc Steels (VUB AI Lab., Belgium) Richard Sutton (GTE Labs., USA) Frederick Toates (The Open University, UK) David Waltz (Thinking Machines Corp., USA) To be published, an article should report substantive new results that significantly advance understanding of adaptive behavior. Critical reviews of existing work will also be considered. Contributions will originate from a range of disciplines including robotics, artificial intelligence, connectionism, classifier systems and genetic algorithms, psychology and cognitive science, behavioral ecology, and ethology among others. Ideally, an article will suggest implications for both natural and artificial systems. Authors should aim to make their results, and the results' significance, clear and understandable to the Journal's multi- disciplinary readership. Very general, speculative, or narrowly specialized papers, papers with substantially incomplete conceptual, experimental, or computational results, or papers irrelevant to the subject of adaptive behavior may be returned to authors without formal review. Submissions should be sent to: Dr. Jean-Arcady Meyer, Editor Adaptive Behavior Groupe de BioInformatique Ecole Normale Superieure 46 rue d'Ulm 75230 Paris Cedex05 FRANCE Please send five (5) copies of all materials. Manuscripts must be in English, with American spelling preferred. Please briefly define terms that may not be familiar outside your specialty. Avoid jargon and non-standard abbreviations. Make every attempt to employ technical terms that are already in use before making up new ones. The following guidelines should be adhered to, or papers may be returned for reformatting prior to review. Double-space all materials. Manuscripts should be typed (or laser printed) on 8 1/2 x 11 inch or A4 paper, one side only, with one-inch margins all around. Every page should be numbered in the upper right hand corner starting with the title page. Manuscript length should not normally exceed the equivalent of twenty journal pages. The title page (page 1) should have: - the paper's title, preferably not too long - the names, affiliations, and complete addresses of the authors, including electronic mail addresses if available - a daytime telephone number for the author with whom the editors should correspond. The second page should contain an abstract of 200 words or less, a list of six or fewer key words, and a shortened title for use as a running head. Begin the text of the article on page 3. Aid the reader by dividing the text into logical sections and subsections. Footnotes may be used sparingly. Follow the text with acknowledgements on a separate page. Begin the reference list on a new page following the acknowledgements page. References and citations should conform to the APA Publication Manual except: (1) do not cite page numbers of any book; (2) use the same format for unpublished references as for published ones. Please carefully check citations and references to be sure thay are correct and consistent. Note that the names of all authors of a publication should be given in the reference list and the first time it is cited in the text; after that "et al." may be used in citations. If a publication has 3 or more authors, "et al." may also be used in the first citation unless ambiguity would result. Include figures and tables at the end of the manuscript. Number them consecutively using Arabic numerals. Include a brief title above each table and a caption below each figure. Indicate in the text an approximate position for each figure and table. Besides graphical material, figures consisting of high quality black and white photographs are acceptable. Submit only clear reproductions of artwork. Authors should retain original artwork until the final version of the manuscript has been accepted. No page charges will be levied. Authors may order reprints when corrected proofs are returned. For subscription information, please contact: MIT Press Journals Circulation Department 55 Hayward Street Cambridge, Ma 02142 tel: 617-253-2889 fax: 617-258-6779 From sussmann at hamilton.rutgers.edu Fri Sep 13 13:02:11 1991 From: sussmann at hamilton.rutgers.edu (sussmann@hamilton.rutgers.edu) Date: Fri, 13 Sep 91 13:02:11 EDT Subject: more RBF refs Message-ID: <9109131702.AA08973@hamilton.rutgers.edu> > Date: Wed, 11 Sep 91 07:39:09 PDT > From: Sean Lehman > > I think you are confusing gradient descent, a mathematical method for > finding a local mininum, with backpropagation, a learning algorithm > for artificial neural networks. I don't quite understand the distinction. Backpropagation is of course "a learning algorithm for artificial neural networks," but it consists of using gradient descent to look for a local minimum of a function. (And, yeah, to compute the gradient one uses the chain rule.) You seem to be saying that, because backpropagation is not gradient descent in general, but gradient descent in a special case, then it's not gradient descent. Or am I missing something? ---Hector Sussmann From watrous at cortex.siemens.com Fri Sep 13 17:14:26 1991 From: watrous at cortex.siemens.com (Ray Watrous) Date: Fri, 13 Sep 91 17:14:26 EDT Subject: Poor Taste Message-ID: <9109132114.AA12637@cortex.siemens.com.noname> I find offensive this practice of unsubscribing to the connectionists list with a petulant note to the whole readership. If people have lost interest, why don't they leave quietly via Connectionists-Request? If they are dissatisfied, and want everyone to know it, they should grow up and act more constructively. Ray Watrous From GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca Sat Sep 14 23:33:57 1991 From: GOLDFARB%unb.ca at UNBMVS1.csd.unb.ca (GOLDFARB%unb.ca@UNBMVS1.csd.unb.ca) Date: Sun, 15 Sep 91 00:33:57 ADT Subject: No subject Message-ID: From mozer at dendrite.cs.colorado.edu Sat Sep 14 15:45:17 1991 From: mozer at dendrite.cs.colorado.edu (Mike Mozer) Date: Sat, 14 Sep 91 13:45:17 -0600 Subject: tech report announcement Message-ID: <199109141945.AA20028@neuron.cs.colorado.edu> Sorry to disappoint you, but this is not another request to be removed from the mailing list. Please do not forward this announcement to other boards. Thank you. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- LEARNING TO SEGMENT IMAGES USING DYNAMIC FEATURE BINDING Michael C. Mozer, Richard S. Zemel, and Marlene Behrmann Despite the fact that complex visual scenes contain multiple, overlapping objects, people perform object recognition with ease and accuracy. One operation that facilitates recognition is an early segmentation process in which features of objects are grouped and labeled according to which object they belong. Current computational systems that perform this operation are based on predefined grouping heuristics. We describe a system called MAGIC that _learns_ how to group features based on a set of presegmented examples. In many cases, MAGIC discovers grouping heuristics similar to those previously proposed, but it also has the capability of finding nonintuitive structural regularities in images. Grouping is performed by a relaxation network that attempts to dynamically bind related features. Features transmit a complex-valued signal (amplitude and phase) to one another; binding can thus be represented by phase locking related features. MAGIC's training procedure is a generalization of back propagation to complex-valued units. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- This TR has been placed in the Neuroprose archive at Ohio State. Instructions for its retrieval are given below. If you are unable to retrieve and print the TR and therefore wish to receive a hardcopy, please send mail to conn_tech_report at cs.colorado.edu. Please do not reply to this message. -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- FTP INSTRUCTIONS NOTE CHANGE OF HOSTNAME FROM cheops TO archive ---------------------------------------------- unix> ftp archive.cis.ohio-state.edu (or 128.146.8.52) Name: anonymous Password: neuron ftp> cd pub/neuroprose ftp> binary ftp> get mozer.segment.ps.Z ftp> quit unix> zcat mozer.segment.ps.Z | lpr From PHARMY at SUMMER.CHEM.SU.OZ.AU Sun Sep 15 20:32:15 1991 From: PHARMY at SUMMER.CHEM.SU.OZ.AU (PHARMY@SUMMER.CHEM.SU.OZ.AU) Date: Mon, 16 Sep 1991 10:32:15 +1000 (EST) Subject: morphology Message-ID: <910916103215.2720065a@SUMMER.CHEM.SU.OZ.AU> hi I am trying to locate a program that does fractal analysis of cell images which is compatible with a IBM or IBM clone. I'd also be greatful for any replies relating to mailing lists dealing with information processing of single neurons in relation to the observed morphology. thank you herbert Jelinek From jim at hydra.maths.unsw.OZ.AU Sun Sep 15 20:15:52 1991 From: jim at hydra.maths.unsw.OZ.AU (jim@hydra.maths.unsw.OZ.AU) Date: Mon, 16 Sep 91 10:15:52 +1000 Subject: interpolation vs generalisation Message-ID: <9109160015.AA23675@hydra.maths.unsw.OZ.AU> What is the `argument that analogical inference is the basic mode of retrieval from memory'? Jim Franklin From marek at iuvax.cs.indiana.edu Mon Sep 16 09:41:47 1991 From: marek at iuvax.cs.indiana.edu (Marek W Lugowski) Date: Mon, 16 Sep 91 08:41:47 -0500 Subject: interpolation vs generalisation Message-ID: would Pentti Kanerva's model of associative memory fit in with your definition of analogical inference? It seems that the input of successive binary vectors to his distributed memory creats a structure in Hamming space, often creating never-before seen resident vectors... which are then evoked. -- Marek P.s. On the other hand, Jim Keeler shows that Kanerva memory and Hopfield nets can have a common notation, so I am confused. After all Hoopfield nets are connectionist. From tackett at ipla00.hac.com Sun Sep 15 11:17:57 1991 From: tackett at ipla00.hac.com (Walter Tackett) Date: Sun, 15 Sep 91 11:17:57 EDT Subject: more RBF refs Message-ID: <9109151817.AA23567@ipla00.ipl.hac.com> uh.... you got the wrong guy. excuse me, person. My posting regarded how to build a many-class classifier, since bp seems to break down rather severely as the number of classes increases. btw, i have been severely dissapointed at the lack of responses. none, to be specific. the mail i rec'd from you regarded rbf's & gradient descent. -thanks, anyway.... walter hughes aircraft and the university of southern california base all policy decisions solely on my opinion. watch out. From hanna at cs.uq.oz.au Sun Sep 15 20:10:33 1991 From: hanna at cs.uq.oz.au (hanna@cs.uq.oz.au) Date: Mon, 16 Sep 91 10:10:33 +1000 Subject: unsubscribe Message-ID: <9109160010.AA08192@client> Please UNSUBSCRIBE From tedwards at wam.umd.edu Mon Sep 16 12:06:18 1991 From: tedwards at wam.umd.edu (Thomas VLSI Edwards) Date: Mon, 16 Sep 91 12:06:18 EDT Subject: backprop vs. gradient descent Message-ID: <9109161606.AA03424@next07pg2.wam.umd.edu> -> From: Tomaso Poggio -> Date: Mon, 9 Sep 91 20:58:53 EDT -> -> Why call gradient descent backpropagation? -> -> -->skl (Sean K. Lehman) LEHMAN2 at llnl.gov lehman at tweety.llnl.gov (128.115.53.23) ("I tot I taw a puddy tat") says.. >I think you are confusing gradient descent, a mathematical >method for finding a local mininum, with backpropagation, a learning >algorithm for artificial neural networks. Probably the best way to deal with this is to consider backpropagation as a manner of obtaining the error gradient of a neural net with multiplicative weights and a typically non-linear transfer function (as you must propagate the error back through the net to obtain the gradient, even if you use it for something like conjugate gradient). Extending the definition of backpropagation to include delta weight=learning rate times gradient confuses people, although it should be mentioned that this simple gradient descent method was often used in early backpropagation applications. -Thomas Edwards From p-mehra at uiuc.edu Mon Sep 16 11:49:05 1991 From: p-mehra at uiuc.edu (Pankaj Mehra) Date: Mon, 16 Sep 91 11:49:05 EDT Subject: Exploiting duality to analyze ANNs Message-ID: <9109161649.AA14738@manip> Several people requested a PostScript copy of the following citation from my previous note: - - ------ 2. ``Principled Constructive Induction,'' by Mehra, Rendell, & Wah, Proc. IJCAI-89. (Extended abstract in Machine Learning Workshop, 1989.) Abstract ideas of Satosi Watanabe on object-predicate duality are given a concrete interpretation for learning systems. This paper introduces inverted spaces similar to sample spaces (somewhat customized for 2-class discrimination). A preliminary result relating the geometry and statistics of feature construction is proved. - - ------ A REVISED version of this paper is now available from the neuroprose archive. The IJCAI paper contained a few oversights; these have been fixed in the recent revision. - - -Pankaj +++++++ HERE'S HOW TO GET THE PAPER +++++++ ftp archive.cis.ohio-state.edu (128.146.8.52) Connected to 128.146.8.52. 220 archive FTP server (SunOS 4.1) ready. Name (128.146.8.52:you): anonymous 331 Guest login ok, send ident as password. Password: neuron 230 Guest login ok, access restrictions apply. ftp> cd pub/neuroprose 250 CWD command successful. ftp> binary 200 Type set to I. ftp> get mehra.duality.ps.Z 200 PORT command successful. 150 Binary data connection for mehra.duality.ps.Z (128.174.31.18,1296) (48242 by tes). 226 Binary Transfer complete. local: mehra.duality.ps.Z remote: mehra.duality.ps.Z 48242 bytes received in 6.4 seconds (7.3 Kbytes/s) ftp> quit 221 Goodbye. uncompress mehra.duality.ps lpr -P mehra.duality.ps From qin at eng.umd.edu Mon Sep 16 15:45:20 1991 From: qin at eng.umd.edu (Si-Zhao Qin) Date: Mon, 16 Sep 91 15:45:20 -0400 Subject: Authors Kuhn & Herzberg Message-ID: <9109161945.AA21518@cm14.eng.umd.edu> I need the authors' initials for the following paper: Kuhn & Herzberg, "Variations on Training of Recurrent Networks", from 24th Conference on Information Sciences and Systems, Princeton, N.J., 3/21/90. I would appreciate if anybody could find the authors' full names. Thanks. Joe at qin at eng.umd.edu From karit at spine.hut.fi Tue Sep 17 11:06:45 1991 From: karit at spine.hut.fi (Kari Torkkola) Date: Tue, 17 Sep 91 11:06:45 DST Subject: RESEARCH POSITIONS IN SPEECH PROCESSING IN SWITZERLAND Message-ID: <9109170806.AA01412@spine.hut.fi.hut.fi> RESEARCH POSITIONS AVAILABLE IN SPEECH PROCESSING The newly created "Institut Dalle Molle d'Intelligence Artificielle Perceptive" (IDIAP) in Martigny, Switzerland seeks to hire qualified researchers in the area of automatic speech recognition. Candidates should be able to conduct independent research in a UNIX environment on the basis of solid theoretical and applied knowledge. Salaries will be aligned with those offered by the Swiss government for equivalent positions. Researchers are expected to begin activity in the beginning of 1992. IDIAP is supported by the Dalle Molle Foundation along with public-sector partners at the local and federal levels (in Switzerland). IDIAP is the third institute of artificial intelligence supported by the Dalle Molle Foundation, the others being ISSCO (attached to the University of Geneva) and IDSIA (situated in Lugano). The new institute maintains close contact with these latter centers as well as with the Polytechnical School of Lausanne and the University of Geneva. Applications for a research position at IDIAP should include the following elements: - a curriculum vitae - sample publications or technical reports - a brief description of the research programme that the candidate wishes to pursue - a list of personal references. Applications are due by December 1, 1991 and may be sent to the address below: Daniel Osherson IDIAP Case Postale 609 CH-1920 Martigny SWITZERLAND For further information by e-mail, contact: osherson at idiap.ch (Daniel Osherson, director) or karit at idiap.ch (Kari Torkkola, researcher) Please use the latter email address only for inquiries concerning speech recognition research. From cutrer at bend.UCSD.EDU Tue Sep 17 11:42:11 1991 From: cutrer at bend.UCSD.EDU (Michelle Cutrer) Date: Tue, 17 Sep 91 08:42:11 PDT Subject: unsubscribe Message-ID: <9109171542.AA26240@bend.UCSD.EDU> unsubscribe From ai-vie!georg at relay.EU.net Tue Sep 17 13:29:58 1991 From: ai-vie!georg at relay.EU.net (Georg Dorffner) Date: Tue, 17 Sep 91 19:29:58 +0200 Subject: meeting: connectionism and cognition Message-ID: <9109171729.AA05125@ai-vie.uucp> ! ! ! Call for Papers ! ! ! Symposium on CONNECTIONISM AND COGNITIVE PROCESSING as part of the Eleventh European Meeting on Cybernetics and Systems Research (EMCSR) April 21 - 24, 1992 Vienna, Austria Chairs: Noel Sharkey (Univ. of Exeter) Georg Dorffner (Univ. of Vienna) After the successes of two sessions on Parallel Distributed Processing at the previous EMCSRs, this symposium is designed to keep abreast with the increasing importance of connectionism (neural networks) in artificial intelligence, cognitive science, as well as in neuroscience and the philosophy and psychology of the mind. Therefore, original papers from all these areas are invited. Descriptions of implemented models are as welcome as theoretical or application-oriented contributions. Papers must not exceed 7 single-spaced A4 pages (max. 50 lines, final size will be 8.5 x 6 inch) and be written in English. They must contain the final text to be submitted, including graphs and figures (these need not be of reducible quality). Please send t h r e e copies of each submission. The deadline for submissions is Oct 15, 1991 (postmarked). However, if a brief note of intent to submit a paper (containing the tentative title) is emailed to georg at ai-vie.uucp (alternatively to georg%ai-vie.uucp at relay.eu.net) by the above date, papers can be sent until Nov 1, 1991 (postmarked) Authors will then be notified about acceptance within three to four weeks. Authors of accepted papers will be provided detailed instructions for the final format for the proceedings to be published by the time of the conference. Send all submissions - marked with the letter 'N' for the connectionist symposium - to EMCSR conference secretariat Oesterreichische Studiengesellschaft fuer Kybernetik Schottengasse 3 A-1010 Vienna, Austria or inquire about more details at the same address, at tel: +43 1 535 32 810 or at email: georg at ai-vie.uucp ------------------------------------------------------ other symposia at the EMCSR will be: A: General Systems Methodology B: Mathematical Systems Theory C: Computer Aided Process Interpretation D: Fuzzy Sets, Approximate Reasoning and Knowledge-Based Systems E: Designing and Systems F: Humanity, Architecture and Conceptualization G: Biocybernetics and Mathematical Biology H: Systems and Ecology I: Cybernetics in Medicine J: Cybernetics of Socio-Economic Systems K: Systems, Management and Organization L: Cybernetics of National Development M: Communication and Computers O: Intelligent Autonomous Systems P: Telepresence, Virtual Environments, and Interactive Fantasy Q: Impacts of Artificial Intelligence R: History of Cybernetics and Systems Research Submissions to these symposia can also be sent to the secretariat no later than Oct 15, 1991 (the above extended deadline only applies to the connectionist meeting). From jfj at m53.limsi.fr Wed Sep 18 04:17:59 1991 From: jfj at m53.limsi.fr (Jean-Francois Jadouin) Date: Wed, 18 Sep 91 10:17:59 +0200 Subject: TIme-unfolding Message-ID: <9109180817.AA16735@m53.limsi.fr> Dear connectionists, I've been doing a little work with Time-Unfolding Networks (first mentioned, I think, in PDP. Paul Werbos has an article out on the technique in the proceedings of the IEEE 1990). I've been getting pretty terrible results. My intuition is that I've misunderstood the model. Does anyone out there use this model ? If so, would you be prepared to exchange benchmark results (or even better, software) and compare notes ? A little discouraged, jfj From jfj at m53.limsi.fr Wed Sep 18 04:57:13 1991 From: jfj at m53.limsi.fr (Jean-Francois Jadouin) Date: Wed, 18 Sep 91 10:57:13 +0200 Subject: Bibliography to a novice ... Message-ID: <9109180857.AA16836@m53.limsi.fr> The most-often cited reference I know of, and one you should definitely read is: J. McClelland, D. Rumelhart, Parallel Distributed Processing vols 1 & 2, MIT press, 1986. The following is a pretty complete collection of the most influential papers in the field, Anderson, Rosenfeld, Neurocomputing, foundations of research, MIT press, 1989. Both these references are from memory and may be a little off - my apologies for this. Oh yes: Artificial Intelligence (# 40, I think), had a special issue on learning, where Hinton wrote a good introduction on connectionist learning procedures. Good luck ! jfj From pollack at cis.ohio-state.edu Wed Sep 18 11:48:22 1991 From: pollack at cis.ohio-state.edu (Jordan B Pollack) Date: Wed, 18 Sep 91 11:48:22 -0400 Subject: Neuroprose Message-ID: <9109181548.AA01754@dendrite.cis.ohio-state.edu> ***Do not forward to other bboards*** As many of you are aware by now, the cis facility at OSU has been unstable the last week of August and first week of September. One of the changes which occured in the spring was that the cheops pyramid computer was replaced with an alias to a new sun server called: archive.cis.ohio-state.edu 128.146.8.52 (Cheops was ..62) One of the changes which recently happened is that the alias mechanism has become unreliable, so transparent access through the "cheops" name has repeatedly failed for people. Please change your Getps and neuroprose scripts accordingly. Also, I'd like to remind the LIST that Neuroprose is a convenience mechanism for the low-cost distribution of preprints and reprints to the MAILING LIST. Neuroprose is NOT a vanity press -- placing a report or book in neuroprose does not constitute publication. Many of the papers in neuroprose are reprints of journal articles and book chapters, THEREFORE NEUROPROSE IS NOT PUBLIC DOMAIN SOFTWARE. While I can appreciate some people wanting to share their results with the highest possible readership, the forwarding of FTP announcements to neuron-digest and/or comp.ai.neural networks will eventually cause major legal problems for somebody. So, when you announce a paper, please include "**do not forward**"'s in the message. Finally, please halt suggestions that the papers be copied all over the world, distributed on floppy disk, or accessed through various public servers. "Don't call me Kinko's" Pollack Assistant Professor CIS Dept/OSU Laboratory for AI Research 2036 Neil Ave Email: pollack at cis.ohio-state.edu Columbus, OH 43210 Phone: (614)292-4890 (then * to fax) ***Do not forward to other ML's or Newsgroups*** From kddlab!hcrlgw.crl.hitachi.co.jp!nitin at uunet.UU.NET Thu Sep 19 14:36:35 1991 From: kddlab!hcrlgw.crl.hitachi.co.jp!nitin at uunet.UU.NET (Nitin Indurkhya) Date: Thu, 19 Sep 91 14:36:35 JST Subject: DP-matching with NN-hardware Message-ID: <9109190536.AA01532@hcrlgw.crl.hitachi.co.jp> one of the problems i'm facing in analyzing time-series data with DP-matching methods is that when the number of templates is very large, the run-time performance is quite poor (i.e. it is definitely not possible to do it in real-time). i recall reading somewhere about DP-matching with NN-hardware. can someone give me a reference on that. also, any pointers to more recent work in this area would be appreciated. i am using the one-stage dp method discussed in the excellent article by hermann ney. thanks in advance, --nitin indurkhya (nitin at crl.hitachi.co.jp) From tackett at ipla00.hac.com Thu Sep 19 08:56:55 1991 From: tackett at ipla00.hac.com (Walter Tackett) Date: Thu, 19 Sep 91 08:56:55 EDT Subject: Bibliography to a novice ... Message-ID: <9109191556.AA07873@ipla00.ipl.hac.com> i would add to jfj's list that IEEE computer had a real good special issue g that contained overview articles concerning ART, BP, Neocog, BAM, Hopfield, etc. And some kohonen, as i recall, by the original researchers. this was in late 88 or early 89. -walter From Renate_Crowley at unixgw.siemens.com Thu Sep 19 17:35:48 1991 From: Renate_Crowley at unixgw.siemens.com (Renate Crowley) Date: 19 Sep 91 17:35:48 Subject: NIPS 1991 submissions Message-ID: <9109192139.AA11973@siemens.siemens.com> Subject: Time:17:42 OFFICE MEMO NIPS 1991 submissions Date:9/19/91 For your information notifications regarding acceptance/rejection have been mailed to all first authors or corresponding authors last week in August. If you have not received a letter please contact me. Renate Crowley Tel 609 734 3311 Fax 609 734 6565 email: renate at siemens.siemens.com From kddlab!hcrlgw.crl.hitachi.co.jp!nitin at uunet.UU.NET Fri Sep 20 10:15:57 1991 From: kddlab!hcrlgw.crl.hitachi.co.jp!nitin at uunet.UU.NET (Nitin Indurkhya) Date: Fri, 20 Sep 91 10:15:57 JST Subject: Ney reference Message-ID: <9109200115.AA14595@hcrlgw.crl.hitachi.co.jp> i've rec'd many requests for a reference to the Ney paper: Hermann Ney, "The use of a one-stage dynamic programming algorithm for connected word recognition", ieee trans. assp 32(2):263-271, apr 1984 --nitin (nitin at crl.hitachi.co.jp) From 7923509%TWNCTU01.BITNET at BITNET.CC.CMU.EDU Fri Sep 20 13:45:00 1991 From: 7923509%TWNCTU01.BITNET at BITNET.CC.CMU.EDU (7923509%TWNCTU01.BITNET@BITNET.CC.CMU.EDU) Date: Fri, 20 Sep 91 13:45 U Subject: Could some one tell me ... ? Message-ID: <01GASD3QJRA8D7PO30@BITNET.CC.CMU.EDU> Hi: Could anyone tell me the following question ? 1.What is the reason that the states of the k th unit Sk = 1 with probability 1 Pk = ------------------- 1+exp(-delta Ek/T) where delta Ek is the energy gap between the 1 and 0 states of the k th unit, T is a parameter which acts like the temperature 2.Why this local decision rule ensures that in thermal equilibrium the relative probability of two global states is determined by their energy difference,and follows a Boltzmann distribution: Pa ---- = exp(-(Ea - Eb))/T Pb where Pa is the probability of being in the a th global state and Ea is the energy of that state. and how do we know that whether the system reaches thermal equilibrium or not? reply should be send to 7923509 at twnctu01 thank's a lot! From david at cns.edinburgh.ac.uk Fri Sep 20 11:53:09 1991 From: david at cns.edinburgh.ac.uk (David Willshaw) Date: Fri, 20 Sep 91 11:53:09 BST Subject: NETWORK Message-ID: <3796.9109201053@subnode.cns.ed.ac.uk> CONTENTS OF NETWORK - COMPUTATION IN NEURAL SYSTEMS Volume 2 Number 3 August 1991 LETTER TO THE EDITOR On the capacity of a neuron with a non-monotone output function. K Kobayashi PAPERS Realistic synaptic inputs for model neural networks. L F Abbott Quantitative study of attractor neural network retrieving at low spike rates: I. Substrate-spikes, rates and neuronal gain. D J Amit and M V Tsodyks Quantitative study of attractor neural network retrieving at low spike rates: II. Low-rate retrieval in symmetric networks. D J Amit and M V Tsodyks Dynamics of an auto-associative neural network model with arbitrary connectivity and noise in the threshold. H F Yanai, Y Sawada and S Yoshizawa A high storage capacity neural network content-addressable memory. E Hartman ABSTRACTS SECTION BOOK REVIEWS --------------------------------------------------------------------- Network welcomes research Papers and Letters where the findings have demonstrable relevance across traditional disciplinary boundaries. Research Papers can be of any length, if that length can be justified by content. Rarely, however, is it expected that a length in excess of 10,000 words will be justified. 2,500 words is the expected limit for research Letters. Articles can be published from authors' TeX source codes. Macros can be supplied to produce papers in the form suitable for refereeing and for IOP house style. For more details contact the Editorial Services Manager at IOP Publishing, Techno House, Redcliffe Way, Bristol BS1 6NX, UK. Telephone: 0272 297481 Fax: 0272 294318 Telex: 449149 INSTP G Email Janet: IOPPL at UK.AC.RL.GB Subscription Information Frequency: quarterly Subscription rates: Institution 125.00 pounds (US$220.00) Individual (UK) 17.30 pounds (Overseas) 20.50 pounds (US$37.90) A microfiche edition is also available at 75.00 pounds (US$132.00) From ross at psych.psy.uq.oz.au Fri Sep 20 08:31:54 1991 From: ross at psych.psy.uq.oz.au (Ross Gayler) Date: Fri, 20 Sep 1991 22:31:54 +1000 Subject: analogy, generalisation & interpolation (LONG!) Message-ID: <9109201231.AA20896@psych.psy.uq.oz.au> One week ago I made a posting that attempted to place analogical inference into the debate on interpolation and generalisation. I have received a number of replies (some direct and some via the mailing list) with sufficient overlap to justify a mass reply via the mailing list - so here it is. - References to my own work in this area Sadly, there are none. Designing a connectionist architecture to support dynamic analogical inference and retrieval is my night job, so I lack the time and resources to produce papers. Most of my work has been in scribbled notes and thought-experiments. My day job is mundane stuff in the computer industry, but I am searching for a new job right now and would love to convert my night job to a day one - so all offers will be considered. - References to other connectionist work on analogical inference The classic is: Holyoak, K.J., & Thagard, P. (1989). Analogical mapping by constraint satisfaction. Cognitive Science, 13(3), 295-355. They produced a program (ACME) that performs analogical inference by a connectionist constraint satisfaction network. The program takes two *symbolic* structures (base and target) and attempts to find a consistent mapping from base to target to fill some gaps in the target structure. For example, the base structure might describe a planetary system and the target structure describe a Rutherford atom. The program tries to map objects and predicates from the base into the target. The fun starts when the structures are not isomorphic, so there is ambiguity as to what is the 'best' mapping. ACME uses a symbolic program to parse the symbolic inputs and create a connectionist net to solve the constraint satisfaction problem. The net uses a localist representation (each unit corresponds to a partial mapping) and the weight between the units encode constraints on combining the partial mappings. After the net has been constructed it is set in action and the 'best' mapping read from the settled state. The Holyoak and Thagard paper is connectionist only in the sense that it uses a connectionist technique to solve the constraint satisafction problem. The theoretical problem they attacked was how to incorporate pragmatic constraints into the mapping. There was no intent for the specific mechanism to be plausible or useful. My aim is to produce a practically useful analogical retrieval mechanism. From jfj at m53.limsi.fr Fri Sep 20 07:03:58 1991 From: jfj at m53.limsi.fr (Jean-Francois Jadouin) Date: Fri, 20 Sep 91 13:03:58 +0200 Subject: Bibliography to a novice ... Message-ID: <9109201103.AA18473@m53.limsi.fr> The references for that IEEE special issue (actually, there are two) : Proceedings of the IEEE, Vol 78 no 9 (sept) & 10 (nov). The first issue addresses more basic models (among which the article by Kohonen tackett speaks of). The second is more appolications oriented. jfj From shawnd at ee.ubc.ca Fri Sep 20 18:19:00 1991 From: shawnd at ee.ubc.ca (shawnd@ee.ubc.ca) Date: Fri, 20 Sep 91 15:19:00 PDT Subject: Preprint Announcement Message-ID: <9109202220.AA01077@fridge.ee.ubc.ca> ** Please do not forward to other mailing lists ** The following preprint is available by ftp from the neuroprose archive at archive.cis.ohio-state.edu: Continuous-Time Temporal Back-Propagation with Adaptable Time Delays Shawn P. Day Michael R. Davenport Departments of Electrical Engineering and Physics University of British Columbia Vancouver, B.C., Canada ABSTRACT We present a generalization of back-propagation for training multilayer feed-forward networks in which all connections have time delays as well as weights. The technique assumes that the network inputs and outputs are continuous time-varying multidimensional signals. Both the weights and the time delays adapt using gradient descent, either in ``epochs'' where they change after each presentation of a training signal, or ``on-line'', where they change continuously. Adaptable time delays allow the network to discover simpler and more accurate mappings than can be achieved with fixed delays. The resulting networks can be used for temporal and spatio-temporal pattern recognition, signal prediction, and signal production. We present simulation results for networks that were trained on-line to predict future values of a chaotic signal using its present value as an input. For a chaotic signal generated by the Mackey-Glass differential-delay equation, networks with adaptable delays typically had less than half the prediction error of networks with fixed delays. Here's how to get the preprint from neuroprose: unix> ftp archive.cis.ohio-state.edu (or 128.146.8.52) Name: anonymous Password: neuron ftp> cd pub/neuroprose ftp> binary ftp> get day.temporal.ps.Z ftp> quit unix> uncompress day.temporal.ps.Z unix> lpr day.temporal.ps (or however you print postscript) Any questions or comments can be addressed to me at: Shawn Day Department of Electrical Engineering 2356 Main Mall University of British Columbia Vancouver, B.C., Canada V6T 1Z4 phone: (604) 264-0024 email: shawnd at ee.ubc.ca From fsegovia at batman.fi.upm.es Fri Sep 20 12:36:40 1991 From: fsegovia at batman.fi.upm.es (fsegovia@batman.fi.upm.es) Date: Fri, 20 Sep 1991 18:36:40 +0200 Subject: Bibliography to a novice Message-ID: <"<9109201636.AA18631@batman.fi.upm.es>*"@MHS> The following articles treat the problem of recurrent NN and their learning procedures. They also include modifications to the original BP applied to recurrent networks (the Rumelhart's trick of unfold in time the network's operation) which make the learning phase more practical (less computations, less time): Williams, R.J., and Zipser, D. 1990. "Gradient-Based learning algo rithms for recurrent connectionist networks". Tech. Rep. NU-CCS-90-9. Northeastern University, College of Computer Science, Boston. Williams, R.J., and Peng, J. 1990. "An Efficient Gradient-Based Algo rithm for On-Line Training of Recurrent Network Trajectories". Neural Computation, Vol. 2, Num 4, 490-501. For an extension to continuous time see: Pearlmutter, B.A. 1989. "Learning state space trjectories in recurrent neural neural networks". Neural Computation, Vol 1, Num 2, 263-269. Sejnowski, T.J., and Fang, Y. 1990. "Faster learning for dynamic recurrent backpropagation". Neural Computation, Vol 2, Num 3, 270-273. The papers mentioned above include experiments and good references for related works. Javier Segovia From jmyers at casbah.acns.nwu.edu Sun Sep 22 21:51:23 1991 From: jmyers at casbah.acns.nwu.edu (J Myers) Date: Sun, 22 Sep 91 20:51:23 CDT Subject: Add Message-ID: <9109230151.AA17154@casbah.acns.nwu.edu> Please add me to the mailing list. Thank you. From mdp at eng.cam.ac.uk Mon Sep 23 10:58:48 1991 From: mdp at eng.cam.ac.uk (Mark Plumbley) Date: Mon, 23 Sep 91 10:58:48 BST Subject: NCM'91: One Day Conference on Applications of NNs Message-ID: <12702.9109230958@dsl.eng.cam.ac.uk> One Day Conference Announcement: NCM'91: APPLICATIONS OF NEURAL NETWORKS October 1, 1991 AT CENTRE FOR NEURAL NETWORKS KING'S COLLEGE LONDON, UK Outline Programme 10.00: Coffee 10.30-12.50 Talks 1) N Biggs, "Learning Algorithms - theory and practice" 2) Dr EPK Tsang and Dr CJ Wang, "A Generic Neural Network Approach for Contraint Satisfaction Problems" 3) D Gorse, "Temporal Processing in Probabilistic RAM nets" 4) GJ Chappell (with J Lee and JG Taylor), "A Review of Medical Diagnostic Applications of Neural networks" 5) DL Toulson, JF Boyce and C Hinton, "Data Representation and Generalisation in an application of a Feedforward Net" 6) IW Ricketts, AY Cairns, S Dickson, M Hudson, K Hussein, M Nimmo, PE Preece, AJ Thompson and C Walker, "Artificial Neural Networks Applied to the Inspection of Medical Images" 12.30-1.45 Buffet Lunch (Provided) 1.45-2.00 AGM of BNNS 2.00-4.00 Talks 7) D Anthony, E Hines, D Hutchins and T Mottram, "Ultrasound Tomography Imaging of Defects using Neural Networks" 8) K Kodaira, H Nakata and M Takamura, "An Apple Sorting System Using Neural Network-Based on Image Processing" 9) EC Mertzanis, "Quadtrees for Neural Network based Position Invariant Pattern Recognition" 10) NA Jalel, AR Mirzai and JR Leigh, "Application of Neural Networks in Process Control" 11) CM Bishop, PM Cox, PS Haynes, CM Roach, TN Todd and DL Trotman, "A Neural Network Approach to Tokamak Equilibrium Control" 12) D Shumsheruddin, "Neural Network Control of Robot Arm Tracking Movements" 4.00-4.30 Tea 4.30-5.30 Talks 13) TG Clarkson, "The pRAM as a hardware-realisable neuron" 14) S Hancock, "A Neural Instruction Set Processor (NISP) and Development System (NDEV)" 15) RE Wright, "The Cognitive Modalities ('CM') System of Knowledge Representation The 'DNA' of Neural Networks?" The talks are each 20 minutes in length (including discussions). ------------------------%<------CUT HERE -----%<------------------------- REGISTRATION SLIP FOR NCM'91 I wish to attend the one day conference on APPLICATIONS OF NEURAL NETWORKS: NAME: ............................................................................... ADDRESS: ............................................................................... ............................................................................... (Please make cheque for 30 pounds sterling payable to `Centre for Neural Networks', and address it to: Prof J. G. Taylor, Centre for Neural Networks, King's College, Strand, London WC2R 2LS, UK) From jose at tractatus.siemens.com Mon Sep 23 18:45:08 1991 From: jose at tractatus.siemens.com (Steve Hanson) Date: Mon, 23 Sep 1991 18:45:08 -0400 (EDT) Subject: NIPS*91 UPDATE Message-ID: NIPS*91 FINAL POSTER PROGRAM: SESSIONS I & II TUESDAY EVENING: SESSION I From jose at tractatus.siemens.com Mon Sep 23 18:39:27 1991 From: jose at tractatus.siemens.com (Steve Hanson) Date: Mon, 23 Sep 1991 18:39:27 -0400 (EDT) Subject: NIPS*91 UPDATE Message-ID: NIPS*91 FINAL ORAL PROGRAM MONDAY DECEMBER 2 After Dinner Talk: Allan Hobson, Harvard Medical School "Models Wanted: Must Fit Dimensions of Sleep and Dreaming" TUESDAY DECEMBER 3 ORAL 1: LEARNING and GENERALIZATION I O.1.1 V. Vapnik, Institute of Control Sciences, Academy of Sciences "Principles of Risk Minimization for Learning Theory" (INVITED TALK) O.1.2 D. MacKay, Caltech "A Practical Bayesian Framework for Backprop Networks" 0.1.3 J .Moody, Yale Computer Science " Generalization, Weight Decay, and Architecture Selection for Nonlinear Learning Systems" O.1.4 D. Haussler UC Santa Cruz M. Kearns, International Computer Science Institute M. Opper, Institute fuer Theoretische Physik R. Schapire, Harvard University "Estimating Average-Case Learning Curves Using Bayesian, Statistical Physics and VC Dimension Methods" ORAL 2: TEMPORAL PROCESSING O.2.1 S. P. Singh, University of Massachusetts "The Efficient Learning of Multiple Task Sequences" O.2.2 G. Tesauro, IBM "Practical Issues in Temporal Difference Learning" O.2.3 H. Hild, Universitat Karlsruhe W. Menzel, Universitat Karlsruhe J. Feulner, Universitat Karlsruhe "A Neural Net For Harmonizing Chorals in the Style of J.S. Bach" O.2.4 M.A. Jabri, Sydney University S. Pickard, Sydney University P. Leong, Sydney University Z. Chi, Sydney University B. Flower, Sydney University "Architectures and Implementation of Right Ventricular Apex Signal Classifiers for Pacemakers" SPOTLIGHT I: TEMPORAL PROCESSING ORAL 3: VISUAL PROCESSING O.3.1 D. A. Robinson, Johns Hopkins University School of Medicine "Information Processing to Create Eye-Movements" (INVITED TALK) O.3.2 S. Becker, University of Toronto G. E. Hinton, University of Toronto "Learning to Make Coherent Predictions in Domains with Discontinuities" O.3.3 K. A. Boahen, Caltech A. G. Andreou, John Hopkins "A Contrast Sensitive Silicon Retina with Reciprocal Synapses" O.3.4 P.A. Viola, MIT S.G. Lisberger, UC San Francisco T.J. Sejnowski, Salk Institute for Biological Science, "Recurrent Eye Tracking Network Using a Distributed Representation of Image Motion" ORAL 4: OPTICAL CHARACTER RECOGNITION O.4.1 F. Faggin, Synaptics "Neural Network Analog VLSI Implementations" (INVITED TALK) O.4.2 J. D. Keeler, MCC D. E. Rumelhart, Stanford University "Self-Organizing Segmentation and Recognition Neural Network" O.4.3 I. Guyon, ATt&T Bell Laboratories V.N. Vapnik, ATt&T Bell Laboratories B.E. Boser, ATt&T Bell Laboratories L.Y. Bottou, ATt&T Bell Laboratories S.A. Solla, ATt&T Bell Laboratories "Structural Risk Minimization for Character Recognition" SPOTLIGHT II: VISUAL PROCESSING AND OCR SPOTLIGHT III: APPLICATIONS and PERFORMANCE COMPARISONS WEDNESDAY DECEMBER 4 ORAL 5: LEARNING and GENERALIZATION II O.5.1 J. S. Judd, Siemens Research "Constant-Time Loading of Shallow 1-Dimensional Networks" O.5.2 J. Alspector, Bellcore A. Jayakumar,Bellcore S. Luna, University of California "Experimental Evaluation of Learning in a Neural Microsystem" O.5.3 C. McMillan, University of Colorado M. C. Mozer, University of Colorado P. Smolensky, University of Colorado "Rule Induction Through A Combination of Symbolic and Subsymbolic Processing" O.5.4 G. Towell, University of Wisconsin-Madison J. W. Shavlik, University of Wisconsin-Madison "Interpretation of Artificial Neural Networks: Mapping Knowledge-Based NN into Rules" SPOTLIGHT IV: LEARNING & ARCHITECTURES ORAL 6: LOCOMOTION, PLANNING & CONTROL O.6.1 A. W. Moore, MIT "Fast, Robust Adaptive Control by Learning only Forward Models" O.6.2 S. B. Thrun, German National Research Center for Computer Science K. Moeller, University of Bonn "Active Exploration in Dynamic Environments" O.6.3 J. Buchanan, Marquette University "Locomotion In A Lower Vertebrate: Studies of The Cellular Basis Of Rhythmogenesis and Oscillator Coupling" O.6.4 M. Lemmon, University of Notre Dame "Oscillatory Neural Networks for Globally Optimal Path Planning" SPOTLIGHT V: NEURAL CONTROL ORAL 7: SELF ORGANIZATION, ARCHITECTURES and LEARNING O.7.1 M. I. Jordan, MIT R. A. Jacobs , MIT "Hierarchies of Adaptive Experts" O.7.3 S. J. Nowlan, University of Toronto G. E. Hinton, University of Toronto "Adaptive Soft Weight Tying using Gaussian Mixtures" O.7.4 D. Rogers, Research Institute for Advanced Computer Science "Friedman's Multivariate Adaptive Regression Splines (MARS)Algorithm with Holland's Genetic Algorithm" O.7.5 D. Wettschereck, Oregon State University T. Dietterich, Oregon State University "Improving the Performance of Radial Basis Function Networks by Learning Center Locations" SPOTLIGHT VI: SPEECH ORAL 8: VISUAL SYSTEM O.8.1 A. Bonds, Vanderbilt University "Dual Inhibitory Mechanisms for Definition of Receptive Field Characteristics in Cat Striate Cortex" O.8.2 K. Obermayer, University of Illinois K. Schulten, University of Illinois G.G. Blasdel, Harvard Medical School "Comparison Between a Neural Network Model for the Formation of Brain Maps and Experimental Data" O.8.3 R. Kessing, Ricoh California Research Center D. Stork, Ricoh California Research Center C.J. Schatz, Stanford University School of Medicine "Retinogeniculate Development: The Role of Competition and Correlated Retinal Activity" SPOTLIGHT VII: LEARNING & ARCHITECTURES THURSDAY DECEMBER 5 ORAL 9: APPLICATIONS O.9.1 M. R. Rutenburg, NeuroMedical Systems, Inc, "PapNET: A Neural Net Based Cytological Screening System" (INVITED TALK) O.9.2 M. Roscheisen Munich Technical University R. Hofmann, Munich Technical University V. Tresp, Siemens AG "Incorporating Prior Knowledge in Parsimonious Networks of Locally-Tuned Units" O.9.3 P. Smyth, JPL J. Mellstrom, JPL "Fault Diagnosis of Antenna Pointing Systems using Hybrid Neural Network and Signal Processing Techniques" O.9.4 C. F. Neugebauer, Caltech A. Yariv, Caltech " A Parallel Analog CCD/CMOS Neural Network IC with Digital I/O" BREAK ORAL 10: SPEECH & SIGNAL PROCESSING O.10.1 K. Church, AT&T Bell Labs "Part of Speech Tagging" (INVITED TALK) O.10.2 A. R. Bulsara, Naval Ocean Systems Center F. E. Moss, Univ. of Missouri "Single Neuron Dynamics: Noise-Enhanced Signal Processing" O.10.3 D. Warland, University of California, Berkeley F. Rieke, NEC Research Institute W. Bialek, NEC Research Institute "Efficient Coding in Sensory Systems" O.10.4 J. Platt, Synaptics F. Faggin, Synaptics "A Network for the Separation of Sources That Are Superimposed And Delayed" O.10.5 A. Waibel, CMU A. N. Jain, CMU A. E. McNair, CMU J. Tebelskis, CMU A. Hauptmann, CMU H. Saito, CMU "JANUS: Speech-to-Speech Translation Using Connectionist and Non-Connectionist Techniques" SPOTLIGHT TALKS (4 Minute Talks) SPOTLIGHT I: TEMPORAL PROCESSING J. Connor, University of Washington L.E. Atlas, University of Washington D. Martin, University of Washington "Advantages of Recurrency and Innovations in Time Series Prediction" C. Brody, ILMAS, UNAM "Fast Learning with Predictive Forward Models" C.G. Atkeson, MIT "Robot Juggling: A Real-Time Implementation of Learning Based on Locally Weighted Regression" SPOTLIGHT II: OCR and VISUAL PROCESSING G.L. Martin, MCC & Eastman Kodak "Centered-Object Integrated Segmentation and Recognition for Visual Character Recognition" G.E. Hinton, University of Toronto C.K.I. Williams, University of Toronto "Adaptive Elastic Models for Character Recognition" S. Ahmad, ICSI "Modelling Visual Attention and Visual Search" D.Z. Anderson, University of Colorado C. Benkert, University of Colorado V. Hebler, University of Colorado J.S. Jang, University of Colorado D. D. Montgomery, University of Colorado M. Saffman, University of Colorado "Optical Implementation of a self-organizing feature extractor" SPOTLIGHT III: APPLICATIONS AND PERFORMANCE COMPARISONS M.W. Goudreau, NEC Research Institute, Inc. C.L. Giles, NEC Research Institute, Inc. "Neural Network Routing for Random Multistage Interconnection Networks" P. Stolorz, Los Alamos National Laboratory A. Lapedes, Los Alamos National Laboratory R. Farber, Los Alamos National Laboratory D. Wolf, Los Alamos National Laboratory Y. Xia, Los Alamos National Laboratory J. Bryngelson, Los Alamos National Laboratory "Prediction of Protein Structure Using Neural Nets and Information Theory" P. Kohn, ICSI J. Bilmes, ICSI N. Morgan, ICSI J. Beck, ICSI "Software for ANN Training on a Ring Array Processor" J. Bernasconi, Asea Brown Boveri Corp. K. Gustafson, University of Colorado "Human and Machine 'Quick Modeling'" L.G.C. Hamey, Macquarie University "Benchmarking feed-forward Neural Networks: models and measures" SPOTLIGHT IV: LEARNING & ARCHITECTURES P. Koistinen, Rolf Nevanlinna Institute L. Holmstroem, Rolf Nevanlinna Institute "Kernel Regression and Backpropagation Training With Noise" K.Y. Siu, Stanford University J. Bruck, IBM Research Division "Neural Computing With Small Weights" SPOTLIGHT V: NEURAL CONTROL T. Anastasio, USC "Learning In the Vestibular System: Simulations Of Vestibular Compensation Using Recurrent Back-Propagation" P. Dean, University of Sheffield E.W. Mayhew, University of Sheffield "A Neural Net Model For Adaptive Control of Saccadic Accuracy By Primate Cerebellum and Brainstem" SPOTLIGHT VI: SPEECH R. Cole, Oregon Graduate Institute of Science and Technology K. Roginski, Oregon Graduate Institute of Science and Technology M. Fanty, Oregon Graduate Institute of Science and Technology "English Alphabet Recognition With Telephone Speech" Y. Bengio, McGill University R. DeMori, McGill University G. Flammia, McGill University R. Kompe, McGill University "Global Optimization of Neural Network - Hidden Markov Model Hybrid" SPOTLIGHT VII: LEARNING AND ARCHITECTURES J. Bridle, R.S.R.E. D.J.C. MacKay, Caltech "Unsupervised Classifiers, Mutual Information And 'Phantom Targets'" P. Simard, AT&T Bell Laboratories B. Victorri, Universite de Caen Y. Le Cun, AT&T Bell Laboratories John Denker, AT&T Bell Laboratories "Tangent Prop - A formalism for specifying selected invariances in an adaptive network" D. Montana, Bolt Beranek and Newman, Inc. "A weighted probabilistic Neural Net" T. Lange, University of California "Dynamically-Adaptive Winner-Take-All Networks" S. Omohundro, International Computer Science Institute "Model-Merging for Improved Generalization" From jose at tractatus.siemens.com Mon Sep 23 18:46:11 1991 From: jose at tractatus.siemens.com (Steve Hanson) Date: Mon, 23 Sep 1991 18:46:11 -0400 (EDT) Subject: NIPS*91 UPDATE Message-ID: <8craunS1GEMn41K2Ih@tractatus.siemens.com> NIPS*91 FINAL POSTER PROGRAM: SESSIONS I & II TUESDAY EVENING: SESSION I From jose at tractatus.siemens.com Mon Sep 23 18:51:11 1991 From: jose at tractatus.siemens.com (Steve Hanson) Date: Mon, 23 Sep 1991 18:51:11 -0400 (EDT) Subject: NIPS*91 UPDATE Message-ID: <0crazT21GEMnA1K3Rn@tractatus.siemens.com> NIPS*91 FINAL POSTER PROGRAM: SESSIONS I & II TUESDAY EVENING: SESSION I LEARNING METHODS AND GENERALIZATION I M. R. Sydorenko, The Johns Hopkins School of Medicine E. D. Young The Johns Hopkins School of Medicine "Analysis of Stationarity of the Strength of Synaptic Coupling Between Pairs of Neurons with Non-Stationary Discharge Properties" P. Dayan, University of Edinburgh G. Goodhill, University of Sussex "Perturbing Hebbian Rules" D. Geiger, Siemens Research R. Pereira, Universita di Trento "Selecting Minimal Surface Data" J. Shawe-Taylor, University of London "Threshold Network Learning in the Presence of Equivalences" B. Pearlmutter, Yale "Asymptotic Convergence of Gradient Descent with Second Order Momentum" P. Munro, University of Pittsburgh "Repeat until bored: a pattern selection strategy" Y. Freund University of California, Santa Cruz D. Haussler, University of California, Santa Cruz "A fast and exact learning rule for a restricted class of Boltzmann machines" IMPLEMENTATION I R.G. Benson, Caltech "Silicon ON-Cell Adaptive Photorecepetors" E. Sackinger, AT&T Bell Laboratories B.E. Boser, AT&T Bell Laboratories L.D. Jackel, AT&T Bell Laboratories "A Neurocomputer Board Based on the ANNA Neural Network Chip" P. Kohn, ICSI J. Bilmes, ICSI N. Morgan, ICSI J. Beck, ICSI "Software for ANN Training on a Ring Array Processor" VISUAL PROCESSING I G.L. Martin, MCC & Eastman Kodak "Centered-Object Integrated Segmentation and Recognition for Visual Character Recognition" S. Ahmad, ICSI "Modelling Visual Attention and Visual Search" E. Mjolsness,Yale University "Visual Grammars and their Neural Nets" M.C. Mozer, University of Colorado R.S. Zemel, University of Toronto M. Behrmann, University of Toronto "Learning to Segment Images using Dynamic Feature Binding" H. Greenspan, Caltech R.M. Goodman, Caltech R. Chellappa, University of Maryland "A Combined Neural Network and Rule Based Framework for Probabilistic Pattern Recognition and Discovery" R. Basri MIT S. Ullman MIT "Linear Operator for Object Recognition" G.E. Hinton, University of Toronto C.K.I. Williams, University of Toronto "Adaptive Elastic Models for Character Recognition" O. Matan, AT&T Bell Laboratories C.J.C. Burges, AT&TBell Laboratories Y. Le Cun, AT&TBell Laboratories J.S. Denker, AT&TBell Laboratories "Multi-Digit Recognition Using a Space-Delay Neural Network" N. Intrator, Brown University J.I. Gold, Brown University H.H. Buelthoff, Brown University S. Edelman, Weizmann Institute of Science Three-Dimensional Object Recognition Using an "Unsupervised Neural Network: Understanding the Distinguishing Feature" ARCHITECTURES AND APPROXIMATION P. Koistinen, Rolf Nevanlinna Institute L. Holmstroem, Rolf Nevanlinna Institute "Kernel Regression and Backpropagation Training With Noise" R. Williamson, Australian National University P. Bartlett, University of Queensland "Piecewise Linear Feedforward Neural Networks" C. Ji, Caltech D. PsaltisCaltech "Storage Capacity and Generalization of Two-Layer Networks With Binary Weights The VC-Dimension vs. The Statistical Capacity" Y. Zhao, MIT C.G. Atkeson, MIT "Some Approximation Properties of Projection Pursuit Learning Networks" J. Moody, Yale N. Yarvin, Yale "Networks with learned unit response functions" N.J. Redding, Electronics Research Laboratory T. Downs, University of Queensland "Networks with Nonsmooth Functions" T.D. Sanger, Massachusetts Institute of Technology R.S. Sutton GTE Laboratories Inc. C.J.Matheus, GTE Laboratories Inc. "Iterative Construction of Sparse Polynomial Approximations" M. Wynne-Jones, RSRE "Node splitting: a constructive algorithm for feed-forward NN" S. Ramachandran, Rutgers L.Y. Pratt, Rutgers "Information Measure Based Skeletonisation" TEMPORAL PROCESSING C. Koch, Caltech H. Schuster, Universitaet Kiel "A Simple Network Showing Burst Synchronization without Frequency-Locking" C. Brody, ILMAS, UNAM "Fast Learning with Predictive Forward Models" C.G. Atkeson, MIT "Robot Juggling: A Real-Time Implementation of Learning Based on Locally Weighted Regression" J.P. Sutton, Harvard Medical School A.N. Mamelak, Harvard Medical School A. Hobson, Harvard Medical School "Network Model of State-Dependent Sequencing" M.C. Mozer, University of Colorado "Connectionist Music Composition and the Induction of Multiscale Temporal Structure" J. Schmidhuber, University of Colorado "History Compression for Adaptive Sequence Chunking" PERFORMANCE COMPARISONS J. Bernasconi,Asea Brown Boveri Corp. K. Gustafson, University of Colorado "Human and Machine 'Quick Modeling'" M. Maechler, University of Washington R. D. Martin, University of Washington J. Schimert, University of Washington J.N. Hwang, University of Washington "A Comparison of Projection Pursuit and NN Regression Modeling" L.G.C. Hamey, Macquarie University "Benchmarking feed-forward Neural Networks: models and measures" APPLICATIONS I P. Stolorz, Los Alamos National Laboratory A. Lapedes, Los Alamos National Laboratory R. Farber, Los Alamos National Laboratory D. Wolf, Los Alamos National Laboratory Y. Xia, Los Alamos National Laboratory J. Bryngelson,Los Alamos National Laboratory "Prediction of Protein Structure Using Neural Nets and Information Theory" J. Connor, University of Washington L.E. Atlas, University of Washington D. Martin, University of Washington "Advantages of Recurrency and Innovations in Time Series Prediction" M.W.Goudreau, NEC Research Institute, Inc. C.L. Giles, NEC Research Institute, Inc. "Neural Network Routing for Random Multistage Interconnection Networks" J. Moody, Yale University J. Utans, Yale University "Selecting Neural Network Architectures via the Prediction Risk: Application to Corporate Bond Rating Prediction" R. Venturini, The Salk Institute W.W. Lytton, The Salk Institute T.J. Sejnowski, The Salk Institute "Neural Network Analysis of Event Related Potentials Predicts Vigilance" SIGNAL PROCESSING D. B. Schwartz, GTE Laboratories, Inc. "Making Precise Measurements with Sloppy Sensors" J. Lazzaro, University of Colorado "Temporal Adaptation in a Silicon Auditory Nerve" M.H. Cohen, The Johns Hopkins University P.O. Pouliquen, The Johns Hopkins University A.G. Andreou, The Johns Hopkins University "Analog VLSI Implementation of an Auto-Adaptive Network for Real-Time Separation of Independent Signals" R. de Ruyter van Steveninck, University Hospital Groningen W. Bialek, NEC Research Institute "Statistical Reliability of a Movement-Sensitive Neuron" B. De Vries, University of Florida J.C. Principe, University of Florida P.Guedes de Olivierra, Universidade de Aviero "Neural Signal Procesing with an Adaptive Dispersive Tapped Delay Line" PATTERN RECOGNITION A.M. Chiang, MIT Lincoln Laboratory M.L. Chuang, MIT Lincoln Laboratory J.R. LaFranchise, MIT Lincoln Laboratory "CCD Neural Network Processors for Pattern Recognition" E.N. Eskandar, National Institute of Mental Health B.J. Richmond, National Institute of Mental Health J.A. Hertz, Nordita L.M. Optican, National Eye Institute "Decoding of Neuronal Signals in Visual Pattern Recognition" B.W. Mel, Caltech "NMDA-Based Pattern Discrimination in a Modeled Cortical Neuron" O. Bernander, Caltech R. Douglas, Oxford, UK & Caltech K. Martin, Oxford, UK C. Koch, Caltech "Synaptic Background Activity Determines Spatio-Temporal Integration in Single Cells" SELF ORGANIZATION A. Zador, Yale University B.J. Clairborne, University of Texas T.H. Brown, Yale University "Nonlinear Processing in Single Hippocampal Neurons with Dendritic Hot and Cold Spots" D.Z. Anderson, University of Colorado C. Benkert, University of Colorado V. Hebler, University of Colorado J. Jang, University of Colorado D.D. Montgomery, University of Colorado "Optical Implementation of a self-organizing feature extractor" T. Bell, Vrije Universiteit Brussel "Self-Organisation in Real Neurons: Gradient Descent In Channel Space?" H.U. Bauer, Universitaet Frankfurt K. Pawelzik, Universitaet Frankfurt F. Wolf, Universitaet Frankfurt T. Geisel, Universitaet Frankfurt "A Topographic Product for the Optimization of Self-Organizing Feature Maps" WEDNESDAY EVENING: SESSION II VISUAL PROCESSING II H.P. Graf, AT&T Bell Laboratories C. Nohl, AT&T Bell Laboratories J. Ben, AT&T Bell Laboratories "Image Segmentation with Networks of Variable Scale" T. Darrell, MIT A. Pentland, MIT "The Multi-Layer Support Process" P. Cooper, Northwestern University P. Prokopowicz, Northwestern University "M. Random Fields can Bridge Levels of Abstraction" A. Shashua, M.I.T. "Illumination and 3D Object Recognition" A. Pouget, The Salk Institute S.A. Fisher, The Salk Institute T.J. Sejnowski,The Salk Institute "Hierarchical Transformation of Space in The Visual System" LANGUAGE A. Jain, CMU "Generalization Performance in PARSEC- A Structured Connectionist Parsing Architecture" G. Pinkas, Washington University "Syntactic Proofs Using Connectionist Constraint Satisfaction" P. Gupta, CMU D.S. Touretzky, CMU "A Connectionist Learning Approach to Analyzing Linguistic Stress" R.A. Sumida, University of California M.G. Dyer, University of California "Propagation Filters in PDS Networks for Sequencing and Ambiguity Resolution Ambiguity Resolution" Y. Muthusamy, Oregon Graduate Institute of Science and Technolgy R. A. Cole, Oregon Graduate Institute of Science and Technolgy "Segment-Based Automatic Language Identification System" R.L. Watrous, Siemens Research C.L. Giles, NEC Research Institute C.B. Miller NEC Research Institute G.M. Kuhn, IDA D. Chen, University of Maryland H.H. Chen, University of Maryland G.Z. Sun, University of Maryland Y.C. Lee, University of Maryland "Induction of Finite State Automata from Discrete-time Recurrent Neural Networks" SPEECH PROCESSING M. Hirayama, ATR E.V. Bateson, ATR M. Kawato, ATR M.I. Jordan MIT "Forward Dynamics Modeling of Speech Motor Control Using Physiological Data" E. Levin, AT&T Bell Laboratories R. Pieraccini, AT&T Bell Laboratories E. Bocchieri, AT&T Bell Laboratories "Time Warping Network: A Hybrid Framework for Speech Recognition" R. Cole, Oregon Graduate Institute of Science and Technology K. Roginski, Oregon Graduate Institute of Science and Technology M. Fanty, Oregon Graduate Institute of Science and Technology "English Alphabet Recognition With Telephone Speech" P. Haffner, Centre National d'Etudes des Telecommunications A. Waibel, CMU "Multi-State Time Delay Neural Networks for Continous Speech Recognition" HYBRID MODELS E. Singer, MIT Lincoln Laboratory R. P. Lippmann, MIT Lincoln Laboratory "Improved Hidden Markov Model Speech Recognition Using Radial Basis Function Networks" S. Renals, ICSI N. Morgan, ICSI H. Bourlard, L&H Speechproducts H. Franco, SRI International Mike Cohen, SRI International "Connectionist Optimisation of Tied Mixture Hidden Markov Models" Y. Bengio, McGill University R. DeMori, McGill University G. Flammia, McGill University R. Kompe, McGill University "Global Optimization of Neural Network - Hidden Markov Model Hybrid" P. Stolorz, Los Alamos National Laboratory "Merging Constrained Optimization with Deterministic Annealing to "Solve"combinatorially hard problems" CONTROL AND PLANNING T. Prescott, University of Sheffield J. Mayhew, University of Sheffield "Obstacle Avoidance through Reinforcement Learning" K. Doya, University of Tokyo S. Yoshizawa, University of Tokyo "Learning of Locomotion Patterns by Recurrent Neural Networks: How can a central pattern generator adapt itself to the physical world?" H. Gomi, ATR M. Kawato, ATR "Learning Closed-Loop Control and Recognition of Manipulated Objects" G.M. Scott, University of Wisconsin J.W. Shavlik, University of Wisconsin W. H. Ray, University of Wisconsin "Refining PID Controllers Using Neural Networks" P. Dean, University of Sheffield E.W. Mayhew, University of Sheffield "A Neural Net Model For Adaptive Control of Saccadic Accuracy By Primate Cerebellum and Brainstem" T. Anastasio, USC "Learning In the Vestibular System: Simulations Of Vestibular Compensation Using Recurrent Back-Propagation" APPLICATIONS II S. Thiria, Universite de Paris Sud C. Mejia, Universite de Paris Sud F. Badran, Universite de Paris Sud M. Creponi, LODYC "Multimodular Neural Architecture for Remote Sensing Operations" A. Manduca, Mayo Foundation P. Christy, Mayo Foundation R. Ehman, Mayo Foundation "Neural Network Diagnosis of Avascular Necrosis From Magnetic Resonance Images" S.L. Gish, IBM M. Blaum, IBM "Adaptive Development of Connectionist Decoders for Complex Error-Correcting Codes" A. C. Tsoi, University of Queensland "Application of Neural Network Methodology to the Modelling of the Yield Strength in a Steel Rolling Plate Mill" D.T. Freeman, University of Pittsburgh School of Medicine "Computer Recognition of Wave Location in Graphical Data by a Neural Network" H. Tunley, University of Sussex "A Neural Network for Motion Detection of Drift-Based Stimuli" MEMORY D. Horn, Tel Aviv University M. Usher, Weizman Institute "Oscillatory Model of Short Term Memory" K-Y. Siu, Stanford University J. Bruck, IBM Research Division "Neural Computing With Small Weights" IMPLEMENTATION D. Kirk, Caltech K. Fleischer, Caltech A. Barr, Caltech "Constrained Optimization Applied to the Parameter Setting Problem for Analog Circuits" J.G. Harris,Caltech "Segmentation Circuits Using Contrained Optimization" KINEMATICS & TRAJECTORIES M. Dornay, ATR Y. Uno,University of Tokyo M. Kawato, ATR R. Suzuki, University of Tokyo "Simulation of Optimal Movements Using the Minimum Muscle-Tension-Change Model" P. Simard, AT&T Bell Laboratories Y. LeCun, AT&T Bell Laboratories "Reverse TDNN: An Architecture for Trajectory Generation" E. Henis, The Weizmann Institute of Science T. Flash, The Weizmann Institute of Science "A Computational Mechanism to Account for Averaged Modified Hand Trajectories" D. DeMers, UC San Diego K. Kreutz-Delgado, UC San Diego "Learning Global Direct Inverse Kinematics" N. E. Berthier,University of Massachusetts S.P. Singh, University of Massachusetts A.G. Barto, University of Massachusetts J.C. Houk, Northwestern University Medical School "A Cortico-Cerebellar Model That Learns To Generate Distributed Motor Commands to Control A Kinematic Arm" ARCHITECTURES AND GENERALIZATION J. Bridle, R.S.R.E. D.J.C. MacKay, Caltech "Unsupervised Classifiers, Mutual Information And 'Phantom Targets'" T. Lange, University of California "Dynamically-Adaptive Winner-Take-All Networks" A. Krogh, The Niels Bohr Institute J.A. Hertz, Nordita "A Simple Weight Decay Can Improve Generalization" M.S. Glassman, Siemens Research "A Network of Localized Linear Discriminants" D. Montana, Bolt Beranek and Newman, Inc. "A weighted probabilistic Neural Net" S. Omohundro, International Computer Science Institute "Model-Merging for Improved Generalization" I. Grebert, Ricoh California Research Center D.G. Stork, Ricoh California Research Center R. Keesing, Ricoh California Research Center S.Mims, Stanford University "Network generalization for production: learning and producing styled letterforms" J.B. Hampshire II, Carnegie Mellon University D.A. Pomerleau, Carnegie Mellon University B.V.K. Vijaya Kumar Carnegie Mellon University "Automatic Feature Selection and Functional Capacity Modulation in MLP Classifiers via Learned Synaptic Correlations" LEARNING METHODS AND GENERALIZATION II P. Simard, AT&T Bell Laboratories B. Victorri, Universite de Caen Y. Le Cun, AT&T Bell Laboratories J. Denker, AT&T Bell Laboratories "Tangent Prop - A formalism for specifying selected invariances in an adaptive networks" G.Z. Sun, University of Maryland H.H. Chen, University of Maryland Y.C. Lee, University of Maryland "Green's Function Method for Fast On-line learning Algorithm of recurrent Method for Fast On-line learning Algorithm of recurrent NN" C. Darken, Yale University J. Moody, Yale University "Faster Stochastic Gradient Search" N.N. Schraudolph, UCSD T. Sejnowski, The Salk Institute "Competitive Anti-Hebbian Learning of Invariants" J. Wiles, University of Queensland A. Bloesch, University of Queensland "Operators and curried functions: training and analysis of simple recurrent networks" A. Bertoni, Universita degli Studi Di Milano P. Campadelli, Universita degli Studi Di Milano A. Morpurgo, Universita degli Studi Di Milano S. Panizza, Universita degli Studi Di Milano "Polynomial Uniform Convergence of Relative Frequencies to Probabilities" T. Kuh, University of Hawaii T. Petsche, Siemens Research R. Rivest, MIT "Mistake Bounds of Incremental Learners when Concepts Drift with Applications to Feedforward Networks" From jai at sol.boeing.com Tue Sep 24 18:21:21 1991 From: jai at sol.boeing.com (Jai Choi) Date: Tue, 24 Sep 91 15:21:21 PDT Subject: TR's available Message-ID: <9109242221.AA06689@sol.boeing.com> The following technical report is available. This's been submitted for FUZZ-IEEE '92, San Diego. A copy will be available by request only to Jai J. Choi Boeing Computer Services P.O.Box 24346, MS 6C-04 Seattle, WA 98124, USA or drop your surface address to "jai at sol.boeing.com". . We propose a real-time diagnostic system using a combination of neural networks and fuzzy logic. This neuro-fuzzy hybrid system utilizes real-time processing, prediction and data fusion. A layer of n trained neural networks processes $n$ independent time series (channels) which can be contaminated with environmental noise. Each network is trained to predict the future behavior of one time series. The prediction error and its rate of change from each channel are computed and sent to a fuzzy logic decision output stage, which contains $n+1$ modules. The (n+1)st final-output module performs data fusion by combining n individual fuzzy decisions that are tuned to match the domain expert's need. From uh311ae at sunmanager.lrz-muenchen.de Wed Sep 25 15:09:31 1991 From: uh311ae at sunmanager.lrz-muenchen.de (Henrik Klagges) Date: 25 Sep 91 21:09:31+0200 Subject: logarithmic/linear weights Message-ID: <9109251909.AA07867@sunmanager.lrz-muenchen.de> Where do I loose more accuracy or information, if I store the weights linear or logarithmic. So to speak, do I need 'high end' resolution or 'low end' resolution ? Cheers, Henrik From harnad at Princeton.EDU Wed Sep 25 17:27:41 1991 From: harnad at Princeton.EDU (Stevan Harnad) Date: Wed, 25 Sep 91 17:27:41 EDT Subject: Connectionits Models of Reading: Call for Commentators Message-ID: <9109252127.AA16450@clarity.Princeton.EDU> The following paper has just appeared in PSYCOLOQUY. Commentaries may be directed to psyc at pucc.bitnet or to psyc at pucc.princeton.edu All contributions will be refereed. --Stevan Harnad From harnad at clarity.princeton.edu Wed Sep 25 11:43:38 1991 From: harnad at clarity.princeton.edu (Stevan Harnad) Date: Wed, 25 Sep 91 11:43:38 EDT Subject: PSYCOLOQUY V2 #8 (2.8.4 Paper: Connectionism/Reading/Skoyles: 288 l) Message-ID: ---------------------------------------------------------------------- From: ucjtprs at ucl.ac.uk Subject: 2.8.4 Paper:Connectionism, Reading and the Limits of Cognition/Skoyles Rationale for Inviting PSYCOLOQUY Commentary: (a) The literature on reading acquisition is becoming fragmented: Here is a theory which can bring together the modellers and those collecting data. (b) I was brought up upon Thomas Kuhn, so the first things I look for are anomalies. I think I found two. What better way to bring them to other people's attention than to show that they are different aspects of the same phenomena? (c) Many literate but phonologically disabled readers exist who were delayed, often to the point of failure, in learning to read. Yet they eventually learn to read, though often only in their teens. This suggests that the processes underlying their reading competence are functionally intact but that something related to their phonological handicap stops their normal development. At present that link between phonological disability and reading failure is only understood in terms of a correlation. By proposing a mechanism I hope to persuade other researchers if not to adopt my theory then propose other causal mechanisms. This is important: Only with causal theories can scientifically based interventions to ameliorate dyslexia develop. Here are some issues and related questions which might be addressed in the PSYCOLOQUY Commentary. (Note the length of my own comments does not indicate importance.) (1) The nature of connectionist training: Much has been written about the training rules used to adjust networks )back-propagation, etc., and effects of the numbers of hidden units). Little (actually I can find none) has been written about the provision of error-correction feedback. Some questions: What happens if error-correction feedback is degraded -- if it is only sometimes correct: How does this affect learning? What happens if this feedback is delayed (one reason why it may be degraded) or its acquisition interferes with the other processes involved in network training? These questions are important because networks learn and function in the real world where error correction feedback may not be synchronised with network training. Or feedback may only be obtainable by sacrificing the efficiency of other processes, including those training the network. (2) Humphries and Evett in a target article in Behavioral and Brain Sciences argued that adult reading was nonphonetic. That was in 1985. Are they and their commentators of the same opinion now as then? (3) What is phonetic reading? It used to be defined in terms of a person's ability to read nonwords. How could nonwords be read except by sequential grapheme-phoneme translation? Connectionism shows that nonword reading can be done purely by processes trained on real words without the use of special grapheme-phoneme translation processes. I defined phonetic reading in terms of word recognition processes which depend for their recognition of words in some way upon a reader's oral vocabulary. Some commentators may want to challenge this. (4) Phonology and success at reading is a greatly debated subject. I have left the exact nature of phonetic reading unstated -- partly because I believe that the mechanisms involved may vary over time and between individuals (although all share a common dependence upon access to oral vocabulary). This may be controversial, however. (5) Dyslexic and phonology -- there is a link, but its nature is still unresolved. Some commentators might want to question this link (e.g., Bruno Breitmeyer and William Lovegrove). (6) Reading research has had little effect on educators. There is the "whole word method" movement, which does not ascribe any importance to phonology. Representatives of various educational methods might want to give their views. (7) Cognitive development rarely discusses the role of error-correction feedback upon cognitive development (although it would hardly be an original idea to suggest it is important). In reading development we have an example of how important it is. I suggest it is not unique. Commentators might wish to suggest other examples (both involving endogenous and external sources of feedback). (8) Dyslexics might like to contribute. I am (or rather was) dyslexic. I would be interested to contact other dyslexic psychologists. Connectionism, Reading and the Limits of Cognition John R. Skoyles, Department of Psychology, University College London, London WC1E 6BT, UK. Abstract You read these written words without using your knowledge of how they sound. A children's ability to do this predicts their success in learning to read. Connectionist (PDP) models of cognitive processes using networks can explain why this is so. Connectionism successfully simulates "nonphonetic" reading skills. Networks, however, learn by error-correction feedback -- but where does the learner-reader get this feedback? I suggest that it is from the phonetic identification of words. This conjecture might help explain (1) the pattern of reading development in children and (2) dyslexia as well as (3) raising questions for research aimed at making learning to read easier. In addition, I suggest that error-correction feedback may have an important and general role as a "bottle-neck" in the development of cognition. Keywords: dyslexia, connectionism, development, error correction, reading. How did you learn to read these words? Recent reading research has thrown up two anomalies in the acquisition of word learning. I propose a theory which links them. It provides a new way of looking at the development of reading and cognition. Anomaly 1: You read these words nonphonetically, that is, you identify them without using your oral knowledge of how words sound (Humphrey and Evett, 1985). But the best predictors of children's success in learning to read are language related skills such as initial phonetic awareness (Stanovitch, Cunningham, & Feeman, 1984) and phonetic instruction (Bradley and Bryant, 1983) - skills which are needed for phonetic reading. Why should skills needed for phonetic reading predict later success in nonphonetic reading? Anomaly 2: Connectionist (PDP) neural network simulations of reading successfully explain many experimental facts found about word recognition (Seidenberg & McClelland, 1989). Like adult reading, these simulations are nonphonetic -- they lack any connection with our oral knowledge about how words sound. But there is a problem with their success: although they are advocated as models of word learning, they can, paradoxically, only learn to recognise words if the word learner can already read. The problem originates in the need for networks to be tutored with error correction feedback. Reading networks learn by having their inner nodes adjusted after the network has read a word. This adjustment (also called training) depends upon whether the network has read a word correctly or not: The network's nodes are adjusted differently depending on whether or not it correctly identifies a word. This error correction, however, puts an element of circularity at the heart of network word learning, for it makes successful word learning depend on the learner's already being a good reader (otherwise the processes training the network cannot know whether or not the network has read a word correctly). This condition in which successful word learning depends on pre-existing reading does correctly describe learning readers, however -- the children who learn to read most easily are those who are good readers already -- if we appreciate that the phonetic identification of words is a form of reading. The first anomaly raises the question of why phonetic reading predicts success in nonphonetic reading. The second anomaly answers it: I conjecture that initial phonetic reading skills are needed to train nonphonetic skills. The better their phonetic skills, the easier learning readers find it to provide error correction feedback (by recognising words phonetically). They are accordingly in a better position to train their developing reading networks and thereby learn to read nonphonetically. I suggest that reading development is a double process, involving a trade off between the different advantages and disadvantages of phonetic reading and nonphonetic reading. The underlying story, I propose, is this: The phonetic identification of words is inefficient for normal reading, both in terms of speed and its use of mental attention (because it depends upon the use of oral vocabulary). In contrast, the nonphonetic recognition of words is suited for the demands of reading as it is quick and undemanding cognitively (an important requirement if the mind is to focus upon what it has read rather than recognising words). However, nonphonetic reading is difficult to acquire because it needs to be trained with error correction feedback. The reading development overcomes this limitation by the use of the less efficient phonetic identification of words to provide this feedback; afterwards, the phonetic identification of words is dispensed with. This suggestion explains other facets of the acquisition of reading skills. It fits the pattern of normal child reading development: Children go through a period of phonetic reading before progressing to nonphonetic reading (Frith, 1985). I suggest that this is due to the advantage mentioned above for the development of nonphonetic reading derived from prior skills in recognising words phonetically. This suggestion may also explain the link between difficulties in recognising words phonetically and dyslexia (Snowling, 1987). The majority of dyslexics have impaired phonological skills (particularly, but not exclusively, phonologically segmenting words), but they seem to have intact nonphonological processes. Indeed, given time, some of them (Campbell & Butterworth, 1985) persevere at learning words and become normal readers -- presumably because they use less efficient means than phonology to training their nonphonetic reading abilities. The notion that phonology may play a role in word learning is not new (e.g., see Jorm and Share, 1983). What is novel, however, is the link between phonology and the network training. This is unexpected. It also allows some important questions to be raised. There must be an optimal time window for error correction to aid network training. How long is this? There must also be a limit on how cognitively demanding obtaining error correction feedback can be before it interferes with training and the reading process, but how demanding? It may be that the importance of phonetic reading is not just that it provides error correction feedback but that it provides this information effectively and nondisruptively when word training processes need it. If these questions can be answered then this may help in the development of better methods for helping people with word learning problems. This model of word learning has wider importance. Word learning in reading may not be unique. Error correction feedback -- the need for learners to know whether or not they have performed the skill they are learning correctly -- may be a general bottle-neck limiting cognitive growth. No one has looked at cognitive development from this perspective, so its importance is largely unknown, but it could explain why cognitive development tends to go in stages: Spurts of cognitive growth could be due to the development of new strategies and means for overcoming problems in obtaining this information. References. Bradley, L., & Bryant, P. E. (1983). Categorisation of sounds and learning to read: A causal connection. Nature 301: 419-421. Campbell, R. & Butterworth, B. (1985). Phonological dyslexia and dysgraphia in a highly literate subject: A development case with associated deficits of phonemic processing and awareness, Quarterly Journal of Experiment Psychology 37A: 435 - 475. Frith, U. (1985). Beneath the surface of developmental dyslexia. In K. E. Patterson, J. C. Marshall, & M. Coltheart (Eds.), Surface dyslexia. London: Routledge and Kegan Paul. Humphries, G. W., & Evett, L. L. (1985). Are there independent lexical and nonlexical routes in word processing? An evaluation of the dual-route model of reading. Behavioral and Brain Sciences 8: 689-740. Joam, A. F. & Share, D. L. (1983). Phonological recoding and reading acquisition. Applied Psycholinguistics 4: 103- 147. Seidenberg, M. S. and McClelland, J. I. (1989). A distributed, developmental model of word recognition and naming. Psychological Review 96: 523-568. Skoyles, J. R. (1988). Training the brain using neural-network models. Nature 333: 401. Snowling, M. (1987). Dyslexia: A cognitive developmental perspective. Oxford: Basil Blackwell. Stanovitch, K. E., Cunningham, A. F., & Feeman, D. J. (1984). Intelligence, cognitive skills and early reading progress. Reading Research Quarterly 19: 278-303. ------------------------------ PSYCOLOQUY is sponsored by the Science Directorate of the American Psychological Association (202) 955-7653 Co-Editors: (scientific discussion) (professional/clinical discussion) Stevan Harnad Perry London, Dean, Cary Cherniss (Assoc Ed.) Psychology Department Graduate School of Applied Graduate School of Applied Princeton University and Professional Psychology and Professional Psychology Rutgers University Rutgers University Assistant Editor: Malcolm Bauer Psychology Department Princeton University End of PSYCOLOQUY Digest ****************************** From kak at max.ee.lsu.edu Wed Sep 25 12:43:11 1991 From: kak at max.ee.lsu.edu (Dr. S. Kak) Date: Wed, 25 Sep 91 11:43:11 CDT Subject: No subject Message-ID: <9109251643.AA03185@max.ee.lsu.edu> Call for Papers Special Issue of INFORMATION SCIENCES Topic: Neural Networks and Artificial Intelligence The motivation for this Special Issue, which will be published in 1992, is to evaluate the advances in the field of neural networks in perspective. Critics have charged that engineering neural computing has turned out to be no more than a new technique of clustering data. From A.G.Hofland at newcastle.ac.uk Wed Sep 25 18:04:31 1991 From: A.G.Hofland at newcastle.ac.uk (Anton Hofland) Date: Wed, 25 Sep 91 18:04:31 BST Subject: Announcement Follow Up Message-ID: Announcement Follow Up Forgot the extensions of Prof. Morris and Dr. Montague. They are on ext. 7342 and ext 7265 respectively. Anton Hofland. From karunani at CS.ColoState.EDU Thu Sep 26 01:09:05 1991 From: karunani at CS.ColoState.EDU (n karunanithi) Date: Wed, 25 Sep 91 23:09:05 MDT Subject: No subject Message-ID: <9109260509.AA01806@zappa> To those who use connectionist networks for sequential prediction ------------------------------------------------------------------ applications. ------------ Background: ----------- I have been using neural network models (both Feed-Forward Nets and Recurrent Nets) in a prediction application and I am getting pretty good results. In fact neural networks approach outperformed many well known analytic models. Similar results have been reported by many researchers in (chaotic) time series predictions. Suppose that X is the independent variable and Y is the dependent variable. Let (x(i),y(i)) represent a sequence of actual input/output values observed at time i = 0,1,2,..,t of a temporal process. Let further that both the input and the output variables are single dimensional variable and can take on a sequence of +ve integers up to a maximum of 2000. Once we train a network with the history of the system up to time "t" we can use the network to predict outputs y(t+h), h=1,..,n for any future input x(t+h). In my application I already have the complete sequence and hence I know what is the maximum value for x and y. Using these maximum I normalized both X and Y over a 0.1 to 0.9 range. (Here I call such normalization as "scaled representation".) Since I have the complete sequence it is possible for me to evaluate how good the networks' predictions are. Now some basic issues: --------------------- 1) How to represent these variables if we don't know in advance what the maximum values are? Scaled representation presupposes the existence of a maximum value. Some may suggest that a linear units can be used at the output layer to get rid of scaling. If so how do I represent the input variable? The standard sigmoidal unit(with temp = 1.0) gets saturated(or railed to 1.0) when the sum is >= 14. However one may suggest that changing the output range of the sigmoidal can help to get rid of saturation effect. Is it a correct approach? 2) In such prediction application, people (including me) compare the predictive accuracy of neural networks with that of parametric models(that are based on analytical reasons). But one main advantage with the parametric models is that their parameters can be calculated using any of the following parameter estimation techniques: least square, maximum likelyhood, Bayesian, Genetic Algorithms or any other method. These parameter estimation techniques do not require any scaling, and hence there is no need for preguessing of the maximum values. However with the scaled representation in neural networks one can not proceed without making guesses about the maximum(or a future) input and/or output. In many real life situations such guesses are infeasible or dangerous. How do we address this situation? ____________________________________________________________________________ N. KARUNANITHI E-Mail: karunani at handel.CS.ColoState.EDU Computer Science Dept, Colorado State University, Collins, CO 80523. ____________________________________________________________________________ From A.G.Hofland at newcastle.ac.uk Wed Sep 25 17:59:07 1991 From: A.G.Hofland at newcastle.ac.uk (Anton Hofland) Date: Wed, 25 Sep 91 17:59:07 BST Subject: Symposium Announcement Message-ID: NEURAL NETWORKS AND ENGINEERING APPLICATIONS International Symposium 23/24 October 1991 Background The annual symposium on aspects of advanced process control, organised in collaboration with major engineering companies, has in the past been highly successful. This year the fifth meeting will continue this tradition when it takes as its theme Neural Networks and Engineering Applications. Learning systems of all kinds are presently being studied for potential application to a wide variety of industrial situations. Artificial Neural Networks, in particular, are one of the fastest growing paradigms. The reason for the significant industrial interest in neural networks is that they hold great promise for solving problems that have proved to be extremely difficult for standard techniques. In fact some industries, especially in the USA and Japan, have already transferred the technology through to successful plant applications. In many industrial situations, mathematical models of the processes are either too complex for companies to develop or not accurate enough to be used for control or optimisation. Neural networks can enable rapid and accurate model development and thus play a significant role in improving the operation and control of a wide range of strategically important processes. Indeed there is a potential for their application in virtually every operation in industry. The underlying theme of the Symposium, organised by the University of Newcastle and the International Neural Networks Industrial Club, will be to look at the methodology of artificial neural networks and their applications. Industrial studies will be presented with the aim of identifying opportunities for cross- fertilisation of ideas and technology between industrial sectors and academic research. The speakers are technical authorities in their area and are drawn from international industry and university research centres. They will all address issues associated with the technology and the potential applications. Poster and demonstration sessions will also be mounted. Ample time will be provided for discussions adding to the spectrum of experience of the delegates. In order to facilitate fruitful discussion periods the number of delegates will be restricted to 75. The symposium will be held at the Imperial Hotel in Newcastle upon Tyne, UK. It will start at 1 pm. on the 23rd October with registration from 12 noon. The Lord Mayor of Newcastle will host a civic reception and dinner on the evening of the 23rd starting at 7.30 pm. The meeting will re-convene on the 24th at 8.30 am and finish at 4.30 pm. PROGRAMME 1) Artificial Neural Networks for Process Control. Prof. Julian Morris, University of Newcastle UK. 2) Network Training Paradigms. Dr. Mark Willis, University of Newcastle, UK. 3) Neural Networks in Process Control. Dr. Tom McAvoy, University of Maryland, USA. 4) Autoassociative and Radial Basis Function Networks. Dr. Mark Kramer, MIT, USA. 5) Fault Detection via Artificial Neural Networks. Dr. David Himmelblau, University of Texas, USA. 6) Neural Networks in Industrial Modelling and Control. Dr. Mike Piovoso, DuPont, USA. 7) Neural Networks in Process Engineering. Dr. Gary Montague, University of Newcastle, UK. 8) Time Series Forecasting using Neural Networks. Mr. Chris Gent, SD Scicon, UK. 9) Neural Network Tools. Dr. Alan Hall, Scientific Computers Ltd., UK. REGISTRATION Registration will close at the 17th October. More information and/or registration forms can be obtained by contacting : Prof. A. Julian Morris Department of Chemical and Process Engineering Merz Court Claremont Road Newcastle upon Tyne NE1 7RU Telephone : +44-91-2226000 ext. ???? Fax : +44-91-2611182 e-mail : julian.morris at newcastle.ac.uk or Dr. Gary Montague Department of Chemical and Process Engineering Merz Court Claremont Road Newcastle upon Tyne NE1 7RU Telephone : +44-91-2226000 ext. ???? Fax : +44-91-2611182 e-mail : gary.montague at newcastle.ac.uk From yoshio at tuna.cis.upenn.edu Fri Sep 27 00:59:19 1991 From: yoshio at tuna.cis.upenn.edu (Yamamoto Yoshio) Date: Fri, 27 Sep 91 00:59:19 EDT Subject: No subject Message-ID: <9109270459.AA14476@tuna.cis.upenn.edu> I am looking for information about Kawato's recent works. Recently I've read his articles about the applications in robot control, one from Neural Networks (1988), another from a book, "Neural Networks for Control", and a couple of others from Biological Cybernetics (1987,1988). Therefore I am particularly interested in his works after 1988 to present. Any pointer will be greatly appreciated. I am also interested in getting information about other people's works on similar subjects beside Grossberg and Kuperstein's work and Miller et. al.'s CMAC type approach. Thanks. - Yoshio Yamamoto GRASP Lab (General Robotics And Sensory Perception Laboratory) University of Pennsilvania From bernard at arti1.vub.ac.be Thu Sep 26 11:30:18 1991 From: bernard at arti1.vub.ac.be (Bernard Manderick) Date: Thu, 26 Sep 91 17:30:18 +0200 Subject: PPSN-92: Call for Papers Message-ID: <9109261530.AA21742@arti1.vub.ac.be> Dear moderator, Can you put this Call for the PPSN-conference in the next number of your electronic magazine? This call is followed by the Latex version so that people may print it out if they want. I apologize for any inconvenience. Many thanks in advance, Bernard Manderick AI Lab VUB Pleinlaan 2 B-1050 Brussel tel.: +32/2/641.35.75 email: bernard at arti.vub.ac.be ------------------------------------------------------------------------------- CUT HERE FOR THE CALL ------------------------------------------------------------------------------- Call for Papers PPSN 92 Parallel Problem Solving from Nature Free University Brussels, Belgium 28-30 September 1992 The unifying theme of the PPSN-conference is ``natural computation'', i.e. the design, the theoretical and empirical understanding, and the comparison of algorithms gleaned from nature as well as their application to real-world problems in science, technology, etc. Characteristic for natural computation is the metaphorical use of concepts, principles, and mechanisms explaining natural systems. Examples are genetic algorithms, evolution strategies, algorithms based on neural networks, immune networks, and so on. A first focus of the conference is on problem solving in general, and learning and adaptiveness in particular. Since natural systems usually operate in a massively parallel way, a second focus is on parallel algorithms and their implementations. The conference scope includes but is not limited to the following topics: Physical metaphors such as simulated annealing, Biological metaphors such as evolution strategies, genetic algorithms, immune networks, classifier systems and neural networks insofar problem solving, learning and adaptability are concerned, and Transfer of other natural metaphors to artificial problem solving. Objectives of this conference are 1) to bring together scientists and practitioners working with these algorithms, 2) to discuss theoretical and empirical results, 3) to compare these algorithms, 4) to discuss various implementations on different parallel computer architectures, 5) to discuss applications in science, technology, administration, etc., and 6) to summarize the state of the art. For practical reasons, there will be both oral and poster presentations. The way of presentation of a paper does not say anything about its quality. Conference Chair: B. Manderick (VUB, Belgium) and H. Bersini (ULB, Belgium) Conference Address: PPSN - p/a D. Roggen - Dienst WEIN - Vrije Universiteit Brussel - Pleinlaan 2 - B-1050 Brussels - Belgium tel. +32/2/641.35.75 fax +32/2/641.28.70 email ppsn at arti.vub.ac.be Organizing Committee: D. Keymeulen, D. Roggen, P. Spiessens, J. Toreele (all VUB) Program Co-chairs: Y. Davidor (Israel) and H.-P. Schwefel (Germany) Program Committee: E.M.L. Aarts (The Netherlands) R.K. Belew (USA) K.A. de Jong (USA) J. Decuyper (Belgium) M. Dorigo (Italy) D.E. Goldberg (USA) M. Gorges-Schleuter (Germany) J.J. Grefenstette (USA) A.W.J. Kolen (The Netherlands) R. Maenner (Germany) W. Ebeling (Germany) J.-A. Meyer (France) H. Muehlenbein (Germany) F. Varela (France) H.-M. Voigt (Germany) Important Dates: April 1, 1992: Submission of papers (four copies) not exceeding 5000 words to be sent to the conference address. May 15, 1992: Notification of acceptance or rejection. June 15, 1992: Camera ready revised versions due. Sept. 28-30, 1992: PPSN-Conference. The proceedings will be published by Elsevier Publishing Company and will be available at the time of the conference. %----------------------------------------------------------------------------- % CUT HERE TO PRINT OUT THE CALL FOR PAPERS %------------------------------------------------------------------------------ % % call.tex Ma, 05.Sep.'91 LaTeX 2.09 % % call-for-papers to PPSN II in 1992 in Brussels, Belgium % \documentstyle{article} \pagestyle{empty} \makeatletter \def\deadline#1#2{% \par\@hangfrom{\hbox to 9em{\it #1\/\hfil}}#2\par } \def\committee#1#2{% \par\@hangfrom{\hbox to 15.5em{\bf #1\hfil}}#2\par } \makeatother \iffalse \topmargin -2.5cm \textwidth 15cm \textheight 25.5cm \else % % dinA4.sty Ho, 14.Dec.'89 LaTeX 2.09 % % LaTeX style modifications to change the paper size to DIN A4 \setlength{\textheight}{8.9in} % 22.6cm \setlength{\textwidth}{6.5in} % 16.5cm \setlength{\headheight}{12pt} % max.possible line heigth \setlength{\headsep}{25pt} \setlength{\footheight}{12pt} \setlength{\footskip}{25pt} \setlength{\oddsidemargin}{0.10in} % + 1in \setlength{\evensidemargin}{0.10in} % + 1in \setlength{\marginparwidth}{0.08in} \setlength{\marginparsep}{0.001in} % 0.1in + 0.08in + 0.001in = 3.0cm \setlength{\marginparpush}{0.4\parindent} \setlength{\topmargin}{-0.54cm} % 1in - 0.54cm = 2.0cm \setlength{\columnsep}{10pt} \setlength{\columnseprule}{0pt} % EOF dinA4.sty \fi \makeatletter % % myList.sty Ho, 14.Feb.'91 LaTeX 2.09 % % private variants of standard LaTeX lists %{ % % \begin{myList}{listname}{topsep}{parsep}{itemsep} % % \item ... % % \end{myList} % % base construct to realize re-defined standard lists % the seperation measures may be also given as \default. % \newenvironment{myList}[4]{% \edef\@myList{#1}% % \edef\@@myList{#2}% \ifx\@@myList\relax \else \setlength{\topsep}{\@@myList} \fi % \edef\@@myList{#3}% \ifx\@@myList\relax \else \setlength{\parsep}{\@@myList} \fi % \edef\@@myList{#4}% \ifx\@@myList\relax \else \setlength{\itemsep}{\@@myList} \fi % \begin{\@myList} }% {% \end{\@myList} }% \newcommand{\default}{} % % \begin{Itemize} % % \item ... % % \end{Itemize} % % single spaced, itemized list % \newenvironment{Itemize}{% \begin{myList}{itemize}{\parskip}{0pt}{0pt} }% {% \end{myList} }% % % \begin{Enumerate} % % \item ... % % \end{Enumerate} % % single spaced, enumerated list % \newenvironment{Enumerate}{% \begin{myList}{enumerate}{\parskip}{0pt}{0pt} }% {% \end{myList} }% % % \begin{enumerate} % % \item ... % % \end{enumerate} % % own enumeration style (1) ... (2) ... % which is a modification of LaTeX's enumerate. % \def\enumerate{% \def\labelEnumi{(\theenumi)}% \def\labelEnumii{(\theenumii)}% \def\labelEnumiii{(\theenumiii)}% \def\labelEnumiv{(\theenumiv)}% % \ifnum \@enumdepth > 3 \@toodeep \else \advance\@enumdepth \@ne \edef\@enumctr{\romannumeral\the\@enumdepth}% \list{% \csname labelEnum\@enumctr\endcsname }{% \usecounter{enum\@enumctr}% \def\makelabel##1{\hss\llap{##1}}% }% \fi } \def\endenumerate{% \endlist \@ignoretrue } % EOF myList.sty %} \makeatother \parskip=0.5\baselineskip \parindent=0pt \begin{document} {\large\em\centerline{Call for Papers}} \bigskip {\large\bf\centerline{PPSN~92}} {\large\bf\centerline{Parallel Problem Solving from Nature}} {\large\bf\centerline{Free University Brussels, Belgium}} {\large\bf\centerline{28--30 September 1992}} \bigskip \normalsize The unifying theme of the PPSN-conference is ``natural computation'', i.e. the design, the theoretical and empirical understanding, and the comparison of algorithms gleaned from nature as well as their application to real-world problems in science, technology, etc. Characteristic for natural computation is the metaphorical use of concepts, principles, and mechanisms explaining natural systems. Examples are genetic algorithms, evolution strategies, algorithms based on neural networks, immune networks, and so on. A first focus of the conference is on problem solving in general, and learning and adaptiveness in particular. Since natural systems usually operate in a massively parallel way, a second focus is on parallel algorithms and their implementations. The conference {\em scope\/} includes but is not limited to the following topics: \begin{Itemize} \item Physical metaphors such as simulated annealing, \item Biological metaphors such as evolution strategies, genetic algorithms, immune networks, classifier systems and neural networks insofar problem solving, learning and adaptability are concerned, and \item Transfer of other natural metaphors to artificial problem solving. \end{Itemize} {\em Objectives\/} of this conference are 1)~to bring together scientists and practitioners working with these algorithms, 2)~to discuss theoretical and empirical results, 3)~to compare these algorithms, 4)~to discuss various implementations on different parallel computer architectures, 5)~to discuss applications in science, technology, administration, etc., and 6)~to summarize the state of the art. For practical reasons, there will be both oral and poster presentations. The way of presentation of a paper does not say anything about its quality. \medskip \committee{Conference Chair:}% {B. Manderick (VUB, Belgium) and H. Bersini (ULB, Belgium)} \committee{Conference Address:}% {PPSN - p/a D. Roggen - Dienst WEIN - Vrije Universiteit Brussel - Pleinlaan 2 -- B-1050 Brussels -- Belgium -- {\bf tel.} +32/2/641.35.75 -- {\bf fax} +32/2/641.28.70 -- {\bf email} ppsn at arti.vub.ac.be} \committee{Organizing Committee:}% {D. Keymeulen, D. Roggen, P. Spiessens, J. Toreele (all VUB)} \committee{Program Co-chairpersons:}% {Y. Davidor (Israel) and H.-P. Schwefel (Germany)} {\bf Program Committee:} \\ \begin{tabular}{@{}lll} E.M.L. Aarts (The Netherlands) & R.K. Belew (USA) & K.A. de Jong (USA) \\ J. Decuyper (Belgium) & M. Dorigo (Italy) & D.E. Goldberg (USA) \\ M. Gorges-Schleuter (Germany) & J.J. Grefenstette (USA) & A.W.J. Kolen (The Netherlands) \\ R. M\"{a}nner (Germany) & W. Ebeling (Germany) & J.-A. Meyer (France) \\ H. M\"{u}hlenbein (Germany) & F. Varela (France) & H.-M. Voigt (Germany) \\ \end{tabular} \medskip \begingroup \deadline{\bf Important Dates:}{} \parskip=0pt \deadline{April 1, 1992:}{Submission of papers (four copies) not exceeding 5000 words to be sent to the conference address.} \deadline{May 15, 1992:}{Notification of acceptance or rejection.} \deadline{June 15, 1992:}{Camera ready revised versions due.} \deadline{Sept.~28-30, 1992:}{PPSN-Conference.} \endgroup The proceedings will be published by Elsevier Publishing Company and will be available at the time of the conference. \end{document} From ashley at spectrum.cs.unsw.oz.au Fri Sep 27 12:51:58 1991 From: ashley at spectrum.cs.unsw.oz.au (Ashley Aitken) Date: Fri, 27 Sep 91 11:51:58 EST Subject: Biological Threshold Function Message-ID: <9109270152.2118@munnari.oz.au> G'day, In browsing literature to help construct a simple information processing model of (biological) neurons I have seen little discussion of the threshold function - that is, how the threshold (membrane potential) of a neuron changes. I understand that there is a refractory period after each particular action potential pulse (which effectively represents a change in the threshold value, and hence limits the burst frequency) and their is a period of super-excitability after this. However, I am interested in changes on a larger timescale. Is there local change in the threshold value (c.f. the biases in artificial neural networks) on a larger timescale ? Is there any global modulation of the threshold values of groups of neurons (say in an area of the cortex) ? If anyone has any comments or references relevant to these questions I would be most grateful if they could email them to me - I will summarize if there is interest and a good response. Regards, Ashley Aitken. -- E-MAIL : ashley at spectrum.cs.unsw.oz.au AARNet SNAIL : University of New South Wales PO Box 1 Kensington NSW 2033 Australia "... any day above ground is a good day ..." J. Reyne? From btan at bluering.cowan.edu.au Fri Sep 27 03:34:56 1991 From: btan at bluering.cowan.edu.au (btan@bluering.cowan.edu.au) Date: Fri, 27 Sep 91 15:34:56 +0800 Subject: Add me messages Message-ID: <9109270734.AA12508@bluering.cowan.edu.au> Dear Sir/Madam My name is Stanley Tan. I am interested to be part of your connection. I am calling from Cowan University. My e_mail address is btan at bluering@cowan.edu.au Please add me messages, many thanks With regards Stanley Tah Australia From smagt at fwi.uva.nl Fri Sep 27 10:12:10 1991 From: smagt at fwi.uva.nl (Patrick van der Smagt) Date: Fri, 27 Sep 91 16:12:10 +0200 Subject: neural robotics motor control Message-ID: <9109271412.AA12259@fwi.uva.nl> Yamamoto Yoshio writes: >I am also interested in getting >information about other people's works on similar subjects beside Grossberg >and Kuperstein's work and Miller et. al.'s CMAC type approach. I have recently published an article on neural robotic control: P. P. van der Smagt & B. J. A. Kr\"ose, `A real-time learning neural robot controller,' Proceedings of the 1991 International Conference on Artificial Neural Networks, Espoo, Finland, 24-28 June, 1991, pp. 351--356. It describes a fast-learning self-supervised robot controller which remains adaptive during operation. Patrick van der Smagt From koch%CitJulie.Bitnet at BITNET.CC.CMU.EDU Fri Sep 27 17:27:32 1991 From: koch%CitJulie.Bitnet at BITNET.CC.CMU.EDU (koch%CitJulie.Bitnet@BITNET.CC.CMU.EDU) Date: Fri, 27 Sep 91 14:27:32 PDT Subject: Threshold changes Message-ID: <910927142732.2080c452@Juliet.Caltech.Edu> Yes, there exists good evidence in the PNS and CNS that various neuro- modulatotion--- can have similar effects. Since individual axons from neurons .quit From tackett at ipla00.dnet.hac.com Mon Sep 30 09:14:16 1991 From: tackett at ipla00.dnet.hac.com (Walter Tackett) Date: Mon, 30 Sep 91 09:14:16 EDT Subject: No subject Message-ID: <9109301614.AA27054@ipla00.ipl.hac.com> Does anyone have a reasonably complete list of papers on conjugate gradient methods for training multilayer perceptrons? If so, i'd appreciate it. From attou at ICSI.Berkeley.EDU Mon Sep 30 13:53:56 1991 From: attou at ICSI.Berkeley.EDU (abdelghani attou) Date: Mon, 30 Sep 91 11:53:56 MDT Subject: neural robotics motor control In-Reply-To: Your message of "Fri, 27 Sep 91 16:12:10 +0100." <9109271412.AA12259@fwi.uva.nl> Message-ID: <9109301853.AA06596@icsib45.berkeley.edu.Berkeley.EDU> hello well i got your email regarding the paper on neural robotic control , is it possible that you can send me a copy . thanks From jb%s1.gov at CARNEGIE.BITNET Fri Sep 20 19:11:50 1991 From: jb%s1.gov at CARNEGIE.BITNET (jb%s1.gov@CARNEGIE.BITNET) Date: Fri, 20 Sep 91 16:11:50 PDT Subject: Could some one tell me ... ? Message-ID: <9109202311.AA08365@perseus.s1.gov> The property (2) is called detailed balance resulting in a Gibbs distribution for the probability to find the system in a particular state. The rule (1) is an update procedure for the spin Sk which ensure detailed balance provided that E is an energy. Both principles are fundamental facts of statistical mechanics of neural networks (or if you prefer result from an maximum entropy analysis of neural nets). The book by Hertz Krogh and Palmer summerizes all that in a nice way. The book title is "Introduction to Neural Computation". Also consult D. Amit's book "Modelling brain functions" from Cambridge University Press. Joachim M. Buhmann University of California Lawrence Livermore National Laboratory 7000 East Ave., P.O.Box 808, L-270 Livermore, California 94550 email address: jb at s1.gov From ray%cs.su.oz.au at CARNEGIE.BITNET Mon Sep 23 07:13:11 1991 From: ray%cs.su.oz.au at CARNEGIE.BITNET (ray%cs.su.oz.au@CARNEGIE.BITNET) Date: Mon, 23 Sep 1991 21:13:11 +1000 Subject: Could some one tell me ... ? Message-ID: <01GAWVDE399CD7PXGF@BITNET.CC.CMU.EDU> I'll try to give a brief, intuitive, answer. For what might be a more formal and satisfying answer, try: van Laarhoven, P and Aarts, E "Simulated Annealing: Theory and Applications" Reidel Publishing (and Kluwer), 1987. Aarts, E, and Korst, J "Simulated Annealing and Boltzmann Machines", Wiley, 1989. > 1.What is the reason that the states of the k th unit Sk = 1 with > probability > 1 > Pk = ------------------- > 1+exp(-delta Ek/T) > > where delta Ek is the energy gap between the 1 and 0 states of the > k th unit, T is a parameter which acts like the temperature Its really only normal simulated annealing, in a slightly different form. Suppose we explicitly used traditional simulated annealing instead to determine the state of a unit. The energy of the s=0 state is E0=0, and the energy of the s=1 state is E1=Ek. Suppose also, for simplicity, that Ek>0. Now, since we must be in one state or the other: P(s=0,n) + P(s=1,n) = 1 Where s=x indicates the state of the unit, and n is the iteration number. At thermal equilibrium: P(s=0,n) = P(s=0,n-1) P(s=1,n) = P(s=1,n-1) So we will drop the second component to the function, and simply write P(s=0) and P(s=1). You can only transfer from s=0 to s=1, and vice versa. Let the probabilty of those transitions (given that you are already in the initial state) be designated P(0->1) and P(1->0) respectively. Since Ek>0, if we find ourself in the s=1 state, we will always immediately drop back to the s=0 state i.e. P(1->0) = 1 If we're in the s=0 state, then: P(0->1) = exp(-Ek/T) i.e. the normal simulated annealing function. At equilibrium, P(s=1) equals the probability of being in state s=0 at the previous iteration , and then making a successful transition to s=1 i.e.: P(s=1) = P(s=0) * P(0->1) = (1 - P(s=1)) * exp(-delta Ek/T) Rearranging the above equation gives - I think! - the Pk expression normally used for the Boltzmann Machine. > 2.Why this local decision rule ensures that in thermal equilibrium the > relative probability of two global states is determined by their > energy difference,and follows a Boltzmann distribution: > > Pa > ---- = exp(-(Ea - Eb))/T > Pb > > where Pa is the probability of being in the a th global state and > Ea is the energy of that state. It follows from noting that exp(x) * exp(y) = exp(x+y). Or, putting it another way P(A->B)*P(B->C) = P(A->C). > and how do we know that whether the system reaches thermal equilibrium > or not? There are formal ways of showing it. In practise, they're no help. The answer is "Run the machine for a very long time." In a lot of cases, "a very long time" is approximately equal to infinity, and this accounts for the mediocre performance of the Boltzmann Machine.