From pablo at iai.es Thu Mar 1 09:16:00 1990 From: pablo at iai.es (Pablo Bustos) Date: 1 Mar 90 15:16 +0100 Subject: linear separability Message-ID: <223*pablo@iai.es> Does anyone know a test that could decide if a given subset of the vertex of a binary hypercube is linearly separable from the rest of the set. We are looking for criteria in the sense of Hamming distance, connectivity , etc. instead of an iterative algorithm (perceptrons already do) Thanks From mclennan%MACLENNAN.CS.UTK.EDU at cs.utk.edu Thu Mar 1 15:18:22 1990 From: mclennan%MACLENNAN.CS.UTK.EDU at cs.utk.edu (mclennan%MACLENNAN.CS.UTK.EDU@cs.utk.edu) Date: Thu, 1 Mar 90 16:18:22 EDT Subject: tech report available Message-ID: <9003012118.AA03926@MACLENNAN.CS.UTK.EDU> ********** DO NOT DISTRIBUTE TO OTHER LISTS ********** The following technical report is available: Field Computation: A Theoretical Framework for Massively Parallel Analog Computation Parts I - IV Bruce MacLennan Department of Computer Science University of Tennessee CS-90-100 ABSTRACT This report comprises the first four parts of a systematic presentation of _field_computation_, a theoretical framework for understanding and designing massively parallel analog computers. This theory treats computation as the continuous transformation of fields: continuous ensembles of continuous-valued data. This theory is technology-independent in that it can be realized through optical and molecular processes, as well as through large neural networks. Part I is an overview of the goals and assumptions of field computation. Part II presents relevant ideas and results from functional analysis, including theorems concerning the field- computation of linear and multilinear operators. Part III is devoted to examples of the field computation of a number of use- ful linear and multilinear operators, including integrals, derivatives, Fourier transforms, convolutions and correlations. Part IV discusses the field computation of nonlinear operators, including: a theoretical basis for universal (general purpose) field computers, ways of replacing field polynomials by sigmoid transformations, and ways of avoiding higher-dimensional fields (since they may be difficult to represent in physical media). ------------------------------------------------------------------------ The report is available in PostScript form by anonymous ftp as follows: unix> ftp cheops.cis.ohio-state.edu (or, ftp 128.146.8.62) Name: anonymous Password: neuron ftp> cd pub/neuroprose ftp> get maclennan.fieldcomp.ps.Z ftp> quit unix> uncompress maclennan.fieldcomp.ps.Z unix> lpr maclennan.fieldcomp.ps (or however you print postscript files) ------------------------------------------------------------------------ Hardcopy will soon be available from: library at cs.utk.edu For all other correspondence: Bruce MacLennan Department of Computer Science 107 Ayres Hall The University of Tennessee Knoxville, TN 37996-1301 (615)974-0994/5067 maclennan at cs.utk.edu From slehar at bucasb.bu.edu Thu Mar 1 13:49:14 1990 From: slehar at bucasb.bu.edu (slehar@bucasb.bu.edu) Date: Thu, 1 Mar 90 13:49:14 EST Subject: Mathematical Tractability of Neural Nets In-Reply-To: "Helen M. Gigley"'s message of Wed, 28 Feb 90 14:55:41 EST <9002281455.aa02466@Note.NSF.GOV> Message-ID: <9003011849.AA01302@bucasd.bu.edu> AAAAH! Now I understand the source of the confusion! Your statement... "It is the subsequent analysis of function corresponding to this **linguistic theory** which underlies the development of the neural analysis of the brain areas at what you consider the **functional level**." (**my emphasis**) reveals that you and I are refering to altogether different types of neural models. You are doubtless refering to the connectionist variants of the Chomsky type linguistic models which represent language in abstract and rigidly functional and hierarchical terms. If you think that such models are excessively rigid and abstract, then you and I are in complete agreement. The neural models to which I refer are more in the Grossberg school of thought. Such models are characterized by a firm founding in quantitative neurological analysis, expression in dynamic systems terms, and are confirmed by psychophysical and behavioral data. In other words these models adhere closely to known biological and behavioral knowledge. For instance the Grossberg neural model for vision [3],[4],[5] (which is more my area of expertise) is built of dynamic neurons defined by differential equations derived from the Hodgkin Huxley equations (from measurement of the squid giant axon) and also from behavioral data [1]. The topology and functionality of the model is based again on neurological observation (such as Hubel & Wiesel intracellular measurement in the visual cortex) together with psychophysical evidence, particularly visual illusions such as perceptual grouping [10], color perception [6], neon color spreading [7],[8], image stabilization experiments [9] and others. The model duplicates and explains such illusory phenomena as well as elucidating aspects of natural vision processing. In their book VISUAL PERCEPTION, THE NEUROPHYSIOLOGICAL FOUNDATIONS (1990), Spillmann & Werner say "Neural models for cortical Boundary Contour System and Feature Contour System interactions have begun to be able to account for and predict a far reaching set of interdisciplinary data as manifestations of basic design principles, notably how the cortex achieves a resolution of uncertainties through its parallel and hierarchical interactions" The point is that this class of models is not based on arbitrary philosophising about abstract concepts, but rather on hard physical and behavioral data, and Grossbergs models have on numerous occasions made behavioral and anatomical predictions which were subsequently confirmed by experiment and histology. Such models therefore cannot be challenged on purely philosophical grounds, but simply on whether they predict the behavioral data, and whether they are neurologically plausible. In this sense, the models are scientifically testable, since they make concrete predictions of how the brain actually processes information, not vague speculations on how it might do so. So, I maintain my original conjecture that the time is ripe for a fusion of knowledge from the diverse fields of neurology, psychology, mathematics and artificial intelligence, and I maintain further that such a fusion is already taking place. REFERENCES ========== Not all of these are pertinant to the discussion at hand, (they were copied from another work) but I leave them in to give you a starting point for further research if you are interested. [1] Stephen Grossberg THE QUANTIZED GEOMETRY OF VISUAL SPACE The Behavioral and Brain Sciences 6, 625 657 (1983) Cambridge University Press. Section 21 Reflectance Processing, Weber Law Modulation, and Adaptation Level in Feedforward Shunting Competitive Networks. In this section Grossberg examines the dynamics of a feedforward on-center off-surround network of shunting neurons and shows how such a topology performs a normalization of the signal, i.e. a factorization of pattern and energy, preserving the pattern and discarding the overall illumination energy. Reprinted in THE ADAPTIVE BRAIN Stephen Grossberg Editor, North-Holland (1987) Chapter 1 Part II section 21 [2] Gail Carpenter & Stephen Grossberg A MASSIVELY PARALLEL ARCHITECTURE FOR A SELF-ORGANIZING NEURAL PATTERN RECOGNITION MACHINE Computer Vision, Graphics, and Image Processing (1987), 37, 54-115 Academic Press, Inc. This is a neural network model of an adaptive pattern classifier (Adaptive Resonance Theory, ART 1) composed of dynamic shunting neurons with interesting properties of stable category formation while maintaining plasticity to new pattern types. This is achieved through the use of resonant feedback between a data level layer and a feature level layer. The original ART1 model has been upgraded by ART2, which handles graded instead of binary patterns, and recently ART3 which uses a more elegant and physiologically plausible neural mechanism while extending the functionality to account for more data. Reprinted in NEURAL NETWORKS AND NATURAL INTELLIGENCE, Stephen Grossberg Editor, MIT Press (1988) Chapter 6. [3] Stephen Grossberg & Ennio Mingolla NEURAL DYNAMICS OF PERCEPTUAL GROUPING: TEXTURES, BOUNDARIES AND EMERGENT SEGMENTATIONS Perception & Psychophysics (1985), 38 (2), 141-171. This work presents the BCS / FCS model with detailed psychophysical justification for the model components and computer simulation of the BCS. [4] Stephen Grossberg & Ennio Mingolla NEURAL DYNAMICS OF SURFACE PERCEPTION: BOUNDARY WEBS, ILLUMINANTS, AND SHAPE-FROM-SHADING. Computer Vision, Graphics and Image Processing (1987) 37, 116-165. This model extends the BCS to explore its response to gradients of illumination. It is mentioned here because of an elegant modification of the second competitive stage that was utilized in our simulations. [5] Stephen Grossberg & Dejan Todorovic NEURAL DYNAMICS OF 1-D AND 2-D BRIGHTNESS PERCEPTION Perception and Psychophysics (1988) 43, 241-277. A beautifully lucid summary of BCS / FCS modules with 1-D and 2-D computer simulations with excellent graphics reproducing several brightness perception illusions. This algorithm dispenses with boundary completion, but in return it simulates the FCS operation. Reprinted in NEURAL NETWORKS AND NATURAL INTELLIGENCE, Stephen Grossberg Editor, MIT Press (1988) Chapter 3. [6] Land, E. H. THE RETINEX THEORY OF COLOR VISION Scientific American (1977) 237, 108-128. A mathematical theory that predicts the human perception of color in Mondrian type images, based on intensity differences at boundaries between color patches. [7] Ejima, Y., Redies, C., Takahashi, S., & Akita, M. THE NEON COLOR EFFECT IN THE EHRENSTEIN PATTERN Vision Research (1984), 24, 1719-1726 [8] Redies, C., Spillmann, L., & Kunz, K. COLORED NEON FLANKS AND LINE GAP ENHANCEMENT Vision Research (1984) 24, 1301-1309 [9] Yarbus, A. L. EYE MOVEMENTS AND VISION New York: Plenum Press (1967) a startling demonstration of featural flow in human vision. [10] Beck, J. TEXTURAL SEGMENTATION, SECOND-ORDER STATISTICS, AND TEXTURAL ELEMENTS. Biological Cybernetics (1983) 48, 125-130 [11] Beck, J., Prazdny, K., & Rosenfeld, A. A THEORY OF TEXTURAL SEGMENTATION in J. Beck, B. Hope, & A. Rosenfeld (Eds.), HUMAN AND MACHINE VISION. New York: Academic Press (1983) [12] Stephen Grossberg SOME PHYSIOLOGICAL AND BIOCHEMICAL CONSEQUENCES OF PSYCHOLOGICAL POSTULATES Proceedings of the National Academy of Sciences (1968) 60, 758-765. Grossbergs original formulation of the dynamic shunting neuron as derived from psychological and neurobiological considerations and subjected to rigorous mathematical analysis. Reprinted in STUDIES OF MIND AND BRAIN Stephen Grossberg, D. Reidel Publishing (1982) Chapter 2. [13] David Marr VISION Freeman & Co. 1982. In a remarkably lucid and well illustrated book Marr presents a theory of vision which includes the Laplacian operator as the front-end feature extractor. In chapter 2 he shows how this operator can be closely approximated with a difference of Gaussians. [14] Daugman J. G., COMPLETE DISCRETE 2-D GABOR TRANSFORMS BY NEURAL NETWORKS FOR IMAGE ANALYSIS AND COMPRESSION I.E.E.E. Trans. Acoustics, Speech, and Signal Processing (1988) Vol. 36 (7), pp 1169-1179. Daugman presents the Gabor filter, the product of an exponential and a trigonometric term, for extracting local spatial frequency information from images; he shows how such filters are similar to receptive fields mapped in the visual cortex, and illustrates their use in feature extraction and image compression. [15] Stephen Grossberg CONTOUR ENHANCEMENT, SHORT TERM MEMORY, AND CONSTANCIES IN REVERBERATING NEURAL NETWORKS Studies in Applied Mathematics (1973) LII, 213-257. Grossberg analyzes the dynamic behavior of a recurrent competitive field of shunting neurons, i.e. a layer wherein the neurons are interconnected with inhibitory synapses and receive excitatory feedback from themselves, as a mechanism for stable short term memory storage. He finds that the synaptic feedback function is critical in determining the dynamics of the system, a faster than linear function such as f(x) = x*x results in a winner-take-all choice, such that only the maximally active node survives and suppresses the others in the layer. A sigmoidal function can be tailored to produce either contrast enhancement or winner-take-all, or any variation in between. From sg at corwin.ccs.northeastern.edu Thu Mar 1 14:05:58 1990 From: sg at corwin.ccs.northeastern.edu (steve gallant) Date: Thu, 1 Mar 90 14:05:58 EST Subject: linear separability Message-ID: <9003011905.AA10169@corwin.CCS.Northeastern.EDU> It's not easy to tell whether a set of vertices is separable or not even with perceptron learning, because you don't know whether the set is nonseparable or whether you just haven't run enough iterations. One approach is to cycle through the training examples and keep track of the weights on the output cell. Either perceptron learning will find a solution (separable case) or a set of weights will reappear (nonseparable case). Another method is the Ho-Kashyap procedure (see Duda & Hart), but there's still no good bound on how much work is required to determine separability. Steve Gallant From yann at lesun.att.com Thu Mar 1 13:25:07 1990 From: yann at lesun.att.com (Yann le Cun) Date: Thu, 01 Mar 90 13:25:07 -0500 Subject: linear separability In-Reply-To: Your message of 01 Mar 90 15:16:00 +0100. Message-ID: <9003011825.AA24715@lesun.> > Does anyone know a test that could decide if a given subset of the vertex of > a binary hypercube is linearly separable from the rest of the set. > ... (perceptrons already do) Perceptrons tell you if two sets are LS, they don't tell you anything if they are not LS: they just keep ocsillating. The Ho-Kashyap procedure (IEEE Trans. Elec. Comp. EC14 october 1965) tells you if two sets are linearly separable, and gives you a solution if they are. It is described in the book by Duda and Hart "pattern classification and scene analysis" Wiley and Son, (1973). - Yann Le Cun, Bell Labs Holmdel. From sayegh at ed.ecn.purdue.edu Thu Mar 1 18:47:15 1990 From: sayegh at ed.ecn.purdue.edu (Samir Sayegh) Date: Thu, 1 Mar 90 18:47:15 -0500 Subject: NN conference Indiana_Purdue Ft Wayne Message-ID: <9003012347.AA00854@ed.ecn.purdue.edu> Third Conference on Neural Networks and Parallel Distributed Processing Indiana-Purdue University A conference on NN and PDP will be held April 12, 13 and 14, 1990 on th common campus of Indiana and Purdue University at Ft Wayne. The emphasis of this conference will be Vision and Robotics although all contributions are w are welcome. People from the Midwest are particularly encouraged to attend and contribute especially since the "major" NN conferences seem to oscillate between the East and West Coast! Send abstracts and inquiries to: Dr. Samir Sayegh Physics Department Indiana Purdue University Ft Wayne, IN 46805 email: sayegh at ed.ecn.purdue.edu sayegh at ipfwcvax.bitnet FAX : (219) 481-6800 Voice: (219) 481-6157 From bates at amos.ucsd.edu Thu Mar 1 21:04:06 1990 From: bates at amos.ucsd.edu (Elizabeth Bates) Date: Thu, 1 Mar 90 18:04:06 PST Subject: Mathematical Tractability of Neural Nets Message-ID: <9003020204.AA10645@amos.ucsd.edu> Although I am grateful for the references -- and very aware that there are alternative approaches and multiple points of view in the world of neural networks -- I must reiterate: my comments to you were not addressed to neural networking per se (my qualifications are relatively limited in that regard) but to the claims you were making on the network regarding what is supposedly "known" about brain organization for language. I can only hope that the references you cite are not trying to fit their models to a false reality (i.e. Broca's area = grammar, and so on). -liz bates From Dave.Touretzky at B.GP.CS.CMU.EDU Fri Mar 2 02:26:21 1990 From: Dave.Touretzky at B.GP.CS.CMU.EDU (Dave.Touretzky@B.GP.CS.CMU.EDU) Date: Fri, 02 Mar 90 02:26:21 EST Subject: language, vision, and connectionism Message-ID: <6501.636362781@DST.BOLTZ.CS.CMU.EDU> Steve Lehar's argument ignores a crucial point: low-level vision is a specialized area with properties that make it more tractable for computational modeling than most other aspects of intelligence. First, low-level vision is concerned with essentially geometric phenomena, like edge detection. Geometric phenomena are especially easy to work with because they involve mostly local computations, and the representations are (relatively!) intuitive and straightforward. Basically they are iconic representations on retinotopic maps. Compare this with high level vision tasks (e.g., 3D object recognition), or language tasks, where the brain's representations are completely unknown, probably not iconic, and almost certainly not intuitive or straightforward. A second aspect of low-level vision is that it involves only the earliest stages of the visual system, which are (relatively!) easily explored in a neuroscience lab. It's not that hard now, after several decades of practice, to make autoradiographs of ocular dominance patterns in LGN, for example, or to work out the mapping from the retina to the surface of area 17. It's much harder to try to explore the higher level visual areas where objects are recognized, in part because the circuitry gets too complex as you go deeper into the brain, and in part because, since the highest areas don't use simple geometric representations, techniques like autoradiography or receptive field mapping are not applicable. As Jim Bower pointed out, even in the earliest stages of vision there are lots of unanswered basic questions, such as why the feedback connections from visual cortex to LGN vastly outnumber the feedforward connections. Also, the wiring of the different layers of cells in primary visual cortex (only a few synapses in from the retina) is not fully known. If we know so little about vision after decades of studies on cats and monkeys, think about how much less we know about language. -- Dave From tmb at ai.mit.edu Fri Mar 2 02:29:32 1990 From: tmb at ai.mit.edu (Thomas M. Breuel) Date: Fri, 2 Mar 90 02:29:32 EST Subject: linear separability Message-ID: <9003020729.AA04172@rice-chex> |It's not easy to tell whether a set of vertices is separable |or not even with perceptron learning, because you don't know whether the |set is nonseparable or whether you just haven't run enough iterations. |One approach is to cycle through the training examples and keep track of |the weights on the output cell. Either perceptron learning will find a |solution (separable case) or a set of weights will reappear (nonseparable |case). Another method is the Ho-Kashyap procedure (see Duda & Hart), but |there's still no good bound on how much work is required to determine |separability. To get a bound on the amount of work required to determined linear separability, observe that linear separability is easily transformed to a linear programming problem. From AMR at IBM.COM Fri Mar 2 09:16:35 1990 From: AMR at IBM.COM (AMR@IBM.COM) Date: Fri, 2 Mar 90 09:16:35 EST Subject: Mathematical Tractability of Neural Nets Message-ID: I would like to commend the content though not the tone of the recent remarks by Bates and Bower. While all their criticisms are certainly right, I would nevertheless hope for more, not less, in the way of attempts (no matter how quixotic) to reach across the gap that separates the disciplines more than it does their respective subject matters. It may be that biologists and linguists (to take two extreme cases) will simply have to work each in their own garden for a century, say, before their results on brain activity begin to converge, and that current attempts at a linkage of the kind that some connectionist literature seems to envisage between neural architecture and cognitive behavior are doomed to failure. On the other hand, it seems to me that the history of science is so full of cases where progress was delayed because people were not ready to see connections between initially distinct areas of study that the only lesson to draw is that if somebody is willing to invest the time and the effort to pursue the connections, let them. I myself, as primarily a linguist, find myself in the following predicament. On the one hand, no one has shown me how linguistic behavior can arise out of neural nets, but, on the other hand, no one has shown me how it can arise out of formal grammars or Turing machines, either. I am very uncomfortable with the idea that, given the obvious primitiveness of current neural models, we should accept, as our idea of what makes language possible, anything like a grammar in the conventional sense. Linguists of an earlier age were very careful (and most non-theoretically inclined linguists still are) in viewing grammars as descriptions of the data, not as models of anything that goes on inside human beings. Unfortunately, the direction of theoretical linguistics has been largely one where the wonderful idea that we SHOULD find out what goes on inside has been confused with the deplorable conceit that we CAN find this out by doing little more than armchair grammar writing but ATTRIBUTING to these grammars what has come to be known as psychological reality. A few of us are trying, in linguistics, to right the balance and to develop an alternative in which the need to have grammars as human-readable descriptions of masses of data does not lead to the assumption that these same grammars are at all useful as models of human cognition or neurology. It may well be that we will find that very few linguistic facts can be explained, given current knowledge, by reference to something more basic, and that the bulk of the routine grammatical information that we know perfectly how to DESCRIBE will continue to elude a psychological (much less a neurological) account for a long time. But it is a case, I submit, of having a choice of a very few pearls or a whole lot of swine. And in the search for the former I am willing to get help anywhere I can find it. Alexis Manaster-Ramer IBM Research POB 704 Yorktown Heights, NY 10510 (914) 784-7239 From AMR at IBM.COM Fri Mar 2 09:53:44 1990 From: AMR at IBM.COM (AMR@IBM.COM) Date: Fri, 2 Mar 90 09:53:44 EST Subject: Generative Power of Neural Nets Message-ID: Some time ago I posed the question of the weak generative power of neural nets, and in particular, their equivalence or otherwise to Turing machines. This elicited a number of responses, and I am by no means done reading the entire literature that has been kindly sent to me. I would like to offer an update, however. (1) There seems to be no consensus on whether the issue is significant, although I feel (as I have said in this forum, without getting any response) that the arguments for ignoring such issues (at least the ones I saw) were based on fundamental misunderstandings of the mathematics and/or its applications. (2) There seems to be a strong feeling among those who like the neural nets that, regardless of what their generative power is, they are interestingly different from, say, Turing machines, but as far as I have been able to determine, no one has any concrete proposals as how such a notion of difference (or a corresponding notion of equivalence) are to be defined. Instead, it would appear that some of the misguided arguments against being concerned with formal issues (mentioned in (1) above) may arise out of a anything-but-misguided intuition that difference or equivalence in terms of weak generative power is not the relevant criterion combined with the indeed-quite-misguided notion that formal theories of computation and the like cannot characterize any other such criterion. (3) There ARE a number of results (or ideas that could easily be turned into results) about the weak generative power, but they conflict at first glance quite a bit with each other. Thus, on some views, neural nets are equivalent to finite state machines, on others to Turing machines, and yet on others they are MORE powerful than Turing machines. The last position sounds at first glance the most intriguing. However, the results that point in this direction are based simply on assuming infinite nets which are essentially (unless I am very wrong) like what you would get if you took the idea of a finite-state machine (or the finite control of a Turing machine, which is the same thing, essentially) and allowed an infinite number of states. In that case, you could easily get all sorts of non-Turing-computable things to "compute", but most would I think dispute that this adds anything to our original idea that "computable" should mean "Turing-computable", since crucially you would need to allow infinite-length computations. On the hand, the arguments that reduce neural nets to finite-state machines are based on the simple idea that actual nets must be strictly finite (without even the infinite tape which we allow Turing machines to have). As has been pointed out, the same applies to any physically realized computer such as the IBM AT I am using to type these words. A Turing machine (or even a PDA) cannot be physically realized, only a finite-state machine can. However, both of these ways of viewing the situation seem to me to be fruitless. The Turing model has been so succesful because, while literally quite false of the computers we build (because of the infinite tape), it has proved useful in understanding what real computation does, anyway. The point is difficult to make briefly, but it boils down to the observation that the finite- state machine model does not distinguish a random finite list of strings, for example, from something like all strings of the form a^n b^n for n up to 20,000. The standard Turing model does do this, by allowing us to pretend in the second case that we are really recognizing the infinite language a^n b^n for all n. As a number of people have pointed out, this is really cheating but it does not appear to do any harm, at least among those who understand the rules of the game. Thus, it seems to me that the important results would be along the lines of showing that various types of neural nets are equivalent to Turing machines, or that, if they are not, then the distinction is NOT simply due either to assuming strict finiteness (i.e. no way to simulate the TM tape) or else by assuming strict infiniteness (i.e. an infinite control). It is by no means clear to me that we have a convincing answer in this case, since it seems that some models that have been defined are NOT (or at least not obviously) as powerful as TMs. I would welcome additional clarification about this point. (4) I personally feel that the next step would have to be to develop a different way of measuring equivalence (as I have been hinting all along), since this seems to me to be the intuition that almost everybody has, but I have seen no ideas directed to this. Some of my colleagues and I have been struggling with doing something like this for a different domain (namely, the comparison of different linguistic formalisms), but progress has been difficult. I suspect that neural nets would be an ideal testing ground for any proposals along these lines, and I would be grateful for any ideas. Just to make this a little more concrete, let me give an example: (5) It is clear that a Turing machine, a Von Neumann machine, and a neural net are not the same thing, yet they may have the same weak generative power. The question is whether there is any way to develop a useful mathematical theory in which the first two are in some sense equivalent to each other but neither is to the third. It seems to me that this is the intuition of many people, including by definition (I would think) all connectionists, but there is no basis currently for deciding whether those who feel so are right. Alexis Manaster-Ramer IBM Research POB 704 Yorktown Heights, NY 10598 From sg at corwin.ccs.northeastern.edu Fri Mar 2 12:40:14 1990 From: sg at corwin.ccs.northeastern.edu (steve gallant) Date: Fri, 2 Mar 90 12:40:14 EST Subject: linear separability Message-ID: <9003021740.AA02286@corwin.CCS.Northeastern.EDU> > To get a bound on the amount of work required to determined linear > separability, observe that linear separability is easily transformed > to a linear programming problem. Good suggestion. The transformation is (presumably) to an LP problem that has a minimum solution of 0 iff the original data is separable. This certainly would give an exponential bound based upon the simplex method. I wonder whether polynomial methods would give a polynomial bound here or not. The potential problem is getting a series of approximations that converge toward 0, but not being able to tell whether the solution is EXACTLY 0. Perhaps you or someone else familiar with polynomial methods could comment on this? Steve Gallant From well!mitsu at apple.com Fri Mar 2 15:30:55 1990 From: well!mitsu at apple.com (Mitsuharu Hadeishi) Date: Fri, 2 Mar 90 12:30:55 pst Subject: Mathematical Tractability of Neural Nets Message-ID: <9003022030.AA20061@well.sf.ca.us> Just as a side note to those who have posted what appear to be personal replies to the whole Connectionists list, to remind you that to send a reply only to the originator of the message (and not to everyone who received it as well), use capital R instead of small r when replying. This is not a complaint, but just a reminder in case people are accidentally sending messages intended to be personal to the entire list. From Scott.Fahlman at B.GP.CS.CMU.EDU Sat Mar 3 09:36:16 1990 From: Scott.Fahlman at B.GP.CS.CMU.EDU (Scott.Fahlman@B.GP.CS.CMU.EDU) Date: Sat, 03 Mar 90 09:36:16 EST Subject: Mathematical Tractability of Neural Nets In-Reply-To: Your message of Fri, 02 Mar 90 12:30:55 -0800. <9003022030.AA20061@well.sf.ca.us> Message-ID: Just as a side note to those who have posted what appear to be personal replies to the whole Connectionists list, to remind you that to send a reply only to the originator of the message (and not to everyone who received it as well), use capital R instead of small r when replying. The proper way to reply will vary from one mail program to another, and there are dozens of different mailers in common use. Please learn to use your own mailer properly, whatever it may be. -- Scott From bates at amos.ucsd.edu Sat Mar 3 15:53:12 1990 From: bates at amos.ucsd.edu (Elizabeth Bates) Date: Sat, 3 Mar 90 12:53:12 PST Subject: Mathematical Tractability of Neural Nets Message-ID: <9003032053.AA05721@amos.ucsd.edu> AMR's point about the need for collaboration is well taken -- and as a scientist who is virtually obsessed with collaboration (e.g. cross-linguistic projects over three continents that we've somehow managed to keep afloat for 15 years) I would be the last to suggest that we work in our own gardens for a few more decades. Indeed, I think we are in the middle of a particularly promising time for an interaction between neuroscience and the various subfields of language research. A few concrete examples: The WAshington University work on brain metabolism during lexical processing, exciting new psycholinguistic research using electrophysiological indices (a 6-dimensional outcome measure that puts old-fashioned button-press techniques to shame) by Kutas, van Petten, Neville, Holcomb, Garnsey and others, new "on-line" studies of aphasia that are telling us a great deal about real-time processing in aphasic patients using techniques that were not possible 10 - 15 years ago, developmental studies of infants with focal brain injury that are looking PROSPECTIVELY at the process of recovery, for the very first time -- and this is just a small sample. The technical advances are great, and the opportunities are even greater. I also believe that connectionism will offer new theoretical and experimental tools for examining language breakdown in aphasia -- such as the Hinton/Shallice or McClelland/Seidenberg/Patterson collaborations that I cited earlier. In short (and I have gone on too long), my point was really a simple one: the old view of brain organization for language appears to have been disconfirmed, quite roundly, and the field of aphasiology is currently seeking a completely new way of characterizing contrasting forms of language impairment following focal brain injury. I was answering Slehar's proposal that we "follow the neurologists" and accept the old story (e.g. Broca's area = the grammar center, and so on). But I would NEVER want or mean to suggest that we give up!! Of course language is difficult to study (compared, as Touretzky points out, with low level vision), but it also has its advantages: (1) it is the only public form of cognition, out there for all of us to say, and (2) for that reason language is perhaps the best understood and most easily measured of all higher cognitive processes. We do indeed live in interesting times, and I am sure we have some real breakthroughs ahead of us in a cognitive neuroscience of language.. -liz From munnari!cluster.cs.su.oz.au!ray at uunet.uu.net Sat Mar 3 02:41:40 1990 From: munnari!cluster.cs.su.oz.au!ray at uunet.uu.net (munnari!cluster.cs.su.oz.au!ray@uunet.uu.net) Date: Sat, 3 Mar 90 18:41:40 +1100 Subject: No subject Message-ID: <9003030743.6898@munnari.oz.au> >From ml-connectionists-request%q.cs.cmu.edu at murtoa.cs.mu.oz Sat Mar 3 14:40:20 1990 From tmb%ai.mit.edu at murtoa.cs.mu.oz Fri Mar 2 02:29:32 1990 From: tmb%ai.mit.edu at murtoa.cs.mu.oz (Thomas M. Breuel) Date: Fri, 2 Mar 90 02:29:32 EST Subject: linear separability Message-ID: <9003020729.AA04172@rice-chex> |It's not easy to tell whether a set of vertices is separable |or not even with perceptron learning, because you don't know whether the |set is nonseparable or whether you just haven't run enough iterations. |One approach is to cycle through the training examples and keep track of |the weights on the output cell. Either perceptron learning will find a |solution (separable case) or a set of weights will reappear (nonseparable |case). Another method is the Ho-Kashyap procedure (see Duda & Hart), but |there's still no good bound on how much work is required to determine |separability. To get a bound on the amount of work required to determined linear separability, observe that linear separability is easily transformed to a linear programming problem. From ato at breeze.bellcore.com Sun Mar 4 13:53:37 1990 From: ato at breeze.bellcore.com (Andrew Ogielski) Date: Sun, 4 Mar 90 13:53:37 -0500 Subject: Linear separability Message-ID: <9003041853.AA19720@breeze.bellcore.com> The question of algorithmic complexity of determinining linear separability of two sets of vectors (with real or binary components, it does not matter) is an old story, and the answer is well known: As was mentioned in a previous posting (and was widely known among early threshold gate enthusiasts in the 1960's) linear separability is easily transformed into a linear programming problem. For those who don't see it right away - just write down linear inequalities for the coordinates of a normal vector for a separating hyperplane ("weights" in the nn jargon). Linear programs can be solved in polynomial time (polynomial in the problem size, i.e. number of variables, number of inequalities, and the number of bits, when needed) unless one restricts the solution to integers. The first proof has been achieved by Khachiyan in his ellipsoid method. A more recent provably polynomial algorithm is due to Karmarkar. As an aside: despite the fact that the simplex method is not polynomial (there exist instances of linear programs where simplex methods would take exponential number of iterations in the program size), it works extremely well in majority of cases, including separability of binary vectors. The latter special case usually is not very sparse , which does not allow to benefit fully from interior point methods such as the Karmarkar's algorithm. A good, recent reference is A. Schrijver, Theory of Linear and Integer Programming, John Wiley & Sons, Chichester 1987. Andy Ogielski P.S. Perhaps it is proper to mention here that relaxation methods for solving systems of linear inequalities ( the "perceptron algorithm" is in this category) have been known and thoroughly explored by mathematicians well before the perceptrons. If I remember correctly, this fact received an overdue acknowledgment in the new edition of the Minsky & Papert book. If anybody needs it, I can dig out proper references. ato From ray at cluster.cs.su.oz.au Sun Mar 4 22:34:40 1990 From: ray at cluster.cs.su.oz.au (ray@cluster.cs.su.oz.au) Date: Mon, 5 Mar 1990 14:34:40 +1100 Subject: No subject Message-ID: <9003050336.10396@munnari.oz.au> >From ml-connectionists-request%q.cs.cmu.edu at murtoa.cs.mu.oz Sat Mar 3 14:40:20 1990 From tmb%ai.mit.edu at murtoa.cs.mu.oz Fri Mar 2 02:29:32 1990 From: tmb%ai.mit.edu at murtoa.cs.mu.oz (Thomas M. Breuel) Date: Fri, 2 Mar 90 02:29:32 EST Subject: linear separability Message-ID: <9003020729.AA04172@rice-chex> |It's not easy to tell whether a set of vertices is separable |or not even with perceptron learning, because you don't know whether the |set is nonseparable or whether you just haven't run enough iterations. |One approach is to cycle through the training examples and keep track of |the weights on the output cell. Either perceptron learning will find a |solution (separable case) or a set of weights will reappear (nonseparable |case). Another method is the Ho-Kashyap procedure (see Duda & Hart), but |there's still no good bound on how much work is required to determine |separability. To get a bound on the amount of work required to determined linear separability, observe that linear separability is easily transformed to a linear programming problem. From om%icsib15.Berkeley.EDU at jade.berkeley.edu Mon Mar 5 15:46:19 1990 From: om%icsib15.Berkeley.EDU at jade.berkeley.edu (Stephen M. Omohundro) Date: Mon, 5 Mar 90 12:46:19 PST Subject: linear separability test. In-Reply-To: Tal Grossman's message of Mon, 05 Mar 90 17:56:17 +0200 <9003052005.AA05814@icsi> Message-ID: <9003052046.AA14381@icsib15.berkeley.edu.> A couple of simple transformations will convert the linear separability question from intersection of two convex hulls to just the containment of a point in a single convex hull. First realize that we can focus attention just on hyperplanes through the origin by embedding the original problem in the "z=1" plane one dimension higher. Each arbitrary hyperplane in the original problem then corresponds to a hyperplane through the origin in this bigger space. Consider the unit vector v which is perpendicular to such a plane. The plane is separating if the dot product of this vector with each point in set1 is negative and with each point in set2 is positive. If we reflect a point through the origin (ie. (x1,...,xn) -> (-x1,...,-xn)) then the sign of its dot product with any vector also changes. Thus if we do such reflection on each point in set1, we have reduced the problem to finding a vector whose dot product with each of a set of points is positive. The existence of such a vector is equivalent to the origin not being included in the convex hull of the points. --Stephen Omohundro From huyser at mithril.stanford.edu Mon Mar 5 18:30:15 1990 From: huyser at mithril.stanford.edu (Karen Huyser) Date: Mon, 5 Mar 90 15:30:15 PST Subject: linear separability Message-ID: <9003052330.AA02094@mithril.Stanford.EDU> There is one Hamming-type test I know of, and that's a test for summability. For a function of N variables to be linearly separable, it must be asummable. To be asummable, it must be 2-asummable, 3-asummable, ... , k-asummable, ... , N-asummable. Any function is k-asummable if it is not k-summable. 2-summability is described below. If a boolean function is 2-summable, there exists the following relationship between its vertices. 100 101 Imagine the faces (2-D subspaces) of a cube (hypercube). 0 ------ 1 Assign one-bit output values to each corner of the cube. | | (If multiple-bit output, make multiple hypercubes.) | | If any face of the hypercube has an assignment like that 1 ------ 0 to the left, the problem is not linearly separable. 000 001 This "exor" relation between pairs of vertices corresponds to the function being "2-summable". Specifically, if we add the vectors that label the 1-corners (000 + 101 = 101) and those that label the 0-corners (100 + 001 = 101), they add together to the same vector value (they sum in pairs, hence 2-summable). Similar relationships between three, four, etc. vertices corresponds to 3-summability, 4-summability, etc. Any function that is 2-summable is not linearly separable. A simple test of 2-summability is to form all possible sets of four training vectors and test for this exor condition. In a Hamming sense, each hypercube face will be composed of a corner, two of its nearest neighbors (one bit different), and the diagonal. (Note each face represents changes in only two bits.) The test is not likely to be efficient, but it will be local. So how can it be used for general boolean functions? Suppose you have an incomplete boolean function of N variables, and that the incomplete function can be divided into subsets each of which is a complete boolean function of Ki variables, where Ki < N. (It's okay if a vertex is in more than one subset, but each vertex must be in some subset.) If each such subset of vertices represents a subspace of fewer than nine dimensions (nine variables), and the subset is a complete boolean function of the variables, then the 2-summability test is also a test for linear separabilty. If the subset fails the test, it is not linearly separable. If it passes the test, it is linearly separable. This is because 2-asummability is the same as complete monotonicity, which has been shown to be a necessary and sufficient condition for linear separability for boolean functions of fewer than nine variables. (See Muroga, 1971) The 2-summability test works well only for completely specified functions. For incomplete boolean functions, (that is, when the subset of vertices to be tested is incomplete), this method will require filling in values for "don't care" vertices in an attempt to prevent the augmented subset from becoming summable. Again, summability tests are generally not thought of as efficient, whereas a linear programming approach will be. (1) Saburo Muroga, "Threshold Logic and its Applications", Wiley, NY, 1971 This text is full of wonderful theory about summability, monotonicity, etc. (2) John Hopcroft, "Synthesis of Threshold Logic Networks", Ph.D. thesis, Stanford Univ., 1964. Hopcroft discusses the relationship between convex hulls and linear separability in his thesis. He uses convex hulls to construct a network-building algorithm you might find useful. Karen Huyser Electrical Engineering Stanford University From kasif at crabcake.cs.jhu.edu Mon Mar 5 19:18:15 1990 From: kasif at crabcake.cs.jhu.edu (Simon Kasif ) Date: Mon, 5 Mar 90 19:18:15 -0500 Subject: linear separability test. Message-ID: Algorithms for convex hull construction in multiple dimensions are usually exponential in the number of dimensions. Thus, at least asymptotically one may prefer linear programming methods. Many interesting special cases to decide whether convex polytopes intersect and consequently determine linear separability (e.g., in 2 or 3 dimensions) are discussed in standard computational geometry books (e.g., Preparata and Shamos). From singer at Think.COM Tue Mar 6 16:00:13 1990 From: singer at Think.COM (singer@Think.COM) Date: Tue, 6 Mar 90 16:00:13 EST Subject: Neural networks on NCube machines Message-ID: <9003062100.AA11858@selli.think.com> I am looking for neural network implementations on NCube machines. If anyone knows of anyone doing such work, could he/she please contact me? Thanks, Alexander Singer Thinking Machines Corp. 617-876-1111 singer at think.com From gotsman%humus.huji.ac.il at VMA.CC.CMU.EDU Wed Mar 7 15:48:02 1990 From: gotsman%humus.huji.ac.il at VMA.CC.CMU.EDU (Chaim Gotsman) Date: Wed, 7 Mar 90 22:48:02 +0200 Subject: Linear separability Message-ID: <9003072048.AA23510@humus.huji.ac.il> As Andy Ogielski has pointed out, the ONLY polynomial (in the dimension) algorithms for solving the linear separability problem are the recent polynomial linear programming algorithms, specifically the Kachiyan ellipsoid method. The perceptron methods (termination set aside) or delta rule modifications can sometimes run in exponential time, as Minsky and Papert themselves confess. This has been recently "rediscovered" by Maass and Turan in "On the Complexity of Learning From Counterexamples" Proc. of IEEE Conf. on Foundations of Computer Science 1989, p.262-267 A word of caution is in order concerning this algorithm. It requires an "oracle" to supply counterexamples to false hypothesis' at each iteration, i.e. if the current guess of weights is incorrect, a point of the hypercube incorrectly classified has to be supplied. If the number of points (on the cube) supplied as input is polynomial, the oracle can be simulated polynomially by exhaustive search. This is probably the case, otherwise exponential time is required just to READ THE INPUT. An interesting offshoot of this question is whether it is possible to determine linear separability of an ENTIRE (on the whole cube) boolean function, when the input is a description of a CIRCUIT computing the function. The circuit description is obviously polynomial, otherwise the function is definitely NOT linearly separable (all linear threshold functions are in NC1). I'd be interested in knowing if anyone can answer this. Craig Gotsman Hebrew University gotsman at humus.huji.ac.il  From C.Foster at massey.ac.nz Thu Mar 8 10:32:25 1990 From: C.Foster at massey.ac.nz (C.Foster@massey.ac.nz) Date: Thu, 8 Mar 90 10:32:25 NZS Subject: formal levels Message-ID: <9003072232.AA21066@sis-a> With reference to the recent discussion of levels, there is a formal theory of levels. I have developed it as part of a larger (formal) theory of strong equivalence of complex systems. This is a PhD thesis in progress for Edinburgh University's Centre for Cognitive Science. It should be available *soon*. Let me know if you want a copy or more information. C. Foster CFoster at massey.ac.nz c/- School of Information Sciences Massey University Palmerston North New Zealand From kasif at crabcake.cs.jhu.edu Wed Mar 7 19:56:34 1990 From: kasif at crabcake.cs.jhu.edu (Simon Kasif ) Date: Wed, 7 Mar 90 19:56:34 -0500 Subject: Linear separability Message-ID: > > An interesting offshoot of this question is whether it is possible to determine linear separability of an ENTIRE (on the whole cube) boolean function, when the input is a description of a CIRCUIT computing the function. The circuit description is obviously polynomial, otherwise the function is definitely NOT linearly separable (all linear threshold functions are in NC1). I'd be interested in knowing if anyone can answer this. > > If I understood your question correctly, this seems to depend on the representation of the boolean (function) circuit. If the function F is given as a formula in CNF, then the question become co-NP. Simply observe that the formula F*G (F and G), where G represents some easy non-linearly separable function (e.g. XOR) over the same input, is a linear threshold function (all inputs map to 0) iff F is unsatisfiable. From haim at grumpy.sarnoff.com Thu Mar 8 11:12:17 1990 From: haim at grumpy.sarnoff.com (Haim Shvaytser x2085) Date: Thu, 8 Mar 90 11:12:17 EST Subject: Linear separability Message-ID: <9003081612.AA07761@vision.sarnoff.com> Two comments: 1- While it is true that the general problem of determining linear seperability is NP Complete (see previous messages), there are many results about interesting cases that are polynomial. For example, Chvatal and Hammer [1] give an O(mn^2) algorithm for determining whether a set of m linear inequalities in n variables is linearly seperable (i.e., the m inequalities can be replaced by a single inequality). Their nice algorithm is for the case in which the coefficients are from {0,1}. They also show that the same problem for general (integer) coefficients is NP Complete. (Notice that this implies that the question of whether a given neural net with one hidden layer can be simulated by a perceptron is NP Complete.) 2- Even though the worst case behavior of the perceptron algorithm is exponential, it was shown to be better than currently known algorithms of linear programming in a certain "average case" sense. See E. Baum's paper in the last NIPS conference. [1] V. Chvatal and P.L. Hammer, Aggregation of Inequalities in Integer Programming, Annals of Discrete Mathematics 1 (1977) 145-162. From grumbach at meduse.enst.fr Thu Mar 8 13:17:49 1990 From: grumbach at meduse.enst.fr (Alain Grumbach) Date: Thu, 8 Mar 90 19:17:49 +0100 Subject: organization levels Message-ID: <9003081817.AA02414@meduse.enst.fr> A couple of weeks ago, I sent this mail : % Being working on hybrid symbolic - connectionist systems, % I am wondering about the notion of "organization level", % which is underlying hybrid models. % A lot of people use this phrase, from neuroscience, to cognitive % psychology, via computer science, Artificial Intelligence : % (Anderson, Newell, Simon, Hofstadter,Marr, Changeux, etc). % But has anybody heard about a formal description of it ? % (formal but understandable !) Thank you very much ... Hendler, Olson, Manester-Ramer, Schrein, Cutrell, Bovet, Tgelder, Sejnowski, Sanger, Sloman, Rohwer ... for your answers. I shall try here a summary of these answers and put forward a framework of a formal point of view. 1. Answers : 1.1 Points of view : A lot of phrases are used to qualify level hierarchy, denoting quite different points of view : organization levels description levels abstraction levels analysis levels integration levels activity levels explanation levels To name each level, a lot of words are used, which, of course, should be associated with some of hierarchy phrases : computational, algorithmic, implementational sensors, synapses, neurons, areas task, implementation, virtual machine Several hierarchy structures are mentioned : linear tree oriented graph without circuits oriented graph with circuits (from processing point of view only) 1.2. References : Many references are given : Steels, Fodor, Fox, Honavar & Uhr, Jacob, Churchland & Sejnowski, Bennett & Hoffman & Prakash the last one : Bennett & Hoffman & Prakash, being a formal theory dealing with levels. Unfortunately I have not read it yet, as it concerns the ninth chapter of a book; R. Rohwer will write a more direct condensed version of it. 2. Sketch of an organization level description : 2.1 Intuitive issues : 2.1.1 Who ? First it must be emphasized (or remembered) that an organization level hierarchy consists in a point of view of an OBSERVER about an OBJECT, a PHENOMENON, etc. It is not an intrinsic caracteristic of the object, the phenomenon, but a property of the situation including the observer (his culture) and the observed entity (from an epistemologic point of view). 2.1.2 What ? I say above that the organization level concerns an object, a phenomenon. Let us give some examples: a book, a house an image, a sentence, this mail a processing unit : computer, engine, tv, living being, etc a set of objects, processing units (society, ant colony) etc Some of them live in interaction with an environment (processing units), others are static, closed entities (book, house). 2.1.3 "Level" , "Organization", "Hierarchy" : From giles at fuzzy.nec.com Fri Mar 9 15:11:50 1990 From: giles at fuzzy.nec.com (Giles L.) Date: Fri, 9 Mar 90 15:11:50 EST Subject: No subject Message-ID: <9003092011.AA00586@fuzzy.nec.com> paper available: This 8-page paper will appear in Advances in Neural Information Processing Systems 2, D.S. Touretzky (ed), Morgan Kaufmann, San Mateo, Ca., 1990. HIGHER ORDER RECURRENT NETWORKS & GRAMMATICAL INFERENCE C. L. Giles*, G. Z. Sun, H. H. Chen, Y. C. Lee, D. Chen Department of Physics and Astronomy and Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20742. *NEC Research Institute, 4 Independence Way, Princeton, N.J. 08540 ABSTRACT We design a higher-order single layer, recursive neural network which easily learns to simulate a deterministic finite state machine and infer simple regular grammars from small training sets. An enhanced version of this neural network state machine is then constructed and connected through a common error term to an external analog stack memory. The resulting hybrid machine can be interpreted as a type of neural net pushdown automata. The neural net finite state machine part is given the primitives, push and pop, and is able to read the top of the stack. Using a gradient descent learning rule derived from a common error function, the hybrid network learns to effectively use the stack actions to manipulate the stack memory and to learn simple context-free grammars. If the neural net pushdown automata are reduced through a heuristic clustering of neuron states and actions, the neural network reduces to correct pushdown automata which recognize the learned context-free grammars. --------------- For a hard copy of the above, please send a request to: gloria at research.nec.com or Gloria Behrens NEC Research Institute 4 Independence Way Princeton, N.J. 08540 From Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU Fri Mar 9 16:15:43 1990 From: Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF1.SLISP.CS.CMU.EDU) Date: Fri, 09 Mar 90 16:15:43 EST Subject: Tech Report Announcement: CMU-CS-90-100 Message-ID: *** Please do not forward this to other mailing lists or newsgroups. *** Tech Report CMU-CS-90-100 is now available, after some unfortunate delays in preparation. This is a somewhat more detailed version of the paper that will be appearing soon in "Advances in Neural Information Processing Systems 2" (also known as the NIPS-89 Proceedings). To request a copy of the TR, send a note containing the TR number and your physical mail address to "catherine.copetas at cs.cmu.edu". Please *try* not to respond to the whole mailing list. People who requested a preprint of our paper at the NIPS conference should be getting this TR soon, so please don't send a redundant request right away. If you don't get something in a week or two, then try again. I'll be making an announcement to this list sometime soon about how to get Common Lisp code implementing the algorithm described in this TR. No C version is available at present. =========================================================================== The Cascade-Correlation Learning Architecture Scott E. Fahlman and Christian Lebiere School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 Technical Report CMU-CS-90-100 ABSTRACT Cascade-Correlation is a new architecture and supervised learning algorithm for artificial neural networks. Instead of just adjusting the weights in a network of fixed topology, Cascade-Correlation begins with a minimal network, then automatically trains and adds new hidden units one by one, creating a multi-layer structure. Once a new hidden unit has been added to the network, its input-side weights are frozen. This unit then becomes a permanent feature-detector in the network, available for producing outputs or for creating other, more complex feature detectors. The Cascade-Correlation architecture has several advantages over existing algorithms: it learns very quickly, the network determines its own size and topology, it retains the structures it has built even if the training set changes, and it requires no back-propagation of error signals through the From gaudiano at bucasb.bu.edu Fri Mar 9 16:31:11 1990 From: gaudiano at bucasb.bu.edu (gaudiano@bucasb.bu.edu) Date: Fri, 9 Mar 90 16:31:11 EST Subject: NNSS update and call for help Message-ID: <9003092131.AA14624@retina.bu.edu> NEURAL NETWORKS STUDENT SOCIETY UPDATE and CALL FOR HELP NOTE: in the near future there may be a special newsgroup for student notes like this one so we don't have to clutter up the mailing lists. For now apologies to those subscribers that do not wish to get this. Thanks to all those that responded to the Society Announcement. The response has been overwhelming (almost 300 so far). We have not been acknowledging individual requests because of the volume, but we will mail out our newsletter at the end of the month. At that point we will send notes to make sure everyone's address (email or snailmail) is correct. CALL FOR SUBMISSIONS: If you are in a recognized academic program for Neural Networks and want other students to know about it, please send us a short description IN YOUR OWN WORDS of the program, including an address for people who want more details. Include things like faculty members, courses, and your opinions (if you want). Send the submission by email to "nnss-request at thalamus.bu.edu" by MARCH 21 for inclusion in our newsletter. We will let you know if there are problems or comments. CALL FOR HELP: In order to handle international memberships, we decided it is best to designate "ambassadors" for each country or geographical area. This is primarily to save people the trouble of having to get $5 exchanged from their own currencies, but ambassadors should also be willing to devote some time for possible future Society-related tasks. If you live outside of the US and are willing to devote some of your time to this endeavour, drop us a note (email nnss-request at thalamus.bu.edu). Paolo Gaudiano Karen Haines gaudiano at thalamus.bu.edu khaines at galileo.ece.cmu.edu From walker at sumex-aim.stanford.edu Fri Mar 9 21:34:38 1990 From: walker at sumex-aim.stanford.edu (Michael G. Walker) Date: Fri, 9 Mar 1990 18:34:38 PST Subject: Expert systems and systematic biology workshop Message-ID: Workshop Announcement: Artificial Intelligence and Modern Computer Methods in Systematic Biology (ARTISYST Workshop) The Systematic Biology Program of the National Science Foundation, is sponsoring a Workshop on Artificial Intelligence, Expert Systems, and Modern Computer Methods in Systematic Biology, to be held September 9 to 14, 1990, at the University of California, Davis. There will be about 45 participants representing an even mixture of biologists and computer scientists. Attendance at the workshop is by invitation only. All expenses for participants (travel, hotel, food) will be paid. These are the subject areas for the workshop: 1. Scientific workstations for systematics; 2. Expert systems, expert workstations and other tools for identification; 3. Phylogenetic inference and mapping characters onto tree topologies; 4. Literature data extraction and geographical data; 5. Machine vision and feature extraction applied to systematics. The workshop will examine state-of-the-art computing methods and particularly Artificial Intelligence methods and the possibilities they offer for applications in systematics. Methods for knowledge representation as they apply to systematics will be a central focus of the workshop. This meeting will provide systematists the opportunity to make productive contacts with computer scientists interested in these applications. It will consist of tutorials, lectures on problems and approaches in each area, working groups and discussion periods, and demonstrations of relevant software. Participants will present their previous or proposed research in a lecture, in a poster session, or in a software demonstration session. If you are interested in participating, complete the application form below. Preference will be given to applicants who are most likely to continue active research and teaching in this area. The Workshop organizers welcome applications from all qualified biologists and computer scientists, and strongly encourage women, minorities, and persons with disabilities to apply. APPLICATIONS RECEIVED AFTER APRIL 15, 1990 WILL NOT BE ACCEPTED ----------------- Application form Name: Address: E-mail address: In your application, please include 1) a short resume, 2) a description of your previous work related to the workshop topic, 3) a description of your planned research and how it relates to the workshop, and 4) whether you, as biologists (or computer scientists) have taken or would like to take steps to establish permanent collaboration with computer scientists (or biologists). A total of two pages or less is preferred. This material will be the primary basis for selecting workshop participants. If you have software that you would like to demonstrate at the workshop, please give a brief description, and indicate the hardware that you need to run the program. Several PC's and workstations will be available at the workshop. Mail your completed application to: Renaud Fortuner, ARTISYST Workshop Chairman, California Department of Food and Agriculture Analysis & Identification, room 340 P.O. Box 942871 Sacramento, CA 94271-0001 USA (916) 445-4521 E-mail: rfortuner at ucdavis.edu For further information, contact Renaud Fortuner, Michael Walker, Program Chairman, (Walker at sumex-aim.stanford.edu), or a member of the steering committee: Jim Diederich, U.C. Davis (dieder at ernie.berkeley.edu) Jack Milton, U.C. Davis (milton at eclipse.stanford.edu) Peter Cheeseman, NASA AMES (cheeseman at pluto.arc.nasa.gov) Eric Horvitz, Stanford University (horvitz at sumex-aim.stanford.edu) Julian Humphries, Cornell University (lqyy at crnlvax5.bitnet) George Lauder, U.C Irvine (glauder at UCIvmsa.bitnet) James Rohlf, SUNY (rohlf at sbbiovm.bitnet) James Woolley, Texas A&M University (woolley at tamento.bitnet) From well!mitsu at apple.com Fri Mar 9 18:13:05 1990 From: well!mitsu at apple.com (Mitsuharu Hadeishi) Date: Fri, 9 Mar 90 15:13:05 pst Subject: organization levels Message-ID: <9003092313.AA20573@well.sf.ca.us> I would guess that such a technique (to describe logical levels of abstraction formally, i.e., syntactically) would fail (except as metaphor) because it is often the case that the abstracted level of a formal system is not itself formally describable (at least not in a manner which is easily defined). That is, one would not be able in all cases to define the set of relations on the "meta-" field in any kind of well-defined manner (though such relations might well exist in the original system). Mathematics versus metamathematics, physics versus "metaphysics". I.e., the "meta-" fields are not describable from within the language of the fields themselves, and this is often, I think, the case because the meta-fields are not formally describable in a well-defined manner (at least not a manner which is easily fathomable by human beings). I am, of course, simply speaking through my hat: this seems to me to be the case, but I have not come up with an ironclad argument as yet. I just put the idea out for consideration by those who might be able to clarify the issue. Mitsu From nina at alpha.ece.jhu.edu Mon Mar 12 18:56:54 1990 From: nina at alpha.ece.jhu.edu (Nina A. Kowalski) Date: Mon, 12 Mar 90 18:56:54 EST Subject: IJCNN-90 San Diego Message-ID: *************************************************************************** IJCNN 1990 - REQUEST FOR VOLUNTEERS *************************************************************************** This is the first call for volunteers to help at the International Joint Committee on Neural Networks (IJCNN) conference, to be held at the San Diego Marriot Hotel in San Diego, CA, June 17-21,1990. Full admittance to the conference and a copy of the proceedings is offered in exchange for your assistance throughout the conference. I would like to point out that STUDENT REGISTRATION DOES NOT INCLUDE PROCEEDINGS OR ADMITTANCE INTO WEDNESDAY NIGHT'S PARTY. In general, each volunteer is expected to work one shift each day of the conference. Hours are approximately: AM shift - 7:00 am - Noon PM shift - Noon - 5:00 pm Available shifts are: Technical Session Ushers Poster Sessions Hospitality/Press Room Registration Material Assistance You will be basically working the same shift each day of the conference. In addition to working one of the above shifts throughout the conference, assistance may be required for the social events. Those interested in signing up, please send me the following information: Name Address phone number email Upon signing up, you be sent a form with a more detailed description of the positions, and a request for shift preference and tutorials. Sign ups will be based on the date of commitment. Tutorials: --------- In addition to volunteering for the conference, we will need help the day of the tutorials. The day of the tutorials is Sunday, June 17. Although conference volunteers are not required to work the tutorials, tutorial volunteers are required to work the conference. To sign up please contact: Nina Kowalski - Volunteer Chariman 209 W. 29th St. FLR 2 Baltimore, MD 21211 message: (301) 889-0587 email: nina at alpha.ece.jhu.edu If you have further questions, please feel free to contact me. Thank you, Nina Kowalski IJCNN Volunteer Chairman ----------------------------------------------------------------------------- end of post ---------------------------------------------------------------------------- From gotsman%humus.huji.ac.il at VMA.CC.CMU.EDU Tue Mar 13 02:28:11 1990 From: gotsman%humus.huji.ac.il at VMA.CC.CMU.EDU (Chaim Gotsman) Date: Tue, 13 Mar 90 09:28:11 +0200 Subject: linear separability Message-ID: <9003130728.AA23292@humus.huji.ac.il> The Baum paper from the last NIPS avoids the problem instances where the classic perceptron algorithm runs in exponential time (where some of the points of the hypercube are at an exponentially small distance from the separating plane), by calling these points malicious and probabilistically eliminating them. I don't see why this makes the algorithm suddenly tractable "on the average". From TESAURO at IBM.COM Tue Mar 13 11:05:27 1990 From: TESAURO at IBM.COM (Gerald Tesauro) Date: Tue, 13 Mar 90 11:05:27 EST Subject: "malicious" training patterns Message-ID: The notion that points close to the decision surface are "malicious" comes as a surprise to me. From the point of view of extracting good generalizations from a limited number of training patterns, such "borderline" training patterns may in fact be the best possible patterns to use. A benevolent teacher might very well explicitly design a bunch of border patterns to clearly mark the boundaries between different conceptual classes. --Gerry Tesauro From tgd at turing.CS.ORST.EDU Tue Mar 13 12:34:57 1990 From: tgd at turing.CS.ORST.EDU (Tom Dietterich) Date: Tue, 13 Mar 90 09:34:57 PST Subject: "malicious" training patterns In-Reply-To: Gerald Tesauro's message of Tue, 13 Mar 90 11:05:27 EST <9003131719.AA05526@CS.ORST.EDU> Message-ID: <9003131734.AA13875@turing.CS.ORST.EDU> Date: Tue, 13 Mar 90 11:05:27 EST From: Gerald Tesauro The notion that points close to the decision surface are "malicious" comes as a surprise to me. From the point of view of extracting good generalizations from a limited number of training patterns, such "borderline" training patterns may in fact be the best possible patterns to use. A benevolent teacher might very well explicitly design a bunch of border patterns to clearly mark the boundaries between different conceptual classes. --Gerry Tesauro This is true for cases where the decision surface is parallel to an axis--in that case, the teacher can give two examples differing in only one feature-value. But in general, the more closely positive and negative examples crowd together, the harder it is to resolve and separate them, especially in noisy circumstances. An average case analysis must always define "average", which is what Baum has nicely done. Do readers have examples of domains that are linearly separable but hard to separate? In my experience, the problem is that simple linear models, while they may give reasonable fits to training data, tend to underfit and hence give poor predictive performance. For example, the following points are (just barely) linearly separable, but a multilayer perceptron or a quadratic model would give better predictive performance: + + + - - - + + + - - - + + + - - - + + + - - - + + + - - - + + + - - - + + + - - - + + + - - - + + + - - - As the number of training examples increases, there is statistical support for hypotheses more complex than simple linear separators. Tom Dietterich From John.Hampshire at SPEECH2.CS.CMU.EDU Tue Mar 13 16:59:24 1990 From: John.Hampshire at SPEECH2.CS.CMU.EDU (John.Hampshire@SPEECH2.CS.CMU.EDU) Date: Tue, 13 Mar 90 16:59:24 EST Subject: 'malicious' training tokens Message-ID: Fundamentally I agree with Gerry Tesauro's argument that training tokens in the vicinity of a random vector's true class boundaries are the best ones to use if you want good generalization --- these tokens will delineate the optimal class boundaries. I think there's a caveat though: Say that you could consistently obtain training tokens in the vicinity of the optimal class boundaries. If you could get an arbitrarily large number of independent training tokens, then you could build a perceptron (or for non-linear class boundaries, an MLP) with appropriate connectivity for good generalization. If, however, you were severely limited in the number of independent training samples you could obtain (again, they're all near the optimal class boundaries), then you'd be faced with insufficient data to avoid bad inferences about your limited data --- and you'd get crummy generalization. This would happen because your classifier needs to be sufficiently parameterized to learn the training tokens; however, this degree of parameterization leads to rotten generalization due to insufficient data. In cases of limited training data, then, it may be better to have training tokens near the modes of the class conditional densities (and away from the optimal boundaries) in order for you to at least make a good inference about the prototypical nature of the RV being classified. These tokens would also require a lower degree of parameterization in your classifier, which would give better performance on disjoint test data. I haven't read Baum's paper and I wouldn't presume to put words in anyone's mouth, but maybe this is what he was getting at by characterizing near-boundary tokens as 'malicious'. Incidentally, Duda & Hart (sec. 5.5) give a convergence proof for the perceptron criterion function that illustrates why it takes so long to learn 'malicious' training tokens near the optimal class boundaries. John From jagota at cs.Buffalo.EDU Tue Mar 13 16:09:48 1990 From: jagota at cs.Buffalo.EDU (Arun Jagota) Date: Tue, 13 Mar 90 16:09:48 EST Subject: TR available Message-ID: <9003132109.AA27057@sybil.cs.Buffalo.EDU> The following technical report is available: A Hopfield-style network for content-addressable memories Arun Jagota Department of Computer Science State University Of New York At Buffalo 90-02 ABSTRACT With the binary Hopfield network as a basis, new learning and energy descent rules are developed. It is shown, using graph theoretic techniques, that the stable states of the network are the maximal cliques in the underlying graph and that the network can store an arbitrary collection of memories without interference (memory loss, unstable fixed points). In that sense (and that sense alone), the network has exponential capacity (upto 2^(n/2) memories can be stored in an n-unit network). Spurious memories can (and are likely) to develop. No analytical results for these are derived, but important links are established between the storage and recall properties of the network and the properties of the memories that are stored. In particular it is shown, partly by analysing the graph underlying the network, that the network retrieval time and other desirable properties depend on the 'sparse-ness' of the memories and whether they have a 'combinatorial' structure (as defined in the report). It is shown that the network converges in <= n iterations and for sparse memories (and initial states) with sparseness k, 0 < k < n, it converges in <= k iterations. ------------------------------------------------------------------------ The report is available in PostScript form by anonymous ftp as follows: unix> ftp cheops.cis.ohio-state.edu (or, ftp 128.146.8.62) Name: anonymous Password: neuron ftp> cd pub/neuroprose/Inbox ftp> binary ftp> get jagota.hsn.ps.Z ftp> quit unix> uncompress jagota.hsn.ps.Z unix> lpr jagota.hsn.ps (use flag your printer needs for Postscript) [sometime soon, the report may be moved to pub/neuroprose] ------------------------------------------------------------------------ It is recommended that hard-copy requests be made only if it is not possible (or too inconvenient) to access the report via ftp. I have developed a software simulator that I am willing to share with individuals who might be interested (now or later). It has been carefully 'tuned' for this particular model, implementing the network algorithms in a most efficient manner. It allows configurability, (any size 1-layer net, other parameters etc) and provides a convenient 'symbolic' interface. For hard-copy (and/or simulator) requests send e-mail (or write to) the following address. Please do not reply with 'r' or 'R' to this message. Arun Jagota e-mail: jagota at cs.buffalo.edu Dept Of Computer Science 226 Bell Hall, State University Of New York At Buffalo, NY - 14260 From marvit at hplpm.hpl.hp.com Wed Mar 14 00:21:22 1990 From: marvit at hplpm.hpl.hp.com (Peter Marvit) Date: Tue, 13 Mar 90 21:21:22 PST Subject: Neural models of attention? Message-ID: <9003140521.AA13072@hplpm.HPL.HP.COM> Early this century, William James said "Everyone knows what attention is." Well, after reading lots of cognitive psychology papers, I'm not so sure. After reading more bio-psych and neuro- papers, I'm even less sure I can give a cogent definition, though I still think I know what it is. Some colleagues and I have been discussing various approaches to studying attention. The question arose: What neural models (either artificial [i.e., connectionist] or naturalistic) of attention are there? Who has tried to explain how attention works, produced a model which attempts to be psychologically valid, or (in the words on a friend) explain where attention "comes from"? For that matter, can connectionist models be used for such a slippery subject? Attending your responses, Peter Marvit : Peter Marvit Hewlett-Packard Labs in Palo Alto, CA (415) 857-6646 : : Internet: uucp: {any backbone}!hplabs!marvit : From jc5e+ at ANDREW.CMU.EDU Wed Mar 14 02:47:52 1990 From: jc5e+ at ANDREW.CMU.EDU (Jonathan D. Cohen) Date: Wed, 14 Mar 90 02:47:52 -0500 (EST) Subject: Neural models of attention? Message-ID: Several investigators have begun to address attentional phenomena using connectionist models. They include Schneider, Mozer and Phaff (see references below). My colleagues (Kevin Dunbar and Jay McClelland) and I have also done some work in this area. Our approach has been to view attention as the modulatory influence that information in one part of the system has on processing in other parts. By modulation, we mean changes in the responsivity of processing units. We have implemented our ideas in a model of the Stroop task, a standard psychological paradigm for studying selective aspects of attention. The reference and abstract for this paper are also included below. Jonathan Cohen ************************* Mozer, M. (1988). A connectionist model of selective attention in visual perception. In the proceedings of the tenth annual conference of the Cognitive Science Society.Hillsdale, NJ: Erlbaum, pp. 195-201. Phaff, R. H. (1986). A connectionist model for attention: Restricting parallel processing through modularity. Unpublished doctoral dissertation, University of Experimental Psychology, University of Leiden, Netherlands. Schneider, W. (1985). Toward a model of attention and the development of automatic processing. In M.I. Posner & O.S.M. Marin (Eds.), Attention and and Performance XI (pp.475-492). Hillsdale, NJ: Lawrence Erlbaum. ****************************** Cohen JD, Dunbar K & McClelland JL (in press). On the control of automatic processes: A parallel distributed processing model of the Stroop effect. Psychological Review, in press. (I can handle a *limited* number of requests for preprints) Abstract A growing body of evidence suggests that traditional views of automaticity are in need of revision. For example, automaticity has often been treated as an all-or-none phenomenon, and traditional theories have held that automatic processes are independent of attention. Yet recent empirical data suggest that automatic processes are continuous, and furthermore are subject to attentional control. In this paper we present a model of attention which addresses these issues. Using a parallel distributed processing framework we propose that the attributes of automaticity depend upon the strength of a processing pathway and that strength increases with training. Using the Stroop effect as an example, we show how automatic processes are continuous and emerge gradually with practice. Specifically, we present a computational model of the Stroop task which simulates the time course of processing as well as the effects of learning. This was accomplished by combining the cascade mechanism described by McClelland (1979) with the back propagation learning algorithm (Rumelhart, Hinton, & Williams, 1986). The model is able to simulate performance in the standard Stroop task, as well as aspects of performance in variants of this task which manipulate SOA, response set, and degree of practice. In the discussion we contrast our model with other models, and indicate how it relates to many of the central issues in the literature on attention, automaticity, and interference. From koch%HAMLET.BITNET at VMA.CC.CMU.EDU Wed Mar 14 01:54:23 1990 From: koch%HAMLET.BITNET at VMA.CC.CMU.EDU (Christof Koch) Date: Tue, 13 Mar 90 22:54:23 PST Subject: Neural network model of attention Message-ID: <900313225351.2020114e@Hamlet.Caltech.Edu> I published a network model of attention, together with SHimon Ullman, in 1984/85. It involves a central saliency map, possibly located in area IT or in parietal cortex. Attention selects the most conspicuous or salient location from this map (without knowing "what" is at that location) and then goes back to the individual feature maps to find out about the properties of the attended location (e.g. red, horizontal etc..). The paper is "Shifts in selective visual attention: towards the underlying neural circuitry", C. Koch and S. Ullman, Human Neurbiology Vol. 4: pp. 219-227, 1985. It also appeared as a MIT AI Memo in 1984. Christof P.S. In this version, attention has to solve the binding problem; that is, combine the various attributes of the object being perceived into a coherent whole. From tsejnowski at UCSD.EDU Wed Mar 14 04:15:23 1990 From: tsejnowski at UCSD.EDU (Terry Sejnowski) Date: Wed, 14 Mar 90 01:15:23 PST Subject: March 15 Deadline for CSH Message-ID: <9003140915.AA22429@sdbio2.UCSD.EDU> March 15 *** Deadline for Student Applications to the Cold Spring Harbor Laboratory 1990 Summer Course July 14 - July 28, 1990 Computational Neuroscience: Learning and Memory Organized by: Michael Jordan, MIT and Terry Sejnowski, Salk and UCSD This is an intensive two week course that includes hands-on computer experience as well as lectures. Topics include invertebrate learning mechanisms, synaptic plasticity in the hippocampus, models of classical conditioning, cortical maps, motor learning, temporal-difference learning, radial basis functions, scaling, generalization, and complexity. Instructors include: Jack Byrne, Tom Brown, Steve Lisberger, Nelson Donegan, Gerry Tesauro, Rich Sutton, Ralph Linsker, Richard Durbin, John Moody, Dave Rumelhart, Yan Le Cun, Chris Atkeson, Tommy Poggio, Mike Kearns. Tuition: $1390. Send applications and requests for scholarships to: Registrar Cold Spring Harbor Laboratory Cold Spring Harbor, NY 11724 For application forms and info call: (516) 367-8343 ----- From ZVONA%AI.AI.MIT.EDU at Mintaka.lcs.mit.edu Wed Mar 14 12:14:00 1990 From: ZVONA%AI.AI.MIT.EDU at Mintaka.lcs.mit.edu (David Chapman) Date: Wed, 14 Mar 90 12:14:00 EST Subject: Neural models of attention? In-Reply-To: Msg of Tue 13 Mar 90 21:21:22 PST from Peter Marvit Message-ID: <710698.900314.ZVONA@AI.AI.MIT.EDU> There are actually many models in the literature; most of the people studying the problem seem not to be aware of each other's work. In addition to those already mentioned, [Feldman+Ballard] have proposed a model inspired by [Treisman+Gelade]. (References are at the end of this message.) [Strong+Whitehead] have implemented a similar model. [Fukushima]'s model is rather different. [Koch+Ullman]'s and [Mozer]'s models seem closest to the psychophysical and neurophysiological data to me. I [Chapman] have implemented [Koch+Ullman]'s proposal successfully and used it to model in detail the psychophysically-based theories of visual search due to [Treisman+Geldade, Treisman+Gormican]. My thesis [Chapman] describes these and other models of attention and tries to sort out their relative strengths. It's probably time for someone to write a review article. Cites: David Chapman, {\em Vision, Instruction, and Action.} PhD Thesis, MIT Artificial Intelligence Laboratory, 1990. Jerome A.~Feldman and Dana Ballard, ``Connectionist Models and their Properties.'' {\em Cognitive Science} {\bf 6} (1982) pp.~205--254. Kunihiko Fukushima, ``A Neural Network Model for Selective Attention in Visual Pattern Recognition.'' {\em Biological Cybernetics} {\bf 55} (1986) pp.~5--15. Christof Koch and Shimon Ullman, ``Selecting One Among the Many: A Simple Network Implementing Shifts in Selective Visual Attention.'' {\em Human Neurobiology} {\bf 4} (1985) pp.~219--227. Also published as MIT AI Memo 770/C.B.I.P.~Paper 003, January, 1984. Michael C.~Mozer, ``A connectionist model of selective attention in visual perception.'' {\em Program of the Tenth Annual Conference of the Cognitive Science Society}, Montreal, 1988, pp.~195--201. Gary W.~Strong and Bruce A.~Whitehead, ``A solution to the tag-assignment problem for neural networks.'' {\em Behavioral and Brain Sciences} (1989) {\bf 12}, pp.~381--433. Anne M.~Treisman and Garry Gelade, ``A Feature-Integration Theory of Attention.'' {\em Cognitive Psychology} {\bf 12} (1980), pp.~97--136. Anne Treisman and Stephen Gormican, ``Feature Analysis in Early Vision: Evidence From Search Asymmetries.'' {\em Psychological Review} Vol.~95 (1988), No.~1, pp.~15--48. Some other relevant references: C.~H.~Anderson and D.~C.~Van Essen, ``Shifter circuits: A computational strategy for dynamic aspects of visual processing.'' {\em Proceedings of the National Academy of Sciences, USA}, Vol.~84, pp.~6297--6301, September 1987. Francis Crick, ``Function of the thalamic reticular complex: The searchlight hypothesis.'' {\em Proceedings of the National Academy of Science}, Vol.~81, pp.~4586--4590, July 1984. Jefferey Moran and Robert Desimone, ``Selective attention gates visual processing in the extrastriate cortex.'' {\em Science} {\bf 229} (1985), pp.~782--784. V.~B.~Mountcastle, B.~C.~Motter, M.~A.~Steinmetz, and A.~K.~Sestokas, ``Common and Differential Effects of Attentive Fixation on the Excitability of Parietal and Prestriate (V4) Cortical Visual Neurons in the Macaque Monkey.'' {\em The Journal of Neuroscience}, July 1987, 7(7), pp.~2239--2255. John K.~Tsotsos, ``Analyzing vision at the complexity level.'' To appear, {\em Behavioral and Brain Sciences}. From B344DSL at UTARLG.ARL.UTEXAS.EDU Wed Mar 14 14:23:00 1990 From: B344DSL at UTARLG.ARL.UTEXAS.EDU (B344DSL@UTARLG.ARL.UTEXAS.EDU) Date: Wed, 14 Mar 90 13:23 CST Subject: NN models of attention Message-ID: I don't recall what the original inquiry was regarding neural network models of attention, but there have been several articles of Grossberg's and of mine that have dealt with some aspect of this problem. Grossberg wrote a review article (Int. Rev. of Neurobiology 1975) on attention, reinforcement, and discrimination learning, and he and I wrote a mathematical paper in J. of Theoretical Biology the same year on attentional biases in a competitive network. Psychologists distinguish selective attention between separate stimuli from selective attention between aspects of a single stimulus. Those 1975 papers mainly deal with the former, but my article with Paul Prueitt in Neural Networks 2:103-116 deals with the latter. We model switches between different reinforcement criteria (e. g. based on color vs. based on shape) on the Wisconsin card sorting test, and how such a switch in attentional modula- tion of categorization is disrupted by frontal lobe damage. Dan Levine (b344dsl at utarlg.bitnet) From FRANKLINS%MEMSTVX1.BITNET at VMA.CC.CMU.EDU Wed Mar 14 14:46:00 1990 From: FRANKLINS%MEMSTVX1.BITNET at VMA.CC.CMU.EDU (FRANKLINS%MEMSTVX1.BITNET@VMA.CC.CMU.EDU) Date: Wed, 14 Mar 90 14:46 CDT Subject: Training with incomplete target data Message-ID: I'm helping to design a neural network based diagnostic system in which findings are represented locally as inputs and diagnoses are represented locally as output. The training patterns are such the target value of a single node is known for each pattern. Target values of the other output nodes are not known. I'd appreciate hearing from anyone who has had some experience with training networks under these constraints. I'd be particularly grateful for references to pertinent articles. -- Stan Franklin From kruschke at ucs.indiana.edu Wed Mar 14 16:40:00 1990 From: kruschke at ucs.indiana.edu (kruschke@ucs.indiana.edu) Date: 14 Mar 90 16:40:00 EST Subject: models of attention Message-ID: All the responses (that I've seen) to Peter Marvit's query about attention have concerned *visual/spatial* attention, and mostly the feature-integration problem at that. But ``attention'' is used in other ways in the psych literature. In particular, it is often used to describe which information in a stimulus is deemed to be relevant by the subject when making ``high-level'' decisions, such as which category the stimulus belongs to. Consider, for example, a medical diagnosis situation, in which several different symptoms are present, but only some are ``attended to'' by the physician, because the physician has learned that only some types of symptoms are relevant. This type of attention has been described in the connectionist literature as well. Two references immediately come to mind (no doubt there are others; readers, please inform us): Mozer and Smolensky, ``Skeletonization...'' in NIPS'88; and Gluck & Chow, ``Dynamic stimulus specific learning rates...'' in NIPS'89. I am presently working on a model, called ALCOVE, that learns to distribute attention across input dimensions during training. It captures many effects observed in human category learning, such as the relative difficulty of different types of category structures, some types of base-rate neglect, learning speeds of rules and exceptions (including U-shaped learning of high-frequency exceptions), etc. It is written up in my PhD thesis, to be announced in a few weeks. ---------------------------------------------------------------- John K. Kruschke kruschke at ucs.indiana.edu Dept. of Psychology kruschke at iubacs.bitnet Indiana University (812) 855-3192 Bloomington, IN 47405-4201 USA ---------------------------------------------------------------- From gaudiano at bucasb.bu.edu Wed Mar 14 16:50:53 1990 From: gaudiano at bucasb.bu.edu (gaudiano@bucasb.bu.edu) Date: Wed, 14 Mar 90 16:50:53 EST Subject: Neural models of attention Message-ID: <9003142150.AA02249@retina.bu.edu> >>>>> In reply to: Peter Marvit In addition to all the work on attention mentioned by the previous replies, there is a large body of work by Grossberg on psychological and physiological aspects of attention. Most of the relevant articles are collected in two volumes: [1] S. Grossberg (1986) {\bf The Adaptive Brain I: Cognition, Learning, Reinforcement, and Rhythm.} Amsterdam: Elsevier/North-Holland. Chapter 1 is one that I found particularly clear. It is a reprint of the article: "A Psychophysiological theory of reinforcement, drive, motivation, and attention." {\em Journal of Theoretical Neurobiology.} V. I, 286-369. 1982. The only problem is that attention surfaces here as an integral part of a model that is based primarily on conditioning. Basically, the idea is that things like "motivation", "attention", "drives", etc. have a clear interpretation within the large framework of this article. This means that if you want to know about attention (or motivation, etc.) you kind of need to look at all the rest of the stuff. You will not find a single, self-sustained section that tells you "here is a neural net for attention". Actually, you will find that (e.g., sec. 37), but it is based on all the rest of the material. If you are willing to get into this, it's an *excellent* article. The other good resource would be: [2] S. Grossberg {\bf Studies of Mind and Brain} Boston: Reidel. 1982. For instance, chapter 6 is a reprint of: "A neural model of Attention, Reinforcement and Discrimination Learning." {\em International Review of Neurobiology,} (Carl Pfeiffer, Ed.), V.18, 263-327. 1975. This is an older, and perhaps even broader-scoped article than the other one. Finally, the latest compilation (hey, why write books when you have hundreds of articles that need to be collected in individual, coherent volumes?) is [3] {\bf Neural Networks and Natural Intelligence.} Cambridge, MA, USA: MIT Press. 1988. This includes at least three articles on computer simulations of some of the models developed in the older articles. Have fun! From John.Hampshire at SPEECH2.CS.CMU.EDU Wed Mar 14 21:21:31 1990 From: John.Hampshire at SPEECH2.CS.CMU.EDU (John.Hampshire@SPEECH2.CS.CMU.EDU) Date: Wed, 14 Mar 90 21:21:31 EST Subject: selective attention Message-ID: Depending on your perspective, the following may or may not seem relevant to the topic of attention in connectionist structures. You decide: A hierarchical MLP classifier that recognizes the speech of many speakers by learning how to integrate speaker-dependent modules. This integration involves a sort of supervisory net that learns how to tell when a particular speaker-dependent module is relevant to the speech recognition process. This dynamic focussing of attention to particular modules or groups of modules (depending on the specific input speech signal) is learned indirectly via what we have previously described on this net as the "Meta-Pi" connection. The training error signal is derived from the global objective of "correct phoneme classification", so there is no explicit attention training of the supervisor net. Nevertheless, it learns to direct its attention to relevant modules pretty well (98.4% recognition rate). Attention evolves as a by-product of the global objective. The operation of the whole contraption is explained in terms of Bayesian probability theory. The original work describing this is "The Meta-Pi Network: Building Distributed Knowledge Representations for Robust Pattern Recognition", CMU tech report CMU-CS-89-166, August, 1989. THIS TR IS BEING SUPERSEDED AS WE SPEAK by CMU-CS-89-166-R, so the new version is a few weeks away. ^ | A part of the TR (uneffected by the revision) will appear in the forthcoming NIPS proceedings. Cheers, John Hampshire & Alex Waibel From tsejnowski at UCSD.EDU Thu Mar 15 01:44:44 1990 From: tsejnowski at UCSD.EDU (Terry Sejnowski) Date: Wed, 14 Mar 90 22:44:44 PST Subject: CSH FAX Message-ID: <9003150644.AA16070@sdbio2.UCSD.EDU> Applications to the Cold Spring Harbor Summer Course on Computational Neuroscience: Learning and Memory can be sent by FAX: (516) 367 8845. Deadline is March 15. Terry ----- From pkube at UCSD.EDU Thu Mar 15 01:35:43 1990 From: pkube at UCSD.EDU (Paul Kube) Date: Wed, 14 Mar 90 22:35:43 PST Subject: models of attention Message-ID: <9003150635.AA19417@kokoro.UCSD.EDU> On the topics of recent discussions about connectionist models of attention, and about the relation between connectionist modelling and neurological facts: In a recent paper (Nature, 30 November 1989, pp. 543-545), Luck, Hillyard, Mangun and Gazzaniga report that split-brain patients are twice as good as normals on Triesman-type conjunctive feature visual search tasks when the stimulus array is distributed across both hemifields, but no better than normals when the array is restricted to one hemifield. This suggests that commissurotomy permits a "splitting of attention" that is impossible with connected hemispheres, and that remains impossible within each hemisphere. I'd be interested to know if any of the models of attention under discussion predict this. --Paul Kube kube at cs.ucsd.edu From janet at psych.psy.uq.OZ.AU Wed Mar 14 00:56:21 1990 From: janet at psych.psy.uq.OZ.AU (Janet Wiles) Date: Wed, 14 Mar 90 16:56:21 +1100 Subject: Australian neural networks conference, Feb 1991 Message-ID: <9003140556.AA08485@psych.psy.uq.oz.au> PRELIMINARY ANNOUNCEMENT SECOND AUSTRALIAN CONFERENCE ON NEURAL NETWORKS (ACNN'91) 4th - 6th February 1991 THE UNIVERSITY OF SYDNEY SYDNEY, AUSTRALIA The second Australian conference on neural networks will be held in Sydney on Feb 4, 5 and 6, 1991, at the Stephen Roberts Theatre, The University of Sydney. This conference is interdisciplinary, with emphasis on cross discipline communication between Neuroscientists, Engineers, Computer Scientists, Mathematicians and Psychologists concerned with understanding the integrative nature of the nervous system and its implementation in hardware/software. Neuroscientists concerned with understanding the integrative function of neural networks in vision, audition, motor, somatosensory and autonomic functions are invited to participate and learn how modelling these systems can be used to sharpen the design of experiments as well as to interpret data. Mathematicians and computer scientists concerned with the various new neural network algorithms that have recently become available, as well as with statistical thermodynamic approaches to network modelling and simulation are also invited to contribute. Engineers concerned with the advantages which parallel and distributed computing architectures offer in the solution of various classes of problems and with the state of the art techniques in the hardware implementation of neural network systems are also invited to participate. Psychologists interested in computational models of cognition and perception are invited to contribute and to learn about neural network techniques and their biological and hardware implementations. ACNN'91 will feature invited keynote speakers in the areas of neuroscience, learning, modelling and implementations. The program will include pre-conference workshops, presentations and poster sessions. Proceedings will be printed and distributed to the attendees. Expression of Interest: ----------------------- Please fill the expression of interest form below and return it to Miss Justine Doherty at the address below. ___ I wish to attend the conference ___ I wish to attend the workshops ___ I wish to present a paper Title: _____________________________________________________ _____________________________________________________ Authors: ___________________________________________________ _____________________________________________________ ___ I wish to be on your mailing list My areas of interests are: ____ Neuroscience ____ Learning ____ Modelling ____ Implementation ____ Applications ____ Other: _______________________________________ First Name: ___________________________ Surname: ______________________________ Title: ________________________________ Position:______________________________ Department: ___________________________________________________________ Institution:___________________________________________________________ Address:_______________________________________________________________ _______________________________________________________________________ City: _______________________________ State: __________________________ Zip Code: _________________________ Country: __________________________ Tel: ______________________________ Fax: ______________________________ Email: _________________________________________ ________________________________________________________________________ Organising committee: Chairman Dr Marwan Jabri, Sydney Co-chairman Professor Max Bennett, Sydney Technical Program Chairman Dr Ah Chung Tsoi, ADFA Technical Program Co-Chairman Professor Bill Levick, ANU Publicity Dr Janet Wiles, Queensland Registration Electrical Engineering Foundation Conference Committee Professor Yanni Attikiousel, WA Professor Max Bennett, Sydney Professor Bob Bogner, Adelaide Professor Richard Brent, ANU Dr Jacob Cybulski, TRL Dr Marwan Jabri, Sydney Professor Bill Levick, ANU Dr Tom Osbourn, UTS Professor Steve Redman, ANU Ass/Prof Sam Reisenfeld, OTC Ltd Professor Graham Rigby, UNSW Professor Steve Schwartz, Queensland Dr Ah Chung Tsoi, ADFA Dr Charles Watson, DSTO Dr Janet Wiles, Queensland Dr Hong Yan, Sydney For further information contact: Miss Justine Doherty Secretariat ACNN'91 Sydney University Electrical Engineering NSW 2006 Australia Tel: (+61-2) 692 3659 Fax: (+61-2) 692 3847 Email: acnn91 at ee.su.oz.au _____________________________________________________________________________ From PF103%phoenix.cambridge.ac.uk at NSFnet-Relay.AC.UK Thu Mar 15 04:35:12 1990 From: PF103%phoenix.cambridge.ac.uk at NSFnet-Relay.AC.UK (Peter Foldiak) Date: Thu, 15 Mar 90 09:35:12 GMT Subject: Neural models of attention? In-Reply-To: Msg of Tue 13 Mar 90 21:21:22 PST from Peter Marvit Message-ID: From marcus%cns.edinburgh.ac.uk at NSFnet-Relay.AC.UK Thu Mar 15 09:09:36 1990 From: marcus%cns.edinburgh.ac.uk at NSFnet-Relay.AC.UK (Marcus Frean) Date: Thu, 15 Mar 90 14:09:36 GMT Subject: TR available Message-ID: <4986.9003151409@subnode.cns.ed.ac.uk> The following technical report is available: The Upstart algorithm : a method for constructing and training feed-forward neural networks Marcus Frean Center for Cognitive Science University of Edinburgh ABSTRACT A general method for building and training multi-layer perceptrons composed of linear threshold units is proposed. A simple recursive rule is used to build the net's structure by adding units as they are needed, while a modified Perceptron algorithm is used to learn the connection strengths. Convergence to zero errors is guaranteed for any Boolean classification on patterns of binary variables. Simulations suggest that this method is efficient in terms of the numbers of units constructed, and the networks it builds can generalise over patterns not in the training set. ---------------------------------------------------------------------- This is available in Postscript form by anonymous ftp as follows: unix> ftp cheops.cis.ohio-state.edu (or, ftp 128.146.8.62) Name: anonymous Password: neuron ftp> cd pub/neuroprose/Inbox ftp> binary ftp> get upstart.ps.Z ftp> quit unix> uncompress upstart.ps.Z unix> lpr upstart.ps (use flag your printer needs for Postscript) [The report should be moved to pub/neuroprose fairly soon] ---------------------------------------------------------------------- Please make requests for hard-copy only if you can't get it by ftp. Marcus Frean email: marcus at cns.ed.ac.uk mail : Center for Cognitive Science University of Edinburgh 2 Buccleuch Place Edinburgh EH8 9LW United Kingdom From Connectionists-Request at CS.CMU.EDU Thu Mar 15 11:23:22 1990 From: Connectionists-Request at CS.CMU.EDU (Connectionists-Request@CS.CMU.EDU) Date: Thu, 15 Mar 90 11:23:22 EST Subject: Fwd: Re: Neural models of attention? Message-ID: <29860.637518202@B.GP.CS.CMU.EDU> This should have gone to the whole list. ------- Forwarded Message From rsun at chaos.cs.brandeis.edu Wed Mar 14 13:19:20 1990 From: rsun at chaos.cs.brandeis.edu (Ron Sun) Date: Wed, 14 Mar 90 13:19:20 -0500 Subject: Neural models of attention? Message-ID: <9003141819.AA00896@chaos> Francis Crick had a paper on "searchlight hypothesis" , which is relevant to the issues of attention. It is reprinted in Neurocomputing: foundation of research (eds. Rosenfeld and Anderson). Also, in (Sun 1989) the issues of attentional mechanisms are touched upon. - --------------------- Sun, R. A discrete neural network model for conceptual representation and reasoning. 11th Cognitive science Conference, 1989 - -------------------- Ron Sun Brandeis University ------- End of Forwarded Message From ajr%engineering.cambridge.ac.uk at NSFnet-Relay.AC.UK Thu Mar 15 14:41:28 1990 From: ajr%engineering.cambridge.ac.uk at NSFnet-Relay.AC.UK (Tony Robinson) Date: Thu, 15 Mar 90 19:41:28 GMT Subject: Summary and tech report and thesis availability (long) Message-ID: <25781.9003151941@dsl.eng.cam.ac.uk> There are three topics in this (long) posting: Summary of replies to my message "problems with large training sets". Tech report availability announcement "Phoneme Recognition from the TIMIT database using Recurrent Error Propagation Networks" Thesis availability announcement "Dynamic Error Propagation Networks" Mail me (ajr at eng.cam.ac.uk) if you would like a copy of the tech report and thesis (I will be at ICASSP if anyone there would like to discuss (or save me some postage)). Tony Robinson /*****************************************************************************/ Subject: Summary of replies to my message "problems with large training sets" Thanks to: Ron Cole, Geoff Hinton, Yann Le Cun, Alexander Singer, Fu-Sheng Tsung, Guy Smith and Rich Sutton for their replies, here is a brief summary: Adaptive learning rates: The paper that was most recommended was: Jacobs, R. A. (1988) Increased rates of convergence through learning rate adaptation. {Neural Networks}, {\em 1} pp 295-307. The scheme described in this paper is nice in that it allows the step size scaling factor (\eta) for each weight to vary independently and variations of two orders of magnitude have been observed. Use a faster machine: Something like a 2.7 GFlop Connection Machine could shake some of these problems away! There are two issues here, one is understanding the problem from which more efficient algorithms naturally develop, the other is the need to get results. I don't know how the two will balance in future, but my guess is that we will need more compute. Combined subset training: Several people have used small subsets for initial training, with later training combining these subsets. The reference I was sent was: Fu-Sheng Tsung and Garrison Cottrell (1989) A Sequential Adder with Recurrent Networks IJCNN 89, June, Washington D.C For reasons of software homogeneity, I prefer to use an increasing momentum term, initially it smooths over one "subset" but this increases until the smoothing is over the whole training set. I've never done a comparison of these techniques. Use of higher order derivatives: A good step size can be estimated from the second order derivatives. To me this looks very promising, but I haven't had time to play with it yet. The reference is: Le Cun, Y.: "Generalization and Network Design Strategies", Tech Report CRG-TR-89-4, Dept. of computer science, University of Toronto, 1989. /*****************************************************************************/ Subject: Tech report availability announcement: Phoneme Recognition from the TIMIT database using Recurrent Error Propagation Networks CUED/F-INFENG/TR.42 Tony Robinson and Frank Fallside Cambridge University Engineering Department, Trumpington Street, Cambridge, England. Enquiries to: ajr at eng.cam.ac.uk This report describes a speaker independent phoneme recognition system based on the recurrent error propagation network recogniser described in (RobinsonFallside89, FallsideLuckeMarslandOSheaOwenPragerRobinsonRussell90). This recogniser employs a preprocessor which generates a range of types of output including bark scaled spectrum, energy and estimates of formant positions. The preprocessor feeds a fully recurrent error propagation network whose outputs are estimates of the probability that the given frame is part of a particular phonetic segment. The network is trained with a new variation on the stochastic gradient descent procedure which updates the weights by an adaptive step size in the direction given by the sign of the gradient. Once trained, a dynamic programming match is made to find the most probable symbol string of phonetic segments. The recognition rate is improved considerably when duration and bigram probabilities are used to constrain the symbol string. A set of recognition results is presented for the trade off between insertion and deletion errors. When these two errors balance the recognition rate for all 61 TIMIT symbols is 68.6% correct (62.5% including insertion errors) and on a reduced 39 symbol set the recognition rate is 75.1% correct (68.9%). This compares favourably with the results of other methods on the same database (ZueGlassPhillipsSeneff89, DigalakisOstendorfRohlicek89, HataokaWaibel89, LeeHon89, LevinsonLibermanLjoljeMiller89). /*****************************************************************************/ Subject: Thesis availability announcement "Dynamic Error Propagation Networks" Please forgive me for the title, a better one would have been "Recurrent Error Propagation Networks". This is my PhD thesis, submitted in Feb 1989 and is a concatenation of the work I had done to that date. Summary: This thesis extends the error propagation network to deal with time varying or dynamic patterns. Examples are given of supervised, reinforcement driven and unsupervised learning. Chapter 1 presents an overview of connectionist models. Chapter 2 introduces the error propagation algorithm for general node types. Chapter 3 discusses the issue of data representation in connectionist models. Chapter 4 describes the use of several types of networks applied to the problem of the recognition of steady state vowels from multiple speakers. Chapter 5 extends the error propagation algorithm to deal with time varying input. Three possible architectures are explored which deal with learning sequences of known length and sequences of unknown and possibly indefinite length. Several simple examples are given. Chapter 6 describes the use of two dynamic nets to form a speech coder. The popular method of Differential Pulse Code Modulation for speech coding employs two linear filters to encoded and decode speech. By generalising these to non-linear filters, implemented as dynamic nets, a reduction in the noise imposed by a limited bandwidth channel is achieved. Chapter 7 describes the application of a dynamic net to the recognition of a large subset of the phonemes of English from continuous speech. The dynamic net is found to give a higher recognition rate both in comparison with a fixed window net and with the established k nearest neighbour technique. Chapter 8 describes a further development of dynamic nets which allows them to be trained by a reinforcement signal which expresses the correctness of the output of the net. Two possible architectures are given and an example of learning to play the game of noughts and crosses is presented. From sayegh at ed.ecn.purdue.edu Thu Mar 15 17:41:16 1990 From: sayegh at ed.ecn.purdue.edu (Samir Sayegh) Date: Thu, 15 Mar 90 17:41:16 -0500 Subject: NN Conference April 12-13-14 Message-ID: <9003152241.AA06442@ed.ecn.purdue.edu> Third Conference on Neural Networks and PDP (Robotics and Vision) Indiana-Purdue University Deadline for submission of a 1 page abstract is March 23. e-mail and FAX submissions OK. Conference fee is $25. Students attend free. Inquiries and abstracts: S.Sayegh Physics Dept. Indiana Purdue University Ft Wayne In 46805 email: sayegh at ed.ecn.purdue.edu sayegh at ipfwcvax.bitnet From reggia at cs.UMD.EDU Thu Mar 15 20:22:20 1990 From: reggia at cs.UMD.EDU (James A. Reggia) Date: Thu, 15 Mar 90 20:22:20 -0500 Subject: research position available Message-ID: <9003160122.AA29318@mimsy.UMD.EDU> RESEARCH SCIENTIST POSITION AVAILABLE IN NEURAL MODELLING The component of The Food and Drug Administration responsible for regulating medical devices has an opening for a research scientist. This is a permanent civil service position available for someone interested in modelling the neural activity of the hippocampus. The candidate will focus his/her research on improving the safety and effectiveness of electro-convulsive therapy devices. The candidate must have a PhD in one of the physical sciences. Any additional training in the biological sciences is highly desirable. For more information call or write to: Dr. C. L. Christman (301) 443-3840 Address: FDA HFZ-133 12721 Twinbrook Pkwy Rockville, MD 20857 (Do NOT send inquiries for further information via email to the individual posting this announcement.) From Tuya at etsiig.uniovi.es Fri Mar 16 09:50:00 1990 From: Tuya at etsiig.uniovi.es (Javier Tuya Gonzalez) Date: 16 Mar 90 16:50 +0200 Subject: Neural Networks application? Message-ID: <100*Tuya@etsiig.uniovi.es> Has anybody information (people who works in, papers published, etc.) about some application of Neural Networks for: Classification of crystal structures using X-ray diffraction data Could be possible to apply Neural Networks for it? Please, post me E-mail if you can help me. Thanks in advance. +--------------------------------------+------------------------------------+ | Pablo Javier Tuya Gonzalez | PSI: PSI%(02145)285060338::TUYA | | E.T.S. Ingenieros Industriales | E-Mail: tuya at etsiig.uniovi.es | | Area de Lenguajes y Sistemas | HEPNET: tuya at 16515.decnet.cern.ch | | Informaticos (Universidad de Oviedo) | : EASTVC::TUYA (16.131) | | Carretera de Castiello s/n. | Phone: (..34-85) 338380 ext 278 | | E-33394 GIJON/SPAIN | FAX: (..34-85) 338538 | +--------------------------------------+------------------------------------+ From skrzypek at CS.UCLA.EDU Fri Mar 16 21:51:51 1990 From: skrzypek at CS.UCLA.EDU (Dr. Josef Skrzypek) Date: Fri, 16 Mar 90 18:51:51 PST Subject: last call for abstracts Message-ID: <9003170251.AA16795@retina.cs.ucla.edu> I am organizing two sessions on Artificial Neural Systems and Computational Neuroscience (ANS/CN) for the IEEE International Conference on Systems, Man and Cybernetics. The conference takes place in Los Angeles in Nov 4-7. One session is focused on vision within the scope of ANS/CN. The deadline for extended abstracts is March 23rd. All contributors of selected abstracts will be invited to submitt a contributed paper. The deadline for contributed papers is April 30.. All contributed papers will reviewed by two referees. Notification about acceptance will be send before July 15th. All accepted papers will be published in IEEE SMC Conference proceedings. Deadline for final typed mats for proceedings will be Aug. 15th. Prof Josef Skrzypek Dir. Machine Perception Laboratory Department of Computer Science 3532 BH UCLA Los Angeles CA 90024-1596 From marcus%cns.edinburgh.ac.uk at NSFnet-Relay.AC.UK Fri Mar 16 09:43:40 1990 From: marcus%cns.edinburgh.ac.uk at NSFnet-Relay.AC.UK (Marcus Frean) Date: Fri, 16 Mar 90 14:43:40 GMT Subject: Obtaining "Upstart algorithm" TR by ftp. Message-ID: <323.9003161443@subnode.cns.ed.ac.uk> Due to problems with transferring the compressed version, I've now put an uncompressed version in it's place, which seems to be okay. The tech report is entitled The Upstart algorithm : a method for constructing and training feed-forward neural networks. Marcus Frean. and you can get it as follows: ----------------------------------------------------------------- unix> ftp cheops.cis.ohio-state.edu (or, ftp 128.146.8.62) Name: anonymous Password: neuron ftp> cd pub/neuroprose/Inbox ftp> get upstart.ps ftp> quit unix> lpr upstart.ps (use flag your printer needs for Postscript) ------------------------------------------------------------------ [The report may be compressed and/or moved to pub/neuroprose soon] Let me know if you can't get it by ftp. Marcus Frean email: marcus at cns.ed.ac.uk mail : Center for Cognitive Science University of Edinburgh 2 Buccleuch Place Edinburgh EH8 9LW United Kingdom From sls at dsl.pitt.edu Mon Mar 19 21:05:38 1990 From: sls at dsl.pitt.edu (Steven L. Small) Date: Mon, 19 Mar 90 21:05:38 -0500 Subject: Mathematical Tractability of Neural Nets Message-ID: <9003200205.AA00586@cadre.dsl.pitt.edu> I agree with Liz Bates about neurological localization in general, and language functions in particular. One major problem with localization is that the data come from experiments of nature in which large areas of brain are damaged; inferences about small areas of the brain are made by comparing individuals to look for intersections in both damaged brain and in deficient cognitive processing abilities. There are lots of problems with this, and a number of assumptions of the enterprise are probably wrong (e.g., that computational organizations across individuals do not differ at the gross neuroanatomical level). I also agree that connectionist networks make a better metaphor for brain computations than do filing cabinets and filing clerks (or the store and retrieve operations of CPUs). Regards, Steve Small (neurologist among other things). From delta at csl.ncsu.edu Tue Mar 20 09:04:10 1990 From: delta at csl.ncsu.edu (Thomas Hildebrandt) Date: Tue, 20 Mar 90 09:04:10 EST Subject: Selective Attention Message-ID: <9003201404.AA22459@csl36h.ncsu.edu> Paul Kube of UCSD mentions: In a recent paper (Nature, 30 November 1989, pp. 543-545), Luck, Hillyard, Mangun and Gazzaniga report that split-brain patients are twice as good as normals on Triesman-type conjunctive feature visual search tasks when the stimulus array is distributed across both hemifields, but no better than normals when the array is restricted to one hemifield. This suggests that commissurotomy permits a "splitting of attention" that is impossible with connected hemispheres, and that remains impossible within each hemisphere. It seems to be fairly obvious that attention is impossible without inhibition, and that attention can be interpreted to be the relative lack of it in a subset of neurons. If you adopt this view, then the results of the paper mentioned by kube can be easily explained: One hemisphere inhibits the other. If the connections between them are cut, then each may act independently -- thus doubling the apparent capacity for attention of the brain as a whole. However, I imagine that a commissurotomy also has some UNdesirable effects. . . . Thomas H. Hildebrandt North Carolina State From mariah!yak at tucson.sie.arizona.edu Wed Mar 21 07:32:48 1990 From: mariah!yak at tucson.sie.arizona.edu (mariah!yak@tucson.sie.arizona.edu) Date: Wed, 21 Mar 90 05:32:48 -0700 Subject: No subject Message-ID: <9003211232.AA13367@tucson.sie.arizona.edu> Dear Connectionists, I have a small NSF grant to investigate statistical aspects of machine learning and its relation to neural networks. Dr. H. Gigley suggested to me that I would find it worthwhile the have my name added to a mailing list in the NN topic area. Attached to her message was a news item bearing your address. If you know of the internet mailing list to which she was referring, please send any information you can. Gratefully, Sid Yakowitz Professor (yak at tucson.sie.arizona.edu) From glb at ecelet.ncsu.edu Wed Mar 21 14:18:01 1990 From: glb at ecelet.ncsu.edu (Griff Bilbro) Date: Wed, 21 Mar 90 14:18:01 EST Subject: Linear Separability Message-ID: <9003211918.AA03102@ecelet.ncsu.edu> The statistical mechanical theory of learning predicts that learning linear separability in the plane depends strongly the location of samples. I have applied theory of learning available in the litera- ture [Tishby, Levin, and Solla, IJCNN, 1989] to the problem of learning from examples the line that separates two classes of points in the plane. When the examples in the training set are chosen uniformly in a unit square bisected by the true separator, learning (as measured by the average prediction probability) begins with the first example. If the training set is chosen at some distance from the line, even more learning occurs. On the other hand, if the training set is chosen close to the line, almost no learning is predicted until the training set size reaches 5 examples, but after that learning is so fast that it exceeds the uniform case by 15 examples. Here learning is measured by the predictive ability of the estimated line rather than its numerical precision. The line may be determined to more significant digits by 5 mali- cious points, but this is not enough if the 6th point is drawn from the same malicious distribution. This doesn't apply to the case when the 6th point is drawn from a dif- ferent distribution. Griff Bilbro. From Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU Thu Mar 22 11:31:10 1990 From: Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF1.SLISP.CS.CMU.EDU) Date: Thu, 22 Mar 90 11:31:10 EST Subject: Cascade-Correlation code available Message-ID: *** Please do not forward this to other mailing lists or newsgroups *** For those of you who want to play with the Cascade-Correlation algorithm, a public-domain Common Lisp version of my Cascade-Correlation simulator is now available for FTP via Internet. This is the same version I've been using for my own experiments, except that a lot of non-portable display and user-interface code has been removed. I believe that this version is now strictly portable Common Lisp. It has been tested on CMU Common Lisp on the IBM RT, Allegro Common Lisp (beta test) for Decstation 3100, and Sun/Lucid Common Lisp on the Sun 3. (In the Lucid system it runs properly, but it spends a lot of time garbage-collecting, so it is very slow; maybe there's some optimization magic in Lucid that I don't know about.) The code is heavily commented, so if you read Lisp at all it should be a straightforward task to translate the program (or just the, inner loops) into the language of your choice. A couple of people have told me they planned to port the code to C, and would share the result, but at present no C version is available. Instructions for obtaining the code via Internet FTP are included at the end of this message. If people can't get it by FTP, contact me by E-mail and I'll try once to mail it to you in a single chunk of 51K bytes. If it bounces or your mailer rejects such a large message, I don't have time to try a lot of other delivery methods. I would appreciate hearing about any interesting applications of this code, and will try to help with any problems people run into. Of course, if the code is incorporated into any products or larger systems, I would appreciate an acknowledgement of where it came from. There are several other programs in the "code" directory mentioned below: versions of Quickprop in Common Lisp and C, and some simulation code written by Tony Robinson for the vowel benchmark he contributed to the benchmark collection. -- Scott *************************************************************************** For people (at CMU, MIT, and soon some other places) with access to the Andrew File System (AFS), you can access the files directly from directory "/afs/cs.cmu.edu/project/connect/code". This file system uses the same syntactic conventions as BSD Unix: case sensitive names, slashes for subdirectories, no version numbers, etc. The protection scheme is a bit different, but that shouldn't matter to people just trying to read these files. For people accessing these files via FTP: 1. Create an FTP connection from wherever you are to machine "pt.cs.cmu.edu". 2. Log in as user "anonymous" with no password. You may see an error message that says "filenames may not have /.. in them" or something like that. Just ignore it. 3. Change remote directory to "/afs/cs/project/connect/code". Any subdirectories of this one should also be accessible. Parent directories may not be. 4. At this point FTP should be able to get a listing of files in this directory and fetch the ones you want. The Cascade-Correlation simulator lives in "cascor1.lisp". If you try to access this directory by FTP and have trouble, please contact me. The exact FTP commands you use to change directories, list files, etc., will vary from one version of FTP to another. From fozzard at alumni.Colorado.EDU Thu Mar 22 12:05:59 1990 From: fozzard at alumni.Colorado.EDU (Richard Fozzard) Date: Thu, 22 Mar 90 10:05:59 -0700 Subject: Comparisons of NetTalk to other approaches Message-ID: <9003221705.AA01306@alumni.colorado.edu> As I remember there was some talk here a while back comparing NetTalk with non-neural net approaches to the same problem. We are putting together a talk to high-level government managers, and would like to mention NetTalk. Anyone out there have any references or thoughts on the advantages and disadvantages of the net approach compared to other methods (eg. statistical, rule-based, anything else)? If I get a number of responses, I will summarize. Thanks for the input! Rich ======================================================================== Richard Fozzard "Serendipity empowers" Univ of Colorado/CIRES/NOAA R/E/FS 325 Broadway, Boulder, CO 80303 fozzard at boulder.colorado.edu (303)497-6011 or 444-3168 From tgd at turing.CS.ORST.EDU Thu Mar 22 14:46:13 1990 From: tgd at turing.CS.ORST.EDU (Tom Dietterich) Date: Thu, 22 Mar 90 11:46:13 PST Subject: Comparisons of NetTalk to other approaches In-Reply-To: Richard Fozzard's message of Thu, 22 Mar 90 10:05:59 -0700 <9003221705.AA01306@alumni.colorado.edu> Message-ID: <9003221946.AA16344@turing.CS.ORST.EDU> There are two studies that I know of comparing NETtalk to decision-tree methods such as Quinlan's ID3. 1. Mooney, R., Shavlik, J., Towell, G., and Gove, A. (1989). An experimental comparison of symbolic and connectionist learning algorithms. {\it IJCAI-89: Eleventh International Joint Conference on Artificial Intelligence}. 775--80. This paper included a simple comparison of ID3 and backprop on the nettalk task. 2. Dietterich, T. G., Hild, H., Bakiri, G. (1990) A comparative study of ID3 and backpropagation for English text-to-speech mapping. To appear in 1990 Machine Learning Conference, Austin, TX. This is a more detailed study. I'll be producing a tech report soon and I'll announce availability to connectionists. The bottom line seems to be that, while backprop is awkward and time-consuming to apply, it does give slightly better results on this task. From DGOLDBER at UA1VM.ua.edu Thu Mar 22 15:32:27 1990 From: DGOLDBER at UA1VM.ua.edu (Dave Goldberg) Date: Thu, 22 Mar 90 14:32:27 CST Subject: Dortmund Workshop Message-ID: FIRST ANNOUNCEMENT and CALL FOR PAPERS International Workshop Parallel Problem Solving from Nature (PPSN) October 1 - 3, 1990 University of Dortmund, Germany F.R. Scope With the appearance of massively parallel computers increased attention has been paid to algorithms which rely upon analogies to natural processes. The workshop scope includes but is not limited to the following topics: - Darwinian methods such as Evolution Strategies and Genetic Algorithms - Boltzmann methods such as Simulated Annealing - Classifier systems and Neural Networks insofar as problem solving predominates - Transfer of other natural metaphors to artificial problem solving The objectives of this workshop are - to bring together scientists and practitioners working on and with such strategies. - to gather theoretical results about as well as experimental comparisons between these algorithms. - to discuss various implementations on different parallel computer architectures (e.g. SIMD, MIMD, LAN). - to look for current and future applications in science, technology, and administration. - to summarize the state of the art in this field which up to now has been scattered so widely among disciplines as well as geographically. Submission of papers, Proceedings Prospective authors are invited to submit 4 copies of an extended abstract of two pages to the conference chair before June 1, 1990. All contributions will be reviewed by the programme committee and up to about 30 papers will be selected for presentation. Authors will get notice about acceptance or rejection of their papers by July 15, 1990. Full papers will be due on September 1, 1990. They will be delivered to all participants at the conference as a prepublication volume. Final papers for the proceedings of the workshop should be finished immediately after the workshop. Details about the format of the camera-ready final papers will be distributed later. Language The official language for papers and presentations is English. Conference Chair: H. Muehlenbein and H.-P. Schwefel Gesellschaft fuer Mathematik University of Dortmund und Datenverarbeitung (GMD) -Z1- Dept. of Computer Science P. O. Box 12 40, Schloss Birlinghoven P. O. Box 50 05 00 D-5205 St. Augustin 1 D-4600 Dortmund 50 F. R. Germany F. R. Germany Tel. +49-2241-142405 Tel. +49-231-755-4590 Fax +49-2241-142889 Fax +49-231-755-2047 bitnet grzia0 at dbngmd21 bitnet uin005 at ddohrz11 Programme Committee: (chair) D.E. Goldberg Univ. of Alabama, Tuscaloosa, USA (chair) R. Maenner Univ. of Heidelberg, FRG Institute of Physics Philosophenweg 12 D-6900 Heidelberg 1 Tel. +49-6221-569363 Fax +49-6221-475733 bitnet maen at dhdmpi50 E.M.L. Aarts Philips Res.Lab. Eindhoven, NL P. Bock Univ. of Washington DC, USA V. Cerny Univ. of Bratislava, CSSR Y. Davidor Weizmann Inst. Rehovot, Israel G. Dueck IBM Heidelberg, FRG J.J. Grefenstette Naval Res.Lab. Washington DC, USA A.W.J. Kolen Univ. of Limburg, Maastricht, NL B. Manderick Univ. of Brussels, Belgium H. Roeck Univ. of Bielefeld, FRG H. Schwaertzel Siemens AG Munich, FRG B. Soucek Univ. of Zagreb, YU H.-M. Voigt Academy of Sciences Berlin, GDR Organization Committee: J. Becker, H. Bracklo, H.-P. Schwefel, E. Speckenmeyer, A. Ultsch Sponsors: Parsytec GmbH and Paracom GmbH, IBM Deutschland GmbH, Siemens AG Deadlines: Abstracts (2 pages) June 1, 1990 Notification of acceptance July 15, 1990 Full papers (for preprints) September 1, 1990 Workshop October 1-3, 1990 Final papers November 1, 1990 Reply Form International Workhop Parallel Problem Solving from Nature (PPSN) Dortmund, October 1-3, 1990 c/o Prof. Dr. H.-P. Schwefel Dept. of Computer Science Tel. +49-2 31/7 55/45 90 P. O. Box 50 05 00 Fax +49-2 31/7 55/20 47 D-4600 Dortmund 50 bitnet uin005 at ddohrz11 F. R. Germany Title First Name Middle Initials Last Name ................................................................. Institution ..................................................... Address ......................................................... ................................................................. ................................................................. ( ) Please send further information ( ) I intend to attend the workhop ( ) I intend to submit an abstract: Title of paper to be presented ................................................................. ................................................................. From P.Refenes at Cs.Ucl.AC.UK Fri Mar 23 05:57:00 1990 From: P.Refenes at Cs.Ucl.AC.UK (P.Refenes@Cs.Ucl.AC.UK) Date: Fri, 23 Mar 90 10:57:00 +0000 Subject: C-Implementation of BOLTZMANN learning. Message-ID: Is anyone out there in possesion of a C implementation of BOLTZMANN learning? Is She/He willing to send us a copy of the sources? Thanks in advance, Paul refenes. From Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU Fri Mar 23 09:10:55 1990 From: Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF1.SLISP.CS.CMU.EDU) Date: Fri, 23 Mar 90 09:10:55 EST Subject: FTPing code from CMU Message-ID: I forgot to include this in my earlier message: For people who need internet numbers instead of machine names, the machine "pt.cs.cmu.edu" is 128.2.254.155 -- Scott From khaines at galileo.ECE.CMU.EDU Fri Mar 23 15:33:58 1990 From: khaines at galileo.ECE.CMU.EDU (Karen Haines) Date: Fri, 23 Mar 90 15:33:58 EST Subject: ICNN - Request for Volunteers Message-ID: <9003232033.AA26990@galileo.ece.cmu.edu> *************************************************************************** ICNN REQUEST FOR VOLUNTEERS July 9-13,1990 Paris, France *************************************************************************** This is the first call for volunteers to help at the ICNN conference, to be held at the Palias Des Congres in Paris, France, on July 9-13,1990. Full admittance to the conference and a copy of the proceedings is offered in exchange for your assistance throughout the conference. In general, each volunteer is expected to work one shift each day of the conference. Hours are approximately: AM shift - 7:00 am - 1:00pm PM shift - Noon - 6:00 pm In addition, assistance may be required for the social events. Below is a description of the available positions. If you are interested in volunteering, please send me the following information: Name Address Phone number Country electronic mail address shift preference Positions are being filled on a first commit first served basis. If you have further questions, please feel free to contact me. Karen Haines Dept. of ECE Carnegie Mellon University Pittsburgh, PA 15213 message: (412) 362-8675 email: khaines at galileo.ece.cmu.edu At this time there is no funding available from the conference to cover traveling/lodging expenses. Nor do I anticipate any funding available. Thank you, Karen Haines ICNN Volunteer Coordinator Volunteer Positions (volunteers needed) - Description (please note that hours are subject to change) --------------------------------------------------------- Exhibits - Stuffing Proceedings (8) - These volunteers will be required to work Sunday 9am-6pm, Monday 8am-6pm, and Tuesday 8am-12pm. Sunday and Monday will be used to stuff proceedings into the bags. Monday/Tuesday they will double in the exhibits area assisting the Exhibits Chair exhibitors. Poster Session (8) - The volunteers will be responsible for assisting the presenters in putting up/taking down their posters. Days that they will be Shifts are AM or PM Tues thru Thurs. (Hours - General) Conference Sessions (16) - The number of Technical sessions that will be occurring each morning and afternoon of the conference is 4. Two volunteers will be used to check badges at the door for each technical session. Volunteers working the technical sessions will be assigned mornings or afternoons in groups of two. Note that they will be working with the same person each day throughout the conference. Shifts are AM or PM, Tues-Fri. (Hours - General) Exhibit Area II (4) : - Two volunteers will be used to check badges at the door. Volunteers will be assigned mornings or afternoons. Shifts are AM or PM, Tues-Fri. (Hours - General) Message Center (4) - Volunteers will be responsible for the message center. Two volunteers in the morning, two in the afternoon. Shifts are AM or PM Mon-Fri. (Hours - General) Reception at the Hotels (24) - Volunteers will be posted at 6 hotels to provide directions to the conference. Working in teams of 2, these volunteers will be required to work Sunday 9am-9pm, Monday 9am-9pm. From koch%HAMLET.BITNET at VMA.CC.CMU.EDU Sun Mar 25 19:07:52 1990 From: koch%HAMLET.BITNET at VMA.CC.CMU.EDU (Christof Koch) Date: Sun, 25 Mar 90 16:07:52 PST Subject: models of attention In-Reply-To: Your message <9003150635.AA19417@kokoro.UCSD.EDU> dated 14-Mar-1990 Message-ID: <900325160731.23a01f3f@Hamlet.Caltech.Edu> This is in response to P. Kube's question re. the recent Gazzaniga report on split-brain patients possibly having two "searchlights" of attention. The model Shimon Ullman and I proposed would accomodate such a finding, by just cutting our Winner-Take-All pyramid in two, leading to two maximally salient points out there in the visual field. The problem, though, is in locating this control structure. F. Crick proposed in 1984 the Reticular Nucleus of the thalamus as the site of this mechanism. However, the NRT, as are all the other thalamic nuclei, is NOT interconnected with the NRT on the contralateral sienuclei, is no In fact, no thalamic nuclei, with the exception of the ventral lateral geniculate nucleus (which is different from the better known dorsal lateral geniculate nucleus relaying visual information to the cortex), has interhemispheric connections. This seems to imply that the structure controlling attention may reside in neocortex proper (including the claustrum). One caveat, of course, is that negative anatomical findings can always be overthrown one day with better techniques. F. Crick and I have a paper coming out where we discuss a lot of these things in relation to iconic and short-term memory, awareness, attention and neuronal oscillations. Christof From zl at aurel.cns.caltech.edu Mon Mar 26 11:54:36 1990 From: zl at aurel.cns.caltech.edu (Zhaoping Li) Date: Mon, 26 Mar 90 08:54:36 PST Subject: No subject Message-ID: <9003261654.AA00950@aurel.cns.caltech.edu> "A model of the olfactory adaptation and sensitivity enhancement in the olfactory bulb" by Zhaoping LI, in Biological Cybernetics p349-361 62/4 1990. Adaptation and enhancement --- Attention!? Central inputs 'similar' to Crick's search light is used for the mechanism. This paper describes a model of a oscillatory neural system in the bulb, agrees with experimental findings and suggests further experiments. From cole at cse.ogi.edu Mon Mar 26 19:48:50 1990 From: cole at cse.ogi.edu (Ron Cole) Date: Mon, 26 Mar 90 16:48:50 -0800 Subject: English Alphabet Recognition Database Message-ID: <9003270048.AA16206@cse.ogi.edu> The ISOLET database is available for research on speaker-independent classification of spoken English letters. ISOLET contains 2 utterances of the letters A-Z by 150 speakers. The sampling rate, microphone and filter were chosen to mimic the TIMIT database. The database (about 150 MB) is available on 1/2 inch tape, Exabyte cassette, Sun or DEC TK50 cartridge. There is a small charge to cover shipping, handling and storage medium costs. For a description of the database and details on how to order it, send mail to vincew at cse.ogi.edu. Ask for TR 90-004. Ronald A. Cole Computer Science and Engineering Oregon Graduate Institute of Science and Technology 19600 N.W. Von Neumann Dr. Beaverton, OR 97006-1999 cole at cse.ogi.edu 503 690 1159 From risto at CS.UCLA.EDU Tue Mar 27 17:31:36 1990 From: risto at CS.UCLA.EDU (Risto Miikkulainen) Date: Tue, 27 Mar 90 14:31:36 pst Subject: abstracts for 2 tech reports in neuroprose Message-ID: <9003272234.AA08980@shemp.cs.ucla.edu> *********** Do not forward to other bboards ************* The following tech reports are available by anonymous ftp from the pub/neuroprose directory at cheops.cis.ohio-state.edu: A DISTRIBUTED FEATURE MAP MODEL OF THE LEXICON Risto Miikkulainen Ailab, Computer Science Department, UCLA Technical Report UCLA-AI-90-04 DISLEX models the human lexical system at the level of physical structures, i.e. maps and pathways. It consists of a semantic memory and a number of modality-specific symbol memories, implemented as feature maps. Distributed representations for the word symbols and their meanings are stored on the maps, and linked with associative connections. The memory organization and the associations are formed in an unsupervised process, based on co-occurrence of the physical symbol and its meaning. DISLEX models processing of ambiguous words, i.e. homonyms and synonyms, and dyslexic errors in input and in production. Lesioning the system produces lexical deficits similar to human aphasia. DISLEX-1 is an AI implementation of the model, which can be used as the lexicon module in distributed natural language processing systems. ----------------------------------------------------------------------- A NEURAL NETWORK MODEL OF SCRIPT PROCESSING AND MEMORY Risto Miikkulainen Ailab, Computer Science Department, UCLA Technical Report UCLA-AI-90-03 DISCERN is a large-scale NLP system built from distributed neural networks. It reads short narratives about stereotypical event sequences, stores them in episodic memory, generates fully expanded paraphrases of the narratives, and answers questions about them. Processing is based on hierarchically organized backpropagation modules, communicating through a central lexicon of word representations. The lexicon is a double feature map, which transforms the physical word symbol into its semantic representation and vice versa. The episodic memory is a hierarchy of feature maps, where memories are stored ``one-shot'' at different locations. Several high-level phenomena emerge automatically from the special properties of distributed neural networks. DISCERN plausibly infers unmentioned events and unspecified role fillers, and exhibits plausible lexical access errors and memory interference behavior. Word semantics, memory organization and the appropriate script inferences are extracted from examples. ----------------------------------------------------------------------- To obtain copies, either: a) use the getps script (by Tony Plate and Jordan Pollack, posted on connectionists a few weeks ago) b) unix> ftp cheops.cis.ohio-state.edu # (or ftp 128.146.8.62) Name (cheops.cis.ohio-state.edu:): anonymous Password (cheops.cis.ohio-state.edu:anonymous): ftp> cd pub/neuroprose ftp> type binary ftp> get (remote-file) miikkulainen.lexicon.ps.Z (local-file) foo.ps.Z ftp> get (remote-file) miikkulainen.discern.ps.Z (local-file) bar.ps.Z ftp> quit unix> uncompress foo.ps bar.ps unix> lpr -P(your_local_postscript_printer) foo.ps bar.ps From jacobs at gluttony.cs.umass.edu Wed Mar 28 09:42:52 1990 From: jacobs at gluttony.cs.umass.edu (jacobs@gluttony.cs.umass.edu) Date: Wed, 28 Mar 90 09:42:52 EST Subject: new technical report available Message-ID: <9003281442.AA08428@ANW.edu> The following technical report is now available: Task Decomposition Through Competition In a Modular Connectionist Architecture: The What and Where Vision Tasks Robert A. Jacobs (UMass) Michael I. Jordan (MIT) Andrew G. Barto (UMASS) COINS Technical Report 90-27 Abstract -------- A novel modular connectionist architecture is presented in which the networks composing the architecture compete to learn the training patterns. An outcome of the competition is that different networks learn different training patterns and, thus, learn to compute different functions. The architecture performs task decomposition in the sense that it learns to partition a task into two or more functionally independent tasks and allocates distinct networks to learn each task. In addition, the architecture tends to allocate to each task the network whose topology is most appropriate to that task. The architecture's performance on ``what'' and ``where'' vision tasks is presented and compared with the performance of two multi--layer networks. Finally, it is noted that function decomposition is an underconstrained problem and, thus, different modular architectures may decompose a function in different ways. We argue that a desirable decomposition can be achieved if the architecture is suitably restricted in the types of functions that it can compute. Appropriate restrictions can be found through the application of domain knowledge. A strength of the modular architecture is that its structure is well--suited for incorporating domain knowledge. If possible, please obtain a postscript version of this technical report from the pub/neuroprose directory at cheops.cis.ohio-state.edu. a) Here are the directions: unix> ftp cheops.cis.ohio-state.edu # (or ftp 128.146.8.62) Name (cheops.cis.ohio-state.edu:): anonymous Password (cheops.cis.ohio-state.edu:anonymous): neuron ftp> cd pub/neuroprose ftp> type binary ftp> get (remote-file) jacobs.modular.ps.Z (local-file) foo.ps.Z ftp> quit unix> uncompress foo.ps.Z unix> lpr -P(your_local_postscript_printer) foo.ps b) You can also use the Getps script posted on the connectionist mailing list a few weeks ago. If you do not have access to a postscript printer, copies of this technical report can be obtained by sending requests to Connie Smith at smith at cs.umass.edu. Remember to ask for COINS Technical Report 90-27. From bates at amos.ucsd.edu Wed Mar 28 12:12:48 1990 From: bates at amos.ucsd.edu (Elizabeth Bates) Date: Wed, 28 Mar 90 09:12:48 PST Subject: aphasia references Message-ID: <9003281712.AA21945@amos.ucsd.edu> I've had a number of requests for references pertaining to our recent discussion about aphasia and brain localization. Here are a few SOME OF THE RELEVANT REFERENCES FROM THE RECENT DEBATE ON APHASIA ON THE CONNECTIONIST NET WOULD INCLUDE THE FOLLOWING: I. Some of the cross-language studies of aphasia from our laboratories: Bates, E. & Wulfeck, B. (1989). Comparative aphasiology: a cross-linguistic approach to language breakdown. Aphasiology, 3, 111-142. Review of the cross-language work. Bates, E. & Wulfeck, B. (1989). Cross-linguistic studies of aphasia. In B. MacWhinney & E. Bates (Eds.), The cross-linguistic study of sentence processing. New York, Cambridge University Press. Another review, nested within a volume summarizing our cross-language work with normals as well. Bates, E., Friederici, A., & Wulfeck, B. (1987a). Comprehension in aphasia: a cross-linguistic study. Brain & Language, 32, 19-67. Among other things, this study shows that "receptive agrammatism" (i.e. partial loss of sensitivity to closed class elements, albeit to a different degree in each language) occurs not only in Broca's, but in Wernicke's, anomics, and in many neurological and non-neurological patients without focal brain injury. In other words, receptive agrammatism may occur in response to generalized stress!! Bates, E., Friederici, A., & Wulfeck, B. (1987b). Grammatical morphology in aphasia: evidence from three languages. Cortex, 23, 545-574. One of the studies that best illustrates how patients use their preserved knowledge to "shape" closed class omission and other typical symptoms. II. A few references from other laboratories on the "new look" in aphasia research: Basso, A., Capitani, E., Laiacona, M. & Luzzatti, C. (1980). Factors influencing type and severity of aphasia. Cortex, 16, 631 - 636 (an archival review of MRI and CT data showing how often the classical teaching re lesion site and aphasia type is violated). Baum, S. (1989). On-line sensitivity to local and long-distance dependencies in Broca's aphasia. Brain & Language, 37, 327-338. Damasio, H. & Damasio, A. (1989). Lesion analysis in neuropsychology. New York: Oxford University Press. Also documents a few of the surprises in brain-behavior mapping. Friederici, A. & Kilborn, K. (1989). Temporal constraints on language processing in Broca's aphasia. Journal of Cognitive Neuroscience, 1, 262-272. A study showing "grammatical priming" in Broca's aphasics. Linebarger, M., Schwartz, M. & Saffran, E. (1983). Sensitivity to grammatical structure in so-called agrammatic aphasics. Cognition, 13, 361-392. The first of what are now many papers demonstrating preservation of grammaticality judgments in "agrammatic" patients. Lukatela, G., Crain, S. & Shankwweiler, D. (1988). Sensitivity to inflectional morphology in agrammatism: investigation of a highly inflected language. Brain & Language, 33, 1 - 15. Miceli, G., Silveri, M., Romani, C. & Caramazza, A. (1989). Variation in the pattern of omissions and substitutions of grammatical morphemes in the spontaneous speech of so-called agrammatic aphasics. Brain & Language, 36, 447-492. This study goes too far in trying to claim that "everything dissociations from everything else", violating a lot of statistical assumptions in the process. Nevertheless, it clearly shows just how much variation can occur among patients from the "same" clinical category, and it also shows that "agrammatic" symptoms are quite common in patients with posterior as opposed to anterior lesions. Milberg, W. & Blumstein, S. (1981). Lexical decision and aphasia: evidence for semantic processing. Brain & Language, 14, 371-385. This is among the first of a series of papers from this laboratory trying to recast the Broca/Wernicke contrast in processing rather than content terms. Ostrin, R. & Schwartz, M. (1986). Reconstructing from a degraded trace: a study of sentence repetition in agrammatism. Brain & Language, 28, 328-345. Similar line of argument to Milberg & Blumstein, although it differs in detail. Shankweiler, D., Crain, S. Gorrell, P. & Tuller, B. (1989). Reception of language in Broca's aphasia. Language and Cognitive Processes, 4, 1 - 33. Still more evidence for preserved grammar in Broca's aphasia. Swinney, D., Zurif, E., Rosenberg, B. & Nicol, J. Modularity and information access in the lexicon: evidence from aphasia. Journal of Cognitive Neuroscience. Sorry I don't have a more specific reference. This paper tries to salvage modularity in aphasia, showing that semantic priming occurs in both Broca's and Wernicke's aphasia, but in slightly different forms. In fact, the paper makes a strong case that aphasic deficits are based on access problems across a preserved knowledge base. Tyler, L. (1989). Syntactic deficits and the construction of local phrases in spoken language comprehension. Cognitive Neuropsychology, 6, 333 - 356. Yet another attempt to rewrite the nature of processing deficits in aphasia, demonstrating that the basic organization of language is preserved. III. Some papers that are relevant to the argument although they do not present new data on aphasic patients. Hinton, G. & Shallice, T. (1989). Lesioning a connectionist network: investigations of acquired dyslexia. (Tech. rep. CRG-TR-89-30. University of Toronto). Funny things can happen when a language net is randomly lesioned -- things that old-style aphasiologists might typically explain with the logic of localization if the same symptoms were observed in a brain-damaged patient. Kutas, M. & Van Petten, C. (1988). Event-related brain potential studies of language. In P.K. Ackles, J. R. Jennings & M. G. H. Coles (Eds.), Advances in psychophysiology, Vol. III. Greenwich, Connecticut, JAI Press, 139 - 187. Posner, M. Petersen, S., Fox, P. & Raichle, M. (1988). Localization of cognitive operations in the human brain. Science, 240, 1627 - 1631. a "new look" at localization based on PET scan data, arguing that components of attention are localized but linguistic content is not. Seidenberg, M., McClelland, J. & Patterson, K. (1987). A distributed developmental model of visual word recognition, naming and dyslexia. Symposium on Connectionism, Annual Meeting of the Experimental Psychological Society (U.K.), Oxford. There is probably a more recent, published version of this reference but I don't have it. Shows how "dyslexic-like" symptoms can arise from random lesions (i.e. non-localized) to a connectionist net. hope these are useful. -liz bates From A.Hurlbert%newcastle.ac.uk at NSFnet-Relay.AC.UK Wed Mar 28 15:23:38 1990 From: A.Hurlbert%newcastle.ac.uk at NSFnet-Relay.AC.UK (Dr A. Hulbert) Date: Wed, 28 Mar 90 15:23:38 BST Subject: abstracts for 2 tech reports in neuroprose Message-ID: From dave at cogsci.indiana.edu Wed Mar 28 15:43:56 1990 From: dave at cogsci.indiana.edu (David Chalmers) Date: Wed, 28 Mar 90 15:43:56 EST Subject: Technical reports available Message-ID: The following two technical reports are now available from the Center for Research on Concepts and Cognition at Indiana University. ------------------------------------------------------------------------------ SYNTACTIC TRANSFORMATIONS ON DISTRIBUTED REPRESENTATIONS David J. Chalmers Center for Research on Concepts and Cognition Indiana University CRCC-TR-40 There has been much interest in the possibility of connectionist models whose representations can be endowed with compositional structure, and a variety of such models have been proposed. These models typically use distributed representations which arise from the functional composition of constituent parts. Functional composition and decomposition alone, however, yield only an implementation of classical symbolic theories. This paper explores the possibility of moving beyond implementation by exploiting holistic structure-sensitive operations on distributed representations. An experiment is performed using Pollack's Recursive Auto-Associative Memory. RAAM is used to construct distributed representations of syntactically structured sentences. A feed-forward network is then trained to operate directly on these representations, modeling syntactic transformations of the represented sentences. Successful training and generalization is obtained, demonstrating that the implicit structure present in these representations can be used for a kind of structure-sensitive processing unique to the connectionist domain. This paper is to appear in CONNECTION SCIENCE. ------------------------------------------------------------------------------ WHY FODOR AND PYLYSHYN WERE WRONG: THE SIMPLEST REFUTATION David J. Chalmers Center for Research on Concepts and Cognition Indiana University CRCC-TR-41 This paper offers both a theoretical and an experimental perspective on the relationship between connectionist and Classical (symbol-processing) models. Firstly, a serious flaw in Fodor and Pylyshyn's argument against connectionism is pointed out: if, in fact, a part of their argument is valid, then it establishes a conclusion quite different from that which they intend, a conclusion which is demonstrably false. The source of this flaw is traced to an underestimation of the differences between localist and distributed representation. It has been claimed that distributed representations cannot support systematic operations, or that if they can, then they will be mere implementations of traditional ideas. This paper presents experimental evidence against this conclusion: distributed representations can be used to support direct structure-sensitive operations, in a manner quite unlike the Classical approach. Finally, it is argued that even if Fodor and Pylyshyn's argument that connectionist models of compositionality must be mere implementations were correct, then this would still not be a serious argument against connectionism as a theory of mind. ------------------------------------------------------------------------------ To obtain a copy of either of these papers, send e-mail to dave at cogsci.indiana.edu. From Clayton.Bridges at GS10.SP.CS.CMU.EDU Wed Mar 28 16:48:05 1990 From: Clayton.Bridges at GS10.SP.CS.CMU.EDU (Clay Bridges) Date: Wed, 28 Mar 90 16:48:05 EST Subject: Size v. Accuracy Message-ID: <5001.638660885@GS10.SP.CS.CMU.EDU> Does anyone know of any theoretical analysis of the tradeoffs between network size and accuracy of generalization? From turing%ctcs.leeds.ac.uk at NSFnet-Relay.AC.UK Thu Mar 29 13:24:02 1990 From: turing%ctcs.leeds.ac.uk at NSFnet-Relay.AC.UK (Turing Conference) Date: Thu, 29 Mar 90 13:24:02 BST Subject: YOUR LAST CHANCE!! Message-ID: <664.9003291224@ctcs.leeds.ac.uk> ___________________________________________________________________________ Computer Studies and Philosophy, University of Leeds, LEEDS, LS2 9JT Friday, 23rd March 1990 TURING 1990 - FINAL REMINDER I would be very grateful if you could bring this notice to the attention of the relevant academic staff and postgraduates in your department, as soon as possible. It concerns a major conference which is taking place in Sussex University the week after next (starting on Tuesday 3rd April), and for which a limited number of places are still available. Because of the uniqueness of the Conference, and its magnificent range of speakers, we are taking the unusual step of providing a last-minute "reminder" for anyone who may have either failed to see our previous notices, or forgotten to register in time. We are keen to provide a final opportunity for British academics and postgraduates who are interested in computers and their philosophical significance, since it is very unlikely that such an impressive list of speakers in this subject area will be assembled on this side of the Atlantic for a long time to come (see below). Yours sincerely, and with many thanks, Peter Millican ___________________________________________________________________________ INVITED GUEST SPEAKERS ANDREW HODGES, author of the much-acclaimed biography Alan Turing: the Enigma of Intelligence, will give the opening address at the Conference. DONALD MICHIE and ROBIN GANDY, both of whom knew Turing personally, will present the first and last major papers. Gandy is a prominent mathematical logician, while Michie is very well known in artificial intelligence circles, as well as being chief scientist at Glasgow's Turing Institute. The two other invited British speakers are CHRISTOPHER PEACOCKE, Waynflete Professor of Philosophy at Oxford, and J.R. LUCAS, who will be speaking on the topic of his famous and controversial paper "Minds, Machines and Godel" in front of an audience which will include some of his fiercest critics! One of these, DOUGLAS HOFSTADTER (Indiana), achieved fame with his Pulitzer Prize winning book Godel, Escher, Bach, which did much to provoke general interest in artificial intelligence. Other major American visitors include PAUL CHURCHLAND (California), perhaps the best known connectionist opponent of folk-psychology; JOSEPH FORD (Georgia), a prominent advocate of the new and exciting theory of chaos; CLARK GLYMOUR (Carnegie-Mellon), a notable philosopher of science, and last, but certainly not least, HERBERT SIMON (Carnegie-Mellon), one of the founding fathers of the science of artificial intelligence, and a Nobel laureate in 1978. OTHER CONTRIBUTORS Authors of the other 18 contributions include many well-known computer scientists, artificial intelligence researchers, and philosophers from America, Australia and Europe as well as from Britain. Their names, and the titles of their papers, are listed in the programme which follows. ___________________________________________________________________________ TURING 1990 - LATE REGISTRATION INFORMATION VENUE The Conference takes place at the University of Sussex, Falmer, which is about 4 miles from Brighton (the frequent trains take about 8 minutes, and the campus is barely 100 yards from Falmer station). Registration is from 11 a.m. until 2 p.m. on Tuesday 3rd April at NORWICH HOUSE, which is where most delegates will be accommodated. Those arriving late should ask the porter at Norwich House for registration materials unless they arrive after he has gone off duty, in which case registration materials, keys etc. can be collected from the permanent duty porter at the adjacent YORK HOUSE. FIRST AND LAST AFTERNOONS The Conference opens at 2 p.m. on Tuesday, with a lecture by Andrew Hodges in ARTS A2. This will be followed by coffee at 3.00, and a paper by Donald Michie (also in Arts A2) at 3.30. Dinner is from 5.00 until 6.30 in the Refectory, Level 2, with a wine reception in the Grapevine Bar (Refectory building) from 6.00 until 8.00, when Clark Glymour will speak in Arts A2. On Friday 6th April, Lunch is from 12.00 p.m. until 2.00, when Robin Gandy will give the closing speech. Coffee at 3.30 marks the official end of the Conference, although at 4.00 Douglas Hofstadter will give an additional open lecture entitled "Hiroshima Ma Mignonne". Dinner on Friday evening is available for those who require it (at a cost of #6.00). REGISTRATION AND ACCOMMODATION COSTS For members of the Mind Association or the Aristotelian Society, and also subscribers to Analysis or Philosophical Quarterly, the registration fee is only #30, thanks to the generous support which we are receiving from these bodies. The registration fee for students is likewise #30. For other academics the fee is #50, while for non-academics the fee is #80. Full board including bed, breakfast and all meals (with the exception of Thursday evening) from Dinner on Tuesday to Lunch on Friday, costs #84. For those wanting these meals alone (and not bed and breakfast), the cost is #33. On Thursday evening the Conference Banquet takes place at the Royal Pavilion in Brighton (for which we charge only the marginal cost of #25), but for those not attending the Banquet, dinner is available in the University at a cost of #6. Please note that places at the Banquet are strictly limited, and will be filled on a first come-first served basis. HOW TO REGISTER LATE Those who wish to book accommodation for the Conference should ring Judith Dennison at Sussex University (0273-678379) immediately, and if she is not available, should leave on her answerphone full details of their meal and accommodation requirements, together with A TELEPHONE NUMBER AT WHICH THEY CAN BE CONTACTED. Those who telephone by 2.00 p.m. ON FRIDAY 30th MARCH can probably be guaranteed accommodation within the University (though not necessarily in Norwich House), and you are asked to meet this deadline if at all possible (assuming that you are able to catch the Friday postal collection, please also send your cheque and written requirements, by first class mail, to the address below). During the following weekend Andy Clark (0273-722942) will be able to provide some information on the number of places remaining, and on Monday Judith Dennison will do her best to fit in those who have left their name in the meantime. Those who arrive on Tuesday without having booked do so, of course, at their own risk! CHEQUES AND WRITTEN REQUIREMENTS TO: Judith Dennison, School of Cognitive and Computing Sciences, University of Sussex, Brighton, BN1 9QN (please use first class post, and do not include cheques if posted after 30th March). PJRM/23rd March 1990 ____________________________________________________________________________ TURING 1990 COLLOQUIUM At the University of Sussex, Brighton, England 3rd - 6th April 1990 PROGRAMME OF SPEAKERS AND GENERAL INFORMATION ____________________________________________________________________________ INVITED SPEAKERS Paul CHURCHLAND (Philosophy, University of California at San Diego) FURTHER THOUGHTS ON LEARNING AND CONCEPTUAL CHANGE Joseph FORD (Physics, Georgia Institute of Technology) CHAOS : ITS PAST, ITS PRESENT, BUT MOSTLY ITS FUTURE Robin GANDY (Mathematical Institute, Oxford) HUMAN VERSUS MECHANICAL INTELLIGENCE Clark GLYMOUR (Philosophy, Carnegie-Mellon) COMPUTABILITY, CONCEPTUAL REVOLUTIONS AND THE LOGIC OF DISCOVERY Andrew HODGES (Oxford, author of "Alan Turing: the enigma of intelligence") BACK TO THE FUTURE : ALAN TURING IN 1950 Douglas HOFSTADTER (Computer Science, Indiana) MENTAL FLUIDITY AND CREATIVITY J.R. LUCAS (Merton College, Oxford) MINDS, MACHINES AND GODEL : A RETROSPECT Donald MICHIE (Turing Institute, Glasgow) MACHINE INTELLIGENCE - TURING AND AFTER Christopher PEACOCKE (Magdalen College, Oxford) PHILOSOPHICAL AND PSYCHOLOGICAL THEORIES OF CONCEPTS Herbert SIMON (Computer Science and Psychology, Carnegie-Mellon) MACHINE AS MIND ____________________________________________________________________________ OTHER SPEAKERS Most of the papers to be given at the Colloquium are interdisciplinary, and should hold considerable interest for those working in any area of Cognitive Science or related disciplines. However the papers below will be presented in paired parallel sessions, which have been arranged as far as possible to minimise clashes of subject area, so that those who have predominantly formal interests, for example, will be able to attend all of the papers which are most relevant to their work, and a similar point applies for those with mainly philosophical, psychological, or purely computational interests. Jonathan Cohen (The Queen's College, Oxford) "Does Belief Exist?" Mario Compiani (ENIDATA, Bologna, Italy) "Remarks on the Paradigms of Connectionism" Martin Davies (Philosophy, Birkbeck College, London) "Facing up to Eliminativism" Chris Fields (Computing Research Laboratory, New Mexico) "Measurement and Computational Description" Robert French (Center for Research on Concepts and Cognition, Indiana) "Subcognition and the Limits of the Turing Test" Beatrice de Gelder (Psychology and Philosophy, Tilburg, Netherlands) "Cognitive Science is Philosophy of Science Writ Small" Peter Mott (Computer Studies and Philosophy, Leeds) "A Grammar Based Approach to Commonsense Reasoning" Aaron Sloman (Cognitive and Computing Sciences, Sussex) "Beyond Turing Equivalence" Antony Galton (Computer Science, Exeter) "The Church-Turing Thesis: its Nature and Status" Ajit Narayanan (Computer Science, Exeter) "The Intentional Stance and the Imitation Game" Jon Oberlander and Peter Dayan (Centre for Cognitive Science, Edinburgh) "Altered States and Virtual Beliefs" Philip Pettit and Frank Jackson (Social Sciences Research, ANU, Canberra) "Causation in the Philosophy of Mind" Ian Pratt (Computer Science, Manchester) "Encoding Psychological Knowledge" Joop Schopman and Aziz Shawky (Philosophy, Utrecht, Netherlands) "Remarks on the Impact of Connectionism on our Thinking about Concepts" Murray Shanahan (Computing, Imperial College London) "Folk Psychology and Naive Physics" Iain Stewart (Computing Laboratory, Newcastle) "The Demise of the Turing Machine in Complexity Theory" Chris Thornton (Artificial Intelligence, Edinburgh) "Why Concept Learning is a Good Idea" Blay Whitby (Cognitive and Computing Sciences, Sussex) "The Turing Test: AI's Biggest Blind Alley?" ____________________________________________________________________________ TURING 1990 COLLOQUIUM At the University of Sussex, Brighton, England 3rd - 6th April 1990 This Conference commemorates the 40th anniversary of the publication in Mind of Alan Turing's influential paper "Computing Machinery and Intelligence". It is hosted by the School of Cognitive and Computing Sciences at the University of Sussex and held under the auspices of the Mind Association. Additional support has been received from the Analysis Committee, the Aristotelian Society, The British Logic Colloquium, The International Union of History and Philosophy of Science, POPLOG, Philosophical Quarterly, and the SERC Logic for IT Initiative. The aim of the Conference is to draw together people working in Philosophy, Logic, Computer Science, Artificial Intelligence, Cognitive Science and related fields, in order to celebrate the intellectual and technological developments which owe so much to Turing's seminal thought. Papers will be presented on the following themes: Alan Turing and the emergence of Artificial Intelligence, Logic and the Theory of Computation, The Church- Turing Thesis, The Turing Test, Connectionism, Mind and Content, Philosophy and Methodology of Artificial Intelligence and Cognitive Science. Invited talks will be given by Paul Churchland, Joseph Ford, Robin Gandy, Clark Glymour, Andrew Hodges, Douglas Hofstadter, J.R. Lucas, Donald Michie, Christopher Peacocke and Herbert Simon, and there are many other prominent contributors, whose names and papers are listed above. The conference will start after lunch on Tuesday 3rd April 1990, and it will end on Friday 6th April after tea. ANYONE WISHING TO REGISTER FOR THIS CONFERENCE SHOULD SEE THE LATE REGISTRATION INFORMATION ABOVE. Conference Organizing Committee Andy Clark (Cognitive and Computing Sciences, Sussex University) David Holdcroft (Philosophy, Leeds University) Peter Millican (Computer Studies and Philosophy, Leeds University) Steve Torrance (Information Systems, Middlesex Polytechnic) ___________________________________________________________________________ PLEASE SEND ON THIS NOTICE to any researchers, lecturers or students in the fields of Artificial Intelligence, Cognitive Science, Computer Science, Logic, Mathematics, Philosophy or Psychology, in Britain or abroad, and to ANY APPROPRIATE BULLETIN BOARDS which have not previously displayed it. From dmcneal at note.nsf.gov Mon Mar 26 14:34:23 1990 From: dmcneal at note.nsf.gov (Douglas McNeal) Date: Mon, 26 Mar 90 14:34:23 EST Subject: NACSIS: NSF offers access to Japanese data bases Message-ID: <9003261434.aa24834@Note.nsf.GOV> NSF now offers U.S. scientists and engineers free on-line access to nine Japanese science data bases. The data bases, which are compiled and updated by Japan's National Center for Science Information System (NACSIS), index such topics as research projects sponsored by Japan's Ministry of Education, Science, and Culture; papers presented at conferences of electronics societies; and all doctoral theses. U.S. researchers may request searches by surface mail to Room 416-A, NSF, Washington, D.C. 20550, or by electronic mail. Researchers may also contact the operator to schedule training at the NSF offices in using the Japanese-language system in person. For further information, request NSF publication 90-33, NACSIS, from NSF's Publications Unit. To request a search, to reserve time on the system, or to discuss research support opportunities, please call the NACSIS operator at (202) 357-7278 between the hours of 1 and 4 p.m., EST, or send a message by electronic mail to nacsis at nsf.gov (Internet) or nacsis at NSF (BitNet) ------- End of Forwarded Message From pablo at iai.es Thu Mar 1 09:16:00 1990 From: pablo at iai.es (Pablo Bustos) Date: 1 Mar 90 15:16 +0100 Subject: linear separability Message-ID: <223*pablo@iai.es> Does anyone know a test that could decide if a given subset of the vertex of a binary hypercube is linearly separable from the rest of the set. We are looking for criteria in the sense of Hamming distance, connectivity , etc. instead of an iterative algorithm (perceptrons already do) Thanks From mclennan%MACLENNAN.CS.UTK.EDU at cs.utk.edu Thu Mar 1 15:18:22 1990 From: mclennan%MACLENNAN.CS.UTK.EDU at cs.utk.edu (mclennan%MACLENNAN.CS.UTK.EDU@cs.utk.edu) Date: Thu, 1 Mar 90 16:18:22 EDT Subject: tech report available Message-ID: <9003012118.AA03926@MACLENNAN.CS.UTK.EDU> ********** DO NOT DISTRIBUTE TO OTHER LISTS ********** The following technical report is available: Field Computation: A Theoretical Framework for Massively Parallel Analog Computation Parts I - IV Bruce MacLennan Department of Computer Science University of Tennessee CS-90-100 ABSTRACT This report comprises the first four parts of a systematic presentation of _field_computation_, a theoretical framework for understanding and designing massively parallel analog computers. This theory treats computation as the continuous transformation of fields: continuous ensembles of continuous-valued data. This theory is technology-independent in that it can be realized through optical and molecular processes, as well as through large neural networks. Part I is an overview of the goals and assumptions of field computation. Part II presents relevant ideas and results from functional analysis, including theorems concerning the field- computation of linear and multilinear operators. Part III is devoted to examples of the field computation of a number of use- ful linear and multilinear operators, including integrals, derivatives, Fourier transforms, convolutions and correlations. Part IV discusses the field computation of nonlinear operators, including: a theoretical basis for universal (general purpose) field computers, ways of replacing field polynomials by sigmoid transformations, and ways of avoiding higher-dimensional fields (since they may be difficult to represent in physical media). ------------------------------------------------------------------------ The report is available in PostScript form by anonymous ftp as follows: unix> ftp cheops.cis.ohio-state.edu (or, ftp 128.146.8.62) Name: anonymous Password: neuron ftp> cd pub/neuroprose ftp> get maclennan.fieldcomp.ps.Z ftp> quit unix> uncompress maclennan.fieldcomp.ps.Z unix> lpr maclennan.fieldcomp.ps (or however you print postscript files) ------------------------------------------------------------------------ Hardcopy will soon be available from: library at cs.utk.edu For all other correspondence: Bruce MacLennan Department of Computer Science 107 Ayres Hall The University of Tennessee Knoxville, TN 37996-1301 (615)974-0994/5067 maclennan at cs.utk.edu From slehar at bucasb.bu.edu Thu Mar 1 13:49:14 1990 From: slehar at bucasb.bu.edu (slehar@bucasb.bu.edu) Date: Thu, 1 Mar 90 13:49:14 EST Subject: Mathematical Tractability of Neural Nets In-Reply-To: "Helen M. Gigley"'s message of Wed, 28 Feb 90 14:55:41 EST <9002281455.aa02466@Note.NSF.GOV> Message-ID: <9003011849.AA01302@bucasd.bu.edu> AAAAH! Now I understand the source of the confusion! Your statement... "It is the subsequent analysis of function corresponding to this **linguistic theory** which underlies the development of the neural analysis of the brain areas at what you consider the **functional level**." (**my emphasis**) reveals that you and I are refering to altogether different types of neural models. You are doubtless refering to the connectionist variants of the Chomsky type linguistic models which represent language in abstract and rigidly functional and hierarchical terms. If you think that such models are excessively rigid and abstract, then you and I are in complete agreement. The neural models to which I refer are more in the Grossberg school of thought. Such models are characterized by a firm founding in quantitative neurological analysis, expression in dynamic systems terms, and are confirmed by psychophysical and behavioral data. In other words these models adhere closely to known biological and behavioral knowledge. For instance the Grossberg neural model for vision [3],[4],[5] (which is more my area of expertise) is built of dynamic neurons defined by differential equations derived from the Hodgkin Huxley equations (from measurement of the squid giant axon) and also from behavioral data [1]. The topology and functionality of the model is based again on neurological observation (such as Hubel & Wiesel intracellular measurement in the visual cortex) together with psychophysical evidence, particularly visual illusions such as perceptual grouping [10], color perception [6], neon color spreading [7],[8], image stabilization experiments [9] and others. The model duplicates and explains such illusory phenomena as well as elucidating aspects of natural vision processing. In their book VISUAL PERCEPTION, THE NEUROPHYSIOLOGICAL FOUNDATIONS (1990), Spillmann & Werner say "Neural models for cortical Boundary Contour System and Feature Contour System interactions have begun to be able to account for and predict a far reaching set of interdisciplinary data as manifestations of basic design principles, notably how the cortex achieves a resolution of uncertainties through its parallel and hierarchical interactions" The point is that this class of models is not based on arbitrary philosophising about abstract concepts, but rather on hard physical and behavioral data, and Grossbergs models have on numerous occasions made behavioral and anatomical predictions which were subsequently confirmed by experiment and histology. Such models therefore cannot be challenged on purely philosophical grounds, but simply on whether they predict the behavioral data, and whether they are neurologically plausible. In this sense, the models are scientifically testable, since they make concrete predictions of how the brain actually processes information, not vague speculations on how it might do so. So, I maintain my original conjecture that the time is ripe for a fusion of knowledge from the diverse fields of neurology, psychology, mathematics and artificial intelligence, and I maintain further that such a fusion is already taking place. REFERENCES ========== Not all of these are pertinant to the discussion at hand, (they were copied from another work) but I leave them in to give you a starting point for further research if you are interested. [1] Stephen Grossberg THE QUANTIZED GEOMETRY OF VISUAL SPACE The Behavioral and Brain Sciences 6, 625 657 (1983) Cambridge University Press. Section 21 Reflectance Processing, Weber Law Modulation, and Adaptation Level in Feedforward Shunting Competitive Networks. In this section Grossberg examines the dynamics of a feedforward on-center off-surround network of shunting neurons and shows how such a topology performs a normalization of the signal, i.e. a factorization of pattern and energy, preserving the pattern and discarding the overall illumination energy. Reprinted in THE ADAPTIVE BRAIN Stephen Grossberg Editor, North-Holland (1987) Chapter 1 Part II section 21 [2] Gail Carpenter & Stephen Grossberg A MASSIVELY PARALLEL ARCHITECTURE FOR A SELF-ORGANIZING NEURAL PATTERN RECOGNITION MACHINE Computer Vision, Graphics, and Image Processing (1987), 37, 54-115 Academic Press, Inc. This is a neural network model of an adaptive pattern classifier (Adaptive Resonance Theory, ART 1) composed of dynamic shunting neurons with interesting properties of stable category formation while maintaining plasticity to new pattern types. This is achieved through the use of resonant feedback between a data level layer and a feature level layer. The original ART1 model has been upgraded by ART2, which handles graded instead of binary patterns, and recently ART3 which uses a more elegant and physiologically plausible neural mechanism while extending the functionality to account for more data. Reprinted in NEURAL NETWORKS AND NATURAL INTELLIGENCE, Stephen Grossberg Editor, MIT Press (1988) Chapter 6. [3] Stephen Grossberg & Ennio Mingolla NEURAL DYNAMICS OF PERCEPTUAL GROUPING: TEXTURES, BOUNDARIES AND EMERGENT SEGMENTATIONS Perception & Psychophysics (1985), 38 (2), 141-171. This work presents the BCS / FCS model with detailed psychophysical justification for the model components and computer simulation of the BCS. [4] Stephen Grossberg & Ennio Mingolla NEURAL DYNAMICS OF SURFACE PERCEPTION: BOUNDARY WEBS, ILLUMINANTS, AND SHAPE-FROM-SHADING. Computer Vision, Graphics and Image Processing (1987) 37, 116-165. This model extends the BCS to explore its response to gradients of illumination. It is mentioned here because of an elegant modification of the second competitive stage that was utilized in our simulations. [5] Stephen Grossberg & Dejan Todorovic NEURAL DYNAMICS OF 1-D AND 2-D BRIGHTNESS PERCEPTION Perception and Psychophysics (1988) 43, 241-277. A beautifully lucid summary of BCS / FCS modules with 1-D and 2-D computer simulations with excellent graphics reproducing several brightness perception illusions. This algorithm dispenses with boundary completion, but in return it simulates the FCS operation. Reprinted in NEURAL NETWORKS AND NATURAL INTELLIGENCE, Stephen Grossberg Editor, MIT Press (1988) Chapter 3. [6] Land, E. H. THE RETINEX THEORY OF COLOR VISION Scientific American (1977) 237, 108-128. A mathematical theory that predicts the human perception of color in Mondrian type images, based on intensity differences at boundaries between color patches. [7] Ejima, Y., Redies, C., Takahashi, S., & Akita, M. THE NEON COLOR EFFECT IN THE EHRENSTEIN PATTERN Vision Research (1984), 24, 1719-1726 [8] Redies, C., Spillmann, L., & Kunz, K. COLORED NEON FLANKS AND LINE GAP ENHANCEMENT Vision Research (1984) 24, 1301-1309 [9] Yarbus, A. L. EYE MOVEMENTS AND VISION New York: Plenum Press (1967) a startling demonstration of featural flow in human vision. [10] Beck, J. TEXTURAL SEGMENTATION, SECOND-ORDER STATISTICS, AND TEXTURAL ELEMENTS. Biological Cybernetics (1983) 48, 125-130 [11] Beck, J., Prazdny, K., & Rosenfeld, A. A THEORY OF TEXTURAL SEGMENTATION in J. Beck, B. Hope, & A. Rosenfeld (Eds.), HUMAN AND MACHINE VISION. New York: Academic Press (1983) [12] Stephen Grossberg SOME PHYSIOLOGICAL AND BIOCHEMICAL CONSEQUENCES OF PSYCHOLOGICAL POSTULATES Proceedings of the National Academy of Sciences (1968) 60, 758-765. Grossbergs original formulation of the dynamic shunting neuron as derived from psychological and neurobiological considerations and subjected to rigorous mathematical analysis. Reprinted in STUDIES OF MIND AND BRAIN Stephen Grossberg, D. Reidel Publishing (1982) Chapter 2. [13] David Marr VISION Freeman & Co. 1982. In a remarkably lucid and well illustrated book Marr presents a theory of vision which includes the Laplacian operator as the front-end feature extractor. In chapter 2 he shows how this operator can be closely approximated with a difference of Gaussians. [14] Daugman J. G., COMPLETE DISCRETE 2-D GABOR TRANSFORMS BY NEURAL NETWORKS FOR IMAGE ANALYSIS AND COMPRESSION I.E.E.E. Trans. Acoustics, Speech, and Signal Processing (1988) Vol. 36 (7), pp 1169-1179. Daugman presents the Gabor filter, the product of an exponential and a trigonometric term, for extracting local spatial frequency information from images; he shows how such filters are similar to receptive fields mapped in the visual cortex, and illustrates their use in feature extraction and image compression. [15] Stephen Grossberg CONTOUR ENHANCEMENT, SHORT TERM MEMORY, AND CONSTANCIES IN REVERBERATING NEURAL NETWORKS Studies in Applied Mathematics (1973) LII, 213-257. Grossberg analyzes the dynamic behavior of a recurrent competitive field of shunting neurons, i.e. a layer wherein the neurons are interconnected with inhibitory synapses and receive excitatory feedback from themselves, as a mechanism for stable short term memory storage. He finds that the synaptic feedback function is critical in determining the dynamics of the system, a faster than linear function such as f(x) = x*x results in a winner-take-all choice, such that only the maximally active node survives and suppresses the others in the layer. A sigmoidal function can be tailored to produce either contrast enhancement or winner-take-all, or any variation in between. From sg at corwin.ccs.northeastern.edu Thu Mar 1 14:05:58 1990 From: sg at corwin.ccs.northeastern.edu (steve gallant) Date: Thu, 1 Mar 90 14:05:58 EST Subject: linear separability Message-ID: <9003011905.AA10169@corwin.CCS.Northeastern.EDU> It's not easy to tell whether a set of vertices is separable or not even with perceptron learning, because you don't know whether the set is nonseparable or whether you just haven't run enough iterations. One approach is to cycle through the training examples and keep track of the weights on the output cell. Either perceptron learning will find a solution (separable case) or a set of weights will reappear (nonseparable case). Another method is the Ho-Kashyap procedure (see Duda & Hart), but there's still no good bound on how much work is required to determine separability. Steve Gallant From yann at lesun.att.com Thu Mar 1 13:25:07 1990 From: yann at lesun.att.com (Yann le Cun) Date: Thu, 01 Mar 90 13:25:07 -0500 Subject: linear separability In-Reply-To: Your message of 01 Mar 90 15:16:00 +0100. Message-ID: <9003011825.AA24715@lesun.> > Does anyone know a test that could decide if a given subset of the vertex of > a binary hypercube is linearly separable from the rest of the set. > ... (perceptrons already do) Perceptrons tell you if two sets are LS, they don't tell you anything if they are not LS: they just keep ocsillating. The Ho-Kashyap procedure (IEEE Trans. Elec. Comp. EC14 october 1965) tells you if two sets are linearly separable, and gives you a solution if they are. It is described in the book by Duda and Hart "pattern classification and scene analysis" Wiley and Son, (1973). - Yann Le Cun, Bell Labs Holmdel. From sayegh at ed.ecn.purdue.edu Thu Mar 1 18:47:15 1990 From: sayegh at ed.ecn.purdue.edu (Samir Sayegh) Date: Thu, 1 Mar 90 18:47:15 -0500 Subject: NN conference Indiana_Purdue Ft Wayne Message-ID: <9003012347.AA00854@ed.ecn.purdue.edu> Third Conference on Neural Networks and Parallel Distributed Processing Indiana-Purdue University A conference on NN and PDP will be held April 12, 13 and 14, 1990 on th common campus of Indiana and Purdue University at Ft Wayne. The emphasis of this conference will be Vision and Robotics although all contributions are w are welcome. People from the Midwest are particularly encouraged to attend and contribute especially since the "major" NN conferences seem to oscillate between the East and West Coast! Send abstracts and inquiries to: Dr. Samir Sayegh Physics Department Indiana Purdue University Ft Wayne, IN 46805 email: sayegh at ed.ecn.purdue.edu sayegh at ipfwcvax.bitnet FAX : (219) 481-6800 Voice: (219) 481-6157 From bates at amos.ucsd.edu Thu Mar 1 21:04:06 1990 From: bates at amos.ucsd.edu (Elizabeth Bates) Date: Thu, 1 Mar 90 18:04:06 PST Subject: Mathematical Tractability of Neural Nets Message-ID: <9003020204.AA10645@amos.ucsd.edu> Although I am grateful for the references -- and very aware that there are alternative approaches and multiple points of view in the world of neural networks -- I must reiterate: my comments to you were not addressed to neural networking per se (my qualifications are relatively limited in that regard) but to the claims you were making on the network regarding what is supposedly "known" about brain organization for language. I can only hope that the references you cite are not trying to fit their models to a false reality (i.e. Broca's area = grammar, and so on). -liz bates From Dave.Touretzky at B.GP.CS.CMU.EDU Fri Mar 2 02:26:21 1990 From: Dave.Touretzky at B.GP.CS.CMU.EDU (Dave.Touretzky@B.GP.CS.CMU.EDU) Date: Fri, 02 Mar 90 02:26:21 EST Subject: language, vision, and connectionism Message-ID: <6501.636362781@DST.BOLTZ.CS.CMU.EDU> Steve Lehar's argument ignores a crucial point: low-level vision is a specialized area with properties that make it more tractable for computational modeling than most other aspects of intelligence. First, low-level vision is concerned with essentially geometric phenomena, like edge detection. Geometric phenomena are especially easy to work with because they involve mostly local computations, and the representations are (relatively!) intuitive and straightforward. Basically they are iconic representations on retinotopic maps. Compare this with high level vision tasks (e.g., 3D object recognition), or language tasks, where the brain's representations are completely unknown, probably not iconic, and almost certainly not intuitive or straightforward. A second aspect of low-level vision is that it involves only the earliest stages of the visual system, which are (relatively!) easily explored in a neuroscience lab. It's not that hard now, after several decades of practice, to make autoradiographs of ocular dominance patterns in LGN, for example, or to work out the mapping from the retina to the surface of area 17. It's much harder to try to explore the higher level visual areas where objects are recognized, in part because the circuitry gets too complex as you go deeper into the brain, and in part because, since the highest areas don't use simple geometric representations, techniques like autoradiography or receptive field mapping are not applicable. As Jim Bower pointed out, even in the earliest stages of vision there are lots of unanswered basic questions, such as why the feedback connections from visual cortex to LGN vastly outnumber the feedforward connections. Also, the wiring of the different layers of cells in primary visual cortex (only a few synapses in from the retina) is not fully known. If we know so little about vision after decades of studies on cats and monkeys, think about how much less we know about language. -- Dave From tmb at ai.mit.edu Fri Mar 2 02:29:32 1990 From: tmb at ai.mit.edu (Thomas M. Breuel) Date: Fri, 2 Mar 90 02:29:32 EST Subject: linear separability Message-ID: <9003020729.AA04172@rice-chex> |It's not easy to tell whether a set of vertices is separable |or not even with perceptron learning, because you don't know whether the |set is nonseparable or whether you just haven't run enough iterations. |One approach is to cycle through the training examples and keep track of |the weights on the output cell. Either perceptron learning will find a |solution (separable case) or a set of weights will reappear (nonseparable |case). Another method is the Ho-Kashyap procedure (see Duda & Hart), but |there's still no good bound on how much work is required to determine |separability. To get a bound on the amount of work required to determined linear separability, observe that linear separability is easily transformed to a linear programming problem. From AMR at IBM.COM Fri Mar 2 09:16:35 1990 From: AMR at IBM.COM (AMR@IBM.COM) Date: Fri, 2 Mar 90 09:16:35 EST Subject: Mathematical Tractability of Neural Nets Message-ID: I would like to commend the content though not the tone of the recent remarks by Bates and Bower. While all their criticisms are certainly right, I would nevertheless hope for more, not less, in the way of attempts (no matter how quixotic) to reach across the gap that separates the disciplines more than it does their respective subject matters. It may be that biologists and linguists (to take two extreme cases) will simply have to work each in their own garden for a century, say, before their results on brain activity begin to converge, and that current attempts at a linkage of the kind that some connectionist literature seems to envisage between neural architecture and cognitive behavior are doomed to failure. On the other hand, it seems to me that the history of science is so full of cases where progress was delayed because people were not ready to see connections between initially distinct areas of study that the only lesson to draw is that if somebody is willing to invest the time and the effort to pursue the connections, let them. I myself, as primarily a linguist, find myself in the following predicament. On the one hand, no one has shown me how linguistic behavior can arise out of neural nets, but, on the other hand, no one has shown me how it can arise out of formal grammars or Turing machines, either. I am very uncomfortable with the idea that, given the obvious primitiveness of current neural models, we should accept, as our idea of what makes language possible, anything like a grammar in the conventional sense. Linguists of an earlier age were very careful (and most non-theoretically inclined linguists still are) in viewing grammars as descriptions of the data, not as models of anything that goes on inside human beings. Unfortunately, the direction of theoretical linguistics has been largely one where the wonderful idea that we SHOULD find out what goes on inside has been confused with the deplorable conceit that we CAN find this out by doing little more than armchair grammar writing but ATTRIBUTING to these grammars what has come to be known as psychological reality. A few of us are trying, in linguistics, to right the balance and to develop an alternative in which the need to have grammars as human-readable descriptions of masses of data does not lead to the assumption that these same grammars are at all useful as models of human cognition or neurology. It may well be that we will find that very few linguistic facts can be explained, given current knowledge, by reference to something more basic, and that the bulk of the routine grammatical information that we know perfectly how to DESCRIBE will continue to elude a psychological (much less a neurological) account for a long time. But it is a case, I submit, of having a choice of a very few pearls or a whole lot of swine. And in the search for the former I am willing to get help anywhere I can find it. Alexis Manaster-Ramer IBM Research POB 704 Yorktown Heights, NY 10510 (914) 784-7239 From AMR at IBM.COM Fri Mar 2 09:53:44 1990 From: AMR at IBM.COM (AMR@IBM.COM) Date: Fri, 2 Mar 90 09:53:44 EST Subject: Generative Power of Neural Nets Message-ID: Some time ago I posed the question of the weak generative power of neural nets, and in particular, their equivalence or otherwise to Turing machines. This elicited a number of responses, and I am by no means done reading the entire literature that has been kindly sent to me. I would like to offer an update, however. (1) There seems to be no consensus on whether the issue is significant, although I feel (as I have said in this forum, without getting any response) that the arguments for ignoring such issues (at least the ones I saw) were based on fundamental misunderstandings of the mathematics and/or its applications. (2) There seems to be a strong feeling among those who like the neural nets that, regardless of what their generative power is, they are interestingly different from, say, Turing machines, but as far as I have been able to determine, no one has any concrete proposals as how such a notion of difference (or a corresponding notion of equivalence) are to be defined. Instead, it would appear that some of the misguided arguments against being concerned with formal issues (mentioned in (1) above) may arise out of a anything-but-misguided intuition that difference or equivalence in terms of weak generative power is not the relevant criterion combined with the indeed-quite-misguided notion that formal theories of computation and the like cannot characterize any other such criterion. (3) There ARE a number of results (or ideas that could easily be turned into results) about the weak generative power, but they conflict at first glance quite a bit with each other. Thus, on some views, neural nets are equivalent to finite state machines, on others to Turing machines, and yet on others they are MORE powerful than Turing machines. The last position sounds at first glance the most intriguing. However, the results that point in this direction are based simply on assuming infinite nets which are essentially (unless I am very wrong) like what you would get if you took the idea of a finite-state machine (or the finite control of a Turing machine, which is the same thing, essentially) and allowed an infinite number of states. In that case, you could easily get all sorts of non-Turing-computable things to "compute", but most would I think dispute that this adds anything to our original idea that "computable" should mean "Turing-computable", since crucially you would need to allow infinite-length computations. On the hand, the arguments that reduce neural nets to finite-state machines are based on the simple idea that actual nets must be strictly finite (without even the infinite tape which we allow Turing machines to have). As has been pointed out, the same applies to any physically realized computer such as the IBM AT I am using to type these words. A Turing machine (or even a PDA) cannot be physically realized, only a finite-state machine can. However, both of these ways of viewing the situation seem to me to be fruitless. The Turing model has been so succesful because, while literally quite false of the computers we build (because of the infinite tape), it has proved useful in understanding what real computation does, anyway. The point is difficult to make briefly, but it boils down to the observation that the finite- state machine model does not distinguish a random finite list of strings, for example, from something like all strings of the form a^n b^n for n up to 20,000. The standard Turing model does do this, by allowing us to pretend in the second case that we are really recognizing the infinite language a^n b^n for all n. As a number of people have pointed out, this is really cheating but it does not appear to do any harm, at least among those who understand the rules of the game. Thus, it seems to me that the important results would be along the lines of showing that various types of neural nets are equivalent to Turing machines, or that, if they are not, then the distinction is NOT simply due either to assuming strict finiteness (i.e. no way to simulate the TM tape) or else by assuming strict infiniteness (i.e. an infinite control). It is by no means clear to me that we have a convincing answer in this case, since it seems that some models that have been defined are NOT (or at least not obviously) as powerful as TMs. I would welcome additional clarification about this point. (4) I personally feel that the next step would have to be to develop a different way of measuring equivalence (as I have been hinting all along), since this seems to me to be the intuition that almost everybody has, but I have seen no ideas directed to this. Some of my colleagues and I have been struggling with doing something like this for a different domain (namely, the comparison of different linguistic formalisms), but progress has been difficult. I suspect that neural nets would be an ideal testing ground for any proposals along these lines, and I would be grateful for any ideas. Just to make this a little more concrete, let me give an example: (5) It is clear that a Turing machine, a Von Neumann machine, and a neural net are not the same thing, yet they may have the same weak generative power. The question is whether there is any way to develop a useful mathematical theory in which the first two are in some sense equivalent to each other but neither is to the third. It seems to me that this is the intuition of many people, including by definition (I would think) all connectionists, but there is no basis currently for deciding whether those who feel so are right. Alexis Manaster-Ramer IBM Research POB 704 Yorktown Heights, NY 10598 From sg at corwin.ccs.northeastern.edu Fri Mar 2 12:40:14 1990 From: sg at corwin.ccs.northeastern.edu (steve gallant) Date: Fri, 2 Mar 90 12:40:14 EST Subject: linear separability Message-ID: <9003021740.AA02286@corwin.CCS.Northeastern.EDU> > To get a bound on the amount of work required to determined linear > separability, observe that linear separability is easily transformed > to a linear programming problem. Good suggestion. The transformation is (presumably) to an LP problem that has a minimum solution of 0 iff the original data is separable. This certainly would give an exponential bound based upon the simplex method. I wonder whether polynomial methods would give a polynomial bound here or not. The potential problem is getting a series of approximations that converge toward 0, but not being able to tell whether the solution is EXACTLY 0. Perhaps you or someone else familiar with polynomial methods could comment on this? Steve Gallant From well!mitsu at apple.com Fri Mar 2 15:30:55 1990 From: well!mitsu at apple.com (Mitsuharu Hadeishi) Date: Fri, 2 Mar 90 12:30:55 pst Subject: Mathematical Tractability of Neural Nets Message-ID: <9003022030.AA20061@well.sf.ca.us> Just as a side note to those who have posted what appear to be personal replies to the whole Connectionists list, to remind you that to send a reply only to the originator of the message (and not to everyone who received it as well), use capital R instead of small r when replying. This is not a complaint, but just a reminder in case people are accidentally sending messages intended to be personal to the entire list. From Scott.Fahlman at B.GP.CS.CMU.EDU Sat Mar 3 09:36:16 1990 From: Scott.Fahlman at B.GP.CS.CMU.EDU (Scott.Fahlman@B.GP.CS.CMU.EDU) Date: Sat, 03 Mar 90 09:36:16 EST Subject: Mathematical Tractability of Neural Nets In-Reply-To: Your message of Fri, 02 Mar 90 12:30:55 -0800. <9003022030.AA20061@well.sf.ca.us> Message-ID: Just as a side note to those who have posted what appear to be personal replies to the whole Connectionists list, to remind you that to send a reply only to the originator of the message (and not to everyone who received it as well), use capital R instead of small r when replying. The proper way to reply will vary from one mail program to another, and there are dozens of different mailers in common use. Please learn to use your own mailer properly, whatever it may be. -- Scott From bates at amos.ucsd.edu Sat Mar 3 15:53:12 1990 From: bates at amos.ucsd.edu (Elizabeth Bates) Date: Sat, 3 Mar 90 12:53:12 PST Subject: Mathematical Tractability of Neural Nets Message-ID: <9003032053.AA05721@amos.ucsd.edu> AMR's point about the need for collaboration is well taken -- and as a scientist who is virtually obsessed with collaboration (e.g. cross-linguistic projects over three continents that we've somehow managed to keep afloat for 15 years) I would be the last to suggest that we work in our own gardens for a few more decades. Indeed, I think we are in the middle of a particularly promising time for an interaction between neuroscience and the various subfields of language research. A few concrete examples: The WAshington University work on brain metabolism during lexical processing, exciting new psycholinguistic research using electrophysiological indices (a 6-dimensional outcome measure that puts old-fashioned button-press techniques to shame) by Kutas, van Petten, Neville, Holcomb, Garnsey and others, new "on-line" studies of aphasia that are telling us a great deal about real-time processing in aphasic patients using techniques that were not possible 10 - 15 years ago, developmental studies of infants with focal brain injury that are looking PROSPECTIVELY at the process of recovery, for the very first time -- and this is just a small sample. The technical advances are great, and the opportunities are even greater. I also believe that connectionism will offer new theoretical and experimental tools for examining language breakdown in aphasia -- such as the Hinton/Shallice or McClelland/Seidenberg/Patterson collaborations that I cited earlier. In short (and I have gone on too long), my point was really a simple one: the old view of brain organization for language appears to have been disconfirmed, quite roundly, and the field of aphasiology is currently seeking a completely new way of characterizing contrasting forms of language impairment following focal brain injury. I was answering Slehar's proposal that we "follow the neurologists" and accept the old story (e.g. Broca's area = the grammar center, and so on). But I would NEVER want or mean to suggest that we give up!! Of course language is difficult to study (compared, as Touretzky points out, with low level vision), but it also has its advantages: (1) it is the only public form of cognition, out there for all of us to say, and (2) for that reason language is perhaps the best understood and most easily measured of all higher cognitive processes. We do indeed live in interesting times, and I am sure we have some real breakthroughs ahead of us in a cognitive neuroscience of language.. -liz From munnari!cluster.cs.su.oz.au!ray at uunet.uu.net Sat Mar 3 02:41:40 1990 From: munnari!cluster.cs.su.oz.au!ray at uunet.uu.net (munnari!cluster.cs.su.oz.au!ray@uunet.uu.net) Date: Sat, 3 Mar 90 18:41:40 +1100 Subject: No subject Message-ID: <9003030743.6898@munnari.oz.au> >From ml-connectionists-request%q.cs.cmu.edu at murtoa.cs.mu.oz Sat Mar 3 14:40:20 1990 From tmb%ai.mit.edu at murtoa.cs.mu.oz Fri Mar 2 02:29:32 1990 From: tmb%ai.mit.edu at murtoa.cs.mu.oz (Thomas M. Breuel) Date: Fri, 2 Mar 90 02:29:32 EST Subject: linear separability Message-ID: <9003020729.AA04172@rice-chex> |It's not easy to tell whether a set of vertices is separable |or not even with perceptron learning, because you don't know whether the |set is nonseparable or whether you just haven't run enough iterations. |One approach is to cycle through the training examples and keep track of |the weights on the output cell. Either perceptron learning will find a |solution (separable case) or a set of weights will reappear (nonseparable |case). Another method is the Ho-Kashyap procedure (see Duda & Hart), but |there's still no good bound on how much work is required to determine |separability. To get a bound on the amount of work required to determined linear separability, observe that linear separability is easily transformed to a linear programming problem. From ato at breeze.bellcore.com Sun Mar 4 13:53:37 1990 From: ato at breeze.bellcore.com (Andrew Ogielski) Date: Sun, 4 Mar 90 13:53:37 -0500 Subject: Linear separability Message-ID: <9003041853.AA19720@breeze.bellcore.com> The question of algorithmic complexity of determinining linear separability of two sets of vectors (with real or binary components, it does not matter) is an old story, and the answer is well known: As was mentioned in a previous posting (and was widely known among early threshold gate enthusiasts in the 1960's) linear separability is easily transformed into a linear programming problem. For those who don't see it right away - just write down linear inequalities for the coordinates of a normal vector for a separating hyperplane ("weights" in the nn jargon). Linear programs can be solved in polynomial time (polynomial in the problem size, i.e. number of variables, number of inequalities, and the number of bits, when needed) unless one restricts the solution to integers. The first proof has been achieved by Khachiyan in his ellipsoid method. A more recent provably polynomial algorithm is due to Karmarkar. As an aside: despite the fact that the simplex method is not polynomial (there exist instances of linear programs where simplex methods would take exponential number of iterations in the program size), it works extremely well in majority of cases, including separability of binary vectors. The latter special case usually is not very sparse , which does not allow to benefit fully from interior point methods such as the Karmarkar's algorithm. A good, recent reference is A. Schrijver, Theory of Linear and Integer Programming, John Wiley & Sons, Chichester 1987. Andy Ogielski P.S. Perhaps it is proper to mention here that relaxation methods for solving systems of linear inequalities ( the "perceptron algorithm" is in this category) have been known and thoroughly explored by mathematicians well before the perceptrons. If I remember correctly, this fact received an overdue acknowledgment in the new edition of the Minsky & Papert book. If anybody needs it, I can dig out proper references. ato From ray at cluster.cs.su.oz.au Sun Mar 4 22:34:40 1990 From: ray at cluster.cs.su.oz.au (ray@cluster.cs.su.oz.au) Date: Mon, 5 Mar 1990 14:34:40 +1100 Subject: No subject Message-ID: <9003050336.10396@munnari.oz.au> >From ml-connectionists-request%q.cs.cmu.edu at murtoa.cs.mu.oz Sat Mar 3 14:40:20 1990 From tmb%ai.mit.edu at murtoa.cs.mu.oz Fri Mar 2 02:29:32 1990 From: tmb%ai.mit.edu at murtoa.cs.mu.oz (Thomas M. Breuel) Date: Fri, 2 Mar 90 02:29:32 EST Subject: linear separability Message-ID: <9003020729.AA04172@rice-chex> |It's not easy to tell whether a set of vertices is separable |or not even with perceptron learning, because you don't know whether the |set is nonseparable or whether you just haven't run enough iterations. |One approach is to cycle through the training examples and keep track of |the weights on the output cell. Either perceptron learning will find a |solution (separable case) or a set of weights will reappear (nonseparable |case). Another method is the Ho-Kashyap procedure (see Duda & Hart), but |there's still no good bound on how much work is required to determine |separability. To get a bound on the amount of work required to determined linear separability, observe that linear separability is easily transformed to a linear programming problem. From om%icsib15.Berkeley.EDU at jade.berkeley.edu Mon Mar 5 15:46:19 1990 From: om%icsib15.Berkeley.EDU at jade.berkeley.edu (Stephen M. Omohundro) Date: Mon, 5 Mar 90 12:46:19 PST Subject: linear separability test. In-Reply-To: Tal Grossman's message of Mon, 05 Mar 90 17:56:17 +0200 <9003052005.AA05814@icsi> Message-ID: <9003052046.AA14381@icsib15.berkeley.edu.> A couple of simple transformations will convert the linear separability question from intersection of two convex hulls to just the containment of a point in a single convex hull. First realize that we can focus attention just on hyperplanes through the origin by embedding the original problem in the "z=1" plane one dimension higher. Each arbitrary hyperplane in the original problem then corresponds to a hyperplane through the origin in this bigger space. Consider the unit vector v which is perpendicular to such a plane. The plane is separating if the dot product of this vector with each point in set1 is negative and with each point in set2 is positive. If we reflect a point through the origin (ie. (x1,...,xn) -> (-x1,...,-xn)) then the sign of its dot product with any vector also changes. Thus if we do such reflection on each point in set1, we have reduced the problem to finding a vector whose dot product with each of a set of points is positive. The existence of such a vector is equivalent to the origin not being included in the convex hull of the points. --Stephen Omohundro From huyser at mithril.stanford.edu Mon Mar 5 18:30:15 1990 From: huyser at mithril.stanford.edu (Karen Huyser) Date: Mon, 5 Mar 90 15:30:15 PST Subject: linear separability Message-ID: <9003052330.AA02094@mithril.Stanford.EDU> There is one Hamming-type test I know of, and that's a test for summability. For a function of N variables to be linearly separable, it must be asummable. To be asummable, it must be 2-asummable, 3-asummable, ... , k-asummable, ... , N-asummable. Any function is k-asummable if it is not k-summable. 2-summability is described below. If a boolean function is 2-summable, there exists the following relationship between its vertices. 100 101 Imagine the faces (2-D subspaces) of a cube (hypercube). 0 ------ 1 Assign one-bit output values to each corner of the cube. | | (If multiple-bit output, make multiple hypercubes.) | | If any face of the hypercube has an assignment like that 1 ------ 0 to the left, the problem is not linearly separable. 000 001 This "exor" relation between pairs of vertices corresponds to the function being "2-summable". Specifically, if we add the vectors that label the 1-corners (000 + 101 = 101) and those that label the 0-corners (100 + 001 = 101), they add together to the same vector value (they sum in pairs, hence 2-summable). Similar relationships between three, four, etc. vertices corresponds to 3-summability, 4-summability, etc. Any function that is 2-summable is not linearly separable. A simple test of 2-summability is to form all possible sets of four training vectors and test for this exor condition. In a Hamming sense, each hypercube face will be composed of a corner, two of its nearest neighbors (one bit different), and the diagonal. (Note each face represents changes in only two bits.) The test is not likely to be efficient, but it will be local. So how can it be used for general boolean functions? Suppose you have an incomplete boolean function of N variables, and that the incomplete function can be divided into subsets each of which is a complete boolean function of Ki variables, where Ki < N. (It's okay if a vertex is in more than one subset, but each vertex must be in some subset.) If each such subset of vertices represents a subspace of fewer than nine dimensions (nine variables), and the subset is a complete boolean function of the variables, then the 2-summability test is also a test for linear separabilty. If the subset fails the test, it is not linearly separable. If it passes the test, it is linearly separable. This is because 2-asummability is the same as complete monotonicity, which has been shown to be a necessary and sufficient condition for linear separability for boolean functions of fewer than nine variables. (See Muroga, 1971) The 2-summability test works well only for completely specified functions. For incomplete boolean functions, (that is, when the subset of vertices to be tested is incomplete), this method will require filling in values for "don't care" vertices in an attempt to prevent the augmented subset from becoming summable. Again, summability tests are generally not thought of as efficient, whereas a linear programming approach will be. (1) Saburo Muroga, "Threshold Logic and its Applications", Wiley, NY, 1971 This text is full of wonderful theory about summability, monotonicity, etc. (2) John Hopcroft, "Synthesis of Threshold Logic Networks", Ph.D. thesis, Stanford Univ., 1964. Hopcroft discusses the relationship between convex hulls and linear separability in his thesis. He uses convex hulls to construct a network-building algorithm you might find useful. Karen Huyser Electrical Engineering Stanford University From kasif at crabcake.cs.jhu.edu Mon Mar 5 19:18:15 1990 From: kasif at crabcake.cs.jhu.edu (Simon Kasif ) Date: Mon, 5 Mar 90 19:18:15 -0500 Subject: linear separability test. Message-ID: Algorithms for convex hull construction in multiple dimensions are usually exponential in the number of dimensions. Thus, at least asymptotically one may prefer linear programming methods. Many interesting special cases to decide whether convex polytopes intersect and consequently determine linear separability (e.g., in 2 or 3 dimensions) are discussed in standard computational geometry books (e.g., Preparata and Shamos). From singer at Think.COM Tue Mar 6 16:00:13 1990 From: singer at Think.COM (singer@Think.COM) Date: Tue, 6 Mar 90 16:00:13 EST Subject: Neural networks on NCube machines Message-ID: <9003062100.AA11858@selli.think.com> I am looking for neural network implementations on NCube machines. If anyone knows of anyone doing such work, could he/she please contact me? Thanks, Alexander Singer Thinking Machines Corp. 617-876-1111 singer at think.com From gotsman%humus.huji.ac.il at VMA.CC.CMU.EDU Wed Mar 7 15:48:02 1990 From: gotsman%humus.huji.ac.il at VMA.CC.CMU.EDU (Chaim Gotsman) Date: Wed, 7 Mar 90 22:48:02 +0200 Subject: Linear separability Message-ID: <9003072048.AA23510@humus.huji.ac.il> As Andy Ogielski has pointed out, the ONLY polynomial (in the dimension) algorithms for solving the linear separability problem are the recent polynomial linear programming algorithms, specifically the Kachiyan ellipsoid method. The perceptron methods (termination set aside) or delta rule modifications can sometimes run in exponential time, as Minsky and Papert themselves confess. This has been recently "rediscovered" by Maass and Turan in "On the Complexity of Learning From Counterexamples" Proc. of IEEE Conf. on Foundations of Computer Science 1989, p.262-267 A word of caution is in order concerning this algorithm. It requires an "oracle" to supply counterexamples to false hypothesis' at each iteration, i.e. if the current guess of weights is incorrect, a point of the hypercube incorrectly classified has to be supplied. If the number of points (on the cube) supplied as input is polynomial, the oracle can be simulated polynomially by exhaustive search. This is probably the case, otherwise exponential time is required just to READ THE INPUT. An interesting offshoot of this question is whether it is possible to determine linear separability of an ENTIRE (on the whole cube) boolean function, when the input is a description of a CIRCUIT computing the function. The circuit description is obviously polynomial, otherwise the function is definitely NOT linearly separable (all linear threshold functions are in NC1). I'd be interested in knowing if anyone can answer this. Craig Gotsman Hebrew University gotsman at humus.huji.ac.il  From C.Foster at massey.ac.nz Thu Mar 8 10:32:25 1990 From: C.Foster at massey.ac.nz (C.Foster@massey.ac.nz) Date: Thu, 8 Mar 90 10:32:25 NZS Subject: formal levels Message-ID: <9003072232.AA21066@sis-a> With reference to the recent discussion of levels, there is a formal theory of levels. I have developed it as part of a larger (formal) theory of strong equivalence of complex systems. This is a PhD thesis in progress for Edinburgh University's Centre for Cognitive Science. It should be available *soon*. Let me know if you want a copy or more information. C. Foster CFoster at massey.ac.nz c/- School of Information Sciences Massey University Palmerston North New Zealand From kasif at crabcake.cs.jhu.edu Wed Mar 7 19:56:34 1990 From: kasif at crabcake.cs.jhu.edu (Simon Kasif ) Date: Wed, 7 Mar 90 19:56:34 -0500 Subject: Linear separability Message-ID: > > An interesting offshoot of this question is whether it is possible to determine linear separability of an ENTIRE (on the whole cube) boolean function, when the input is a description of a CIRCUIT computing the function. The circuit description is obviously polynomial, otherwise the function is definitely NOT linearly separable (all linear threshold functions are in NC1). I'd be interested in knowing if anyone can answer this. > > If I understood your question correctly, this seems to depend on the representation of the boolean (function) circuit. If the function F is given as a formula in CNF, then the question become co-NP. Simply observe that the formula F*G (F and G), where G represents some easy non-linearly separable function (e.g. XOR) over the same input, is a linear threshold function (all inputs map to 0) iff F is unsatisfiable. From haim at grumpy.sarnoff.com Thu Mar 8 11:12:17 1990 From: haim at grumpy.sarnoff.com (Haim Shvaytser x2085) Date: Thu, 8 Mar 90 11:12:17 EST Subject: Linear separability Message-ID: <9003081612.AA07761@vision.sarnoff.com> Two comments: 1- While it is true that the general problem of determining linear seperability is NP Complete (see previous messages), there are many results about interesting cases that are polynomial. For example, Chvatal and Hammer [1] give an O(mn^2) algorithm for determining whether a set of m linear inequalities in n variables is linearly seperable (i.e., the m inequalities can be replaced by a single inequality). Their nice algorithm is for the case in which the coefficients are from {0,1}. They also show that the same problem for general (integer) coefficients is NP Complete. (Notice that this implies that the question of whether a given neural net with one hidden layer can be simulated by a perceptron is NP Complete.) 2- Even though the worst case behavior of the perceptron algorithm is exponential, it was shown to be better than currently known algorithms of linear programming in a certain "average case" sense. See E. Baum's paper in the last NIPS conference. [1] V. Chvatal and P.L. Hammer, Aggregation of Inequalities in Integer Programming, Annals of Discrete Mathematics 1 (1977) 145-162. From grumbach at meduse.enst.fr Thu Mar 8 13:17:49 1990 From: grumbach at meduse.enst.fr (Alain Grumbach) Date: Thu, 8 Mar 90 19:17:49 +0100 Subject: organization levels Message-ID: <9003081817.AA02414@meduse.enst.fr> A couple of weeks ago, I sent this mail : % Being working on hybrid symbolic - connectionist systems, % I am wondering about the notion of "organization level", % which is underlying hybrid models. % A lot of people use this phrase, from neuroscience, to cognitive % psychology, via computer science, Artificial Intelligence : % (Anderson, Newell, Simon, Hofstadter,Marr, Changeux, etc). % But has anybody heard about a formal description of it ? % (formal but understandable !) Thank you very much ... Hendler, Olson, Manester-Ramer, Schrein, Cutrell, Bovet, Tgelder, Sejnowski, Sanger, Sloman, Rohwer ... for your answers. I shall try here a summary of these answers and put forward a framework of a formal point of view. 1. Answers : 1.1 Points of view : A lot of phrases are used to qualify level hierarchy, denoting quite different points of view : organization levels description levels abstraction levels analysis levels integration levels activity levels explanation levels To name each level, a lot of words are used, which, of course, should be associated with some of hierarchy phrases : computational, algorithmic, implementational sensors, synapses, neurons, areas task, implementation, virtual machine Several hierarchy structures are mentioned : linear tree oriented graph without circuits oriented graph with circuits (from processing point of view only) 1.2. References : Many references are given : Steels, Fodor, Fox, Honavar & Uhr, Jacob, Churchland & Sejnowski, Bennett & Hoffman & Prakash the last one : Bennett & Hoffman & Prakash, being a formal theory dealing with levels. Unfortunately I have not read it yet, as it concerns the ninth chapter of a book; R. Rohwer will write a more direct condensed version of it. 2. Sketch of an organization level description : 2.1 Intuitive issues : 2.1.1 Who ? First it must be emphasized (or remembered) that an organization level hierarchy consists in a point of view of an OBSERVER about an OBJECT, a PHENOMENON, etc. It is not an intrinsic caracteristic of the object, the phenomenon, but a property of the situation including the observer (his culture) and the observed entity (from an epistemologic point of view). 2.1.2 What ? I say above that the organization level concerns an object, a phenomenon. Let us give some examples: a book, a house an image, a sentence, this mail a processing unit : computer, engine, tv, living being, etc a set of objects, processing units (society, ant colony) etc Some of them live in interaction with an environment (processing units), others are static, closed entities (book, house). 2.1.3 "Level" , "Organization", "Hierarchy" : From giles at fuzzy.nec.com Fri Mar 9 15:11:50 1990 From: giles at fuzzy.nec.com (Giles L.) Date: Fri, 9 Mar 90 15:11:50 EST Subject: No subject Message-ID: <9003092011.AA00586@fuzzy.nec.com> paper available: This 8-page paper will appear in Advances in Neural Information Processing Systems 2, D.S. Touretzky (ed), Morgan Kaufmann, San Mateo, Ca., 1990. HIGHER ORDER RECURRENT NETWORKS & GRAMMATICAL INFERENCE C. L. Giles*, G. Z. Sun, H. H. Chen, Y. C. Lee, D. Chen Department of Physics and Astronomy and Institute for Advanced Computer Studies, University of Maryland, College Park, MD 20742. *NEC Research Institute, 4 Independence Way, Princeton, N.J. 08540 ABSTRACT We design a higher-order single layer, recursive neural network which easily learns to simulate a deterministic finite state machine and infer simple regular grammars from small training sets. An enhanced version of this neural network state machine is then constructed and connected through a common error term to an external analog stack memory. The resulting hybrid machine can be interpreted as a type of neural net pushdown automata. The neural net finite state machine part is given the primitives, push and pop, and is able to read the top of the stack. Using a gradient descent learning rule derived from a common error function, the hybrid network learns to effectively use the stack actions to manipulate the stack memory and to learn simple context-free grammars. If the neural net pushdown automata are reduced through a heuristic clustering of neuron states and actions, the neural network reduces to correct pushdown automata which recognize the learned context-free grammars. --------------- For a hard copy of the above, please send a request to: gloria at research.nec.com or Gloria Behrens NEC Research Institute 4 Independence Way Princeton, N.J. 08540 From Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU Fri Mar 9 16:15:43 1990 From: Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF1.SLISP.CS.CMU.EDU) Date: Fri, 09 Mar 90 16:15:43 EST Subject: Tech Report Announcement: CMU-CS-90-100 Message-ID: *** Please do not forward this to other mailing lists or newsgroups. *** Tech Report CMU-CS-90-100 is now available, after some unfortunate delays in preparation. This is a somewhat more detailed version of the paper that will be appearing soon in "Advances in Neural Information Processing Systems 2" (also known as the NIPS-89 Proceedings). To request a copy of the TR, send a note containing the TR number and your physical mail address to "catherine.copetas at cs.cmu.edu". Please *try* not to respond to the whole mailing list. People who requested a preprint of our paper at the NIPS conference should be getting this TR soon, so please don't send a redundant request right away. If you don't get something in a week or two, then try again. I'll be making an announcement to this list sometime soon about how to get Common Lisp code implementing the algorithm described in this TR. No C version is available at present. =========================================================================== The Cascade-Correlation Learning Architecture Scott E. Fahlman and Christian Lebiere School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 Technical Report CMU-CS-90-100 ABSTRACT Cascade-Correlation is a new architecture and supervised learning algorithm for artificial neural networks. Instead of just adjusting the weights in a network of fixed topology, Cascade-Correlation begins with a minimal network, then automatically trains and adds new hidden units one by one, creating a multi-layer structure. Once a new hidden unit has been added to the network, its input-side weights are frozen. This unit then becomes a permanent feature-detector in the network, available for producing outputs or for creating other, more complex feature detectors. The Cascade-Correlation architecture has several advantages over existing algorithms: it learns very quickly, the network determines its own size and topology, it retains the structures it has built even if the training set changes, and it requires no back-propagation of error signals through the From gaudiano at bucasb.bu.edu Fri Mar 9 16:31:11 1990 From: gaudiano at bucasb.bu.edu (gaudiano@bucasb.bu.edu) Date: Fri, 9 Mar 90 16:31:11 EST Subject: NNSS update and call for help Message-ID: <9003092131.AA14624@retina.bu.edu> NEURAL NETWORKS STUDENT SOCIETY UPDATE and CALL FOR HELP NOTE: in the near future there may be a special newsgroup for student notes like this one so we don't have to clutter up the mailing lists. For now apologies to those subscribers that do not wish to get this. Thanks to all those that responded to the Society Announcement. The response has been overwhelming (almost 300 so far). We have not been acknowledging individual requests because of the volume, but we will mail out our newsletter at the end of the month. At that point we will send notes to make sure everyone's address (email or snailmail) is correct. CALL FOR SUBMISSIONS: If you are in a recognized academic program for Neural Networks and want other students to know about it, please send us a short description IN YOUR OWN WORDS of the program, including an address for people who want more details. Include things like faculty members, courses, and your opinions (if you want). Send the submission by email to "nnss-request at thalamus.bu.edu" by MARCH 21 for inclusion in our newsletter. We will let you know if there are problems or comments. CALL FOR HELP: In order to handle international memberships, we decided it is best to designate "ambassadors" for each country or geographical area. This is primarily to save people the trouble of having to get $5 exchanged from their own currencies, but ambassadors should also be willing to devote some time for possible future Society-related tasks. If you live outside of the US and are willing to devote some of your time to this endeavour, drop us a note (email nnss-request at thalamus.bu.edu). Paolo Gaudiano Karen Haines gaudiano at thalamus.bu.edu khaines at galileo.ece.cmu.edu From walker at sumex-aim.stanford.edu Fri Mar 9 21:34:38 1990 From: walker at sumex-aim.stanford.edu (Michael G. Walker) Date: Fri, 9 Mar 1990 18:34:38 PST Subject: Expert systems and systematic biology workshop Message-ID: Workshop Announcement: Artificial Intelligence and Modern Computer Methods in Systematic Biology (ARTISYST Workshop) The Systematic Biology Program of the National Science Foundation, is sponsoring a Workshop on Artificial Intelligence, Expert Systems, and Modern Computer Methods in Systematic Biology, to be held September 9 to 14, 1990, at the University of California, Davis. There will be about 45 participants representing an even mixture of biologists and computer scientists. Attendance at the workshop is by invitation only. All expenses for participants (travel, hotel, food) will be paid. These are the subject areas for the workshop: 1. Scientific workstations for systematics; 2. Expert systems, expert workstations and other tools for identification; 3. Phylogenetic inference and mapping characters onto tree topologies; 4. Literature data extraction and geographical data; 5. Machine vision and feature extraction applied to systematics. The workshop will examine state-of-the-art computing methods and particularly Artificial Intelligence methods and the possibilities they offer for applications in systematics. Methods for knowledge representation as they apply to systematics will be a central focus of the workshop. This meeting will provide systematists the opportunity to make productive contacts with computer scientists interested in these applications. It will consist of tutorials, lectures on problems and approaches in each area, working groups and discussion periods, and demonstrations of relevant software. Participants will present their previous or proposed research in a lecture, in a poster session, or in a software demonstration session. If you are interested in participating, complete the application form below. Preference will be given to applicants who are most likely to continue active research and teaching in this area. The Workshop organizers welcome applications from all qualified biologists and computer scientists, and strongly encourage women, minorities, and persons with disabilities to apply. APPLICATIONS RECEIVED AFTER APRIL 15, 1990 WILL NOT BE ACCEPTED ----------------- Application form Name: Address: E-mail address: In your application, please include 1) a short resume, 2) a description of your previous work related to the workshop topic, 3) a description of your planned research and how it relates to the workshop, and 4) whether you, as biologists (or computer scientists) have taken or would like to take steps to establish permanent collaboration with computer scientists (or biologists). A total of two pages or less is preferred. This material will be the primary basis for selecting workshop participants. If you have software that you would like to demonstrate at the workshop, please give a brief description, and indicate the hardware that you need to run the program. Several PC's and workstations will be available at the workshop. Mail your completed application to: Renaud Fortuner, ARTISYST Workshop Chairman, California Department of Food and Agriculture Analysis & Identification, room 340 P.O. Box 942871 Sacramento, CA 94271-0001 USA (916) 445-4521 E-mail: rfortuner at ucdavis.edu For further information, contact Renaud Fortuner, Michael Walker, Program Chairman, (Walker at sumex-aim.stanford.edu), or a member of the steering committee: Jim Diederich, U.C. Davis (dieder at ernie.berkeley.edu) Jack Milton, U.C. Davis (milton at eclipse.stanford.edu) Peter Cheeseman, NASA AMES (cheeseman at pluto.arc.nasa.gov) Eric Horvitz, Stanford University (horvitz at sumex-aim.stanford.edu) Julian Humphries, Cornell University (lqyy at crnlvax5.bitnet) George Lauder, U.C Irvine (glauder at UCIvmsa.bitnet) James Rohlf, SUNY (rohlf at sbbiovm.bitnet) James Woolley, Texas A&M University (woolley at tamento.bitnet) From well!mitsu at apple.com Fri Mar 9 18:13:05 1990 From: well!mitsu at apple.com (Mitsuharu Hadeishi) Date: Fri, 9 Mar 90 15:13:05 pst Subject: organization levels Message-ID: <9003092313.AA20573@well.sf.ca.us> I would guess that such a technique (to describe logical levels of abstraction formally, i.e., syntactically) would fail (except as metaphor) because it is often the case that the abstracted level of a formal system is not itself formally describable (at least not in a manner which is easily defined). That is, one would not be able in all cases to define the set of relations on the "meta-" field in any kind of well-defined manner (though such relations might well exist in the original system). Mathematics versus metamathematics, physics versus "metaphysics". I.e., the "meta-" fields are not describable from within the language of the fields themselves, and this is often, I think, the case because the meta-fields are not formally describable in a well-defined manner (at least not a manner which is easily fathomable by human beings). I am, of course, simply speaking through my hat: this seems to me to be the case, but I have not come up with an ironclad argument as yet. I just put the idea out for consideration by those who might be able to clarify the issue. Mitsu From nina at alpha.ece.jhu.edu Mon Mar 12 18:56:54 1990 From: nina at alpha.ece.jhu.edu (Nina A. Kowalski) Date: Mon, 12 Mar 90 18:56:54 EST Subject: IJCNN-90 San Diego Message-ID: *************************************************************************** IJCNN 1990 - REQUEST FOR VOLUNTEERS *************************************************************************** This is the first call for volunteers to help at the International Joint Committee on Neural Networks (IJCNN) conference, to be held at the San Diego Marriot Hotel in San Diego, CA, June 17-21,1990. Full admittance to the conference and a copy of the proceedings is offered in exchange for your assistance throughout the conference. I would like to point out that STUDENT REGISTRATION DOES NOT INCLUDE PROCEEDINGS OR ADMITTANCE INTO WEDNESDAY NIGHT'S PARTY. In general, each volunteer is expected to work one shift each day of the conference. Hours are approximately: AM shift - 7:00 am - Noon PM shift - Noon - 5:00 pm Available shifts are: Technical Session Ushers Poster Sessions Hospitality/Press Room Registration Material Assistance You will be basically working the same shift each day of the conference. In addition to working one of the above shifts throughout the conference, assistance may be required for the social events. Those interested in signing up, please send me the following information: Name Address phone number email Upon signing up, you be sent a form with a more detailed description of the positions, and a request for shift preference and tutorials. Sign ups will be based on the date of commitment. Tutorials: --------- In addition to volunteering for the conference, we will need help the day of the tutorials. The day of the tutorials is Sunday, June 17. Although conference volunteers are not required to work the tutorials, tutorial volunteers are required to work the conference. To sign up please contact: Nina Kowalski - Volunteer Chariman 209 W. 29th St. FLR 2 Baltimore, MD 21211 message: (301) 889-0587 email: nina at alpha.ece.jhu.edu If you have further questions, please feel free to contact me. Thank you, Nina Kowalski IJCNN Volunteer Chairman ----------------------------------------------------------------------------- end of post ---------------------------------------------------------------------------- From gotsman%humus.huji.ac.il at VMA.CC.CMU.EDU Tue Mar 13 02:28:11 1990 From: gotsman%humus.huji.ac.il at VMA.CC.CMU.EDU (Chaim Gotsman) Date: Tue, 13 Mar 90 09:28:11 +0200 Subject: linear separability Message-ID: <9003130728.AA23292@humus.huji.ac.il> The Baum paper from the last NIPS avoids the problem instances where the classic perceptron algorithm runs in exponential time (where some of the points of the hypercube are at an exponentially small distance from the separating plane), by calling these points malicious and probabilistically eliminating them. I don't see why this makes the algorithm suddenly tractable "on the average". From TESAURO at IBM.COM Tue Mar 13 11:05:27 1990 From: TESAURO at IBM.COM (Gerald Tesauro) Date: Tue, 13 Mar 90 11:05:27 EST Subject: "malicious" training patterns Message-ID: The notion that points close to the decision surface are "malicious" comes as a surprise to me. From the point of view of extracting good generalizations from a limited number of training patterns, such "borderline" training patterns may in fact be the best possible patterns to use. A benevolent teacher might very well explicitly design a bunch of border patterns to clearly mark the boundaries between different conceptual classes. --Gerry Tesauro From tgd at turing.CS.ORST.EDU Tue Mar 13 12:34:57 1990 From: tgd at turing.CS.ORST.EDU (Tom Dietterich) Date: Tue, 13 Mar 90 09:34:57 PST Subject: "malicious" training patterns In-Reply-To: Gerald Tesauro's message of Tue, 13 Mar 90 11:05:27 EST <9003131719.AA05526@CS.ORST.EDU> Message-ID: <9003131734.AA13875@turing.CS.ORST.EDU> Date: Tue, 13 Mar 90 11:05:27 EST From: Gerald Tesauro The notion that points close to the decision surface are "malicious" comes as a surprise to me. From the point of view of extracting good generalizations from a limited number of training patterns, such "borderline" training patterns may in fact be the best possible patterns to use. A benevolent teacher might very well explicitly design a bunch of border patterns to clearly mark the boundaries between different conceptual classes. --Gerry Tesauro This is true for cases where the decision surface is parallel to an axis--in that case, the teacher can give two examples differing in only one feature-value. But in general, the more closely positive and negative examples crowd together, the harder it is to resolve and separate them, especially in noisy circumstances. An average case analysis must always define "average", which is what Baum has nicely done. Do readers have examples of domains that are linearly separable but hard to separate? In my experience, the problem is that simple linear models, while they may give reasonable fits to training data, tend to underfit and hence give poor predictive performance. For example, the following points are (just barely) linearly separable, but a multilayer perceptron or a quadratic model would give better predictive performance: + + + - - - + + + - - - + + + - - - + + + - - - + + + - - - + + + - - - + + + - - - + + + - - - + + + - - - As the number of training examples increases, there is statistical support for hypotheses more complex than simple linear separators. Tom Dietterich From John.Hampshire at SPEECH2.CS.CMU.EDU Tue Mar 13 16:59:24 1990 From: John.Hampshire at SPEECH2.CS.CMU.EDU (John.Hampshire@SPEECH2.CS.CMU.EDU) Date: Tue, 13 Mar 90 16:59:24 EST Subject: 'malicious' training tokens Message-ID: Fundamentally I agree with Gerry Tesauro's argument that training tokens in the vicinity of a random vector's true class boundaries are the best ones to use if you want good generalization --- these tokens will delineate the optimal class boundaries. I think there's a caveat though: Say that you could consistently obtain training tokens in the vicinity of the optimal class boundaries. If you could get an arbitrarily large number of independent training tokens, then you could build a perceptron (or for non-linear class boundaries, an MLP) with appropriate connectivity for good generalization. If, however, you were severely limited in the number of independent training samples you could obtain (again, they're all near the optimal class boundaries), then you'd be faced with insufficient data to avoid bad inferences about your limited data --- and you'd get crummy generalization. This would happen because your classifier needs to be sufficiently parameterized to learn the training tokens; however, this degree of parameterization leads to rotten generalization due to insufficient data. In cases of limited training data, then, it may be better to have training tokens near the modes of the class conditional densities (and away from the optimal boundaries) in order for you to at least make a good inference about the prototypical nature of the RV being classified. These tokens would also require a lower degree of parameterization in your classifier, which would give better performance on disjoint test data. I haven't read Baum's paper and I wouldn't presume to put words in anyone's mouth, but maybe this is what he was getting at by characterizing near-boundary tokens as 'malicious'. Incidentally, Duda & Hart (sec. 5.5) give a convergence proof for the perceptron criterion function that illustrates why it takes so long to learn 'malicious' training tokens near the optimal class boundaries. John From jagota at cs.Buffalo.EDU Tue Mar 13 16:09:48 1990 From: jagota at cs.Buffalo.EDU (Arun Jagota) Date: Tue, 13 Mar 90 16:09:48 EST Subject: TR available Message-ID: <9003132109.AA27057@sybil.cs.Buffalo.EDU> The following technical report is available: A Hopfield-style network for content-addressable memories Arun Jagota Department of Computer Science State University Of New York At Buffalo 90-02 ABSTRACT With the binary Hopfield network as a basis, new learning and energy descent rules are developed. It is shown, using graph theoretic techniques, that the stable states of the network are the maximal cliques in the underlying graph and that the network can store an arbitrary collection of memories without interference (memory loss, unstable fixed points). In that sense (and that sense alone), the network has exponential capacity (upto 2^(n/2) memories can be stored in an n-unit network). Spurious memories can (and are likely) to develop. No analytical results for these are derived, but important links are established between the storage and recall properties of the network and the properties of the memories that are stored. In particular it is shown, partly by analysing the graph underlying the network, that the network retrieval time and other desirable properties depend on the 'sparse-ness' of the memories and whether they have a 'combinatorial' structure (as defined in the report). It is shown that the network converges in <= n iterations and for sparse memories (and initial states) with sparseness k, 0 < k < n, it converges in <= k iterations. ------------------------------------------------------------------------ The report is available in PostScript form by anonymous ftp as follows: unix> ftp cheops.cis.ohio-state.edu (or, ftp 128.146.8.62) Name: anonymous Password: neuron ftp> cd pub/neuroprose/Inbox ftp> binary ftp> get jagota.hsn.ps.Z ftp> quit unix> uncompress jagota.hsn.ps.Z unix> lpr jagota.hsn.ps (use flag your printer needs for Postscript) [sometime soon, the report may be moved to pub/neuroprose] ------------------------------------------------------------------------ It is recommended that hard-copy requests be made only if it is not possible (or too inconvenient) to access the report via ftp. I have developed a software simulator that I am willing to share with individuals who might be interested (now or later). It has been carefully 'tuned' for this particular model, implementing the network algorithms in a most efficient manner. It allows configurability, (any size 1-layer net, other parameters etc) and provides a convenient 'symbolic' interface. For hard-copy (and/or simulator) requests send e-mail (or write to) the following address. Please do not reply with 'r' or 'R' to this message. Arun Jagota e-mail: jagota at cs.buffalo.edu Dept Of Computer Science 226 Bell Hall, State University Of New York At Buffalo, NY - 14260 From marvit at hplpm.hpl.hp.com Wed Mar 14 00:21:22 1990 From: marvit at hplpm.hpl.hp.com (Peter Marvit) Date: Tue, 13 Mar 90 21:21:22 PST Subject: Neural models of attention? Message-ID: <9003140521.AA13072@hplpm.HPL.HP.COM> Early this century, William James said "Everyone knows what attention is." Well, after reading lots of cognitive psychology papers, I'm not so sure. After reading more bio-psych and neuro- papers, I'm even less sure I can give a cogent definition, though I still think I know what it is. Some colleagues and I have been discussing various approaches to studying attention. The question arose: What neural models (either artificial [i.e., connectionist] or naturalistic) of attention are there? Who has tried to explain how attention works, produced a model which attempts to be psychologically valid, or (in the words on a friend) explain where attention "comes from"? For that matter, can connectionist models be used for such a slippery subject? Attending your responses, Peter Marvit : Peter Marvit Hewlett-Packard Labs in Palo Alto, CA (415) 857-6646 : : Internet: uucp: {any backbone}!hplabs!marvit : From jc5e+ at ANDREW.CMU.EDU Wed Mar 14 02:47:52 1990 From: jc5e+ at ANDREW.CMU.EDU (Jonathan D. Cohen) Date: Wed, 14 Mar 90 02:47:52 -0500 (EST) Subject: Neural models of attention? Message-ID: Several investigators have begun to address attentional phenomena using connectionist models. They include Schneider, Mozer and Phaff (see references below). My colleagues (Kevin Dunbar and Jay McClelland) and I have also done some work in this area. Our approach has been to view attention as the modulatory influence that information in one part of the system has on processing in other parts. By modulation, we mean changes in the responsivity of processing units. We have implemented our ideas in a model of the Stroop task, a standard psychological paradigm for studying selective aspects of attention. The reference and abstract for this paper are also included below. Jonathan Cohen ************************* Mozer, M. (1988). A connectionist model of selective attention in visual perception. In the proceedings of the tenth annual conference of the Cognitive Science Society.Hillsdale, NJ: Erlbaum, pp. 195-201. Phaff, R. H. (1986). A connectionist model for attention: Restricting parallel processing through modularity. Unpublished doctoral dissertation, University of Experimental Psychology, University of Leiden, Netherlands. Schneider, W. (1985). Toward a model of attention and the development of automatic processing. In M.I. Posner & O.S.M. Marin (Eds.), Attention and and Performance XI (pp.475-492). Hillsdale, NJ: Lawrence Erlbaum. ****************************** Cohen JD, Dunbar K & McClelland JL (in press). On the control of automatic processes: A parallel distributed processing model of the Stroop effect. Psychological Review, in press. (I can handle a *limited* number of requests for preprints) Abstract A growing body of evidence suggests that traditional views of automaticity are in need of revision. For example, automaticity has often been treated as an all-or-none phenomenon, and traditional theories have held that automatic processes are independent of attention. Yet recent empirical data suggest that automatic processes are continuous, and furthermore are subject to attentional control. In this paper we present a model of attention which addresses these issues. Using a parallel distributed processing framework we propose that the attributes of automaticity depend upon the strength of a processing pathway and that strength increases with training. Using the Stroop effect as an example, we show how automatic processes are continuous and emerge gradually with practice. Specifically, we present a computational model of the Stroop task which simulates the time course of processing as well as the effects of learning. This was accomplished by combining the cascade mechanism described by McClelland (1979) with the back propagation learning algorithm (Rumelhart, Hinton, & Williams, 1986). The model is able to simulate performance in the standard Stroop task, as well as aspects of performance in variants of this task which manipulate SOA, response set, and degree of practice. In the discussion we contrast our model with other models, and indicate how it relates to many of the central issues in the literature on attention, automaticity, and interference. From koch%HAMLET.BITNET at VMA.CC.CMU.EDU Wed Mar 14 01:54:23 1990 From: koch%HAMLET.BITNET at VMA.CC.CMU.EDU (Christof Koch) Date: Tue, 13 Mar 90 22:54:23 PST Subject: Neural network model of attention Message-ID: <900313225351.2020114e@Hamlet.Caltech.Edu> I published a network model of attention, together with SHimon Ullman, in 1984/85. It involves a central saliency map, possibly located in area IT or in parietal cortex. Attention selects the most conspicuous or salient location from this map (without knowing "what" is at that location) and then goes back to the individual feature maps to find out about the properties of the attended location (e.g. red, horizontal etc..). The paper is "Shifts in selective visual attention: towards the underlying neural circuitry", C. Koch and S. Ullman, Human Neurbiology Vol. 4: pp. 219-227, 1985. It also appeared as a MIT AI Memo in 1984. Christof P.S. In this version, attention has to solve the binding problem; that is, combine the various attributes of the object being perceived into a coherent whole. From tsejnowski at UCSD.EDU Wed Mar 14 04:15:23 1990 From: tsejnowski at UCSD.EDU (Terry Sejnowski) Date: Wed, 14 Mar 90 01:15:23 PST Subject: March 15 Deadline for CSH Message-ID: <9003140915.AA22429@sdbio2.UCSD.EDU> March 15 *** Deadline for Student Applications to the Cold Spring Harbor Laboratory 1990 Summer Course July 14 - July 28, 1990 Computational Neuroscience: Learning and Memory Organized by: Michael Jordan, MIT and Terry Sejnowski, Salk and UCSD This is an intensive two week course that includes hands-on computer experience as well as lectures. Topics include invertebrate learning mechanisms, synaptic plasticity in the hippocampus, models of classical conditioning, cortical maps, motor learning, temporal-difference learning, radial basis functions, scaling, generalization, and complexity. Instructors include: Jack Byrne, Tom Brown, Steve Lisberger, Nelson Donegan, Gerry Tesauro, Rich Sutton, Ralph Linsker, Richard Durbin, John Moody, Dave Rumelhart, Yan Le Cun, Chris Atkeson, Tommy Poggio, Mike Kearns. Tuition: $1390. Send applications and requests for scholarships to: Registrar Cold Spring Harbor Laboratory Cold Spring Harbor, NY 11724 For application forms and info call: (516) 367-8343 ----- From ZVONA%AI.AI.MIT.EDU at Mintaka.lcs.mit.edu Wed Mar 14 12:14:00 1990 From: ZVONA%AI.AI.MIT.EDU at Mintaka.lcs.mit.edu (David Chapman) Date: Wed, 14 Mar 90 12:14:00 EST Subject: Neural models of attention? In-Reply-To: Msg of Tue 13 Mar 90 21:21:22 PST from Peter Marvit Message-ID: <710698.900314.ZVONA@AI.AI.MIT.EDU> There are actually many models in the literature; most of the people studying the problem seem not to be aware of each other's work. In addition to those already mentioned, [Feldman+Ballard] have proposed a model inspired by [Treisman+Gelade]. (References are at the end of this message.) [Strong+Whitehead] have implemented a similar model. [Fukushima]'s model is rather different. [Koch+Ullman]'s and [Mozer]'s models seem closest to the psychophysical and neurophysiological data to me. I [Chapman] have implemented [Koch+Ullman]'s proposal successfully and used it to model in detail the psychophysically-based theories of visual search due to [Treisman+Geldade, Treisman+Gormican]. My thesis [Chapman] describes these and other models of attention and tries to sort out their relative strengths. It's probably time for someone to write a review article. Cites: David Chapman, {\em Vision, Instruction, and Action.} PhD Thesis, MIT Artificial Intelligence Laboratory, 1990. Jerome A.~Feldman and Dana Ballard, ``Connectionist Models and their Properties.'' {\em Cognitive Science} {\bf 6} (1982) pp.~205--254. Kunihiko Fukushima, ``A Neural Network Model for Selective Attention in Visual Pattern Recognition.'' {\em Biological Cybernetics} {\bf 55} (1986) pp.~5--15. Christof Koch and Shimon Ullman, ``Selecting One Among the Many: A Simple Network Implementing Shifts in Selective Visual Attention.'' {\em Human Neurobiology} {\bf 4} (1985) pp.~219--227. Also published as MIT AI Memo 770/C.B.I.P.~Paper 003, January, 1984. Michael C.~Mozer, ``A connectionist model of selective attention in visual perception.'' {\em Program of the Tenth Annual Conference of the Cognitive Science Society}, Montreal, 1988, pp.~195--201. Gary W.~Strong and Bruce A.~Whitehead, ``A solution to the tag-assignment problem for neural networks.'' {\em Behavioral and Brain Sciences} (1989) {\bf 12}, pp.~381--433. Anne M.~Treisman and Garry Gelade, ``A Feature-Integration Theory of Attention.'' {\em Cognitive Psychology} {\bf 12} (1980), pp.~97--136. Anne Treisman and Stephen Gormican, ``Feature Analysis in Early Vision: Evidence From Search Asymmetries.'' {\em Psychological Review} Vol.~95 (1988), No.~1, pp.~15--48. Some other relevant references: C.~H.~Anderson and D.~C.~Van Essen, ``Shifter circuits: A computational strategy for dynamic aspects of visual processing.'' {\em Proceedings of the National Academy of Sciences, USA}, Vol.~84, pp.~6297--6301, September 1987. Francis Crick, ``Function of the thalamic reticular complex: The searchlight hypothesis.'' {\em Proceedings of the National Academy of Science}, Vol.~81, pp.~4586--4590, July 1984. Jefferey Moran and Robert Desimone, ``Selective attention gates visual processing in the extrastriate cortex.'' {\em Science} {\bf 229} (1985), pp.~782--784. V.~B.~Mountcastle, B.~C.~Motter, M.~A.~Steinmetz, and A.~K.~Sestokas, ``Common and Differential Effects of Attentive Fixation on the Excitability of Parietal and Prestriate (V4) Cortical Visual Neurons in the Macaque Monkey.'' {\em The Journal of Neuroscience}, July 1987, 7(7), pp.~2239--2255. John K.~Tsotsos, ``Analyzing vision at the complexity level.'' To appear, {\em Behavioral and Brain Sciences}. From B344DSL at UTARLG.ARL.UTEXAS.EDU Wed Mar 14 14:23:00 1990 From: B344DSL at UTARLG.ARL.UTEXAS.EDU (B344DSL@UTARLG.ARL.UTEXAS.EDU) Date: Wed, 14 Mar 90 13:23 CST Subject: NN models of attention Message-ID: I don't recall what the original inquiry was regarding neural network models of attention, but there have been several articles of Grossberg's and of mine that have dealt with some aspect of this problem. Grossberg wrote a review article (Int. Rev. of Neurobiology 1975) on attention, reinforcement, and discrimination learning, and he and I wrote a mathematical paper in J. of Theoretical Biology the same year on attentional biases in a competitive network. Psychologists distinguish selective attention between separate stimuli from selective attention between aspects of a single stimulus. Those 1975 papers mainly deal with the former, but my article with Paul Prueitt in Neural Networks 2:103-116 deals with the latter. We model switches between different reinforcement criteria (e. g. based on color vs. based on shape) on the Wisconsin card sorting test, and how such a switch in attentional modula- tion of categorization is disrupted by frontal lobe damage. Dan Levine (b344dsl at utarlg.bitnet) From FRANKLINS%MEMSTVX1.BITNET at VMA.CC.CMU.EDU Wed Mar 14 14:46:00 1990 From: FRANKLINS%MEMSTVX1.BITNET at VMA.CC.CMU.EDU (FRANKLINS%MEMSTVX1.BITNET@VMA.CC.CMU.EDU) Date: Wed, 14 Mar 90 14:46 CDT Subject: Training with incomplete target data Message-ID: I'm helping to design a neural network based diagnostic system in which findings are represented locally as inputs and diagnoses are represented locally as output. The training patterns are such the target value of a single node is known for each pattern. Target values of the other output nodes are not known. I'd appreciate hearing from anyone who has had some experience with training networks under these constraints. I'd be particularly grateful for references to pertinent articles. -- Stan Franklin From kruschke at ucs.indiana.edu Wed Mar 14 16:40:00 1990 From: kruschke at ucs.indiana.edu (kruschke@ucs.indiana.edu) Date: 14 Mar 90 16:40:00 EST Subject: models of attention Message-ID: All the responses (that I've seen) to Peter Marvit's query about attention have concerned *visual/spatial* attention, and mostly the feature-integration problem at that. But ``attention'' is used in other ways in the psych literature. In particular, it is often used to describe which information in a stimulus is deemed to be relevant by the subject when making ``high-level'' decisions, such as which category the stimulus belongs to. Consider, for example, a medical diagnosis situation, in which several different symptoms are present, but only some are ``attended to'' by the physician, because the physician has learned that only some types of symptoms are relevant. This type of attention has been described in the connectionist literature as well. Two references immediately come to mind (no doubt there are others; readers, please inform us): Mozer and Smolensky, ``Skeletonization...'' in NIPS'88; and Gluck & Chow, ``Dynamic stimulus specific learning rates...'' in NIPS'89. I am presently working on a model, called ALCOVE, that learns to distribute attention across input dimensions during training. It captures many effects observed in human category learning, such as the relative difficulty of different types of category structures, some types of base-rate neglect, learning speeds of rules and exceptions (including U-shaped learning of high-frequency exceptions), etc. It is written up in my PhD thesis, to be announced in a few weeks. ---------------------------------------------------------------- John K. Kruschke kruschke at ucs.indiana.edu Dept. of Psychology kruschke at iubacs.bitnet Indiana University (812) 855-3192 Bloomington, IN 47405-4201 USA ---------------------------------------------------------------- From gaudiano at bucasb.bu.edu Wed Mar 14 16:50:53 1990 From: gaudiano at bucasb.bu.edu (gaudiano@bucasb.bu.edu) Date: Wed, 14 Mar 90 16:50:53 EST Subject: Neural models of attention Message-ID: <9003142150.AA02249@retina.bu.edu> >>>>> In reply to: Peter Marvit In addition to all the work on attention mentioned by the previous replies, there is a large body of work by Grossberg on psychological and physiological aspects of attention. Most of the relevant articles are collected in two volumes: [1] S. Grossberg (1986) {\bf The Adaptive Brain I: Cognition, Learning, Reinforcement, and Rhythm.} Amsterdam: Elsevier/North-Holland. Chapter 1 is one that I found particularly clear. It is a reprint of the article: "A Psychophysiological theory of reinforcement, drive, motivation, and attention." {\em Journal of Theoretical Neurobiology.} V. I, 286-369. 1982. The only problem is that attention surfaces here as an integral part of a model that is based primarily on conditioning. Basically, the idea is that things like "motivation", "attention", "drives", etc. have a clear interpretation within the large framework of this article. This means that if you want to know about attention (or motivation, etc.) you kind of need to look at all the rest of the stuff. You will not find a single, self-sustained section that tells you "here is a neural net for attention". Actually, you will find that (e.g., sec. 37), but it is based on all the rest of the material. If you are willing to get into this, it's an *excellent* article. The other good resource would be: [2] S. Grossberg {\bf Studies of Mind and Brain} Boston: Reidel. 1982. For instance, chapter 6 is a reprint of: "A neural model of Attention, Reinforcement and Discrimination Learning." {\em International Review of Neurobiology,} (Carl Pfeiffer, Ed.), V.18, 263-327. 1975. This is an older, and perhaps even broader-scoped article than the other one. Finally, the latest compilation (hey, why write books when you have hundreds of articles that need to be collected in individual, coherent volumes?) is [3] {\bf Neural Networks and Natural Intelligence.} Cambridge, MA, USA: MIT Press. 1988. This includes at least three articles on computer simulations of some of the models developed in the older articles. Have fun! From John.Hampshire at SPEECH2.CS.CMU.EDU Wed Mar 14 21:21:31 1990 From: John.Hampshire at SPEECH2.CS.CMU.EDU (John.Hampshire@SPEECH2.CS.CMU.EDU) Date: Wed, 14 Mar 90 21:21:31 EST Subject: selective attention Message-ID: Depending on your perspective, the following may or may not seem relevant to the topic of attention in connectionist structures. You decide: A hierarchical MLP classifier that recognizes the speech of many speakers by learning how to integrate speaker-dependent modules. This integration involves a sort of supervisory net that learns how to tell when a particular speaker-dependent module is relevant to the speech recognition process. This dynamic focussing of attention to particular modules or groups of modules (depending on the specific input speech signal) is learned indirectly via what we have previously described on this net as the "Meta-Pi" connection. The training error signal is derived from the global objective of "correct phoneme classification", so there is no explicit attention training of the supervisor net. Nevertheless, it learns to direct its attention to relevant modules pretty well (98.4% recognition rate). Attention evolves as a by-product of the global objective. The operation of the whole contraption is explained in terms of Bayesian probability theory. The original work describing this is "The Meta-Pi Network: Building Distributed Knowledge Representations for Robust Pattern Recognition", CMU tech report CMU-CS-89-166, August, 1989. THIS TR IS BEING SUPERSEDED AS WE SPEAK by CMU-CS-89-166-R, so the new version is a few weeks away. ^ | A part of the TR (uneffected by the revision) will appear in the forthcoming NIPS proceedings. Cheers, John Hampshire & Alex Waibel From tsejnowski at UCSD.EDU Thu Mar 15 01:44:44 1990 From: tsejnowski at UCSD.EDU (Terry Sejnowski) Date: Wed, 14 Mar 90 22:44:44 PST Subject: CSH FAX Message-ID: <9003150644.AA16070@sdbio2.UCSD.EDU> Applications to the Cold Spring Harbor Summer Course on Computational Neuroscience: Learning and Memory can be sent by FAX: (516) 367 8845. Deadline is March 15. Terry ----- From pkube at UCSD.EDU Thu Mar 15 01:35:43 1990 From: pkube at UCSD.EDU (Paul Kube) Date: Wed, 14 Mar 90 22:35:43 PST Subject: models of attention Message-ID: <9003150635.AA19417@kokoro.UCSD.EDU> On the topics of recent discussions about connectionist models of attention, and about the relation between connectionist modelling and neurological facts: In a recent paper (Nature, 30 November 1989, pp. 543-545), Luck, Hillyard, Mangun and Gazzaniga report that split-brain patients are twice as good as normals on Triesman-type conjunctive feature visual search tasks when the stimulus array is distributed across both hemifields, but no better than normals when the array is restricted to one hemifield. This suggests that commissurotomy permits a "splitting of attention" that is impossible with connected hemispheres, and that remains impossible within each hemisphere. I'd be interested to know if any of the models of attention under discussion predict this. --Paul Kube kube at cs.ucsd.edu From janet at psych.psy.uq.OZ.AU Wed Mar 14 00:56:21 1990 From: janet at psych.psy.uq.OZ.AU (Janet Wiles) Date: Wed, 14 Mar 90 16:56:21 +1100 Subject: Australian neural networks conference, Feb 1991 Message-ID: <9003140556.AA08485@psych.psy.uq.oz.au> PRELIMINARY ANNOUNCEMENT SECOND AUSTRALIAN CONFERENCE ON NEURAL NETWORKS (ACNN'91) 4th - 6th February 1991 THE UNIVERSITY OF SYDNEY SYDNEY, AUSTRALIA The second Australian conference on neural networks will be held in Sydney on Feb 4, 5 and 6, 1991, at the Stephen Roberts Theatre, The University of Sydney. This conference is interdisciplinary, with emphasis on cross discipline communication between Neuroscientists, Engineers, Computer Scientists, Mathematicians and Psychologists concerned with understanding the integrative nature of the nervous system and its implementation in hardware/software. Neuroscientists concerned with understanding the integrative function of neural networks in vision, audition, motor, somatosensory and autonomic functions are invited to participate and learn how modelling these systems can be used to sharpen the design of experiments as well as to interpret data. Mathematicians and computer scientists concerned with the various new neural network algorithms that have recently become available, as well as with statistical thermodynamic approaches to network modelling and simulation are also invited to contribute. Engineers concerned with the advantages which parallel and distributed computing architectures offer in the solution of various classes of problems and with the state of the art techniques in the hardware implementation of neural network systems are also invited to participate. Psychologists interested in computational models of cognition and perception are invited to contribute and to learn about neural network techniques and their biological and hardware implementations. ACNN'91 will feature invited keynote speakers in the areas of neuroscience, learning, modelling and implementations. The program will include pre-conference workshops, presentations and poster sessions. Proceedings will be printed and distributed to the attendees. Expression of Interest: ----------------------- Please fill the expression of interest form below and return it to Miss Justine Doherty at the address below. ___ I wish to attend the conference ___ I wish to attend the workshops ___ I wish to present a paper Title: _____________________________________________________ _____________________________________________________ Authors: ___________________________________________________ _____________________________________________________ ___ I wish to be on your mailing list My areas of interests are: ____ Neuroscience ____ Learning ____ Modelling ____ Implementation ____ Applications ____ Other: _______________________________________ First Name: ___________________________ Surname: ______________________________ Title: ________________________________ Position:______________________________ Department: ___________________________________________________________ Institution:___________________________________________________________ Address:_______________________________________________________________ _______________________________________________________________________ City: _______________________________ State: __________________________ Zip Code: _________________________ Country: __________________________ Tel: ______________________________ Fax: ______________________________ Email: _________________________________________ ________________________________________________________________________ Organising committee: Chairman Dr Marwan Jabri, Sydney Co-chairman Professor Max Bennett, Sydney Technical Program Chairman Dr Ah Chung Tsoi, ADFA Technical Program Co-Chairman Professor Bill Levick, ANU Publicity Dr Janet Wiles, Queensland Registration Electrical Engineering Foundation Conference Committee Professor Yanni Attikiousel, WA Professor Max Bennett, Sydney Professor Bob Bogner, Adelaide Professor Richard Brent, ANU Dr Jacob Cybulski, TRL Dr Marwan Jabri, Sydney Professor Bill Levick, ANU Dr Tom Osbourn, UTS Professor Steve Redman, ANU Ass/Prof Sam Reisenfeld, OTC Ltd Professor Graham Rigby, UNSW Professor Steve Schwartz, Queensland Dr Ah Chung Tsoi, ADFA Dr Charles Watson, DSTO Dr Janet Wiles, Queensland Dr Hong Yan, Sydney For further information contact: Miss Justine Doherty Secretariat ACNN'91 Sydney University Electrical Engineering NSW 2006 Australia Tel: (+61-2) 692 3659 Fax: (+61-2) 692 3847 Email: acnn91 at ee.su.oz.au _____________________________________________________________________________ From PF103%phoenix.cambridge.ac.uk at NSFnet-Relay.AC.UK Thu Mar 15 04:35:12 1990 From: PF103%phoenix.cambridge.ac.uk at NSFnet-Relay.AC.UK (Peter Foldiak) Date: Thu, 15 Mar 90 09:35:12 GMT Subject: Neural models of attention? In-Reply-To: Msg of Tue 13 Mar 90 21:21:22 PST from Peter Marvit Message-ID: From marcus%cns.edinburgh.ac.uk at NSFnet-Relay.AC.UK Thu Mar 15 09:09:36 1990 From: marcus%cns.edinburgh.ac.uk at NSFnet-Relay.AC.UK (Marcus Frean) Date: Thu, 15 Mar 90 14:09:36 GMT Subject: TR available Message-ID: <4986.9003151409@subnode.cns.ed.ac.uk> The following technical report is available: The Upstart algorithm : a method for constructing and training feed-forward neural networks Marcus Frean Center for Cognitive Science University of Edinburgh ABSTRACT A general method for building and training multi-layer perceptrons composed of linear threshold units is proposed. A simple recursive rule is used to build the net's structure by adding units as they are needed, while a modified Perceptron algorithm is used to learn the connection strengths. Convergence to zero errors is guaranteed for any Boolean classification on patterns of binary variables. Simulations suggest that this method is efficient in terms of the numbers of units constructed, and the networks it builds can generalise over patterns not in the training set. ---------------------------------------------------------------------- This is available in Postscript form by anonymous ftp as follows: unix> ftp cheops.cis.ohio-state.edu (or, ftp 128.146.8.62) Name: anonymous Password: neuron ftp> cd pub/neuroprose/Inbox ftp> binary ftp> get upstart.ps.Z ftp> quit unix> uncompress upstart.ps.Z unix> lpr upstart.ps (use flag your printer needs for Postscript) [The report should be moved to pub/neuroprose fairly soon] ---------------------------------------------------------------------- Please make requests for hard-copy only if you can't get it by ftp. Marcus Frean email: marcus at cns.ed.ac.uk mail : Center for Cognitive Science University of Edinburgh 2 Buccleuch Place Edinburgh EH8 9LW United Kingdom From Connectionists-Request at CS.CMU.EDU Thu Mar 15 11:23:22 1990 From: Connectionists-Request at CS.CMU.EDU (Connectionists-Request@CS.CMU.EDU) Date: Thu, 15 Mar 90 11:23:22 EST Subject: Fwd: Re: Neural models of attention? Message-ID: <29860.637518202@B.GP.CS.CMU.EDU> This should have gone to the whole list. ------- Forwarded Message From rsun at chaos.cs.brandeis.edu Wed Mar 14 13:19:20 1990 From: rsun at chaos.cs.brandeis.edu (Ron Sun) Date: Wed, 14 Mar 90 13:19:20 -0500 Subject: Neural models of attention? Message-ID: <9003141819.AA00896@chaos> Francis Crick had a paper on "searchlight hypothesis" , which is relevant to the issues of attention. It is reprinted in Neurocomputing: foundation of research (eds. Rosenfeld and Anderson). Also, in (Sun 1989) the issues of attentional mechanisms are touched upon. - --------------------- Sun, R. A discrete neural network model for conceptual representation and reasoning. 11th Cognitive science Conference, 1989 - -------------------- Ron Sun Brandeis University ------- End of Forwarded Message From ajr%engineering.cambridge.ac.uk at NSFnet-Relay.AC.UK Thu Mar 15 14:41:28 1990 From: ajr%engineering.cambridge.ac.uk at NSFnet-Relay.AC.UK (Tony Robinson) Date: Thu, 15 Mar 90 19:41:28 GMT Subject: Summary and tech report and thesis availability (long) Message-ID: <25781.9003151941@dsl.eng.cam.ac.uk> There are three topics in this (long) posting: Summary of replies to my message "problems with large training sets". Tech report availability announcement "Phoneme Recognition from the TIMIT database using Recurrent Error Propagation Networks" Thesis availability announcement "Dynamic Error Propagation Networks" Mail me (ajr at eng.cam.ac.uk) if you would like a copy of the tech report and thesis (I will be at ICASSP if anyone there would like to discuss (or save me some postage)). Tony Robinson /*****************************************************************************/ Subject: Summary of replies to my message "problems with large training sets" Thanks to: Ron Cole, Geoff Hinton, Yann Le Cun, Alexander Singer, Fu-Sheng Tsung, Guy Smith and Rich Sutton for their replies, here is a brief summary: Adaptive learning rates: The paper that was most recommended was: Jacobs, R. A. (1988) Increased rates of convergence through learning rate adaptation. {Neural Networks}, {\em 1} pp 295-307. The scheme described in this paper is nice in that it allows the step size scaling factor (\eta) for each weight to vary independently and variations of two orders of magnitude have been observed. Use a faster machine: Something like a 2.7 GFlop Connection Machine could shake some of these problems away! There are two issues here, one is understanding the problem from which more efficient algorithms naturally develop, the other is the need to get results. I don't know how the two will balance in future, but my guess is that we will need more compute. Combined subset training: Several people have used small subsets for initial training, with later training combining these subsets. The reference I was sent was: Fu-Sheng Tsung and Garrison Cottrell (1989) A Sequential Adder with Recurrent Networks IJCNN 89, June, Washington D.C For reasons of software homogeneity, I prefer to use an increasing momentum term, initially it smooths over one "subset" but this increases until the smoothing is over the whole training set. I've never done a comparison of these techniques. Use of higher order derivatives: A good step size can be estimated from the second order derivatives. To me this looks very promising, but I haven't had time to play with it yet. The reference is: Le Cun, Y.: "Generalization and Network Design Strategies", Tech Report CRG-TR-89-4, Dept. of computer science, University of Toronto, 1989. /*****************************************************************************/ Subject: Tech report availability announcement: Phoneme Recognition from the TIMIT database using Recurrent Error Propagation Networks CUED/F-INFENG/TR.42 Tony Robinson and Frank Fallside Cambridge University Engineering Department, Trumpington Street, Cambridge, England. Enquiries to: ajr at eng.cam.ac.uk This report describes a speaker independent phoneme recognition system based on the recurrent error propagation network recogniser described in (RobinsonFallside89, FallsideLuckeMarslandOSheaOwenPragerRobinsonRussell90). This recogniser employs a preprocessor which generates a range of types of output including bark scaled spectrum, energy and estimates of formant positions. The preprocessor feeds a fully recurrent error propagation network whose outputs are estimates of the probability that the given frame is part of a particular phonetic segment. The network is trained with a new variation on the stochastic gradient descent procedure which updates the weights by an adaptive step size in the direction given by the sign of the gradient. Once trained, a dynamic programming match is made to find the most probable symbol string of phonetic segments. The recognition rate is improved considerably when duration and bigram probabilities are used to constrain the symbol string. A set of recognition results is presented for the trade off between insertion and deletion errors. When these two errors balance the recognition rate for all 61 TIMIT symbols is 68.6% correct (62.5% including insertion errors) and on a reduced 39 symbol set the recognition rate is 75.1% correct (68.9%). This compares favourably with the results of other methods on the same database (ZueGlassPhillipsSeneff89, DigalakisOstendorfRohlicek89, HataokaWaibel89, LeeHon89, LevinsonLibermanLjoljeMiller89). /*****************************************************************************/ Subject: Thesis availability announcement "Dynamic Error Propagation Networks" Please forgive me for the title, a better one would have been "Recurrent Error Propagation Networks". This is my PhD thesis, submitted in Feb 1989 and is a concatenation of the work I had done to that date. Summary: This thesis extends the error propagation network to deal with time varying or dynamic patterns. Examples are given of supervised, reinforcement driven and unsupervised learning. Chapter 1 presents an overview of connectionist models. Chapter 2 introduces the error propagation algorithm for general node types. Chapter 3 discusses the issue of data representation in connectionist models. Chapter 4 describes the use of several types of networks applied to the problem of the recognition of steady state vowels from multiple speakers. Chapter 5 extends the error propagation algorithm to deal with time varying input. Three possible architectures are explored which deal with learning sequences of known length and sequences of unknown and possibly indefinite length. Several simple examples are given. Chapter 6 describes the use of two dynamic nets to form a speech coder. The popular method of Differential Pulse Code Modulation for speech coding employs two linear filters to encoded and decode speech. By generalising these to non-linear filters, implemented as dynamic nets, a reduction in the noise imposed by a limited bandwidth channel is achieved. Chapter 7 describes the application of a dynamic net to the recognition of a large subset of the phonemes of English from continuous speech. The dynamic net is found to give a higher recognition rate both in comparison with a fixed window net and with the established k nearest neighbour technique. Chapter 8 describes a further development of dynamic nets which allows them to be trained by a reinforcement signal which expresses the correctness of the output of the net. Two possible architectures are given and an example of learning to play the game of noughts and crosses is presented. From sayegh at ed.ecn.purdue.edu Thu Mar 15 17:41:16 1990 From: sayegh at ed.ecn.purdue.edu (Samir Sayegh) Date: Thu, 15 Mar 90 17:41:16 -0500 Subject: NN Conference April 12-13-14 Message-ID: <9003152241.AA06442@ed.ecn.purdue.edu> Third Conference on Neural Networks and PDP (Robotics and Vision) Indiana-Purdue University Deadline for submission of a 1 page abstract is March 23. e-mail and FAX submissions OK. Conference fee is $25. Students attend free. Inquiries and abstracts: S.Sayegh Physics Dept. Indiana Purdue University Ft Wayne In 46805 email: sayegh at ed.ecn.purdue.edu sayegh at ipfwcvax.bitnet From reggia at cs.UMD.EDU Thu Mar 15 20:22:20 1990 From: reggia at cs.UMD.EDU (James A. Reggia) Date: Thu, 15 Mar 90 20:22:20 -0500 Subject: research position available Message-ID: <9003160122.AA29318@mimsy.UMD.EDU> RESEARCH SCIENTIST POSITION AVAILABLE IN NEURAL MODELLING The component of The Food and Drug Administration responsible for regulating medical devices has an opening for a research scientist. This is a permanent civil service position available for someone interested in modelling the neural activity of the hippocampus. The candidate will focus his/her research on improving the safety and effectiveness of electro-convulsive therapy devices. The candidate must have a PhD in one of the physical sciences. Any additional training in the biological sciences is highly desirable. For more information call or write to: Dr. C. L. Christman (301) 443-3840 Address: FDA HFZ-133 12721 Twinbrook Pkwy Rockville, MD 20857 (Do NOT send inquiries for further information via email to the individual posting this announcement.) From Tuya at etsiig.uniovi.es Fri Mar 16 09:50:00 1990 From: Tuya at etsiig.uniovi.es (Javier Tuya Gonzalez) Date: 16 Mar 90 16:50 +0200 Subject: Neural Networks application? Message-ID: <100*Tuya@etsiig.uniovi.es> Has anybody information (people who works in, papers published, etc.) about some application of Neural Networks for: Classification of crystal structures using X-ray diffraction data Could be possible to apply Neural Networks for it? Please, post me E-mail if you can help me. Thanks in advance. +--------------------------------------+------------------------------------+ | Pablo Javier Tuya Gonzalez | PSI: PSI%(02145)285060338::TUYA | | E.T.S. Ingenieros Industriales | E-Mail: tuya at etsiig.uniovi.es | | Area de Lenguajes y Sistemas | HEPNET: tuya at 16515.decnet.cern.ch | | Informaticos (Universidad de Oviedo) | : EASTVC::TUYA (16.131) | | Carretera de Castiello s/n. | Phone: (..34-85) 338380 ext 278 | | E-33394 GIJON/SPAIN | FAX: (..34-85) 338538 | +--------------------------------------+------------------------------------+ From skrzypek at CS.UCLA.EDU Fri Mar 16 21:51:51 1990 From: skrzypek at CS.UCLA.EDU (Dr. Josef Skrzypek) Date: Fri, 16 Mar 90 18:51:51 PST Subject: last call for abstracts Message-ID: <9003170251.AA16795@retina.cs.ucla.edu> I am organizing two sessions on Artificial Neural Systems and Computational Neuroscience (ANS/CN) for the IEEE International Conference on Systems, Man and Cybernetics. The conference takes place in Los Angeles in Nov 4-7. One session is focused on vision within the scope of ANS/CN. The deadline for extended abstracts is March 23rd. All contributors of selected abstracts will be invited to submitt a contributed paper. The deadline for contributed papers is April 30.. All contributed papers will reviewed by two referees. Notification about acceptance will be send before July 15th. All accepted papers will be published in IEEE SMC Conference proceedings. Deadline for final typed mats for proceedings will be Aug. 15th. Prof Josef Skrzypek Dir. Machine Perception Laboratory Department of Computer Science 3532 BH UCLA Los Angeles CA 90024-1596 From marcus%cns.edinburgh.ac.uk at NSFnet-Relay.AC.UK Fri Mar 16 09:43:40 1990 From: marcus%cns.edinburgh.ac.uk at NSFnet-Relay.AC.UK (Marcus Frean) Date: Fri, 16 Mar 90 14:43:40 GMT Subject: Obtaining "Upstart algorithm" TR by ftp. Message-ID: <323.9003161443@subnode.cns.ed.ac.uk> Due to problems with transferring the compressed version, I've now put an uncompressed version in it's place, which seems to be okay. The tech report is entitled The Upstart algorithm : a method for constructing and training feed-forward neural networks. Marcus Frean. and you can get it as follows: ----------------------------------------------------------------- unix> ftp cheops.cis.ohio-state.edu (or, ftp 128.146.8.62) Name: anonymous Password: neuron ftp> cd pub/neuroprose/Inbox ftp> get upstart.ps ftp> quit unix> lpr upstart.ps (use flag your printer needs for Postscript) ------------------------------------------------------------------ [The report may be compressed and/or moved to pub/neuroprose soon] Let me know if you can't get it by ftp. Marcus Frean email: marcus at cns.ed.ac.uk mail : Center for Cognitive Science University of Edinburgh 2 Buccleuch Place Edinburgh EH8 9LW United Kingdom From sls at dsl.pitt.edu Mon Mar 19 21:05:38 1990 From: sls at dsl.pitt.edu (Steven L. Small) Date: Mon, 19 Mar 90 21:05:38 -0500 Subject: Mathematical Tractability of Neural Nets Message-ID: <9003200205.AA00586@cadre.dsl.pitt.edu> I agree with Liz Bates about neurological localization in general, and language functions in particular. One major problem with localization is that the data come from experiments of nature in which large areas of brain are damaged; inferences about small areas of the brain are made by comparing individuals to look for intersections in both damaged brain and in deficient cognitive processing abilities. There are lots of problems with this, and a number of assumptions of the enterprise are probably wrong (e.g., that computational organizations across individuals do not differ at the gross neuroanatomical level). I also agree that connectionist networks make a better metaphor for brain computations than do filing cabinets and filing clerks (or the store and retrieve operations of CPUs). Regards, Steve Small (neurologist among other things). From delta at csl.ncsu.edu Tue Mar 20 09:04:10 1990 From: delta at csl.ncsu.edu (Thomas Hildebrandt) Date: Tue, 20 Mar 90 09:04:10 EST Subject: Selective Attention Message-ID: <9003201404.AA22459@csl36h.ncsu.edu> Paul Kube of UCSD mentions: In a recent paper (Nature, 30 November 1989, pp. 543-545), Luck, Hillyard, Mangun and Gazzaniga report that split-brain patients are twice as good as normals on Triesman-type conjunctive feature visual search tasks when the stimulus array is distributed across both hemifields, but no better than normals when the array is restricted to one hemifield. This suggests that commissurotomy permits a "splitting of attention" that is impossible with connected hemispheres, and that remains impossible within each hemisphere. It seems to be fairly obvious that attention is impossible without inhibition, and that attention can be interpreted to be the relative lack of it in a subset of neurons. If you adopt this view, then the results of the paper mentioned by kube can be easily explained: One hemisphere inhibits the other. If the connections between them are cut, then each may act independently -- thus doubling the apparent capacity for attention of the brain as a whole. However, I imagine that a commissurotomy also has some UNdesirable effects. . . . Thomas H. Hildebrandt North Carolina State From mariah!yak at tucson.sie.arizona.edu Wed Mar 21 07:32:48 1990 From: mariah!yak at tucson.sie.arizona.edu (mariah!yak@tucson.sie.arizona.edu) Date: Wed, 21 Mar 90 05:32:48 -0700 Subject: No subject Message-ID: <9003211232.AA13367@tucson.sie.arizona.edu> Dear Connectionists, I have a small NSF grant to investigate statistical aspects of machine learning and its relation to neural networks. Dr. H. Gigley suggested to me that I would find it worthwhile the have my name added to a mailing list in the NN topic area. Attached to her message was a news item bearing your address. If you know of the internet mailing list to which she was referring, please send any information you can. Gratefully, Sid Yakowitz Professor (yak at tucson.sie.arizona.edu) From glb at ecelet.ncsu.edu Wed Mar 21 14:18:01 1990 From: glb at ecelet.ncsu.edu (Griff Bilbro) Date: Wed, 21 Mar 90 14:18:01 EST Subject: Linear Separability Message-ID: <9003211918.AA03102@ecelet.ncsu.edu> The statistical mechanical theory of learning predicts that learning linear separability in the plane depends strongly the location of samples. I have applied theory of learning available in the litera- ture [Tishby, Levin, and Solla, IJCNN, 1989] to the problem of learning from examples the line that separates two classes of points in the plane. When the examples in the training set are chosen uniformly in a unit square bisected by the true separator, learning (as measured by the average prediction probability) begins with the first example. If the training set is chosen at some distance from the line, even more learning occurs. On the other hand, if the training set is chosen close to the line, almost no learning is predicted until the training set size reaches 5 examples, but after that learning is so fast that it exceeds the uniform case by 15 examples. Here learning is measured by the predictive ability of the estimated line rather than its numerical precision. The line may be determined to more significant digits by 5 mali- cious points, but this is not enough if the 6th point is drawn from the same malicious distribution. This doesn't apply to the case when the 6th point is drawn from a dif- ferent distribution. Griff Bilbro. From Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU Thu Mar 22 11:31:10 1990 From: Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF1.SLISP.CS.CMU.EDU) Date: Thu, 22 Mar 90 11:31:10 EST Subject: Cascade-Correlation code available Message-ID: *** Please do not forward this to other mailing lists or newsgroups *** For those of you who want to play with the Cascade-Correlation algorithm, a public-domain Common Lisp version of my Cascade-Correlation simulator is now available for FTP via Internet. This is the same version I've been using for my own experiments, except that a lot of non-portable display and user-interface code has been removed. I believe that this version is now strictly portable Common Lisp. It has been tested on CMU Common Lisp on the IBM RT, Allegro Common Lisp (beta test) for Decstation 3100, and Sun/Lucid Common Lisp on the Sun 3. (In the Lucid system it runs properly, but it spends a lot of time garbage-collecting, so it is very slow; maybe there's some optimization magic in Lucid that I don't know about.) The code is heavily commented, so if you read Lisp at all it should be a straightforward task to translate the program (or just the, inner loops) into the language of your choice. A couple of people have told me they planned to port the code to C, and would share the result, but at present no C version is available. Instructions for obtaining the code via Internet FTP are included at the end of this message. If people can't get it by FTP, contact me by E-mail and I'll try once to mail it to you in a single chunk of 51K bytes. If it bounces or your mailer rejects such a large message, I don't have time to try a lot of other delivery methods. I would appreciate hearing about any interesting applications of this code, and will try to help with any problems people run into. Of course, if the code is incorporated into any products or larger systems, I would appreciate an acknowledgement of where it came from. There are several other programs in the "code" directory mentioned below: versions of Quickprop in Common Lisp and C, and some simulation code written by Tony Robinson for the vowel benchmark he contributed to the benchmark collection. -- Scott *************************************************************************** For people (at CMU, MIT, and soon some other places) with access to the Andrew File System (AFS), you can access the files directly from directory "/afs/cs.cmu.edu/project/connect/code". This file system uses the same syntactic conventions as BSD Unix: case sensitive names, slashes for subdirectories, no version numbers, etc. The protection scheme is a bit different, but that shouldn't matter to people just trying to read these files. For people accessing these files via FTP: 1. Create an FTP connection from wherever you are to machine "pt.cs.cmu.edu". 2. Log in as user "anonymous" with no password. You may see an error message that says "filenames may not have /.. in them" or something like that. Just ignore it. 3. Change remote directory to "/afs/cs/project/connect/code". Any subdirectories of this one should also be accessible. Parent directories may not be. 4. At this point FTP should be able to get a listing of files in this directory and fetch the ones you want. The Cascade-Correlation simulator lives in "cascor1.lisp". If you try to access this directory by FTP and have trouble, please contact me. The exact FTP commands you use to change directories, list files, etc., will vary from one version of FTP to another. From fozzard at alumni.Colorado.EDU Thu Mar 22 12:05:59 1990 From: fozzard at alumni.Colorado.EDU (Richard Fozzard) Date: Thu, 22 Mar 90 10:05:59 -0700 Subject: Comparisons of NetTalk to other approaches Message-ID: <9003221705.AA01306@alumni.colorado.edu> As I remember there was some talk here a while back comparing NetTalk with non-neural net approaches to the same problem. We are putting together a talk to high-level government managers, and would like to mention NetTalk. Anyone out there have any references or thoughts on the advantages and disadvantages of the net approach compared to other methods (eg. statistical, rule-based, anything else)? If I get a number of responses, I will summarize. Thanks for the input! Rich ======================================================================== Richard Fozzard "Serendipity empowers" Univ of Colorado/CIRES/NOAA R/E/FS 325 Broadway, Boulder, CO 80303 fozzard at boulder.colorado.edu (303)497-6011 or 444-3168 From tgd at turing.CS.ORST.EDU Thu Mar 22 14:46:13 1990 From: tgd at turing.CS.ORST.EDU (Tom Dietterich) Date: Thu, 22 Mar 90 11:46:13 PST Subject: Comparisons of NetTalk to other approaches In-Reply-To: Richard Fozzard's message of Thu, 22 Mar 90 10:05:59 -0700 <9003221705.AA01306@alumni.colorado.edu> Message-ID: <9003221946.AA16344@turing.CS.ORST.EDU> There are two studies that I know of comparing NETtalk to decision-tree methods such as Quinlan's ID3. 1. Mooney, R., Shavlik, J., Towell, G., and Gove, A. (1989). An experimental comparison of symbolic and connectionist learning algorithms. {\it IJCAI-89: Eleventh International Joint Conference on Artificial Intelligence}. 775--80. This paper included a simple comparison of ID3 and backprop on the nettalk task. 2. Dietterich, T. G., Hild, H., Bakiri, G. (1990) A comparative study of ID3 and backpropagation for English text-to-speech mapping. To appear in 1990 Machine Learning Conference, Austin, TX. This is a more detailed study. I'll be producing a tech report soon and I'll announce availability to connectionists. The bottom line seems to be that, while backprop is awkward and time-consuming to apply, it does give slightly better results on this task. From DGOLDBER at UA1VM.ua.edu Thu Mar 22 15:32:27 1990 From: DGOLDBER at UA1VM.ua.edu (Dave Goldberg) Date: Thu, 22 Mar 90 14:32:27 CST Subject: Dortmund Workshop Message-ID: FIRST ANNOUNCEMENT and CALL FOR PAPERS International Workshop Parallel Problem Solving from Nature (PPSN) October 1 - 3, 1990 University of Dortmund, Germany F.R. Scope With the appearance of massively parallel computers increased attention has been paid to algorithms which rely upon analogies to natural processes. The workshop scope includes but is not limited to the following topics: - Darwinian methods such as Evolution Strategies and Genetic Algorithms - Boltzmann methods such as Simulated Annealing - Classifier systems and Neural Networks insofar as problem solving predominates - Transfer of other natural metaphors to artificial problem solving The objectives of this workshop are - to bring together scientists and practitioners working on and with such strategies. - to gather theoretical results about as well as experimental comparisons between these algorithms. - to discuss various implementations on different parallel computer architectures (e.g. SIMD, MIMD, LAN). - to look for current and future applications in science, technology, and administration. - to summarize the state of the art in this field which up to now has been scattered so widely among disciplines as well as geographically. Submission of papers, Proceedings Prospective authors are invited to submit 4 copies of an extended abstract of two pages to the conference chair before June 1, 1990. All contributions will be reviewed by the programme committee and up to about 30 papers will be selected for presentation. Authors will get notice about acceptance or rejection of their papers by July 15, 1990. Full papers will be due on September 1, 1990. They will be delivered to all participants at the conference as a prepublication volume. Final papers for the proceedings of the workshop should be finished immediately after the workshop. Details about the format of the camera-ready final papers will be distributed later. Language The official language for papers and presentations is English. Conference Chair: H. Muehlenbein and H.-P. Schwefel Gesellschaft fuer Mathematik University of Dortmund und Datenverarbeitung (GMD) -Z1- Dept. of Computer Science P. O. Box 12 40, Schloss Birlinghoven P. O. Box 50 05 00 D-5205 St. Augustin 1 D-4600 Dortmund 50 F. R. Germany F. R. Germany Tel. +49-2241-142405 Tel. +49-231-755-4590 Fax +49-2241-142889 Fax +49-231-755-2047 bitnet grzia0 at dbngmd21 bitnet uin005 at ddohrz11 Programme Committee: (chair) D.E. Goldberg Univ. of Alabama, Tuscaloosa, USA (chair) R. Maenner Univ. of Heidelberg, FRG Institute of Physics Philosophenweg 12 D-6900 Heidelberg 1 Tel. +49-6221-569363 Fax +49-6221-475733 bitnet maen at dhdmpi50 E.M.L. Aarts Philips Res.Lab. Eindhoven, NL P. Bock Univ. of Washington DC, USA V. Cerny Univ. of Bratislava, CSSR Y. Davidor Weizmann Inst. Rehovot, Israel G. Dueck IBM Heidelberg, FRG J.J. Grefenstette Naval Res.Lab. Washington DC, USA A.W.J. Kolen Univ. of Limburg, Maastricht, NL B. Manderick Univ. of Brussels, Belgium H. Roeck Univ. of Bielefeld, FRG H. Schwaertzel Siemens AG Munich, FRG B. Soucek Univ. of Zagreb, YU H.-M. Voigt Academy of Sciences Berlin, GDR Organization Committee: J. Becker, H. Bracklo, H.-P. Schwefel, E. Speckenmeyer, A. Ultsch Sponsors: Parsytec GmbH and Paracom GmbH, IBM Deutschland GmbH, Siemens AG Deadlines: Abstracts (2 pages) June 1, 1990 Notification of acceptance July 15, 1990 Full papers (for preprints) September 1, 1990 Workshop October 1-3, 1990 Final papers November 1, 1990 Reply Form International Workhop Parallel Problem Solving from Nature (PPSN) Dortmund, October 1-3, 1990 c/o Prof. Dr. H.-P. Schwefel Dept. of Computer Science Tel. +49-2 31/7 55/45 90 P. O. Box 50 05 00 Fax +49-2 31/7 55/20 47 D-4600 Dortmund 50 bitnet uin005 at ddohrz11 F. R. Germany Title First Name Middle Initials Last Name ................................................................. Institution ..................................................... Address ......................................................... ................................................................. ................................................................. ( ) Please send further information ( ) I intend to attend the workhop ( ) I intend to submit an abstract: Title of paper to be presented ................................................................. ................................................................. From P.Refenes at Cs.Ucl.AC.UK Fri Mar 23 05:57:00 1990 From: P.Refenes at Cs.Ucl.AC.UK (P.Refenes@Cs.Ucl.AC.UK) Date: Fri, 23 Mar 90 10:57:00 +0000 Subject: C-Implementation of BOLTZMANN learning. Message-ID: Is anyone out there in possesion of a C implementation of BOLTZMANN learning? Is She/He willing to send us a copy of the sources? Thanks in advance, Paul refenes. From Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU Fri Mar 23 09:10:55 1990 From: Scott.Fahlman at SEF1.SLISP.CS.CMU.EDU (Scott.Fahlman@SEF1.SLISP.CS.CMU.EDU) Date: Fri, 23 Mar 90 09:10:55 EST Subject: FTPing code from CMU Message-ID: I forgot to include this in my earlier message: For people who need internet numbers instead of machine names, the machine "pt.cs.cmu.edu" is 128.2.254.155 -- Scott From khaines at galileo.ECE.CMU.EDU Fri Mar 23 15:33:58 1990 From: khaines at galileo.ECE.CMU.EDU (Karen Haines) Date: Fri, 23 Mar 90 15:33:58 EST Subject: ICNN - Request for Volunteers Message-ID: <9003232033.AA26990@galileo.ece.cmu.edu> *************************************************************************** ICNN REQUEST FOR VOLUNTEERS July 9-13,1990 Paris, France *************************************************************************** This is the first call for volunteers to help at the ICNN conference, to be held at the Palias Des Congres in Paris, France, on July 9-13,1990. Full admittance to the conference and a copy of the proceedings is offered in exchange for your assistance throughout the conference. In general, each volunteer is expected to work one shift each day of the conference. Hours are approximately: AM shift - 7:00 am - 1:00pm PM shift - Noon - 6:00 pm In addition, assistance may be required for the social events. Below is a description of the available positions. If you are interested in volunteering, please send me the following information: Name Address Phone number Country electronic mail address shift preference Positions are being filled on a first commit first served basis. If you have further questions, please feel free to contact me. Karen Haines Dept. of ECE Carnegie Mellon University Pittsburgh, PA 15213 message: (412) 362-8675 email: khaines at galileo.ece.cmu.edu At this time there is no funding available from the conference to cover traveling/lodging expenses. Nor do I anticipate any funding available. Thank you, Karen Haines ICNN Volunteer Coordinator Volunteer Positions (volunteers needed) - Description (please note that hours are subject to change) --------------------------------------------------------- Exhibits - Stuffing Proceedings (8) - These volunteers will be required to work Sunday 9am-6pm, Monday 8am-6pm, and Tuesday 8am-12pm. Sunday and Monday will be used to stuff proceedings into the bags. Monday/Tuesday they will double in the exhibits area assisting the Exhibits Chair exhibitors. Poster Session (8) - The volunteers will be responsible for assisting the presenters in putting up/taking down their posters. Days that they will be Shifts are AM or PM Tues thru Thurs. (Hours - General) Conference Sessions (16) - The number of Technical sessions that will be occurring each morning and afternoon of the conference is 4. Two volunteers will be used to check badges at the door for each technical session. Volunteers working the technical sessions will be assigned mornings or afternoons in groups of two. Note that they will be working with the same person each day throughout the conference. Shifts are AM or PM, Tues-Fri. (Hours - General) Exhibit Area II (4) : - Two volunteers will be used to check badges at the door. Volunteers will be assigned mornings or afternoons. Shifts are AM or PM, Tues-Fri. (Hours - General) Message Center (4) - Volunteers will be responsible for the message center. Two volunteers in the morning, two in the afternoon. Shifts are AM or PM Mon-Fri. (Hours - General) Reception at the Hotels (24) - Volunteers will be posted at 6 hotels to provide directions to the conference. Working in teams of 2, these volunteers will be required to work Sunday 9am-9pm, Monday 9am-9pm. From koch%HAMLET.BITNET at VMA.CC.CMU.EDU Sun Mar 25 19:07:52 1990 From: koch%HAMLET.BITNET at VMA.CC.CMU.EDU (Christof Koch) Date: Sun, 25 Mar 90 16:07:52 PST Subject: models of attention In-Reply-To: Your message <9003150635.AA19417@kokoro.UCSD.EDU> dated 14-Mar-1990 Message-ID: <900325160731.23a01f3f@Hamlet.Caltech.Edu> This is in response to P. Kube's question re. the recent Gazzaniga report on split-brain patients possibly having two "searchlights" of attention. The model Shimon Ullman and I proposed would accomodate such a finding, by just cutting our Winner-Take-All pyramid in two, leading to two maximally salient points out there in the visual field. The problem, though, is in locating this control structure. F. Crick proposed in 1984 the Reticular Nucleus of the thalamus as the site of this mechanism. However, the NRT, as are all the other thalamic nuclei, is NOT interconnected with the NRT on the contralateral sienuclei, is no In fact, no thalamic nuclei, with the exception of the ventral lateral geniculate nucleus (which is different from the better known dorsal lateral geniculate nucleus relaying visual information to the cortex), has interhemispheric connections. This seems to imply that the structure controlling attention may reside in neocortex proper (including the claustrum). One caveat, of course, is that negative anatomical findings can always be overthrown one day with better techniques. F. Crick and I have a paper coming out where we discuss a lot of these things in relation to iconic and short-term memory, awareness, attention and neuronal oscillations. Christof From zl at aurel.cns.caltech.edu Mon Mar 26 11:54:36 1990 From: zl at aurel.cns.caltech.edu (Zhaoping Li) Date: Mon, 26 Mar 90 08:54:36 PST Subject: No subject Message-ID: <9003261654.AA00950@aurel.cns.caltech.edu> "A model of the olfactory adaptation and sensitivity enhancement in the olfactory bulb" by Zhaoping LI, in Biological Cybernetics p349-361 62/4 1990. Adaptation and enhancement --- Attention!? Central inputs 'similar' to Crick's search light is used for the mechanism. This paper describes a model of a oscillatory neural system in the bulb, agrees with experimental findings and suggests further experiments. From cole at cse.ogi.edu Mon Mar 26 19:48:50 1990 From: cole at cse.ogi.edu (Ron Cole) Date: Mon, 26 Mar 90 16:48:50 -0800 Subject: English Alphabet Recognition Database Message-ID: <9003270048.AA16206@cse.ogi.edu> The ISOLET database is available for research on speaker-independent classification of spoken English letters. ISOLET contains 2 utterances of the letters A-Z by 150 speakers. The sampling rate, microphone and filter were chosen to mimic the TIMIT database. The database (about 150 MB) is available on 1/2 inch tape, Exabyte cassette, Sun or DEC TK50 cartridge. There is a small charge to cover shipping, handling and storage medium costs. For a description of the database and details on how to order it, send mail to vincew at cse.ogi.edu. Ask for TR 90-004. Ronald A. Cole Computer Science and Engineering Oregon Graduate Institute of Science and Technology 19600 N.W. Von Neumann Dr. Beaverton, OR 97006-1999 cole at cse.ogi.edu 503 690 1159 From risto at CS.UCLA.EDU Tue Mar 27 17:31:36 1990 From: risto at CS.UCLA.EDU (Risto Miikkulainen) Date: Tue, 27 Mar 90 14:31:36 pst Subject: abstracts for 2 tech reports in neuroprose Message-ID: <9003272234.AA08980@shemp.cs.ucla.edu> *********** Do not forward to other bboards ************* The following tech reports are available by anonymous ftp from the pub/neuroprose directory at cheops.cis.ohio-state.edu: A DISTRIBUTED FEATURE MAP MODEL OF THE LEXICON Risto Miikkulainen Ailab, Computer Science Department, UCLA Technical Report UCLA-AI-90-04 DISLEX models the human lexical system at the level of physical structures, i.e. maps and pathways. It consists of a semantic memory and a number of modality-specific symbol memories, implemented as feature maps. Distributed representations for the word symbols and their meanings are stored on the maps, and linked with associative connections. The memory organization and the associations are formed in an unsupervised process, based on co-occurrence of the physical symbol and its meaning. DISLEX models processing of ambiguous words, i.e. homonyms and synonyms, and dyslexic errors in input and in production. Lesioning the system produces lexical deficits similar to human aphasia. DISLEX-1 is an AI implementation of the model, which can be used as the lexicon module in distributed natural language processing systems. ----------------------------------------------------------------------- A NEURAL NETWORK MODEL OF SCRIPT PROCESSING AND MEMORY Risto Miikkulainen Ailab, Computer Science Department, UCLA Technical Report UCLA-AI-90-03 DISCERN is a large-scale NLP system built from distributed neural networks. It reads short narratives about stereotypical event sequences, stores them in episodic memory, generates fully expanded paraphrases of the narratives, and answers questions about them. Processing is based on hierarchically organized backpropagation modules, communicating through a central lexicon of word representations. The lexicon is a double feature map, which transforms the physical word symbol into its semantic representation and vice versa. The episodic memory is a hierarchy of feature maps, where memories are stored ``one-shot'' at different locations. Several high-level phenomena emerge automatically from the special properties of distributed neural networks. DISCERN plausibly infers unmentioned events and unspecified role fillers, and exhibits plausible lexical access errors and memory interference behavior. Word semantics, memory organization and the appropriate script inferences are extracted from examples. ----------------------------------------------------------------------- To obtain copies, either: a) use the getps script (by Tony Plate and Jordan Pollack, posted on connectionists a few weeks ago) b) unix> ftp cheops.cis.ohio-state.edu # (or ftp 128.146.8.62) Name (cheops.cis.ohio-state.edu:): anonymous Password (cheops.cis.ohio-state.edu:anonymous): ftp> cd pub/neuroprose ftp> type binary ftp> get (remote-file) miikkulainen.lexicon.ps.Z (local-file) foo.ps.Z ftp> get (remote-file) miikkulainen.discern.ps.Z (local-file) bar.ps.Z ftp> quit unix> uncompress foo.ps bar.ps unix> lpr -P(your_local_postscript_printer) foo.ps bar.ps From jacobs at gluttony.cs.umass.edu Wed Mar 28 09:42:52 1990 From: jacobs at gluttony.cs.umass.edu (jacobs@gluttony.cs.umass.edu) Date: Wed, 28 Mar 90 09:42:52 EST Subject: new technical report available Message-ID: <9003281442.AA08428@ANW.edu> The following technical report is now available: Task Decomposition Through Competition In a Modular Connectionist Architecture: The What and Where Vision Tasks Robert A. Jacobs (UMass) Michael I. Jordan (MIT) Andrew G. Barto (UMASS) COINS Technical Report 90-27 Abstract -------- A novel modular connectionist architecture is presented in which the networks composing the architecture compete to learn the training patterns. An outcome of the competition is that different networks learn different training patterns and, thus, learn to compute different functions. The architecture performs task decomposition in the sense that it learns to partition a task into two or more functionally independent tasks and allocates distinct networks to learn each task. In addition, the architecture tends to allocate to each task the network whose topology is most appropriate to that task. The architecture's performance on ``what'' and ``where'' vision tasks is presented and compared with the performance of two multi--layer networks. Finally, it is noted that function decomposition is an underconstrained problem and, thus, different modular architectures may decompose a function in different ways. We argue that a desirable decomposition can be achieved if the architecture is suitably restricted in the types of functions that it can compute. Appropriate restrictions can be found through the application of domain knowledge. A strength of the modular architecture is that its structure is well--suited for incorporating domain knowledge. If possible, please obtain a postscript version of this technical report from the pub/neuroprose directory at cheops.cis.ohio-state.edu. a) Here are the directions: unix> ftp cheops.cis.ohio-state.edu # (or ftp 128.146.8.62) Name (cheops.cis.ohio-state.edu:): anonymous Password (cheops.cis.ohio-state.edu:anonymous): neuron ftp> cd pub/neuroprose ftp> type binary ftp> get (remote-file) jacobs.modular.ps.Z (local-file) foo.ps.Z ftp> quit unix> uncompress foo.ps.Z unix> lpr -P(your_local_postscript_printer) foo.ps b) You can also use the Getps script posted on the connectionist mailing list a few weeks ago. If you do not have access to a postscript printer, copies of this technical report can be obtained by sending requests to Connie Smith at smith at cs.umass.edu. Remember to ask for COINS Technical Report 90-27. From bates at amos.ucsd.edu Wed Mar 28 12:12:48 1990 From: bates at amos.ucsd.edu (Elizabeth Bates) Date: Wed, 28 Mar 90 09:12:48 PST Subject: aphasia references Message-ID: <9003281712.AA21945@amos.ucsd.edu> I've had a number of requests for references pertaining to our recent discussion about aphasia and brain localization. Here are a few SOME OF THE RELEVANT REFERENCES FROM THE RECENT DEBATE ON APHASIA ON THE CONNECTIONIST NET WOULD INCLUDE THE FOLLOWING: I. Some of the cross-language studies of aphasia from our laboratories: Bates, E. & Wulfeck, B. (1989). Comparative aphasiology: a cross-linguistic approach to language breakdown. Aphasiology, 3, 111-142. Review of the cross-language work. Bates, E. & Wulfeck, B. (1989). Cross-linguistic studies of aphasia. In B. MacWhinney & E. Bates (Eds.), The cross-linguistic study of sentence processing. New York, Cambridge University Press. Another review, nested within a volume summarizing our cross-language work with normals as well. Bates, E., Friederici, A., & Wulfeck, B. (1987a). Comprehension in aphasia: a cross-linguistic study. Brain & Language, 32, 19-67. Among other things, this study shows that "receptive agrammatism" (i.e. partial loss of sensitivity to closed class elements, albeit to a different degree in each language) occurs not only in Broca's, but in Wernicke's, anomics, and in many neurological and non-neurological patients without focal brain injury. In other words, receptive agrammatism may occur in response to generalized stress!! Bates, E., Friederici, A., & Wulfeck, B. (1987b). Grammatical morphology in aphasia: evidence from three languages. Cortex, 23, 545-574. One of the studies that best illustrates how patients use their preserved knowledge to "shape" closed class omission and other typical symptoms. II. A few references from other laboratories on the "new look" in aphasia research: Basso, A., Capitani, E., Laiacona, M. & Luzzatti, C. (1980). Factors influencing type and severity of aphasia. Cortex, 16, 631 - 636 (an archival review of MRI and CT data showing how often the classical teaching re lesion site and aphasia type is violated). Baum, S. (1989). On-line sensitivity to local and long-distance dependencies in Broca's aphasia. Brain & Language, 37, 327-338. Damasio, H. & Damasio, A. (1989). Lesion analysis in neuropsychology. New York: Oxford University Press. Also documents a few of the surprises in brain-behavior mapping. Friederici, A. & Kilborn, K. (1989). Temporal constraints on language processing in Broca's aphasia. Journal of Cognitive Neuroscience, 1, 262-272. A study showing "grammatical priming" in Broca's aphasics. Linebarger, M., Schwartz, M. & Saffran, E. (1983). Sensitivity to grammatical structure in so-called agrammatic aphasics. Cognition, 13, 361-392. The first of what are now many papers demonstrating preservation of grammaticality judgments in "agrammatic" patients. Lukatela, G., Crain, S. & Shankwweiler, D. (1988). Sensitivity to inflectional morphology in agrammatism: investigation of a highly inflected language. Brain & Language, 33, 1 - 15. Miceli, G., Silveri, M., Romani, C. & Caramazza, A. (1989). Variation in the pattern of omissions and substitutions of grammatical morphemes in the spontaneous speech of so-called agrammatic aphasics. Brain & Language, 36, 447-492. This study goes too far in trying to claim that "everything dissociations from everything else", violating a lot of statistical assumptions in the process. Nevertheless, it clearly shows just how much variation can occur among patients from the "same" clinical category, and it also shows that "agrammatic" symptoms are quite common in patients with posterior as opposed to anterior lesions. Milberg, W. & Blumstein, S. (1981). Lexical decision and aphasia: evidence for semantic processing. Brain & Language, 14, 371-385. This is among the first of a series of papers from this laboratory trying to recast the Broca/Wernicke contrast in processing rather than content terms. Ostrin, R. & Schwartz, M. (1986). Reconstructing from a degraded trace: a study of sentence repetition in agrammatism. Brain & Language, 28, 328-345. Similar line of argument to Milberg & Blumstein, although it differs in detail. Shankweiler, D., Crain, S. Gorrell, P. & Tuller, B. (1989). Reception of language in Broca's aphasia. Language and Cognitive Processes, 4, 1 - 33. Still more evidence for preserved grammar in Broca's aphasia. Swinney, D., Zurif, E., Rosenberg, B. & Nicol, J. Modularity and information access in the lexicon: evidence from aphasia. Journal of Cognitive Neuroscience. Sorry I don't have a more specific reference. This paper tries to salvage modularity in aphasia, showing that semantic priming occurs in both Broca's and Wernicke's aphasia, but in slightly different forms. In fact, the paper makes a strong case that aphasic deficits are based on access problems across a preserved knowledge base. Tyler, L. (1989). Syntactic deficits and the construction of local phrases in spoken language comprehension. Cognitive Neuropsychology, 6, 333 - 356. Yet another attempt to rewrite the nature of processing deficits in aphasia, demonstrating that the basic organization of language is preserved. III. Some papers that are relevant to the argument although they do not present new data on aphasic patients. Hinton, G. & Shallice, T. (1989). Lesioning a connectionist network: investigations of acquired dyslexia. (Tech. rep. CRG-TR-89-30. University of Toronto). Funny things can happen when a language net is randomly lesioned -- things that old-style aphasiologists might typically explain with the logic of localization if the same symptoms were observed in a brain-damaged patient. Kutas, M. & Van Petten, C. (1988). Event-related brain potential studies of language. In P.K. Ackles, J. R. Jennings & M. G. H. Coles (Eds.), Advances in psychophysiology, Vol. III. Greenwich, Connecticut, JAI Press, 139 - 187. Posner, M. Petersen, S., Fox, P. & Raichle, M. (1988). Localization of cognitive operations in the human brain. Science, 240, 1627 - 1631. a "new look" at localization based on PET scan data, arguing that components of attention are localized but linguistic content is not. Seidenberg, M., McClelland, J. & Patterson, K. (1987). A distributed developmental model of visual word recognition, naming and dyslexia. Symposium on Connectionism, Annual Meeting of the Experimental Psychological Society (U.K.), Oxford. There is probably a more recent, published version of this reference but I don't have it. Shows how "dyslexic-like" symptoms can arise from random lesions (i.e. non-localized) to a connectionist net. hope these are useful. -liz bates From A.Hurlbert%newcastle.ac.uk at NSFnet-Relay.AC.UK Wed Mar 28 15:23:38 1990 From: A.Hurlbert%newcastle.ac.uk at NSFnet-Relay.AC.UK (Dr A. Hulbert) Date: Wed, 28 Mar 90 15:23:38 BST Subject: abstracts for 2 tech reports in neuroprose Message-ID: From dave at cogsci.indiana.edu Wed Mar 28 15:43:56 1990 From: dave at cogsci.indiana.edu (David Chalmers) Date: Wed, 28 Mar 90 15:43:56 EST Subject: Technical reports available Message-ID: The following two technical reports are now available from the Center for Research on Concepts and Cognition at Indiana University. ------------------------------------------------------------------------------ SYNTACTIC TRANSFORMATIONS ON DISTRIBUTED REPRESENTATIONS David J. Chalmers Center for Research on Concepts and Cognition Indiana University CRCC-TR-40 There has been much interest in the possibility of connectionist models whose representations can be endowed with compositional structure, and a variety of such models have been proposed. These models typically use distributed representations which arise from the functional composition of constituent parts. Functional composition and decomposition alone, however, yield only an implementation of classical symbolic theories. This paper explores the possibility of moving beyond implementation by exploiting holistic structure-sensitive operations on distributed representations. An experiment is performed using Pollack's Recursive Auto-Associative Memory. RAAM is used to construct distributed representations of syntactically structured sentences. A feed-forward network is then trained to operate directly on these representations, modeling syntactic transformations of the represented sentences. Successful training and generalization is obtained, demonstrating that the implicit structure present in these representations can be used for a kind of structure-sensitive processing unique to the connectionist domain. This paper is to appear in CONNECTION SCIENCE. ------------------------------------------------------------------------------ WHY FODOR AND PYLYSHYN WERE WRONG: THE SIMPLEST REFUTATION David J. Chalmers Center for Research on Concepts and Cognition Indiana University CRCC-TR-41 This paper offers both a theoretical and an experimental perspective on the relationship between connectionist and Classical (symbol-processing) models. Firstly, a serious flaw in Fodor and Pylyshyn's argument against connectionism is pointed out: if, in fact, a part of their argument is valid, then it establishes a conclusion quite different from that which they intend, a conclusion which is demonstrably false. The source of this flaw is traced to an underestimation of the differences between localist and distributed representation. It has been claimed that distributed representations cannot support systematic operations, or that if they can, then they will be mere implementations of traditional ideas. This paper presents experimental evidence against this conclusion: distributed representations can be used to support direct structure-sensitive operations, in a manner quite unlike the Classical approach. Finally, it is argued that even if Fodor and Pylyshyn's argument that connectionist models of compositionality must be mere implementations were correct, then this would still not be a serious argument against connectionism as a theory of mind. ------------------------------------------------------------------------------ To obtain a copy of either of these papers, send e-mail to dave at cogsci.indiana.edu. From Clayton.Bridges at GS10.SP.CS.CMU.EDU Wed Mar 28 16:48:05 1990 From: Clayton.Bridges at GS10.SP.CS.CMU.EDU (Clay Bridges) Date: Wed, 28 Mar 90 16:48:05 EST Subject: Size v. Accuracy Message-ID: <5001.638660885@GS10.SP.CS.CMU.EDU> Does anyone know of any theoretical analysis of the tradeoffs between network size and accuracy of generalization? From turing%ctcs.leeds.ac.uk at NSFnet-Relay.AC.UK Thu Mar 29 13:24:02 1990 From: turing%ctcs.leeds.ac.uk at NSFnet-Relay.AC.UK (Turing Conference) Date: Thu, 29 Mar 90 13:24:02 BST Subject: YOUR LAST CHANCE!! Message-ID: <664.9003291224@ctcs.leeds.ac.uk> ___________________________________________________________________________ Computer Studies and Philosophy, University of Leeds, LEEDS, LS2 9JT Friday, 23rd March 1990 TURING 1990 - FINAL REMINDER I would be very grateful if you could bring this notice to the attention of the relevant academic staff and postgraduates in your department, as soon as possible. It concerns a major conference which is taking place in Sussex University the week after next (starting on Tuesday 3rd April), and for which a limited number of places are still available. Because of the uniqueness of the Conference, and its magnificent range of speakers, we are taking the unusual step of providing a last-minute "reminder" for anyone who may have either failed to see our previous notices, or forgotten to register in time. We are keen to provide a final opportunity for British academics and postgraduates who are interested in computers and their philosophical significance, since it is very unlikely that such an impressive list of speakers in this subject area will be assembled on this side of the Atlantic for a long time to come (see below). Yours sincerely, and with many thanks, Peter Millican ___________________________________________________________________________ INVITED GUEST SPEAKERS ANDREW HODGES, author of the much-acclaimed biography Alan Turing: the Enigma of Intelligence, will give the opening address at the Conference. DONALD MICHIE and ROBIN GANDY, both of whom knew Turing personally, will present the first and last major papers. Gandy is a prominent mathematical logician, while Michie is very well known in artificial intelligence circles, as well as being chief scientist at Glasgow's Turing Institute. The two other invited British speakers are CHRISTOPHER PEACOCKE, Waynflete Professor of Philosophy at Oxford, and J.R. LUCAS, who will be speaking on the topic of his famous and controversial paper "Minds, Machines and Godel" in front of an audience which will include some of his fiercest critics! One of these, DOUGLAS HOFSTADTER (Indiana), achieved fame with his Pulitzer Prize winning book Godel, Escher, Bach, which did much to provoke general interest in artificial intelligence. Other major American visitors include PAUL CHURCHLAND (California), perhaps the best known connectionist opponent of folk-psychology; JOSEPH FORD (Georgia), a prominent advocate of the new and exciting theory of chaos; CLARK GLYMOUR (Carnegie-Mellon), a notable philosopher of science, and last, but certainly not least, HERBERT SIMON (Carnegie-Mellon), one of the founding fathers of the science of artificial intelligence, and a Nobel laureate in 1978. OTHER CONTRIBUTORS Authors of the other 18 contributions include many well-known computer scientists, artificial intelligence researchers, and philosophers from America, Australia and Europe as well as from Britain. Their names, and the titles of their papers, are listed in the programme which follows. ___________________________________________________________________________ TURING 1990 - LATE REGISTRATION INFORMATION VENUE The Conference takes place at the University of Sussex, Falmer, which is about 4 miles from Brighton (the frequent trains take about 8 minutes, and the campus is barely 100 yards from Falmer station). Registration is from 11 a.m. until 2 p.m. on Tuesday 3rd April at NORWICH HOUSE, which is where most delegates will be accommodated. Those arriving late should ask the porter at Norwich House for registration materials unless they arrive after he has gone off duty, in which case registration materials, keys etc. can be collected from the permanent duty porter at the adjacent YORK HOUSE. FIRST AND LAST AFTERNOONS The Conference opens at 2 p.m. on Tuesday, with a lecture by Andrew Hodges in ARTS A2. This will be followed by coffee at 3.00, and a paper by Donald Michie (also in Arts A2) at 3.30. Dinner is from 5.00 until 6.30 in the Refectory, Level 2, with a wine reception in the Grapevine Bar (Refectory building) from 6.00 until 8.00, when Clark Glymour will speak in Arts A2. On Friday 6th April, Lunch is from 12.00 p.m. until 2.00, when Robin Gandy will give the closing speech. Coffee at 3.30 marks the official end of the Conference, although at 4.00 Douglas Hofstadter will give an additional open lecture entitled "Hiroshima Ma Mignonne". Dinner on Friday evening is available for those who require it (at a cost of #6.00). REGISTRATION AND ACCOMMODATION COSTS For members of the Mind Association or the Aristotelian Society, and also subscribers to Analysis or Philosophical Quarterly, the registration fee is only #30, thanks to the generous support which we are receiving from these bodies. The registration fee for students is likewise #30. For other academics the fee is #50, while for non-academics the fee is #80. Full board including bed, breakfast and all meals (with the exception of Thursday evening) from Dinner on Tuesday to Lunch on Friday, costs #84. For those wanting these meals alone (and not bed and breakfast), the cost is #33. On Thursday evening the Conference Banquet takes place at the Royal Pavilion in Brighton (for which we charge only the marginal cost of #25), but for those not attending the Banquet, dinner is available in the University at a cost of #6. Please note that places at the Banquet are strictly limited, and will be filled on a first come-first served basis. HOW TO REGISTER LATE Those who wish to book accommodation for the Conference should ring Judith Dennison at Sussex University (0273-678379) immediately, and if she is not available, should leave on her answerphone full details of their meal and accommodation requirements, together with A TELEPHONE NUMBER AT WHICH THEY CAN BE CONTACTED. Those who telephone by 2.00 p.m. ON FRIDAY 30th MARCH can probably be guaranteed accommodation within the University (though not necessarily in Norwich House), and you are asked to meet this deadline if at all possible (assuming that you are able to catch the Friday postal collection, please also send your cheque and written requirements, by first class mail, to the address below). During the following weekend Andy Clark (0273-722942) will be able to provide some information on the number of places remaining, and on Monday Judith Dennison will do her best to fit in those who have left their name in the meantime. Those who arrive on Tuesday without having booked do so, of course, at their own risk! CHEQUES AND WRITTEN REQUIREMENTS TO: Judith Dennison, School of Cognitive and Computing Sciences, University of Sussex, Brighton, BN1 9QN (please use first class post, and do not include cheques if posted after 30th March). PJRM/23rd March 1990 ____________________________________________________________________________ TURING 1990 COLLOQUIUM At the University of Sussex, Brighton, England 3rd - 6th April 1990 PROGRAMME OF SPEAKERS AND GENERAL INFORMATION ____________________________________________________________________________ INVITED SPEAKERS Paul CHURCHLAND (Philosophy, University of California at San Diego) FURTHER THOUGHTS ON LEARNING AND CONCEPTUAL CHANGE Joseph FORD (Physics, Georgia Institute of Technology) CHAOS : ITS PAST, ITS PRESENT, BUT MOSTLY ITS FUTURE Robin GANDY (Mathematical Institute, Oxford) HUMAN VERSUS MECHANICAL INTELLIGENCE Clark GLYMOUR (Philosophy, Carnegie-Mellon) COMPUTABILITY, CONCEPTUAL REVOLUTIONS AND THE LOGIC OF DISCOVERY Andrew HODGES (Oxford, author of "Alan Turing: the enigma of intelligence") BACK TO THE FUTURE : ALAN TURING IN 1950 Douglas HOFSTADTER (Computer Science, Indiana) MENTAL FLUIDITY AND CREATIVITY J.R. LUCAS (Merton College, Oxford) MINDS, MACHINES AND GODEL : A RETROSPECT Donald MICHIE (Turing Institute, Glasgow) MACHINE INTELLIGENCE - TURING AND AFTER Christopher PEACOCKE (Magdalen College, Oxford) PHILOSOPHICAL AND PSYCHOLOGICAL THEORIES OF CONCEPTS Herbert SIMON (Computer Science and Psychology, Carnegie-Mellon) MACHINE AS MIND ____________________________________________________________________________ OTHER SPEAKERS Most of the papers to be given at the Colloquium are interdisciplinary, and should hold considerable interest for those working in any area of Cognitive Science or related disciplines. However the papers below will be presented in paired parallel sessions, which have been arranged as far as possible to minimise clashes of subject area, so that those who have predominantly formal interests, for example, will be able to attend all of the papers which are most relevant to their work, and a similar point applies for those with mainly philosophical, psychological, or purely computational interests. Jonathan Cohen (The Queen's College, Oxford) "Does Belief Exist?" Mario Compiani (ENIDATA, Bologna, Italy) "Remarks on the Paradigms of Connectionism" Martin Davies (Philosophy, Birkbeck College, London) "Facing up to Eliminativism" Chris Fields (Computing Research Laboratory, New Mexico) "Measurement and Computational Description" Robert French (Center for Research on Concepts and Cognition, Indiana) "Subcognition and the Limits of the Turing Test" Beatrice de Gelder (Psychology and Philosophy, Tilburg, Netherlands) "Cognitive Science is Philosophy of Science Writ Small" Peter Mott (Computer Studies and Philosophy, Leeds) "A Grammar Based Approach to Commonsense Reasoning" Aaron Sloman (Cognitive and Computing Sciences, Sussex) "Beyond Turing Equivalence" Antony Galton (Computer Science, Exeter) "The Church-Turing Thesis: its Nature and Status" Ajit Narayanan (Computer Science, Exeter) "The Intentional Stance and the Imitation Game" Jon Oberlander and Peter Dayan (Centre for Cognitive Science, Edinburgh) "Altered States and Virtual Beliefs" Philip Pettit and Frank Jackson (Social Sciences Research, ANU, Canberra) "Causation in the Philosophy of Mind" Ian Pratt (Computer Science, Manchester) "Encoding Psychological Knowledge" Joop Schopman and Aziz Shawky (Philosophy, Utrecht, Netherlands) "Remarks on the Impact of Connectionism on our Thinking about Concepts" Murray Shanahan (Computing, Imperial College London) "Folk Psychology and Naive Physics" Iain Stewart (Computing Laboratory, Newcastle) "The Demise of the Turing Machine in Complexity Theory" Chris Thornton (Artificial Intelligence, Edinburgh) "Why Concept Learning is a Good Idea" Blay Whitby (Cognitive and Computing Sciences, Sussex) "The Turing Test: AI's Biggest Blind Alley?" ____________________________________________________________________________ TURING 1990 COLLOQUIUM At the University of Sussex, Brighton, England 3rd - 6th April 1990 This Conference commemorates the 40th anniversary of the publication in Mind of Alan Turing's influential paper "Computing Machinery and Intelligence". It is hosted by the School of Cognitive and Computing Sciences at the University of Sussex and held under the auspices of the Mind Association. Additional support has been received from the Analysis Committee, the Aristotelian Society, The British Logic Colloquium, The International Union of History and Philosophy of Science, POPLOG, Philosophical Quarterly, and the SERC Logic for IT Initiative. The aim of the Conference is to draw together people working in Philosophy, Logic, Computer Science, Artificial Intelligence, Cognitive Science and related fields, in order to celebrate the intellectual and technological developments which owe so much to Turing's seminal thought. Papers will be presented on the following themes: Alan Turing and the emergence of Artificial Intelligence, Logic and the Theory of Computation, The Church- Turing Thesis, The Turing Test, Connectionism, Mind and Content, Philosophy and Methodology of Artificial Intelligence and Cognitive Science. Invited talks will be given by Paul Churchland, Joseph Ford, Robin Gandy, Clark Glymour, Andrew Hodges, Douglas Hofstadter, J.R. Lucas, Donald Michie, Christopher Peacocke and Herbert Simon, and there are many other prominent contributors, whose names and papers are listed above. The conference will start after lunch on Tuesday 3rd April 1990, and it will end on Friday 6th April after tea. ANYONE WISHING TO REGISTER FOR THIS CONFERENCE SHOULD SEE THE LATE REGISTRATION INFORMATION ABOVE. Conference Organizing Committee Andy Clark (Cognitive and Computing Sciences, Sussex University) David Holdcroft (Philosophy, Leeds University) Peter Millican (Computer Studies and Philosophy, Leeds University) Steve Torrance (Information Systems, Middlesex Polytechnic) ___________________________________________________________________________ PLEASE SEND ON THIS NOTICE to any researchers, lecturers or students in the fields of Artificial Intelligence, Cognitive Science, Computer Science, Logic, Mathematics, Philosophy or Psychology, in Britain or abroad, and to ANY APPROPRIATE BULLETIN BOARDS which have not previously displayed it. From dmcneal at note.nsf.gov Mon Mar 26 14:34:23 1990 From: dmcneal at note.nsf.gov (Douglas McNeal) Date: Mon, 26 Mar 90 14:34:23 EST Subject: NACSIS: NSF offers access to Japanese data bases Message-ID: <9003261434.aa24834@Note.nsf.GOV> NSF now offers U.S. scientists and engineers free on-line access to nine Japanese science data bases. The data bases, which are compiled and updated by Japan's National Center for Science Information System (NACSIS), index such topics as research projects sponsored by Japan's Ministry of Education, Science, and Culture; papers presented at conferences of electronics societies; and all doctoral theses. U.S. researchers may request searches by surface mail to Room 416-A, NSF, Washington, D.C. 20550, or by electronic mail. Researchers may also contact the operator to schedule training at the NSF offices in using the Japanese-language system in person. For further information, request NSF publication 90-33, NACSIS, from NSF's Publications Unit. To request a search, to reserve time on the system, or to discuss research support opportunities, please call the NACSIS operator at (202) 357-7278 between the hours of 1 and 4 p.m., EST, or send a message by electronic mail to nacsis at nsf.gov (Internet) or nacsis at NSF (BitNet) ------- End of Forwarded Message