From tesauro at watson.ibm.com Tue Jun 1 20:07:38 1993 From: tesauro at watson.ibm.com (Gerald Tesauro) Date: Tue, 1 Jun 93 20:07:38 EDT Subject: TD-Gammon paper available in neuroprose Message-ID: The following paper, which has been accepted for publication in Neural Computation, has been placed in the neuroprose archive at Ohio State. Instructions for retrieving the paper by anonymous ftp are appended below. --------------------------------------------------------------- TD-Gammon, A Self-Teaching Backgammon Program, Achieves Master-Level Play Gerald Tesauro IBM Thomas J. Watson Research Center P. O. Box 704 Yorktown Heights, NY 10598 (tesauro at watson.ibm.com) Abstract: TD-Gammon is a neural network that is able to teach itself to play backgammon solely by playing against itself and learning from the results, based on the TD(lambda) reinforcement learning algorithm (Sutton, 1988). Despite starting from random initial weights (and hence random initial strategy), TD-Gammon achieves a surprisingly strong level of play. With zero knowledge built in at the start of learning (i.e. given only a ``raw'' description of the board state), the network learns to play at a strong intermediate level. Furthermore, when a set of hand-crafted features is added to the network's input representation, the result is a truly staggering level of performance: the latest version of TD-Gammon is now estimated to play at a strong master level that is extremely close to the world's best human players. --------------------------------------------------------------- FTP INSTRUCTIONS unix% ftp archive.cis.ohio-state.edu (or 128.146.8.52) Name: anonymous Password: (use your e-mail address) ftp> cd pub/neuroprose ftp> binary ftp> get tesauro.tdgammon.ps.Z ftp> bye unix% uncompress tesauro.tdgammon.ps unix% lpr tesauro.tdgammon.ps From joachim at fitmail.fit.qut.edu.au Tue Jun 1 23:52:47 1993 From: joachim at fitmail.fit.qut.edu.au (Joachim Diederich) Date: Wed, 2 Jun 93 13:52:47 +1000 Subject: Brisbane Neural Network Workshop Message-ID: <2929F4367ADF20D062@qut.edu.au> First Brisbane Neural Network Workshop -------------------------------------- Queensland University of Technology Brisbane Q 4001, AUSTRALIA Gardens Point Campus, ITE 303 4 June 1993 The first Brisbane Neural Network Workshop is intended to bring together those interested in neurocomputing and neural network applications. The objective of the workshop is to provide a discussion platform for researchers and practitioners interested in theoretical and applied aspects of neurocomputing. The workshop should be of interest to computer scientists and engineers, as well as to biologists, cognitive scientists and others interested in the application of neural networks. This is the first of a series of workshops and seminars with the objective of enhancing collaboration between neural network researchers and practitioners in Queensland. A second workshop is planned for the end of July. The First Brisbane Neural Network Workshop will be held at Queensland University of Technology, Gardens Point Campus (ITE 303) on June 4, 1993 from 8:00am to 6:00pm. Programme 8:00-8:15 Welcome Joachim Diederich, QUT-FIT-CS Neurocomputing 8:15-8:45 Janet Wiles, University of Queensland, Departments of Computer Science and Psychology Representations in hidden unit space 8:45-9:15 Paul Bakker, University of Queensland, Departments of Computer Science and Psychology Examining Learning Dynamics with the Hyperplane Animator 9:15-9:45 Simon Dennis, University of Queensland Department of Computer Science Introducing Learning into Models of Human Memory 9:45-10:15 Steven Phillips, University of Queensland Department of Computer Science Systematicity and Feedforward Networks: Exponential Generalizations from Polynomial Examples 10:15-10:45 Coffee Break 10:45-11:15 Joachim Diederich, QUT-FIT-CS Neurocomputing Cows, Bulls & Tarzan: Preliminary results on animal breeding advice using neural networks 11:15-11:45 Joaquin Sitte, QUT-FIT-CS Neurocomputing Learning control in simple dynamics systems 11:45-12:15 Shlomo Geva, QUT-FIT-CS Neurocomputing Constrained gradient descent 12:15-12:45 Ray Lister, University of Queensland Department of Electrical Engineering On Seeing the World in a Grain of Sand: Hidden Unit Self-Organization, and Super Criticality 12:45-2:00 Lunch Break 2:00-2:30 David Abramson, Griffith University, School of Computing and Information Technology High Performance Computation for Simulated Annealing and Genetic Algorithms 2:30-3:00 John D. Pettigrew, University of Queensland, Vision, Touch & Hearing Research Centre The owl & the pussycat: comparative study of the networks underlying binocular vision. 3:00-3:30 Tom Downs/Ah Chung Tsoi, University of Queensland, Department of Electrical Engineering Directions of research in the UQ EE department 3:30-4:00 David Lovell, University of Queensland, Department of Electrical Engineering An improved version of the neocognitron 4:00-4:30 Coffee Break 4:30-5:00 Ron Ganier, University of Queensland, Department of Electrical Engineering Generalization in artificial neural networks 5:00-5:30 Paul Murtagh, University of Queensland, Department of Electrical Engineering Fault tolerance and VLSI design for artificial neural networks 5:30-6:00 Robert Young, Queensland Department of Primary Industries QDPI Neural Network Applications Enquiries should be sent to Professor Joachim Diederich Neurocomputing Research Concentration Area School of Computing Science Queensland University of Technology GPO Box 2434 Brisbane Q 4001 Phone: (07) 864-2143 Fax: (07) 864-1801 Email: joachim at fitmail.fit.qut.edu.au From fmurtagh at eso.org Thu Jun 3 03:43:22 1993 From: fmurtagh at eso.org (fmurtagh@eso.org) Date: Thu, 3 Jun 93 09:43:22 +0200 Subject: Announcement: conferences calendar available in Neuroprose archive Message-ID: <9306030743.AA08508@st2.hq.eso.org> FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/murtagh.calendar.txt.Z The file murtagh.calendar.txt.Z is available for copying from the Neuroprose repository. It is a CALENDAR of forthcoming conferences and workshops in the neural net and related fields. It is about 1300 lines in length, consists of brief details (date, title, location, contact), and is valid from mid-May 1993 onwards. The intention is to update it in about 3 months. F. Murtagh (fmurtagh at eso.org) From sabine at i4.informatik.rwth-aachen.de Fri Jun 4 14:16:26 1993 From: sabine at i4.informatik.rwth-aachen.de (sabine@i4.informatik.rwth-aachen.de) Date: 4 Jun 1993 14:16:26 MEZ-1 Subject: Workshop on Neural Networks at Aachen, Germany Message-ID: CALL FOR PARTICIPATION "LECTURES AND WORKSHOP ON NEURAL NETWORKS AACHEN '93" Aachen University of Technology D - 52056 Aachen, Germany Introductory Lectures June 21-30 1993 Workshop July 12-13 1993 The first Workshop on Neural Networks at Aachen intends to convey ideas on neural methods to a wide audience interested in neurocomputing and neurocomputers. The 15 distinguished invited speakers will cover topics that range from biological issues and the modelling of consciousness to neurocomputers. The workshop will be complemented by a poster session presenting research projects at Aachen University in the field of neural networks. COMMITEE Honorary Chairman: Prof. I. Aleksander, Imperial College, London Prof. Dr. rer. nat. O. Spaniol Forum Informatik, Graduate College "Methods and tools of computer science and their application in technical systems", Aachen University of Technology, D-52056 Aachen, Germany Neural Networks Special Interest Group INN Harald Huening, Sabine Neuhauser, Michael Raus, Wolf Ritschel, Christiane Schmidt FINAL PROGRAMME INTRODUCTORY LECTURES JUNE 21-30, 1993: June 21, 93, 5 pm, AH III Prof. E.J.H. Kerckhoffs, Delft University of Technology (NL) "An Introduction to Neural Computing" June 22, 93, 5 pm, AH II Prof. C. von der Malsburg, Ruhr-Universitaet Bochum (D) "Neural Networks and the Brain" (German language) June 25, 93, 2 pm, AH IV Dr. U. Ramacher, Siemens AG, Munich (D) "A Computer-Architecture for the Simulation of Artificial Neural Networks and the further Development of Neurochips" (German language) June 30, 93, 5 pm, GH 3, Klinikum Prof. V. Braitenberg, Max-Planck-Institut Tuebingen (D) "New Ideas about the Function of the Cerebellum" (German language) WORKSHOP PROGRAMME, JULY 12, 1993, 9:00 AM - 5:30 PM (AULA II) 9:00 - 9:30 am Welcome and Introduction 9:30 - 10:15 am Prof. I. Aleksander, Imperial College, London (UK) "Iconic State Machines and their Cognitive Properties" 10:15 - 10:45 am Coffee Break, Poster Session 10:45 - 11:30 am Prof. E.J.H. Kerckhoffs, Delft University of Technology (NL) "Thoughts on Conjoint Numeric, Symbolic and Neural Computing" 11:30 am - 12:15 pm Dr. P. DeWilde, Imperial College, London (UK) "Reduction of Representations and the Modelling of Consciousness" 12:15 - 2:00 pm Lunch Break, Poster Session 2:00 - 2:45 pm Dr. M. Erb, Philipps-Universitaet Marburg (D) "Synchronized Activity in Biological and Artificial Dynamic Neural Networks: Experimental Results and Simulations" 2:45 - 3:30 pm drs. E. Postma, University of Limburg, Maastricht (NL) "Towards Scalable Neurocomputers" 3:30 - 4:00 pm Coffee Break, Poster Session 4:00 - 4:45 pm J. Heemskerk, Leiden University, Leiden (NL) "Neurocomputers: Design Principles for a Brain" 4:45 - 5:30 pm Prof. U. Rueckert, Technical University of Hamburg-Harburg, (D) "Microelectronic Implementation of Neural Networks" WORKSHOP PROGRAMME, JULY 14, 1993, 9:00 AM - 3:00 PM (AULA II): 9:00 - 9:45 am Dr. J. H. Schmidhuber, Technical University of Munich (D) "Continuous History Compression" 9:45 - 10:30 am K. Weigl, INRIA, Sophia-Antipolis (F) "Metric Tensors and Non-orthogonal Functional Bases" 10:30 - 11:00 am Coffee Break, Poster Session 11:00 - 11:45 am Dr. F. Castillo, Univ. Politecnica de Catalunya, Barcelona (E) "Statistics and Neural Network Classifiers: A Review from Multilayered Perceptrons to Incremental Neural Networks" 11:45 am - 1:30 pm Lunch Break, Poster Session 1:30 - 2:15 pm Dr. J. Mrsic-Floegel, Imperial College, London (UK) "A Review of RAM-based Weightless Nodes" 2:15 - 3:00 pm J. Schaefer, Aachen University of Technology (D) "Neural Networks and Fuzzy Technologies" LOCATIONS The workshop lectures will be performed at the lecture-hall Aula II, Aachen University of Technology, Ahornstrasse 55, D-52074 Aachen, Germany. The introductory lectures are performed in one of the following lecture-halls, as indicated in the programme: AH II, AH III, AH IV, Ahornstrasse 55, D-52074 Aachen, Germany and GH3, Klinikum Aachen, Pauwelstrasse, D- 52074 Aachen, Germany. Ahornstrasse can be reached by bus routes no. 23 or 33: - bus route 33 to "Klinikum" or "Vaals", stop at "Paedagogische Hochschule"; - bus route 23 to "Hoern", stop at "Paedagogische Hochschule". Klinikum can be reached by bus route 33 as well, stop at "Klinikum". To reach bus routes 23 or 33, take a bus from the station to "Bushof". PARTICIPATION is free of charge. Please register by e-mail to the organizing commitee: Harald Huening: harry at dfv.rwth-aachen.de Sabine Neuhauser: sabine at informatik.rwth-aachen.de Michael Raus: raus at rog1.rog.rwth-aachen.de Wolf Ritschel: ri at mtq03.wzl.rwth-aachen.de PROCEEDINGS: H. Huening, S. Neuhauser, M. Raus, W. Ritschel (eds.): "Workshop on Neural networks at RWTH Aachen", Aachener Beitraege zur Informatik ABI, Band 2, Verlag der Augustinus Buchhandlung, 227 pages contain the articles of the workshop + the article "Am I Thinking Assemblies ?" of Prof. C. von der Malsburg. Proceedings can be ordered from Augustinus Buchhandlung Pontstrasse 66/68 D-52062 Aachen at a price of 36.- DM plus postal coverage and postage (about 3.-DM within Germany). During the Workshop the book will be sold at a reduced price by Augustinus bookstore. LANGUAGE English will be the official conference language. sabine at informatik.rwth-aachen.de _______________________________________________________ Sabine Neuhauser Aachen University of Technology Computer Science Department (Informatik IV) Ahornstrasse 55, W- 5100 Aachen, Germany !!! please note the new postal code for !!! Aachen University of Technology !!! valid from 1.7.93 : D-52056 Aachen (postal address) From gasser at cs.indiana.edu Fri Jun 4 15:25:14 1993 From: gasser at cs.indiana.edu (Michael Gasser) Date: Fri, 4 Jun 1993 14:25:14 -0500 Subject: Paper on lexical acquisition Message-ID: FTP-host: cs.indiana.edu (129.79.254.191) FTP-filename: /pub/techreports/TR382.ps.Z The following report is available in compressed postscript form by anonymous ftp from the site given above (note: NOT neuroprose). The paper is 23 pages long. If you have trouble printing it out, please contact me. Michael Gasser gasser at cs.indiana.edu ================================================================= Learning Noun and Adjective Meanings: A Connectionist Account Michael Gasser Computer Science and Linguistics Departments Linda B. Smith Psychology Department Indiana University Abstract Why do children learn nouns such as {\it cup\/} faster than dimensional adjectives such as {\it big\/}? Most explanations of this well-known phenomenon rely on prior knowledge in the child of the noun-adjective distinction or on the logical priority of nouns as the arguments of predicates. In this paper we examine an alternative account, one which seeks to explain the relative ease of nouns over adjectives in terms of the response of the learner to various properties of the semantic categories to be learned and of the word learning task itself. We isolate four such properties: the relative size and the relative compactness of the regions in representational space associated with the categories, the presence or absence of lexical dimensions in the linguistic context of a word ({\it what color is it?\/} vs. {\it what is it?\/}), and the number of words of a particular type to be learned. In a set of five experiments, we trained a simple connectionist categorization device to label input objects, in particular linguistic contexts, as nouns or adjectives. We show that, for the network, the first three of the above properties favor the more rapid learning of nouns, while the fourth favors the more rapid learning of adjectives. Our experiments demonstrate that the advantage for nouns over adjectives does not require prior knowledge of the distinction between nouns and adjectives and suggest that this distinction may instead emerge as the child learns to associate the different properties of noun and adjective categories with the different morphosyntactic contexts which elicit them. From sabine at i4.informatik.rwth-aachen.de Fri Jun 4 15:37:29 1993 From: sabine at i4.informatik.rwth-aachen.de (sabine@i4.informatik.rwth-aachen.de) Date: 4 Jun 1993 15:37:29 MEZ-1 Subject: workshop on neural networks, Aachen 93 Message-ID: Dear organizers of this list, I've just sent an announcement about the Lectures and Workshop on Neural Networks at Aachen '93 to the connectionists-address. The dates for this workshop are June 21-30 for the Introductory Lectures July 12-13 1993 for the Workshop. Unfortunately, I've mentioned a wrong date for the second day of the workshop: in the second "Workshop programme..." header, I've mentioned the date "July, 14" instead of "July, 13". I'd be very pleased, if you could change this before posting it to the whole list. Thanks in advance, I'm really sorry for that mistake, Sabine Neuhauser sabine at informatik.rwth-aachen.de _______________________________________________________ Sabine Neuhauser Aachen University of Technology Computer Science Department (Informatik IV) Ahornstrasse 55, W- 5100 Aachen, Germany !!! please note the new postal code for !!! Aachen University of Technology !!! valid from 1.7.93 : D-52056 Aachen (postal address) From bap at learning.siemens.com Mon Jun 7 11:22:41 1993 From: bap at learning.siemens.com (Barak Pearlmutter) Date: Mon, 7 Jun 93 11:22:41 EDT Subject: Preprint Available Message-ID: <9306071522.AA17817@gull.siemens.com> I have placed the preprint whose abstract appears below in the neuroprose archives. My thanks to Jordan Pollack for providing this valuable service to the community. ---------------- Fast Exact Multiplication by the Hessian Barak A. Pearlmutter Just storing the Hessian $H$ (the matrix of second derivatives of the error $E$ with respect to each pair of weights) of a large neural network is difficult. Since a common use of a large matrix like $H$ is to compute its product with various vectors, we derive a technique that directly calculates $Hv$, where $v$ is an arbitrary vector. To calculate $Hv$, we first define a differential operator $R{f(w)} = (d/dr) f(w+rv) |_{r=0}$, note that $R{dE/dw} = Hv$ and $R{w} = v$, and then apply $R{}$ to the equations used to compute $dE/dw$. The result is an exact and numerically stable procedure for computing $Hv$, which takes about as much computation, and is about as local, as a gradient evaluation. We then apply the technique to a one pass gradient calculation algorithm (backpropagation), a relaxation gradient calculation algorithm (recurrent backpropagation), and two stochastic gradient calculation algorithms (Boltzmann Machines and weight perturbation). Finally, we show that this technique can be used at the heart of many iterative techniques for computing various properties of $H$, obviating any need to calculate the full Hessian. [12 pages; 42k; pearlmutter.hessian.ps.Z; To appear in Neural Computation] From SCHOLTES at ALF.LET.UVA.NL Mon Jun 7 10:41:00 1993 From: SCHOLTES at ALF.LET.UVA.NL (SCHOLTES@ALF.LET.UVA.NL) Date: Mon, 7 Jun 93 10:41 MET Subject: PhD Dissertation available Message-ID: =================================================================== As I had to disapoint many people because I run out of copies in the first batch, a high-quality reprint has been made from....................................... ........REPRINT........ Ph.D. DISSERTATION AVAILABLE on Neural Networks, Natural Language Processing, Information Retrieval 292 pages and over 350 references =================================================================== A Copy of the dissertation "Neural Networks in Natural Language Processing and Information Retrieval" by Johannes C. Scholtes can be obtained for cost price and fast airmail- delivery at US$ 25,-. Payment by Major Creditcards (VISA, AMEX, MC, Diners) is accepted and encouraged. Please include Name on Card, Number and Exp. Date. Your Credit card will be charged for Dfl. 47,50. Within Europe one can also send a Euro-Cheque for Dfl. 47,50 to: (include 4 or 5 digit number on back of cheque!) University of Amsterdam J.C. Scholtes Dufaystraat 1 1075 GR Amsterdam The Netherlands scholtes at alf.let.uva.nl Do not forget to mention a surface shipping address. Please allow 2-4 weeks for delivery. Abstract 1.0 Machine Intelligence For over fifty years the two main directions in machine intelligence (MI), neural networks (NN) and artificial intelligence (AI), have been studied by various persons with many dfferent backgrounds. NN and AI seemed to conflict with many of the traditional sciences as well as with each other. The lack of a long research history and well defined foundations has always been an obstacle for the general acceptance of machine intelligence by other fields. At the same time, traditional schools of science such as mathematics and physics developed their own tradition of new or "intelligent" algorithms. Progress made in the field of statistical reestimation techniques such as the Hidden Markov Models (HMM) started a new phase in speech recognition. Another application of the progress of mathematics can be found in the application of the Kalman filter in the interpretation of sonar and radar signals. Much more examples of such "intelligent" algorithms can be found in the statistical classification en filtering techniques of the study of pattern recognition (PR). Here, the field of neural networks is studied with that of pattern recognition in mind. Although only global qualitative comparisons are made, the importance of the relation between them is not to be underestimated. In addition it is argued that neural networks do indeed add something to the fields of MI and PR, instead of competing or conflicting with them. 2.0 Natural Language Processing The study of natural language processing (NLP) exists even longer than that of MI. Already in the beginning of this century people tried to analyse human language with machines. However, serious efforts had to wait until the development of the digital computer in the 1940s, and even then, the possibilities were limited. For over 40 years, symbolic AI has been the most important approach in the study of NLP. That this has not always been the case, may be concluded from the early work on NLP by Harris. As a matter of fact, Chomsky's Syntactic Structures was an attack on the lack of structural proper-ties in the mathematical methods used in those days. But, as the latter's work remained the standard in NLP, the former has been forgotten completely until recently. As the scientific community in NLP devoted all its attention to the symbolic AI-like theories, the only use- ful practical implementation of NLP systems were those that were based on statistics rather than on linguistics. As a result, more and more scientists are redirecting their attention towards the statistical techniques a vailable in NLP. The field of connectionist NLP can be considered as a special case of these mathematical methods in NLP. More than one reason can be given to explain this turn in approach. On the one hand, many problems in NLP have never been addressed properly by symbolic AI. Some examples are robust behavior in noisy environments, disambiguation driven by different kinds of knowledge, commensense generalizations, and learning (or training) abilities. On the other hand, mathematical methods have become much stronger and more sensitive to spe- cific properties of language such as hierarchical structures. Last but not least, the relatively high degree of success of mathematical techniques in commercial NLP systems might have set the trend towards the implementation of simple, but straightforward algorithms. In this study, the implementation of hierarchical structures and semantical features in mathematical objects such as vectors and matrices is given much attention. These vectors can then be used in models such as neural networks, but also in sequential statistical procedures implementing similar characteristics. 3.0 Information Retrieval The study of information retrieval (IR) was traditionally related to libraries on the one hand and military applications on the other. However, as PC's grew more popular, most common users loose track of the data they produced over the last couple of years. This, together with the introduction of various "small platform" computer programs made the field of IR relevant to ordinary users. However, most of these systems still use techniques that have been developed over thirty years ago and that implement nothing more than a global surface analysis of the textual (layout) properties. No deep structure whatsoever, is incorporated in the decision whether or not to retrieve a text. There is one large dilemma in IR research. On the one hand, the data collections are so incredibly large, that any method other than a global surface analysis would fail. On the other hand, such a global analysis could never implement a contextually sensitive method to restrict the number of possible candidates returned by the retrieval system. As a result, all methods that use some linguistic knowledge exist only in laboratories and not in the real world. Conversely, all methods that are used in the real world are based on technological achievements from twenty to thirty years ago. Therefore, the field of information retrieval would be greatly indebted to a method that could incorporate more context without slowing down. As computers are only capable of processing numbers within reasonable time limits, such a method should be based on vectors of numbers rather than on symbol manipulations. This is exactly where the challenge is: on the one hand keep up the speed, and on the other hand incorporate more context. If possible, the data representation of the contextual information must not be restricted to a single type of media. It should be possible to incorporate symbolic language as well as sound, pictures and video concurrently in the retrieval phase, although one does not know exactly how yet... Here, the emphasis is more on real-time filtering of large amounts of dynamic data than on document retrieval from large (static) data bases. By incorporating more contextual information, it should be possible to implement a model that can process large amounts of unstructured text without providing the end-user with an overkill of information. 4.0 The Combination As this study is a very multi-disciplinary one, the risk exists that it remains restricted to a surface discussion of many different problems without analyzing one in depth. To avoid this, some central themes, applications and tools are chosen. The themes in this work are self- organization, distributed data representations and context. The applications are NLP and IR, the tools are (variants of) Kohonen feature maps, a well known model from neural network research. Self-organization and context are more related to each other than one may suspect. First, without the proper natural context, self-organization shall not be possible. Next, self-organization enables one to discover contextual relations that were not known before. Distributed data representation may solve many of the unsolved problems in NLP and IR by introducing a powerful and efficient knowledge integration and generalization tool. However, distributed data representation and self-organization trigger new problems that should be solved in an elegant manner. Both NLP and IR work on symbolic language. Both have properties in common but both focus on different features of language. In NLP hierarchical structures and semantical features are important. In IR the amount of data sets the limitations of the methods used. However, as computers grow more powerful and the data sets get larger and larger, both approaches get more and more common ground. By using the same models on both applications, a better understanding of both may be obtained. Both neural networks and statistics would be able to implement self-organization, distributed data and context in the same manner. In this thesis, the emphasis is on Kohonen feature maps rather than on statistics. However, it may be possible to implement many of the techniques used with regular sequential mathematical algorithms. So, the true aim of this work can be formulated as the understanding of self-organization, distributed data representation, and context in NLP and IR, by in depth analysis of Kohonen feature maps. ============================================================================== From haussler at cse.ucsc.edu Tue Jun 8 14:11:16 1993 From: haussler at cse.ucsc.edu (David Haussler) Date: Tue, 8 Jun 1993 11:11:16 -0700 Subject: COLT `93: Early registration deadline June 15 Message-ID: <199306081811.AA25547@arapaho.ucsc.edu> COLT '93 Sixth ACM Conference on Computational Learning Theory Monday, July 26 through Wednesday, July 28, 1993 University of California, Santa Cruz, California EARLY REGISTRATION DEADLINE: JUNE 15 The workshop will be held on campus, which is hidden away in the redwoods on the Pacific coast of Northern California. The workshop is sponsored by the ACM Special Interest Group on Algorithms and Computation Theory (SIGACT) and the ACM Special Interest Group on Artificial Intelligence (SIGART). The long version of this document is available by anonymous ftp from ftp.cse.ucsc.edu. To ftp the document you do the following: step 1) ftp ftp.cse.ucsc.edu, and login as "anonymous", 2) cd pub/colt, 3) binary, 4) get colt93.registration.ps. REGISTRATION INFORMATION ------------------------ Please fill in the information needed on the registration sheet Make your payment by check or international money order, in U.S. dollars and payable through a U.S. bank, to COLT '93. Mail the form together with payment (by June 15 to avoid the late fee) to: COLT '93 Dept. of Computer Science University of California Santa Cruz, California 95064 ACCOMMODATIONS AND DINING Accommodation fees are $57 per person for a double and $70 for a single per night at the College Eight Apartments. Cafeteria style breakfast (7:45 to 8:30am), lunch (12:30 to 1:30pm), and dinner (6:00 to 7:00pm) will be served in the College Eight Dining Hall. Doors close at the end of the time indicated, but dining may continue beyond this time. The first meal provided is dinner on the day of arrival and the last meal is lunch on the day you leave. NO REFUNDS can be given after June 15. Those with uncertain plans should make reservations at an off-campus hotel. Each attendee should pick one of the five accomdation packages. For shorter stays, longer stays, and other special requirements, you can get other accommodations through the Conference Office. Make reservations directly with them at (408)459-2611, fax (408)459-3422, and do this soon as on-campus rooms for the summer fill up well in advance. Off-campus hotels include the Dream Inn (408)426-4330 and the Holiday Inn (408)426-7100. Questions: e-mail colt93 at cse.ucsc.edu, fax (408)429-4829. Confirmations will be sent by e-mail. Anyone needing special arrangements to accommodate a disability should enclose a note with their registration. If you don't receive confirmation within three weeks of payment, let us know. Get updated versions of this document by anonymous ftp from ftp.cse.ucsc.edu. CONFERENCE REGISTRATION FORM (see accompanying information for details) Name: ___________________________________ Affiliation: ___________________________________ Address: ___________________________________ City: ________________ State: ____________ Zip: ________________ Country: ____________________ Telephone: (____) ________________ Email: ________________________ The registration fee includes a copy of the proceedings. ACM/SIG Members: $165 (with banquet) $___________ Non-Members: $185 (with banquet) $___________ Late: $220 (postmarked after June 15) $___________ Full time students: $80 (no banquet) $___________ Extra banquet tickets: ___ (quantity) x $18 = $___________ How many in your party have dietary restrictions? Vegetarian: ___________ Other: ___________ Shirt size, please circle one: small medium large x-large ACCOMODATIONS: Pick one package: _____ Package 1: Sun, Mon, Tue nights: $171 double, $210 single. _____ Package 2: Sat, Sun, Mon, Tue nights: $228 double, $280 single. _____ Package 3: Sun, Mon, Tues, Wed nights: $228 double, $280 single. _____ Package 4: Sat, Sun, Mon, Tue, Wed nights: $285 double, $350 single. ______Other housing arrangement. Each 4-person apartment has a living room, a kitchen, two common bathrooms, and either four single separate rooms, two double rooms, or two single and one double room. We need the following information to make room assignments: Gender (M/F): __________ Smoker (Y/N): __________ Roommate Preference: ____________________ AMOUNT ENCLOSED: Registration $___________________ Banquet tickets $___________________ Accommodations $___________________ TOTAL $___________________ Mail this form together with payment (by June 15 to avoid the late fee) to: COLT '93, Dept. of Computer Science, Univ. California, Santa Cruz, CA 95064 COLT '93 --- Conference Schedule Sixth ACM Conference on Computational Learning Theory Monday, July 26 through Wednesday, July 28, 1993 University of California, Santa Cruz, California SUNDAY, JULY 25 4:00 - 6:00 pm, Housing Registration, College Eight Satellite Office. 7:00 - 10:00 pm, Reception, Oakes Learning Center. Preregistered attendees may check in at the reception. Note: All technical sessions will take place in Oakes 105 . MONDAY, JULY 26 Session 1: Learning with Queries Chair: Dana Angluin 8:20-8:40 Learning Sparse Polynomials over Fields with Queries and Counterexamples. Robert E. Schapire and Linda M. Sellie 8:40-9:00 Learning Branching Programs with Queries. Vijay Raghavan and Dawn Wilkins 9:00-9:10 Linear Time Deterministic Learning of k-term DNF. Ulf Berggren 9:10-9:30 Asking Questions to Minimize Errors. Nader H. Bshouty, Sally A. Goldman, Thomas R. Hancock, and Sleiman Matar 9:30-9:40 Parameterized Learning Complexity. Rodney G. Downey, Patricia Evans, and Michael R. Fellows 9:40-10:00 On the Query Complexity of Learning. Sampath K. Kannan 10:00 - 10:30 BREAK Session 2: New Learning Models and Problems Chair: Sally Goldman 10:30-10:50 Teaching a Smarter Learner. Sally A. Goldman and H. David Mathias 10:50-11:00 Learning and Robust Learning of Product Distributions. Klaus-U. Hoffgen 11:00-11:20 A Model of Sequence Extrapolation. Philip Laird, Ronald Saul and Peter Dunning 11:20-11:30 On Polynomial-Time Probably Almost Discriminative Learnability. Kenji Yamanishi 11:30-11:50 Learning from a Population of Hypotheses. Michael Kearns and Sebastian Seung 11:50-12:00 On Probably Correct Classification of Concepts. S.R. Kulkarni and O. Zeitouni 12:00 - 1:40 LUNCH Session 3: Inductive Inference; Neural Nets Chair: Bob Daley 1:40-2:00 On the Structure of Degrees of Inferability. Martin Kummer and Frank Stephan 2:00-2:20 Language Learning in Dependence on the Space of Hypotheses. Steffen Lange and Thomas Zeugmann 2:20-2:30 On the Power of Sigmoid Neural Networks. Joe Kilian and Hava T. Siegelmann 2:30-2:40 Lower Bounds on the Vapnik-Chervonenkis Dimension of Multi-layer Threshold Networks. Peter L. Bartlett 2:40-2:50 Average Case Analysis of the Clipped Hebb Rule for Nonoverlapping Perceptron Networks. Mostefa Golea and Mario Marchand 2:50-3:00 On the Power of Polynomial Discriminators and Radial Basis Function Networks. Martin Anthony and Sean B. Holden 3:00 - 3:30 BREAK 3:30-4:30 Invited Talk by Geoffrey Hinton The Minimum Description Length Principle and Neural Networks. 4:45 - ? Impromptu talks, open problems, etc. 7:00 - 10:00 pm, Banquet, barbeque pit outside Porter Dining Hall. TUESDAY, JULY 27 Session 4: Inductive Inference Chair: Rolf Wiehagen 8:20-8:40 The Impact of Forgetting on Learning Machines. Rusins Freivalds, Efim Kinber, and Carl H. Smith 8:40-8:50 On Parallel Learning. Efim Kinber, Carl H. Smith, Mahendran Velauthapillai, and Rolf Wiehagen 8:50-9:10 Capabilities of Probabilistic Learners with Bounded Mind Changes. Robert Daley and Bala Kalyanasundaram 9:10-9:20 Probability is More Powerful than Team for Language Identification from Positive Data. Sanjay Jain and Arun Sharma 9:20-9:40 Capabilities of Fallible FINite Learning. Robert Daley, Bala Kalyanasundaram, and Mahendran Velauthapillai 9:40-9:50 On Learning in the Limit and Non-uniform (epsilon, delta)-Learning. Shai Ben-David and Michal Jacovi 9:50 - 10:20 BREAK Session 5: Formal Languages, Rectangles, and Noise Chair: Takeshi Shinohara 10:20-10:40 Learning Fallible Deterministic Finite Automata. Dana Ron and Ronitt Rubinfeld 10:40-11:00 Learning Two-Tape Automata from Queries and Counterexamples. Takashi Yokomori 11:00-11:10 Efficient Identification of Regular Expressions from Representative Examples. Alvis Brazma 11:10-11:30 Learning Unions of Two Rectangles in the Plane with Equivalence Queries. Zhixiang Chen 11:30-11:50 On-line Learning of Rectangles in Noisy Environments. Peter Auer 11:50-12:00 Statistical Queries and Faulty PAC Oracles. Scott Evan Decatur 12:00 - 1:40 LUNCH Session 6: New Models; Linear Thresholds Chair: Wray Buntine 1:40-2:00 Learning an Unknown Randomized Algorithm from its Behavior. William Evans, Sridhar Rajagopalan, and Umesh Vazirani 2:00-2:20 Piecemeal Learning of an Unknown Environment. Margrit Betke, Ronald L. Rivest, and Mona Singh 2:20-2:40 Learning with Restricted Focus of Attention. Shai Ben-David and Eli Dichterman 2:40-2:50 Polynomial Learnability of Linear Threshold Approximations. Tom Bylander 2:50-3:00 Rate of Approximation Results Motivated by Robust Neural Network Learning. Christian Darken, Michael Donahue, Leonid Gurvits, and Eduardo Sontag 3:00-3:10 On the Average Tractability of Binary Integer Programming and the Curious Transition to Generalization in Learning Majority Functions. Shao C. Fang and Santosh S. Venkatesh 3:10 - 3:30 BREAK 3:30-4:30 Invited Talk by John Grefenstette Genetic Algorithms and Machine Learning 4:45 - ? Impromptu talks, open problems, etc. 7:00 - 8:30 Poster Session and Dessert Oakes Learning Center 8:30 - 10:00 Business Meeting Oakes 105 WEDNESDAY, JULY 28 Session 7: Pac Learning Chair: Yishay Mansour 8:20-8:40 On Learning Visual Concepts and DNF Formulae. Eyal Kushilevitz and Dan Roth 8:40-9:00 Localization vs. Identification of Semi-Algebraic Sets. Shai Ben-David and Michael Lindenbaum 9:00-9:20 On Learning Embedded Symmetric Concepts. Avrim Blum, Prasad Chalasani, and Jeffrey Jackson 9:20-9:30 Amplification of Weak Learning Under the Uniform Distribution. Dan Boneh and Richard J. Lipton 9:30-9:50 Learning Decision Trees on the Uniform Distribution. Thomas R. Hancock 9:50 - 10:20 BREAK Session 8: VC dimension, Learning Complexity, and Lower Bounds Chair: Sebastian Seung 10:20-10:40 Bounding the Vapnik-Chervonenkis Dimension of Concept Classes Parameterized by Real Numbers. Paul Goldberg and Mark Jerrum 10:40-10:50 Occam's Razor for Functions. B.K. Natarajan 10:50-11:00 Conservativeness and Monotonicity for Learning Algorithms. Eiji Takimoto and Akira Maruoka 11:00-11:20 Lower Bounds for PAC Learning with Queries. Gyorgy Turan 11:20-11:40 On the Complexity of Function Learning. Peter Auer, Philip M. Long, Wolfgang Maass, and Gerhard J. Woeginger 11:40-12:00 General Bounds on the Number of Examples Needed for Learning Probabilistic Concepts. Hans Ulrich Simon NOON: Check-out of Rooms 12:00 - 1:40 LUNCH Session 9: On-Line Learning Chair: Kenji Yamanishi 1:40-2:00 On-line Learning with Linear Loss Constraints. Nick Littlestone and Philip M. Long 2:00-2:10 The `Lob-Pass' Problem and an On-line Learning Model of Rational Choice. Naoki Abe and Jun-ichi Takeuchi 2:10-2:30 Worst-case Quadratic Loss Bounds for a Generalization of the Widrow-Hoff Rule. Nicolo Cesa-Bianchi, Philip M. Long, and Manfred K. Warmuth 2:30-2:40 On-line Learning of Functions of Bounded Variation under Various Sampling Schemes. S.E. Posner and S.R. Kulkarni 2:40-2:50 Acceleration of Learning in Binary Choice Problems. Yoshiyuki Kabashima and Shigeru Shinomoto 2:50-3:10 Learning Binary Relations Using Weighted Majority Voting. Sally A. Goldman and Manfred K. Warmuth 3:10 CONFERENCE ENDS 3:10 - ? Last fling on the Boardwalk. From rob at comec4.mh.ua.edu Tue Jun 8 22:35:30 1993 From: rob at comec4.mh.ua.edu (Robert Elliott Smith.dat) Date: Tue, 08 Jun 93 20:35:30 -0600 Subject: ICGA workshop proposal/participation request Message-ID: <9306090135.AA13062@comec4.mh.ua.edu> Call for Workshop Proposals and Workshop Participation ICGA-93 The Fifth International Conference on Genetic Algorithms 17-21 July, 1993 University of Illinois at Urbana-Champaign Early this Spring, the organizers of ICGA solicited proposals for workshops. Proposals for six workshops have been received and accepted thus far. These workshops are listed below. ICGA attendees are encouraged to contact the organizers of workshops in which they would like to participate. Email addresses for workshop organizers are included below. The organizers would also like to encourage proposals for additional workshops. If you would like to organize and chair a workshop, please submit a one-paragraph proposal, including a description of the workshop's topic, and some idea of how the workshop will be organized. Workshop proposals will be accepted by email only at icga93 at pele.cs.unm.edu At the ICGA91 (in San Diego), the workshops served an important role, providing smaller, less formal meetings for the discussion of specific topics related to genetic algorithms research. The organizers hope that this tradition will continue at ICGA93. ICGA93 workshops (if you wish to partipate, please write directly to the workshop's organizer): ------------------------------------------------------------------------ Genetic Programming Organizer: Kim Kinnear (kim.kinnear at sun.com) Engineering Applications of GAs (structural shape and topology optimization) Organizer: Mark Jakiela (jakiela at MIT.EDU) Discovery of long-action chains and emergence of hierarchies in classifier systems Organizers: Alex Shevorshkon Erhard Bruderer (Erhard.Bruderer at um.cc.umich.edu) Niching Methods Organizer: Alan Schultz (schultz at aic.nrl.navy.mil) Sam Mahfoud (mahfoud at gal4.ge.uiuc.edu) Combinations of GAs and Neural Nets (COGANN) Organizer: J. David Schaffer (ds1 at philabs.Philips.Com) GAs in control systems Organizer: Terry Fogarty (tc_fogar at pat.uwe-bristol.ac.uk) From Scott_Fahlman at SEF1.SLISP.CS.CMU.EDU Wed Jun 9 13:46:48 1993 From: Scott_Fahlman at SEF1.SLISP.CS.CMU.EDU (Scott_Fahlman@SEF1.SLISP.CS.CMU.EDU) Date: Wed, 09 Jun 93 13:46:48 -0400 Subject: Quick survey: Cascor and Quickprop Message-ID: Distributing code by anonymous FTP is convenient for everyone with decent internet connections, but it has the disadvantage that it is hard to keep track of who is using the code. Every so often we need to justify our existence to someone and need to show them that there are a non-trivial number of real users out there. If you are now using, or have recently used, any of my neural net algorithms or programs (Quickprop, Cascade-Correlation, Recurrent Cascade-Correlation), I would very much appreciate it if you would send me a quick E-mail message with your name, organization, and (if it's not a secret) just a few words about what you are doing with it. (For example: "classifying textures in satellite photos".) For those of you who don't know about the availability of this code (and related papers), I enclose below some instructions on how to get these things by anonymous FTP. Thanks, Scott =========================================================================== Scott E. Fahlman Internet: sef+ at cs.cmu.edu Senior Research Scientist Phone: 412 268-2575 School of Computer Science Fax: 412 681-5739 Carnegie Mellon University Latitude: 40:26:33 N 5000 Forbes Avenue Longitude: 79:56:48 W Pittsburgh, PA 15213 =========================================================================== Public-domain simulation programs for the Quickprop, Cascade-Correlation, and Recurrent Cascade-Correlation learning algorithms are available via anonymous FTP on the Internet. This code is distributed without charge on an "as is" basis. There is no warranty of any kind by the authors or by Carnegie-Mellon University. Instructions for obtaining the code via FTP are included below. If you can't get it by FTP, contact me by E-mail (sef+ at cs.cmu.edu) and I'll try *once* to mail it to you. Specify whether you want the C or Lisp version. If it bounces or your mailer rejects such a large message, I don't have time to try a lot of other delivery methods. HOW TO GET IT: For people (at CMU, MIT, and soon some other places) with access to the Andrew File System (AFS), you can access the files directly from directory "/afs/cs.cmu.edu/project/connect/code". This file system uses the same syntactic conventions as BSD Unix: case sensitive names, slashes for subdirectories, no version numbers, etc. The protection scheme is a bit different, but that shouldn't matter to people just trying to read these files. For people accessing these files via FTP: 1. Create an FTP connection from wherever you are to machine "ftp.cs.cmu.edu". The internet address of this machine is 128.2.206.173, for those who need it. 2. Log in as user "anonymous" with your own ID as password. You may see an error message that says "filenames may not have /.. in them" or something like that. Just ignore it. 3. Change remote directory to "/afs/cs/project/connect/code". NOTE: You must do this in a single operation. Some of the super directories on this path are protected against outside users. 4. At this point FTP should be able to get a listing of files in this directory with DIR and fetch the ones you want with GET. (The exact FTP commands you use depend on your local FTP server.) Partial contents: quickprop1.lisp Original Common Lisp version of Quickprop. quickprop1.c C version by Terry Regier, U. Cal. Berkeley. backprop.lisp Overlay for quickprop1.lisp. Turns it into backprop. cascor1.lisp Original Common Lisp version of Cascade-Correlation. cascor1.c C version by Scott Crowder, Carnegie Mellon rcc1.lisp Common Lisp version of Recurrent Cascade-Correlation. rcc1.c C version, trans. by Conor Doherty, Univ. Coll. Dublin nevprop1.15.shar Better quickprop implementation in C from U. of Nevada. --------------------------------------------------------------------------- Tech reports describing these algorithms can also be obtained via FTP. These are Postscript files, processed with the Unix compress/uncompress program. unix> ftp ftp.cs.cmu.edu (or 128.2.206.173) Name: anonymous Password: ftp> cd /afs/cs/project/connect/tr ftp> binary ftp> get filename.ps.Z ftp> quit unix> uncompress filename.ps.Z unix> lpr filename.ps (or however you print postscript files) For "filename", sustitute the following: qp-tr Paper on Quickprop and other backprop speedups. cascor-tr Cascade-Correlation paper. rcc-tr Recurrent Cascade-Correlation paper. precision Hoehfeld-Fahlman paper on Cascade-Correlation with limited numerical precision. --------------------------------------------------------------------------- The following are the published conference and journal versions of the above (in some cases shortened and revised): Scott E. Fahlman (1988) "Faster-Learning Variations on Back-Propagation: An Empirical Study" in (\it Proceedings, 1988 Connectionist Models Summer School}, D. S. Touretzky, G. E. Hinton, and T. J. Sejnowski (eds.), Morgan Kaufmann Publishers, Los Altos CA, pp. 38-51. Scott E. Fahlman and Christian Lebiere (1990) "The Cascade-Correlation Learning Architecture", in {\it Advances in Neural Information Processing Systems 2}, D. S. Touretzky (ed.), Morgan Kaufmann Publishers, Los Altos CA, pp. 524-532. Scott E. Fahlman (1991) "The Recurrent Cascade-Correlation Architecture" in {\it Advances in Neural Information Processing Systems 3}, R. P. Lippmann, J. E. Moody, and D. S. Touretzky (eds.), Morgan Kaufmann Pulishers, Los Altos CA, pp. 190-196. Marcus Hoehfeld and Scott E. Fahlman (1992) "Learning with Limited Numerical Precision Using the Cascade-Correlation Learning Algorithm" in IEEE Transactions on Neural Networks, Vol. 3, no. 4, July 1992, pp. 602-611. From jose at learning.siemens.com Wed Jun 9 09:21:35 1993 From: jose at learning.siemens.com (Steve Hanson,(U,6500,,p)) Date: Wed, 9 Jun 1993 09:21:35 -0400 (EDT) Subject: NIPS5 Oversight Message-ID: <0g5SDTG1GEMnEpEfFi@tractatus.siemens.com> NIPS-5 attendees: Due to an oversight we regret the inadvertent exclusion of 3 papers from the recent NIPS-5 volume. These papers were: Mark Plutowski, Garrison Cottrell and Halbert White: Learning Mackey-Glass from 25 examples, Plus or Minus 2 Yehuda Salu: Classification of Multi-Spectral Pixels by the Binary Diamond Neural Network A. C. Tsoi, D.S.C. So and A. Sergejew: Classification of Electroencephalograms using Artificial Neural Networks We are writing this note to (1) acknowledge our error (2) point out where you can obtain a present copy of the author's papers and (3) inform you that they will appear in their existing form or an updated form in NIPS Vol. 6. Presently, Morgan Kaufmann will be sending a bundle of the 3 formatted papers to all NIPS-5 attendees, these will be marked as NIPS-5 Addendum. You should also be able to retrieve an official copy from NEUROPROSE archive. Again, we apologize for the oversight to the authors. Stephen J. Hanson, General Chair Jack Cowan, Program Chair C. Lee Giles, Publications Chair From sutton at gte.com Fri Jun 11 13:44:08 1993 From: sutton at gte.com (Rich Sutton) Date: Fri, 11 Jun 93 13:44:08 -0400 Subject: Reinforcement Learning Workshop - Call for Participation Message-ID: <9306111744.AA08858@bunny.gte.com> LAST CALL FOR PARTICIPATION "REINFORCEMENT LEARNING: What We Know, What We Need" an Informal Workshop to follow ML93 (10th Int. Conf. on Machine Learning) June 30 & July 1, University of Massachusetts, Amherst Reinforcement learning is a simple way of framing the problem of an autonomous agent learning and interacting with the world to achieve a goal. This has been an active area of machine learning research for the last 5 years. The objective of this workshop is to present concisely the current state of the art in reinforcement learning and to identify and highlight critical open problems. The intended audience is all learning researchers interested in reinforcement learning. The first half of the workshop will be mainly tutorial while the second half will define and explore open problems. The entire workshop will last approximately one and three-quarters days. It is possible to register for the workshop but not the conference, but attending the conference is highly recommended as many new RL results will be presented in the conference and these will not be repeated in the workshop. Registration information is given at the end of this message. Program Committee: Rich Sutton (chair), Nils Nilsson, Leslie Kaelbling, Satinder Singh, Sridhar Mahadevan, Andy Barto, Steve Whitehead ............................................................................ PROGRAM INFORMATION The following draft program is divided into "sessions", each consisting of a set of presentations on a single topic. The earlier sessions are more "What we know" and the later sessions are more "What we Need", although some of each will be covered in all sessions. Sessions last 60-120 minutes and are separated by 30 minute breaks. Each session has an organizer and a series of speakers, one of which is likely to be the organizer herself. In most cases the speakers are meant to cover a body of work, not just their own, as a survey directed at identifying and explaining the key issues and open problems. The organizer works with the speakers to assure this (the organizer also has primary responsibility for picking the speakers, and chairs the session). ***************************************************************************** PRELIMINARY SCHEDULE: June 30: 9:00--10:30 Session 1: Defining Features of RL 10:30--11:00 Break 11:00--12:30 Session 2: RL and Dynamic Programming 12:30--2:00 Lunch 2:00--3:30 Session 3: Theory: Stochastic Approximation and Convergence 3:30--4:00 Break 4:00--5:00 Session 4: Hidden State and Short-Term Memory July 1: 9:00--11:00 Session 5: Structural Generalization: Scaling RL to Large State Spaces 11:00--11:30 Break 11:30--12:30 Session 6: Hierarchy and Abstraction 12:30--1:30 Lunch 1:30--2:30 Session 7: Strategies for Exploration 2:30--3:30 Session 8: Relationships to Neuroscience and Evolution ***************************************************************************** PRELIMINARY PROGRAM --------------------------------------------------------------------------- Session 1: Defining Features of Reinforcement Learning Organizer: Rich Sutton, rich at gte.com "Welcome and Announcements" by Rich Sutton, GTE (10 minutes) "History of RL" by Harry Klopf, WPAFB (25 minutes) "Delayed Reward: TD Learning and TD-Gammon" by Rich Sutton, GTE (50 minutes) The intent of the first two talks is to start getting across certain key ideas about reinforcement learning: 1) RL is a problem, not a class of algorithms, 2) the distinguishing features of the RL problem are trial-and-error search and delayed reward. The third talk is a tutorial presentation of temporal-difference learning, the basis of learning methods for handling delayed reward. This talk will also present Gerry Tesauro's TD-Gammon, a TD-learning system that learned to play backgammon at a grandmaster level. (There is still an outside chance that Tesauro will be able to attend the workshop and present TD-Gammon himself.) --------------------------------------------------------------------------- Session 2: RL and Dynamic Programming Organizer: Andy Barto, barto at cs.umass.edu "Q-learning" by Chris Watkins, Morning Side Inc (30 minutes) "RL and Planning" by Andrew Moore, MIT (30 minutes) "Asynchronous Dynamic Programming" by Andy Barto, UMass (30 minutes) These talks will cover the basic ideas of RL and its relationship to dynamic programming and planning. Including Markov Decision Tasks. --------------------------------------------------------------------------- Session 3: New Results in RL and Asynchronous DP Organizer: Satinder Singh, singh at cs.umass.edu "Introduction, Notation, and Theme" by Satinder P. Singh, UMass "Stochastic Approximation: Convergence Results" by T Jaakkola & M Jordan, MIT "Asychronous Policy Iteration" by Ron Williams, Northeastern "Convergence Proof of Adaptive Asynchronous DP" by Vijaykumar Gullapalli, UMass "Discussion of *some* Future Directions for Theoretical Work" by ? This session consists of two parts. In the first part we present a new and fairly complete theory of (asymptotic) convergence for reinforcement learning (with lookup tables as function approximators). This theory explains RL algorithms as replacing the full-backup operator of classical dynamic programming algorithms by a random backup operator that is unbiased. We present an extension to classical stochastic approximation theory (e.g., Dvoretzky's) to derive probability one convergence proofs for Q-learning, TD(0), and TD(lambda), that are different, and perhaps simpler, than previously available proofs. We will also use the stochastic approximation framework to highlight the contribution made by reinforcement learning algorithms such as TD, and Q-learning, to the entire class of iterative methods for solving the Bellman equations associated with Markovian Decision Tasks. The second part deals with contributions by RL researchers to asynchronous DP. Williams will present a set of algorithms (and convergence results) that are asynchronous at a finer grain than classical asynchronous value iteration, but still use "full" backup operators. These algorithms are related to the modified policy iteration algorithm of Puterman and Shin, as well as to the ACE/ASE (actor-critic) architecture of Barto, Sutton and Anderson. Subsequently, Gullapalli will present a proof of convergence for "adaptive" asynchronous value iteration that shows that in order to ensure convergence with probability one, one has to place constraints on how many model-building steps have to be be performed between two consecutive updates of the value function. Lastly we will discuss some pressing theoretical questions regarding rate of convergence for reinforcement learning algorithms. --------------------------------------------------------------------------- Session 4: Hidden State and Short-Term Memory Organizer: Lonnie Chrisman, lonnie.chrisman at cs.cmu.edu Speakers: Lonnie Chrisman & Michael Littman, CMU Many realistic agents cannot directly observe every relevant aspect of their environment at every moment in time. Such hidden state causes problems for many reinforcement learning algorithms, often causing temporal differencing methods to become unstable and making policies that simply map sensory input to action insufficient. In this session we will examine the problems of hidden state and of learning how to best organize short-term memory. I will review and compare existing approaches such as those of Whitehead & Ballard, Chrisman, Lin & Mitchell, McCallum, and Ring. I will also give a tutorial on the theories of Partially Observable Markovian Decision Processes, Hidden Markov Models, and related learning algorithms such as Balm-Welsh/EM as they are relevant to reinforcement learning. Note: Andrew McCallum will present a paper on this topic as part of the conference; that material will not be repeated in the workshop. --------------------------------------------------------------------------- Session 5: Structural Generalization: Scaling RL to Large State Spaces Organizer: Sridhar Mahadevan, sridhar at watson.ibm.com "Motivation and Introduction" by Sridhar Mahadevan, IBM "Neural Nets" by Long-Ji Lin, Siemens "CMAC" by Tom Miller, Univ. New Hampshire "Kd-trees and CART" by Marcos Salganicoff, UPenn "Learning Teleo-Reactive Trees" by Nils Nilsson, Stanford "Function Approximation in RL: Issues and Approaches" by Richard Yee, UMass "RL with Analog State and Action Vectors", Leemon Baird, WPAFB RL is slow to converge in tasks with high-dimensional continuous state spaces, particularly given sparse rewards. One fundamental issue in scaling RL to such tasks is structural credit assignment, which deals with inferring rewards in novel situations. This problem can be viewed as a supervised learning task, the goal being to learn a function from instances of states, actions, and rewards. Of course, the function cannot be stored exhaustively as a table, and the challenge is devise more compact storage methods. In this session we will discuss some of the different approaches to the structural generalization problem. Note: Steve Whitehead & Rich Sutton will present a paper on this topic as part of the confernece; that material will not be repeated in the workshop. --------------------------------------------------------------------------- Session 6: Hierarchy and Abstraction Organizer: Leslie Kaelbling, lpk at cs.brown.edu Speakers: To be determined Too much of RL is concerned with low-level actions and low-level (single time step) models. How can we model the world, and plan about actions, at a higher level, or over longer time scales? How can we integrate models and actions at different time scales and levels of abstraction? To address these questions, several researchers have proposed models of hierarchical learning and planning, e.g., Satinder Singh, Mark Ring, Chris Watkins, Long-ji Lin, Leslie Kaelbling, and Peter Dayan & Geoff Hinton. The format for this session will be a brief introduction to the problem by the session organizer followed by short talks and discussion. Speakers have not yet been determined. Note: Kaelbling will also speak on this topic as part of the conference; that material will not be repeated in the workshop. ----------------------------------------------------------------------------- Session 7: Strategies for Exploration Organizer: Steve Whitehead, swhitehead at gte.com Exploration is essential to reinforcement learning, since it is through exploration, that an agent learns about its environment. Naive exploration can easily result in intractably slow learning. On the other hand, exploration strategies that are carefully structured or exploit external sources of bias can do much better. A variety of approaches to exploration have been devised over the last few years (e.g., Kaelbling, Sutton, Thrun, Koenig, Lin, Clouse, Whitehead). The goal of this session is to review these techniques, understand their similarities and differences, understand when and why they work, determine their impact on learning time, and to the extent possible organize them taxonomically. The session will consist of a short introduction by the session organizer followed by a open discussion. The discussion will be informal but aimed at issues raised during the monologue. An informal panel of researchers will be on hand to participate in the discussion and answer questions about their work in this area. ----------------------------------------------------------------------------- Session 8: Relationships to Neuroscience and Evolution Organizer: Rich Sutton, rich at gte.com We close the workshop with a reminder of RL's links to neuroscience and to Genetic Algorithms / Classifier Systems: "RL in the Brain: Developing Connections Through Prediction" by R Montague, Salk "RL and Genetic Classifier Systems" by Stewart Wilson, Roland Institute Abstract of first talk: Both vertebrates and invertebrate possess diffusely projecting neuromodulatory systems. In the vertebrate, it is known that these systems are involved in the development of cerebral cortical structures and can deliver reward and/or salience signals to the cerebral cortex and other structures to influence learning in the adult. Recent data in primates suggest that this latter influence obtains because changes in firing in nuclei that deliver the neuromodulators reflect the difference in the predicted and actual reward, i.e., a prediction error. This relationship is qualitatively similar to that predicted by Sutton and Barto's classical conditioning theory. These systems innervate large expanses of cortical and subcortical turf through extensive axonal projections that originate in midbrain and basal forebrain nuclei and deliver such compounds as dopamine, serotonin, norepinephrine, and acetylcholine to their targets. The small number of neurons comprising these subcortical nuclei relative to the extent of the territory their axons innervate suggests that the nuclei are reporting scalar signals to their target structures. These facts are synthesized into a single framework which relates the development of brain structures and conditioning in adult brains. We postulate a modification to Hebbian accounts of self-organization: Hebbian learning is conditional on a incorrect prediction of future delivered reinforcement from a diffuse neuromodulatory system. The reinforcement signal is derived both from externally driven contingencies such as proprioception from eye movements as well as from internal pathways leading from cortical areas to subcortical nuclei. We suggest a specific model for how such predictions are made in the vertebrate and invertebrate brain. We illustrate the framework with examples ranging from the development of sensory and sensory-motor maps to foraging behavior in bumble-bees. ****************************************************************************** GENERAL INFO ON REGISTERING FOR ML93 AND WORKSHOPS: Tenth International Conference on Machine Learning (ML93) --------------------------------------------------------- The conference will be held at the University of Massachusetts in Amherst, Massachusetts, from June 27 (Sunday) through June 29 (Tuesday). The con- ference will feature four invited talks and forty-six paper presentations. The invited speakers are Leo Breiman (U.C. Berkeley, Statistics), Micki Chi (U. Pittsburgh, Psychology), Michael Lloyd-Hart (U. Arizona, Adaptive Optics Group of Steward Observatory), and Pat Langley (Siemens, Machine Learning). Following the conference, there will be three informal workshops: Workshop #A: Reinforcement Learning: What We Know, What We Need (June 30 - July 1) Organizers: R. Sutton (chair), N. Nilsson, L. Kaelbling, S. Singh, S. Mahadevan, A. Barto, S. Whitehead Workshop #B: Fielded Applications of Machine Learning (June 30 - July 1) Organizers: P. Langley, Y. Kodratoff Workshop #C: Knowledge Compilation and Speedup Learning (June 30) Organizers: D. Subramanian, D. Fisher, P. Tadepalli Options and fees: Conference registration fee $140 regular $110 student Breakfast/lunch meal plan (June 27-29) $33 Dormitory housing (nights of June 26-28) $63 single occupancy $51 double occupancy Workshop A (June 30-July 1) $40 Workshop B (June 30-July 1) $40 Breakfast/lunch meal plan (June 30-July 1) $22 Dormitory housing (nights of June 29-30) $42 single occupancy $34 double occupancy Workshop C (June 30) $20 Breakfast/lunch meal plan (June 30) $11 Dormitory housing (night of June 29) $21 single occupancy $17 double occupancy Administrative fee (required) $10 Late fee (received after May 10) $30 To obtain a FAX of the registration form, send an email request to Paul Utgoff ml93 at cs.umass.edu or utgoff at cs.umass.edu From gary at cs.ucsd.edu Fri Jun 11 18:53:33 1993 From: gary at cs.ucsd.edu (Gary Cottrell) Date: Fri, 11 Jun 93 15:53:33 -0700 Subject: NIPS5 Oversight Message-ID: <9306112253.AA06151@odin.ucsd.edu> FYI, to retrieve Plutowski, Cottrell and White: Learning Mackey-Glass from 25 examples, Plus or Minus 2 The file on neuroprose is: pluto.nips92.ps.Z A script file is attached at the end of this note. Gary Cottrell 619-534-6640 Reception: 619-534-6005 FAX: 619-534-7029 Computer Science and Engineering 0114 University of California San Diego La Jolla, Ca. 92093 gary at cs.ucsd.edu (INTERNET) gcottrell at ucsd.edu (BITNET, almost anything) ..!uunet!ucsd!gcottrell (UUCP) RE: From gjacobs at qualcomm.com Fri Jun 11 14:00:40 1993 From: gjacobs at qualcomm.com (Gary Jacobs) Date: Fri, 11 Jun 1993 11:00:40 -0700 Subject: WCCI '94 Announcement and Call for Papers Message-ID: <9306111801.AA17745@harvey> Gary Jacobs gjacobs at qualcomm.com (619)597-5029 voice (619)452-9096 fax HARD FACT IN A WORLD OF FANTASY A world of sheer fantasy awaits your arrival at the IEEE World Congress on Computational Intelligence next year; our host is Walt Disney World in Orlando Florida. Simultaneous Neural Network, Fuzzy Logic and Evolutionary Programming conferences will provide an unprecedented opportunity for technical development while the charms of the nearby Magic Kingdom and Epcot Center attempt to excite your fancies. The role imagination has played in the development of Computational Intelligence techniques is well known; before they became "innovative" the various CI technologies were dismissed as "fantasies" of brilliant minds. Now these tools are real; perhaps it's only appropriate that they should be further explored and their creators honored in a world of the imagination, a world where dreams come true. Share your facts at Disney World; share your imagination. Come to the IEEE World Congress on Computational Intelligence. It's as new as tomorrow. ___________________________________________________________________________ ***CALL FOR PAPERS*** ___________________________________________________ IEEE WORLD CONGRESS ON COMPUTATIONAL INTELLIGENCE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * IEEE International Conference on Neural Networks * * FUZZ/IEEE '94 * * IEEE International Symposium on Evolutionary Computation * June 26 - July 2, 1994 Walt Disney World Dolphin Hotel, Lake Buena Vista, Florida Sponsored by the IEEE Neural Networks Council --------------------------------------------------------------------- IEEE INTERNATIONAL CONFERENCE ON NEURAL NETWORKS Steven K. Rogers, General Chair rogers at afit.af.mil Topics: Applications, architectures, artificially intelligent neural networks, artificial life, associative memory, computational intelligence, cognitive science, embedology, filtering, fuzzy neural systems, hybrid systems, image processing, implementations, intelligent control, learning and memory, machine vision, motion analysis, neurobiology, neurocognition, neurodynamics, optimization, pattern recognition, prediction, robotics, sensation and perception, sensorimotor systems, speech, hearing and language, system identification, supervised and unsupervised learning, tactile sensors, and time series analysis. ------------------------------------------- FUZZ/IEEE '94 Piero P. Bonissone, General Chair bonissone at crd.ge.ge.com Topics: Basic principles and foundations of fuzzy logic, relations between fuzzy logic and other approximate reasoning methods, qualitative and approximate-reasoning modeling, hardware implementations of fuzzy- logic algorithms, design, analysis, and synthesis of fuzzy-logic controllers, learning and acquisition of approximate models, relations between fuzzy logic and neural networks, integration of fuzzy logic and neural networks, integration of fuzzy logic and evolutionary computing, and applications. ------------------------------------------- IEEE CONFERENCE ON EVOLUTIONARY COMPUTATION Zbigniew Michalewicz, General Chair zbyszek at mosaic.uncc.edu Topics: Theory of evolutionary computation, evolutionary computation applications, efficiency and robustness comparisons with other direct search algorithms, parallel computer applications, new ideas incorporating further evolutionary principles, artificial life, evolutionary algorithms for computational intelligence, comparisons between different variants of evolutionary algorithms, machine learning applications, evolutionary computation for neural networks, and fuzzy logic in evolutionary algorithms. --------------------------------------------------------------------- INSTRUCTIONS FOR ALL THREE CONFERENCES Papers must be received by December 10, 1993. Papers will be reviewed by senior researchers in the field, and all authors will be informed of the decisions at the end of the review proces. All accepted papers will be published in the Conference Proceedings. Six copies (one original and five copies) of the paper must be submitted. Original must be camera ready, on 8.5x11-inch white paper, one-column format in Times or similar fontstyle, 10 points or larger with one-inch margins on all four sides. Do not fold or staple the original camera-ready copy. Four pages are encouraged. The paper must not exceed six pages including figures, tables, and references, and should be written in English. Centered at the top of the first page should be the complete title, author name(s), affiliation(s) and mailing address(es). In the accompanying letter, the following information must be included: 1) Full title of paper, 2) Corresponding authors name, address, telephone and fax numbers, 3) First and second choices of technical session, 4) Preference for oral or poster presentation, and 5) Presenter's name, address, telephone and fax numbers. Mail papers to (and/or obtain further information from): World Congress on Computational Intelligence, Meeting Management, 5665 Oberlin Drive, #110, San Diego, California 92121, USA (email: 70750.345 at compuserve.com, telephone: 619-453-6222). From gasser at cs.indiana.edu Mon Jun 14 11:06:46 1993 From: gasser at cs.indiana.edu (Michael Gasser) Date: Mon, 14 Jun 1993 10:06:46 -0500 Subject: TR on language acquisition Message-ID: FTP-host: cs.indiana.edu (129.79.254.191) FTP-filename: /pub/techreports/TR384.ps.Z The following paper is available in compressed postscript form by anonymous ftp from the Indiana University Computer Science Department ftp archive (see above). The paper is 60 pages long. Hardcopies won't be available till September, I'm afraid. Comments welcome. Michael Gasser gasser at cs.indiana.edu ================================================================= Learning Words in Time: Towards a Modular Connectionist Account of the Acquisition of Receptive Morphology Michael Gasser Computer Science and Linguistics Departments Indiana University To have learned the morphology of a natural language is to have the capacity both to recognize and to produce words consisting of novel combinations of familiar morphemes. Most recent work on the acquisition of morphology takes the perspective of production, but it is receptive morphology which comes first in the child. This paper presents a connectionist model of the acquisition of the capacity to recognize morphologically complex words. The model takes sequences of phonetic segments as inputs and maps them onto output units representing the meanings of lexical and grammatical morphemes. It consists of a simple recurrent network with separate hidden-layer modules for the tasks of recognizing the root and the grammatical morphemes of the input word. Experiments with artificial language stimuli demonstrate that the model generalizes to novel words for morphological rules of all but one of the major types found in natural languages and that a version of the network with unassigned hidden-layer modules can learn to assign them to the output recognition tasks in an efficient manner. I also argue that for rules involving reduplication, that is, the copying of portions of a root, the network requires separate recurrent subnetworks for sequences of larger units such as syllables. The network can learn to develop its own syllable representations which not only support the recognition of reduplication but also provide the basis for learning to produce, as well as recognize, morphologically complex words. The model makes many detailed predictions about the learning difficulty of particular morphological rules. From dlovell at s1.elec.uq.oz.au Tue Jun 15 16:07:13 1993 From: dlovell at s1.elec.uq.oz.au (David Lovell) Date: Tue, 15 Jun 93 15:07:13 EST Subject: ACNN'94 Call for papers Message-ID: <9306150507.AA21234@c10.elec.uq.oz.au> First Announcement ACNN'94 FIFTH AUSTRALIAN CONFERENCE ON NEURAL NETWORKS 31st Jan - 2nd Feb 1994 Univeristy of Queensland St Lucia Queensland AUSTRALIA The Fifth Australian Conference on Neural Networks will be held in Brisbane on 31st January - 2nd February, 1994, at the University of Queensland. ACNN'94 is the annual national meeting of the Australian neural network community. It is a multi-disciplinary meeting and seeks contributions from Neuroscientists, Engineers, Computer Scientists, Mathematicians, Physicists and Psychologists. ACNN'94 will feature an invited keynote speaker. The program will include presentations and poster sessions. Proceedings will be printed and distributed to the attendees. Keynote Speaker Professor Teuvo Kohonen, Helsinki Technical University Call for Papers Papers on innovative applications of artificial neural networks and other research topics in ANNs are sought. Research topics of interest include, but not restricted to: Neuroscience: Integrative function of neural networks in vision, audition, motor, somatosensory and autonomic functions. Synaptic function: Cellular information processing; Theory: Learning; Generalisation; Complexity; Scaling; Stability; Dynamics; Implementation: Hardware implementation of neural networks; Analog and digital VLSI implementation; Optical implementation; Architecture and Learning Algorithms: New Architectures and learning algorithms; Hierarchy; Modularity; Learning pattern sequences; information integration; Cognitive Science and AI: Computational models of cognition and perception; Reasoning; Concept formation; Language acquisition; Neural network implementation of expert systems; Applications: Application of neural networks to signal processing and analysis; Pattern recognition; Speech, machine vision; Motor control; Robotics. The conference will also include a poster stream which is designed as a forum for presentation of work in an environment which will encourage informal discussions about methodologies and techniques. The posters will be displayed for a significant period of time, and time will be allocated for authors to be present at their poster in the conference program. Software demonstrations will be possible for authors who bring with them portable computers. The organizing committee will attempt to provide full-size compatible screens. Limited video facilities may be available. As in previous years, there will be a special poster session for postgraduate students. All students are encouraged to present research issues and preliminary findings in this session. Initial Submission of Papers As this is a multi-disciplinary meeting, papers are required to be comprehensible to an informed researcher outside the particular stream of the author in addition to the normal requirements of technical merit. Papers should be submitted as close as possible to final form and must not exceed four A4 pages (2-column format). The first page should include the title and abstract, and should leave space for, but not include the authors' names and affiliations. A cover page should be supplied giving the title of the paper, the name, and affiliation of each author, together with the postal address, the electronic mail address, the phone number, and the fax number of a designated contact author. The type font should be no smaller than 10 point except in footnotes. A serif font such as Times to New Century Schoolbook is preferred. Four copies of the paper and the front cover should be supplied. This initial submission must be on hard copy to reach us by Friday, 27th August, 1993. Each manuscript should clearly indicate submission category (from the six as listed) and author preference for oral or poster presentations. Papers should be sent to: Tracey Carney ACNN'94 Secretariat, Department of Electrical and Computer Engineering University of Queensland St Lucia, Queensland 4072 Australia. Submission Deadlines Friday, 27 August 1993 Deadline for receipt of paper submissions Friday 29th October 1993 Notification of acceptance/rejection Friday 10 December 1993 Final papers ready for camera-ready form for printing Monday 24th January 1994 Deadline for receipt of Student Session abstracts Venue University of Queensland, Brisbane, Australia ACNN'94 Organizing Committee General Chair Ah Chung Tsoi (Univ of Queensland) Technical Chair Tom Downs (Univ of Queensland) Technical Committee Yianni Attikouzel (Univ of Western Aust) Peter Bartlett (Aust National Univ) Robert Bogner (Univ of Adelaide) Terry Caelli (Univ of Melbourne) Max Coltheart (Macquarie Univ) George Coghill (Univ of Auckland) Phil Diamond (Univ of Queensland) Joachim Diederich (Queensland Univ of Tech) Tom Downs (Univ of Queensland) Simon Goss (Def Scient & Tech Org) Graeme Halford (Univ of Queensland) Richard Heath (Univ of Newcastle) Michael Humphreys (Univ of Queensland) Marwan Jabri (Sydney Univ) Andrew Jennings (Telecom) Bill Levick (Aust National Univ) Adam Kowalczyk (Telecom) Dennis Longstaff (Univ of Queensland) D Nandagopal (Def Scient & Tech Org) M Palaniswami (Univ of Melbourne) Jack Pettigrew (Univ of Queensland) Nick Redding (Def Scient & Tech Org) Janet Wiles (Univ of Queensland) Robert Williamson (Aust National Univ) Local Committee (tentative) Andrew Back (Univ of Queensland) Phil Diamond (Univ of Queensland) Joachim Diederich (Queensland Univ of Tech) Shlomo Geva (Queensland Univ of Tech) Graeme Halford (Univ of Queensland) Michael Humphreys (Univ of Queensland) Ray Lister (Univ of Queensland) Brian Lovell (Univ of Queensland) David Lovell (Univ of Queensland) Mark Schulz (Univ of Queensland) Joaquin Sitte (Queensland Univ of Tech) Guy Smith (Univ of Queensland) Janet Wiles (Univ of Queensland) Robert Young (Queensland Dept of Primary Industries) Registration The registration fee to attend ACNN'94 is: Full Time Students A $80 + $30 Late fee Academics A $200 + $50 Late fee Other A $275 + $75 Late fee Late fees will apply to any registrations posted after December 10th 1993. To be eligible for the Full Time Student rate, a letter from the Head of Department as verification of enrolment is required. Accommodation Accommodation has been block booked at King's College (to be confirmed), University of Queensland. Copies of this brochure and registration forms are available from the Secretariat Postgraduate Students Poster Session Students are invited to submit research papers to the conference program in the normal way. However, there are many cases where preliminary work is not in a suitable form for the main program, or is not complete by the submission deadline. Thus, as in past years, there will be a special poster session for students. The main aim of the session is to promote interaction among students, and to give students a chance to present their thesis research, and gain feedback from fellow students and conference attendees. All students are accepted for the session, and indeed, all are strongly encouraged to participate. To take part in the session, send 1-page giving title of paper, name, affiliation, email address, and a 200-word abstract outlining the content of the poster, to Dr Janet Wiles Department of Computer Science, University of Queensland, QLD 4072, Australia, by Monday 24th January, 1994. For questions regarding the student session, write to the above address, or email janetw at cs.uq.oz.au Note that abstracts for the student session are not referreed and will not appear in the conference proceedings. From lpratt at franklinite.Mines.Colorado.EDU Tue Jun 15 12:13:39 1993 From: lpratt at franklinite.Mines.Colorado.EDU (Lorien Y. Pratt) Date: Tue, 15 Jun 93 09:13:39 -0700 Subject: Thesis available: Transfer between neural networks Message-ID: <9306151613.AA05154@franklinite.Mines.Colorado.EDU> FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/pratt.thesis.ps.Z The file pratt.thesis.ps.Z is now available for copying from the Neuroprose repository: Transferring Previously Learned Back Propagation Networks to New Learning Tasks 144 pages Lorien Y. Pratt Colorado School of Mines (Dissertation written at Rutgers University) ABSTRACT: When people learn a new task, they often build on their ability to solve related problems. For example, a doctor moving to a new country can use prior experience to aid in diagnosing patients. A chess player can use experience with one set of end-games to aid in solving a different, but related, set. However, although people are able to perform this sort of skill transfer between tasks, most neural network training methods in use today are unable to build on their prior experience. Instead, every new task is learned from scratch. This dissertation explores how a back-propagation neural network learner can build on its previous experience. We present an algorithm, called Discriminability-Based Transfer (DBT), that facilitates the transfer of information from the learned weights of one network to the initial weights of another. Through evaluation of DBT on several benchmark tasks we demonstrate that it can speed up learning on a new task. We also show that DBT is more robust than simpler methods for transfer. -------- Sample session for obtaining this file via anonymous ftp: > ftp archive.cis.ohio-state.edu Name: anonymous Password: (type your email address her) ftp> cd pub/neuroprose ftp> binary ftp> get pratt.thesis.ps.Z ftp> quit > uncompress pratt.thesis.ps.Z > lpr pratt.thesis.ps --------- For this with email but no ftp access, you can use the mail server at Rutgers, where this document is ML-TR-37. Send email to mth at cs.rutgers.edu to request the document, or send the word `help' in a message to: ftpmail at decwrl.dec.com --------- Those without any network access at all can receive copies of this document by requesting ML-TR-37 from the following address: Technical Reports Librarian Computer Science Department Rutgers University New Brunswick, NJ 08903 USA [Currently there is no charge.] Dr. L. Y. Pratt Dept. of Math and Computer Science lpratt at mines.colorado.edu Colorado School of Mines (303) 273-3878 (work) 402 Stratton (303) 278-4552 (home) Golden, CO 80401, USA Note: I'll be travelling out of the country for the next month. If you have trouble printing this paper, I'll be available to help after I return on July 16. Send me email -Lori From cowan at synapse.uchicago.edu Tue Jun 15 18:47:17 1993 From: cowan at synapse.uchicago.edu (Jack Cowan) Date: Tue, 15 Jun 93 15:47:17 -0700 Subject: loss of Ed Posner Message-ID: I am sorry to have to transmit the very sad news that Ed Posner, the President of the NIPS Foundation, was killed this morning in a bicycling accident in Pasadena. All his many friends and colleagues will want to join me in expressing our painful feelings at such a tragic loss. Jack Cowan From biehl at connect.nbi.dk Wed Jun 16 17:21:12 1993 From: biehl at connect.nbi.dk (Michael Biehl) Date: Wed, 16 Jun 93 17:21:12 WETDST Subject: No subject Message-ID: FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/biehl.unsupervised.ps.Z *** Hardcopies cannot be provided *** The following paper has been placed in the Neuroprose archive (see above for ftp-host) in file biehl.unsupervised.ps.Z (8 pages of output) email address of author: biehl at physik.uni-wuerzburg.de ---------------------------------------------------------------- "An exactly solvable model of unsupervised learning" by Michael Biehl Abstract: A model for unsupervised learning from N-dimensional data is studied. Random training examples are drawn such that the distribution of their overlaps with a vector B is a mixture of two Gaussians of unit width and a separation rho. A student vector is generated by an online algorithm, using each example only once. The evolution of its overlap with B can be calculated exactly in the thermodynamic limit (infinite N). As a specific example Oja's rule is investigated. Its dynamics and approach to the stationary solution are solved for both a constant and an optimally chosen time-dependent learning rate. ----------------------------------------------------------------- From mdavies at psy.ox.ac.uk Thu Jun 17 11:20:34 1993 From: mdavies at psy.ox.ac.uk (Martin Davies) Date: Thu, 17 Jun 93 11:20:34 BST Subject: Euro-SPP 93: Registration and Programme Message-ID: <9306171020.AA16978@uk.ac.oxford.psy> EUROPEAN SOCIETY FOR PHILOSOPHY AND PSYCHOLOGY The Second Annual Meeting of the European Society for Philosophy and Psychology will be held at the University of Sheffield, England, from the afternoon of Saturday 3 July to the morning of Tuesday 6 July, 1993. It is still possible to register for this conference. The Registration Fee is 15 pounds sterling (or 10 pounds for students). The cost of meals and accommodation is 120 pounds sterling. For participants not requiring accommodation, the cost of meals is 70 pounds. In order to register, please contact: Professor Peter Carruthers Hang Seng Centre for Cognitive Studies Department of Philosophy University of Sheffield Sheffield S10 2TN UK Email: P.Carruthers at sheffield.ac.uk Fax: +44-742-824604 **************************************************************** PROGRAMME All sessions will take place in Stephenson Hall of Residence, Oakholme Road, Sheffield S10 3DG. SATURDAY 3 JULY Conference desk open from 12 noon 3.00 - 5.00 pm SYMPOSIUM 1: Body and Self Chair: Naomi Eilan (Philosophy, KCRC Cambridge) Speakers: John Campbell (Philosophy, Oxford) Anthony Marcel (Psychology, APU Cambridge) Michael Martin (Philosophy, London) 5.00 - 5.30 pm Tea 6.15 - 7.45pm INVITED LECTURE 1 Chair: Larry Weiskrantz (Psychology, Oxford) Speaker: Ruth Millikan (Philosophy, Connecticut) 'Synthetic Concepts: A Philosopher's Thoughts about Categorization' 8.15 pm DINNER in Firth Hall, University of Sheffield SUNDAY 4 JULY 9.00 - 11.00 am SYMPOSIUM 2: Explanation by Intentional States Chair: Christopher Peacocke (Philosophy, Oxford) Speakers: John Campbell (Philosophy, Oxford) Pascal Engel (Philosophy, CREA Paris) Gabriel Segal (Philosophy, London) 11.00 - 11.30 am Coffee 11.30 am - 1.00 pm INVITED LECTURE 2 Chair: Speaker: Marc Jeannerod (Psychology, INSERM Lyon) 'The Representing Brain: Neural Correlates of Motor Intention and Imagery' 1.00 - 2.00 pm LUNCH 2.00 - 4.00 pm SUBMITTED PAPERS Chair: Martin Davies (Philosophy, Oxford) Speaker: Manuel Garcia Carpintero (Philosophy, Barcelona) 'The Teleological Account of Content' Comments: Mike Oaksford (Psychology, Bangor) Speaker: Gregory Mulhauser (Philosophy, Edinburgh) 'Chaotic Dynamics and Introspectively Transparent Brain Processes' Comments: Peter Smith (Philosophy, Sheffield) 4.00 - 4.30 pm Tea 4.30 - 6.00 pm INVITED LECTURE 3 Chair: Speaker: Deirdre Wilson (Linguistics, London) 'Truth, Coherence and Relevance' 6.15 pm BUSINESS MEETING followed by a RECEPTION 8.00 pm DINNER at Stephenson Hall of Residence MONDAY 5 JULY 9.00 - 10.45 am SYMPOSIUM 3: The Autonomy of Social Explanation Chair: Daniel Andler (Philosophy, CREA Paris) Speakers: Pascal Boyer (Anthropology, KCRC Cambridge) John Shotter (Communication, New Hampshire) Chris Sinha (Psychology, Aarhus) 10.45 - 11.15 am Coffee 11.15 am - 1.00 pm ROUND TABLE: Neuropsychological Approaches Chair: Beatrice de Gelder (Philosophy and Psychology, Tilburg and Brussels) Speakers: Marcel Kinsbourne (Psychology, Tufts) David Perrett (Psychology, St. Andrews) Tim Shallice (Psychology, London) 1.00 - 2.00 pm LUNCH 2.00 - 4.00 pm SUBMITTED PAPERS Chair: Anthony Marcel (Psychology, APU Cambridge) Speaker: Leslie Stevenson (Philosophy, St. Andrews) 'Merleau-Ponty on the Epistemology of Touch' Comments: Naomi Eilan (Philosophy, KCRC Cambridge) Speaker: Thomas Metzinger (Philosophy, Giessen) 'Subjectivity and Mental Representation' Comments: Barry Smith (Philosophy, London) 4.00 - 4.30 pm Tea 4.30 - 6.15 pm SYMPOSIUM 4: Mindblindness: Autism and Theory of Mind Chair: Peter Carruthers (Philosophy, Sheffield) Speakers: Simon Baron-Cohen (Psychology, London) Juan Carlos Gomez (Psychology, Madrid) Pierre Jacob (Philosophy, CREA Paris) 6.30 pm A visit to Chatsworth House, including DINNER TUESDAY 6 JULY Depart after breakfast From mm at santafe.edu Fri Jun 18 14:35:36 1993 From: mm at santafe.edu (mm@santafe.edu) Date: Fri, 18 Jun 93 12:35:36 MDT Subject: paper available Message-ID: <9306181835.AA09937@lyra> The following paper is available by public ftp. Dynamics, Computation, and the ``Edge of Chaos'': A Re-Examination Melanie Mitchell James P. Crutchfield Peter T. Hraber Santa Fe Institute UC Berkeley Santa Fe Institute Santa Fe Institute Working Paper 93-06-040 Abstract In this paper we review previous work and present new work concerning the relationship between dynamical systems theory and computation. In particular, we review work by Langton (1990) and Packard (1988) on the relationship between dynamical behavior and computational capability in cellular automata (CA). We present results from an experiment similar to the one described in Packard (1988), that was cited there as evidence for the hypothesis that rules capable of performing complex computations are most likely to be found at a phase transition between ordered and chaotic behavioral regimes for CA (the "edge of chaos"). Our experiment produced very different results from the original experiment, and we suggest that the interpretation of the original results is not correct. We conclude by discussing general issues related to dynamics, computation, and the "edge of chaos" in cellular automata. To appear in G. Cowan, D. Pines, and D. Melzner (editors), _Integrative Themes_. Santa Fe Institute Stuides in the Sciences of Complexity, Proceedings Volume 19. Reading, MA: Addison-Wesley. Note: This paper is a much shorter version of our paper "Revisiting the Edge of Chaos" (SFI Working Paper 93-03-014, announced previously on this newsgroup). It contains an expanded review of previous work on relationships between dynamical systems theory and computation. To obtain an electronic copy: ftp santafe.edu login: anonymous password: cd /pub/Users/mm binary get sfi-93-06-040.ps.Z quit Then at your system: uncompress sfi-93-06-040.ps.Z lpr -P sfi-93-06-040.ps To obtain a hard copy (only if you cannot obtain an electronic copy), send a request to dlu at santafe.edu. From maass at igi.tu-graz.ac.at Mon Jun 21 09:43:41 1993 From: maass at igi.tu-graz.ac.at (Wolfgang Maass) Date: Mon, 21 Jun 93 15:43:41 +0200 Subject: new paper in neuroprose Message-ID: <9306211343.AA28749@figids01.tu-graz.ac.at> FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/maass.super.ps.Z The file maass.super.ps.Z is now available for copying from the Neuroprose repository. This is a 7-page long paper. Hardcopies are not available. NEURAL NETS WITH SUPERLINEAR VC-DIMENSION by Wolfgang Maass Institute for Theoretical Computer Science Technische Universitaet Graz, A-8010 Graz, Austria email: maass at igi.tu-graz.ac.at Abstract: We construct arbitrarily large feedforward neural nets of depth 3 (i.e. with 2 hidden layers) and O(w) edges that have a VC-dimension of at least w log w. The same construction can be carried out for any depth larger than 3. This construction proves that the well-known upper bound for the VC-dimension of a neural net by Cover, Baum, and Haussler is in fact asymptotically optimal for any depth 3 or larger. The Vapnik-Chervonenkis dimension (VC-dimension) is an important parameter of any neural net, since it predicts how many training examples are needed for training the net (in Valiant's model for probably approximately correct learning). One may also view our result as mathematical evidence for some type of "connectionism thesis": that a network of neuron-like elements is more than just the sum of its elements. Our result shows that in a large neural net a single weight contributes more than a constant to the VC-dimension of the neural net, and that its contribution may increase with the total size of the neural net. The current paper improves the corresponding result by the author from last year (for depth 4), and it provides the first complete write-up of the construction. From mehra at ptolemy.arc.nasa.gov Mon Jun 21 18:58:56 1993 From: mehra at ptolemy.arc.nasa.gov (Pankaj Mehra) Date: Mon, 21 Jun 93 15:58:56 PDT Subject: edited collection of ANN papers; discount Message-ID: <9306212258.AA12857@tatertot.arc.nasa.gov> Fellow Connectionists: Some of you may have already seen ``Artificial Neural Networks: Concepts and Theory,'' edited by [yours truly] and Ben Wah. It was published by IEEE Computer Society Press in August, 1992. The table of contents are attached at the end of this message. The book is hardback and has 667 pages of which approx 100 are from chapter introductions written by the editors. The list price is $70 [$55 for IEEE members]. My intent in sending this message is not so much to announce the availability of our book as it is to bring to your notice the following offer of discount: If I place an order, I get an author's discount of 40% off list price; if a school bookstore places the order, they get a 32% discount. The IEEE order no. for the book is 1997; 1-800-CS-BOOKS. If you are planning to teach a graduate course on neural networks, you will probably find that our collection of papers as well as the up-to-date bibliography at the end of each chapter introduction provide excellent starting points for independent research. -Pankaj Mehra 415/604-0165 mehra at ptolemy.arc.nasa.gov NASA - Ames Research Center, M/S 269-3 Moffett Field, CA 94035-1000 USA __________________________________________________________________________ TABLE OF CONTENTS: page ----------------- Chapter 1: INTRODUCTION Introduction by editors 1-12 An Introduction to Computing with Neural Nets, Lippmann 13-31 An Introduction to Neural Computing, Kohonen 32-46 Chapter 2: CONNECTIONIST PRIMITIVES Introduction by editors 47-55 A General Framework for Parallel Distributed Processing, Rumelhart, Hinton, & McClelland 56-82 Multilayer Feedforward Potential Function Network, Lee & Kil 83-93 Learning, Invariance, and Generalization in High-Order Networks, Giles & Maxwell 94-100 The Subspace Learning Algorithm as a Formalism for Pattern Recognition and Neural Networks, Oja & Kohonen 101-108 Chapter 3: KNOWLEDGE REPRESENTATION Introduction by editors 109-116 BoltzCONS: Reconciling Connectionism with the Recursive Nature of Stacks and Tree, Touretzky 117-125 Holographic Reduced Representations: Convolution Algebra for Compositional Distributed Representations, Plate 126-131 Efficient Inference with Multi-Place Predicates and Variables in a Connectionist System, Ajjanagadde and Shastri 132-139 Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming, Sutton 140-148 Chapter 4: LEARNING ALGORITHMS I Introduction by editors 149-166 Connectionist Learning Procedures, Hinton 167-216 30 Years of Adaptive Neural Networks: Perceptron, Madaline, and back-Propagation, Widrow and Lehr 217-244 Supervised Learning and Systems with Excess Degrees of Freedom, Jordan 245-285 The Cascade-Correlation Learning Architecture, Fahlman 286-294 Learning to Predict by the Methods of Temporal Differences, Sutton 295-330 A Theoretical Framework for Back-Propagation, le Cun 331-338 Two Problems with Backpropagation and other Steepest-Descent Learning Procedures for Networks, Sutton 339-348 Chapter 5: LEARNING ALGORITHMS II Introduction by editors 349-358 The Self-Organizing Map, Kohonen 359-375 The ART of Adaptive Pattern Recognition by a Self-Organizing Neural Network, Grossberg 376-387 Unsupervised Learning in Noise, Kosko 388-401 A Learning Algorithm for Boltzmann Machines, Ackley, Hinton & Sejnowski 402-424 Learning Algorithms and Probability Distributions in Feed- forward and Feed-back Networks, Hopfield 425-429 A Mean Field Theory Learning Algorithm for Neural Networks, Peterson & Anderson 430-454 On the Use of Backpropagation in Associative Reinforcement Learning, Williams 455-462 Chapter 6: COMPUTATIONAL LEARNING THEORY Introduction by editors 463-473 Information Theory, Complexity, and Neural Networks, Abu-Mostafa 474-478 Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition, Cover 479-487 Approximation by Superpositions of a Sigmoidal Function, Cybenko 488-499 Approximation and Estimation Bounds for Artificial Neural Networks, Barron 500-506 Generalizing the PAC Model: Sample Size Bounds From Metric Dimension-based Uniform Convergence Results, Haussler 507-512 Complete Representations for Learning from Examples, Baum 513-534 A Statistical Approach to Learning and Generalization in Neural Networks, Levin, Tishby & Solla 535-542 Chapter 7: STABILITY AND CONVERGENCE Introduction by editors 543-550 Convergence in Neural Nets, Hirsch 551-561 Statistical Neurodynamics of Associative Memory, Amari & Maginu 562-572 Stability and Adaptation in Artificial Neural Systems, Schurmann 573-580 Dynamics and Architecture for Neural Computation, Pineda 581-610 Oscillations and Synchronizations in Neural Networks: An Exploration of the Labeling Hypothesis, Atiya & Baldi 611-632 Chapter 8: EMPIRICAL STUDIES Introduction by editors 633-639 Scaling Relationships in Back-Propagation Learning: Dependence on Training Set Size, Tesauro 640-645 An Empirical Comparison of Pattern Recognition, Neural Nets, and Machine Learning Classification Methods, Weiss & Kapouleas 646-652 Basins of Attraction of Neural Network Models, Keeler 653-657 Parallel Distributed Approaches to Combinatorial Optimization: Benchmark Studies on Traveling Salesman Problem, Peterson 658-666 From P.Refenes at cs.ucl.ac.uk Tue Jun 22 04:33:31 1993 From: P.Refenes at cs.ucl.ac.uk (P.Refenes@cs.ucl.ac.uk) Date: Tue, 22 Jun 93 09:33:31 +0100 Subject: NEURAL NETWORKS IN THE CAPITAL MARKETS Message-ID: ___________________________________________________________ CALL FOR PAPERS 1ST INTERNATIONAL WORKSHOP NEURAL NETWORKS IN THE CAPITAL MARKETS LONDON BUSINESS SCHOOL, NOVEMBER 18-19 1993 Neural Networks have now been applied to a number of live systems in the capital markets and in many cases have demonstrated better performance than competing approaches. Now is the time to take a critical look at their successes and limitations and to assess their capabilities, research issues and future directions. This workshop invites original papers which represent new and significant research, development and applications in finance & investment and which cover key areas of time series forecasting, multivariate dataset analysis, classification and pattern recognition. TOPICS Full papers are invited in (but not limited to) the following areas: - Bond and Stock Valuation and Trading - Univariate time series analysis - Asset allocation and risk management - Multivariate data analysis - Foreign exchange rate prediction - Classification and ranking - Commodity price forecasting - Pattern Recognition - Portfolio management - Hybrid systems Short communications will be accepted if they contain original topical material. SUBMISSION Deadline for submission : 15 September 1993 Notification of acceptance: 15 October 1993 Format: up to a maximum of twenty, single-spaced A4 pages. PROGRAMME COMMITTEE Prof. N. Biggs - London School of Economics Prof. D. Bunn - London Business School Dr J. Moody - Oregon Graduate Institute Dr A. Refenes - London Business School Prof. M. Steiner - Universitaet Munster Dr A. Weigend - University of Colorado ADDRESS FOR PAPERS Dr A. N. Refenes London Business School Department of Decision Science Sussex Place, Regents Park London NW1 4SA, England Tel: ++44 (71) 380 73 29 Fax: ++44 (71) 387 13 97 Email: refenes at cs.ucl.ac.uk ____________________________________________________________________ From greiner at learning.siemens.com Tue Jun 22 08:48:57 1993 From: greiner at learning.siemens.com (Russell Greiner) Date: Tue, 22 Jun 93 08:48:57 EDT Subject: CLNL'93 - Revised Deadlines and General Info Message-ID: <9306221248.AA05363@learning.siemens.com> re: deadlines for Computational Learning and Natural Learning (CLNL'93) Due to popular requests, we have decided to extend the deadline for CLNL'93 submission by one week, until 7/July/93. Below is the revised call for papers, with updated "Important Dates" and "Programme Committee" entries, as well as general registration information. We look forward to receiving your papers, and also hope that you will attend the workshop this September! Russ Greiner (Chair, CLNL'93) PS: People who plan to attend this workshop are still strongly encouraged to register by 30/June. ------------- CLNL'93 -- Call for Submissions Computational Learning and Natural Learning Provincetown, Massachusetts 10-12 September 1993 CLNL'93 is the fourth of an ongoing series of workshops designed to bring together researchers from a diverse set of disciplines --- including computational learning theory, AI/machine learning, connectionist learning, statistics, and control theory --- to explore issues at the intersection of theoretical learning research and natural learning systems. Theme: To be useful, the learning methods used by our fields must be able to handle the complications inherent in real-world tasks. We therefore encourage researchers to submit papers that discuss extensions to learning systems that let them address issues such as: * handling many irrelevant features * dealing with large amounts of noise * inducing very complex concepts * mining enormous sets of data * learning over extended periods of time * exploiting large amounts of background knowledge We welcome theoretical analyses, comparative studies of existing algorithms, psychological models of learning in complex domains, and reports on relevant new techniques. Submissions: Authors should submit three copies of an abstract (100 words or less) and a summary (2000 words or less) of original research to: CLNL'93 Workshop Learning Systems Department Siemens Corporate Research 755 College Road East Princeton, NJ 08540-6632 by 30 June 1993. We will also accept plain-text, stand-alone LaTeX or Postscript submissions sent by electronic mail to clnl93 at learning.scr.siemens.com Each submission will be refereed by the workshop organizers and evaluated based on its relevance to the theme, originality, clarity, and significance. Copies of accepted abstracts will be distributed at the workshop, and MIT Press has agreed to publish an edited volume that incorporates papers from the meeting, subject to revisions and additional reviewing. Invited Talks: Tom Dietterich Oregon State University Ron Rivest Massachusetts Institute of Technology Leo Breiman University of California, Berkeley Yann le Cun Bell Laboratories Important Dates: Deadline for submissions: 7 July 1993 Notification of acceptance: 27 July 1993 CLNL'93 Workshop: 10-12 September 1993 Programme Committee: Andrew Barron, Russell Greiner, Steve Hanson, Robert Holte, Michael Jordan, Stephen Judd, Pat Langley, Thomas Petsche, Tomaso Poggio, Ron Rivest, Eduardo Sontag, Steve Whitehead Workshop Sponsors: Siemens Corporate Research and MIT Laboratory of Computer Science CLNL'93 General Information Dates: The workshop officially begins at 9am Friday 10/Sept, and concludes by 3pm Sunday 12/Sept, in time to catch the 3:30pm Provincetown-Boston ferry. Location: All sessions will take place in the Provincetown Inn (800 942-5388). We encourage registrants to stay there; please sign up in the enclosed registration form. Notice the $74/night does correspond to $37/person per night double-occupancy, if two people share one room. Cost: The cost to attend this workshop is $50/person in general; $25/student. This includes * attendance at all presentation and poster sessions, including the four invited talks; * the banquet dinner on Saturday night; and * a copy of the accepted abstracts. Transportation: Provincetown is located at the very tip of Cape Cod, jutting into the Atlantic Ocean. The drive from Boston to Provincetown requires approximately two hours. There is also a daily ferry (run by Bay State Cruise Lines, 617 723-7800) that leaves Commonwealth Pier in Boston Harbor at 9:30am and arrives in Provincetown at 12:30pm; the return trip departs Provincetown at 3:30pm, arriving at Commonwealth Pier at 6:30pm. Its cost is $15/person, one way. There are also cabs, busses and commuter airplanes (CapeAir, 800 352-0714) that service this Boston-Provincetown route. Reception (Tentative): If there is sufficient interest (as indicated by signing up on the form below), we will hold a reception on a private ferry that leaves Commonwealth Pier for Provincetown at 6:30pm 9/Sept. The additional (Siemens-subsidized) cost for ferry and reception is $40/person, which also includes the return Provincetown-Boston ferry trip on 12/Sept. You must sign up by 30/June; we will announce by 13/July whether this private ferry will be used (and refund the money otherwise). Inquiries: For additional information about CLNL'93, contact clnl93 at learning.scr.siemens.com or the above address. To learn more about Provincetown, contact their Chamber of Commerce at 508 487-3424. CLNL'93 Registration Name: ________________________________________________ Affiliation: ________________________________________________ Address: ________________________________________________ ________________________________________________ Telephone: ____________________ E-mail: ____________________ Select the appropriate options and fees: Workshop registration fee ($50 regular; $25 student) ___________ Ferry transportation + reception ($40) ___________ Hotel room(*) ($74 = 1 night deposit) ___________ Arrival date ___________ Departure date _____________ Name of person sharing room (optional) __________________ # of breakfasts desired ($7.50/bkfst; no deposit req'd) ___ Total amount enclosed: ___________ (*) This is at the Provincetown Inn. For minimum stay of 2 nights. The total cost for three nights is $222 = $74 x 3, plus optional breakfasts. The block of rooms held for CLNL'93 will be released on 30 June 93; room reservations received after this date are accepted subject to availability. See hotel for cancellation policy. If you are not using a credit card, make your check payable in U.S. dollars to "Provincetown Inn/CLNL'93", and mail your completed registration form to Provincetown Inn/CLNL P.O. Box 619 Provincetown, MA 02657. If you are using Visa or MasterCard, please fill out the following, which you may mail to above address, or FAX to 508 487-2911. Signature: ______________________________________________ Visa/MasterCard #: ______________________________________________ Expiration: ______________________________________________ From barnhill at Hudson.Stanford.EDU Tue Jun 22 16:59:31 1993 From: barnhill at Hudson.Stanford.EDU (Joleen Barnhill) Date: Tue, 22 Jun 93 13:59:31 PDT Subject: Neural Nets course Message-ID: The Western Institute in Computer Science announces a one-week course in Neural Networks to be held on the Stanford campus from August 16-20, 1993. Aimed at those technical managers or engineers unfamiliar with Neural Networks who are looking for a comprehensive overview of the field, the course will provide an introduction and an understanding of basic terminology. Included in the introduction will be several hands-on sessions. The course will cover recent developments in the field and their impact from an applications perspective. Students will be given detailed coverage of the application of neural networks to scientific, engineering, and commercial problems. Lastly, future directions of the field will be discussed. Throughout the course, students will have contact with leading researchers, application engineers and managers experienced with the technology. INSTRUCTORS: DR. DAVID BISANT received the Ph.D. from George Washington University and is an Advanced Studies Fellow at STanford University. He holds several adjunct faculty positions and has held positions at International Medical Corporation and with the Department of Defense. His research involves neural network applications to image processing and sequence analysis. DR. DAVID RUMELHART is a Professor at STanford University. He is a Fellow of the NAS, the AAAS and the American Academy of Arts and Sciences. He received a MacArthur Foundation Fellowship for his work on cognitive modeling and he co-founded the Institute of Cognitive Science at UC- San Diego. He received the Ph.D. from STanford. For a complete brochure of all WICS offerings, including this one, send your name and mailing address to barnhill at hudson.Stanford.EDU or call (916) 873-0575 from 8 a.m.-5 p.m. Pacific Time. From jbower at smaug.cns.caltech.edu Thu Jun 24 14:17:20 1993 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Thu, 24 Jun 93 11:17:20 PDT Subject: Edward C. Posner Memorial Fellowhip Fund Message-ID: <9306241817.AA24017@smaug.cns.caltech.edu> ********************************************************** Dear Colleague, As you may be aware, last week Ed Posner was killed in an unfortunate bicycle accident in Pasadena. Those of us who knew him well feel the loss very deeply. However, Ed's strong commitment to the communities to which he belonged makes his death even more unfortunate. Throughout his career, Ed was actively involved in academic education and organization. For example, in recent years, he was the first chairman and principle organizer of the Neural Information Processing Systems (NIPS) meeting. One of his many legacies to this meeting is a strong commitment to student travel awards. He also spent much of the last year of his life working hard to establish the NIPS Foundation so that NIPS as well as other related meetings could be on sound financial and legal footing. In addition to his professional activities, Ed was also deeply involved in educational activities at Caltech and JPL. His commitment to students was legend at both institutions. Ed was particularly heavily involved in the SURF program at Caltech through which undergraduates (both from Caltech and from other institutions) carry out independent research projects during the summer months. Ed was an active member of the SURF Administrative Committee. He was also one of the most active SURF research sponsors, having served as mentor to 13 students since 1984. Three more students were preparing to work with him this summer. In addition, Ed co-founded the SURFSAT satellite program in 1986. In this program, successive teams of SURF students are designing, building, and testing a small communications satellite to support the research objectives of NASA's Deep Space Network. Since its inception, 43 students have participated in SURFSAT. Ed's persistent commitment to the scientific education of young people stretches far and touches many. Just a few days prior to his death, for example, Ed had begun to work with the Dean of Graduate Education at Caltech to organize yet another educational program, in this case to increase the number of underrepresented minorities in engineering. Given Ed's strong interest in science education, research, and students, Mrs. Posner has asked that memorial gifts be designated to support Caltech's SURF Program. It is our hope that gifts might ultimately fund an Edward C. Posner SURF Fellowship Fund. Once funded, the Posner SURF Fellowship would annually support an under-represented minority student for a research project in a field related to Ed's own professional interests. Those individuals, or institutions interested in making a contribution to the Edward C. Posner SURF fellowship fund in his memory should contact: Dore Charbonneau Director of Special Gifts Development, mail code 105-40 Caltech Pasadena, CA. 91125 818 - 356-6285 Thank you for your interest and we hope to hear from you. Carolyn Merkel Director, SURF program James M. Bower Computation and Neural Systems Program-Caltech From mackay at mrao.cam.ac.uk Sat Jun 26 12:34:00 1993 From: mackay at mrao.cam.ac.uk (David J.C. MacKay) Date: Sat, 26 Jun 93 12:34 BST Subject: Hyperparameters: optimise, or integrate out? Message-ID: The following preprint is now available by anonymous ftp. ======================================================================== Hyperparameters: optimise, or integrate out? David J.C. MacKay University of Cambridge Cavendish Laboratory Madingley Road Cambridge CB3 0HE mackay at mrao.cam.ac.uk I examine two computational methods for implementation of Bayesian hierarchical models, that is, models which include unknown hyperparameters such as regularisation constants. In the `evidence framework' the model parameters are {\em integrated} over, and the resulting evidence is {\em maximised} over the hyperparameters. In the alternative `MAP' method, the `true posterior probability' is found by {\em integrating} over the hyperparameters, and this is then {\em maximised} over the model parameters. The similarities of the two approaches, and their relative merits, are discussed. In severely ill-posed problems, it is shown that significant biases arise in the second method. ======================================================================== The preprint "Hyperparameters: optimise, or integrate out?" may be obtained as follows: ftp 131.111.48.8 anonymous (your name) cd pub/mackay binary get alpha.ps.Z quit uncompress alpha.ps.Z This document is 16 pages long Table of contents: Outline Making inferences The ideal approach The Evidence framework The MAP method The effective $\a$ of the general MAP method Pros and cons In favour of the MAP method Magnifying the differences An example The curvature of the true prior, and MAP error bars Discussion Appendices: Conditions for the evidence approximation Distance between probability distributions A method for evaluating distances D( p(t) , q(t) ) What I mean by saying that the approximation `works' Predictions The evidence \sigma_N and \sigma_N-1 From mackay at mrao.cam.ac.uk Sat Jun 26 12:36:00 1993 From: mackay at mrao.cam.ac.uk (David J.C. MacKay) Date: Sat, 26 Jun 93 12:36 BST Subject: Energy Prediction Competition Message-ID: The following preprint is now available by anonymous ftp. *********************************************************************** Bayesian Non-linear Modeling for the Energy Prediction Competition David J.C. MacKay University of Cambridge Cavendish Laboratory Madingley Road Cambridge CB3 0HE mackay at mrao.cam.ac.uk Bayesian probability theory provides a unifying framework for data modeling. A model space may include numerous control parameters which influence the complexity of the model (for example regularisation constants). Bayesian methods can automatically set such parameters so that the model becomes probabilistically well-matched to the data. The 1993 energy prediction competition involved the prediction of a series of building energy loads from a series of environmental input variables. Non-linear regression using `neural networks' is a popular technique for such modeling tasks. Since it is not obvious how large a time-window of inputs is appropriate, or what preprocessing of inputs is best, this can be viewed as a regression problem in which there are many possible input variables, some of which may actually be irrelevant to the prediction of the output variable. Because a finite data set will show random correlations between the irrelevant inputs and the output, any conventional neural network (even with `weight decay') will not set the coefficients for these junk inputs to zero. Thus the irrelevant variables will hurt the model's performance. The Automatic Relevance Determination (ARD) model puts a prior over the regression parameters which embodies the concept of relevance. This is done in a simple and `soft' way by introducing multiple `weight decay' constants, one `$\alpha$' associated with each input. Using Bayesian methods, the decay rates for junk inputs are automatically inferred to be large, preventing those inputs from causing significant overfitting. An entry using the ARD model won the prediction competition by a significant margin. *********************************************************************** The preprint "Bayesian Non-linear Modeling for the Energy Prediction Competition" may be obtained as follows: ftp 131.111.48.8 anonymous (your name) cd pub/mackay binary mget pred.*.ps.Z quit uncompress pred.*.ps.Z This preprint is 24 pages long and contains a large number of figures. A more concise version may be released later. Table of contents: Overview of Bayesian modeling methods Neural networks for regression Neural network learning as inference Setting regularisation constants $\a$ and $\b$ Automatic Relevance Determination Prediction competition: part A The task Preliminaries Round 1 Round 2 Creating a committee Results Additional performance criteria How much did ARD help? How much did the use of a committee help? Prediction competition: part B The task Preprocessing Time--delayed and time--filtered inputs Results Summary and Discussion What I might have done differently How to better model this sort of problem. Appendix Training data: problem A Omitted data Coding of holidays Pre-- and post-processing From joe at cogsci.edinburgh.ac.uk Tue Jun 29 06:46:17 1993 From: joe at cogsci.edinburgh.ac.uk (Joe Levy) Date: Tue, 29 Jun 93 11:46:17 +0100 Subject: 2nd Neural Computation and Psychology Workshop Message-ID: <9150.9306291046@grogan.cogsci.ed.ac.uk> 2nd Neural Computation and Psychology Workshop Connectionist Models of Memory and Language University of Edinburgh, Scotland 10th - 13th September 1993 AIMS AND OBJECTIVES This workshop is the second in a series, following on from last year's very successful Neurodynamics and Psychology Workshop at the University of Wales, Bangor. This year it is to be hosted by the Connectionism and Cognition Research Group at the University of Edinburgh. The general aim is to bring together researchers from such diverse disciplines as neurobiology, psychology, cognitive science, artificial intelligence, applied mathematics and computer science to discuss their work on the connectionist modelling of memory and language. Likely special sessions include memory, speech processes and models of reading. The workshop will have an invited list of speakers and a limited number of participants to allow single-track papers and ease of discussion. It will run from the evening of Friday 10th through to lunch time on Monday 13th. PROVISIONAL SESSION CHAIRS AND INVITED SPEAKERS INCLUDE: Bob Damper (Southampton) Trevor Harley (Warwick) Jacob Murre (APU, Cambridge) Noel Sharkey (Exeter) Leslie Smith (Stirling) Keith Stenning (Edinburgh) John Taylor (KC, London) David Willshaw (Edinburgh) POSTER SESSION So that other participants have a chance of presenting their work on connectionist models of memory and language we are including refereed poster sessions. If interested, please send a one page abstract with the registration form. REGISTRATION, FOOD AND ACCOMMODATION The workshop will be held in Pollock Halls, an attractive student residence close to the centre of Edinburgh with views of Arthur's Seat and ample parking. The conference registration fee (which includes morning and afternoon teas and coffees, the three lunches, a reception and meal on the Friday evening and dinner on Saturday) is 80 pounds (or 65 pounds for full time students). A special Conference Dinner will be arranged for the Sunday evening costing 15 pounds. Bed and breakfast accommodation on site is 70 pounds (total for the three nights). ORGANISING COMMITTEE Joe Levy (HCRC, Edinburgh) Dimitris Bairaktaris (HCRC, Edinburgh) John Bullinaria (Psychology, Edinburgh) Paul Cairns (Cognitive Science, Edinburgh) FURTHER DETAILS AND REGISTRATION Contact: Dr. Joe Levy, NCPW 93, University of Edinburgh, Human Communication Research Centre, 2 Buccleuch Place, Edinburgh, EH8 9LW, Scotland. Email: joe at uk.ac.ed.cogsci Phone: +44 31 650 4450. Fax: +44 31 650 4587. REGISTRATION FORM: 2nd Neural Computation and Psychology Workshop Connectionist Models of Memory and Language University of Edinburgh, Scotland 10th - 13th September 1993 Completed registration forms, full payment and any abstracts must be received by 12th July. Full Name: ___________________________________________________________ Institution: _________________________________________________________ Mailing Address: _____________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ Telephone: _________________________________________ Fax: _______________________________________________ E-mail: ____________________________________________ Poster abstract enclosed: Yes / No Money enclosed: Registration Fee: #80 ___________ (includes 3 lunches and 2 dinners) (#65 for full time students - proof required) Conference Dinner on Sunday evening: #15 ___________ (Special dietary requirements: Vegetarian Vegan Other........ ) Accommodation: #70 ___________ (total for 3 nights B & B) (indicate if adjacent rooms required) TOTAL =========== Please make cheques payable to 'University of Edinburgh'. Mail to: Dr. Joe Levy (NCPW 93) University of Edinburgh Human Communication Research Centre 2 Buccleuch Place Edinburgh EH8 9LW Scotland UK From greiner at learning.siemens.com Tue Jun 29 23:25:52 1993 From: greiner at learning.siemens.com (Russell Greiner) Date: Tue, 29 Jun 93 23:25:52 EDT Subject: CLNL'93 - Revised deadline Message-ID: <9306300325.AA10121@learning.siemens.com> re: deadlines for Computational Learning and Natural Learning (CLNL'93) Due to popular requests, we have decided to extend the deadline for CLNL'93 submission by one week, until 7/July/93. Below is the revised call for papers, with updated "Important Dates" and "Programme Committee" entries, as well as general registration information. We look forward to receiving your papers, and also hope that you will attend the workshop this September! Russ Greiner (Chair, CLNL'93) ------------- CLNL'93 -- Call for Submissions Computational Learning and Natural Learning Provincetown, Massachusetts 10-12 September 1993 CLNL'93 is the fourth of an ongoing series of workshops designed to bring together researchers from a diverse set of disciplines --- including computational learning theory, AI/machine learning, connectionist learning, statistics, and control theory --- to explore issues at the intersection of theoretical learning research and natural learning systems. Theme: To be useful, the learning methods used by our fields must be able to handle the complications inherent in real-world tasks. We therefore encourage researchers to submit papers that discuss extensions to learning systems that let them address issues such as: * handling many irrelevant features * dealing with large amounts of noise * inducing very complex concepts * mining enormous sets of data * learning over extended periods of time * exploiting large amounts of background knowledge We welcome theoretical analyses, comparative studies of existing algorithms, psychological models of learning in complex domains, and reports on relevant new techniques. Submissions: Authors should submit three copies of an abstract (100 words or less) and a summary (2000 words or less) of original research to: CLNL'93 Workshop Learning Systems Department Siemens Corporate Research 755 College Road East Princeton, NJ 08540-6632 by 30 June 1993. We will also accept plain-text, stand-alone LaTeX or Postscript submissions sent by electronic mail to clnl93 at learning.scr.siemens.com Each submission will be refereed by the workshop organizers and evaluated based on its relevance to the theme, originality, clarity, and significance. Copies of accepted abstracts will be distributed at the workshop, and MIT Press has agreed to publish an edited volume that incorporates papers from the meeting, subject to revisions and additional reviewing. Invited Talks: Tom Dietterich Oregon State University Ron Rivest Massachusetts Institute of Technology Leo Breiman University of California, Berkeley Yann le Cun Bell Laboratories Important Dates: Deadline for submissions: 7 July 1993 Notification of acceptance: 27 July 1993 CLNL'93 Workshop: 10-12 September 1993 Programme Committee: Andrew Barron, Russell Greiner, Steve Hanson, Robert Holte, Michael Jordan, Stephen Judd, Pat Langley, Thomas Petsche, Tomaso Poggio, Ron Rivest, Eduardo Sontag, Steve Whitehead Workshop Sponsors: Siemens Corporate Research and MIT Laboratory of Computer Science CLNL'93 General Information Dates: The workshop officially begins at 9am Friday 10/Sept, and concludes by 3pm Sunday 12/Sept, in time to catch the 3:30pm Provincetown-Boston ferry. Location: All sessions will take place in the Provincetown Inn (800 942-5388). We encourage registrants to stay there; please sign up in the enclosed registration form. Notice the $74/night does correspond to $37/person per night double-occupancy, if two people share one room. Cost: The cost to attend this workshop is $50/person in general; $25/student. This includes * attendance at all presentation and poster sessions, including the four invited talks; * the banquet dinner on Saturday night; and * a copy of the accepted abstracts. Transportation: Provincetown is located at the very tip of Cape Cod, jutting into the Atlantic Ocean. The drive from Boston to Provincetown requires approximately two hours. There is also a daily ferry (run by Bay State Cruise Lines, 617 723-7800) that leaves Commonwealth Pier in Boston Harbor at 9:30am and arrives in Provincetown at 12:30pm; the return trip departs Provincetown at 3:30pm, arriving at Commonwealth Pier at 6:30pm. Its cost is $15/person, one way. There are also cabs, busses and commuter airplanes (CapeAir, 800 352-0714) that service this Boston-Provincetown route. Reception (Tentative): If there is sufficient interest (as indicated by signing up on the form below), we will hold a reception on a private ferry that leaves Commonwealth Pier for Provincetown at 6:30pm 9/Sept. The additional (Siemens-subsidized) cost for ferry and reception is $40/person, which also includes the return Provincetown-Boston ferry trip on 12/Sept. You must sign up by 30/June; we will announce by 13/July whether this private ferry will be used (and refund the money otherwise). Inquiries: For additional information about CLNL'93, contact clnl93 at learning.scr.siemens.com or the above address. To learn more about Provincetown, contact their Chamber of Commerce at 508 487-3424. CLNL'93 Registration Name: ________________________________________________ Affiliation: ________________________________________________ Address: ________________________________________________ ________________________________________________ Telephone: ____________________ E-mail: ____________________ Select the appropriate options and fees: Workshop registration fee ($50 regular; $25 student) ___________ Ferry transportation + reception ($40) ___________ Hotel room(*) ($74 = 1 night deposit) ___________ Arrival date ___________ Departure date _____________ Name of person sharing room (optional) __________________ # of breakfasts desired ($7.50/bkfst; no deposit req'd) ___ Total amount enclosed: ___________ (*) This is at the Provincetown Inn. For minimum stay of 2 nights. The total cost for three nights is $222 = $74 x 3, plus optional breakfasts. The block of rooms held for CLNL'93 will be released on 30 June 93; room reservations received after this date are accepted subject to availability. See hotel for cancellation policy. If you are not using a credit card, make your check payable in U.S. dollars to "Provincetown Inn/CLNL'93", and mail your completed registration form to Provincetown Inn/CLNL P.O. Box 619 Provincetown, MA 02657. If you are using Visa or MasterCard, please fill out the following, which you may mail to above address, or FAX to 508 487-2911. Signature: ______________________________________________ Visa/MasterCard #: ______________________________________________ Expiration: ______________________________________________ From cic!john!ostrem at unix.sri.com Tue Jun 22 16:48:32 1993 From: cic!john!ostrem at unix.sri.com (John Ostrem) Date: Tue, 22 Jun 93 13:48:32 PDT Subject: job openings Message-ID: <9306222048.AA17348@john.noname> Communication Intelligence Corporation (CIC) is a leader in handwriting recognition and other pen input technologies. We currently market recognizers for English, Western European, and Asian languages on a variety of platforms (e.g., DOS, Windows, Macintosh, and so on). These systems enable the pen to serve as the sole input and control device, combining the functions of both keyboard and mouse, and adding new capabilities. Advanced development is directed toward integrated discrete/cursive recognizers, and future integration with voice recognition, OCR, and similar technologies. CIC was founded in 1981 in conjunction with SRI International (formerly Stanford Research Institute). CIC is headquartered in Redwood Shores, California, and has an international subsidiary, CIC Japan, Inc., in Tokyo, Japan. CIC currently has immediate openings for the following positions: ----------------------------------------------------------------------------- POSITION: Software Engineer QUALIFICATIONS: 1. 3-5 years experience in designing and coding for large software projects in a UNIX environment 2. Good communication skills and works well with other people. 3. Expert C programmer (at least 3-5 years experience) 4. BS or MS in Computer Science or the equivalent 5. Experience in graphics programming and user interfaces a plus 6. The following are additional pluses: a. Experience in handwriting recognition (on-line or off-line) b. Linguistic experience (particularly statistical linguistics) c. Experience planning/executing complex projects d. Experience in commercial companies e. Experience in SunOS system administration JOB DESCRIPTION: 1. Work with and support researchers working on handwriting and speech recognition 2. Design, implement, and support data collection software and analysis tools ----------------------------------------------------------------------------- POSITION: Pattern Recognition Specialist/project leader QUALIFICATIONS: 1. Strong background in statistics, pattern recognition, algorithm development 2. Experience in OCR a plus 3. 3-5 years experience in designing and coding for large software projects in a UNIX environment 4. Good communication skills and works well with other people. 5. Expert C programmer (at least 3-5 years experience) 6. Ph.D. or substantial experience in Computer Science, Electrical Engineering or the equivalent 7. The following are additional pluses: a. Experience in handwriting recognition (on-line or off-line) b. Linguistic experience (particularly statistical linguistics) c. Experience planning/executing complex projects d. Experience in commercial companies JOB DESCRIPTION 1. Work with a team of researchers on the next generation of handwriting recognition systems (both off-line and on-line) for the commercial market 2. Develop into a project leader/manager ----------------------------------------------------------------------------- Please reply to cic!ostrem at unix.sri.com (or cic\!ostrem at unix.sri.com in command mode), or write or fax to John S. Ostrem Communication Intelligence Corporation 275 Shoreline Drive, 6th Floor Redwood Shores, CA 94065-1413 Fax: (415) 802-7777 From tesauro at watson.ibm.com Tue Jun 1 20:07:38 1993 From: tesauro at watson.ibm.com (Gerald Tesauro) Date: Tue, 1 Jun 93 20:07:38 EDT Subject: TD-Gammon paper available in neuroprose Message-ID: The following paper, which has been accepted for publication in Neural Computation, has been placed in the neuroprose archive at Ohio State. Instructions for retrieving the paper by anonymous ftp are appended below. --------------------------------------------------------------- TD-Gammon, A Self-Teaching Backgammon Program, Achieves Master-Level Play Gerald Tesauro IBM Thomas J. Watson Research Center P. O. Box 704 Yorktown Heights, NY 10598 (tesauro at watson.ibm.com) Abstract: TD-Gammon is a neural network that is able to teach itself to play backgammon solely by playing against itself and learning from the results, based on the TD(lambda) reinforcement learning algorithm (Sutton, 1988). Despite starting from random initial weights (and hence random initial strategy), TD-Gammon achieves a surprisingly strong level of play. With zero knowledge built in at the start of learning (i.e. given only a ``raw'' description of the board state), the network learns to play at a strong intermediate level. Furthermore, when a set of hand-crafted features is added to the network's input representation, the result is a truly staggering level of performance: the latest version of TD-Gammon is now estimated to play at a strong master level that is extremely close to the world's best human players. --------------------------------------------------------------- FTP INSTRUCTIONS unix% ftp archive.cis.ohio-state.edu (or 128.146.8.52) Name: anonymous Password: (use your e-mail address) ftp> cd pub/neuroprose ftp> binary ftp> get tesauro.tdgammon.ps.Z ftp> bye unix% uncompress tesauro.tdgammon.ps unix% lpr tesauro.tdgammon.ps From joachim at fitmail.fit.qut.edu.au Tue Jun 1 23:52:47 1993 From: joachim at fitmail.fit.qut.edu.au (Joachim Diederich) Date: Wed, 2 Jun 93 13:52:47 +1000 Subject: Brisbane Neural Network Workshop Message-ID: <2929F4367ADF20D062@qut.edu.au> First Brisbane Neural Network Workshop -------------------------------------- Queensland University of Technology Brisbane Q 4001, AUSTRALIA Gardens Point Campus, ITE 303 4 June 1993 The first Brisbane Neural Network Workshop is intended to bring together those interested in neurocomputing and neural network applications. The objective of the workshop is to provide a discussion platform for researchers and practitioners interested in theoretical and applied aspects of neurocomputing. The workshop should be of interest to computer scientists and engineers, as well as to biologists, cognitive scientists and others interested in the application of neural networks. This is the first of a series of workshops and seminars with the objective of enhancing collaboration between neural network researchers and practitioners in Queensland. A second workshop is planned for the end of July. The First Brisbane Neural Network Workshop will be held at Queensland University of Technology, Gardens Point Campus (ITE 303) on June 4, 1993 from 8:00am to 6:00pm. Programme 8:00-8:15 Welcome Joachim Diederich, QUT-FIT-CS Neurocomputing 8:15-8:45 Janet Wiles, University of Queensland, Departments of Computer Science and Psychology Representations in hidden unit space 8:45-9:15 Paul Bakker, University of Queensland, Departments of Computer Science and Psychology Examining Learning Dynamics with the Hyperplane Animator 9:15-9:45 Simon Dennis, University of Queensland Department of Computer Science Introducing Learning into Models of Human Memory 9:45-10:15 Steven Phillips, University of Queensland Department of Computer Science Systematicity and Feedforward Networks: Exponential Generalizations from Polynomial Examples 10:15-10:45 Coffee Break 10:45-11:15 Joachim Diederich, QUT-FIT-CS Neurocomputing Cows, Bulls & Tarzan: Preliminary results on animal breeding advice using neural networks 11:15-11:45 Joaquin Sitte, QUT-FIT-CS Neurocomputing Learning control in simple dynamics systems 11:45-12:15 Shlomo Geva, QUT-FIT-CS Neurocomputing Constrained gradient descent 12:15-12:45 Ray Lister, University of Queensland Department of Electrical Engineering On Seeing the World in a Grain of Sand: Hidden Unit Self-Organization, and Super Criticality 12:45-2:00 Lunch Break 2:00-2:30 David Abramson, Griffith University, School of Computing and Information Technology High Performance Computation for Simulated Annealing and Genetic Algorithms 2:30-3:00 John D. Pettigrew, University of Queensland, Vision, Touch & Hearing Research Centre The owl & the pussycat: comparative study of the networks underlying binocular vision. 3:00-3:30 Tom Downs/Ah Chung Tsoi, University of Queensland, Department of Electrical Engineering Directions of research in the UQ EE department 3:30-4:00 David Lovell, University of Queensland, Department of Electrical Engineering An improved version of the neocognitron 4:00-4:30 Coffee Break 4:30-5:00 Ron Ganier, University of Queensland, Department of Electrical Engineering Generalization in artificial neural networks 5:00-5:30 Paul Murtagh, University of Queensland, Department of Electrical Engineering Fault tolerance and VLSI design for artificial neural networks 5:30-6:00 Robert Young, Queensland Department of Primary Industries QDPI Neural Network Applications Enquiries should be sent to Professor Joachim Diederich Neurocomputing Research Concentration Area School of Computing Science Queensland University of Technology GPO Box 2434 Brisbane Q 4001 Phone: (07) 864-2143 Fax: (07) 864-1801 Email: joachim at fitmail.fit.qut.edu.au From fmurtagh at eso.org Thu Jun 3 03:43:22 1993 From: fmurtagh at eso.org (fmurtagh@eso.org) Date: Thu, 3 Jun 93 09:43:22 +0200 Subject: Announcement: conferences calendar available in Neuroprose archive Message-ID: <9306030743.AA08508@st2.hq.eso.org> FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/murtagh.calendar.txt.Z The file murtagh.calendar.txt.Z is available for copying from the Neuroprose repository. It is a CALENDAR of forthcoming conferences and workshops in the neural net and related fields. It is about 1300 lines in length, consists of brief details (date, title, location, contact), and is valid from mid-May 1993 onwards. The intention is to update it in about 3 months. F. Murtagh (fmurtagh at eso.org) From sabine at i4.informatik.rwth-aachen.de Fri Jun 4 14:16:26 1993 From: sabine at i4.informatik.rwth-aachen.de (sabine@i4.informatik.rwth-aachen.de) Date: 4 Jun 1993 14:16:26 MEZ-1 Subject: Workshop on Neural Networks at Aachen, Germany Message-ID: CALL FOR PARTICIPATION "LECTURES AND WORKSHOP ON NEURAL NETWORKS AACHEN '93" Aachen University of Technology D - 52056 Aachen, Germany Introductory Lectures June 21-30 1993 Workshop July 12-13 1993 The first Workshop on Neural Networks at Aachen intends to convey ideas on neural methods to a wide audience interested in neurocomputing and neurocomputers. The 15 distinguished invited speakers will cover topics that range from biological issues and the modelling of consciousness to neurocomputers. The workshop will be complemented by a poster session presenting research projects at Aachen University in the field of neural networks. COMMITEE Honorary Chairman: Prof. I. Aleksander, Imperial College, London Prof. Dr. rer. nat. O. Spaniol Forum Informatik, Graduate College "Methods and tools of computer science and their application in technical systems", Aachen University of Technology, D-52056 Aachen, Germany Neural Networks Special Interest Group INN Harald Huening, Sabine Neuhauser, Michael Raus, Wolf Ritschel, Christiane Schmidt FINAL PROGRAMME INTRODUCTORY LECTURES JUNE 21-30, 1993: June 21, 93, 5 pm, AH III Prof. E.J.H. Kerckhoffs, Delft University of Technology (NL) "An Introduction to Neural Computing" June 22, 93, 5 pm, AH II Prof. C. von der Malsburg, Ruhr-Universitaet Bochum (D) "Neural Networks and the Brain" (German language) June 25, 93, 2 pm, AH IV Dr. U. Ramacher, Siemens AG, Munich (D) "A Computer-Architecture for the Simulation of Artificial Neural Networks and the further Development of Neurochips" (German language) June 30, 93, 5 pm, GH 3, Klinikum Prof. V. Braitenberg, Max-Planck-Institut Tuebingen (D) "New Ideas about the Function of the Cerebellum" (German language) WORKSHOP PROGRAMME, JULY 12, 1993, 9:00 AM - 5:30 PM (AULA II) 9:00 - 9:30 am Welcome and Introduction 9:30 - 10:15 am Prof. I. Aleksander, Imperial College, London (UK) "Iconic State Machines and their Cognitive Properties" 10:15 - 10:45 am Coffee Break, Poster Session 10:45 - 11:30 am Prof. E.J.H. Kerckhoffs, Delft University of Technology (NL) "Thoughts on Conjoint Numeric, Symbolic and Neural Computing" 11:30 am - 12:15 pm Dr. P. DeWilde, Imperial College, London (UK) "Reduction of Representations and the Modelling of Consciousness" 12:15 - 2:00 pm Lunch Break, Poster Session 2:00 - 2:45 pm Dr. M. Erb, Philipps-Universitaet Marburg (D) "Synchronized Activity in Biological and Artificial Dynamic Neural Networks: Experimental Results and Simulations" 2:45 - 3:30 pm drs. E. Postma, University of Limburg, Maastricht (NL) "Towards Scalable Neurocomputers" 3:30 - 4:00 pm Coffee Break, Poster Session 4:00 - 4:45 pm J. Heemskerk, Leiden University, Leiden (NL) "Neurocomputers: Design Principles for a Brain" 4:45 - 5:30 pm Prof. U. Rueckert, Technical University of Hamburg-Harburg, (D) "Microelectronic Implementation of Neural Networks" WORKSHOP PROGRAMME, JULY 14, 1993, 9:00 AM - 3:00 PM (AULA II): 9:00 - 9:45 am Dr. J. H. Schmidhuber, Technical University of Munich (D) "Continuous History Compression" 9:45 - 10:30 am K. Weigl, INRIA, Sophia-Antipolis (F) "Metric Tensors and Non-orthogonal Functional Bases" 10:30 - 11:00 am Coffee Break, Poster Session 11:00 - 11:45 am Dr. F. Castillo, Univ. Politecnica de Catalunya, Barcelona (E) "Statistics and Neural Network Classifiers: A Review from Multilayered Perceptrons to Incremental Neural Networks" 11:45 am - 1:30 pm Lunch Break, Poster Session 1:30 - 2:15 pm Dr. J. Mrsic-Floegel, Imperial College, London (UK) "A Review of RAM-based Weightless Nodes" 2:15 - 3:00 pm J. Schaefer, Aachen University of Technology (D) "Neural Networks and Fuzzy Technologies" LOCATIONS The workshop lectures will be performed at the lecture-hall Aula II, Aachen University of Technology, Ahornstrasse 55, D-52074 Aachen, Germany. The introductory lectures are performed in one of the following lecture-halls, as indicated in the programme: AH II, AH III, AH IV, Ahornstrasse 55, D-52074 Aachen, Germany and GH3, Klinikum Aachen, Pauwelstrasse, D- 52074 Aachen, Germany. Ahornstrasse can be reached by bus routes no. 23 or 33: - bus route 33 to "Klinikum" or "Vaals", stop at "Paedagogische Hochschule"; - bus route 23 to "Hoern", stop at "Paedagogische Hochschule". Klinikum can be reached by bus route 33 as well, stop at "Klinikum". To reach bus routes 23 or 33, take a bus from the station to "Bushof". PARTICIPATION is free of charge. Please register by e-mail to the organizing commitee: Harald Huening: harry at dfv.rwth-aachen.de Sabine Neuhauser: sabine at informatik.rwth-aachen.de Michael Raus: raus at rog1.rog.rwth-aachen.de Wolf Ritschel: ri at mtq03.wzl.rwth-aachen.de PROCEEDINGS: H. Huening, S. Neuhauser, M. Raus, W. Ritschel (eds.): "Workshop on Neural networks at RWTH Aachen", Aachener Beitraege zur Informatik ABI, Band 2, Verlag der Augustinus Buchhandlung, 227 pages contain the articles of the workshop + the article "Am I Thinking Assemblies ?" of Prof. C. von der Malsburg. Proceedings can be ordered from Augustinus Buchhandlung Pontstrasse 66/68 D-52062 Aachen at a price of 36.- DM plus postal coverage and postage (about 3.-DM within Germany). During the Workshop the book will be sold at a reduced price by Augustinus bookstore. LANGUAGE English will be the official conference language. sabine at informatik.rwth-aachen.de _______________________________________________________ Sabine Neuhauser Aachen University of Technology Computer Science Department (Informatik IV) Ahornstrasse 55, W- 5100 Aachen, Germany !!! please note the new postal code for !!! Aachen University of Technology !!! valid from 1.7.93 : D-52056 Aachen (postal address) From gasser at cs.indiana.edu Fri Jun 4 15:25:14 1993 From: gasser at cs.indiana.edu (Michael Gasser) Date: Fri, 4 Jun 1993 14:25:14 -0500 Subject: Paper on lexical acquisition Message-ID: FTP-host: cs.indiana.edu (129.79.254.191) FTP-filename: /pub/techreports/TR382.ps.Z The following report is available in compressed postscript form by anonymous ftp from the site given above (note: NOT neuroprose). The paper is 23 pages long. If you have trouble printing it out, please contact me. Michael Gasser gasser at cs.indiana.edu ================================================================= Learning Noun and Adjective Meanings: A Connectionist Account Michael Gasser Computer Science and Linguistics Departments Linda B. Smith Psychology Department Indiana University Abstract Why do children learn nouns such as {\it cup\/} faster than dimensional adjectives such as {\it big\/}? Most explanations of this well-known phenomenon rely on prior knowledge in the child of the noun-adjective distinction or on the logical priority of nouns as the arguments of predicates. In this paper we examine an alternative account, one which seeks to explain the relative ease of nouns over adjectives in terms of the response of the learner to various properties of the semantic categories to be learned and of the word learning task itself. We isolate four such properties: the relative size and the relative compactness of the regions in representational space associated with the categories, the presence or absence of lexical dimensions in the linguistic context of a word ({\it what color is it?\/} vs. {\it what is it?\/}), and the number of words of a particular type to be learned. In a set of five experiments, we trained a simple connectionist categorization device to label input objects, in particular linguistic contexts, as nouns or adjectives. We show that, for the network, the first three of the above properties favor the more rapid learning of nouns, while the fourth favors the more rapid learning of adjectives. Our experiments demonstrate that the advantage for nouns over adjectives does not require prior knowledge of the distinction between nouns and adjectives and suggest that this distinction may instead emerge as the child learns to associate the different properties of noun and adjective categories with the different morphosyntactic contexts which elicit them. From sabine at i4.informatik.rwth-aachen.de Fri Jun 4 15:37:29 1993 From: sabine at i4.informatik.rwth-aachen.de (sabine@i4.informatik.rwth-aachen.de) Date: 4 Jun 1993 15:37:29 MEZ-1 Subject: workshop on neural networks, Aachen 93 Message-ID: Dear organizers of this list, I've just sent an announcement about the Lectures and Workshop on Neural Networks at Aachen '93 to the connectionists-address. The dates for this workshop are June 21-30 for the Introductory Lectures July 12-13 1993 for the Workshop. Unfortunately, I've mentioned a wrong date for the second day of the workshop: in the second "Workshop programme..." header, I've mentioned the date "July, 14" instead of "July, 13". I'd be very pleased, if you could change this before posting it to the whole list. Thanks in advance, I'm really sorry for that mistake, Sabine Neuhauser sabine at informatik.rwth-aachen.de _______________________________________________________ Sabine Neuhauser Aachen University of Technology Computer Science Department (Informatik IV) Ahornstrasse 55, W- 5100 Aachen, Germany !!! please note the new postal code for !!! Aachen University of Technology !!! valid from 1.7.93 : D-52056 Aachen (postal address) From bap at learning.siemens.com Mon Jun 7 11:22:41 1993 From: bap at learning.siemens.com (Barak Pearlmutter) Date: Mon, 7 Jun 93 11:22:41 EDT Subject: Preprint Available Message-ID: <9306071522.AA17817@gull.siemens.com> I have placed the preprint whose abstract appears below in the neuroprose archives. My thanks to Jordan Pollack for providing this valuable service to the community. ---------------- Fast Exact Multiplication by the Hessian Barak A. Pearlmutter Just storing the Hessian $H$ (the matrix of second derivatives of the error $E$ with respect to each pair of weights) of a large neural network is difficult. Since a common use of a large matrix like $H$ is to compute its product with various vectors, we derive a technique that directly calculates $Hv$, where $v$ is an arbitrary vector. To calculate $Hv$, we first define a differential operator $R{f(w)} = (d/dr) f(w+rv) |_{r=0}$, note that $R{dE/dw} = Hv$ and $R{w} = v$, and then apply $R{}$ to the equations used to compute $dE/dw$. The result is an exact and numerically stable procedure for computing $Hv$, which takes about as much computation, and is about as local, as a gradient evaluation. We then apply the technique to a one pass gradient calculation algorithm (backpropagation), a relaxation gradient calculation algorithm (recurrent backpropagation), and two stochastic gradient calculation algorithms (Boltzmann Machines and weight perturbation). Finally, we show that this technique can be used at the heart of many iterative techniques for computing various properties of $H$, obviating any need to calculate the full Hessian. [12 pages; 42k; pearlmutter.hessian.ps.Z; To appear in Neural Computation] From SCHOLTES at ALF.LET.UVA.NL Mon Jun 7 10:41:00 1993 From: SCHOLTES at ALF.LET.UVA.NL (SCHOLTES@ALF.LET.UVA.NL) Date: Mon, 7 Jun 93 10:41 MET Subject: PhD Dissertation available Message-ID: =================================================================== As I had to disapoint many people because I run out of copies in the first batch, a high-quality reprint has been made from....................................... ........REPRINT........ Ph.D. DISSERTATION AVAILABLE on Neural Networks, Natural Language Processing, Information Retrieval 292 pages and over 350 references =================================================================== A Copy of the dissertation "Neural Networks in Natural Language Processing and Information Retrieval" by Johannes C. Scholtes can be obtained for cost price and fast airmail- delivery at US$ 25,-. Payment by Major Creditcards (VISA, AMEX, MC, Diners) is accepted and encouraged. Please include Name on Card, Number and Exp. Date. Your Credit card will be charged for Dfl. 47,50. Within Europe one can also send a Euro-Cheque for Dfl. 47,50 to: (include 4 or 5 digit number on back of cheque!) University of Amsterdam J.C. Scholtes Dufaystraat 1 1075 GR Amsterdam The Netherlands scholtes at alf.let.uva.nl Do not forget to mention a surface shipping address. Please allow 2-4 weeks for delivery. Abstract 1.0 Machine Intelligence For over fifty years the two main directions in machine intelligence (MI), neural networks (NN) and artificial intelligence (AI), have been studied by various persons with many dfferent backgrounds. NN and AI seemed to conflict with many of the traditional sciences as well as with each other. The lack of a long research history and well defined foundations has always been an obstacle for the general acceptance of machine intelligence by other fields. At the same time, traditional schools of science such as mathematics and physics developed their own tradition of new or "intelligent" algorithms. Progress made in the field of statistical reestimation techniques such as the Hidden Markov Models (HMM) started a new phase in speech recognition. Another application of the progress of mathematics can be found in the application of the Kalman filter in the interpretation of sonar and radar signals. Much more examples of such "intelligent" algorithms can be found in the statistical classification en filtering techniques of the study of pattern recognition (PR). Here, the field of neural networks is studied with that of pattern recognition in mind. Although only global qualitative comparisons are made, the importance of the relation between them is not to be underestimated. In addition it is argued that neural networks do indeed add something to the fields of MI and PR, instead of competing or conflicting with them. 2.0 Natural Language Processing The study of natural language processing (NLP) exists even longer than that of MI. Already in the beginning of this century people tried to analyse human language with machines. However, serious efforts had to wait until the development of the digital computer in the 1940s, and even then, the possibilities were limited. For over 40 years, symbolic AI has been the most important approach in the study of NLP. That this has not always been the case, may be concluded from the early work on NLP by Harris. As a matter of fact, Chomsky's Syntactic Structures was an attack on the lack of structural proper-ties in the mathematical methods used in those days. But, as the latter's work remained the standard in NLP, the former has been forgotten completely until recently. As the scientific community in NLP devoted all its attention to the symbolic AI-like theories, the only use- ful practical implementation of NLP systems were those that were based on statistics rather than on linguistics. As a result, more and more scientists are redirecting their attention towards the statistical techniques a vailable in NLP. The field of connectionist NLP can be considered as a special case of these mathematical methods in NLP. More than one reason can be given to explain this turn in approach. On the one hand, many problems in NLP have never been addressed properly by symbolic AI. Some examples are robust behavior in noisy environments, disambiguation driven by different kinds of knowledge, commensense generalizations, and learning (or training) abilities. On the other hand, mathematical methods have become much stronger and more sensitive to spe- cific properties of language such as hierarchical structures. Last but not least, the relatively high degree of success of mathematical techniques in commercial NLP systems might have set the trend towards the implementation of simple, but straightforward algorithms. In this study, the implementation of hierarchical structures and semantical features in mathematical objects such as vectors and matrices is given much attention. These vectors can then be used in models such as neural networks, but also in sequential statistical procedures implementing similar characteristics. 3.0 Information Retrieval The study of information retrieval (IR) was traditionally related to libraries on the one hand and military applications on the other. However, as PC's grew more popular, most common users loose track of the data they produced over the last couple of years. This, together with the introduction of various "small platform" computer programs made the field of IR relevant to ordinary users. However, most of these systems still use techniques that have been developed over thirty years ago and that implement nothing more than a global surface analysis of the textual (layout) properties. No deep structure whatsoever, is incorporated in the decision whether or not to retrieve a text. There is one large dilemma in IR research. On the one hand, the data collections are so incredibly large, that any method other than a global surface analysis would fail. On the other hand, such a global analysis could never implement a contextually sensitive method to restrict the number of possible candidates returned by the retrieval system. As a result, all methods that use some linguistic knowledge exist only in laboratories and not in the real world. Conversely, all methods that are used in the real world are based on technological achievements from twenty to thirty years ago. Therefore, the field of information retrieval would be greatly indebted to a method that could incorporate more context without slowing down. As computers are only capable of processing numbers within reasonable time limits, such a method should be based on vectors of numbers rather than on symbol manipulations. This is exactly where the challenge is: on the one hand keep up the speed, and on the other hand incorporate more context. If possible, the data representation of the contextual information must not be restricted to a single type of media. It should be possible to incorporate symbolic language as well as sound, pictures and video concurrently in the retrieval phase, although one does not know exactly how yet... Here, the emphasis is more on real-time filtering of large amounts of dynamic data than on document retrieval from large (static) data bases. By incorporating more contextual information, it should be possible to implement a model that can process large amounts of unstructured text without providing the end-user with an overkill of information. 4.0 The Combination As this study is a very multi-disciplinary one, the risk exists that it remains restricted to a surface discussion of many different problems without analyzing one in depth. To avoid this, some central themes, applications and tools are chosen. The themes in this work are self- organization, distributed data representations and context. The applications are NLP and IR, the tools are (variants of) Kohonen feature maps, a well known model from neural network research. Self-organization and context are more related to each other than one may suspect. First, without the proper natural context, self-organization shall not be possible. Next, self-organization enables one to discover contextual relations that were not known before. Distributed data representation may solve many of the unsolved problems in NLP and IR by introducing a powerful and efficient knowledge integration and generalization tool. However, distributed data representation and self-organization trigger new problems that should be solved in an elegant manner. Both NLP and IR work on symbolic language. Both have properties in common but both focus on different features of language. In NLP hierarchical structures and semantical features are important. In IR the amount of data sets the limitations of the methods used. However, as computers grow more powerful and the data sets get larger and larger, both approaches get more and more common ground. By using the same models on both applications, a better understanding of both may be obtained. Both neural networks and statistics would be able to implement self-organization, distributed data and context in the same manner. In this thesis, the emphasis is on Kohonen feature maps rather than on statistics. However, it may be possible to implement many of the techniques used with regular sequential mathematical algorithms. So, the true aim of this work can be formulated as the understanding of self-organization, distributed data representation, and context in NLP and IR, by in depth analysis of Kohonen feature maps. ============================================================================== From haussler at cse.ucsc.edu Tue Jun 8 14:11:16 1993 From: haussler at cse.ucsc.edu (David Haussler) Date: Tue, 8 Jun 1993 11:11:16 -0700 Subject: COLT `93: Early registration deadline June 15 Message-ID: <199306081811.AA25547@arapaho.ucsc.edu> COLT '93 Sixth ACM Conference on Computational Learning Theory Monday, July 26 through Wednesday, July 28, 1993 University of California, Santa Cruz, California EARLY REGISTRATION DEADLINE: JUNE 15 The workshop will be held on campus, which is hidden away in the redwoods on the Pacific coast of Northern California. The workshop is sponsored by the ACM Special Interest Group on Algorithms and Computation Theory (SIGACT) and the ACM Special Interest Group on Artificial Intelligence (SIGART). The long version of this document is available by anonymous ftp from ftp.cse.ucsc.edu. To ftp the document you do the following: step 1) ftp ftp.cse.ucsc.edu, and login as "anonymous", 2) cd pub/colt, 3) binary, 4) get colt93.registration.ps. REGISTRATION INFORMATION ------------------------ Please fill in the information needed on the registration sheet Make your payment by check or international money order, in U.S. dollars and payable through a U.S. bank, to COLT '93. Mail the form together with payment (by June 15 to avoid the late fee) to: COLT '93 Dept. of Computer Science University of California Santa Cruz, California 95064 ACCOMMODATIONS AND DINING Accommodation fees are $57 per person for a double and $70 for a single per night at the College Eight Apartments. Cafeteria style breakfast (7:45 to 8:30am), lunch (12:30 to 1:30pm), and dinner (6:00 to 7:00pm) will be served in the College Eight Dining Hall. Doors close at the end of the time indicated, but dining may continue beyond this time. The first meal provided is dinner on the day of arrival and the last meal is lunch on the day you leave. NO REFUNDS can be given after June 15. Those with uncertain plans should make reservations at an off-campus hotel. Each attendee should pick one of the five accomdation packages. For shorter stays, longer stays, and other special requirements, you can get other accommodations through the Conference Office. Make reservations directly with them at (408)459-2611, fax (408)459-3422, and do this soon as on-campus rooms for the summer fill up well in advance. Off-campus hotels include the Dream Inn (408)426-4330 and the Holiday Inn (408)426-7100. Questions: e-mail colt93 at cse.ucsc.edu, fax (408)429-4829. Confirmations will be sent by e-mail. Anyone needing special arrangements to accommodate a disability should enclose a note with their registration. If you don't receive confirmation within three weeks of payment, let us know. Get updated versions of this document by anonymous ftp from ftp.cse.ucsc.edu. CONFERENCE REGISTRATION FORM (see accompanying information for details) Name: ___________________________________ Affiliation: ___________________________________ Address: ___________________________________ City: ________________ State: ____________ Zip: ________________ Country: ____________________ Telephone: (____) ________________ Email: ________________________ The registration fee includes a copy of the proceedings. ACM/SIG Members: $165 (with banquet) $___________ Non-Members: $185 (with banquet) $___________ Late: $220 (postmarked after June 15) $___________ Full time students: $80 (no banquet) $___________ Extra banquet tickets: ___ (quantity) x $18 = $___________ How many in your party have dietary restrictions? Vegetarian: ___________ Other: ___________ Shirt size, please circle one: small medium large x-large ACCOMODATIONS: Pick one package: _____ Package 1: Sun, Mon, Tue nights: $171 double, $210 single. _____ Package 2: Sat, Sun, Mon, Tue nights: $228 double, $280 single. _____ Package 3: Sun, Mon, Tues, Wed nights: $228 double, $280 single. _____ Package 4: Sat, Sun, Mon, Tue, Wed nights: $285 double, $350 single. ______Other housing arrangement. Each 4-person apartment has a living room, a kitchen, two common bathrooms, and either four single separate rooms, two double rooms, or two single and one double room. We need the following information to make room assignments: Gender (M/F): __________ Smoker (Y/N): __________ Roommate Preference: ____________________ AMOUNT ENCLOSED: Registration $___________________ Banquet tickets $___________________ Accommodations $___________________ TOTAL $___________________ Mail this form together with payment (by June 15 to avoid the late fee) to: COLT '93, Dept. of Computer Science, Univ. California, Santa Cruz, CA 95064 COLT '93 --- Conference Schedule Sixth ACM Conference on Computational Learning Theory Monday, July 26 through Wednesday, July 28, 1993 University of California, Santa Cruz, California SUNDAY, JULY 25 4:00 - 6:00 pm, Housing Registration, College Eight Satellite Office. 7:00 - 10:00 pm, Reception, Oakes Learning Center. Preregistered attendees may check in at the reception. Note: All technical sessions will take place in Oakes 105 . MONDAY, JULY 26 Session 1: Learning with Queries Chair: Dana Angluin 8:20-8:40 Learning Sparse Polynomials over Fields with Queries and Counterexamples. Robert E. Schapire and Linda M. Sellie 8:40-9:00 Learning Branching Programs with Queries. Vijay Raghavan and Dawn Wilkins 9:00-9:10 Linear Time Deterministic Learning of k-term DNF. Ulf Berggren 9:10-9:30 Asking Questions to Minimize Errors. Nader H. Bshouty, Sally A. Goldman, Thomas R. Hancock, and Sleiman Matar 9:30-9:40 Parameterized Learning Complexity. Rodney G. Downey, Patricia Evans, and Michael R. Fellows 9:40-10:00 On the Query Complexity of Learning. Sampath K. Kannan 10:00 - 10:30 BREAK Session 2: New Learning Models and Problems Chair: Sally Goldman 10:30-10:50 Teaching a Smarter Learner. Sally A. Goldman and H. David Mathias 10:50-11:00 Learning and Robust Learning of Product Distributions. Klaus-U. Hoffgen 11:00-11:20 A Model of Sequence Extrapolation. Philip Laird, Ronald Saul and Peter Dunning 11:20-11:30 On Polynomial-Time Probably Almost Discriminative Learnability. Kenji Yamanishi 11:30-11:50 Learning from a Population of Hypotheses. Michael Kearns and Sebastian Seung 11:50-12:00 On Probably Correct Classification of Concepts. S.R. Kulkarni and O. Zeitouni 12:00 - 1:40 LUNCH Session 3: Inductive Inference; Neural Nets Chair: Bob Daley 1:40-2:00 On the Structure of Degrees of Inferability. Martin Kummer and Frank Stephan 2:00-2:20 Language Learning in Dependence on the Space of Hypotheses. Steffen Lange and Thomas Zeugmann 2:20-2:30 On the Power of Sigmoid Neural Networks. Joe Kilian and Hava T. Siegelmann 2:30-2:40 Lower Bounds on the Vapnik-Chervonenkis Dimension of Multi-layer Threshold Networks. Peter L. Bartlett 2:40-2:50 Average Case Analysis of the Clipped Hebb Rule for Nonoverlapping Perceptron Networks. Mostefa Golea and Mario Marchand 2:50-3:00 On the Power of Polynomial Discriminators and Radial Basis Function Networks. Martin Anthony and Sean B. Holden 3:00 - 3:30 BREAK 3:30-4:30 Invited Talk by Geoffrey Hinton The Minimum Description Length Principle and Neural Networks. 4:45 - ? Impromptu talks, open problems, etc. 7:00 - 10:00 pm, Banquet, barbeque pit outside Porter Dining Hall. TUESDAY, JULY 27 Session 4: Inductive Inference Chair: Rolf Wiehagen 8:20-8:40 The Impact of Forgetting on Learning Machines. Rusins Freivalds, Efim Kinber, and Carl H. Smith 8:40-8:50 On Parallel Learning. Efim Kinber, Carl H. Smith, Mahendran Velauthapillai, and Rolf Wiehagen 8:50-9:10 Capabilities of Probabilistic Learners with Bounded Mind Changes. Robert Daley and Bala Kalyanasundaram 9:10-9:20 Probability is More Powerful than Team for Language Identification from Positive Data. Sanjay Jain and Arun Sharma 9:20-9:40 Capabilities of Fallible FINite Learning. Robert Daley, Bala Kalyanasundaram, and Mahendran Velauthapillai 9:40-9:50 On Learning in the Limit and Non-uniform (epsilon, delta)-Learning. Shai Ben-David and Michal Jacovi 9:50 - 10:20 BREAK Session 5: Formal Languages, Rectangles, and Noise Chair: Takeshi Shinohara 10:20-10:40 Learning Fallible Deterministic Finite Automata. Dana Ron and Ronitt Rubinfeld 10:40-11:00 Learning Two-Tape Automata from Queries and Counterexamples. Takashi Yokomori 11:00-11:10 Efficient Identification of Regular Expressions from Representative Examples. Alvis Brazma 11:10-11:30 Learning Unions of Two Rectangles in the Plane with Equivalence Queries. Zhixiang Chen 11:30-11:50 On-line Learning of Rectangles in Noisy Environments. Peter Auer 11:50-12:00 Statistical Queries and Faulty PAC Oracles. Scott Evan Decatur 12:00 - 1:40 LUNCH Session 6: New Models; Linear Thresholds Chair: Wray Buntine 1:40-2:00 Learning an Unknown Randomized Algorithm from its Behavior. William Evans, Sridhar Rajagopalan, and Umesh Vazirani 2:00-2:20 Piecemeal Learning of an Unknown Environment. Margrit Betke, Ronald L. Rivest, and Mona Singh 2:20-2:40 Learning with Restricted Focus of Attention. Shai Ben-David and Eli Dichterman 2:40-2:50 Polynomial Learnability of Linear Threshold Approximations. Tom Bylander 2:50-3:00 Rate of Approximation Results Motivated by Robust Neural Network Learning. Christian Darken, Michael Donahue, Leonid Gurvits, and Eduardo Sontag 3:00-3:10 On the Average Tractability of Binary Integer Programming and the Curious Transition to Generalization in Learning Majority Functions. Shao C. Fang and Santosh S. Venkatesh 3:10 - 3:30 BREAK 3:30-4:30 Invited Talk by John Grefenstette Genetic Algorithms and Machine Learning 4:45 - ? Impromptu talks, open problems, etc. 7:00 - 8:30 Poster Session and Dessert Oakes Learning Center 8:30 - 10:00 Business Meeting Oakes 105 WEDNESDAY, JULY 28 Session 7: Pac Learning Chair: Yishay Mansour 8:20-8:40 On Learning Visual Concepts and DNF Formulae. Eyal Kushilevitz and Dan Roth 8:40-9:00 Localization vs. Identification of Semi-Algebraic Sets. Shai Ben-David and Michael Lindenbaum 9:00-9:20 On Learning Embedded Symmetric Concepts. Avrim Blum, Prasad Chalasani, and Jeffrey Jackson 9:20-9:30 Amplification of Weak Learning Under the Uniform Distribution. Dan Boneh and Richard J. Lipton 9:30-9:50 Learning Decision Trees on the Uniform Distribution. Thomas R. Hancock 9:50 - 10:20 BREAK Session 8: VC dimension, Learning Complexity, and Lower Bounds Chair: Sebastian Seung 10:20-10:40 Bounding the Vapnik-Chervonenkis Dimension of Concept Classes Parameterized by Real Numbers. Paul Goldberg and Mark Jerrum 10:40-10:50 Occam's Razor for Functions. B.K. Natarajan 10:50-11:00 Conservativeness and Monotonicity for Learning Algorithms. Eiji Takimoto and Akira Maruoka 11:00-11:20 Lower Bounds for PAC Learning with Queries. Gyorgy Turan 11:20-11:40 On the Complexity of Function Learning. Peter Auer, Philip M. Long, Wolfgang Maass, and Gerhard J. Woeginger 11:40-12:00 General Bounds on the Number of Examples Needed for Learning Probabilistic Concepts. Hans Ulrich Simon NOON: Check-out of Rooms 12:00 - 1:40 LUNCH Session 9: On-Line Learning Chair: Kenji Yamanishi 1:40-2:00 On-line Learning with Linear Loss Constraints. Nick Littlestone and Philip M. Long 2:00-2:10 The `Lob-Pass' Problem and an On-line Learning Model of Rational Choice. Naoki Abe and Jun-ichi Takeuchi 2:10-2:30 Worst-case Quadratic Loss Bounds for a Generalization of the Widrow-Hoff Rule. Nicolo Cesa-Bianchi, Philip M. Long, and Manfred K. Warmuth 2:30-2:40 On-line Learning of Functions of Bounded Variation under Various Sampling Schemes. S.E. Posner and S.R. Kulkarni 2:40-2:50 Acceleration of Learning in Binary Choice Problems. Yoshiyuki Kabashima and Shigeru Shinomoto 2:50-3:10 Learning Binary Relations Using Weighted Majority Voting. Sally A. Goldman and Manfred K. Warmuth 3:10 CONFERENCE ENDS 3:10 - ? Last fling on the Boardwalk. From rob at comec4.mh.ua.edu Tue Jun 8 22:35:30 1993 From: rob at comec4.mh.ua.edu (Robert Elliott Smith.dat) Date: Tue, 08 Jun 93 20:35:30 -0600 Subject: ICGA workshop proposal/participation request Message-ID: <9306090135.AA13062@comec4.mh.ua.edu> Call for Workshop Proposals and Workshop Participation ICGA-93 The Fifth International Conference on Genetic Algorithms 17-21 July, 1993 University of Illinois at Urbana-Champaign Early this Spring, the organizers of ICGA solicited proposals for workshops. Proposals for six workshops have been received and accepted thus far. These workshops are listed below. ICGA attendees are encouraged to contact the organizers of workshops in which they would like to participate. Email addresses for workshop organizers are included below. The organizers would also like to encourage proposals for additional workshops. If you would like to organize and chair a workshop, please submit a one-paragraph proposal, including a description of the workshop's topic, and some idea of how the workshop will be organized. Workshop proposals will be accepted by email only at icga93 at pele.cs.unm.edu At the ICGA91 (in San Diego), the workshops served an important role, providing smaller, less formal meetings for the discussion of specific topics related to genetic algorithms research. The organizers hope that this tradition will continue at ICGA93. ICGA93 workshops (if you wish to partipate, please write directly to the workshop's organizer): ------------------------------------------------------------------------ Genetic Programming Organizer: Kim Kinnear (kim.kinnear at sun.com) Engineering Applications of GAs (structural shape and topology optimization) Organizer: Mark Jakiela (jakiela at MIT.EDU) Discovery of long-action chains and emergence of hierarchies in classifier systems Organizers: Alex Shevorshkon Erhard Bruderer (Erhard.Bruderer at um.cc.umich.edu) Niching Methods Organizer: Alan Schultz (schultz at aic.nrl.navy.mil) Sam Mahfoud (mahfoud at gal4.ge.uiuc.edu) Combinations of GAs and Neural Nets (COGANN) Organizer: J. David Schaffer (ds1 at philabs.Philips.Com) GAs in control systems Organizer: Terry Fogarty (tc_fogar at pat.uwe-bristol.ac.uk) From Scott_Fahlman at SEF1.SLISP.CS.CMU.EDU Wed Jun 9 13:46:48 1993 From: Scott_Fahlman at SEF1.SLISP.CS.CMU.EDU (Scott_Fahlman@SEF1.SLISP.CS.CMU.EDU) Date: Wed, 09 Jun 93 13:46:48 -0400 Subject: Quick survey: Cascor and Quickprop Message-ID: Distributing code by anonymous FTP is convenient for everyone with decent internet connections, but it has the disadvantage that it is hard to keep track of who is using the code. Every so often we need to justify our existence to someone and need to show them that there are a non-trivial number of real users out there. If you are now using, or have recently used, any of my neural net algorithms or programs (Quickprop, Cascade-Correlation, Recurrent Cascade-Correlation), I would very much appreciate it if you would send me a quick E-mail message with your name, organization, and (if it's not a secret) just a few words about what you are doing with it. (For example: "classifying textures in satellite photos".) For those of you who don't know about the availability of this code (and related papers), I enclose below some instructions on how to get these things by anonymous FTP. Thanks, Scott =========================================================================== Scott E. Fahlman Internet: sef+ at cs.cmu.edu Senior Research Scientist Phone: 412 268-2575 School of Computer Science Fax: 412 681-5739 Carnegie Mellon University Latitude: 40:26:33 N 5000 Forbes Avenue Longitude: 79:56:48 W Pittsburgh, PA 15213 =========================================================================== Public-domain simulation programs for the Quickprop, Cascade-Correlation, and Recurrent Cascade-Correlation learning algorithms are available via anonymous FTP on the Internet. This code is distributed without charge on an "as is" basis. There is no warranty of any kind by the authors or by Carnegie-Mellon University. Instructions for obtaining the code via FTP are included below. If you can't get it by FTP, contact me by E-mail (sef+ at cs.cmu.edu) and I'll try *once* to mail it to you. Specify whether you want the C or Lisp version. If it bounces or your mailer rejects such a large message, I don't have time to try a lot of other delivery methods. HOW TO GET IT: For people (at CMU, MIT, and soon some other places) with access to the Andrew File System (AFS), you can access the files directly from directory "/afs/cs.cmu.edu/project/connect/code". This file system uses the same syntactic conventions as BSD Unix: case sensitive names, slashes for subdirectories, no version numbers, etc. The protection scheme is a bit different, but that shouldn't matter to people just trying to read these files. For people accessing these files via FTP: 1. Create an FTP connection from wherever you are to machine "ftp.cs.cmu.edu". The internet address of this machine is 128.2.206.173, for those who need it. 2. Log in as user "anonymous" with your own ID as password. You may see an error message that says "filenames may not have /.. in them" or something like that. Just ignore it. 3. Change remote directory to "/afs/cs/project/connect/code". NOTE: You must do this in a single operation. Some of the super directories on this path are protected against outside users. 4. At this point FTP should be able to get a listing of files in this directory with DIR and fetch the ones you want with GET. (The exact FTP commands you use depend on your local FTP server.) Partial contents: quickprop1.lisp Original Common Lisp version of Quickprop. quickprop1.c C version by Terry Regier, U. Cal. Berkeley. backprop.lisp Overlay for quickprop1.lisp. Turns it into backprop. cascor1.lisp Original Common Lisp version of Cascade-Correlation. cascor1.c C version by Scott Crowder, Carnegie Mellon rcc1.lisp Common Lisp version of Recurrent Cascade-Correlation. rcc1.c C version, trans. by Conor Doherty, Univ. Coll. Dublin nevprop1.15.shar Better quickprop implementation in C from U. of Nevada. --------------------------------------------------------------------------- Tech reports describing these algorithms can also be obtained via FTP. These are Postscript files, processed with the Unix compress/uncompress program. unix> ftp ftp.cs.cmu.edu (or 128.2.206.173) Name: anonymous Password: ftp> cd /afs/cs/project/connect/tr ftp> binary ftp> get filename.ps.Z ftp> quit unix> uncompress filename.ps.Z unix> lpr filename.ps (or however you print postscript files) For "filename", sustitute the following: qp-tr Paper on Quickprop and other backprop speedups. cascor-tr Cascade-Correlation paper. rcc-tr Recurrent Cascade-Correlation paper. precision Hoehfeld-Fahlman paper on Cascade-Correlation with limited numerical precision. --------------------------------------------------------------------------- The following are the published conference and journal versions of the above (in some cases shortened and revised): Scott E. Fahlman (1988) "Faster-Learning Variations on Back-Propagation: An Empirical Study" in (\it Proceedings, 1988 Connectionist Models Summer School}, D. S. Touretzky, G. E. Hinton, and T. J. Sejnowski (eds.), Morgan Kaufmann Publishers, Los Altos CA, pp. 38-51. Scott E. Fahlman and Christian Lebiere (1990) "The Cascade-Correlation Learning Architecture", in {\it Advances in Neural Information Processing Systems 2}, D. S. Touretzky (ed.), Morgan Kaufmann Publishers, Los Altos CA, pp. 524-532. Scott E. Fahlman (1991) "The Recurrent Cascade-Correlation Architecture" in {\it Advances in Neural Information Processing Systems 3}, R. P. Lippmann, J. E. Moody, and D. S. Touretzky (eds.), Morgan Kaufmann Pulishers, Los Altos CA, pp. 190-196. Marcus Hoehfeld and Scott E. Fahlman (1992) "Learning with Limited Numerical Precision Using the Cascade-Correlation Learning Algorithm" in IEEE Transactions on Neural Networks, Vol. 3, no. 4, July 1992, pp. 602-611. From jose at learning.siemens.com Wed Jun 9 09:21:35 1993 From: jose at learning.siemens.com (Steve Hanson,(U,6500,,p)) Date: Wed, 9 Jun 1993 09:21:35 -0400 (EDT) Subject: NIPS5 Oversight Message-ID: <0g5SDTG1GEMnEpEfFi@tractatus.siemens.com> NIPS-5 attendees: Due to an oversight we regret the inadvertent exclusion of 3 papers from the recent NIPS-5 volume. These papers were: Mark Plutowski, Garrison Cottrell and Halbert White: Learning Mackey-Glass from 25 examples, Plus or Minus 2 Yehuda Salu: Classification of Multi-Spectral Pixels by the Binary Diamond Neural Network A. C. Tsoi, D.S.C. So and A. Sergejew: Classification of Electroencephalograms using Artificial Neural Networks We are writing this note to (1) acknowledge our error (2) point out where you can obtain a present copy of the author's papers and (3) inform you that they will appear in their existing form or an updated form in NIPS Vol. 6. Presently, Morgan Kaufmann will be sending a bundle of the 3 formatted papers to all NIPS-5 attendees, these will be marked as NIPS-5 Addendum. You should also be able to retrieve an official copy from NEUROPROSE archive. Again, we apologize for the oversight to the authors. Stephen J. Hanson, General Chair Jack Cowan, Program Chair C. Lee Giles, Publications Chair From sutton at gte.com Fri Jun 11 13:44:08 1993 From: sutton at gte.com (Rich Sutton) Date: Fri, 11 Jun 93 13:44:08 -0400 Subject: Reinforcement Learning Workshop - Call for Participation Message-ID: <9306111744.AA08858@bunny.gte.com> LAST CALL FOR PARTICIPATION "REINFORCEMENT LEARNING: What We Know, What We Need" an Informal Workshop to follow ML93 (10th Int. Conf. on Machine Learning) June 30 & July 1, University of Massachusetts, Amherst Reinforcement learning is a simple way of framing the problem of an autonomous agent learning and interacting with the world to achieve a goal. This has been an active area of machine learning research for the last 5 years. The objective of this workshop is to present concisely the current state of the art in reinforcement learning and to identify and highlight critical open problems. The intended audience is all learning researchers interested in reinforcement learning. The first half of the workshop will be mainly tutorial while the second half will define and explore open problems. The entire workshop will last approximately one and three-quarters days. It is possible to register for the workshop but not the conference, but attending the conference is highly recommended as many new RL results will be presented in the conference and these will not be repeated in the workshop. Registration information is given at the end of this message. Program Committee: Rich Sutton (chair), Nils Nilsson, Leslie Kaelbling, Satinder Singh, Sridhar Mahadevan, Andy Barto, Steve Whitehead ............................................................................ PROGRAM INFORMATION The following draft program is divided into "sessions", each consisting of a set of presentations on a single topic. The earlier sessions are more "What we know" and the later sessions are more "What we Need", although some of each will be covered in all sessions. Sessions last 60-120 minutes and are separated by 30 minute breaks. Each session has an organizer and a series of speakers, one of which is likely to be the organizer herself. In most cases the speakers are meant to cover a body of work, not just their own, as a survey directed at identifying and explaining the key issues and open problems. The organizer works with the speakers to assure this (the organizer also has primary responsibility for picking the speakers, and chairs the session). ***************************************************************************** PRELIMINARY SCHEDULE: June 30: 9:00--10:30 Session 1: Defining Features of RL 10:30--11:00 Break 11:00--12:30 Session 2: RL and Dynamic Programming 12:30--2:00 Lunch 2:00--3:30 Session 3: Theory: Stochastic Approximation and Convergence 3:30--4:00 Break 4:00--5:00 Session 4: Hidden State and Short-Term Memory July 1: 9:00--11:00 Session 5: Structural Generalization: Scaling RL to Large State Spaces 11:00--11:30 Break 11:30--12:30 Session 6: Hierarchy and Abstraction 12:30--1:30 Lunch 1:30--2:30 Session 7: Strategies for Exploration 2:30--3:30 Session 8: Relationships to Neuroscience and Evolution ***************************************************************************** PRELIMINARY PROGRAM --------------------------------------------------------------------------- Session 1: Defining Features of Reinforcement Learning Organizer: Rich Sutton, rich at gte.com "Welcome and Announcements" by Rich Sutton, GTE (10 minutes) "History of RL" by Harry Klopf, WPAFB (25 minutes) "Delayed Reward: TD Learning and TD-Gammon" by Rich Sutton, GTE (50 minutes) The intent of the first two talks is to start getting across certain key ideas about reinforcement learning: 1) RL is a problem, not a class of algorithms, 2) the distinguishing features of the RL problem are trial-and-error search and delayed reward. The third talk is a tutorial presentation of temporal-difference learning, the basis of learning methods for handling delayed reward. This talk will also present Gerry Tesauro's TD-Gammon, a TD-learning system that learned to play backgammon at a grandmaster level. (There is still an outside chance that Tesauro will be able to attend the workshop and present TD-Gammon himself.) --------------------------------------------------------------------------- Session 2: RL and Dynamic Programming Organizer: Andy Barto, barto at cs.umass.edu "Q-learning" by Chris Watkins, Morning Side Inc (30 minutes) "RL and Planning" by Andrew Moore, MIT (30 minutes) "Asynchronous Dynamic Programming" by Andy Barto, UMass (30 minutes) These talks will cover the basic ideas of RL and its relationship to dynamic programming and planning. Including Markov Decision Tasks. --------------------------------------------------------------------------- Session 3: New Results in RL and Asynchronous DP Organizer: Satinder Singh, singh at cs.umass.edu "Introduction, Notation, and Theme" by Satinder P. Singh, UMass "Stochastic Approximation: Convergence Results" by T Jaakkola & M Jordan, MIT "Asychronous Policy Iteration" by Ron Williams, Northeastern "Convergence Proof of Adaptive Asynchronous DP" by Vijaykumar Gullapalli, UMass "Discussion of *some* Future Directions for Theoretical Work" by ? This session consists of two parts. In the first part we present a new and fairly complete theory of (asymptotic) convergence for reinforcement learning (with lookup tables as function approximators). This theory explains RL algorithms as replacing the full-backup operator of classical dynamic programming algorithms by a random backup operator that is unbiased. We present an extension to classical stochastic approximation theory (e.g., Dvoretzky's) to derive probability one convergence proofs for Q-learning, TD(0), and TD(lambda), that are different, and perhaps simpler, than previously available proofs. We will also use the stochastic approximation framework to highlight the contribution made by reinforcement learning algorithms such as TD, and Q-learning, to the entire class of iterative methods for solving the Bellman equations associated with Markovian Decision Tasks. The second part deals with contributions by RL researchers to asynchronous DP. Williams will present a set of algorithms (and convergence results) that are asynchronous at a finer grain than classical asynchronous value iteration, but still use "full" backup operators. These algorithms are related to the modified policy iteration algorithm of Puterman and Shin, as well as to the ACE/ASE (actor-critic) architecture of Barto, Sutton and Anderson. Subsequently, Gullapalli will present a proof of convergence for "adaptive" asynchronous value iteration that shows that in order to ensure convergence with probability one, one has to place constraints on how many model-building steps have to be be performed between two consecutive updates of the value function. Lastly we will discuss some pressing theoretical questions regarding rate of convergence for reinforcement learning algorithms. --------------------------------------------------------------------------- Session 4: Hidden State and Short-Term Memory Organizer: Lonnie Chrisman, lonnie.chrisman at cs.cmu.edu Speakers: Lonnie Chrisman & Michael Littman, CMU Many realistic agents cannot directly observe every relevant aspect of their environment at every moment in time. Such hidden state causes problems for many reinforcement learning algorithms, often causing temporal differencing methods to become unstable and making policies that simply map sensory input to action insufficient. In this session we will examine the problems of hidden state and of learning how to best organize short-term memory. I will review and compare existing approaches such as those of Whitehead & Ballard, Chrisman, Lin & Mitchell, McCallum, and Ring. I will also give a tutorial on the theories of Partially Observable Markovian Decision Processes, Hidden Markov Models, and related learning algorithms such as Balm-Welsh/EM as they are relevant to reinforcement learning. Note: Andrew McCallum will present a paper on this topic as part of the conference; that material will not be repeated in the workshop. --------------------------------------------------------------------------- Session 5: Structural Generalization: Scaling RL to Large State Spaces Organizer: Sridhar Mahadevan, sridhar at watson.ibm.com "Motivation and Introduction" by Sridhar Mahadevan, IBM "Neural Nets" by Long-Ji Lin, Siemens "CMAC" by Tom Miller, Univ. New Hampshire "Kd-trees and CART" by Marcos Salganicoff, UPenn "Learning Teleo-Reactive Trees" by Nils Nilsson, Stanford "Function Approximation in RL: Issues and Approaches" by Richard Yee, UMass "RL with Analog State and Action Vectors", Leemon Baird, WPAFB RL is slow to converge in tasks with high-dimensional continuous state spaces, particularly given sparse rewards. One fundamental issue in scaling RL to such tasks is structural credit assignment, which deals with inferring rewards in novel situations. This problem can be viewed as a supervised learning task, the goal being to learn a function from instances of states, actions, and rewards. Of course, the function cannot be stored exhaustively as a table, and the challenge is devise more compact storage methods. In this session we will discuss some of the different approaches to the structural generalization problem. Note: Steve Whitehead & Rich Sutton will present a paper on this topic as part of the confernece; that material will not be repeated in the workshop. --------------------------------------------------------------------------- Session 6: Hierarchy and Abstraction Organizer: Leslie Kaelbling, lpk at cs.brown.edu Speakers: To be determined Too much of RL is concerned with low-level actions and low-level (single time step) models. How can we model the world, and plan about actions, at a higher level, or over longer time scales? How can we integrate models and actions at different time scales and levels of abstraction? To address these questions, several researchers have proposed models of hierarchical learning and planning, e.g., Satinder Singh, Mark Ring, Chris Watkins, Long-ji Lin, Leslie Kaelbling, and Peter Dayan & Geoff Hinton. The format for this session will be a brief introduction to the problem by the session organizer followed by short talks and discussion. Speakers have not yet been determined. Note: Kaelbling will also speak on this topic as part of the conference; that material will not be repeated in the workshop. ----------------------------------------------------------------------------- Session 7: Strategies for Exploration Organizer: Steve Whitehead, swhitehead at gte.com Exploration is essential to reinforcement learning, since it is through exploration, that an agent learns about its environment. Naive exploration can easily result in intractably slow learning. On the other hand, exploration strategies that are carefully structured or exploit external sources of bias can do much better. A variety of approaches to exploration have been devised over the last few years (e.g., Kaelbling, Sutton, Thrun, Koenig, Lin, Clouse, Whitehead). The goal of this session is to review these techniques, understand their similarities and differences, understand when and why they work, determine their impact on learning time, and to the extent possible organize them taxonomically. The session will consist of a short introduction by the session organizer followed by a open discussion. The discussion will be informal but aimed at issues raised during the monologue. An informal panel of researchers will be on hand to participate in the discussion and answer questions about their work in this area. ----------------------------------------------------------------------------- Session 8: Relationships to Neuroscience and Evolution Organizer: Rich Sutton, rich at gte.com We close the workshop with a reminder of RL's links to neuroscience and to Genetic Algorithms / Classifier Systems: "RL in the Brain: Developing Connections Through Prediction" by R Montague, Salk "RL and Genetic Classifier Systems" by Stewart Wilson, Roland Institute Abstract of first talk: Both vertebrates and invertebrate possess diffusely projecting neuromodulatory systems. In the vertebrate, it is known that these systems are involved in the development of cerebral cortical structures and can deliver reward and/or salience signals to the cerebral cortex and other structures to influence learning in the adult. Recent data in primates suggest that this latter influence obtains because changes in firing in nuclei that deliver the neuromodulators reflect the difference in the predicted and actual reward, i.e., a prediction error. This relationship is qualitatively similar to that predicted by Sutton and Barto's classical conditioning theory. These systems innervate large expanses of cortical and subcortical turf through extensive axonal projections that originate in midbrain and basal forebrain nuclei and deliver such compounds as dopamine, serotonin, norepinephrine, and acetylcholine to their targets. The small number of neurons comprising these subcortical nuclei relative to the extent of the territory their axons innervate suggests that the nuclei are reporting scalar signals to their target structures. These facts are synthesized into a single framework which relates the development of brain structures and conditioning in adult brains. We postulate a modification to Hebbian accounts of self-organization: Hebbian learning is conditional on a incorrect prediction of future delivered reinforcement from a diffuse neuromodulatory system. The reinforcement signal is derived both from externally driven contingencies such as proprioception from eye movements as well as from internal pathways leading from cortical areas to subcortical nuclei. We suggest a specific model for how such predictions are made in the vertebrate and invertebrate brain. We illustrate the framework with examples ranging from the development of sensory and sensory-motor maps to foraging behavior in bumble-bees. ****************************************************************************** GENERAL INFO ON REGISTERING FOR ML93 AND WORKSHOPS: Tenth International Conference on Machine Learning (ML93) --------------------------------------------------------- The conference will be held at the University of Massachusetts in Amherst, Massachusetts, from June 27 (Sunday) through June 29 (Tuesday). The con- ference will feature four invited talks and forty-six paper presentations. The invited speakers are Leo Breiman (U.C. Berkeley, Statistics), Micki Chi (U. Pittsburgh, Psychology), Michael Lloyd-Hart (U. Arizona, Adaptive Optics Group of Steward Observatory), and Pat Langley (Siemens, Machine Learning). Following the conference, there will be three informal workshops: Workshop #A: Reinforcement Learning: What We Know, What We Need (June 30 - July 1) Organizers: R. Sutton (chair), N. Nilsson, L. Kaelbling, S. Singh, S. Mahadevan, A. Barto, S. Whitehead Workshop #B: Fielded Applications of Machine Learning (June 30 - July 1) Organizers: P. Langley, Y. Kodratoff Workshop #C: Knowledge Compilation and Speedup Learning (June 30) Organizers: D. Subramanian, D. Fisher, P. Tadepalli Options and fees: Conference registration fee $140 regular $110 student Breakfast/lunch meal plan (June 27-29) $33 Dormitory housing (nights of June 26-28) $63 single occupancy $51 double occupancy Workshop A (June 30-July 1) $40 Workshop B (June 30-July 1) $40 Breakfast/lunch meal plan (June 30-July 1) $22 Dormitory housing (nights of June 29-30) $42 single occupancy $34 double occupancy Workshop C (June 30) $20 Breakfast/lunch meal plan (June 30) $11 Dormitory housing (night of June 29) $21 single occupancy $17 double occupancy Administrative fee (required) $10 Late fee (received after May 10) $30 To obtain a FAX of the registration form, send an email request to Paul Utgoff ml93 at cs.umass.edu or utgoff at cs.umass.edu From gary at cs.ucsd.edu Fri Jun 11 18:53:33 1993 From: gary at cs.ucsd.edu (Gary Cottrell) Date: Fri, 11 Jun 93 15:53:33 -0700 Subject: NIPS5 Oversight Message-ID: <9306112253.AA06151@odin.ucsd.edu> FYI, to retrieve Plutowski, Cottrell and White: Learning Mackey-Glass from 25 examples, Plus or Minus 2 The file on neuroprose is: pluto.nips92.ps.Z A script file is attached at the end of this note. Gary Cottrell 619-534-6640 Reception: 619-534-6005 FAX: 619-534-7029 Computer Science and Engineering 0114 University of California San Diego La Jolla, Ca. 92093 gary at cs.ucsd.edu (INTERNET) gcottrell at ucsd.edu (BITNET, almost anything) ..!uunet!ucsd!gcottrell (UUCP) RE: From gjacobs at qualcomm.com Fri Jun 11 14:00:40 1993 From: gjacobs at qualcomm.com (Gary Jacobs) Date: Fri, 11 Jun 1993 11:00:40 -0700 Subject: WCCI '94 Announcement and Call for Papers Message-ID: <9306111801.AA17745@harvey> Gary Jacobs gjacobs at qualcomm.com (619)597-5029 voice (619)452-9096 fax HARD FACT IN A WORLD OF FANTASY A world of sheer fantasy awaits your arrival at the IEEE World Congress on Computational Intelligence next year; our host is Walt Disney World in Orlando Florida. Simultaneous Neural Network, Fuzzy Logic and Evolutionary Programming conferences will provide an unprecedented opportunity for technical development while the charms of the nearby Magic Kingdom and Epcot Center attempt to excite your fancies. The role imagination has played in the development of Computational Intelligence techniques is well known; before they became "innovative" the various CI technologies were dismissed as "fantasies" of brilliant minds. Now these tools are real; perhaps it's only appropriate that they should be further explored and their creators honored in a world of the imagination, a world where dreams come true. Share your facts at Disney World; share your imagination. Come to the IEEE World Congress on Computational Intelligence. It's as new as tomorrow. ___________________________________________________________________________ ***CALL FOR PAPERS*** ___________________________________________________ IEEE WORLD CONGRESS ON COMPUTATIONAL INTELLIGENCE ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ * IEEE International Conference on Neural Networks * * FUZZ/IEEE '94 * * IEEE International Symposium on Evolutionary Computation * June 26 - July 2, 1994 Walt Disney World Dolphin Hotel, Lake Buena Vista, Florida Sponsored by the IEEE Neural Networks Council --------------------------------------------------------------------- IEEE INTERNATIONAL CONFERENCE ON NEURAL NETWORKS Steven K. Rogers, General Chair rogers at afit.af.mil Topics: Applications, architectures, artificially intelligent neural networks, artificial life, associative memory, computational intelligence, cognitive science, embedology, filtering, fuzzy neural systems, hybrid systems, image processing, implementations, intelligent control, learning and memory, machine vision, motion analysis, neurobiology, neurocognition, neurodynamics, optimization, pattern recognition, prediction, robotics, sensation and perception, sensorimotor systems, speech, hearing and language, system identification, supervised and unsupervised learning, tactile sensors, and time series analysis. ------------------------------------------- FUZZ/IEEE '94 Piero P. Bonissone, General Chair bonissone at crd.ge.ge.com Topics: Basic principles and foundations of fuzzy logic, relations between fuzzy logic and other approximate reasoning methods, qualitative and approximate-reasoning modeling, hardware implementations of fuzzy- logic algorithms, design, analysis, and synthesis of fuzzy-logic controllers, learning and acquisition of approximate models, relations between fuzzy logic and neural networks, integration of fuzzy logic and neural networks, integration of fuzzy logic and evolutionary computing, and applications. ------------------------------------------- IEEE CONFERENCE ON EVOLUTIONARY COMPUTATION Zbigniew Michalewicz, General Chair zbyszek at mosaic.uncc.edu Topics: Theory of evolutionary computation, evolutionary computation applications, efficiency and robustness comparisons with other direct search algorithms, parallel computer applications, new ideas incorporating further evolutionary principles, artificial life, evolutionary algorithms for computational intelligence, comparisons between different variants of evolutionary algorithms, machine learning applications, evolutionary computation for neural networks, and fuzzy logic in evolutionary algorithms. --------------------------------------------------------------------- INSTRUCTIONS FOR ALL THREE CONFERENCES Papers must be received by December 10, 1993. Papers will be reviewed by senior researchers in the field, and all authors will be informed of the decisions at the end of the review proces. All accepted papers will be published in the Conference Proceedings. Six copies (one original and five copies) of the paper must be submitted. Original must be camera ready, on 8.5x11-inch white paper, one-column format in Times or similar fontstyle, 10 points or larger with one-inch margins on all four sides. Do not fold or staple the original camera-ready copy. Four pages are encouraged. The paper must not exceed six pages including figures, tables, and references, and should be written in English. Centered at the top of the first page should be the complete title, author name(s), affiliation(s) and mailing address(es). In the accompanying letter, the following information must be included: 1) Full title of paper, 2) Corresponding authors name, address, telephone and fax numbers, 3) First and second choices of technical session, 4) Preference for oral or poster presentation, and 5) Presenter's name, address, telephone and fax numbers. Mail papers to (and/or obtain further information from): World Congress on Computational Intelligence, Meeting Management, 5665 Oberlin Drive, #110, San Diego, California 92121, USA (email: 70750.345 at compuserve.com, telephone: 619-453-6222). From gasser at cs.indiana.edu Mon Jun 14 11:06:46 1993 From: gasser at cs.indiana.edu (Michael Gasser) Date: Mon, 14 Jun 1993 10:06:46 -0500 Subject: TR on language acquisition Message-ID: FTP-host: cs.indiana.edu (129.79.254.191) FTP-filename: /pub/techreports/TR384.ps.Z The following paper is available in compressed postscript form by anonymous ftp from the Indiana University Computer Science Department ftp archive (see above). The paper is 60 pages long. Hardcopies won't be available till September, I'm afraid. Comments welcome. Michael Gasser gasser at cs.indiana.edu ================================================================= Learning Words in Time: Towards a Modular Connectionist Account of the Acquisition of Receptive Morphology Michael Gasser Computer Science and Linguistics Departments Indiana University To have learned the morphology of a natural language is to have the capacity both to recognize and to produce words consisting of novel combinations of familiar morphemes. Most recent work on the acquisition of morphology takes the perspective of production, but it is receptive morphology which comes first in the child. This paper presents a connectionist model of the acquisition of the capacity to recognize morphologically complex words. The model takes sequences of phonetic segments as inputs and maps them onto output units representing the meanings of lexical and grammatical morphemes. It consists of a simple recurrent network with separate hidden-layer modules for the tasks of recognizing the root and the grammatical morphemes of the input word. Experiments with artificial language stimuli demonstrate that the model generalizes to novel words for morphological rules of all but one of the major types found in natural languages and that a version of the network with unassigned hidden-layer modules can learn to assign them to the output recognition tasks in an efficient manner. I also argue that for rules involving reduplication, that is, the copying of portions of a root, the network requires separate recurrent subnetworks for sequences of larger units such as syllables. The network can learn to develop its own syllable representations which not only support the recognition of reduplication but also provide the basis for learning to produce, as well as recognize, morphologically complex words. The model makes many detailed predictions about the learning difficulty of particular morphological rules. From dlovell at s1.elec.uq.oz.au Tue Jun 15 16:07:13 1993 From: dlovell at s1.elec.uq.oz.au (David Lovell) Date: Tue, 15 Jun 93 15:07:13 EST Subject: ACNN'94 Call for papers Message-ID: <9306150507.AA21234@c10.elec.uq.oz.au> First Announcement ACNN'94 FIFTH AUSTRALIAN CONFERENCE ON NEURAL NETWORKS 31st Jan - 2nd Feb 1994 Univeristy of Queensland St Lucia Queensland AUSTRALIA The Fifth Australian Conference on Neural Networks will be held in Brisbane on 31st January - 2nd February, 1994, at the University of Queensland. ACNN'94 is the annual national meeting of the Australian neural network community. It is a multi-disciplinary meeting and seeks contributions from Neuroscientists, Engineers, Computer Scientists, Mathematicians, Physicists and Psychologists. ACNN'94 will feature an invited keynote speaker. The program will include presentations and poster sessions. Proceedings will be printed and distributed to the attendees. Keynote Speaker Professor Teuvo Kohonen, Helsinki Technical University Call for Papers Papers on innovative applications of artificial neural networks and other research topics in ANNs are sought. Research topics of interest include, but not restricted to: Neuroscience: Integrative function of neural networks in vision, audition, motor, somatosensory and autonomic functions. Synaptic function: Cellular information processing; Theory: Learning; Generalisation; Complexity; Scaling; Stability; Dynamics; Implementation: Hardware implementation of neural networks; Analog and digital VLSI implementation; Optical implementation; Architecture and Learning Algorithms: New Architectures and learning algorithms; Hierarchy; Modularity; Learning pattern sequences; information integration; Cognitive Science and AI: Computational models of cognition and perception; Reasoning; Concept formation; Language acquisition; Neural network implementation of expert systems; Applications: Application of neural networks to signal processing and analysis; Pattern recognition; Speech, machine vision; Motor control; Robotics. The conference will also include a poster stream which is designed as a forum for presentation of work in an environment which will encourage informal discussions about methodologies and techniques. The posters will be displayed for a significant period of time, and time will be allocated for authors to be present at their poster in the conference program. Software demonstrations will be possible for authors who bring with them portable computers. The organizing committee will attempt to provide full-size compatible screens. Limited video facilities may be available. As in previous years, there will be a special poster session for postgraduate students. All students are encouraged to present research issues and preliminary findings in this session. Initial Submission of Papers As this is a multi-disciplinary meeting, papers are required to be comprehensible to an informed researcher outside the particular stream of the author in addition to the normal requirements of technical merit. Papers should be submitted as close as possible to final form and must not exceed four A4 pages (2-column format). The first page should include the title and abstract, and should leave space for, but not include the authors' names and affiliations. A cover page should be supplied giving the title of the paper, the name, and affiliation of each author, together with the postal address, the electronic mail address, the phone number, and the fax number of a designated contact author. The type font should be no smaller than 10 point except in footnotes. A serif font such as Times to New Century Schoolbook is preferred. Four copies of the paper and the front cover should be supplied. This initial submission must be on hard copy to reach us by Friday, 27th August, 1993. Each manuscript should clearly indicate submission category (from the six as listed) and author preference for oral or poster presentations. Papers should be sent to: Tracey Carney ACNN'94 Secretariat, Department of Electrical and Computer Engineering University of Queensland St Lucia, Queensland 4072 Australia. Submission Deadlines Friday, 27 August 1993 Deadline for receipt of paper submissions Friday 29th October 1993 Notification of acceptance/rejection Friday 10 December 1993 Final papers ready for camera-ready form for printing Monday 24th January 1994 Deadline for receipt of Student Session abstracts Venue University of Queensland, Brisbane, Australia ACNN'94 Organizing Committee General Chair Ah Chung Tsoi (Univ of Queensland) Technical Chair Tom Downs (Univ of Queensland) Technical Committee Yianni Attikouzel (Univ of Western Aust) Peter Bartlett (Aust National Univ) Robert Bogner (Univ of Adelaide) Terry Caelli (Univ of Melbourne) Max Coltheart (Macquarie Univ) George Coghill (Univ of Auckland) Phil Diamond (Univ of Queensland) Joachim Diederich (Queensland Univ of Tech) Tom Downs (Univ of Queensland) Simon Goss (Def Scient & Tech Org) Graeme Halford (Univ of Queensland) Richard Heath (Univ of Newcastle) Michael Humphreys (Univ of Queensland) Marwan Jabri (Sydney Univ) Andrew Jennings (Telecom) Bill Levick (Aust National Univ) Adam Kowalczyk (Telecom) Dennis Longstaff (Univ of Queensland) D Nandagopal (Def Scient & Tech Org) M Palaniswami (Univ of Melbourne) Jack Pettigrew (Univ of Queensland) Nick Redding (Def Scient & Tech Org) Janet Wiles (Univ of Queensland) Robert Williamson (Aust National Univ) Local Committee (tentative) Andrew Back (Univ of Queensland) Phil Diamond (Univ of Queensland) Joachim Diederich (Queensland Univ of Tech) Shlomo Geva (Queensland Univ of Tech) Graeme Halford (Univ of Queensland) Michael Humphreys (Univ of Queensland) Ray Lister (Univ of Queensland) Brian Lovell (Univ of Queensland) David Lovell (Univ of Queensland) Mark Schulz (Univ of Queensland) Joaquin Sitte (Queensland Univ of Tech) Guy Smith (Univ of Queensland) Janet Wiles (Univ of Queensland) Robert Young (Queensland Dept of Primary Industries) Registration The registration fee to attend ACNN'94 is: Full Time Students A $80 + $30 Late fee Academics A $200 + $50 Late fee Other A $275 + $75 Late fee Late fees will apply to any registrations posted after December 10th 1993. To be eligible for the Full Time Student rate, a letter from the Head of Department as verification of enrolment is required. Accommodation Accommodation has been block booked at King's College (to be confirmed), University of Queensland. Copies of this brochure and registration forms are available from the Secretariat Postgraduate Students Poster Session Students are invited to submit research papers to the conference program in the normal way. However, there are many cases where preliminary work is not in a suitable form for the main program, or is not complete by the submission deadline. Thus, as in past years, there will be a special poster session for students. The main aim of the session is to promote interaction among students, and to give students a chance to present their thesis research, and gain feedback from fellow students and conference attendees. All students are accepted for the session, and indeed, all are strongly encouraged to participate. To take part in the session, send 1-page giving title of paper, name, affiliation, email address, and a 200-word abstract outlining the content of the poster, to Dr Janet Wiles Department of Computer Science, University of Queensland, QLD 4072, Australia, by Monday 24th January, 1994. For questions regarding the student session, write to the above address, or email janetw at cs.uq.oz.au Note that abstracts for the student session are not referreed and will not appear in the conference proceedings. From lpratt at franklinite.Mines.Colorado.EDU Tue Jun 15 12:13:39 1993 From: lpratt at franklinite.Mines.Colorado.EDU (Lorien Y. Pratt) Date: Tue, 15 Jun 93 09:13:39 -0700 Subject: Thesis available: Transfer between neural networks Message-ID: <9306151613.AA05154@franklinite.Mines.Colorado.EDU> FTP-host: archive.cis.ohio-state.edu FTP-file: pub/neuroprose/pratt.thesis.ps.Z The file pratt.thesis.ps.Z is now available for copying from the Neuroprose repository: Transferring Previously Learned Back Propagation Networks to New Learning Tasks 144 pages Lorien Y. Pratt Colorado School of Mines (Dissertation written at Rutgers University) ABSTRACT: When people learn a new task, they often build on their ability to solve related problems. For example, a doctor moving to a new country can use prior experience to aid in diagnosing patients. A chess player can use experience with one set of end-games to aid in solving a different, but related, set. However, although people are able to perform this sort of skill transfer between tasks, most neural network training methods in use today are unable to build on their prior experience. Instead, every new task is learned from scratch. This dissertation explores how a back-propagation neural network learner can build on its previous experience. We present an algorithm, called Discriminability-Based Transfer (DBT), that facilitates the transfer of information from the learned weights of one network to the initial weights of another. Through evaluation of DBT on several benchmark tasks we demonstrate that it can speed up learning on a new task. We also show that DBT is more robust than simpler methods for transfer. -------- Sample session for obtaining this file via anonymous ftp: > ftp archive.cis.ohio-state.edu Name: anonymous Password: (type your email address her) ftp> cd pub/neuroprose ftp> binary ftp> get pratt.thesis.ps.Z ftp> quit > uncompress pratt.thesis.ps.Z > lpr pratt.thesis.ps --------- For this with email but no ftp access, you can use the mail server at Rutgers, where this document is ML-TR-37. Send email to mth at cs.rutgers.edu to request the document, or send the word `help' in a message to: ftpmail at decwrl.dec.com --------- Those without any network access at all can receive copies of this document by requesting ML-TR-37 from the following address: Technical Reports Librarian Computer Science Department Rutgers University New Brunswick, NJ 08903 USA [Currently there is no charge.] Dr. L. Y. Pratt Dept. of Math and Computer Science lpratt at mines.colorado.edu Colorado School of Mines (303) 273-3878 (work) 402 Stratton (303) 278-4552 (home) Golden, CO 80401, USA Note: I'll be travelling out of the country for the next month. If you have trouble printing this paper, I'll be available to help after I return on July 16. Send me email -Lori From cowan at synapse.uchicago.edu Tue Jun 15 18:47:17 1993 From: cowan at synapse.uchicago.edu (Jack Cowan) Date: Tue, 15 Jun 93 15:47:17 -0700 Subject: loss of Ed Posner Message-ID: I am sorry to have to transmit the very sad news that Ed Posner, the President of the NIPS Foundation, was killed this morning in a bicycling accident in Pasadena. All his many friends and colleagues will want to join me in expressing our painful feelings at such a tragic loss. Jack Cowan From biehl at connect.nbi.dk Wed Jun 16 17:21:12 1993 From: biehl at connect.nbi.dk (Michael Biehl) Date: Wed, 16 Jun 93 17:21:12 WETDST Subject: No subject Message-ID: FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/biehl.unsupervised.ps.Z *** Hardcopies cannot be provided *** The following paper has been placed in the Neuroprose archive (see above for ftp-host) in file biehl.unsupervised.ps.Z (8 pages of output) email address of author: biehl at physik.uni-wuerzburg.de ---------------------------------------------------------------- "An exactly solvable model of unsupervised learning" by Michael Biehl Abstract: A model for unsupervised learning from N-dimensional data is studied. Random training examples are drawn such that the distribution of their overlaps with a vector B is a mixture of two Gaussians of unit width and a separation rho. A student vector is generated by an online algorithm, using each example only once. The evolution of its overlap with B can be calculated exactly in the thermodynamic limit (infinite N). As a specific example Oja's rule is investigated. Its dynamics and approach to the stationary solution are solved for both a constant and an optimally chosen time-dependent learning rate. ----------------------------------------------------------------- From mdavies at psy.ox.ac.uk Thu Jun 17 11:20:34 1993 From: mdavies at psy.ox.ac.uk (Martin Davies) Date: Thu, 17 Jun 93 11:20:34 BST Subject: Euro-SPP 93: Registration and Programme Message-ID: <9306171020.AA16978@uk.ac.oxford.psy> EUROPEAN SOCIETY FOR PHILOSOPHY AND PSYCHOLOGY The Second Annual Meeting of the European Society for Philosophy and Psychology will be held at the University of Sheffield, England, from the afternoon of Saturday 3 July to the morning of Tuesday 6 July, 1993. It is still possible to register for this conference. The Registration Fee is 15 pounds sterling (or 10 pounds for students). The cost of meals and accommodation is 120 pounds sterling. For participants not requiring accommodation, the cost of meals is 70 pounds. In order to register, please contact: Professor Peter Carruthers Hang Seng Centre for Cognitive Studies Department of Philosophy University of Sheffield Sheffield S10 2TN UK Email: P.Carruthers at sheffield.ac.uk Fax: +44-742-824604 **************************************************************** PROGRAMME All sessions will take place in Stephenson Hall of Residence, Oakholme Road, Sheffield S10 3DG. SATURDAY 3 JULY Conference desk open from 12 noon 3.00 - 5.00 pm SYMPOSIUM 1: Body and Self Chair: Naomi Eilan (Philosophy, KCRC Cambridge) Speakers: John Campbell (Philosophy, Oxford) Anthony Marcel (Psychology, APU Cambridge) Michael Martin (Philosophy, London) 5.00 - 5.30 pm Tea 6.15 - 7.45pm INVITED LECTURE 1 Chair: Larry Weiskrantz (Psychology, Oxford) Speaker: Ruth Millikan (Philosophy, Connecticut) 'Synthetic Concepts: A Philosopher's Thoughts about Categorization' 8.15 pm DINNER in Firth Hall, University of Sheffield SUNDAY 4 JULY 9.00 - 11.00 am SYMPOSIUM 2: Explanation by Intentional States Chair: Christopher Peacocke (Philosophy, Oxford) Speakers: John Campbell (Philosophy, Oxford) Pascal Engel (Philosophy, CREA Paris) Gabriel Segal (Philosophy, London) 11.00 - 11.30 am Coffee 11.30 am - 1.00 pm INVITED LECTURE 2 Chair: Speaker: Marc Jeannerod (Psychology, INSERM Lyon) 'The Representing Brain: Neural Correlates of Motor Intention and Imagery' 1.00 - 2.00 pm LUNCH 2.00 - 4.00 pm SUBMITTED PAPERS Chair: Martin Davies (Philosophy, Oxford) Speaker: Manuel Garcia Carpintero (Philosophy, Barcelona) 'The Teleological Account of Content' Comments: Mike Oaksford (Psychology, Bangor) Speaker: Gregory Mulhauser (Philosophy, Edinburgh) 'Chaotic Dynamics and Introspectively Transparent Brain Processes' Comments: Peter Smith (Philosophy, Sheffield) 4.00 - 4.30 pm Tea 4.30 - 6.00 pm INVITED LECTURE 3 Chair: Speaker: Deirdre Wilson (Linguistics, London) 'Truth, Coherence and Relevance' 6.15 pm BUSINESS MEETING followed by a RECEPTION 8.00 pm DINNER at Stephenson Hall of Residence MONDAY 5 JULY 9.00 - 10.45 am SYMPOSIUM 3: The Autonomy of Social Explanation Chair: Daniel Andler (Philosophy, CREA Paris) Speakers: Pascal Boyer (Anthropology, KCRC Cambridge) John Shotter (Communication, New Hampshire) Chris Sinha (Psychology, Aarhus) 10.45 - 11.15 am Coffee 11.15 am - 1.00 pm ROUND TABLE: Neuropsychological Approaches Chair: Beatrice de Gelder (Philosophy and Psychology, Tilburg and Brussels) Speakers: Marcel Kinsbourne (Psychology, Tufts) David Perrett (Psychology, St. Andrews) Tim Shallice (Psychology, London) 1.00 - 2.00 pm LUNCH 2.00 - 4.00 pm SUBMITTED PAPERS Chair: Anthony Marcel (Psychology, APU Cambridge) Speaker: Leslie Stevenson (Philosophy, St. Andrews) 'Merleau-Ponty on the Epistemology of Touch' Comments: Naomi Eilan (Philosophy, KCRC Cambridge) Speaker: Thomas Metzinger (Philosophy, Giessen) 'Subjectivity and Mental Representation' Comments: Barry Smith (Philosophy, London) 4.00 - 4.30 pm Tea 4.30 - 6.15 pm SYMPOSIUM 4: Mindblindness: Autism and Theory of Mind Chair: Peter Carruthers (Philosophy, Sheffield) Speakers: Simon Baron-Cohen (Psychology, London) Juan Carlos Gomez (Psychology, Madrid) Pierre Jacob (Philosophy, CREA Paris) 6.30 pm A visit to Chatsworth House, including DINNER TUESDAY 6 JULY Depart after breakfast From mm at santafe.edu Fri Jun 18 14:35:36 1993 From: mm at santafe.edu (mm@santafe.edu) Date: Fri, 18 Jun 93 12:35:36 MDT Subject: paper available Message-ID: <9306181835.AA09937@lyra> The following paper is available by public ftp. Dynamics, Computation, and the ``Edge of Chaos'': A Re-Examination Melanie Mitchell James P. Crutchfield Peter T. Hraber Santa Fe Institute UC Berkeley Santa Fe Institute Santa Fe Institute Working Paper 93-06-040 Abstract In this paper we review previous work and present new work concerning the relationship between dynamical systems theory and computation. In particular, we review work by Langton (1990) and Packard (1988) on the relationship between dynamical behavior and computational capability in cellular automata (CA). We present results from an experiment similar to the one described in Packard (1988), that was cited there as evidence for the hypothesis that rules capable of performing complex computations are most likely to be found at a phase transition between ordered and chaotic behavioral regimes for CA (the "edge of chaos"). Our experiment produced very different results from the original experiment, and we suggest that the interpretation of the original results is not correct. We conclude by discussing general issues related to dynamics, computation, and the "edge of chaos" in cellular automata. To appear in G. Cowan, D. Pines, and D. Melzner (editors), _Integrative Themes_. Santa Fe Institute Stuides in the Sciences of Complexity, Proceedings Volume 19. Reading, MA: Addison-Wesley. Note: This paper is a much shorter version of our paper "Revisiting the Edge of Chaos" (SFI Working Paper 93-03-014, announced previously on this newsgroup). It contains an expanded review of previous work on relationships between dynamical systems theory and computation. To obtain an electronic copy: ftp santafe.edu login: anonymous password: cd /pub/Users/mm binary get sfi-93-06-040.ps.Z quit Then at your system: uncompress sfi-93-06-040.ps.Z lpr -P sfi-93-06-040.ps To obtain a hard copy (only if you cannot obtain an electronic copy), send a request to dlu at santafe.edu. From maass at igi.tu-graz.ac.at Mon Jun 21 09:43:41 1993 From: maass at igi.tu-graz.ac.at (Wolfgang Maass) Date: Mon, 21 Jun 93 15:43:41 +0200 Subject: new paper in neuroprose Message-ID: <9306211343.AA28749@figids01.tu-graz.ac.at> FTP-host: archive.cis.ohio-state.edu FTP-filename: /pub/neuroprose/maass.super.ps.Z The file maass.super.ps.Z is now available for copying from the Neuroprose repository. This is a 7-page long paper. Hardcopies are not available. NEURAL NETS WITH SUPERLINEAR VC-DIMENSION by Wolfgang Maass Institute for Theoretical Computer Science Technische Universitaet Graz, A-8010 Graz, Austria email: maass at igi.tu-graz.ac.at Abstract: We construct arbitrarily large feedforward neural nets of depth 3 (i.e. with 2 hidden layers) and O(w) edges that have a VC-dimension of at least w log w. The same construction can be carried out for any depth larger than 3. This construction proves that the well-known upper bound for the VC-dimension of a neural net by Cover, Baum, and Haussler is in fact asymptotically optimal for any depth 3 or larger. The Vapnik-Chervonenkis dimension (VC-dimension) is an important parameter of any neural net, since it predicts how many training examples are needed for training the net (in Valiant's model for probably approximately correct learning). One may also view our result as mathematical evidence for some type of "connectionism thesis": that a network of neuron-like elements is more than just the sum of its elements. Our result shows that in a large neural net a single weight contributes more than a constant to the VC-dimension of the neural net, and that its contribution may increase with the total size of the neural net. The current paper improves the corresponding result by the author from last year (for depth 4), and it provides the first complete write-up of the construction. From mehra at ptolemy.arc.nasa.gov Mon Jun 21 18:58:56 1993 From: mehra at ptolemy.arc.nasa.gov (Pankaj Mehra) Date: Mon, 21 Jun 93 15:58:56 PDT Subject: edited collection of ANN papers; discount Message-ID: <9306212258.AA12857@tatertot.arc.nasa.gov> Fellow Connectionists: Some of you may have already seen ``Artificial Neural Networks: Concepts and Theory,'' edited by [yours truly] and Ben Wah. It was published by IEEE Computer Society Press in August, 1992. The table of contents are attached at the end of this message. The book is hardback and has 667 pages of which approx 100 are from chapter introductions written by the editors. The list price is $70 [$55 for IEEE members]. My intent in sending this message is not so much to announce the availability of our book as it is to bring to your notice the following offer of discount: If I place an order, I get an author's discount of 40% off list price; if a school bookstore places the order, they get a 32% discount. The IEEE order no. for the book is 1997; 1-800-CS-BOOKS. If you are planning to teach a graduate course on neural networks, you will probably find that our collection of papers as well as the up-to-date bibliography at the end of each chapter introduction provide excellent starting points for independent research. -Pankaj Mehra 415/604-0165 mehra at ptolemy.arc.nasa.gov NASA - Ames Research Center, M/S 269-3 Moffett Field, CA 94035-1000 USA __________________________________________________________________________ TABLE OF CONTENTS: page ----------------- Chapter 1: INTRODUCTION Introduction by editors 1-12 An Introduction to Computing with Neural Nets, Lippmann 13-31 An Introduction to Neural Computing, Kohonen 32-46 Chapter 2: CONNECTIONIST PRIMITIVES Introduction by editors 47-55 A General Framework for Parallel Distributed Processing, Rumelhart, Hinton, & McClelland 56-82 Multilayer Feedforward Potential Function Network, Lee & Kil 83-93 Learning, Invariance, and Generalization in High-Order Networks, Giles & Maxwell 94-100 The Subspace Learning Algorithm as a Formalism for Pattern Recognition and Neural Networks, Oja & Kohonen 101-108 Chapter 3: KNOWLEDGE REPRESENTATION Introduction by editors 109-116 BoltzCONS: Reconciling Connectionism with the Recursive Nature of Stacks and Tree, Touretzky 117-125 Holographic Reduced Representations: Convolution Algebra for Compositional Distributed Representations, Plate 126-131 Efficient Inference with Multi-Place Predicates and Variables in a Connectionist System, Ajjanagadde and Shastri 132-139 Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming, Sutton 140-148 Chapter 4: LEARNING ALGORITHMS I Introduction by editors 149-166 Connectionist Learning Procedures, Hinton 167-216 30 Years of Adaptive Neural Networks: Perceptron, Madaline, and back-Propagation, Widrow and Lehr 217-244 Supervised Learning and Systems with Excess Degrees of Freedom, Jordan 245-285 The Cascade-Correlation Learning Architecture, Fahlman 286-294 Learning to Predict by the Methods of Temporal Differences, Sutton 295-330 A Theoretical Framework for Back-Propagation, le Cun 331-338 Two Problems with Backpropagation and other Steepest-Descent Learning Procedures for Networks, Sutton 339-348 Chapter 5: LEARNING ALGORITHMS II Introduction by editors 349-358 The Self-Organizing Map, Kohonen 359-375 The ART of Adaptive Pattern Recognition by a Self-Organizing Neural Network, Grossberg 376-387 Unsupervised Learning in Noise, Kosko 388-401 A Learning Algorithm for Boltzmann Machines, Ackley, Hinton & Sejnowski 402-424 Learning Algorithms and Probability Distributions in Feed- forward and Feed-back Networks, Hopfield 425-429 A Mean Field Theory Learning Algorithm for Neural Networks, Peterson & Anderson 430-454 On the Use of Backpropagation in Associative Reinforcement Learning, Williams 455-462 Chapter 6: COMPUTATIONAL LEARNING THEORY Introduction by editors 463-473 Information Theory, Complexity, and Neural Networks, Abu-Mostafa 474-478 Geometrical and Statistical Properties of Systems of Linear Inequalities with Applications in Pattern Recognition, Cover 479-487 Approximation by Superpositions of a Sigmoidal Function, Cybenko 488-499 Approximation and Estimation Bounds for Artificial Neural Networks, Barron 500-506 Generalizing the PAC Model: Sample Size Bounds From Metric Dimension-based Uniform Convergence Results, Haussler 507-512 Complete Representations for Learning from Examples, Baum 513-534 A Statistical Approach to Learning and Generalization in Neural Networks, Levin, Tishby & Solla 535-542 Chapter 7: STABILITY AND CONVERGENCE Introduction by editors 543-550 Convergence in Neural Nets, Hirsch 551-561 Statistical Neurodynamics of Associative Memory, Amari & Maginu 562-572 Stability and Adaptation in Artificial Neural Systems, Schurmann 573-580 Dynamics and Architecture for Neural Computation, Pineda 581-610 Oscillations and Synchronizations in Neural Networks: An Exploration of the Labeling Hypothesis, Atiya & Baldi 611-632 Chapter 8: EMPIRICAL STUDIES Introduction by editors 633-639 Scaling Relationships in Back-Propagation Learning: Dependence on Training Set Size, Tesauro 640-645 An Empirical Comparison of Pattern Recognition, Neural Nets, and Machine Learning Classification Methods, Weiss & Kapouleas 646-652 Basins of Attraction of Neural Network Models, Keeler 653-657 Parallel Distributed Approaches to Combinatorial Optimization: Benchmark Studies on Traveling Salesman Problem, Peterson 658-666 From P.Refenes at cs.ucl.ac.uk Tue Jun 22 04:33:31 1993 From: P.Refenes at cs.ucl.ac.uk (P.Refenes@cs.ucl.ac.uk) Date: Tue, 22 Jun 93 09:33:31 +0100 Subject: NEURAL NETWORKS IN THE CAPITAL MARKETS Message-ID: ___________________________________________________________ CALL FOR PAPERS 1ST INTERNATIONAL WORKSHOP NEURAL NETWORKS IN THE CAPITAL MARKETS LONDON BUSINESS SCHOOL, NOVEMBER 18-19 1993 Neural Networks have now been applied to a number of live systems in the capital markets and in many cases have demonstrated better performance than competing approaches. Now is the time to take a critical look at their successes and limitations and to assess their capabilities, research issues and future directions. This workshop invites original papers which represent new and significant research, development and applications in finance & investment and which cover key areas of time series forecasting, multivariate dataset analysis, classification and pattern recognition. TOPICS Full papers are invited in (but not limited to) the following areas: - Bond and Stock Valuation and Trading - Univariate time series analysis - Asset allocation and risk management - Multivariate data analysis - Foreign exchange rate prediction - Classification and ranking - Commodity price forecasting - Pattern Recognition - Portfolio management - Hybrid systems Short communications will be accepted if they contain original topical material. SUBMISSION Deadline for submission : 15 September 1993 Notification of acceptance: 15 October 1993 Format: up to a maximum of twenty, single-spaced A4 pages. PROGRAMME COMMITTEE Prof. N. Biggs - London School of Economics Prof. D. Bunn - London Business School Dr J. Moody - Oregon Graduate Institute Dr A. Refenes - London Business School Prof. M. Steiner - Universitaet Munster Dr A. Weigend - University of Colorado ADDRESS FOR PAPERS Dr A. N. Refenes London Business School Department of Decision Science Sussex Place, Regents Park London NW1 4SA, England Tel: ++44 (71) 380 73 29 Fax: ++44 (71) 387 13 97 Email: refenes at cs.ucl.ac.uk ____________________________________________________________________ From greiner at learning.siemens.com Tue Jun 22 08:48:57 1993 From: greiner at learning.siemens.com (Russell Greiner) Date: Tue, 22 Jun 93 08:48:57 EDT Subject: CLNL'93 - Revised Deadlines and General Info Message-ID: <9306221248.AA05363@learning.siemens.com> re: deadlines for Computational Learning and Natural Learning (CLNL'93) Due to popular requests, we have decided to extend the deadline for CLNL'93 submission by one week, until 7/July/93. Below is the revised call for papers, with updated "Important Dates" and "Programme Committee" entries, as well as general registration information. We look forward to receiving your papers, and also hope that you will attend the workshop this September! Russ Greiner (Chair, CLNL'93) PS: People who plan to attend this workshop are still strongly encouraged to register by 30/June. ------------- CLNL'93 -- Call for Submissions Computational Learning and Natural Learning Provincetown, Massachusetts 10-12 September 1993 CLNL'93 is the fourth of an ongoing series of workshops designed to bring together researchers from a diverse set of disciplines --- including computational learning theory, AI/machine learning, connectionist learning, statistics, and control theory --- to explore issues at the intersection of theoretical learning research and natural learning systems. Theme: To be useful, the learning methods used by our fields must be able to handle the complications inherent in real-world tasks. We therefore encourage researchers to submit papers that discuss extensions to learning systems that let them address issues such as: * handling many irrelevant features * dealing with large amounts of noise * inducing very complex concepts * mining enormous sets of data * learning over extended periods of time * exploiting large amounts of background knowledge We welcome theoretical analyses, comparative studies of existing algorithms, psychological models of learning in complex domains, and reports on relevant new techniques. Submissions: Authors should submit three copies of an abstract (100 words or less) and a summary (2000 words or less) of original research to: CLNL'93 Workshop Learning Systems Department Siemens Corporate Research 755 College Road East Princeton, NJ 08540-6632 by 30 June 1993. We will also accept plain-text, stand-alone LaTeX or Postscript submissions sent by electronic mail to clnl93 at learning.scr.siemens.com Each submission will be refereed by the workshop organizers and evaluated based on its relevance to the theme, originality, clarity, and significance. Copies of accepted abstracts will be distributed at the workshop, and MIT Press has agreed to publish an edited volume that incorporates papers from the meeting, subject to revisions and additional reviewing. Invited Talks: Tom Dietterich Oregon State University Ron Rivest Massachusetts Institute of Technology Leo Breiman University of California, Berkeley Yann le Cun Bell Laboratories Important Dates: Deadline for submissions: 7 July 1993 Notification of acceptance: 27 July 1993 CLNL'93 Workshop: 10-12 September 1993 Programme Committee: Andrew Barron, Russell Greiner, Steve Hanson, Robert Holte, Michael Jordan, Stephen Judd, Pat Langley, Thomas Petsche, Tomaso Poggio, Ron Rivest, Eduardo Sontag, Steve Whitehead Workshop Sponsors: Siemens Corporate Research and MIT Laboratory of Computer Science CLNL'93 General Information Dates: The workshop officially begins at 9am Friday 10/Sept, and concludes by 3pm Sunday 12/Sept, in time to catch the 3:30pm Provincetown-Boston ferry. Location: All sessions will take place in the Provincetown Inn (800 942-5388). We encourage registrants to stay there; please sign up in the enclosed registration form. Notice the $74/night does correspond to $37/person per night double-occupancy, if two people share one room. Cost: The cost to attend this workshop is $50/person in general; $25/student. This includes * attendance at all presentation and poster sessions, including the four invited talks; * the banquet dinner on Saturday night; and * a copy of the accepted abstracts. Transportation: Provincetown is located at the very tip of Cape Cod, jutting into the Atlantic Ocean. The drive from Boston to Provincetown requires approximately two hours. There is also a daily ferry (run by Bay State Cruise Lines, 617 723-7800) that leaves Commonwealth Pier in Boston Harbor at 9:30am and arrives in Provincetown at 12:30pm; the return trip departs Provincetown at 3:30pm, arriving at Commonwealth Pier at 6:30pm. Its cost is $15/person, one way. There are also cabs, busses and commuter airplanes (CapeAir, 800 352-0714) that service this Boston-Provincetown route. Reception (Tentative): If there is sufficient interest (as indicated by signing up on the form below), we will hold a reception on a private ferry that leaves Commonwealth Pier for Provincetown at 6:30pm 9/Sept. The additional (Siemens-subsidized) cost for ferry and reception is $40/person, which also includes the return Provincetown-Boston ferry trip on 12/Sept. You must sign up by 30/June; we will announce by 13/July whether this private ferry will be used (and refund the money otherwise). Inquiries: For additional information about CLNL'93, contact clnl93 at learning.scr.siemens.com or the above address. To learn more about Provincetown, contact their Chamber of Commerce at 508 487-3424. CLNL'93 Registration Name: ________________________________________________ Affiliation: ________________________________________________ Address: ________________________________________________ ________________________________________________ Telephone: ____________________ E-mail: ____________________ Select the appropriate options and fees: Workshop registration fee ($50 regular; $25 student) ___________ Ferry transportation + reception ($40) ___________ Hotel room(*) ($74 = 1 night deposit) ___________ Arrival date ___________ Departure date _____________ Name of person sharing room (optional) __________________ # of breakfasts desired ($7.50/bkfst; no deposit req'd) ___ Total amount enclosed: ___________ (*) This is at the Provincetown Inn. For minimum stay of 2 nights. The total cost for three nights is $222 = $74 x 3, plus optional breakfasts. The block of rooms held for CLNL'93 will be released on 30 June 93; room reservations received after this date are accepted subject to availability. See hotel for cancellation policy. If you are not using a credit card, make your check payable in U.S. dollars to "Provincetown Inn/CLNL'93", and mail your completed registration form to Provincetown Inn/CLNL P.O. Box 619 Provincetown, MA 02657. If you are using Visa or MasterCard, please fill out the following, which you may mail to above address, or FAX to 508 487-2911. Signature: ______________________________________________ Visa/MasterCard #: ______________________________________________ Expiration: ______________________________________________ From barnhill at Hudson.Stanford.EDU Tue Jun 22 16:59:31 1993 From: barnhill at Hudson.Stanford.EDU (Joleen Barnhill) Date: Tue, 22 Jun 93 13:59:31 PDT Subject: Neural Nets course Message-ID: The Western Institute in Computer Science announces a one-week course in Neural Networks to be held on the Stanford campus from August 16-20, 1993. Aimed at those technical managers or engineers unfamiliar with Neural Networks who are looking for a comprehensive overview of the field, the course will provide an introduction and an understanding of basic terminology. Included in the introduction will be several hands-on sessions. The course will cover recent developments in the field and their impact from an applications perspective. Students will be given detailed coverage of the application of neural networks to scientific, engineering, and commercial problems. Lastly, future directions of the field will be discussed. Throughout the course, students will have contact with leading researchers, application engineers and managers experienced with the technology. INSTRUCTORS: DR. DAVID BISANT received the Ph.D. from George Washington University and is an Advanced Studies Fellow at STanford University. He holds several adjunct faculty positions and has held positions at International Medical Corporation and with the Department of Defense. His research involves neural network applications to image processing and sequence analysis. DR. DAVID RUMELHART is a Professor at STanford University. He is a Fellow of the NAS, the AAAS and the American Academy of Arts and Sciences. He received a MacArthur Foundation Fellowship for his work on cognitive modeling and he co-founded the Institute of Cognitive Science at UC- San Diego. He received the Ph.D. from STanford. For a complete brochure of all WICS offerings, including this one, send your name and mailing address to barnhill at hudson.Stanford.EDU or call (916) 873-0575 from 8 a.m.-5 p.m. Pacific Time. From jbower at smaug.cns.caltech.edu Thu Jun 24 14:17:20 1993 From: jbower at smaug.cns.caltech.edu (Jim Bower) Date: Thu, 24 Jun 93 11:17:20 PDT Subject: Edward C. Posner Memorial Fellowhip Fund Message-ID: <9306241817.AA24017@smaug.cns.caltech.edu> ********************************************************** Dear Colleague, As you may be aware, last week Ed Posner was killed in an unfortunate bicycle accident in Pasadena. Those of us who knew him well feel the loss very deeply. However, Ed's strong commitment to the communities to which he belonged makes his death even more unfortunate. Throughout his career, Ed was actively involved in academic education and organization. For example, in recent years, he was the first chairman and principle organizer of the Neural Information Processing Systems (NIPS) meeting. One of his many legacies to this meeting is a strong commitment to student travel awards. He also spent much of the last year of his life working hard to establish the NIPS Foundation so that NIPS as well as other related meetings could be on sound financial and legal footing. In addition to his professional activities, Ed was also deeply involved in educational activities at Caltech and JPL. His commitment to students was legend at both institutions. Ed was particularly heavily involved in the SURF program at Caltech through which undergraduates (both from Caltech and from other institutions) carry out independent research projects during the summer months. Ed was an active member of the SURF Administrative Committee. He was also one of the most active SURF research sponsors, having served as mentor to 13 students since 1984. Three more students were preparing to work with him this summer. In addition, Ed co-founded the SURFSAT satellite program in 1986. In this program, successive teams of SURF students are designing, building, and testing a small communications satellite to support the research objectives of NASA's Deep Space Network. Since its inception, 43 students have participated in SURFSAT. Ed's persistent commitment to the scientific education of young people stretches far and touches many. Just a few days prior to his death, for example, Ed had begun to work with the Dean of Graduate Education at Caltech to organize yet another educational program, in this case to increase the number of underrepresented minorities in engineering. Given Ed's strong interest in science education, research, and students, Mrs. Posner has asked that memorial gifts be designated to support Caltech's SURF Program. It is our hope that gifts might ultimately fund an Edward C. Posner SURF Fellowship Fund. Once funded, the Posner SURF Fellowship would annually support an under-represented minority student for a research project in a field related to Ed's own professional interests. Those individuals, or institutions interested in making a contribution to the Edward C. Posner SURF fellowship fund in his memory should contact: Dore Charbonneau Director of Special Gifts Development, mail code 105-40 Caltech Pasadena, CA. 91125 818 - 356-6285 Thank you for your interest and we hope to hear from you. Carolyn Merkel Director, SURF program James M. Bower Computation and Neural Systems Program-Caltech From mackay at mrao.cam.ac.uk Sat Jun 26 12:34:00 1993 From: mackay at mrao.cam.ac.uk (David J.C. MacKay) Date: Sat, 26 Jun 93 12:34 BST Subject: Hyperparameters: optimise, or integrate out? Message-ID: The following preprint is now available by anonymous ftp. ======================================================================== Hyperparameters: optimise, or integrate out? David J.C. MacKay University of Cambridge Cavendish Laboratory Madingley Road Cambridge CB3 0HE mackay at mrao.cam.ac.uk I examine two computational methods for implementation of Bayesian hierarchical models, that is, models which include unknown hyperparameters such as regularisation constants. In the `evidence framework' the model parameters are {\em integrated} over, and the resulting evidence is {\em maximised} over the hyperparameters. In the alternative `MAP' method, the `true posterior probability' is found by {\em integrating} over the hyperparameters, and this is then {\em maximised} over the model parameters. The similarities of the two approaches, and their relative merits, are discussed. In severely ill-posed problems, it is shown that significant biases arise in the second method. ======================================================================== The preprint "Hyperparameters: optimise, or integrate out?" may be obtained as follows: ftp 131.111.48.8 anonymous (your name) cd pub/mackay binary get alpha.ps.Z quit uncompress alpha.ps.Z This document is 16 pages long Table of contents: Outline Making inferences The ideal approach The Evidence framework The MAP method The effective $\a$ of the general MAP method Pros and cons In favour of the MAP method Magnifying the differences An example The curvature of the true prior, and MAP error bars Discussion Appendices: Conditions for the evidence approximation Distance between probability distributions A method for evaluating distances D( p(t) , q(t) ) What I mean by saying that the approximation `works' Predictions The evidence \sigma_N and \sigma_N-1 From mackay at mrao.cam.ac.uk Sat Jun 26 12:36:00 1993 From: mackay at mrao.cam.ac.uk (David J.C. MacKay) Date: Sat, 26 Jun 93 12:36 BST Subject: Energy Prediction Competition Message-ID: The following preprint is now available by anonymous ftp. *********************************************************************** Bayesian Non-linear Modeling for the Energy Prediction Competition David J.C. MacKay University of Cambridge Cavendish Laboratory Madingley Road Cambridge CB3 0HE mackay at mrao.cam.ac.uk Bayesian probability theory provides a unifying framework for data modeling. A model space may include numerous control parameters which influence the complexity of the model (for example regularisation constants). Bayesian methods can automatically set such parameters so that the model becomes probabilistically well-matched to the data. The 1993 energy prediction competition involved the prediction of a series of building energy loads from a series of environmental input variables. Non-linear regression using `neural networks' is a popular technique for such modeling tasks. Since it is not obvious how large a time-window of inputs is appropriate, or what preprocessing of inputs is best, this can be viewed as a regression problem in which there are many possible input variables, some of which may actually be irrelevant to the prediction of the output variable. Because a finite data set will show random correlations between the irrelevant inputs and the output, any conventional neural network (even with `weight decay') will not set the coefficients for these junk inputs to zero. Thus the irrelevant variables will hurt the model's performance. The Automatic Relevance Determination (ARD) model puts a prior over the regression parameters which embodies the concept of relevance. This is done in a simple and `soft' way by introducing multiple `weight decay' constants, one `$\alpha$' associated with each input. Using Bayesian methods, the decay rates for junk inputs are automatically inferred to be large, preventing those inputs from causing significant overfitting. An entry using the ARD model won the prediction competition by a significant margin. *********************************************************************** The preprint "Bayesian Non-linear Modeling for the Energy Prediction Competition" may be obtained as follows: ftp 131.111.48.8 anonymous (your name) cd pub/mackay binary mget pred.*.ps.Z quit uncompress pred.*.ps.Z This preprint is 24 pages long and contains a large number of figures. A more concise version may be released later. Table of contents: Overview of Bayesian modeling methods Neural networks for regression Neural network learning as inference Setting regularisation constants $\a$ and $\b$ Automatic Relevance Determination Prediction competition: part A The task Preliminaries Round 1 Round 2 Creating a committee Results Additional performance criteria How much did ARD help? How much did the use of a committee help? Prediction competition: part B The task Preprocessing Time--delayed and time--filtered inputs Results Summary and Discussion What I might have done differently How to better model this sort of problem. Appendix Training data: problem A Omitted data Coding of holidays Pre-- and post-processing From joe at cogsci.edinburgh.ac.uk Tue Jun 29 06:46:17 1993 From: joe at cogsci.edinburgh.ac.uk (Joe Levy) Date: Tue, 29 Jun 93 11:46:17 +0100 Subject: 2nd Neural Computation and Psychology Workshop Message-ID: <9150.9306291046@grogan.cogsci.ed.ac.uk> 2nd Neural Computation and Psychology Workshop Connectionist Models of Memory and Language University of Edinburgh, Scotland 10th - 13th September 1993 AIMS AND OBJECTIVES This workshop is the second in a series, following on from last year's very successful Neurodynamics and Psychology Workshop at the University of Wales, Bangor. This year it is to be hosted by the Connectionism and Cognition Research Group at the University of Edinburgh. The general aim is to bring together researchers from such diverse disciplines as neurobiology, psychology, cognitive science, artificial intelligence, applied mathematics and computer science to discuss their work on the connectionist modelling of memory and language. Likely special sessions include memory, speech processes and models of reading. The workshop will have an invited list of speakers and a limited number of participants to allow single-track papers and ease of discussion. It will run from the evening of Friday 10th through to lunch time on Monday 13th. PROVISIONAL SESSION CHAIRS AND INVITED SPEAKERS INCLUDE: Bob Damper (Southampton) Trevor Harley (Warwick) Jacob Murre (APU, Cambridge) Noel Sharkey (Exeter) Leslie Smith (Stirling) Keith Stenning (Edinburgh) John Taylor (KC, London) David Willshaw (Edinburgh) POSTER SESSION So that other participants have a chance of presenting their work on connectionist models of memory and language we are including refereed poster sessions. If interested, please send a one page abstract with the registration form. REGISTRATION, FOOD AND ACCOMMODATION The workshop will be held in Pollock Halls, an attractive student residence close to the centre of Edinburgh with views of Arthur's Seat and ample parking. The conference registration fee (which includes morning and afternoon teas and coffees, the three lunches, a reception and meal on the Friday evening and dinner on Saturday) is 80 pounds (or 65 pounds for full time students). A special Conference Dinner will be arranged for the Sunday evening costing 15 pounds. Bed and breakfast accommodation on site is 70 pounds (total for the three nights). ORGANISING COMMITTEE Joe Levy (HCRC, Edinburgh) Dimitris Bairaktaris (HCRC, Edinburgh) John Bullinaria (Psychology, Edinburgh) Paul Cairns (Cognitive Science, Edinburgh) FURTHER DETAILS AND REGISTRATION Contact: Dr. Joe Levy, NCPW 93, University of Edinburgh, Human Communication Research Centre, 2 Buccleuch Place, Edinburgh, EH8 9LW, Scotland. Email: joe at uk.ac.ed.cogsci Phone: +44 31 650 4450. Fax: +44 31 650 4587. REGISTRATION FORM: 2nd Neural Computation and Psychology Workshop Connectionist Models of Memory and Language University of Edinburgh, Scotland 10th - 13th September 1993 Completed registration forms, full payment and any abstracts must be received by 12th July. Full Name: ___________________________________________________________ Institution: _________________________________________________________ Mailing Address: _____________________________________________________ ______________________________________________________________________ ______________________________________________________________________ ______________________________________________________________________ Telephone: _________________________________________ Fax: _______________________________________________ E-mail: ____________________________________________ Poster abstract enclosed: Yes / No Money enclosed: Registration Fee: #80 ___________ (includes 3 lunches and 2 dinners) (#65 for full time students - proof required) Conference Dinner on Sunday evening: #15 ___________ (Special dietary requirements: Vegetarian Vegan Other........ ) Accommodation: #70 ___________ (total for 3 nights B & B) (indicate if adjacent rooms required) TOTAL =========== Please make cheques payable to 'University of Edinburgh'. Mail to: Dr. Joe Levy (NCPW 93) University of Edinburgh Human Communication Research Centre 2 Buccleuch Place Edinburgh EH8 9LW Scotland UK From greiner at learning.siemens.com Tue Jun 29 23:25:52 1993 From: greiner at learning.siemens.com (Russell Greiner) Date: Tue, 29 Jun 93 23:25:52 EDT Subject: CLNL'93 - Revised deadline Message-ID: <9306300325.AA10121@learning.siemens.com> re: deadlines for Computational Learning and Natural Learning (CLNL'93) Due to popular requests, we have decided to extend the deadline for CLNL'93 submission by one week, until 7/July/93. Below is the revised call for papers, with updated "Important Dates" and "Programme Committee" entries, as well as general registration information. We look forward to receiving your papers, and also hope that you will attend the workshop this September! Russ Greiner (Chair, CLNL'93) ------------- CLNL'93 -- Call for Submissions Computational Learning and Natural Learning Provincetown, Massachusetts 10-12 September 1993 CLNL'93 is the fourth of an ongoing series of workshops designed to bring together researchers from a diverse set of disciplines --- including computational learning theory, AI/machine learning, connectionist learning, statistics, and control theory --- to explore issues at the intersection of theoretical learning research and natural learning systems. Theme: To be useful, the learning methods used by our fields must be able to handle the complications inherent in real-world tasks. We therefore encourage researchers to submit papers that discuss extensions to learning systems that let them address issues such as: * handling many irrelevant features * dealing with large amounts of noise * inducing very complex concepts * mining enormous sets of data * learning over extended periods of time * exploiting large amounts of background knowledge We welcome theoretical analyses, comparative studies of existing algorithms, psychological models of learning in complex domains, and reports on relevant new techniques. Submissions: Authors should submit three copies of an abstract (100 words or less) and a summary (2000 words or less) of original research to: CLNL'93 Workshop Learning Systems Department Siemens Corporate Research 755 College Road East Princeton, NJ 08540-6632 by 30 June 1993. We will also accept plain-text, stand-alone LaTeX or Postscript submissions sent by electronic mail to clnl93 at learning.scr.siemens.com Each submission will be refereed by the workshop organizers and evaluated based on its relevance to the theme, originality, clarity, and significance. Copies of accepted abstracts will be distributed at the workshop, and MIT Press has agreed to publish an edited volume that incorporates papers from the meeting, subject to revisions and additional reviewing. Invited Talks: Tom Dietterich Oregon State University Ron Rivest Massachusetts Institute of Technology Leo Breiman University of California, Berkeley Yann le Cun Bell Laboratories Important Dates: Deadline for submissions: 7 July 1993 Notification of acceptance: 27 July 1993 CLNL'93 Workshop: 10-12 September 1993 Programme Committee: Andrew Barron, Russell Greiner, Steve Hanson, Robert Holte, Michael Jordan, Stephen Judd, Pat Langley, Thomas Petsche, Tomaso Poggio, Ron Rivest, Eduardo Sontag, Steve Whitehead Workshop Sponsors: Siemens Corporate Research and MIT Laboratory of Computer Science CLNL'93 General Information Dates: The workshop officially begins at 9am Friday 10/Sept, and concludes by 3pm Sunday 12/Sept, in time to catch the 3:30pm Provincetown-Boston ferry. Location: All sessions will take place in the Provincetown Inn (800 942-5388). We encourage registrants to stay there; please sign up in the enclosed registration form. Notice the $74/night does correspond to $37/person per night double-occupancy, if two people share one room. Cost: The cost to attend this workshop is $50/person in general; $25/student. This includes * attendance at all presentation and poster sessions, including the four invited talks; * the banquet dinner on Saturday night; and * a copy of the accepted abstracts. Transportation: Provincetown is located at the very tip of Cape Cod, jutting into the Atlantic Ocean. The drive from Boston to Provincetown requires approximately two hours. There is also a daily ferry (run by Bay State Cruise Lines, 617 723-7800) that leaves Commonwealth Pier in Boston Harbor at 9:30am and arrives in Provincetown at 12:30pm; the return trip departs Provincetown at 3:30pm, arriving at Commonwealth Pier at 6:30pm. Its cost is $15/person, one way. There are also cabs, busses and commuter airplanes (CapeAir, 800 352-0714) that service this Boston-Provincetown route. Reception (Tentative): If there is sufficient interest (as indicated by signing up on the form below), we will hold a reception on a private ferry that leaves Commonwealth Pier for Provincetown at 6:30pm 9/Sept. The additional (Siemens-subsidized) cost for ferry and reception is $40/person, which also includes the return Provincetown-Boston ferry trip on 12/Sept. You must sign up by 30/June; we will announce by 13/July whether this private ferry will be used (and refund the money otherwise). Inquiries: For additional information about CLNL'93, contact clnl93 at learning.scr.siemens.com or the above address. To learn more about Provincetown, contact their Chamber of Commerce at 508 487-3424. CLNL'93 Registration Name: ________________________________________________ Affiliation: ________________________________________________ Address: ________________________________________________ ________________________________________________ Telephone: ____________________ E-mail: ____________________ Select the appropriate options and fees: Workshop registration fee ($50 regular; $25 student) ___________ Ferry transportation + reception ($40) ___________ Hotel room(*) ($74 = 1 night deposit) ___________ Arrival date ___________ Departure date _____________ Name of person sharing room (optional) __________________ # of breakfasts desired ($7.50/bkfst; no deposit req'd) ___ Total amount enclosed: ___________ (*) This is at the Provincetown Inn. For minimum stay of 2 nights. The total cost for three nights is $222 = $74 x 3, plus optional breakfasts. The block of rooms held for CLNL'93 will be released on 30 June 93; room reservations received after this date are accepted subject to availability. See hotel for cancellation policy. If you are not using a credit card, make your check payable in U.S. dollars to "Provincetown Inn/CLNL'93", and mail your completed registration form to Provincetown Inn/CLNL P.O. Box 619 Provincetown, MA 02657. If you are using Visa or MasterCard, please fill out the following, which you may mail to above address, or FAX to 508 487-2911. Signature: ______________________________________________ Visa/MasterCard #: ______________________________________________ Expiration: ______________________________________________ From cic!john!ostrem at unix.sri.com Tue Jun 22 16:48:32 1993 From: cic!john!ostrem at unix.sri.com (John Ostrem) Date: Tue, 22 Jun 93 13:48:32 PDT Subject: job openings Message-ID: <9306222048.AA17348@john.noname> Communication Intelligence Corporation (CIC) is a leader in handwriting recognition and other pen input technologies. We currently market recognizers for English, Western European, and Asian languages on a variety of platforms (e.g., DOS, Windows, Macintosh, and so on). These systems enable the pen to serve as the sole input and control device, combining the functions of both keyboard and mouse, and adding new capabilities. Advanced development is directed toward integrated discrete/cursive recognizers, and future integration with voice recognition, OCR, and similar technologies. CIC was founded in 1981 in conjunction with SRI International (formerly Stanford Research Institute). CIC is headquartered in Redwood Shores, California, and has an international subsidiary, CIC Japan, Inc., in Tokyo, Japan. CIC currently has immediate openings for the following positions: ----------------------------------------------------------------------------- POSITION: Software Engineer QUALIFICATIONS: 1. 3-5 years experience in designing and coding for large software projects in a UNIX environment 2. Good communication skills and works well with other people. 3. Expert C programmer (at least 3-5 years experience) 4. BS or MS in Computer Science or the equivalent 5. Experience in graphics programming and user interfaces a plus 6. The following are additional pluses: a. Experience in handwriting recognition (on-line or off-line) b. Linguistic experience (particularly statistical linguistics) c. Experience planning/executing complex projects d. Experience in commercial companies e. Experience in SunOS system administration JOB DESCRIPTION: 1. Work with and support researchers working on handwriting and speech recognition 2. Design, implement, and support data collection software and analysis tools ----------------------------------------------------------------------------- POSITION: Pattern Recognition Specialist/project leader QUALIFICATIONS: 1. Strong background in statistics, pattern recognition, algorithm development 2. Experience in OCR a plus 3. 3-5 years experience in designing and coding for large software projects in a UNIX environment 4. Good communication skills and works well with other people. 5. Expert C programmer (at least 3-5 years experience) 6. Ph.D. or substantial experience in Computer Science, Electrical Engineering or the equivalent 7. The following are additional pluses: a. Experience in handwriting recognition (on-line or off-line) b. Linguistic experience (particularly statistical linguistics) c. Experience planning/executing complex projects d. Experience in commercial companies JOB DESCRIPTION 1. Work with a team of researchers on the next generation of handwriting recognition systems (both off-line and on-line) for the commercial market 2. Develop into a project leader/manager ----------------------------------------------------------------------------- Please reply to cic!ostrem at unix.sri.com (or cic\!ostrem at unix.sri.com in command mode), or write or fax to John S. Ostrem Communication Intelligence Corporation 275 Shoreline Drive, 6th Floor Redwood Shores, CA 94065-1413 Fax: (415) 802-7777