From COSCAVR at rivendell.otago.ac.nz  Tue Sep  1 12:15:44 1998
From: COSCAVR at rivendell.otago.ac.nz (ANTHONY ROBINS.)
Date: Tue, 01 Sep 1998 16:03:44 -0012
Subject: What have neural networks achieved?
Message-ID: <01J1APXR7F4I94E2TK@rivendell.otago.ac.nz>

> From: "Randall C. O'Reilly" <oreilly at grey.colorado.edu>
>
> Another angle on the hippocampal story has to do with the phenomenon
> of catestrophic interference (McCloskey & Cohen, 1989), and the notion
> that the hippocampus and the cortex are complementary learning systems


The catastrophic forgetting (catastrophic interference, serial
learning) problem has come up in this thread.  In most neural networks
most of the time, learning new information disrupts (even eliminates)
old information.  I want to quickly describe what we think is an
interesting and general solution to this problem.

First, a comment on rehearsal.  The catastrophic forgetting 
problem can be solved with rehearsal - relearning old items as
new items are learned.  A range of rehearsal regimes have been
explored (see for example Murre, 1992; Robins, 1995; and the
"interleaved learning" referred to earlier in this thread by Jay
McClelland from McClelland, McNaughton, & O'Reilly, 1995).

Rehearsal is an effective solution as long as the previously learned
items are actually available for relearning.  It may be, however, that
the old items have been lost, or it is not practical for some reason
to store them.  In any case, retaining old items for rehearsal in a
network seems somewhat artificial, as it requires that they be
available on demand from some other source, which would seem to make
the network itself redundant.

It is possible to achieve the benefits of rehearsal, however, even
when there is no access to old items.  This "pseudorehearsal"
mechanism, introduced in Robins (1995), is based on the relearning of
artificially constructed populations of "pseudoitems" instead of the
actual old items.

In MLP / backprop type networks a pseudoitem is constructed by
generating a new input vector at random, and passing it forward
through a network in the standard way.  Whatever output vector this
input generates becomes the associated target output.  Rehearsing
these pseudoitems during new learning protects the old items in the
same way that rehearsing the real old items does.  Why does it work?
The essence of preventing catastrophic forgetting is to localise
changes to the function instantiated by the network so that it changes
only in the immediate vicinity of the new item to be learned.
Rehearsal localises changes by relearning ("fixing") the original
training data points.  Pseudorehearsal localises change by relearning
("fixing") other points randomly chosen from the function (the
pseudoitems).  (Work in progress suggests that simply using a "local"
learning algorithm such as an RBF is not enough).

Pseudorehearsal is the generation of approximations of old knowledge
to be rehearsed as needed.  The method is very effective, and has been 
further explored in a number of papers (Robins, 1996; Frean & Robins, 
1998; Ans & Rousset, 1997; French, 1997; and as a part of work 
described in Silver & Mercer, 1998).  Pseudorehearsal enables sequential 
learning (the learning of new information at any time) in a neural 
network.

Extending these ideas to dynamical networks (such as Hopfield nets),
we can rehearse randomly chosen attractors to preserve previously
learned items / attractors during new learning (Robins & McCallum,
1998). Here the distinction between rehearsal and pseudorehearsal
starts to break down, as randomly chosen attractors naturally contain
a mixture of both real old items / learned attractors and pseudoitems
/ spurious attractors.

We have already linked pseudorehearsal in MLP networks to the
consolidation of information during sleep (Robins, 1996).  In the
context of Hopfield type nets another proposed solution to
catastrophic forgetting based on unlearning spurious attractors has
also been linked to sleep (eg Hopfield, Feinstein & Palmer, 1983;
Crick & Mitchison, 1983; Christos, 1996).  We are currently exploring
the relationship between this *unlearning* and our *relearning* based
accounts.  Details of the input patterns, architecture, and learning
algorithm are all significant in determining the efficacy of the two
approaches (we think our approach has advantages, but this is work in
progress!).


References

Ans,B. & Rousset,S. (1997) Avoiding Catastrophic Forgetting by Coupling
Two Reverberating Neural Networks.  Academie des Sciences, Sciences de la
vie, 320, 989 - 997.

Christos, G. (1996) Investigation of the Crick-Mitchison Reverse-Learning
Dream Sleep Hypothesis in a Dynamic Setting. Neural Networks, 9, 427 -
434.

Crick,F. & Mitchison,G (1983) The Function of Dream Sleep.  Nature, 304,
111 -114.

Frean,M.R. & Robins,A.V. (1998). Catastrophic forgetting and
"pseudorehearsal" in linear networks. In Downs T, Frean M & Gallagher M
(Eds) Proceedings of the Ninth Australian Conference on Neural Networks
Brisbane: University of Queensland (1998) 173 - 178.

French,R.M. (1997) Pseudo-recurrent Connectionist Networks:  An Approach
to the Sensitivity Stability Dilemma.  Connection Science, 9, 353 - 380.

Hopfield,J., Feinstein, D. & Palmer,R. (1983) 'Unlearning' has a
Stabilizing Effect in Collective Memories.  Nature, 304. 158 - 159.

McClelland,J., McNaughton,B. & O'Reilly,R. (1995) Why there are 
complementary learning systems in the hippocampus and neocortex: 
Insights from the successes and failures of connectionist models of 
learning and memory.  Psychological Review, 102, 419-457.

Murre,J.M.J. (1992) Learning and Categorization in Modular Neural
Networks.  Hillsdale, NJ:  Earlbaum.

Robins,A. (1995) Catastrophic Forgetting, Rehearsal, and Pseudorehearsal.
Connection Science, 7, 123 - 146.

Robins,A. (1996)  Consolidation in Neural Networks and in the Sleeping
Brain.  Connection Science, 8, 259 - 275.

Robins, A. & McCallum, S. (1998).  Pseudorehearsal and the Catastrophic
Forgetting Solution in Hopfield Type Networks.  Connection Science, 7 :
121 - 135.

Silver,D. & Mercer,R. (1998) The Task Rehearsal Method of Sequential
Learning.  Department of Computer Science University of Western Ontario
Technical Report # 517.



Anthony Robins  ----------------------------------------------------
Computer Science                                 coscavr at otago.ac.nz
University of Otago                             ph:    +64 3 4798314
Dunedin, NEW ZEALAND                            fax:   +64 3 4798529


From wahba at stat.wisc.edu  Tue Sep  1 15:00:30 1998
From: wahba at stat.wisc.edu (Grace Wahba)
Date: Tue, 1 Sep 1998 14:00:30 -0500 (CDT)
Subject: Gaussian statistical models, Hilbert spaces
Message-ID: <199809011900.OAA14197@hera.stat.wisc.edu>


Readers of 
...............
  http://www.santafe.edu/~zhuh/draft/edmc.ps.gz 

		Error Decomposition and Model Complexity

			     Huaiyu Zhu

  Bayesian information geometry provides a general error decomposition
  theorem for arbitrary statistical models and a family of information
  deviations that include Kullback-Leibler information as a special case.
  When applied to Gaussian measures it takes the classical Hilbert space
  (Sobolev space) theories for estimation (regression, filtering,
  approximation, smoothing) as a special case.  When the statistical and
  computational models are properly distinguished, the dilemmas of
  over-fitting and ``curse of dimensionality'' disappears, and the optimal
  model order disregarding computing cost is always infinity.
.............

  will do doubt be interested in the long history of the relationship 
  between reproducing kernel Hilbert spaces (rkhs), gaussian measures  
  and regularization,  -
    see 
    
    1962 Proccedings of the Symposium on Time Series Analysis 
    edited by Murray Rosenblatt, Wiley 1962, esp. the paper by Parzen
    1962 J. Hajek On linear statistical problems in stochastic processes
	 Czech Math J. v 87. 
    1971 Kimeldorf and Wahba, Some results on Tchebycheffian spline functions, 
    J. Math Anal. Applic. v 33.
    1990 G. Wahba, Spline Models for Observational Data, SIAM 
    1997 F. Girosi, An equivalence between sparse approximation and 
      support vector machines, to appear Neural Comp
    1997 G. Wahba, Support vector vachines, reproducing kernel Hilbert 
      spaces and the randomized GACV, to appear, Schoelkopf, Burges 
      and Smola, eds, forthcoming book on Support Vector Machines, MIT Press 
    1981 C. Micchelli and G. Wahba, Design problems for optimal
      surface interpolation, Approximation Theory and Applications, 
      Z. Ziegler ed, Academic press. Also numerous works by 
      L. Plaskota and others on optimal bases. First k eigenfunctions 
      of the reproducing kernel are well known to have certain 
      optimal properties under restricted circumstances, 
      see e.g. Ch 12 of Spline Models and references there, but 
      if there are n observations, then the Bayes estimates 
      are found in an at most  n-dimensional subspace of the rkhs 
      associated with the prior, KW 1971.   B. Silverman 1982
      `On the estimation of a probability density fuction by 
      the maximum penalized likelihood method', Ann. Statist 1982 
      will also be of interest - convergence rates are related
      to the rate of decay of the eigenvalues of the reproducing
      kernel. 
	     Grace Wahba http://www.stat.wisc.edu/~wahba/  


From ericr at ee.usyd.edu.au  Tue Sep  1 21:11:01 1998
From: ericr at ee.usyd.edu.au (Eric Ronco)
Date: Wed, 2 Sep 1998 11:11:01 +1000
Subject: Gated Modular Neural Networks for Control Oriented Modelling
Message-ID: <199809020111.LAA00815@merlot.ee.usyd.edu.au.usyd.edu.au>

Dear Connectionists,

A new technical report can be found in the on-line data base
of the Systems and Control Laboratory of Sydney University.
 
Its title is: Gated Modular Neural Networks for Control Oriented
Modelling.

The (gziped) PostScript file is accessible at:

       http://www.ee.usyd.edu.au/~ericr/pub

or alternatively at

       http://merlot.ee.usyd.edu.au/tech_rep/


The authors are: Eric Ronco, Peter J. Gawthrop and David J. Hill.

The keywords are: Non-linear modelling and control; Neural Network;
Modularity; 

The abstract:

This study is an attempt to review the main ``Gated Modular Neural
Networks'' (GMNNs) which are particularly suitable for modelling
oriented toward control. A GMNN consists of a network of computing
modules and a gating system. The computing modules are the operating
part of the architecture and corresponds to linear or simple
non-linear models or controllers. The simplicity of the computing
modules is the fundamental advantage of this approach as this often
clarifies and simplifies the modelling and control design. The gating
system is used for the selection of the computing module(s) valid for
the current state of the plant. Three types of GMNN are distinguished
according to the gating strategy they implement which are respectively
based on spatial clustering, modelling performance and probable
performance of the computing modules. The modelling oriented control
properties of these approaches are assessed in terms of adaptability,
performance, control design, implementation and analysis
properties. Conclusion are drawn to highlight required future works to
generalise the applicability of these approaches.

 
The number of this technical report is: EE-98009

Bye,

Eric
-------------------------------------------------------------------
Eric Ronco, PhD                    Tel: +61 2 9351 7680
Dt of Electrical Engineering       Fax: +61 2 9351 5132
Bldg J13, Sydney University        Email: ericr\@ee.usyd.edu.au
NSW 2006, Australia                http://www.ee.usyd.edu.au/~ericr
-------------------------------------------------------------------

From szepes at iserv.iki.kfki.hu  Wed Sep  2 04:57:40 1998
From: szepes at iserv.iki.kfki.hu (Csaba Szepesvari)
Date: Wed, 2 Sep 1998 10:57:40 +0200 (MET)
Subject: new publications
Message-ID: <Pine.A32.3.91.980902105239.22745A-100000@iserv.iki.kfki.hu>

Dear Connectionists,

I have updated my homepage which now contains a link to my
online publications. Below you can find a list of the publications
(title, abstract). They can be accessed from the page
http://sneaker.mindmaker.kfkipark.hu/~szepes/research/OnlinePubs.htm
in the form of gzipped postscript files.

Yours Sincerely,

  Csaba Szepesvari

====================================Reinforcement Learning & Markovian Decision Problems
-------------------------------------------------------------------------

Module-Based Reinforcement Learning: Experiments with a Real Robot

Zs. Kalmár, Cs. Szepesvári and A. Lorincz
Machine Learning 31:55-85, 1998. ps

Autonomous Robots 5:273-295, 1998. (this was a joint Special Issue of the
two journals..)

The behavior of reinforcement learning (RL) algorithms is best understood
in completely observable, discrete-time controlled Markov chains with
finite state and action spaces. In contrast, robot-learning domains are
inherently continuous both in time and space, and moreover are partially
observable. Here we suggest a systematic approach to solve such problems
in which the available qualitative and quantitative knowledge is used to
reduce the complexity of learning task. The steps of the design process
are to: i) decompose the task into subtasks using the qualitative
knowledge at hand; ii) design local controllers to solve the subtasks
using the available quantitative knowledge and iii) learn a coordination
of these controllers by means of reinforcement learning. It is argued
that the approach enables fast, semi-automatic, but still high quality
robot-control as no fine-tuning of the local controllers is needed. The
approach was verified on a non-trivial real-life robot task. Several RL
algorithms were compared by ANOVA and it was found that the model-based
approach worked significantly better than the model-free approach. The
learnt switching strategy performed comparably to a handcrafted version.
Moreover, the learnt strategy seemed to exploit certain properties of the
environment which were not foreseen in advance, thus supporting the view
that adaptive algorithms are advantageous to non-adaptive ones in complex
environments.

Non-Markovian Policies in Sequential Decision Problems

Cs. Szepesvári
Acta Cybernetica (to appear) 1998. ps

In this article we prove the validity of the Bellman Optimality Equation
and related results for sequential decision problems with a general
recursive structure. The characteristic feature of our approach is that
also non-Markovian policies are taken into account. The theory is
motivated by some experiments with a learning robot.

Convergence Results for Single-Step On-Policy Reinforcement-Learning
Algorithms

S. Singh, T. Jaakkola, M.L. Littman and Cs. Szepesvári
Machine Learning, to appear, 1998. ps

An important application of reinforcement learning (RL) is to
finite-state control problems and one of the most difficult problems in
learning for control is balancing the exploration/exploitation tradeoff.
Existing theoretical results for RL give very little guidance on
reasonable ways to perform exploration. In this paper, we examine the
convergence of single-step on-policy RL algorithms for control. On-policy
algorithms cannot separate exploration from learning and therefore must
confront the exploration problem directly. We prove convergence results
for several related on-policy algorithms with both decaying exploration
and persistent exploration. We also provide examples of exploration
strategies that can be followed during learning that result in
convergence to both optimal values and optimal policies.

Multi-criteria Reinforcement Learning

Z. Gábor, Zs. Kalmár and Cs. Szepesvári
In Proceedings of International Conference of Machine Learning, in press,
1998. ps

We consider multi-criteria sequential decision making problems where the
vector-valued evaluations are compared by a given, fixed total ordering.
Conditions for the optimality of stationary policies and the Bellman
optimality equation are given for a special, but important class of
problems when the evaluation of policies can be computed for the criteria
independently of each other. The analysis requires special care as the
topology introduced by pointwise convergence and the order-topology
introduced by the preference order are in general incompatible.
Reinforcement learning algorithms are proposed and analyzed. Preliminary
computer experiments confirm the validity of the derived algorithms.
These type of multi-criteria problems are most useful when there are
several optimal solutions to a problem and one wants to choose the one
among these which is optimal according to another fixed criterion.
Possible application in robotics and repeated games are outlined.

Multi-criteria Reinforcement Learning

Z. Gábor, Zs. Kalmár and Cs. Szepesvári
Technical Report TR-98-115, "Attila József" University, Research Group on
Artificial Intelligence Szeged, HU-6700, 1998 (submitted in 1997). ps

We consider multi-criteria sequential decision making problems where the
vector-valued evaluations are compared by a given, fixed total ordering.
Conditions for the optimality of stationary policies and the Bellman
optimality equation are given for a special, but important class of
problems when the evaluation of policies can be computed for the criteria
independently of each other. The analysis requires special care as the
topology introduced by pointwise convergence and the order-topology
introduced by the preference order are in general incompatible.
Reinforcement learning algorithms are proposed and analyzed. Preliminary
computer experiments confirm the validity of the derived algorithms.
These type of multi-criteria problems are most useful when there are
several optimal solutions to a problem and one wants to choose the one
among these which is optimal according to another fixed criterion.
Possible application in robotics and repeated games are outlined.

The Asymptotic Convergence-Rate of Q-learning

Cs. Szepesvári
In Proceedings of Neural Information Processing Systems 10, pp.
1064-1070, 1997. ps

In this paper we show that for discounted MDPs with discount factor
\gamma>1/2 the asymptotic rate of convergence of Q-learning is
O(1/t^{R(1-\gamma)}) if R(1-\gamma)<1/2 and O(\sqrt{\log\log t/ t})
otherwise provided that the state-action pairs are sampled from a fixed
probability distribution. Here R=\pmin/\pmax is the ratio of the minimum
and maximum state-action occupation frequencies. The results extend to
convergent on-line learning provided that \pmin>0, where \pmin and \pmax
now become the minimum and maximum state-action occupation frequencies
corresponding to the stationary distribution.

Learning and Exploitation do not Conflict Under Minimax Optimality

Cs. Szepesvári
In Proceedings of 9th European Conference of Machine Learning, pp.
242-249, 1997. ps

We show that adaptive real time dynamic programming extended with the
action selection strategy which chooses the best action according to the
latest estimate of the cost function yields asymptotically optimal
policies within finite time under the minimax optimality criterion. From
this it follows that learning and exploitation do not conflict under this
special optimality criterion. We relate this result to learning optimal
strategies in repeated two-player zero-sum deterministic games.

A unified analysis of value-function-based reinforcement-learning algorithms

Cs. Szepesvári and M. L. Littman

Submitted for review, 1997 ps

Reinforcement learning is the problem of generating optimal behavior in a
sequential decision-making environment given the opportunity of
interacting with it. Many algorithms for solving reinforcement-learning
problems work by computing improved estimates of the optimal value
function. We extend prior analyses of reinforcement-learning algorithms
and present a powerful new theorem that can provide a unified analysis of
value-function-based reinforcement-learning algorithms. The usefulness of
the theorem lies in how it allows the asynchronous convergence of a
complex reinforcement-learning algorithm to be proven by verifying that a
simpler synchronous algorithm converges. We illustrate the application of
the theorem by analyzing the convergence of Q-learning, model-based
reinforcement learning, Q-learning with multi-state updates, Q-learning
for Markov games, and risk-sensitive reinforcement learning.

Some basic facts concerning minimax sequential decision processes

Cs. Szepesvári
Technical Report TR-96-100, "Attila József" University, Research Group on
Artificial Intelligence Szeged, HU-6700, 1996. ps

It is shown that for discounted minimax sequential decision processes the
evaluation function of a stationary policy is the fixed point of the
so-called policy evaluation operator which is a contraction mapping.
Using this we prove that Bellman's principle of optimality holds. We also
prove that the asynchronous value iteration algorithm converges to the
optimal value function.

Adaptive (Neuro-)Control
---------------------------------------------------------------------
Uncertainty and Performance of Adaptive Controllers for Functionally
Uncertain Output Feedback Systems

M. French, Cs. Szepesvári and E. Rogers
In Proc. of 1998 IEEE Conference on Decision and Decision, 1998. ps

We consider nonlinear systems in an output feedback form which are
functionally known up to a L2 measure of uncertainty. The control task is
to drive the output of the system to some neighbourhood of the origin. A
modified L2 measure of transient performance (penalising both state and
control effort) is given, and the performance of a class of model based
adaptive controllers is studied. An upper performance bound is derived.

Uncertainty, Performance and Model Dependency in Approximate Adaptive
Nonlinear Control

M. French, Cs. Szepesvári and E. Rogers
In Proc. of 1997 IEEE Conference on Decision and Decision, San Diego,
California, 1997. ps

We consider systems satisfying a matching condition which are
functionally known up to a L2 measure of uncertainty. A modified L2
performance measure is given, and the performance of a class of model
based adaptive controllers is studied. An upper performance bound is
derived in terms of the uncertainty measure and measures of the
approximation error of the model. Asymptotic analyses of the bounds under
increasing model size are undertaken, and sufficient conditions are given
on the model that ensure the performance bounds are bounded independent
of the model size.

Uncertainty, Performance and Model Dependency in Approximate Adaptive
Nonlinear Control

M. French, Cs. Szepesvári and E. Rogers
Submitted for journal publication 1997. ps

We consider systems satisfying a matching condition which are
functionally known up to weighted L2 and L-infinity measures of
uncertainty. A modified LQ measure of control and state transient
performance is given, and the performance of a class of approximate model
based adaptive controllers is studied. An upper performance bound is
derived in terms of the uncertainty models (stability and the state
transient bounds require only the L2 uncertainty model; control effort
bounds require both L2 and L-infinity uncertainty models), and various
structural properties of the model basis. Sufficient conditions are
sgiven to ensure that the performance is bounded independent of the model
basis size.



From brander at csee.uq.edu.au  Wed Sep  2 14:28:01 1998
From: brander at csee.uq.edu.au (Rafael Brander)
Date: Thu, 3 Sep 1998 04:28:01 +1000 (EST)
Subject: What have neural networks achieved?
In-Reply-To: <01J1APXR7F4I94E2TK@rivendell.otago.ac.nz>
Message-ID: <Pine.GSO.3.95q.980903042457.15706A-100000@fife>


These look like achievements of neural networks (see detail below):

(1) Suggested by Randall O'Reilly,  "...the neural network
approach provides a principled basis for understanding why we have a
hippocampus, and what its functional characteristics should be."

The catastrophic interference literature has also given a plausible
explanation -- sparseness -- for why, given that our brains look very
much like neural networks, we have any memory at all.

(2) I think that the catastrophic interference and generalisation
literature suggests the possibility that in order to have generalisation
capability, the human weakness with discrete symbolic memory may be an
inevitability -- in artificial as well as biological computers. This
bolsters the common view that AI will only be achieved with
human-comparable computer hardware. It also explains the typical human
complaint "why do I have such a terrible memory..." (relative to
silicon chip computers).

Apologies for length of this email, see my Masters abstract at bottom.
Rafael Brander.

Randall O'Reilly wrote:

"Another angle on the hippocampal story has to do with the phenomenon
of catestrophic interference (McCloskey & Cohen, 1989), and the notion
that the hippocampus and the cortex are complementary learning systems
that each optimize different functional objectives (McClelland,
McNaughton, & O'Reilly, 1995).  In this case, the neural network
approach provides a principled basis for understanding why we have a
hippocampus, and what its functional characteristics should be.
Interestingly, one of the "sucesses" of neural networks in this case
was their dramatic failure in the form of the catestrophic
interference phenomenon.  This failure tells us something about the
limitations of the cortical memory system, and thus, why we might need
a hippocampus."


I agree. And further to this, research results involving sparse vectors
show that hippocampally realistic settings of certain of the parameters
of an otherwise standard MLP are sufficient to equal the
intermediary-term memory performance of human subjects. In other words,
current knowledge of sparse neural networks is actually consistent with
human intermediary-term memory performance, which is associated with the
hippocampus.
See bottom of email for my Masters abstract.

Jay McClelland wrote:

[text deleted]
"To allow rapid learning of the contents of a particular experience,
the arguement goes, a second learning system, complementary to the
first [neocortex..],
 is needed; such a system has a higher learning rate and recodes
inputs using what we call 'sparse, random conjunctive coding' to
minimize interference (while simultaneously reducing the adequacy of
generalization).  These characteristics are just the ones that appear
to characterize the hippocampal system: it is the part of the brain
known to be crucial for the rapid learning of the contents of specific
experiences; it is massively plastic; and neuronal recording studies
indicate that it does indeed use sparse, random conjunctive coding."


My research mentioned above, which found simple parameters allowing a
sparse network to equal human intermediary-term memory performance on
the tasks studied confirm Jay's comments. He also refers to sparseness
"simultaneously reducing the adequacy of generalization". I also studied
the issue of this generalisation/memory trade-off in my thesis. I found
that indeed generalisation can be wiped out by sparseness if the domain
is combinatorial. By *combinatorial*, I mean where any input features can
be combined freely in any combination to form an input vector. I also
found, affirming results of French and Lewandowsky, that generalisation
was not affected in simpler, noncombinatorial (actually classification)
domains.

This problem in combinatorial domains of sparseness damaging
generalisation, a classic utility of neural networks, is a bit
discouraging for applications. However, any working algorithm must find
a way to both sufficiently separate memories to avoid interference, and 
to combine them sensibly to perform tasks requiring recognition of shared
features. In this vein, current knowledge actually suggests that the
human brain has traded off detailed discrete memory against
generalisation capability. Relative to conventional silicon chip Von
Neumann computers, everyone is aware of human frustration with detailed
discrete symbolic memory. This, despite the colossal hardware available
in the human brain. Note that the "catastrophic interference" of naive
networks referred to in the literature is catastrophic relative to
humans; but humans are also catastrophic relative to computers. This
frustration relative to silicon chip computers may stem from
*partially* overlapping semantic feature based representations, which may
be necessary for generalisation and content addressability. According to
the literature on catastrophic interference in artificial networks,
overlap the representations too much and memory disappears; too little,
and generalisation vanishes (at least in a combinatorial domain).
Humans can perform generalisation on tasks spanning periods well under
the few months or years that memories take to become established
in the neocortex, hence the hippocampus may be involved. This line of
thinking, based on the catastrophic interference literature, suggests
the possibility that in order to have generalisation capability, the
human weakness with discrete symbolic memory may be an inevitability --
in artificial as well as biological computers. Most AI researchers
believe that they will need human-comparable computer hardware to achieve
human level performance; I think this is further evidence for that view.


Bryan Thompson wrote:

    Max writes,

    Think about the structure of this argument for a moment. It runs
    thus:

    1. Neural networks suffer from catastrophic interference.
    2. Therefore the cortical memory system suffers from catastrophic
    interference.  3. That's why we might need a hippocampus.

    Is everyone happy with the idea that (1) implies (2)?

    Max
                  max at currawong.bhs.mq.edu.au

  "I am not happy with the conclusion (1), above.  Catastrophic
  interference is a function of the global quality of the weights
  involved in the network.  More local networks are, of necessity, less
  prone to such interference as less overlapping subsets of the weights
  are used to maps the transformation from input to output space.  Modifying
  some weights may have *no* effect on some other predictions. In
  the extreme case of table lookups, it is clear that catastropic
  interference completely disappears (along with most options for
  generalization, etc.:) In many ways, it seems that this statement is
  true for supervised learning networks in which weights are more global
  than not.  Other, more practical counter examples would include
  (differentiable) CMACs and radial basis function networks.
  (text deleted..)"


This mostly ties in. But a couple of elaborations. Although sparser
networks in earlier research always reduced interference, the memory
performance of multilayered sparse networks was generally much below
what one might intuitively expect (see Masters abstract below). The
reasons for this I explain below, but if you set up the network
correctly then your comments about progressively more severe sparseness
reducing interference are correct. Regarding radial basis function
networks, I might expect an RBF net with narrow activation bases to be
analogous to sparse MLPs, although Robins' work in progress (just below)
seems pessimistic about its memory capabilities. Some of the unexpected
but correctable problems which I found with sparse networks might have
their analogues in narrowed RBFs.


Anthony Robins wrote:

[stuff deleted]...
In any case, retaining old items for rehearsal in a
network seems somewhat artificial, as it requires that they be
available on demand from some other source, which would seem to make
the network itself redundant.

[I agree. Retaining items for rehearsal requires memory overhead in a
system which is supposed to be trying to optimise memory...]

It is possible to achieve the benefits of rehearsal, however, even
when there is no access to old items.  This "pseudorehearsal"
mechanism, introduced in Robins (1995), is based on the relearning of
artificially constructed populations of "pseudoitems" instead of the
actual old items.

In MLP / backprop type networks a pseudoitem is constructed by
generating a new input vector at random, and passing it forward
through a network in the standard way.  Whatever output vector this
input generates becomes the associated target output.
[stuff deleted] ....
(Work in progress suggests that simply using a "local"
learning algorithm such as an RBF is not enough).

We have already linked pseudorehearsal in MLP networks to the
consolidation of information during sleep (Robins, 1996).
[stuff deleted]...


Considerations of some kind of rehearsal for consolidation of information
during sleep, and over the long-term in general sound very interesting.
Regarding shorter memory tasks, say over a number of minutes such
as the ones I studied in my Masters, it seems less likely to me that
the brain would have the time and memory capacity to implement
pseudorehearsal. From some of your data, rather large pseudopopulation
overhead seems to be needed to slow the swinging off-target (as shown by
dot products with target) of the original learned output vectors.


The abstract of my Masters thesis is appended below, but I mention a
few bits of extra relevant detail here. As mentioned above, it was
successfully demonstrated that hidden-layer sparseness, in combination
with three other hippocampally realistic network conditions, can
eliminate catastrophic interference. Some of the simulations were a
resolution of McCloskey and Cohen's [1989] expose of catastrophic
interference. The other three necessary anti-interference factors were
context dominance (for context-dependent tasks of course), initial
weightsize and the bias influence.
Context dominance refers to large (i.e. larger than 1) context unit
activation values.  It was set to 4 for the human-equal
context-dependent simulation, which actually matched the whole human
forgetting curve well. Comparing with the hippocampus, there is no easy
way to determine just how much "attention" it pays to list context. Much
larger than normal initial weightsizes -- at values typical of
in-training sizes -- were also found to be necessary for both tasks in
avoiding catastrophic interference. One would expect weight sizes in the
brain to always be at "in-training" sizes. Finally, a small bias
influence relative to the input layer, such as is typically found in
large networks (the hippocampus is a huge network), was required.

Here is a summarised explanation for how the four factors
influence interference. For sparseness, it's the obvious one given many
times by other researchers; sparseness reduces overlap, thus reduces
weight-learning interference. However, in my simulations sparseness by
itself only ever got first list retention just off the floor;
interference from the second list was still catastrophic. For the
context-dependent task, the most important other factor as context
dominance, which works as follows. If context activation values are too
small, switching list context is not going to change the total summed
inputs to the hidden layer by much. Consequently, the network will
train mostly the same weights for second list items as it previously
did for the first list, wiping out memory of list 1. Turning to initial
weightsizes. Firstly, training on list 1 naturally pushed the weights
most involved to the "intraining" sizes -- much greater than traditional
initialisations. During list 2 training, backpropagation of error
through these enlarged weights was far greater than through small
weights, which directly encouraged the new associations to be learned
using those same weights over again. Initialising weights around the
in-training range removed this quite substantial interference effect.
Regarding bias influence. In a small model network, the bias for each
node, traditionally set to 1, is significant in comparison to the
previous layer's fan-in to the node. When a hidden node is "switched
on" during early training (list 1), its bias is naturally increased to
speed up the training. The result is that the hidden node's activation
threshold is now low, and it is therefore much more likely to be
turned on during the learning of list 2. Thus early and late training
tends strongly to use the same hidden nodes, increasing interference.
This problem vanished for smaller bias settings.

It was not obvious that low initial weight-sizes and high bias influence
could cause substantial interference, and I point out that these factors
should obtain across a wide variety of commonly used networks.

I'll probably submit material from the Masters catastrophic interference
sections for publication in the near future (and probably some SOM
stuff too). If someone asks me about it, I can put the latest version of
the thesis on the web in a few weeks, I'm finishing some
examiner-initiated modifications. The thesis has a great deal more
references on catastrophic interference than I've appended below.

\title{On Sparse Neural Network Representations: Catastrophic Interference,
 Generalisation, and Combinatoriality}
\author{Rafael Antony Brander\\
 B.Sc.(Hons. Pure Math; Hons. Applied Math), G.Dip.(Comp.Sci.)

{\it A thesis submitted for the degree of Master of Science}

School of Information Technology\\
The University of Queensland\\
Australia}
\date{September 30, 1997}

Abstract:

  Memory is fundamental to human psychology, being necessary for any
thought processing. Hence the importance of research involving human
memory modelling, which aims to give us a better understanding of how the
human memory system works. Learning and forgetting are of course the most
important aspects of memory, and any worthwhile model of human memory
must be expected to exhibit these phenomena in a manner qualitatively
similar to that of a human. It has been claimed (see below) that standard
artificial neural networks cannot fulfil this elementary expectation,
suffering from {\it catastrophic interference}, and sparseness of neural
network representations has been employed, directly and indirectly, in
many of the attempts to answer this claim. Part of the motivation for the
employment of sparseness has been the fact that sparse vectors have been
observed in human neurological memory systems [Barnes, McNaughton,
Mizumori, Leonard and Lin, 1990]. In the broader field of neural
networks, sparse representations have recently become a popular tool of
research. This thesis aimed to discover what fundamental properties of
sparseness might justify the counter-claims alluded to above, and in so
doing uncovered a more general characterisation of the effects of sparse
vector representations in neural networks.

As yet, little formal knowledge of the concept of sparseness itself has
been reported in the literature; we developed some formal definitions and
measures, which served as foundational background theory for the thesis.
We also discussed several representative methods for implementing
sparsification.

We initially conjectured that the main problem of sparsification in the
case of a boolean space might be that of finding ``base'' clusters
without being concerned with orientation with respect to an origin. This
pointed us towards a clustering algorithm. We employed simulations and
theory to show that a particular sparse representation, which we derived
from a neural network cluster-based learning algorithm, the Self
Organising Map ({\it SOM}) [Kohonen 1982], is an ineffective basis for
even simple learning and generalisation tasks, {\it in combinatorial
domains}. The SOM is generally regarded as a good performer in
classification tasks, which is the noncombinatorial domain.

We then turned to the well known problem referred to earlier, where
neural networks are observed to fail to model a basic fundamental
property of human memory. Since McCloskey and Cohen [1989] and Ratcliff
[1990] first brought it to the attention of neural network researchers,
the problem of {\it catastrophic interference} in standard feedforward,
multilayer, backpropagation neural network models of human memory has
continued to generate research aimed at its resolution. Interference is
termed ``catastrophic'' when the learning of a second list almost
completely removes memory of a list learned earlier, and when forgetting
of this extreme nature does not occur in humans.

In previous research [French, 1991; McRae \& Hetherington, 1993],
sparseness at the hidden layer, either directly or indirectly induced,
has been shown to substantially reduce catastrophic interference in
memory tasks where no context is required to distinguish between the two
lists.

Our interference studies investigated the degree to which sparsification
algorithms can eliminate the serious problem of catastrophic
interference, by virtue of a comparison to the human performance data
[Barnes \& Underwood, 1959; Barnes-McGovern, 1964; Garskof, 1968], a
match to  which data had not yet been achieved with standard MLPs in the
literature. These studies investigated both the AB AC and AB CD list
learning paradigms, which represent instances of context dependent and
context independent tasks respectively.

It was successfully demonstrated that sparseness, in combination with
three other realistic network conditions, can eliminate catastrophic
interference. The other three necessary anti-interference factors were
context dominance, initial weightsize and the bias influence. Context
dominance was here definitionally implemented as setting the context
units to have large (i.e. larger than 1) activation values. Much larger
than normal initial weightsizes -- at values typical of in-training sizes
 -- were also found to be necessary in avoiding catastrophic
interference. Finally, a small bias influence relative to the input
layer, such as is typically found in large networks, was required. The
explanation for sparseness' removal of catastrophic interference was
argued to be that it reduces relative overlap between unrelated vectors.

However, it is believed [French, 1991; McRae \& Hetherington, 1993;
Lewandowsky, 1994; Sharkey \& Sharkey, 1995] that there is a trade-off
between sparseness and generalisation. We also addressed this trade-off
issue, and showed that the sparsification algorithms used above to
eliminate catastrophic interference concomitantly incur a great loss in
a neural network's generalisation capability {\it in combinatorial
domains}; although there was no such loss (as agreed by French [1992] and
Lewandowsky [1994]) if the domain was noncombinatorial.

Combining the results of the studies of the SOM and effects of sparseness
on generalisation, we suggested that sparseness has no effect on learning
or generalisation in noncombinatorial domains, and that in combinatorial
domains generalisation is removed while learning can only occur by
exhaustively training in a supervised scheme on all exemplars. Further,
it was shown that a more abstract result predicts -- and gives an
intuitive explanation for -- the specific results of all our experiments
discussed above. This more abstract, and intuitively expected result is
the following: {\it sparseness at the hidden layer of a standard network
has the effect, in combinatorial domains in particular, of reducing the
network's general operational similarity dependence on the similarity of
input vectors}.

The results of this thesis clarify understanding of the way the human
memory system works, which is of interest in psychology and neuroscience.
They also provide important insights into the functionality of the SOM
and the MLP.

[Barnes \& Underwood, 1959]{BU} Barnes, J. M. and Underwood, B.
 J. (1959).
``Fate'' of first-list associations in transfer theory.
 {\it Journal of Experimental Psychology}, {\bf 58}(2), 97-105.

[Barnes-McGovern, 1964] Barnes-McGovern, J. M. (1964).
Extinction of associations in four transfer paradigms.
 {\it Psychological Monographs: General and Applied}, Whole No. 593,
 {\bf 78}(16), 1-21.

[Barnes et al., 1990]{Ba} Barnes, C. A., McNaughton, B. L.,
 Mizumori, S. J.
 and Lim, L. H. (1990).
 Comparison of spatial and temporal characteristics of neuronal 
activity in sequential stages of hippocampal processing.
  {\it Progress in Brain Research}, {\bf 83}, 287-300.

[French, 1991]{F91} French, R. (1991).
 Using semi-distributed representations to overcome 
	catastrophic forgetting in connectionist networks.
 {\it Proceedings 
	of the 13th Annual Conference of the Cognitive Science Society},
 173-178. Hillsdale, NJ: Erlbaum.

[French, 1992]{F} French, R. M. (1992).
Semi-distributed representations and catastrophic
 forgetting in connectionist networks.                      
{\it Connection Science}, {\bf 4}, (3/4), 365-378.

[Garskof, 1968] Garskof, B. E. (1968).
Unlearning as a function of degree of interpolated learning and method of
testing in the A-B, A-C and A-B, C-D paradigms.
 {\it Journal of Experimental Psychology}, {\bf 76}(4), 579-583.

[Kohonen, 1982]{K} Kohonen, T. (1982).
 Self-organised formation of topologically correct feature maps.
{\it Biological Cybernetics}, {\bf 43}, 59-69.

[Lewandowsky, 1994]{L94} Lewandowsky, S. (1994).
 On the relation between catastrophic
	interference and generalization in connectionist networks.
Journal of Biological Systems, Vol. 2(3), 307-333.

[McCloskey \& Cohen, 1989]{MC} McCloskey, M. and Cohen, N. J.
 (1989).
  Catastrophic interference in connectionist networks: the
sequential learning problem.
In (Ed.), G. H. Bower,
{\it The Psychology of Learning and Motivation}, {\bf 24}, 109-165.

[McRae \& Hetherington, 1993]{MH}  McRae, K. and 
Hetherington, P. A. (1993).
 Catastrophic interference is eliminated in pretrained networks.
 {\it Proceedings of the
Fifteenth Annual Conference of the Cognitive Science Society}, 723-728.
Hillsdale, NJ: Erlbaum.

[Ratcliff, 1990]{R} Ratcliff, R. (1990).
  Connectionist models of recognition memory: constraints imposed
 by learning and forgetting functions.
{\it Psychological Review}, {\bf 97}(2), 285-308.

Sharkey, NE and Sharkey, AJC (1995).
 An analysis of Catastrophic Interference.
Connection Science, 7, 301-329 


From murre at psy.uva.nl  Wed Sep  2 15:12:13 1998
From: murre at psy.uva.nl (Jaap Murre)
Date: Wed, 02 Sep 98 21:12:13 0200
Subject: What have neural networks achieved?
Message-ID: <199809021909.AA19979@uvapsy.psy.uva.nl>

Max Coltheart wrote:

	Suppose a net learned Task A to criterion and then was trained on Tas
	B to criterion without any further exposure to A (no interleaving,
	nothing corresponding to rehearsal of A). Then retest on A will reveal
	catastrophic forgetting.

	What happens to people here? If I spend 1998 learning to play golf,
	and 1999 learning to play tennis and doing *nothing at all about golf*,
	I would not expect my golf game to be have been completely blown away
	when I try it again on Jan 1, 2000.

	Isn't this a big difference between how neural nets learn and how
	people learn?

	Circularity needs to be avoided here e.g. it would not be good to
	reply: If your golf game is still there, you must have been rehearsing
	it in 1999.

If one only focusses on one element of current hippocampal models this 
circularity may appear.

In fact, there are two independent problems here:

1. How neural networks are able to learn sequential tasks without much 
interference.

2. How the brain (e.g., hippocampus-cortex architecture) accomplishes this.

Neural networks that have very distributed architectures (such as 
backpropagation) will tend to show catastrophic interference. That is,
when compared to the human data (e.g., Osgood, 1949), they will either
show too much forgetting or--what is less well known--they will show 
too *much* learning (Murre, 1996a). As was pointed out in the debate, 
this effect can be reduced by interleaved learning of various kinds, 
bringing the network behavior in line with the human data. (It can also 
be reduced by using localist, modular or semi-distributed architectures.)

Hippocampus models that deal with the effects of hippocampal lesions,
must be able to explain why such lesions tend to obliterate *recent* rather
than old memories (called the Ribot effect). In normal forgetting these
recent memories are most readily accessible; under lesioning they are 
the first to go. This is typically explained by assuming (1) that 
memories are first stored in or via the hippocampus (or medial
temporal lobe complex) and (2) that there is a process of consolidation
whereby the memories are strengthened at a cortical level. At least
three models have been published with neural network simulations of such 
a process (Alvarez and Squire, 1994; McClelland, McNaughton, and O'Reilly,
1995; Murre, 1996b). Consolidation in these models is implemented by
selecting representations in the hippocampus by some random process and
giving them extra learning trials at a cortical level. (There is also
some process by which representations are gradually lost from the
hippocampus.) Some neurobiological data exists supporting such a
process (Wilson and McNaughton, 1994).

McClelland et al. use the catestrophic interference effect as an 
in-principle argument why this consolidation process exists, which
resembles interleaved learning. Other arguments have been put 
forward. We, for example, stress the fact that the cortex has a 
'connectivity problem' making it somewhat time-consuming to set up 
the long-range connections that underlie episodic memories (Murre 
and Sturdy, 1995). 

Evidence for the consolidation process is still a little thin 
at a neurobiological level. Some new data in neuropsychology has
recently emerged that seems to carry the thought experiment
"What would happen if consolidation could *not* take place, 
i.e., the case where the brain remains dependent primarily on the 
representations in the hippocampus (and some remnants in the 
cortex). This seems to be the case in a newly discovered form of 
dementia, called semantic dementia, whereby the semantic 
representations disappear but the episodic memory remains relatively 
preserved. On the basis of modelling work, we have predicted and 
found several new characteristics of semantic dementia(Graham and 
Hodges, 1997; Murre, Graham, and Hodges, submitted). 

Though there is clearly an enormous amount of work to be done, I
think that it is fair to say that neural network models have
contributed and continue to contribute towards our understanding 
of human (and animal)learning and memory and that one cannot rule 
out hippocampus/amnesia models on the basis of circularity.


References

Alvarez, P., & L.R. Squire (1994). Memory consolidation and the 
	medial temporal lobe: a simple network model. Proceedings 
	of National Academy of Sciences (USA), 91, 7041-7045.

Graham, K.S., & J.R. Hodges (1997). Differentiating the roles of 
	the hippocampal complex and the neocortex in long-term memory 
	storage: evidence from the study of semantic dementia and 
	Alzheimer's disease, Neuropsychology, 11, 1-13.

McClelland, J.L., B.L., McNaughton, & R.C. O'Reilly (1995). Why 
	there are complementary learning systems in the hippocampus 
	and neocortex: insights from the successes and failures of 
	connectionist models of learning and memory. Psychological 
	Review, 102, 419-457.

Murre, J.M.J. (1996a). Hypertransfer in neural networks. Connection 
	Science, 8, 225-234.

Murre, J.M.J. (1996b). TraceLink: a model of amnesia and consolidation 
	of memory. Hippocampus, 6, 675-684.

Murre, J.M.J., & D.P.F. Sturdy (1995). The connectivity of the brain: 
	multi-level quantitative analysis. Biological Cybernetics, 
	73, 529-545.

Murre, J.M.J., K.S. Graham and J.R. Hodges (submitted). Semantic 
	dementia: new constraints on connectionist models of long-term 
	memory. Submitted to Psychological Bulletin.

Osgood, C.E. (1949). The similarity paradox in human learning: a 
	resolution. Psychological Review, 56, 132-143.

Wilson, M.A., & B.L. McNaughton (1994). Reactivation of hippocampal 
	ensemble memories during sleep. Science, 255, 676-679.

From mike at deathstar.psych.ualberta.ca  Wed Sep  2 21:56:10 1998
From: mike at deathstar.psych.ualberta.ca (Michael R.W. Dawson)
Date: Wed, 2 Sep 1998 19:56:10 -0600 (MDT)
Subject: Job Openings At U. of A.
Message-ID: <Pine.LNX.3.96.980902195530.32579A-100000@emperor.psych.ualberta.ca>

Three Tenure-Track Assistant Professor Positions in Brain & Behaviour


The Department of Psychology, Faculty of Science at the University of
Alberta, is seeking to expand its development in the area of Brain and
Behaviour. Over the next two years, three tenure-track positions at the
level of assistant professor (salary floor $40,638) will be open to
competition. The first appointment, will be effective July 1, 1999; the
second and third appointments will be effective July 1, 2000.

Applicants should have expertise in any one of the following or related
approaches to the study of brain and behavior: neural plasticity and
development, brain dysfunction, computational or systems, and comparative
cognition.  Applicants must have completed their PhD or equivalent degree
by July 1, 1999.  The expectation is that the successful candidate will
secure NSERC, MRC, AHFMR, or equivalent funding.  Hiring decisions will be
made on the basis of demonstrated research capability, teaching ability,
the potential for interactions with colleagues, and fit with departmental
needs.

A curriculum vitae, a description of current and planned research, copies
of recent publications, and at least three letters of reference should be
sent to: Dr Charles H M Beck, Acting Chair, Department of Psychology, P220
Biological Sciences Building, University of Alberta, Edmonton, Alberta,
Canada  T6G 2E9. The closing date for applications for the position
available July 1, 1999, is November 15, 1998.  


In accordance with Canadian Immigration requirements, this competition is
directed at Canadian Citizens and permanent residents of Canada.  The
University of Alberta is committed to the principle of equity in
employment.  As an employer we welcome diversity in the workplace and
encourage applications from all qualified women and men, including
Aboriginal peoples, persons with disabilities, and members of visible
minorities.

Further information is available on the web at URL
http://web.psych.ualberta.ca/people.htmld.


--
Professor Michael R.W. Dawson | mike at bcp.psych.ualberta.ca | (403)-492-5175
Biological Computation Project, Dept. of Psychology, University of Alberta
Edmonton, AB, CANADA T6G 2E9 | http://www.bcp.psych.ualberta.ca/~mike/


From terry at salk.edu  Wed Sep  2 22:22:24 1998
From: terry at salk.edu (Terry Sejnowski)
Date: Wed, 2 Sep 1998 19:22:24 -0700 (PDT)
Subject: NEURAL COMPUTATION 10:7
Message-ID: <199809030222.TAA25746@helmholtz.salk.edu>

Neural Computation -  Contents Volume 10, Number 7 - October 1, 1998

VIEW

Analog Versus Digital:  Extrapolating from Electronics to Neurobiology
Rahul Sarpeshkar

NOTE

Employing The Z-Transform to Optimize the Calculation of the Synaptic 
Conductance of NMDA-and Other Synaptic Channels in Network Simulations
J. Kohn and F. Worgotter

LETTERS

Site-Selective Autophosphorylation of Ca2+/Calmodulin-
Dependent Protein Kinase II as a Synaptic Encoding Mechanism
C. J. Coomber

Ion Channel Stochasticity May Be Critical in Determining the 
Reliability and Precision of Spike Timing
Elad Schneidman, Barry Freedman, and Idan Segev

Fast Temporal Encoding and Decoding with Spiking Neurons
David Horn and Sharon Levanda

Linearization of F-I Curves by Adaptation
Bard Ermentrout

Mutual Information, Fisher Information and Population Coding
Nicolas Brunel and Jean-Pierre Nadal

Synaptic Pruning in Development:  A Computational Account
Gal Chechik, Isaac Meilijson and Eytan Ruppin

Spatial Decorrelation in Orientation-Selective Cortical Cells
Alexander Dimitrov and Jack D. Cowan

Receptive Field Formation in Natural Scene Environments:  
Comparison of Single-Cell Learning Rules
Brian S. Blais, N. Intrator, H. Shouval and Leon N. Cooper

Neural Feature Abstraction from Judgements of Similarity
Michael D. Lee

Classification of Temporal Patterns In Dynamic Biological Networks
Patrick D. Roberts

Kernel-Based Equiprobable Topographic Map Formation
Marc M. Van Hulle

An Energy Function and Continuous Edit Process for Graph Matching
Andrew M. Finch, Richard C. Wilson, and Edwin R. Hancock

Approximate Statistical Test for Comparing Supervised Classification 
Learning Algorithms
Thomas G. Dietterich

Probability Density Estimation Using Entropy Maximization
Gad Miller and David Horn

-----

ABSTRACTS -  http://mitpress.mit.edu/NECO/

SUBSCRIPTIONS - 1998 - VOLUME 10 - 8 ISSUES

                  USA    Canada*   Other Countries
Student/Retired   $50    $53.50      $78
Individual        $82    $87.74     $110
Institution      $285   $304.95     $318
 * includes 7% GST

(Back issues from Volumes 1-9 are regularly available for
$28 each to institutions and $14 each for individuals.
Add $5 for postage per issue outside USA and Canada.
Add +7% GST for Canada.)

MIT Press Journals, 5 Cambridge Center, Cambridge, MA 02142-9902.
Tel: (617) 253-2889  FAX: (617) 258-6779 mitpress-orders at mit.edu

-----

From elizondo at axone.u-strasbg.fr  Thu Sep  3 03:36:04 1998
From: elizondo at axone.u-strasbg.fr (David ELIZONDO)
Date: Thu, 3 Sep 98 09:36:04 +0200
Subject: catastrophic forgetting 
Message-ID: <9809030736.AA00658@axone.u-strasbg.fr>

The Recurcive Deterministic Perceptron (RDP) is an example of a  
neural network model that does not suffer from catastrophic  
interference. This feedforward multilayer neural network is a  
generalization of the single layer perceptron topology (SLPT) that  
can handle both linearly separable and non linearly separable  
problems.

Due to the incremental learning  nature of the RDP neural networks,  
the problem of catastrophic interference will not arise with this  
learning method. The latter because the topology is build one step at  
the time by adding an intermediate neuron (IN) to the topology. Once  
a new IN is added, its weights are frozen. 


Here are two references describing this model:

	M. Tajine and D. Elizondo.
	The recursive deterministic perceptron neural network. 

	Neural Networks (Pergamon Press). Acceptance date : 

	Mars 6, 1998

	M. Tajine and D. Elizondo. 

	Growing Methods for constructing Recursive eterministic 

	Perceptron Neural Networks and Knowledge extraction.   	
	Artificial Intelligence (Elsevier).
	Acceptance date : May 6, 1998

A limited number of pre-print hard copies are available.

From thorpe at cerco.ups-tlse.fr  Thu Sep  3 08:14:08 1998
From: thorpe at cerco.ups-tlse.fr (Simon Thorpe)
Date: Thu, 3 Sep 1998 13:14:08 +0100
Subject: Multiprocessor boards for neural network simulations...
Message-ID: <v03007825b21438635d45@[193.54.228.14]>

Hi,

	Over the past few months I have been in discussion with Neil Carson
from Causality Ltd. http://www.causality.com/ in the UK about the
possibility of developping hardware for doing neural network simulations.
Right now, the project is looking very promising,  and Neil has said that
he would be happy to go ahead and build a first batch of 25 boards as soon
as he can be reasonably confident that there will be enough buyers. I
myself will be buying 8-10 boards, but we need a few other interested
people to get the project off the ground. I was wondering whether anyone on
the comp-neuro mailing list might be interested.

	Briefly, what we are proposing is the following. The basic board
will be a standard PCI board that could be used in PCs, Macs or indeed any
other computer with PCI slots in it. Each board would be fitted with 6
processor modules, three on each side, which would plug into standard
SODIMM memory slots (Small Outline Dual In-Line Memory Modules), the same
type of slots that are used for memory expansion on laptop computers. Each
of these daughter boards would have a StrongARM  SA-110 micro-processor, a
Digital 21285-A PCI bridge circuit, and 32 Mbytes of SDRAM. In effect, each
board would be a computer in its own right, and would run a version of Unix
(Linux or NetBSD). It would also have an IP address, allowing messages to
be sent efficiently via the PCI bus from processor to processor. Neil and
the other programmers at Causality will look after the message passing
mechanisms, probably using I20 protocols (if you know what that is).

	In our case, we want to use these boards to run a parallel version
of SpikeNET, our asynchronous spiking neuronal network simulator. It turns
out that this particular program would parallelise very nicely on such a
system, because the communication bandwidth between processors is kept very
low. If you're interested in SpikeNET, just let me know - we're seriously
thinking about making the program available to whoever wants to use it. But
in fact, the same hardware could also be used for any sort of parallel
program that could be run using PVM or MPI message passing protocols,
including (I presume) Parallel Genesis.

	There are some limitations though. The StrongARM SA-110 doesn't
have a floating point unit, but as long as you only want to do integer
calculations, it goes like a rocket. Our own software (which is integer
only) runs as fast on the StrongARM as on a Pentium II of the same clock
speed. This is really impressive, since the StrongARM doesn't have a second
level cache (unlike the Pentium II) and the code has (as yet) not been
optimised for the StrongARM at all. I don't know what the situation is with
Genesis - my guess is that there's a lot of floating point in it, but I'm
not sure that this could be recoded for fixed point - after all, most
neurones don't have voltages that reach 10 to the power 200 volts ;-)

	The reasons for choosing the StrongARM are pretty straightforward.
It is very small, doesn't get hot (< I Watt) and is cheap. This means that
it becomes perfectly feasible to imagine packing large numbers of
StrongARMs in a very small space without having to worry about overheating
(imagine trying to do the same thing with Pentium IIs). In addition,
although the future of StrongARM was once in doubt (it was co-developped by
Digital and Advanced Risc Machines), it has now been bought up by Intel who
have recently announced that they will be investing heavilly in StrongARM
development. See http://developer.intel.com/design/strong/ for details.

	The current top-of-the-range StrongARM runs at 233 MHz, and this is
what Neil Carson is proposing to use in this first batch. However, in the
not to distant future there will be 400 MHz StrongARMs with 100 MHz SDRAM
memory busses. And there will be a new StrongARM processor (the SA-1500)
which will have a separate floating point unit for multimedia operations.
One of the nice features of this daughter board arrangement is that it
would be pretty simple and cost effective to do a new batch of boards using
whatever the best technology is at that moment. Another advantage of using
this sort of parallel hardware is that  even last years technology will
still be useful to you - not like conventional PCs where you feel that you
have to buy a new computer every six months if you don't want to be
obsolete.

	So, what about prices you may be saying. Well, if you are
interested it should be possible to do such a board for 1200 pounds ($2000)
on this first run.  Each board would only take one PCI slot, so with four
free PCI slots you could put up to 24 processors in a single PC! If we can
round up enough interested people, we should be able to get the boards done
in about 2 months. Please note that I am not personally going to making any
money on this, and Causality are only expecting to break even on it.
However, both Neil and I are confident that this could be a really
promising approach - we just need to get enough support to get the ball
rolling. Obviously, the more people that are interested, the cheaper it
gets....

	If you want more information, don't hesitate to contact either me
or Neil at neil at causality.com.

	Best wishes

	Simon Thorpe





__________________
Simon Thorpe
Centre de Recherche Cerveau et Cognition
133, route de Narbonne
31062 Toulouse France
Tel 33 (0)5 62 17 28 03. Fax 33 (0)5 62 17 28 09
__________________



From aminai at ececs.uc.edu  Thu Sep  3 10:19:09 1998
From: aminai at ececs.uc.edu (Ali Minai)
Date: Thu, 3 Sep 1998 10:19:09 -0400 (EDT)
Subject: function of hippocampus
Message-ID: <199809031419.KAA19536@holmes.ececs.uc.edu>

	From trengove at socs.uts.EDU.AU  Thu Sep  3 04:37:08 1998
	X-Sender: trengove at linus
	Subject: Re: function of hippocampus
	MIME-Version: 1.0

	  ...the above quotes concerning the role of hippocampus in categories of
	memory such as episodic memory as well as tasks such as spatial cognition
	suggests to me we consider from a top down, psychological standpoint why
	spatial cognition and episodic memory should be tied together
	functionally, and hence why we shouldn't be surprised that a single part
	of the brain is involved in both.
	  From my own subjective observations, if I am trying to remember a
	thought that I had, or a particular piece of information that came up in a
	conversation, often the best way to proceed is for me to remember where I
	was when I had the thought.  Once I have remembered the place where I had
	the thought I have access to a rich pool of cues that can help to trigger
	the particular thought I'm after.
	  It thus makes sense to me that spatial cognition should be involved in
	the laying down of new memories.  In the course of a day I will have
	perhaps hundreds of disctinct cognitive experiences to remember, but the
	number of distinct _places_ in which I dwell whilst having those
	experiences is likely to be at least an order of magnitude smaller.
	  Thus it makes good sense to organise memory around the memories of the
	places one has been during the course of a day.

There has been debate among hippocampal theorists (mainly in the context of
the rodent hippocampus) about whether the hippocampus is dedicated solely or
predominantly to spatial cognition. The alternative --- advanced most prominently
by Howard Eichenbaum and co-workers --- is that the hippocampus helps construct
memories involving complex relationships between cues, conditions, contexts, etc.
(relational memory). In this view, purely spatial representations, such as the
cognitive map of O'Keefe and Nadel, are special cases of relational memory of
complex episodes.

Let us think of an episode as a spatio-temporal structure in some very
high-dimensional space, where the dimensions correspond to sensory cues,
features, contexts, motivations, internal states, etc. In any particular
case, most of the possible dimensions will be irrelevant, and the episodic
representation will lie in a subspace of the full space of possibilities.
When this subspace is purely spatial, we see a place representation. When
the subspace is predominantly spatial but also includes other dimensions,
we see conditional place representations (e.g., context-dependent,
directional, reward-related, etc.) In particular cases --- e.g., experiments
designed to make place irrelevant --- we would see non-spatial representations
(e.g., in Eichenbaum's olfactory experiments). In general, as the quotation
above points out, place is an extremely important component of any episodic
experience, and it would not be surprising to find strong place-dependencies
in any neural representation of episodic memory. I believe that most
hippocampal theory now implicitly acknowledges this.

Theories which see the hippocampus as primarily involved in spatial cognition
are based on rodent data with its place and head-direction cells. There is no
question that, in many instances, one is hard pressed to find any correlate
of a hippocampal pyramidal cell's activity other than location. However, in
strongly directed environments (e.g., arm mazes), direction becomes a factor
too. And several reports have demonstrated that cells fire in response to
task contingencies when these are made important. It is tempting to think that,
given a rat's limited mental life (am I being speceist:-), place is almost
the entire default sub-space of experience, except at particular times. Thus,
episodic representations appear --- especially when measured one cell at a
time --- as place representations. In higher animals such as primates, episodes
have much richer content, and space is only a part of the picture --- hence
the absence of place cells. Interestingly, one does find view cells in primates
(e.g., in the work of Rolls and co-workers) which fire when the animal is looking
at a particular scene. Perhaps this means that, for primates, what is seen during
an episode is more significant than where one is located. Also, it is possible
that primates have a better ability to project experience to places that they
can see but where they are not currently located.


        ......  I believe one idea in
	circulation e.g. discussed by Rolls and coworkers, is that the hippocampus
	provides a short term 'buffer' for storing memories, which are later
	'transferred' to the neocortex for long term storage.

This idea has been around for a while in various forms. In terms of modeling,
I think the work by Gluck and Mayers, Squire and Alvarez, O'Reilly, McClelland
and McNaughton, and recently, by Redish and Touretzky, offers interesting
perspectives.


	  Back to the specific idea given above, I'm curious whether specialists
	of the hippocampus find it (a) highly dubious, implausible or naive; or
	(b) too obvious to be worth mentioning; or (c) a potentially useful way
	to look at the role of the hippocampus.

Without claiming specialist status, my answer is (c), provided we think of the
big picture that includes the whole constellation of available data. I believe
strongly that big theoretical ideas --- such as the cognitive mapping theory
--- are crucial to our understanding of neural function, even when the theories
turn out to be less than perfect in the end.

Ali Minai


-----------------------------------------------------------------------------
Ali A. Minai
Assistant Professor
Complex Adaptive Systems Laboratory
Department of Electrical
   & Computer Engineering
               and Computer Science
University of Cincinnati
Cincinnati, OH 45221-0030

Phone: (513) 556-4783
Fax:   (513) 556-7326
Email: Ali.Minai at uc.edu

Internet: http://www.ececs.uc.edu/~aminai/

From arbib at pollux.usc.edu  Thu Sep  3 11:53:55 1998
From: arbib at pollux.usc.edu (Michael Arbib)
Date: Thu, 03 Sep 1998 08:53:55 -0700
Subject: What have neural networks achieved?
Message-ID: <199809031555.IAA14614@pollux.usc.edu>


The following extract from the debate on what AI has achieved may be of
interest to connectionists engaged in the present discussion, and in the
one on
symbolic representation.

One might ask:  Does AI need neural networks to be really successful?


Subject: Re: What have neural networks achieved?
>Date: Wed, 2 Sep 1998 20:50:12 -0700 (PDT)
>From: John McCarthy <jmc at Steam.Stanford.EDU>
>Subject: challenges to AI
>
>Paul Rosenbloom asks David McAllester:
>
>   Can you elaborate a bit on what
>   you would find necessary before you would see "any evidence for the
>   feasibility of the grand goals"?
>
>David McAllester replied:
>
>     Some convincing ability to discuss, say, daily events in the
>     life of a child.  A human-level theorem proving machine for
>     UNRESTRICTED conceptual mathematics would also be evidence
>     for me.  I am quite familiar with the current state of the
>     art in theorem proving and I feel like I have climbed a tree
>     while staring at the moon.  While current formal methods do
>     have applications, I do not believe that current
>     applications constitute evidence for the grand goals.  It
>     seems popular among AI researchers to take a "sour grapes"
>     attitude toward grand-goal problems like the Turing test and
>     human-level understanding of general conceptual mathematics
>     --- "oh those can't be done but they're not important
>     anyway".
>
>       David
>
>David McAllester's complaints are similar in some respects to those
>Hubert Dreyfus and Lotfi Zadeh presented at the recent Wonderfest in
>Berkeley.  The difference is that McAllester knows a lot more about
>AI.  At the Berkeley meeting, I tried to get Dreyfus and Zadeh to say
>what was the *easiest* task they considered infeasible to AI.  After
>some discussion, this came down to what task would not be accomplished 
>in the next ten years.  I think I got something out of each of them.
>
>Dreyfus gave the example of "Jane saw the puppy in the window of the
>pet store.  She pressed her nose against it."  The problem is to
>get the referent of "it" to be the window in an honest way, i.e. not
>building in too much.  This requires returning to the task of using
>world knowledge to get the referents of pronouns.  Mere scripts
>wouldn't do it.
>
>Zadeh's example was to get a car out of a parking garage with columns
>and lots of other cars some of which would have to be moved.  That one 
>doesn't seem very hard.  Zadeh thinks AI without fuzzy logic
>won't work.
>
>I have some sympathy with McAllester's point of view.  Much
>present work in AI uses methodologies that are limited in what
>they can ultimately do.  I discuss this in my
>
>FROM HERE TO HUMAN-LEVEL AI
>
>http://www-formal.stanford.edu/jmc/human.html.
>
>That paper lists some of the problems that must be solved to
>reach human-level AI.  I reread it and plan to improve it.
>
>I would like David McAllester to list some of the problems that
>he sees.  If there are any relatively concrete problems that he
>thinks can't be solved in the next ten years, this would serve as 
>a worthwhile challenge to AI research.  He should list the
>*easiest* problem he thinks won't be solved in ten years.
>
>It hasn't helped AI much to be challenged only by the ignorant,
>but McAllester isn't ignorant, so his challenges, if he can make
>them more concrete than in his message, will be helpful.
>
>To give an example, I don't think the methods that have been
>moderately successful at chess will succeed with Go.
>I say "moderately successful", because I think the amount of
>computer power used by Deep Blue is disgracefully large and
>conceals a lack of understanding.  Almost all of those 200
>million positions examined each second would be rightfully
>ignored by a more sophisticated program.
>
>See http://www-formal.stanford.edu/jmc/newborn.html which
>appeared in _Science_.
>
>
>
> 
> 
*************************
Michael Arbib
Director, USC Brain Project
University of Southern California
Los Angeles, CA 90089-2520
(213) 743-6452     FAX (213) 740-5687
arbib at pollux.usc.edu
http://www-hbp.usc.edu/HBP/ 

From trengove at socs.uts.EDU.AU  Thu Sep  3 04:36:53 1998
From: trengove at socs.uts.EDU.AU (Chris Trengove)
Date: Thu, 3 Sep 1998 18:36:53 +1000 (EST)
Subject: function of hippocampus
In-Reply-To: <199808270518.BAA06571@holmes.ececs.uc.edu>
Message-ID: <Pine.GSO.4.02A.9809031737480.26360-100000@linus>

As a non-expert in regards to the hippocampus I would like to offer a
thought triggered by the following remarks of Minai, to see what people
think:

On Thu, 27 Aug 1998, Ali Minai wrote:
> That having been said, I do think (and others can marshall the evidence
> better than I can) that a preponderance of evidence favors a hippocampal
> involvement in episodic memory and, at least in rodents, spatial cognition.
..
> The issue of hippocampal involvement in spatial cognition in rodents...
> is given overwhelming credibility ... by the undeniable existence of
> place cells and head-direction cells. 
> ... provides
> convincing evidence that the hippocampus ``knows'' a great deal about
> the animal's spatial environment, is very sensitive to it, and responds
> robustly to disruptions of landmarks, etc. 
.. 
> I do not think we really understand what role the rodent hippocampus plays
> in spatial cognition, but it is hard to dispute that it plays some ---
> possibly many --- important roles. I think that, as theories about
> hippocampal function begin to place the hippocampus in the larger context
> of other interconnected systems (e.g., in the work of Redish and Touretzky),
> we will move away from the urge to say, ``Here! This is what the hippocampus
> does'' and towards the recognition that it is probably an important part
> in a larger system for spatial cognition.
.. 
> Finally, one issue that is particularly relevant to hippocampal theories
> is the possibility that the categories of memory (e.g., episodic, declarative,
> etc.) or task (DNMS, spatial memory, working memory, etc.) that
> we use in our theorizing may not match up with the categories relevant to
> actual hippocampal functionality. Perhaps we are trying to build a science
> of chemistry based on air, water, fire, and earth. 

  So, the above quotes concerning the role of hippocampus in categories of
memory such as episodic memory as well as tasks such as spatial cognition
suggests to me we consider from a top down, psychological standpoint why
spatial cognition and episodic memory should be tied together
functionally, and hence why we shouldn't be surprised that a single part
of the brain is involved in both.
  From my own subjective observations, if I am trying to remember a
thought that I had, or a particular piece of information that came up in a
conversation, often the best way to proceed is for me to remember where I
was when I had the thought.  Once I have remembered the place where I had
the thought I have access to a rich pool of cues that can help to trigger
the particular thought I'm after.
  It thus makes sense to me that spatial cognition should be involved in
the laying down of new memories.  In the course of a day I will have
perhaps hundreds of disctinct cognitive experiences to remember, but the
number of distinct _places_ in which I dwell whilst having those
experiences is likely to be at least an order of magnitude smaller.
  Thus it makes good sense to organise memory around the memories of the
places one has been during the course of a day.  I believe one idea in
circulation e.g. discussed by Rolls and coworkers, is that the hippocampus
provides a short term 'buffer' for storing memories, which are later
'transferred' to the neocortex for long term storage.  This idea is in a
similar vein.
  On a more general note, this kind of thinking suggests that eventually
we will find that there is a harmonious correspondence between the
functional interrelationships of certain, various aspects of cognition on
the one hand and the manner in which various brain structures (areas and
pathways) are especially involved in these aspects of cognition.  Thus the
feedback between neuroscience and psychology ought to give us insights
into the functional organisation of cognition which could not otherwise be
found; i.e. to help us to find the right categories, to go beyond 'air
water fire and earth' c.f. Minai, above.
  Back to the specific idea given above, I'm curious whether specialists
of the hippocampus find it (a) highly dubious, implausible or naive; or
(b) too obvious to be worth mentioning; or (c) a potentially useful way
to look at the role of the hippocampus.

Chris Trengove
School of Mathematical Sciences,
University of Technology, Sydney.


From ucganlb at ucl.ac.uk  Mon Sep  7 13:06:25 1998
From: ucganlb at ucl.ac.uk (Neil Burgess - Anatomy UCL London)
Date: Mon, 07 Sep 98 17:06:25 +0000
Subject: Hippocampus, spatio-temporal context and episodic memory
Message-ID: <68743.9809071606@link-1.ts.bcc.ac.uk>

Several of the recent contributions to this list have considerd the 
relationship between the hippocampus and episodic and spatial memory. 
These appear to be converging on idea of the hippocampus providing a spatial
(and perhaps temporal) context in which events are embedded, so as
to form an episodic memory and facilitate their subsequent recall. 

This view of the role of the (right) human hippocampus was presented as 
part of O'Keefe and Nadel's idea of the hippocampus as a cognitve map
(for a concise discussion see O'Keefe and Nadel, 1979; pp 493 and 527).
Further discussion of this point of view, and the relationship between
hippocampal and parietal regions in spatial and mnemonic tasks can be 
found in the introduction and several of the chapters of the forthcoming
book (see below).

Best wishes,

Neil


O'Keefe and Nadel (1979) The Behavioural and Brain Sciences 2, 487-533
(Precis of The hippocampus as a cognitive map, and peer commentary).

The Hippocampal and Parietal Foundations of Spatial Cognition
(Eds: N Burgess, KJ Jeffery and J O'Keefe) Oxford University Press 
(1998 - should be out in time for the Soc. Neurosci. conference).


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dr Neil Burgess                        
Institute of Cognitive Neuroscience & Dept. of Anatomy
University College London              
London WC1E 6BT, U.K.                  
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

From erdi at rmki.kfki.hu  Mon Sep  7 12:54:13 1998
From: erdi at rmki.kfki.hu (Erdi Peter)
Date: Mon, 7 Sep 1998 18:54:13 +0200 (MDT)
Subject: Computational Neuroscience in Europe
Message-ID: <Pine.SGI.3.95.980907185232.19954A-100000@sgiserv.rmki.kfki.hu>


COMPUTATIONAL NEUROSCIENCE in EUROPE:  Where we are and where to go?


	Within the framework of the Forum of European Neuroscience held in
Berlin (27 June - 1 July 1998) a special workshop "Spectrum of
Computational Neuroscience in Europe" was organized (see:
http://www.hirn.uni-duesseldorf.de/~rk/ena98sw5.htm).

	In addition to the Workshop, an informal discussion on
Computational Neuroscience was held. This report summarizes the central
ideas: 

	The fast growing field of Computational Neuroscience plays an
important role in integrating the results of structural,
functional and dynamic approaches to the nervous system. In Europe, an
active community is emerging investigating structure/function
relationships of the nervous system with computational tools at
subcellular, cellular, network, and system levels.  

	To enhance the growth of this field in Europe it was thought that
the following specific undertakings should be considered:
	
1. To make a tentative list of the European Computational Neuroscience
(cns) labs and of their webpages;

2. To hear about the position of the cns labs within the national
neuroscience communities;
	
3. To search for funding programs;
	
4. To think on the formation of a network of European laboraories (i) to
apply for common grants; (ii) to organize a regular series of cns
meetings.
	
5. To consider the creation and utilization of Neuroscience databases as a
joint effort in collaboration with experimentalists. 	 
	The participants were well aware of not representing the field of
Computational Neuroscience in Europe and of not having any mandate. It was
felt, however, that this initiative could be a first step towards the
creation of a more representative voice. 

	The participants agreed that the creation of a network
among the European computational neuroscientists would be useful. 
We would highly appreciate your input at this point.  In the first
step we should like to make a tentative list of Computational
Neuroscience laboratories as mentioned in 1. Any pointers to
relevant web sites or email contacts are most welcome.  

	Concerning the second step we should like to hear if and how the
field of Computational Neuroscience is publicly represented in different
countries and whether it is considered useful to initiate the foundation
of such sections within the national (and thereby the European)
neuroscience societies. 	
	

	Prof. Peter Erdi
	Dept. Biophysics
	KFKI Research Institute for 
	Particle and Nuclear Physics
	of the Hungarian Academy of Sciences
	H-1525 Budapest, P.O. Box 49, Hungary
	Tel: (36-1)-395-92-20/2505 ext.

Fax: (36-1)-395-9151; 
	http://www.rmki.kfki.hu/biofiz/biophysics.html
	
	Dr. Rolf Kotter
	Center for Anatomy and Brain Research
	Heinrich Heine University, 
	Moorenstr. 5, D-40225 Dusseldorf, FRG 
	Tel./Fax: +49-211-81-12095
	http://www.hirn.uni-duesseldorf.de/~rk/
	


From austin at minster.cs.york.ac.uk  Mon Sep  7 14:22:00 1998
From: austin at minster.cs.york.ac.uk (Jim Austin)
Date: Mon, 7 Sep 1998 19:22:00 +0100
Subject: Connectionist symbol processing: any progress? 
Message-ID: <9809071922.ZM17338@minster.cs.york.ac.uk>



Its time to add one more (!) line of work on the symbolic neural networks
debate. We have been working on these systems for about 5 years, in the
context of binary neural networks. We have concentrated on the use
of outer-product methods for storing and accessing information in
many (commercially relevant) problems. We have not concentrated on the
cognitive significance (although I believe what we have done has some). We
exploit them for the following reasons;

1) The use of outer-product based methods are very fast in both learning and
access, this comes from the sparce representations we use, along with the
simple learning rules.

2) They are very memory efficient if used with distributed representations.

3) They have the ability to offer any-time properties (in this I mean
they can give a sensible result at any time after a fixed initial
processing time has passed).

4) They can be used in a modular way.

5) They are fault tolerant (probably)

6) They map to hardware very efficiently.

Our main work has been in there use in search engines, rule based systems
and image analysis.

The approach is based on the use of tensor products for binding data
before presentation to a CMM (an outer product based memory), the other
features are;
The use of binary distributed representations of data.
The use of threshold logic to select interesting matches.
The use of pre-processing to orthoganalise data for efficient representation.
The use of superposition to maintain fixed length representations.

Our work has looked at the use of large numbers of CMM systems for solving
hard problems. For example, we have done work using them for
molecule databases, where they are used for structure matching. We
have been using them to recognise images (simple ones), where they can
be made to generalise to translations and scale varying images.

We have built hardware to perform binary outer-products and other operations,
and are now scaling up the architecture to support large numbers of CMMs
using these cards.

The largest CMM we have used (i.e. an outer product based representation) is
750Mb on a text database.

Details of this work can be found on our web pages and in the following
papers;

The basic methods of tensor binding;
%A    J Austin
%E    N Kasabov
%J    International Journal on Fuzzy Sets and Systems
%T    Distributed Associative Memories for High Speed Symbolic Reasoning
%P 223-233
%V 82
%D 1996


Some more of the same;
%A J Austin
%B Conectionist Symbolic Integration
%D 1997
%E Ron Sun
%E Frederic Alexander
%I Lawrence Erlbaum Associates
%C 15
%P 265-278
%T A Distributed Associative Memory for High Speed Symbolic Reasoning
%K BBBB+
%O ISBN 0-8058-2348-4


The properties of a CMM that we use to store the bindings;
%A Mick Turner
%A Jim Austin
%T Matching Performance of Binary Correlation Matrix Memories
%J Neural Networks
%D 1997
%I Elsevier Science
%P 1637-1648
%N 9
%V 10

The use of the symbolic methods to do molecular database searching;
(A new paper just sent to Neural networks is also on this)
%A M Turner
%A J Austin
%T A neural network technique for chemical graph matching
%D July 1997
%E M Niranjan
%I IEE
%B Fifth International Conference on Artificial Neural Networks, Cambridge, UK

Image understanding architecture;
%A C Orovas
%A J Austin
%D 1997
%J 5th IEEE Int. Workshop on Cellular Neural Networks and their application.
%D 14-17 April 1998
%T A cellular system for pattern recognition using associative neural networks

The hardware;
%A J Austin
%A J Kennedy
%T PRESENCE, a hardware implementation of binary neural networks
%J International Conference on Artificial Neural Networks
%C Sweden
%D 1998





Jim

-- 

Dr. Jim Austin, Senior Lecturer, Department of Computer Science, University of York, York,
YO1 5DD, UK.
Tel : 01904 43 2734
Fax : 01904 43 2767
web pages: http://www.cs.york.ac.uk/arch/


From reggia at cs.umd.edu  Tue Sep  8 14:44:15 1998
From: reggia at cs.umd.edu (James A. Reggia)
Date: Tue, 8 Sep 1998 14:44:15 -0400 (EDT)
Subject: Faculty Position in Neural Modeling
Message-ID: <199809081844.OAA05767@avion.cs.umd.edu>


      FACULTY POSITION AVAILABLE

The position is intended for an entry level, full time
faculty member at the Maryland Psychiatric Research Center, 
a dedicated research facility, part of the University of Maryland's
Department of Psychiatry, in Baltimore.

This funded position is meant to provide imaging, computational, and
research assistant resources.  We are looking for someone who wants to
use fMRI, possibly supplemented by evoked response potential data, to
develop and test models of brain-behavior relationships.

Using the high spatial and temporal relationships provided by fMRI and ERP
studies, the investigator should be able to generate mathematical
models to account for observed and predicted phenomena.  

The investigator should have a Ph.D. in a field that is directly
concerned with biological modeling or system engineering.  Applied 
mathematics, computer science, electrical engineering, and neuroscience
(with an emphasis on system modeling) are appropriate areas of 
concentration.  

Substantial resources for fMRI and ERP research are available for
the investigator.  

A competitive salary will be offered.

Interested applicants should contact
Henry H Holcomb, M.D.
email = hholcomb at mprc.umaryland.edu
phone = 410 719 6817
fax = 410 719 6882
address = MPRC, P.O. Box 21247, Baltimore, MD 21228-0247.

From ehartman at pav.com  Tue Sep  8 15:49:00 1998
From: ehartman at pav.com (Eric Hartman)
Date: Tue, 08 Sep 98 14:49:00 CDT
Subject: NN commercial successes
Message-ID: <35F58AC9@pav.com>


>Michael Arbib wrote:
>
> b) What are the "big success stories" (i.e., of the kind the general
> public could understand) for neural networks contributing to the
> construction of "artificial" brains, i.e., successfully fielded
> applications of NN hardware and software that have had a major
> commercial or other impact?

Pavilion Technologies, Inc., is a very strong NN-based commerical   
 success story.
See Pavilion's web site
   http://www.pavtech.com
for extensive information about Pavilion.

Pavilion produces NN-based software for modeling, optimizating, and   
controling
  continuous process manufacturing processes.
Hundreds of on-line applications are in operation across a number of   
major industries
  world-wide.

Pavilion's many software products include:
 - Process Insights, primarily geared toward steady-state modeling and   
optimization
 - Process Perfecter, a (nonlinear/linear) dynamic, closed-loop   
controller/optimizer
 - Software CEM
 - Virtual OnLine Analyzer
 - BOOST

Pavilion's primary customer base spans blue chip corporations in the   
petrochemical,
power generation, refining, pulp and paper, and food processing markets.   
   

The technology is widely applicable to all continuous manufacturing   
processes.
Pavilion maintains its headquarters in Austin, with offices in New   
Jersey, Brussels,
Frankfurt, and Tokoyo. Pavilion currently employs over 100 people   
worldwide.

Pavilion was incorporated in Austin, Texas in September 1991. Pavilion   
has been honored
with numerous awards for both technical and business accomplishments,   
including:
 - Process Insights received a second consecutive Reader's Choice Award
      for "Best Neural Network Software" from Control Magazine.
 - Process Perfecter received Product of the Year title at the
      1997 KPMG Austin ICE High Tech Awards program.
 - Pavilion has twice been recognized as one of the 20 fastest growing
      companies in Austin by the Austin Business Journal and Ernst &   
Young.
 - Pavilion has been awarded 8 patents for technology by the U.S. Patent   
Office,
      with 10 additional patents pending.

Application Example:
  Chevron Chemical s Experience with Pavilion's Process Perfecter

"The installation of Pavilion s Perfecter controllers on our BP-licensed,   

gas-phase high density polyethylene (HDPE) and linear low density
 polyethylene (LLDPE) reactors has been a very rewarding project," says
 a Chevron spokesman. "The controllers have been well received by the
operating group.  We chose Pavilion for the ease of modeling from   
historical
data, nonlinear capabilities of their dynamic multivariable controller,   
and the
polymer experience and competence of the Pavilion engineers."

"Results from this project have been impressive. Our reactor regulatory,
quality and product transition control have been significantly improved
using Pavilion s dynamic multivariable controller and neural net based
virtual on-line analyzers. The results have exceeded our expectations,"
says the Chevron spokesman.

"We have been working with Pavilion for three or four years, using their
Process Insights modeling software," says Russ Clinton, Supervisor of
Automated Systems for the Chevron site.  "Pavilion s expertise and unique   

controller capabilities were well matched to the needs of ourapplication.   

We have recently decided to extend Pavilion s control technology to our
autoclave low density polyethylene (LDPE) process."
   

For more information about Pavilion, please visit Pavilion's web site:
  http://www.pavtech.com

====================================================

Dr. Eric Hartman
Chief Scientist
hartman at pav.com

(512) 438-1534
(512) 438-1401 fax

Pavilion Technologies, Inc.
11100 Metric Blvd., #700
Austin, TX 78758-4018
USA

From granger at uci.edu  Tue Sep  8 20:52:38 1998
From: granger at uci.edu (Richard Granger)
Date: Tue, 8 Sep 1998 17:52:38 -0700
Subject: What have neural networks achieved?
Message-ID: <v02140b01b21b8169d34e@[128.200.120.16]>


Michael Arbib wrote:
>>  So: I would like to see responses of the form:
>>  "Models A and B have shown the role of brain regions C and D in functions E
>>  and F - see specific references G and H".

Models of the induction and expression (storage and retrieval) mechanisms
of synaptic long-term potentiation (LTP), the actual biological change in
connection strength that underlies at least some forms of real
telencephalic memory in humans, have led to novel hypotheses of the
functions LTP gives rise to in the actual circuitries in which it occurs.

One instance is the olfactory bulb-cortex system:  LTP in this system
(which was shown by Kanter and Haberly, Brain Research, 525:175-179, 1990;
and Jung et al., Synapse, 6: 279-283, 1990), endogenously induced by the 5
Hz "theta" rhythm that occurs during exploration and learning (Komisaruk,
J.Comp.Physiol.Psychol., 70: 482-492, 1970; Macrides, Behav.Biol., 14:
295-308, 1975; Otto et al., Hippocampus, 1991), was shown in models to lead
to an unexpected function of not just remembering odors but organizing
those memories hierarchically and producing successively finer-grained
recognition of an odor over iterative (theta) cycles of operation
(Ambros-Ingerson et al., Science, 247: 1344-1348, 1990; Kilborn et al.,
J.Cog.Neurosci., 8: 338-353, 1996).

This is an instance in which modeling of physiological activity in
anatomical circuitry gave rise to an operation that was unexpected from
behavioral studies, and had been little-studied in the related
psychological and behavioral literatures.


>>  The real interest comes when claims appear to conflict.

Other studies of the olfactory system have yielded quite different
predictions; this raises the question of whether animals cluster and
subcluster odors behaviorally, and whether paleocortical cells respond
selectively to different sampling cycles of clusters of similar odors.
These important issues are far from resolved; some relevant experimental
evidence on behavioral learning of odors is found in (Granger et al.,
Psychol. Sci., 2: 116-118, 1991); and on unit cell activity in cortex
during learning in (McCollum et al., J.Cog.Neurosci., 3: 293-299, 1991; and
see Granger & Lynch, Curr.Biol., 1: 209-214, 1991, for a review).


>>  What about the role of hippocampus in both spatial navigation and
>>  consolidation of short term memory?

Many studies begin with observed behaviors linked to medial temporal
regions by lesion studies and some chronic recording; it has been pointed
out that the connection specifically to hippocampus, as opposed to
surrounding perirhinal cortical regions, is difficult.  Moreover, the range
of behaviors, from spatial navigation to short-term memories, are
suggestive of emergent operations arising from combinations of more
fundamental mechanisms that may be occurring within the various modules of
these brain areas.

Not only is the medial temporal region composed of hippocampus, subiculum,
and overlying cortical structures, but these naming conventions occlude the
richness of circuitries within.  The "hippocampus" is composed of three
extraordinarily distinct structures (dentate, CA3 and CA1), each of which
consists of very different cell types, synaptic connections and local
circuits, and which are strongly connected with neighboring structures
(subiculum, pre- and parasubiculum, and superficial and deep entorhinal
cortex).  Of interest from a modeling point of view are the distinct
functions that emerge from the physiological operations of these disparate
circuits as well as composite functions arising from interactions among
them.  It will be interesting to uncover not only how these circuits
participate in well-studied behavioral circumstances such as navigation and
memory consolidation, but also what heretofore unexpected functions may be
found to arise from their action and interaction (Lynch & Granger,
J.Cog.Neurosci., 4: 189-199, 1992; Granger et al., Hippocampus, 6:
567-578).

It's worth mentioning that a special issue of the journal "Hippocampus"
dedicated to "computational models of hippocampal function in memory"
appeared as Volume 6, number 6 (1996); it may be a useful reference for
this part of the discussion.

- Rick Granger




From becker at curie.psychology.mcmaster.ca  Wed Sep  9 11:04:39 1998
From: becker at curie.psychology.mcmaster.ca (Sue Becker)
Date: Wed, 9 Sep 1998 11:04:39 -0400 (EDT)
Subject: correction to NIPS*98 workshop announcements
Message-ID: <Pine.SGI.3.96.980909110335.29540G-100000@meitner.psychology.mcmaster.ca>

Michael Kearn's name was inadvertently left off the organizing list
for the NIPS*98 workshop on Integrating Supervised and Unsupervised 
Learning. A corrected announcement for that workshop appears below. 

**************************************************************

TITLE: Integrating Supervised and Unsupervised Learning

This workshop will debate the relationship between supervised and
unsupervised learning.  The discussion will run the gamut from
examining the view that supervised learning can be performed by
unsupervised learning of the joint distribution between the inputs and
targets, to discussion of how natural learning systems do supervised
learning without explicit labels, to the presentation of practical
methods of combining supervised and unsupervised learning by using
unsupervised clustering or unlabelled data to augment a labelled
corpus.  The debate should be fun because some attendees believe
supervised learning has clear advantages, while others believe
unsupervised learning is the only game worth playing in the long run.

More information (including a call for abstracts) can be found at
www.cs.cmu.edu/~mccallum/supunsup.

ORGANIZERS:   Rich Caruana
              Virginia de Sa
              Michael Kearns
              Andrew McCallum




From bert at mbfys.kun.nl  Wed Sep  9 04:11:48 1998
From: bert at mbfys.kun.nl (Bert Kappen)
Date: Wed, 9 Sep 1998 10:11:48 +0200 (MET DST)
Subject: PhD position available
Message-ID: <199809090811.KAA23643@bertus.mbfys.kun.nl>

PhD position for neural network research at SNN, University of 
Nijmegen, the Netherlands.

Background:
The SNN neural networks research group at the university of Nijmegen 
consists of 10 researchers and PhD students and conducts
theoretical and applied research on neural networks and graphical
models. The group is part
of the Laboratory of Biophysics which is involved in experimental brain
science.

Recent research of the group has focussed on theoretical description of
learning processes using the theory of stochastic processes and the design
of efficient learning rules for Boltzmann machines using techniques from
statistical mechanics; the extraction of rules from data and the
integration of knowledge and data for modeling; the design of robust
methods for confidence estimation with neural networks; applications in
medical diagnosis and prediction of consumer behaviour. 

Research project:
The modern view on AI, neural networks as well as parts of statistics, is to
describe learning and reasoning using a probabilistic framework. A particular
advantage of the probabilistic framework is that domain knowledge in the form
of rules and data can be easily combined in model construction. The main
drawback is that inference and learning in large probabilistic networks 
is intractible.
Therefore, robust approximation schemes are needed to apply this technology to
large real world applications. 
The topic of research is to develop learning rules for neural networks
and graphical models using techniques from statistical mechanics.

Requirements:
The candidate should have a strong background in theoretical physics or
mathematics.

The PhD position:
Appointment will be full-time for four years. Gross salary will be NLG 2184 
per month in the first year, increasing to NLG 3899 in the fourth year. 

More information:
Details about the research can be found at
http://www.mbfys.kun.nl/SNN or by contacting dr. H.J.
Kappen (bert at mbfys.kun.nl, ++31243614241). 

Applications (three copies) should include a curriculum vitae and a
statement of the candidate's professional interests and goals, and one
copy of recent work (e.g., thesis, article). Applications 
should be sent before October 10 to the
Personnel Department of the Faculty of Natural Sciences, University of
Nijmegen, Toernooiveld 1, 6525 ED Nijmegen, vacancy number 98-52.


From dhw at santafe.edu  Wed Sep  9 14:28:27 1998
From: dhw at santafe.edu (dhw@santafe.edu)
Date: Wed, 9 Sep 1998 12:28:27 -0600
Subject: New Journal Announcement
Message-ID: <199809091828.MAA05358@santafe.santafe.edu>


We apologize if you receive multiple copies of this message.
 

****************************************
	NEW JOURNAL ANNOUNCEMENT
****************************************


                   JOURNAL OF COMPLEX SYSTEMS, Vol. 1, # 1

 
 
CONTENTS

Editorial						9
E. Bonabeau 

Modelling migration and economic agglomeration with 
active brownian particles				11
F. Schweitzer 

Amplitude spectra of fitness landscapes			39
W. Hordijk and P. F. Stadler 


From halici at rorqual.cc.metu.edu.tr  Thu Sep 10 06:20:29 1998
From: halici at rorqual.cc.metu.edu.tr (Ugur Halici)
Date: Thu, 10 Sep 1998 13:20:29 +0300
Subject: CFP: HYBRID APPROACHES ON SOFT COMPUTING TECHNIQUES
Message-ID: <35F7A7ED.16F0@rorqual.cc.metu.edu.tr>

CALL FOR PAPERS 
--------------------------------------------------------------------------------
Special Session on 
HYBRID APPROACHES FOR SOFT COMPUTING TECHNIQUES
--------------------------------------------------------------------------------
in SOCO'99, SOFT COMPUTING, June 1-4, 1999 at the Palazzo Ducale in
Genova, Italy


High quality research papers are sought on the  hybrid approaches or
comparative studies  using at least two of Neural Networks, Genetic
Algorithms or Fuzzy Logic. If you are interested in presenting a paper
in this special session in SOCO'99 please send me
(halici at rorqual.cc.metu.edu.tr)

1. A draft title of your paper                       now
2. One page abstract of the paper and a short CV     by September 20

Then the deadlines are:
3. Draft paper   (at most 7 pages)                   October   15
4. Full paper                                        January   31

You may find detailed information on SOCO  at site
SOCO'99 http://www.icsc.ab.ca/soco99.htm

------------------------------------------------
Ugur Halici, Prof. Dr., (Session Organizer)
Dept of Electrical and Electronics Eng.
Middle East Technical University,
06531, Ankara, Turkey

email: halici at rorqual.cc.metu.edu.tr or ugur-halici at metu.edu.tr
http://www.metu.edu.tr/~wwwnng/ugur/halici.html
Tel: (+90) 312 210 2333
Fax: (+90) 312 210 1261

From austin at minster.cs.york.ac.uk  Thu Sep 10 13:15:55 1998
From: austin at minster.cs.york.ac.uk (Jim Austin)
Date: Thu, 10 Sep 1998 18:15:55 +0100
Subject: Connectionist symbol processing: any progress?
Message-ID: <9809101815.ZM19721@minster.cs.york.ac.uk>


Another outline of symbolic/neural work taking place at York, UK, that
may be of interest to the debate.

Jim Austin


\author{Victoria J. Hodge and Jim Austin}
e-mail: vicky,austin at cs.york.ac.uk

We are proposing a unified-connectionist-distributed system.  The system is
currently theoretical, proposing a logical architecture that we aim to map onto
the AURA \cite{Austin_96}, \cite{AURA_web} modular, distributed neural network
methodology.

We posit a flexible, generic hierarchical topology with three fundamental
layers: features, case, and classes.  There is no repetition, each concept is
represented by a single node in the hierarchy, maintaining consistency and
allowing multifarious data types to be represented thus permitting generic
domains to be accommodated.

The front-end is an implicit, self-organising, unsupervised approach
similar to the Growing Cell Structures of Fritzke
\cite{Fritzke_93:TR} but constructing a hierarchy on top of the clusters and
generating feature descriptions and weighted connections for all nodes.  This
hierarchy will be mapped onto the binary representations of AURA and input to a
hierarchically arranged CMM topology representing features, cases and classes;
all partitioned into CMMs according to the natural partitions inherent in the
hierarchy.

A constraint elimination process will be implemented initially to eliminate
implausible concepts and context-sensitively reduce the search space.  A
spreading activation (SA)-type process (purely theoretical at present) will be
initiated on the required features (i.e., sub-conceptually) and allowed to
spread via the weighted links throughout the hierarchy.  SA is postulated as
psychologically plausible and can implement context effects (semantically
focussing  retrieval) and priming of recently retrieved concepts.  The highest
activated case(s) and class(es) will be retrieved as the best match.

New classes, cases and features can be aggregated into the hierarchy anytime,
simply and efficiently merely by incorporating new node and connections.  We
also aim to implement a deletion procedure that will ensure our hierarchy
remains within finite bounds (i.e., is asymptotically limited).  When a
predetermined size is reached, nodes are generalised that have least utility,
i.e., are covered by other nodes and least frequently accessed.  This allows
forgetting as a new addition results in the generalisation of older concepts.

Future investigation includes: structured concepts; more complexity for classes
(including hierarchically divided classes); weight adaptation where the weights
in the hierarchy are adjusted if the retrieved case(s) or class(es) are a poor
match; solution adaptation allowing solutions to be generated for new cases and
possibly generated from subsections of other solutions and aggregated together;
and, an explanation procedure.

@misc{AURA_web,
	title	= {{The AURA Homepage:
	\emph{http://www.cs.york.ac.uk/arch/nn/aura.html}}},
	}

@Article{Austin_96,
	author 	= {Austin, J. },
	title 	= {{Distributed associative memories for high speed symbolic
				reasoning}},
	journal = {Fuzzy Sets and Systems},
	volume 	= 82,
	pages 	= {223--233},
	year 	= 1996
	}

@Techreport{Fritzke_93:TR,
	author 	= {Fritzke, Bernd},
	title 	= {{Growing Cell Structures - a Self-organizing Network for
		Unsupervised and  Supervised Learning}},
	institution	= {International Computer Science Institute},
	address = {Berkeley, CA},
	number 	= {TR-93-026},
	year 	= 1993,
	}

--


-- 

Dr. Jim Austin, Senior Lecturer, Department of Computer Science, University of York, York,
YO1 5DD, UK.
Tel : 01904 43 2734
Fax : 01904 43 2767
web pages: http://www.cs.york.ac.uk/arch/


From bernabe at cnmx4-fddi0.imse.cnm.es  Fri Sep 11 04:44:00 1998
From: bernabe at cnmx4-fddi0.imse.cnm.es (Bernabe Linares B.)
Date: Fri, 11 Sep 1998 10:44:00 +0200
Subject: ART Microchips
Message-ID: <199809110844.KAA08430@cnm12.cnm.es>


Book Announcement:

ADAPTIVE RESONANCE THEORY MICROCHIPS, Circuit Design Techniques
Authors: T. Serrano-Gotarredona, B. Linares-Barranco and A. G. Andreou

is now available for purchase from Kluwer Academic Publishers
at "http://www.wkap.nl/book.htm/0-7923-8231-5". Table of content and
preface can be copied from "http://www.imse.cnm.es/~bernabe".


ADAPTIVE RESONANCE THEORY MICROCHIPS, Circuit 
Design Techniques, describes circuit strategies resulting in 
efficient and functional adaptive resonance theory (ART) 
hardware systems. While ART algorithms have been developed 
in software by their creators, this is the first book that addresses 
efficient VLSI design of ART systems. All systems described in 
the book have been designed and fabricated (or are nearing 
completion) as VLSI microchips in anticipation of the 
impending proliferation of ART applications to autonomous 
intelligent systems. To accomodate these systems, the book not 
only provides circuit design techniques, but also validates them 
through experimental measurements. The book also includes a 
chapter tutorially describing four ART architectures (ART1, 
ARTMAP, Fuzzy-ART and Fuzzy-ARTMAP) while providing 
easily understandable MATLAB code examples to implement 
these four algorithms in software. In addition, an entire chapter 
is devoted to other potential applications in real-time data 
clustering and category learning. 


From uwe.zimmer at gmd.de  Fri Sep 11 09:15:15 1998
From: uwe.zimmer at gmd.de (Uwe R. Zimmer)
Date: Fri, 11 Sep 1998 15:15:15 +0200
Subject: PostDoc Pos. at GMD Japan Research Lab. (Robotics)
Message-ID: <35F92261.4783DF31@gmd.de>

PosDoc-Pos-Announcement


     --------------------------------------------------------    
             Post-Doctoral Research Positions in

           Autonomous Robotics in Open Environments
   --------------------------------------------------------



Two new post-doctoral positions are open at GMD Japan Research 
Laboratory, Kitakyushu, Japan and will be filled at the earliest 
convenience. The new laboratory (starting officially at first of 
November with a team of 8 scientists and 3 support staff, where 
these first 8 positions should be filled completely until March 
'99) is based on long term cooperations with the Japanese research
community and focuses on the robotics and the telecommunications research fields.

Investigated questions (in the area of robotics) are:

   - How to localize and move in many DoF without global 
     correlation? 
   - Interpretation / integration / abstraction / compression of 
     complex sensor signals? 
   - How to build models of previously unknown environments? 
   - Direct sensor-actuator prediction 
   - How to coordinate multiple loosely coupled robots?

Underwater robotics is regarded as one of the most promising
experimental and application environment in this context. Real six
degrees of freedom, real dynamic environments and real autonomy 
(which is required in most setups here), settle these questions in
a fruitful area. The overall goal is of course not 'just' 
implementing prototype systems, but to get a better understanding 
of autonomy, and situatedness. Modeling, adaptation, clustering, 
prediction, communication, or - from the perspective of robotics -
spatial and behavioral modeling, localization, navigation, and 
exploration are cross-topics addressed in most questions.

Although we are an independent research group, there are of course
close connections to the robotics activities in the institute of 
Thomas Christaller at GMD (German National Research Center for 
Information Technology) in Sankt Augustin, Germany.

Techniques employed and developed up to now include dynamical 
systems, connectionist approaches, behavior-based techniques, rule
based systems, and systems theory. Experiments are based on 
physical robots (not yet underwater!). Thus the discussion of 
experimental setups and particularly the meaning of embodiment 
became topics in itself.

If the above challenges rose your interest, please proceed to our 
expectations regarding the ideal candidate:

   - Ph.D. / doctoral degree in computer sciences, electrical 
     engineering, physics, mathematics, biology, or related 
     disciplines. 
   - Experiences in experimenting with autonomous systems 
   - Theoretical foundations in mathematics, control, 
     connectionism, dynamical systems, or systems theory 
   - Interest in joining an international team of motivated 
     researchers

Furthermore it is expected that the candidate evolves/introduces 
her/his own perspective on the topic, and pushes the goals of the 
whole group at the same time.

Salary starts at 8 Mill. Yen per year depending on experience.

For any further information, and applications (including addresses
of referees, two recent publications, and a letter of interest!) 
please contact:

Uwe R. Zimmer (address below)

link to related activities: http://www.gmd.de/AutoSys/

___________________________________________ 
                                     ____________________________|
    Dr. Uwe R. Zimmer  -  GMD                                 ___|
                    Schloss Birlinghoven                         |
                    53754 St. Augustin, Germany                  |
  _______________________________________________________________.
          Voice: +49 2241 14 2373 - Fax: +49 2241 14 2384        |
             http://www.gmd.de/People/Uwe.Zimmer/                |

From schierwa at informatik.uni-leipzig.de  Fri Sep 11 07:07:46 1998
From: schierwa at informatik.uni-leipzig.de (Andreas Schierwagen)
Date: Fri, 11 Sep 1998 13:07:46 +0200
Subject: Call for Participation: FNS '99
Message-ID: <35F90482.7713@informatik.uni-leipzig.de>

Dear Colleagues,

Below is a brief annoucement of the 6th International Workshop
"Fuzzy-Neuro Systems '99" taking place in Leipzig, Germany on
March 18-19, 1999.
We have published a web page with further details. See

http://www.informatik.uni-leipzig.de/~brewka/FNS/

Andreas Schierwagen

------------------------------------------------------------------------


                        6th International Workshop

                         Fuzzy-Neuro Systems '99


Fuzzy-Neuro Systems `99 is the sixth event of a well established series
of workshops with international participation. Its aim is to give an
overview of the state of the art in research and development of fuzzy
systems and artificial neural networks. Another aim is to highlight
applications of these methods and to forge innovative links between
theory and application by means of creative discussions.

Fuzzy-Neuro Systems `99 is being organized by the Research Committee 1.2
"Inference Systems" (Fachausschuss 1.2 "Inferenzsysteme") of the German
Society of Computer Science GI (Gesellschaft f?r Informatik e. V.) and
Universit?t Leipzig, March 18 - 19, 1999

Workshop Chairs: Gerhard Brewka and Siegfried Gottwald
Local Organization: Ralf Der and Andreas Schierwagen

Topics of interest include:

theory and principles of multivalued logic and fuzzy logic
representation of fuzzy knowledge
approximate reasoning
fuzzy control in theory and practice
fuzzy logic in data analysis, signal processing and pattern recognition
fuzzy classification systems
fuzzy decision support systems
fuzzy logic in non-technical areas like business administration,
	management etc.
fuzzy databases
theory and principles of artificial neural networks
hybrid learning algorithms
neural networks in pattern recognition, classification, process
	monitoring and production control
theory and principles of evolutionary algorithms: genetic algorithms and
	evolution strategies
discrete parameter and structure optimization
hybrid systems like neuro-fuzzy systems, connectionist expert systems
	etc.
special hardware and software

Submissions should be extended abstracts of 4-6 pages. Please send 5
copies of your contribution to Gerhard Brewka, Universit?t Leipzig,
Institut f?r Informatik, Augustusplatz 10-11, 04109 Leipzig, Germany.
Proceedings containing full versions of accepted papers will be
published.

Preliminary schedule:


      Submission of papers: Nov. 8th, 1998
      Notification of authors: Dec. 15th, 1998
      Final version of accepted papers: Jan. 16th, 1999
      Workshop: March 18-19, 1999

The conference language will be English.

From cmbishop at microsoft.com  Fri Sep 11 10:20:57 1998
From: cmbishop at microsoft.com (Christopher Bishop)
Date: Fri, 11 Sep 1998 07:20:57 -0700
Subject: Cambridge-Microsoft Postdoctoral Research Fellowship
Message-ID: <3FF8121C9B6DD111812100805F31FC0D06C00E25@RED-MSG-59>


                 Darwin College Cambridge

               Microsoft Research Fellowship

          http://research.microsoft.com/cambridge  
                http://www.dar.cam.ac.uk/ 

The Governing Body of Darwin College Cambridge, and Microsoft Research
Limited (MSR), jointly invite applications for a stipendary
post-doctoral Research Fellowship supporting research in the field of
adaptive computing (including topics such as pattern recognition,
probabilistic inference, handwriting recognition, statistical learning
theory, computer vision and speech recognition). Applicants should
hold a PhD or should be expecing to have completed their thesis prior
to commencement of the Fellowship.

The Fellowship, which is funded by MSR, will be tenable for two years
commencing on 1 January 1999 (or any other mutually convenient date).
The successful candiate will work closely with Professor C M Bishop at
the MSR laboratory in Cambridge. In addition to a salary, the
Fellowship provides funding for conference participation.

College accommodation will be provided, subject to availability, or an
accommodation allowance will be paid in lieu.

Applicants should send their curriculum vitae, a list of publications,
and the names and addresses of three referees, via email (with the
subject line Darwin-Microsoft Research Fellowship) to
jmg39 at hermes.cam.ac.uk

Hard copies may be sent by surface mail to the Master's Secretary,
Darwin College, Cambridge, CB3 9EU.

   * The closing date for applications is 28 September 1998. *


From jjameson at bayarea.net  Sat Sep 12 19:22:39 1998
From: jjameson at bayarea.net (John Jameson)
Date: Sat, 12 Sep 1998 16:22:39 -0700
Subject: could you post this?
Message-ID: <35FB023E.77DFD006@bayarea.net>

Autonomous Systems, a small startup located in the Bay Area, CA, is
seeking one person to join us.  Our focus is shifting to computer
vision and the consumer PC market.  This person should have a good
background in C++, machine learning (especially neural networks), and
mathematical analysis.  Experience in vision, as well as an
entrepreneurial blood type, are plusses.

Email us if you would like more details.  If you prefer, attach your
resume and what kinds of problems interest you (Word 97 or prior,
postscript, text, pdf).

Best regards,

John Jameson
CTO
Autonomous Systems
San Carlos, CA
jjameson at bayarea.net
http://cdr.stanford.edu/~jameson/autonomous/public_html



From jbower at bbb.caltech.edu  Mon Sep 14 14:05:06 1998
From: jbower at bbb.caltech.edu (James M. Bower)
Date: Mon, 14 Sep 1998 10:05:06 -0800
Subject: Neural networks and neuroscience
Message-ID: <v0401171db22307d6cd8d@[131.215.137.89]>

I would point out that Michael Arbib's request:

>>  "Models A and B have shown the role of brain regions C and D in functions E
>>  and F - see specific references G and H".

does not necessarily involve "neural networks" in the strict sense at all.
My understanding is that the original question raised by Michael involved
the value of "neural network" research to brain science.

There are many models of brain function that have no relationship to what
are generally accepted as "neural network" forms (connectionist models,
backprop, etc).

Further, I would claim that there are very few examples where "neural
network" type models have had much to say at all about neurobiology.  A
quick look at the NN component of the table of contents of the NIPS
proceedings (NIPS being the long running gold standard for neural
network/connectionist research) should make clear the lack of connection
(or real interest) of most NN practisioners in real neurobiology.
Similarly, a survey of the usual traffic on this mailing list reveals the
same.

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


With respect to Richard Granger's post,  the statement:

(Various neuro-biological results)  was shown in models to lead
to an unexpected function of not just remembering odors but organizing
those memories hierarchically and producing successively finer-grained
recognition of an odor over iterative (theta) cycles of operation
(Ambros-Ingerson et al., Science, 247: 1344-1348, 1990)

Was not, in fact, unexpected, as the model was specifically designed to do
just this.


Jim Bower



From gary at cs.ucsd.edu  Mon Sep 14 20:37:40 1998
From: gary at cs.ucsd.edu (Gary Cottrell)
Date: Mon, 14 Sep 1998 17:37:40 -0700 (PDT)
Subject: What have neural networks achieved?
Message-ID: <199809150037.RAA20208@gremlin.ucsd.edu>

(Hi Jay)

I like the Seidenberg & McClelland account of reading, as
well as PMSP. However, the PMSP result (nonword reading)
really relies on a very well engineered representation of
the outputs. That is, it is very difficult to get a poor
pronunciation with that representation, which already encodes
many of the rules of English pronunciation. Thus, it seems
like one of Lachter and Bever's TRICS (The Representation It
Crucially Supposes). This does not contradict the fact that
a single mechanism account has been demonstrated. But one of
Jay's "loose ends" is: How would a network develop such a
representation in the first place? Which I think is a crucial
question.

g.

From jbower at bbb.caltech.edu  Mon Sep 14 20:11:57 1998
From: jbower at bbb.caltech.edu (James M. Bower)
Date: Mon, 14 Sep 1998 16:11:57 -0800
Subject: nn and neuroscience
Message-ID: <v04011742b2235f566544@[131.215.137.89]>

I should probably say, to somewhat temper my earlier email, that the one
area where NNs and neuroscience HAVE interacted in a more direct and
potentially useful way, is in the general area of thinking about memory
storage.  This includes not only the work on the hippocampus being
discussed at present in this mail group, but also the work of Mike Hasselmo
on neuromodulation of olfactory and hippocampal networks.

However, the vast majority of those in NNs have little direct interest in
the details of brain function -- accordingly, it is not particularly
surprising that much of the work is not particularly related to this
subject.


Jim Bower



From dario at cns.nyu.edu  Tue Sep 15 09:21:45 1998
From: dario at cns.nyu.edu (Dario Ringach)
Date: Tue, 15 Sep 1998 09:21:45 -0400
Subject: Postdoctoral position
Message-ID: <980915092145.ZM25831@alberich.cns.nyu.edu>


Applications are invited for a postdoctoral position to investigate
mechanisms of cortical processing in primate visual cortex using
single and multi-electrode techniques.  The emphasis of the research
is on the measurement and mathematical modeling of neural dynamics in
assemblies of visual cortical neurons.  The Departments of
Neurobiology and Psychology at UCLA provide an excellent research
environment to combine psychophysical, computational and experimental
studies of vision.  Details of past research may be found at
http://cns.nyu.edu/home/dario.  Candidates with previous experience in
primate cortical electrophysiology are preferred.  Candidates with
mathematical or engineering background are also encouraged to apply.
The position is available starting Jan 1st, 1999.  Applicants should
send a CV, the names of three references, and a summary of research
interests and experience to:

Before Dec 15th, 1998:

	Dario Ringach
	Center for Neural Science, Rm 809
	New York University
	4 Washington Place
	New York, NY 10003

	email: dario at cns.nyu.edu

After Dec 15th, 1998:

	Dario Ringach
	Dept of Psychology & Neurobiology
	Franz Hall
	405 Hilgard Ave
	Los Angeles, CA 90095-1563

From mharm at CNBC.cmu.edu  Tue Sep 15 13:07:13 1998
From: mharm at CNBC.cmu.edu (Mike Harm)
Date: Tue, 15 Sep 1998 13:07:13 EDT
Subject: Paper available: Phonology, Reading Acquisition, and Dyslexia
Message-ID: <199809151707.NAA24361@CNBC.CMU.EDU>

Hi.

The following paper has been accepted for publication in Psychological
Review.  It bears on much of the recent discussion on connectionist
models of reading.

=================================================================

         Phonology, Reading Acquisition, and Dyslexia:
             Insights from Connectionist Models


            Michael W. Harm, Mark S. Seidenberg

             University of Southern California


Abstract:

The development of reading skill and the bases of developmental
dyslexia were explored using a connectionist model of word
recognition.  Four issues were examined: the acquisition of
phonological knowledge prior to reading, how this knowledge
facilitates learning to read, the bases of phonological and
non-phonological types of dyslexia, and the effects of literacy on
phonological representation. Representing phonological knowledge in an
attractor network yielded improved acquisition and generalization
compared to simple feedforward networks.  Phonological and surface
forms of developmental dyslexia, which are usually attributed to
impairments in distinct lexical and nonlexical processing routes, were
derived from different types of damage to the network. The results
provide a computationally explicit account of the role of phonological
representations in normal and disordered reading and how they are in
turn shaped by their participation in the reading task. They also show
that connectionist principles that have been applied to skilled
reading and reading impairments following brain injury account for
many aspects of reading acquisition.

========================================================

Send me email at mharm at cnbc.cmu.edu for directions where to download
electronic (postscript or pdf) versions of the paper from.

I apologize that I have not provided a URL for the paper, and that you
must contact me for such information.  My understanding is that the
APA has rules against making such things available in a direct form:

    "If a paper is accepted for publication and the copyright has been
     transferred to the American Psychological Association, the author
     must remove the full text of the article from the Web site. They
     author may leave an abstract up and, on request, the author may 
     send a copy of the full article (electronically or by other means) 
     to the requestor."

     (from http://www.apa.org/journals/posting.html)


cheers,

Mike Harm
mharm at cnbc.cmu.edu
http://www.cnbc.cmu.edu/~mharm/
-----------------------------------------

  Big Science.  Every man, every man for himself.
  Big Science.  Hallelujah.  Yodellayehoo.

                                 -Laurie Andersen


From granger at uci.edu  Tue Sep 15 15:16:15 1998
From: granger at uci.edu (Richard Granger)
Date: Tue, 15 Sep 1998 12:16:15 -0700
Subject: Unexpected hypotheses arising from brain circuit simulation
Message-ID: <v02140b01b2233a42ccab@[128.200.120.16]>


jbower wrote that a set of results of ours, published in Science (1990), ...

> "... Was not, in fact, unexpected, as the model was specifically designed
>to do
> just this."

Thanks for giving us the credit for this design.  Since nothing like this
had ever been suggested as a hypothesis of olfactory function (indeed, it's
still studied, and still controversial), if we designed it, it would be a
testament to our inventiveness.

Nonetheless, we admit that the findings actually were quite surprising to
us.  The result emerged only after considerable observation and analysis of
a series of brain network simulations of the anatomical circuits of the
olfactory bulb and cortex (as characterized by Price, Haberly, Shepherd,
and others), operating under normal physiological conditions (as identified
by both in vivo and in vitro work from many labs, including Macrides,
Kauer, Freeman, Haberly,  Eichenbaum, Otto, Komisaruk, Staubli, et al.) and
in response to the physiological induction and expression rules for
synaptic long-term potentiation (LTP; Kanter & Haberly, '90; Jung et al.,
'90).  Extensive references can be found in the many published papers on
the topic; a glance at Medline will readily find most of our papers,
containing these references.  Anyone interested in more detail is welcome
to contact us:  granger at uci.edu

Back to the science of the thing: the hypothesis in question is that the
operation of the bulb-cortex system not only acts to 'remember' odors but
operates (via feedback inhibition over iterative samples of an odor) to
produce first recognition of the general 'category' (or cluster) of an
odor, followed by successively finer-grained recognition over sequential
(theta) cycles of operation, thereby re-using cortical cells over
successive samples to effectively read out a hierarchical description of
the odor.  It's interesting to note that this remains an intriguing
candidate hypothesis that continues to be cited and studied both
behaviorally and physiologically, since its initial publication
(Ambros-Ingerson et al., Science, 247: 1344-1348, 1990).

Many relevant articles from our lab and others' have appeared over the
years; we can send (or post) a list to any interested parties.

-Rick Granger




From uipodola at jetta.if.uj.edu.pl  Tue Sep 15 11:53:05 1998
From: uipodola at jetta.if.uj.edu.pl (Igor T. Podolak)
Date: Tue, 15 Sep 1998 17:53:05 +0200 (MET DST)
Subject: Connectionist symbol processing: any progress?
In-Reply-To: <199808231449.AAA02154@numbat.cs.rmit.edu.au>
Message-ID: <Pine.SOL.3.90.980915173548.16805A-100000@jetta>

Arun Jagota and B. Garner wrote:

> * 
> * It would be nice if some sort of a record of the "Connectionist
> * Symbol Processing" debate were to be produced and archived for
> * the benefit of the community.
> * 
> I think this would be a good idea.. because there were so many 
> interesting ideas expressed.

  I have done some homework towards it.  You can find a first 
approach towards an archive of the discussion at my www home page

	http://www.ii.uj.edu.pl/~podolak

  With help of a Perl program I have moved all the emails into 
separate html files, and the discussion is organized as a time 
sorted liste, name sorted list, and a list of consecutive 
postings starting with Dave Touretzky's email.  For ease of use 
all the email addresses are turned into 'mailto:' links, and, 
most important, all the given web adresses of various pages and 
documents are turned into 'http:' links ready to use.

  In some places, where algorithms were described, the Perl 
program made them harder to read, but I shall work on.  

  Please email me if you find it usable.

					igor


Igor T. Podolak, PhD               phone (48 12) 6323355 ext 320
Computer Science Department        fax   (48 12) 6341865
Jagiellonian University            home  (48 12) 6331324
Nawojki 11, 30 072 Krakow, Poland
e-mail:uipodola at if.uj.edu.pl       http://www.ii.uj.edu.pl/~podolak/index.html


From Otto_Schnurr-A11505 at email.mot.com  Tue Sep 15 14:42:40 1998
From: Otto_Schnurr-A11505 at email.mot.com (Otto Schnurr-A11505)
Date: Tue, 15 Sep 1998 13:42:40 -0500
Subject: What have neural networks achieved?
Message-ID: <35FEB520.DFBC8AAC@ccrl.mot.com>

Michael Arbib wrote:
> b) What are the "big success stories" (i.e., of the kind the general public
> could understand) for neural networks contributing to the construction of
> "artificial" brains, i.e., successfully fielded applications of NN hardware
> and software that have had a major commercial or other impact?
>
>  *********************************
>  Michael A. Arbib
>  USC Brain Project
>  University of Southern California
>  Los Angeles, CA 90089-2520, USA
>  arbib at pollux.usc.edu
 
 
While our application does not address brain function, it does
represent a successful example of how neural networks are able learn
and synthesize human behavior.
 
Motorola has developed a text-to-speech synthesizer that utilizes
multiple cooperating neural networks, each specializing in a
particular area of human language ability.  This use of neural
networks for both linguistic and acoustic processing produces speech
with exceptional naturalness.  Speech produced by our system has been
found to be more acceptable to listeners than that of other commercial
systems [11].
 
The system excels in learning the specific characteristics of a given
speaker and allows us to develop new dialects and languages rapidly
when compared to other methods.
 
To date, we have developed four voices: two male speakers of American
English, one female speaker of American English and one male speaker
of Mandarin.  Additional linguistic processing has also produced
speech in Spanish, Greek and Turkish with an American accent.
 
 
Regards,
 
Otto Schnurr
 
Speech Processing Research Lab
Chicago Corporate Research Laboratories
Motorola
schnurr at ccrl.mot.com
 
--
 
  [1] Karaali, O., Corrigan, G., Massey, N., Miller, C., Schnurr, O.,
      & Mackie, A. (1998).  A High Quality Text-To-Speech System Composed
      of Multiple Neural Networks.  International Conference on
      Acoustics, Speech and Signal Processing.  Seattle.
 
  [2] Miller, Corey (to appear).  Individuation of Postlexical
      Phonology for Speech Synthesis.  ESCA/COCOSDA Third
      International Workshop on Speech Synthesis, Jenolan Caves
      Australia.
 
  [3] Miller, Corey (1998).  Pronunciation Modeling in Speech Synthesis.
      Doctoral dissertation, University of Pennsylvania.  Philadelphia,
      Pennsylvania.  Published as Technical Report 98-09, Institute for
      Research in Cognitive Science, University of Pennsylvania.
 
  [4] Miller, C., Karaali, O., Massey, N., (1998).  Learning
      Postlexical Variation in an Individual.  Paper presented at the
      Linguistics Society of America Annual Meeting, New York.
 
  [5] Miller, C., Massey, N., Karaali, O. (1998).  Exploring the Nature
      of Postlexical Processes.  Paper presented at the Penn Linguistics
      Colloquium.
 
  [6] Corrigan, G., Massey, N., & Karaali, O. (1997).  Generating Segment
      Durations in a Text-to-Speech System: A Hybrid Rule-Based/Neural
      Network Approach.  In Proceedings of Eurospeech '97.
      pp. 2675-2678.  Rhodes, Greece.
 
  [7] Karaali, O., Corrigan, G., Gerson, I., & Massey, N., (1997).
      Text-to-Speech Conversion with Neural Networks: A Recurrent TDNN
      Approach.  In Proceedings of Eurospeech '97.  pp. 561-564.  Rhodes,
      Greece.
 
  [8] Miller, C., Karaali, O., & Massey, N. (1997).  Variation and
      Synthetic Speech.  Paper presented at NWAVE 26, Quebec, Canada.
 
  [9] Gerson, I., Karaali, O., Corrigan, G., & Massey, N. (1996).  Neural
      Network Speech Synthesis.  Speech Science and Technology (SST-96).
      Australia.
 
 [10] Karaali, O., Corrigan, G., & Gerson, I. (1996).  Speech Synthesis
      with Neural Networks.  Invited paper, World Congress on Neural
      Networks (WCNN-96).  pp. 40-50.  San Diego.
 
 [11] Nusbaum, H., & Luks, T. (1995).  Comparative Evaluation of the
      Quality of Synthetic Speech Produced at Motorola.  Technical Report
      1, University of Chicago.  Chicago, Illinois.

From gary at cs.ucsd.edu  Tue Sep 15 19:29:50 1998
From: gary at cs.ucsd.edu (Gary Cottrell)
Date: Tue, 15 Sep 1998 16:29:50 -0700 (PDT)
Subject: Blended memory for faces: preprint
Message-ID: <199809152329.QAA12129@gremlin.ucsd.edu>

The following paper has been accepted for publication
in NIPS-11. It is available from my web page (given
below):

Dailey, Matthew N., Cottrell, Garrison W. and Busey, Thomas A. (1999)
Facial memory is kernel density estimation (almost).

To appear in Advances in Neural Information Processing Systems
11, MIT Press, Cambridge, MA.

We compare the ability of three exemplar models, each
using three different face stimulus representations, to
account for the probability a human subject responded
``old'' in an old/new facial memory experiment.  The
models are 1) the Generalized Context Model, 2) a
probabilistic sampling model, and 3) a novel model related
to kernel density estimation that explicitly encodes
stimulus distinctiveness.  The representations are 1)
positions of stimuli in MDS ``face space,'' 2) projections
of test faces onto the eigenfaces of the study set, and 3)
a representation based on response to a grid of Gabor
filters.  Of the 9 model/representation combinations, only
the distinctiveness model in MDS space predicts the
observed ``morph familiarity inversion'' effect, in which
subjects' false alarm rate for morphs between similar
parents is higher than their hit rate for the studied
parents of the morphs.  This evidence is consistent with
the hypothesis that human memory for faces is a kernel
density estimation task, with the caveat that distinctive
faces require larger kernels.


Gary Cottrell 619-534-6640 FAX: 619-534-7029 
Faculty Assistant Joy Gorback: 619-534-5948
Computer Science and Engineering 0114
IF USING FED EX INCLUDE THE FOLLOWING LINE:          "Only connect"
3101 Applied Physics and Math Building
University of California San Diego			-E.M. Forster
La Jolla, Ca. 92093-0114

Email: gary at cs.ucsd.edu or gcottrell at ucsd.edu
Home page: http://www-cse.ucsd.edu/~gary/


From backhaus at zedat.fu-berlin.de  Wed Sep 16 04:44:46 1998
From: backhaus at zedat.fu-berlin.de (PD Dr. Backhaus)
Date: Tue, 15 Sep 1998 23:44:46 -0900 (PDT)
Subject: Naples/Ischia Course: Final Call for Abstracts  
In-Reply-To: <Pine.SUN.3.91.980913111408.27536A-100000@berg> 
Message-ID: <Pine.PCW.3.91.980915233921.23310P-100000@Theorie1.biologie.fu-berlin.de>


See:    http://www.fu-berlin.de/backhaus/circul.html

Call for Abstracts (deadline: 19.9.98)

ISTITUTO ITALIANO PER GLI STUDI FILOSOFICI
STUDY PROGRAM ON "FROM NEURONAL CODING TO CONSCIOUSNESS"

INTERNATIONAL SCHOOL OF BIOCYBERNETICS
NEURONAL CODING OF PERCEPTUAL SYSTEMS

Isle of Ischia (Naples), Italy
October 12-17, 1998

OPENING CEREMONY AND INTRODUCTORY LECTURE:
Naples, morning of October 12, 1998

TOPICS:

[1] Vision: Neuronal Coding of Colour, Space, Motion, and Polarized Light
    Perception

[2] Hearing and Touch: Neuronal Coding of Auditory and
    Mechano Perception,
[3] Taste and Smell: Neuronal Coding of Chemical Perception,
[4] Neuronal Coding of Temperature, Pain, Electro, and Magneto Perception
[5] Neuronal Coding, Internal Representations, Qualia and Sensations
    (Consciousness)

ADVISORY BOARD:
A. Clark (USA), M. Kavaliers (C), L. Maffei (I), T. Radil, (CZ), U. Thurm,
(D), G. Tratteur (I), R. de Valois, (USA), R. Wehner, (CH), J. S. Werner, (USA),


SCHOOL DIRECTOR
Werner Backhaus
Freie Universit?t Berlin

For program and registration form see:

http://www.fu-berlin.de/backhaus/circul.html
   


W. Backhaus
homepage:
http://www.fu-berlin.de/backhaus
with a link to our book "Color Vision -
Perspectives from Different Disciplines, eds.
W. Backhaus, R. Kliegl, and J.S. Werner. De Gruyter,
Berlin - New York, 1998.

New Address:
W. Backhaus
Theoretical and Experimental Biology
Freie Universitaet Berlin
Villa, Koenigin-Luise-Str. 29
14195 Berlin

Tel./Fax.: +49-30-838 2692
e-mail: backhaus at zedat.fu-berlin.de




From ken at phy.ucsf.EDU  Wed Sep 16 05:22:21 1998
From: ken at phy.ucsf.EDU (Ken Miller)
Date: Wed, 16 Sep 1998 02:22:21 -0700 (PDT)
Subject: Paper Available: Model of V1 Development
Message-ID: <13823.33613.834512.602479@coltrane.ucsf.edu>

FTP-host:	ftp.keck.ucsf.edu
FTP-filename:   pub/ken/jn-erwin.ps.gz
URL:		ftp://ftp.keck.ucsf.edu/pub/ken/jn-erwin.ps.gz
Uncompressed version is available by omitting the '.gz'. 

The following paper is available by anonymous ftp.  It can also be
obtained from my web page:

http://www.keck.ucsf.edu/~ken
  (click on 'Publications';
   or alternatively, go directly to
   http://www.keck.ucsf.edu/~ken/miller.htm#references)

--------------------------------------------------------------------------

E. Erwin and K.D. Miller (1998).  ``Correlation-Based Development of
Ocularly-Matched Orientation and Ocular Dominance Maps: Determination
of Required Input Activities.''  In press, Journal of Neuroscience.

ABSTRACT:
We extend previous models for separate development of ocular dominance
and orientation selectivity in cortical layer 4 by exploring
conditions permitting combined organization of both properties.  These
conditions are expressed in terms of functions describing the degree
of correlation in the firing of two inputs from the lateral geniculate
nucleus (LGN), as a function of their retinotopic separation and their
``type'' (ON-center or OFF-center, left-eye or right-eye).

The development of ocular dominance requires that an input's
correlations with other inputs from the same eye be stronger than or
equal to its correlations with inputs of the opposite eye, and
strictly stronger at small retinotopic separations.  This must be true
after summing correlations with inputs of both center types.  The
development of orientation-selective simple cells requires that (1) an
input's correlations with other inputs of the same center type be
stronger than its correlations with inputs of the opposite center type
at small retinotopic separation; and (2) this relationship reverse at
larger retinotopic separations within an arbor radius (the radius over
which LGN cells can project to a common cortical point).  This must be
true after summing correlations with inputs serving both eyes.

For orientations to become matched in the two eyes, correlated
activity within the receptive fields must be maximized by specific
between-eye alignments of ON and OFF subregions.  Thus the
correlations between the eyes must differ depending on center type,
and this difference must vary with retinotopic separation within an
arbor radius.

These principles are satisfied by a wide class of correlation
functions.  Combined development of ocularly matched orientation maps
and ocular dominance maps can be achieved either simultaneously or
sequentially.  In the latter case, the model can produce a correlation
between the locations of orientation map singularities and local
ocular dominance peaks similar to that observed physiologically.

The model's main prediction is that the above correlations should
exist among inputs to cortical layer 4 simple cells before vision.  In
addition, mature simple cells are predicted to have certain
relationships between the locations of the ON and OFF subregions of
the left- and right-eyes' receptive fields.

--------------------------------------------------------------------

Ken Miller
 
        Kenneth D. Miller               telephone: (415) 476-8217
        Dept. of Physiology		fax: (415) 476-4929
        UCSF                            internet: ken at phy.ucsf.edu
        513 Parnassus			www: http://www.keck.ucsf.edu/~ken
        San Francisco, CA 94143-0444    


From wahba at stat.wisc.edu  Wed Sep 16 17:09:51 1998
From: wahba at stat.wisc.edu (Grace Wahba)
Date: Wed, 16 Sep 1998 16:09:51 -0500 (CDT)
Subject: Bias-Variance, GACV paper
Message-ID: <199809162109.QAA15339@hera.stat.wisc.edu>

The following paper has been accepted for oral
presentation at NIPS*98:  Available as University
of Wisconsin-Madison Statistics Dept TR997 in 

http://www.stat.wisc.edu/~wahba -> TRLIST
..................................................
The Bias-Variance Tradeoff and the Randomized GACV

Grace Wahba*, Xiwu Lin, Fangyu Gao,  Dong Xiang, 
Ronald Klein MD and Barbara Klein MD


We propose a new in-sample cross validation based 
method (randomized GACV) for choosing smoothing 
or bandwidth parameters that govern the bias-variance 
or fit-complexity tradeoff in `soft' classification. 
Soft classification refers to a learning procedure 
which estimates the probability
that an example with a given attribute vector is in 
class 1 {\it vs} class 0. The target for optimizing the 
the tradeoff is the Kullback-Liebler distance between 
the estimated probability distribution and the `true' 
probability distribution, representing knowledge 
of an infinite population. The method uses a 
randomized estimate of the trace of a Hessian and mimics 
cross validation at the cost of a single relearning with 
perturbed outcome data.

*corresponding author wahba at stat.wisc.edu 
.................................................


From jbower at bbb.caltech.edu  Wed Sep 16 19:36:19 1998
From: jbower at bbb.caltech.edu (James M. Bower)
Date: Wed, 16 Sep 1998 15:36:19 -0800
Subject: which came first the idea or the model?
Message-ID: <v04011785b225afa8cd49@[131.215.137.89]>

Not to pick nits, but,

In response to Richard Granger:

In his original email he stated:

>This (iterative finer grained representation of an odor) is an instance in
>which modeling of physiological activity in anatomical circuitry gave rise to
>an operation that was unexpected from behavioral studies,

However, the idea that cortical (olfactory) processing involved iterative
response refinement and specificity was actually a central thesis of the
monograph published by Lynch in 1986.  This monograph includes a discussion
of supporting behavioral data.

Reference:  G. Lynch, Synapses, Circuits, and the beginnings of memory. MIT
Press. 1986

It was clearly the objective of the subsequent model by Granger and Lynch
to see if this specific idea could be incorporated into a "cortical like"
structure.  Thus, as I indicated earlier, the model essentially served to
demonstrate a particular idea, which as Richard points out is still
controversial.

Second, the first reference I know for the Granger model was actually in
1988, two years before publication of the physiological studies claimed to
serve as its foundation:

Reference:  Granger et al., Partitioning of sensory data by a cortical
network, Neural Information Processing Systems. D. Anderson Ed. AIP, 1988}.

To quote previous email:


>"physiological induction and expression rules for
>synaptic long-term potentiation (LTP; Kanter & Haberly, '90; Jung et al.,
>'90)."

In fact, as I remember, the Granger model assumed that the only LTP was in
the synaptic connections made by the Lateral olfactory tract (LOT) not in
the association fiber system.  Kanter and Haberly (1990) actually showed
that the association fiber system is the major source of LTP in olfactory
cortex.

Thus, in summary, models can indeed generate novel ideas about brain
function.  And I agree completely with Richard that physiologically and
anatomically based models are much more likely to do so.  We ourselves have
built and "mined" many such models.  However, it is very important that a
clear distinction be made between models of this type, and those intended
to demonstrate a previously proposed functional idea using a mix of
convenient "neurobiological-like" structures and mechanisms.  There is
nothing wrong with such demonstration models, it is just not appropriate to
claim that they originated the ideas that they were actually designed to
demonstrate.


Jim Bower

           ***************************************
                       James M. Bower
                     Division of Biology
                     Mail code:  216-76
                           Caltech
                     Pasadena, CA 91125
                      (626) 395-6817
                      (626) 795-2088 FAX

      WWW addresses for:
        laboratory                 http://www.bbb.caltech.edu/bowerlab
        GENESIS:                   http://www.bbb.caltech.edu/GENESIS
        science education reform   http://www.caltech.edu/~capsi and
				   http://www.nas.edu/rise/examp81.htm

	J. Computational Neuroscience  http://www.bbb.caltech.edu/JCNS/
	Annual CNS meetings	http://www.bbb.caltech.edu/cns-meetings

From reza at bme.jhu.edu  Thu Sep 17 09:30:08 1998
From: reza at bme.jhu.edu (Reza Shadmehr)
Date: Thu, 17 Sep 1998 09:30:08 -0400 (EDT)
Subject: a paper on human adaptive control
Message-ID: <199809171330.JAA04456@bme.jhu.edu>


Dear Connectionists:

An abridged version of the following paper on human adaptive
control will be presented at NIPS this year.  It is available from  

http://www.bme.jhu.edu/~reza/nb_paper.pdf


   Computational Nature of Human Adaptive Control During
       Learning of Reaching Movements in Force Fields

                Nikhil Bhushan and Reza Shadmehr

Learning to make reaching movements in force fields was used as a
paradigm to explore the system architecture of the biological adaptive
controller.  We compared the performance of a number of candidate
control systems that acted on a model of the neuromuscular system of
the human arm and asked how well the dynamics of the candidate system
compared with the behavior of the biological controller.  We found
that control via a supra-spinal system that utilized an adaptive
inverse model resulted in dynamics that were similar to that observed
in our subjects, but lacked essential characteristics.  These
characteristics pointed to a different architecture where descending
commands were influenced by an adaptive forward model.  However, we
found that control via a forward model alone also resulted in dynamics
that did not match the behavior of the human arm.  We considered a
third control architecture where a forward model was used in
conjunction with an inverse model and found that the resulting
dynamics were remarkably similar to that observed in the experimental
data.  The essential property of this control architecture was that it
predicted a complex pattern of near-discontinuities in hand trajectory
in the novel force field. A nearly identical pattern was observed in
our subjects, suggesting that generation of descending motor commands
was likely through a control system architecture that included both
adaptive forward and inverse models.  We further demonstrate that as
subjects learned to make reaching movements, adaptation rates for the
forward and inverse models could be independently estimated and the
resulting changes in performance of subjects from movement to movement
could be accurately accounted for.  It appeared that in learning to
make reaching movements, adaptation of the forward model played a very
significant role in reducing the errors in performance.  Finally, we
found that after a period of consolidation, the rates of adaptation in
the models were significantly larger than those observed before the
memory had consolidated.  This suggested that consolidation of motor
memory may have coincided with freeing of certain computational
resources for subsequent learning.

From rafal at idsia.ch  Thu Sep 17 10:04:18 1998
From: rafal at idsia.ch (Rafal Salustowicz)
Date: Thu, 17 Sep 1998 16:04:18 +0200 (MET DST)
Subject: Prediction and Automatic Task Decomposition
Message-ID: <Pine.GSO.3.92.980917160123.4324C-100000@broccolo>


      LEARNING TO PREDICT THROUGH PROBABILISTIC INCREMENTAL
      PROGRAM  EVOLUTION  AND  AUTOMATIC TASK DECOMPOSITION

      Rafal Salustowicz                 Juergen Schmidhuber

                 Technical Report IDSIA-11-98

Analog gradient-based  recurrent  neural  nets  can learn complex
prediction tasks.  Most,   however,  tend to fail in case of long
minimal time lags between relevant training events.  On the other
hand, discrete methods such as search in a space of event-memori-
zing programs are not  necessarily  affected  at all by long time
lags:  we show that  discrete  "Probabilistic Incremental Program
Evolution" (PIPE) can solve several long time lag tasks that have
been successfully solved by only one analog method ("Long Short-
Term Memory" - LSTM).  In fact,  sometimes  PIPE even outperforms
LSTM. Existing discrete methods, however, cannot easily deal with
problems  whose solutions  exhibit comparatively high algorithmic
complexity.   We overcome this drawback by introducing filtering,
a novel,  general,  data-driven  divide-and-conquer technique for
automatic  task decomposition that is not limited to a particular
learning method. We compare PIPE plus filtering to various analog
recurrent net methods.

      ftp://ftp.idsia.ch/pub/rafal/TR-11-98-filter_pipe.ps.gz
      http://www.idsia.ch/~rafal/research.html

Rafal & Juergen, IDSIA, Switzerland                  www.idsia.ch


From Bill_Warren at Brown.edu  Thu Sep 17 08:57:51 1998
From: Bill_Warren at Brown.edu (Bill Warren)
Date: Thu, 17 Sep 1998 08:57:51 -0400 (EDT)
Subject: Please post -- thanks!
Message-ID: <v03010d02b2267ee6a8d6@[128.148.208.31]>


FACULTY POSITION IN VISUAL PERCEPTION, BROWN UNIVERSITY:  The Department of
Cognitive and Linguistic Sciences invites applications for a position in
visual perception beginning July 1, 1999.  An appointment will be made
either as a three-year renewable tenure-track Assistant Professor, or a
tenured Associate Professor. Applicants must have a strong experimental
research program combined with strong computational or theoretical
interests in vision, a broad teaching ability in cognitive science at both
the undergraduate and graduate levels, and an interest in contributing to
an interdisciplinary vision group spanning the departments of applied
mathematics, neuroscience, psychology, engineering, and computer science.
Applicants should have completed all Ph.D. requirements by no later than
July 1, 1999.  Women and minorities are especially encouraged to apply.
Send curriculum vitae, three letters of reference, reprints, and preprints
of publications, and a one-page statement of research interests to
Perception Search Committee, Dept. Of Cognitive and Linguistic Sciences,
Brown University, Providence, R.I. 02912, by January 1, 1999.

Brown University is an Equal Opportunity/Affirmative Action Employer.

                        -- Bill

                        William H. Warren, Professor
                        Dept. of Cognitive & Linguistic Sciences
                        Box 1978
                        Brown University
                        Providence, RI  02912
                        (401) 863-3980 ofc, 863-2255 FAX
                        Bill_Warren at brown.edu



From M.Usher at ukc.ac.uk  Thu Sep 17 11:24:55 1998
From: M.Usher at ukc.ac.uk (M.Usher@ukc.ac.uk)
Date: Thu, 17 Sep 1998 16:24:55 +0100
Subject: article on LATERAL INTERACTIONS
Message-ID: <199809171524.QAA17844@snipe.ukc.ac.uk>


The following article, to appear in SPATIAL VISION
(Special Issue on "Long Range Spatial Interactions in Vision"),
can now be accessed from:
http://ukc.ac.uk/psychology/people/usherm/ (at recent  publications)

The article addresses psychophysical data that indicate facilitatory
lateral interaction in visual processing, and presents a computational
model based on principles from neural information processing and signal
detection theory, to explain those interactions.

-Marius Usher

Department of Psychology
University of Kent

--------------------------------------------------------------

     MECHANISMS FOR SPATIAL INTEGRATION IN VISUAL DETECTION:
        A model based on lateral interactions

    Marius Usher, Yoram Bonneh, Dov Sagi & Michael Herrmann

                   Abstract

Studies of visual detection of multiple targets  show a weak
improvement of thresholds with the number of targets, which
corresponds to a fourth-root power law. We find this result to 
be inconsistent with probability summation models, and account
for it by a model of ``physiological'' integration that is based 
on excitatory lateral interactions in the visual cortex.
The model explains several phenomena which are confirmed by the
experimental data, such as the absence of spatial and temporal
uncertainty effects, temporal summation curves, and facilitation
by a pedestal in 2AFC tasks. The summation exponents are dependent 
on the strength of the lateral interactions, and on the distance 
and orientation relationship between the elements.


From cmerz at saanen.ics.uci.edu  Thu Sep 17 12:48:43 1998
From: cmerz at saanen.ics.uci.edu (Chris Merz)
Date: Thu, 17 Sep 1998 09:48:43 -0700
Subject: Articles on combining multiple models
Message-ID: <9809170948.aa29866@paris.ics.uci.edu>


I am announcing the availability of several articles related to
classification and regression by combining models. The first list
below contains the titles, url's and FEATURES of each article. The
second list contains the abstracts.

Please contact me at cmerz at ics.uci.edu with any questions.

Thanks,
Chris Merz

================== Titles, URL's and *** FEATURES *** =================

1. My dissertation:  

       "Classification and Regression by Combining Models"
       Merz, Christopher J. (1998) 
       http://www.ics.uci.edu/~cmerz/thesis.ps

       *** DESCRIBES TWO ROBUST METHODS FOR COMBINING LEARNED MODELS ***
       *** USING TECHNIQUES BASED ON SINGULAR VALUE DECOMPOSITION.   ***
       *** CONTAINS COMPREHENSIVE BACKGROUND AND SURVEY CHAPTERS.    ***

2. Preprints of two accepted Machine Learning Journal articles:

       "A Principal Components Approach to Combining Regression Estimates", 
       Merz, C. J., Pazzani, M. J. (1997) 
       To appear in the Special Issue of Machine Learning on Integrating 
       Multiple Learned Models. 
       http://www.ics.uci.edu/~cmerz/jr.html/mlj.pcr.ps

       *** SHOWS HOW PCA MAY BE USED TO SYSTEMATICALLY EXPLORE  ***
       *** WEIGHT SETS WITH VARYING DEGREES OF REGULARIZATION.  ***

       "Using Correspondence Analysis to Combine Classifiers", 
       Merz, C. J. (1997) 
       To appear in the Special Issue of Machine Learning on Integrating 
       Multiple Learned Models. 
       http://www.ics.uci.edu/~cmerz/jr.html/mlj.scann.ps

       *** SHOWS THAT THE SCANN METHOD COMBINES BOOSTED MODEL  ***
       *** SETS BETTER THAN BOOSTING DOES.                     ***

3. A bibtex file of the references in my dissertation survey:

       http://www.ics.uci.edu/~cmerz/bib.html/survey.bib

       *** COMPREHENSIVE BIBLIOGRAPHY - MANY ABSTRACTS INCLUDED ***

======================== Abstracts ==========================

1.      "Classification and Regression by Combining Models"

     Two novel methods for combining predictors are introduced in this
thesis; one for the task of regression, and the other for the task of
classification. The goal of combining the predictions of a set of
models is to form an improved predictor. This dissertation
demonstrates how a combining scheme can rely on the stability of the
consensus opinion and, at the same time, capitalize on the unique
contributions of each model.

     An empirical evaluation reveals that the new methods consistently
perform as well or better than existing combining schemes for a
variety of prediction problems. The success of these algorithms is
explained empirically and analytically by demonstrating how they
adhere to a set of theoretical and heuristic guidelines.

     A byproduct of the empirical investigation is the evidence that
existing combining methods fail to satisfy one or more of the
guidelines defined. The new combining approaches satisfy these
criteria by relying upon Singular Value Decomposition as a tool for
filtering out the redundancy and noise in the predictions of the learn
models, and for characterizing the areas of the example space where
each model is superior. The SVD-based representation used in the new
combining methods aids in avoiding sensitivity to correlated
predictions without discarding any learned models. Therefore, the
unique contributions of each model can still be discovered and
exploited. An added advantage of the combining algorithms derived in
this dissertation is that they are not limited to models generated by
a single algorithm; they may be applied to model sets generated by a
diverse collection of machine learning and statistical modeling
methods.

     The three main contributions of this dissertation are: 
        1. The introduction of two new combining methods capable of 
           robustly combining classification and regression estimates, and
           applicable to a broad range of model sets. 
        2. An in-depth analysis revealing how the new methods address the 
           specific problems encountered in combining multiple learned
           models. 
        3. A detailed account of existing combining methods and an 
           assessment of where they fall short in the criteria for 
           combining approaches. 

----------------

2. Preprints of two accepted Machine Learning Journal articles:

"A Principal Components Approach to Combining Regression Estimates"
Christopher J. Merz and Michael J. Pazzani

  Abstract

  The goal of combining the predictions of multiple learned 
  models is to form an improved estimator.  A combining strategy must be 
  able to robustly handle the inherent correlation, or 
  multicollinearity, of the learned models while identifying the unique 
  contributions of each.  A progression of existing approaches and their 
  limitations with respect to these two issues are discussed.  A new 
  approach, PCR*, based on principal components regression is proposed 
  to address these limitations.  An evaluation of the new approach on a 
  collection of domains reveals that 1) PCR* was the most robust 
  combining method, 2) correlation could be handled without eliminating 
  any of the learned models, and 3) the principal components of the 
  learned models provided a continuum of ``regularized'' weights from 
  which PCR* could choose.


"Using Correspondence Analysis to Combine Classifiers"
Christopher J. Merz

  Abstract

  Several effective methods have been developed recently for improving
  predictive performance by generating and combining multiple learned
  models.  The general approach is to create a set of learned models
  either by applying an algorithm repeatedly to different versions of
  the training data, or by applying different learning algorithms to the
  same data.  The predictions of the models are then combined according
  to a voting scheme.  This paper focuses on the task of combining the
  predictions of a set of learned models.  The method described uses the
  strategies of stacking and Correspondence Analysis to model the
  relationship between the learning examples and their classification by
  a collection of learned models.  A nearest neighbor method is then
  applied within the resulting representation to classify previously
  unseen examples.  The new algorithm does not perform worse than, and
  frequently performs significantly better than other combining
  techniques on a suite of data sets.

----------------

3. The bibtex file contains all of the references in my dissertation,
   including the survey. I've managed to paste in the abstracts of 
   many of the articles. I am willing to update this bibliography
   if any authors want to contribute references, abstracts and/or URL's.


From jbower at bbb.caltech.edu  Thu Sep 17 14:36:16 1998
From: jbower at bbb.caltech.edu (James M. Bower)
Date: Thu, 17 Sep 1998 10:36:16 -0800
Subject: plausibility
Message-ID: <v040117b6b227065373b9@[131.215.137.89]>

A non-text attachment was scrubbed...
Name: not available
Type: multipart/alternative
Size: 2160 bytes
Desc: not available
Url : https://mailman.srv.cs.cmu.edu/mailman/private/connectionists/attachments/00000000/aa92f5db/attachment.bin

From jagota at cse.ucsc.edu  Thu Sep 17 15:26:37 1998
From: jagota at cse.ucsc.edu (Arun Jagota)
Date: Thu, 17 Sep 1998 12:26:37 -0700
Subject: new survey-type publication
Message-ID: <199809171926.MAA15141@arapaho.cse.ucsc.edu>



New refereed e-publication (action editor: Risto Miikkulainen)

comprehensive area bibliography with thematic and keyword indices

Samuel Kaski, Jari Kangas, Teuvo Kohonen, 
Bibliography of Self-Organizing Map (SOM) Papers: 1981--1997, 
Neural Computing Surveys, 1, 102--350, 1998, 3343 references.
http://www.icsi.berkeley.edu/~jagota/NCS

Abstract:

The Self-Organizing Map (SOM) algorithm has attracted an ever
increasing amount of interest among researchers and practitioners in
a wide variety of fields. The SOM and a variant of it, the LVQ, have
been analyzed extensively, a number of variants of them have been
developed and, perhaps most notably, they have been applied
extensively within fields ranging from engineering sciences to
medicine, biology, and economics. We have collected a comprehensive
list of 3343 scientific papers that use the algorithms, have
benefited from them, or contain analyses of them. The list is
intended to serve as a source for literature surveys. We have
provided both a thematic and a keyword index to help finding
articles of interest.


From joe at cs.caltech.edu  Thu Sep 17 17:28:59 1998
From: joe at cs.caltech.edu (Joe Sill)
Date: 17 Sep 1998 21:28:59 GMT
Subject: Special issue on VC dimension
Message-ID: <6truur$h4m@gap.cco.caltech.edu>

Machine learning theorists may be interested in a recent issue
of the journal Discrete Applied Mathematics (Vol 86, Number 1, 
August 18, 1998). This special issue, edited by John Shawe-Taylor, is 
devoted entirely to the VC dimension.

Contents:

"Combinatorial variability of Vapnik-Chervonenkis classes with 
applications to sample compression schemes"
S. Ben-David and A. Litman

"A graph-theoretic generalization of the Sauer-Shelah lemma"
N. Cesa-Bianchi and D. Haussler

"Scale-sensitive dimensions and skeleton estimates for classification"
M. Horvath and G. Lugosi

"Vapnik-Chervonenkis dimension of recurrent neural networks"
P. Koiran and E.D. Sontag

"The degree of approximation of sets in euclidean space using sets
with bounded Vapnik-Chervonenkis dimension"
V. Maiorov and J. Ratsaby

"The capacity of monotonic functions"
J. Sill

"Fluctuation bounds for sock-sorting and other stochastic processes"
D. Steinsaltz

From granger at uci.edu  Thu Sep 17 20:20:39 1998
From: granger at uci.edu (Richard Granger)
Date: Thu, 17 Sep 1998 17:20:39 -0700
Subject: Unexpected hypotheses arising from brain circuit simulation
Message-ID: <v02140b00b226dfa12849@[128.200.120.16]>


  Jim writes:
> the idea that cortical (olfactory) processing involved iterative
> response refinement and specificity was actually a central thesis of the
> monograph published by Lynch in 1986.
>
>Reference:  G. Lynch, Synapses, Circuits, and the beginnings of memory. MIT
>Press. 1986

If that's so, then its author doesn't know it.  The monograph neither
contains nor presages the hypothesis that appears in our 1990 Science
paper.  (Perhaps this is being confused with operations of excitatory
associational feedback fibers, which Lynch in 1986 hypothesized might cycle
repeatedly in response to a single input, rapidly building a
"representation" of an odor.  Such an operation is of course utterly
unrelated to the finding being discussed: there's no mention of bulb, no
mention of theta cycles, no mention of inhibitory feedback, no hierarchy.
The author states that the hypothesis that appears in the Science paper was
not even conceived of at the time of the 1986 monograph.)

The 1986 monograph was an early and fruitful step in the field of modeling
of real biological systems.  It is replete with attempts at identifying
computational concomitants of a range of biological phenomena, and with
compiling and integrating data related to research on LTP, the olfactory
system, and the hippocampus.  In particular, a central phenomenon is the
(4-8 Hz) theta rhythm, which:
  i) entrains cells throughout the olfactory system, hippocampus and much
of neocortex exclusively during learning and exploration (not during sleep,
or in a home cage, or passive restraint, etc.), (Macrides, 1975; Macrides
et al., 1982; Komisaruk, 1970; Otto et al., 1991; Kauer 1991), and
  ii) has been shown to be the optimal stimulation pattern for induction of
LTP (Larson et al., 1986; Diamond et al., 1987) due to an endogenously
occurring time-dependent gaba-b inactivation of gaba terminals (Mott &
Lewis, 1991).

For historical interest, after the monograph, what one finds is a
succession of papers by us, all on studies of this system (two papers in
1988 and four in '89), all attempting to identify various aspects of
function from the structure and operation of the system.  Our studies
focused in particular on the theta rhythm (such as the marvelous fact that
in small mammals the rhythm literally drives overt behavior (sniffing,
moving whiskers, etc., all at 5 Hz) during exploration).  Those papers make
the historical case glaringly:  they contain some interesting early
findings on the computational advantages of the synchrony itself provided
by theta, and on local lateral inhibition, and on refinement of recognition
with episodes of training and incremental steps of LTP, but nothing at all
on hierarchical clustering (the topic of the 1990 Science paper).   In
particular, the 1988 papers made the point that the initial sniff clustered
inputs (as was expected from work by other researchers), but nothing on
later sniffs subclustering.

Then we took on the puzzling observation that the cortical model was
producing different outputs over time, as it sampled a single input.  We
showed that not only do the initial cortical responses tend to empirically
cluster inputs, as mentioned, but also that later cortical responses,
paradoxically, tend to differentiate those same inputs.  We observed that
the system's inhibitory feedback (Price, 1973), and the long-lasting nature
of the resulting bulb granule cell IPSPs (Nicoll, 1969; Mori, 1987) tended
to selectively remove the same portions of the inputs that were giving rise
to the initial responses, and that therefore the system might not simply be
first clustering and then differentiating, but actually clustering,
sub-clustering, sub-sub-clustering, etc., over iterative cycles of the
theta rhythm.  This constituted a new hypothesis; not successive episodes
of learning; not steps of LTP; not cycling associational pathways; but
successive iterative feedforward excitation and feedback inhibition, giving
rise to a sequence of distinct different outputs on successive theta
cycles, traversing a hierarchical tree from general (clusters) to specific
(subclusters).  We wrote this up in 1989 and submitted it to Science, where
it was accepted and appeared in 1990.  It turned out that a relationship
could be shown between this circuit behavior and a nonstandard method of
hierarchical clustering; and it further turned out that this method was
unusually efficient with respect to time and space complexity.

Thus a puzzling operation that arose from interactions among a large number
of features in a biological simulation, turned out to yield an unusual
computational function, and did so in an unusually efficient manner.  It
was this finding that was surprising (to us, and to the reviewers and
editors of Science).

>It was clearly the objective of the subsequent model by Granger and Lynch
>to see if this specific idea could be incorporated into a "cortical like"
>structure.

Clearly not, given the above.  It was clearly our objective to explore the
model for its behaviors and functions, and to attempt to identify and
characterize the emergent properties of its many, many parts.


Our hypothesis is that a function of the olfactory system is the iterative
decomposition of inputs over successive rhythmic cycles of operation:
specifically, that sequential outputs of the cortex (every 200 msec) give
different information, corresponding to successive hierarchical clusters
and subclusters.  This is a novel hypothesis (or was, in 1990), and has
been much cited and studied for its behavioral, neurobiological,
psychological, and computational consequences.  It was derived directly
from, and incorporates, the detailed characteristics of the biological
system itself: theta, LTP induction and expression rules, sparse
connectivity, feedforward excitation, LOT and associational pathways,
feedback inhibition, mitral, tufted and granule cells, local lateral
inhibition, differential time courses of EPSPs and IPSPs in different cell
types, axonal arborization radius of inhibitory interneurons, etc.  We'd be
glad simply to take the credit for having come up with it by inspection, if
that were true.  The fact that we actually came up with it only after
extensive construction and observation of anatomically and physiologically
realistic models is a notable methodological point.  That this new idea has
given rise to a fruitful series of subsequent behavioral, physiological and
theoretical studies (in our labs and others) is a notable consequence.
That the hypothesis will undoubtedly turn out to be wrong in many ways is
the point of ongoing scientific inquiry.  We should continue to doubt, and
to study, and to counter-hypothesize, and to experiment.

-Rick Granger



[Footnote:

>In fact, as I remember, the Granger model assumed that the only LTP was in
>the synaptic connections made by the Lateral olfactory tract (LOT) not in
>the association fiber system.  Kanter and Haberly (1990) actually showed
>that the association fiber system is the major source of LTP in olfactory
>cortex.

No, actually, our models have LTP both in the LOT and the association fiber
system.  We have even studied the individual effects of these two pathways,
and their possible differential contributions to the function of the
overall system.]




From M.Usher at ukc.ac.uk  Fri Sep 18 09:40:58 1998
From: M.Usher at ukc.ac.uk (M.Usher@ukc.ac.uk)
Date: Fri, 18 Sep 1998 14:40:58 +0100
Subject: web-site correction
Message-ID: <199809181340.OAA05419@snipe.ukc.ac.uk>

There was an error in the web-site address I posted yesterday for
our article, to appear in SPATIAL VISION
(Special Issue on "Long Range Spatial Interactions in Vision"),

The correct address is:

http://www.ukc.ac.uk/psychology/people/usherm/public.html

I appologize for the error

-Marius Usher
 
Department of Psychology
University of Kent
 
--------------------------------------------------------------
 
     MECHANISMS FOR SPATIAL INTEGRATION IN VISUAL DETECTION:
        A model based on lateral interactions
 
    Marius Usher, Yoram Bonneh, Dov Sagi & Michael Herrmann
 
                   Abstract
 
Studies of visual detection of multiple targets  show a weak
improvement of thresholds with the number of targets, which
corresponds to a fourth-root power law. We find this result to
be inconsistent with probability summation models, and account
for it by a model of ``physiological'' integration that is based
on excitatory lateral interactions in the visual cortex.
The model explains several phenomena which are confirmed by the
experimental data, such as the absence of spatial and temporal
uncertainty effects, temporal summation curves, and facilitation
by a pedestal in 2AFC tasks. The summation exponents are dependent
on the strength of the lateral interactions, and on the distance
and orientation relationship between the elements.
 

From xw3f at avery.med.virginia.edu  Fri Sep 18 10:39:29 1998
From: xw3f at avery.med.virginia.edu (Xiangbao Wu)
Date: Fri, 18 Sep 1998 10:39:29 -0400
Subject: Postdoctoral Position Available
Message-ID: <199809181439.KAA213090@avery.med.Virginia.EDU>

            COMPUTATIONAL NEUROSCIENCE POSTDOCTORAL RESEARCH ASSOCIATE 
		         UNIVERSITY OF VIRGINIA
		       CHARLOTTESVILLE, VIRGINIA

A postdoctoral research associate position is available in the
laboratory of Dr. William B Levy.

The applicant will work on a research project in at least one of
four areas: (1) quantitative neural network theory of hippocampal
function using network simulations; (2) computational analysis
of neural network solutions to cognitive tasks; (3) effects of activity
fluctuations and noise to hippocampal function by computational simulations;
(4) a theory of hippocampal neocortical interactions.

Applicants should have a strong background in quantitative research and some
basic knowledge of neuroscience or cognitive science.  Candidates will also
be judged on their relevant research experience and communication skills.

The starting date is flexible. The position is available for 1-2 years
depending on accomplishment.

Please send a CV, a letter describing research interests and background,
and three (3) references by post or email to:

William B Levy
Department of Neurosurgery
Health Sciences Center Box 420
University of Virginia
Charlottesville, Va 22908
Email: wbl at virginia.edu

From jbower at bbb.caltech.edu  Thu Sep 17 14:36:16 1998
From: jbower at bbb.caltech.edu (James M. Bower)
Date: Thu, 17 Sep 1998 10:36:16 -0800
Subject: plausibility
Message-ID: <mailman.956.1149540385.24850.connectionists@cs.cmu.edu>

In response to my recent post, I received the following note:


>> actually designed to demonstrate.
>                          ^
> You surely meant to say test.
>


In fact I meant "demonstrate", and this is a very important distinction and
issue in brain-like modeling.  It is my view that the majority of the
models generated in this field to date (especially those of the NN type)
are actually demonstration type models, and not in any real sense "tests".
In order to be a test, there must be some mechanism for formally evaluating
the plausibility of a particular model, given the available neurobiological
data.  We have recently published a paper suggesting one (some would say
the only) formal approach to this problem:

Baldi, P.,  Vanier, M.C., and Bower, J.M.  (1998)  On the use of Bayesian
methods for evaluating compartment neural models.  J. Computational
Neurosci. 5: 285-314.

However, at present there are no accepted standards for such an evaluation
(in fact there is almost no discussion of this issue).  Instead, far too
much modeling involves twiddling the right knobs to get the functional
results you want.  Those few experimentalists interested in modeling
usually evaluate a models plausibility based mostly on intuition.

It is for this reason that the question of prior functional assumptions is
so important, and why I continue to try to draw a strong distinction
between modeling based first on anatomy and physiology and efforts intended
to demonstrate the plausibility of a particular preconceived idea (c.f.
Bower, J.M. (1995)  Reverse engineering the nervous system: an in vito, in
vitro, and in computo  approach to understanding the mammalian olfactory
system.  In: An Introduction to Neural and Electronic Networks, Second
Edition. S. Zornetzer, J. Davis, and C. Lau, editors.  Academic Press.  pp.
3-28.).  If the functional idea truly comes about as a result of the
modeling, and not vice versa, then it is more likely (although still far
from certain) that the revealed mechanisms have something to do with the
real brain.  Of course, this is the same reason that some modelers try to
blur this distinction.


Jim Bower

================ Message 2 of 4 ================


From jbower at bbb.caltech.edu  Fri Sep 18 15:55:56 1998
From: jbower at bbb.caltech.edu (James M. Bower)
Date: Fri, 18 Sep 1998 11:55:56 -0800
Subject: iterative processing
Message-ID: <mailman.957.1149540385.24850.connectionists@cs.cmu.edu>

Read carefully, Richard's response to my previous email indicates pretty
clearly the degree to which prior thinking about how the olfactory system
works influenced the development of the olfactory model.  This is the sole
point I have been trying to make.

In particular he states that:

"The 1986 monograph was an early and fruitful step in the field of modeling
of real biological systems."

Of course there was no model in the monograph -- the discussion involved
speculations based on biological data and certain assumptions concerning
olfactory processing (e.g. iterative refinement of olfactory response).
Richard's long history makes pretty clear that it was those assumptions
that drove the subsequent modeling.  This, I presume, is what Richard meant
by a "fruitful step".  The "fruit" in this case, being the model.

Richard's recounting of the history also makes it clear that the original
claim that the model was based on a wide range of biological data,
including data published in 1990 and after is somewhat difficult to
reconcile with the statement that:

"two papers (were written) in 1988 and four in '89, all attempting to
identify various aspects of function from the structure and operation of
the system."


Finally, there are many technical and biological issues that could be
raised concerning the assumptions and conclusions of this particular
modeling effort (The location of LTP, or evidence that rats can apparently
recognize odors on a single sniff (theta cycle), or the issue of whether
olfactory perception space is really heirarchically clustered), however, it
was never my intent to argue about the model itself.  Instead, I was trying
to make the point that one has to be very careful to distinguish between
models that assume a particular function, and then try to identify a
biologically plausible structure that might provide it, and models that
start with structure, and try to infer function.   It may be that this
distinction is blurry to some -- however, if one has done the later, the
distinction is obvious and important.

Jim Bower


================ Message 3 of 4 ================


From jbower at bbb.caltech.edu  Fri Sep 18 17:08:19 1998
From: jbower at bbb.caltech.edu (James M. Bower)
Date: Fri, 18 Sep 1998 13:08:19 -0800
Subject: cerebellum
Message-ID: <mailman.958.1149540385.24850.connectionists@cs.cmu.edu>

Just catching up on the NN/Brain discussions.

And another cautionary note not unrelated to my previous emails:  There is
growing evidence that the cerebellum may not be a "motor control system" in
the classical sense.  If correct then models that were constructed under
this assumption will need to be revisited.


Jim Bower


================ Message 4 of 4 ================


From jbower at bbb.caltech.edu  Fri Sep 18 17:32:48 1998
From: jbower at bbb.caltech.edu (James M. Bower)
Date: Fri, 18 Sep 1998 13:32:48 -0800
Subject: Chickens and eggs again -- the sequence matters
Message-ID: <mailman.959.1149540385.24850.connectionists@cs.cmu.edu>

Sorry everyone for being so tight about this.

DeLiang Wang wrote:


>His theory and prediction led to the two first
>confirmative reports by Echorn et al. (1988) and Gray et
>al. (1989).

However, in fact, at least Charlie Gray did this work because of his
interest in oscillations arising out of his work in the olfactory system
with Walter Freeman.  I believe that Echorn's group also did the work
without knowing about the theory.



>Since then numerous experiments have been conducted that
>confirm the theory (not without some controversy),

this is vastly too strong a statement.  There are major issues outstanding
about the basic idea and its evidence.  While it is true that Christof's
theory has generated a lot of interest and discussion, it remains to be
seen whether, in the long run, that discussion was useful in figuring out
the significance of cortical oscillations.  It may have been a distraction.



>But one would not dispute that his neural network theory has
>generated major impact on neuroscience.
>


This can not be disputed.  One can wish or not that it was otherwise.


Jim Bower

From hagai at phy.ucsf.EDU  Sun Sep 20 11:17:24 1998
From: hagai at phy.ucsf.EDU (Hagai Attias)
Date: Sun, 20 Sep 98 08:17:24 -0700
Subject: Paper available -- Independent Factor Analysis
Message-ID: <199809201517.IAA08579@phy.ucsf.EDU>


A new paper on a simple graphical model approach to the problem of 
blind separation of independent sources, using exact and variational EM, 
is available at

   http://keck.ucsf.edu/~hagai/papers.html


- ---------------------------------------------------


	  INDEPENDENT FACTOR ANALYSIS

	       Hagai Attias, UCSF

         (Neural Computation, in press)


We introduce the independent factor analysis (IFA) method for recovering 
independent hidden sources from their observed mixtures. IFA generalizes 
and unifies ordinary factor analysis (FA), principal component analysis (PCA), 
and independent component analysis (ICA), and can handle not only square
noiseless mixing, but also the general case where the number of mixtures
differs from the number of sources and the data are noisy. 

IFA is a two-step procedure. In the first step, the source densities, mixing 
matrix and noise covariance are estimated from the observed data by maximum 
likelihood. For this purpose we present an expectation-maximization (EM) 
algorithm, which performs unsupervised learning of an associated probabilistic 
model of the mixing situation. Each source in our model is described by a 
mixture of Gaussians, thus all the probabilistic calculations can be performed 
analytically. In the second step, the sources are reconstructed from the 
observed data by an optimal non-linear estimator. 

A variational approximation of this algorithm is derived for cases with a large 
number of sources, where the exact algorithm becomes intractable.

Our IFA algorithm reduces to the one for ordinary FA when the sources become 
Gaussian, and to an EM algorithm for PCA in the zero-noise limit. We derive an 
additional EM algorithm specifically for noiseless IFA. This algorithm is shown 
to be superior to ICA since it can learn arbitrary source densities from the 
data. Beyond blind separation, IFA can be used for modeling multi-dimensional 
data by a highly constrained mixture of Gaussians, and as a tool for non-linear 
signal encoding.

From tgd at iiia.csic.es  Mon Sep 21 12:01:51 1998
From: tgd at iiia.csic.es (Thomas Dietterich)
Date: Mon, 21 Sep 1998 18:01:51 +0200 (MET DST)
Subject: A Computing Research Repository
Message-ID: <199809211601.SAA25899@sinera.iiia.csic.es>

Annoucing A Computing Research Repository

Researchers have made their papers available by putting them on personal
web pages, departmental pages, and on various ad hoc sites known only
to cognoscenti.  Until now, there has not been a single repository to 
which researchers from the whole field of computing can submit reports.  

This is about to change.  Through a partnership of ACM, the Los Alamos 
e-Print archive, and NCSTRL (Networked Computer Science Technical
Reference Library), an online Computing Research Repository (CoRR) is 
being established.  The Repository has been integrated into the 
collection of over 20,000 computer science research reports and other 
material available through NCSTRL (http://www.ncstrl.org) and will be 
linked with the ACM Digital Library.  Most importantly, the Repository 
will be available to all members of the community at no charge.

We encourage you to start using the Repository right away.  For more details,
see http://xxx.lanl.gov/archive/cs/intro.html.  That site provides 
information on how to submit documents, browse, search, and subscribe to 
get notification of new articles of interest.  Please spread the word 
among your colleagues and students.  CoRR will only gain in value as
more researchers use it.  

See http://www.acm.org/repository for a more detailed description of CoRR.    

From aminai at ececs.uc.edu  Mon Sep 21 16:20:47 1998
From: aminai at ececs.uc.edu (Ali Minai)
Date: Mon, 21 Sep 1998 16:20:47 -0400 (EDT)
Subject: ICCS'98 Focus Session
Message-ID: <199809212020.QAA19332@holmes.ececs.uc.edu>


                               ANNOUNCEMENT
                               ------------
   

      Focus Session on Neural Computation, Cognition, and Complex Systems
      -------------------------------------------------------------------

          Second International Conference on Complex Systems (ICCS'98)
                               Nashua, NH
                           October 25-30, 1998
                 

A focus session on neural computation, cognition, and complex systems will
be held on Oct. 29, 7:00 - 10:00 p.m., as part of the Second International
Conference on Complex Systems. This invited session brings together a
group of researchers who are working at the active interface of complex
systems and neuroscience, and who have thought very deeply about these issues.
The session will be chaired by Prof. Walter Freeman, University of California,
Berkeley, who will give a keynote talk on the afternoon of Wednesday,
October 28. The speakers include:

Walter Freeman (University of California, Berkeley) - CHAIR
Steve Bressler (Center for Complex Systems, Florida Atlantic Univ.)
Michael Hasselmo (Boston University)
Jorge Jose (Northeastern University)
John Lisman (Volen Center for Complex Systems - Brandeis University)
Randy McIntosh (Rotman Research Institute, Toronto)
John Symons (Boston University)

The session will include presentations and a panel discussion. The
focus of discussion will be:


    How, and to what extent, can the study of spatio-temporal
         dynamics in the brain help explain cognition?


with the following specific issues:


1. What sort of dynamic processes and structures in the brain underlie
    the processes of cognition? At what scales do they occur, and are
    they subject to global principles of organization such as
    cooperation, competition, synchronization, etc. across scales?

2. What types of experiments do we need to probe these dynamic processes
    and construct useful neurobiologically grounded theories of cognitive
    function?

3. Is the theoretical/mathematical framework in which we model neural
    systems (e.g., compartmental models, local learning rules, patterns
    of activity, spike train statistics, etc.) sufficient to capture the
    processes of interest?

4. Are the emerging sciences of complexity, with ideas like self-organization,
    self-similarity, chaos, and scaling, likely to provide a useful paradigm
    for relating biology to cognition? How successful have attempts to
    apply these ideas been so far?

5. Is the information processing metaphor for the brain still a viable one,
    or should it be expanded/modified in some way?



The International Conference on Complex Systems is organized by the
New England Complex Systems Institute (NECSI), and is an important effort
to establish complex systems as a research area in its own right. The first
conference last year (also in Nashua) brought together several hundred people
and produced very animated discussions. In addition to the speakers at the
neural computation session, this year's conference speakers include Stephen
Kosslyn, Scott Kelso, Per Bak, Doyne Farmer, Phillip Anderson, Matt Wilson
and many others whose ideas speak to issues of interest to neural systems
researchers.

I will post more information on this as it becomes available. Interested
readers should check out the ICCS'98 website at http://necsi.org/html/iccs2.html
or send mail to iccs at necsi.org for information on registration, etc.


Ali A. Minai
Complex Adaptive Systems Laboratory
Department of Electrical
   & Computer Engineering
               and Computer Science
University of Cincinnati
Cincinnati, OH 45221-0030

Phone: (513) 556-4783
Fax:   (513) 556-7326
Email: Ali.Minai at uc.edu

Internet: http://www.ececs.uc.edu/~aminai/

From jagota at cse.ucsc.edu  Mon Sep 21 19:57:55 1998
From: jagota at cse.ucsc.edu (Arun Jagota)
Date: Mon, 21 Sep 1998 16:57:55 -0700
Subject: Call for volunteers: NIPS*98
Message-ID: <199809212357.QAA28989@arapaho.cse.ucsc.edu>


		NIPS*98 Call For Volunteers
		===========================

NIPS*98 needs student volunteers to assist onsite at the tutorials, 
main conference, and workshops. In exchange for approximately 9 hours 
of work a volunteer will receive free registration to the component 
of NIPS (tutorials, conference, or workshops) that (s)he volunteers 
time towards. Volunteers can look forward to this being an educational 
and fun experience!

To apply check out the procedure at

	http://www.cse.ucsc.edu/~jagota/

	
For questions (but first try the web site) contact me by e-mail,

Arun Jagota 			jagota at cse.ucsc.edu
Local arrangements, NIPS*98

From hochreit at informatik.tu-muenchen.de  Mon Sep 21 11:06:47 1998
From: hochreit at informatik.tu-muenchen.de (Josef Hochreiter)
Date: Mon, 21 Sep 1998 17:06:47 +0200
Subject: paper on ICA and LOCOCODE
Message-ID: <98Sep21.170648+0200_met_dst.7646-25738+67@papa.informatik.tu-muenchen.de>



                 Feature extraction through LOCOCODE                

   Sepp Hochreiter,  TUM                Juergen Schmidhuber, IDSIA       

   Neural Computation, in press   (28 pages, 0.7MB, 5MB gunzipped) 

LOw-COmplexity COding and DEcoding  (LOCOCODE)  is a novel approach to 
sensory coding and unsupervised learning.   Unlike previous methods it
explicitly takes into account the  information-theoretic complexity of 
the code generator:      it computes lococodes that convey information 
about the input data and can be computed and decoded by low-complexity 
mappings.  We implement LOCOCODE by training autoassociators with Flat
Minimum Search (Neural Computation 9(1):1-42, 1997),  a general method
for discovering low-complexity neural nets.     It turns out that this
approach can unmix an  unknown number of  independent  data sources by 
extracting a minimal number of  low-complexity features  necessary for
representing the data.   Experiments show:  unlike codes obtained with
standard autoencoders, lococodes are based on feature detectors, never
unstructured, usually sparse, sometimes factorial or local  (depending 
on statistical properties of the data).       Although LOCOCODE is not 
explicitly designed to enforce sparse or factorial codes,  it extracts 
optimal codes for  difficult  versions of the  bars benchmark problem, 
whereas ICA and PCA do not.        It produces familiar,  biologically 
plausible  feature  detectors  when applied to real world images,  and 
codes with fewer bits per pixel than ICA and PCA.   Unlike ICA it does 
not need to know the number of independent sources.  As a preprocessor 
for a  vowel  recognition  benchmark  problem it  sets  the  stage for 
excellent classification performance.            Our results reveil an 
interesting,  previously  ignored  connection  between  two  important 
fields: regularizer research, and ICA-related research.       They may 
represent  a  first  step  towards  unification  of regularization and 
unsupervised learning.

   ftp://ftp.idsia.ch/pub/juergen/lococode.ps.gz
   ftp://flop.informatik.tu-muenchen.de/pub/articles-etc/
   hochreiter.lococode.ps.gz

   http://www7.informatik.tu-muenchen.de/~hochreit/pub.html
   http://www.idsia.ch/~juergen/onlinepub.html

   Conference spin-offs:
1. Low-complexity coding and decoding. In K. M. Wong, I. King, D. 
   Yeung, eds., Proc. TANC'97, 297-306, Springer, 1997.
2. Unsupervised coding with LOCOCODE. In W. Gerstner, A. Germond, M. 
   Hasler, J.-D.  Nicoud, eds., ICANN'97, 655-660, Springer, 1997.
3. LOCOCODE versus PCA and ICA. In L. Niklasson, M. Boden, T. Ziemke,
   eds., ICANN'98, 669-674, Springer, 1998.
4. Source separation as a by-product of regularization.  
   To be presented at NIPS'98, 1998.

Sepp & Juergen




From sylee at eekaist.kaist.ac.kr  Tue Sep 22 21:13:29 1998
From: sylee at eekaist.kaist.ac.kr (sylee)
Date: Wed, 23 Sep 1998 10:13:29 +0900
Subject: Several Post Doc Positions in Korea
Message-ID: <199809230133.KAA08525@eekaist.kaist.ac.kr>

                   Immediate Opening:
           Post Doc Positions Available
     at Brain Science Research Center

Korean Ministry of Science and Technology just initiated a 10 year national
research program on Braintech'21, which consists of neuroscience,
cognitive science, and artificial neural networks. The Brain Science
Research Center (BSRC) is selected as the main research organization
for the Braintech'21. Although the BSRC is located at Korea Advanced
Institute of
Science and Technology (KAIST) at Taejon, Korea (South), it supports the
whole
Korean brain research community. More than 70 professors and researchers
from all over the Korea are affiliated to the BSRC.

Several Post Doc positions are available starting from November 1998 or
later.
The research areas cover all aspects of neuroscience, cognitive science,
and artificial neural networks. Currently we have 3 inter-disciplinary
projects and 6 basic-research projects.

o Interdicsiplinary Projects
   - artifical vision and speech recognition system
     (from biological models to hardware implementations, i.e.
      artificial retina and artificial cochlea chips)
   - inferrence system
     (from biology to neural network models)
   - EEG classification system
o Basic-research projects
   - Neuroscience: molecular level
   - Neuroscience: system level
   - Cognitive science
   - Artificial neural network models
   - Neuro-chip hardware impementation
   - Neural network applications (control and telecommunications)

Applicants should have a strong background in quantitative research and
some
basic knowledge of neuroscience, cognitive science, artificial neural
networks,
or VLSI design.  Interests in multidisciplinary researches are
advantageous.

In colllaobation with many academic and research organizations in Korea,
the BSRC will provide excellent research environments. Researchers from
diversified research backgrounds join together to stimulate each other
and come up with excellent multidisciplinary researches.

The starting date is flexible. The position is available for 1-3 years
depending on accomplishment.

Applicants should send a CV, the names and e-mail addresses of three
references, 
and a summary of research interests and experience to:

Prof. Soo-Young Lee
Director, Brain Science Research Center
3rd Floor, LG Semicon Hall
KAIST
373-1 Kusong-dong, Yusong-gu
Taejon 305-701
Korea (South)
Tel: +82-42-869-3431
Fax: +82-42-869-8570
E-mail: sylee at ee.kaist.ac.kr




From friess at acse.shef.ac.uk  Wed Sep 23 08:09:01 1998
From: friess at acse.shef.ac.uk (Thomas Friess)
Date: Wed, 23 Sep 1998 13:09:01 +0100 (BST)
Subject: new RR on support vector neural networks 
Message-ID: <Pine.SOL.3.94.980923125756.6731A-100000@puffin>



a new research report on support vector neural networks is
available at:
http://www.brunner-edv.com/friess/index.html


Support Vector Neural Networks: The Kernel Adatron with Bias 
and Soft Margin


Abstract: The kernel Adatron with bias and soft margin (KAb) is a new 
neural network alternative to support vector (SV) machines. It can learn
large-margin decision functions in kernel feature spaces in an iterative
"on-line" fashion which are identical to support vector machines.
Support vector learning is batch learning and is strongly based on solving
constrained quadratic programming problems which are nontrivial to
implement and may be subject to stability problems.
The kernel Adatron algorithm (KA), which has been developed as a joint
project, has been introduced recently. So far it has been assumed that the
bias parameter of the plane in feature space is always zero, and that all
patterns can be correctly classified by the learning machine. These
assumptions cannot always be made.
At first perceptrons in the data dependent representation, support vector
machines, and the kernel Adatron will be reviewd. Then the kernel Adatron
with bias and soft margin will be introduced.
The algorithm is conceptually simple and implements an iterative form of
unconstrained quadratic programming.
Experimental results using benchmarks and real data are provided which
allow to compare the performance and speed of kernel Adatrons and
SV machines.







From cns-cas at cns.bu.edu  Wed Sep 23 11:08:43 1998
From: cns-cas at cns.bu.edu (Boston University - Cognitive and Neural Systems)
Date: Wed, 23 Sep 1998 11:08:43 -0400
Subject: Graduate Training in CNS at Boston University
Message-ID: <199809231508.LAA20573@mattapan.bu.edu>

*******************************************************************
GRADUATE TRAINING IN THE
DEPARTMENT OF COGNITIVE AND NEURAL SYSTEMS (CNS)
AT BOSTON UNIVERSITY
*******************************************************************

The Boston University Department of Cognitive and Neural Systems
offers comprehensive graduate training in the neural and computational
principles, mechanisms, and architectures that underlie human and
animal behavior, and the application of neural network architectures
to the solution of technological problems.

Applications for Fall, 1999, admission and financial aid are now being
accepted for both the MA and PhD degree programs.

To obtain a brochure describing the CNS Program and a set of application
materials, write, telephone, or fax:

DEPARTMENT OF COGNITIVE AND NEURAL SYSTEMS
Boston University
677 Beacon Street
Boston, MA 02215

617/353-9481 (phone)
617/353-7755 (fax)

or send via e-mail your full name and mailing address to the attention
of Mr. Robin Amos at:

inquiries at cns.bu.edu

Applications for admission and financial aid should be received by the
Graduate School Admissions Office no later than January 15.  Late
applications will be considered until May 1; after that date
applications will be considered only as special cases.

Applicants are required to submit undergraduate (and, if applicable,
graduate) transcripts, three letters of recommendation, and Graduate
Record Examination (GRE) scores. The Advanced Test should be in the
candidate's area of departmental specialization. GRE scores may be
waived for MA candidates and, in exceptional cases, for PhD
candidates, but absence of these scores will decrease an applicant's
chances for admission and financial aid.

Non-degree students may also enroll in CNS courses on a part-time
basis.

Stephen Grossberg, Chairman
Gail A. Carpenter, Director of Graduate Studies

Description of the CNS Department:

The Department of Cognitive and Neural Systems (CNS) provides
advanced training and research experience for graduate students
interested in the neural and computational principles,
mechanisms, and architectures that underlie human and animal
behavior, and the application of neural network architectures to
the solution of outstanding technological problems. Students are
trained in a broad range of areas concerning cognitive and
neural systems, including vision and image processing; speech
and language understanding; adaptive pattern recognition;
cognitive information processing; self-organization; associative
learning and long-term memory; cooperative and competitive
network dynamics and short-term memory; reinforcement,
motivation, and attention; adaptive sensory-motor control and
robotics; and biological rhythms; as well as the mathematical
and computational methods needed to support modeling research
and applications. The CNS Department awards MA, PhD, and BA/MA
degrees.

The CNS Department embodies a number of unique features. It has
developed a curriculum that consists of interdisciplinary
graduate courses, each of which integrates the psychological,
neurobiological, mathematical, and computational information
needed to theoretically investigate fundamental issues
concerning mind and brain processes and the applications of
neural networks to technology. Additional advanced courses,
including research seminars, are also offered. Each course is
typically taught once a week in the afternoon or evening to make
the program available to qualified students, including working
professionals, throughout the Boston area. Students develop a
coherent area of expertise by designing a program that includes
courses in areas such as biology, computer science, engineering,
mathematics, and psychology, in addition to courses in the CNS
curriculum.

The CNS Department prepares students for thesis research with
scientists in one of several Boston University research centers
or groups, and with Boston-area scientists collaborating with
these centers. The unit most closely linked to the department is
the Center for Adaptive Systems.  Students interested in neural
network hardware work with researchers in CNS, at the College
of Engineering, and at MIT Lincoln Laboratory.  Other research
resources include distinguished research groups in neurophysiology,
neuroanatomy, and neuropharmacology at the Medical School and
the Charles River Campus; in sensory robotics, biomedical
engineering, computer and systems engineering, and neuromuscular
research within the College of Engineering; in dynamical systems
within the Mathematics Department; in theoretical computer science
within the Computer Science Department; and in biophysics and
computational physics within the Physics Department.

In addition to its basic research and training program, the
department conducts a seminar series, as well as conferences and
symposia, which bring together distinguished scientists from
both experimental and theoretical disciplines.

The department is housed in its own new four-story building
which includes ample space for faculty and student offices and
laboratories, as well as an auditorium, classroom and seminar
rooms, a library, and a faculty-student lounge.

Below are listed departmental faculty, courses and labs.


1998-99 CAS MEMBERS and CNS FACULTY:

Thomas J. Anastasio
Visiting Scholar, Department of Cognitive and Neural Systems (9/1/98-6/30/99)
Associate Professor, Molecular & Integrative Physiology,
Univ. of Illinois, Urbana/Champaign
PhD, McGill University
Computational modeling of vestibular, oculomotor, and other sensorimotor
systems.

Jelle Atema
Professor of Biology
Director, Boston University Marine Program (BUMP)
PhD, University of Michigan
Sensory physiology and behavior.

Aijaz Baloch
Adjunct Assistant Professor of Cognitive and Neural Systems
PhD, Electrical Engineering, Boston University
Neural modeling of role of visual attention in recognition, learning and
motor control, computational vision, adaptive control systems, reinforcement
learning.

Helen Barbas
Associate Professor, Department of Health Sciences
PhD, Physiology/Neurophysiology, McGill University
Organization of the prefrontal cortex, evolution of the neocortex.

Jacob Beck
Research Professor of Cognitive and Neural Systems
PhD, Psychology, Cornell University
Visual perception, psychophysics, computational models.

Daniel H. Bullock
Associate Professor of Cognitive and Neural Systems and Psychology
PhD, Psychology, Stanford University
Real-time neural systems, sensory-motor learning and control, evolution of
intelligence, cognitive development.

Gail A.Carpenter
Professor of Cognitive and Neural Systems and Mathematics
Director of Graduate Studies, Department of Cognitive and Neural Systems
PhD, Mathematics, University of Wisconsin, Madison
Pattern recognition, machine learning, differential equations, technology
transfer.

Gert Cauwenberghs
Visiting Scholar, Department of Cognitive and Neural Systems (6/1/98-8/31/99)
Associate Professor of Electrical And Computer Engineering,
The Johns Hopkins University
PhD, Electrical Engineering, California Institute of Technology
VLSI circuits, systems and algorithms for parallel analog signal processing
and adaptive neural computation.

Laird Cermak
Director, Memory Disorders Research Center, Boston Veterans Affairs Medical
Center
Professor of Neuropsychology, School of Medicine
Professor of Occupational Therapy, Sargent College
PhD, Ohio State University
Memory disorders.

Michael A. Cohen
Associate Professor of Cognitive and Neural Systems and Computer Science
PhD, Psychology, Harvard University
Speech and language processing, measurement theory, neural modeling,
dynamical systems.

H. Steven Colburn
Professor of Biomedical Engineering
PhD, Electrical Engineering, Massachusetts Institute of Technology
Audition, binaural interaction, signal processing models of hearing.

Howard Eichenbaum
Professor of Psychology
PhD, Psychology, University of Michigan
Neurophysiological studies of how the hippocampal system is involved in
reinforcement learning, spatial orientation, and declarative memory.

William D. Eldred III
Associate Professor of Biology
PhD, University of Colorado, Health Science Center
Visual neural biology.

Gil Engel
Research Fellow, Department of Cognitive and Neural Systems
Chief Engineer, Vision Applications, Inc.
Senior Design Engineer, Analog Devices, CTS Division
MS, Polytechnic University, New York
Space-variant active vision systems for use in human-computer interactive
control.

Bruce Fischl
Research Fellow, Department of Cognitive and Neural Systems
PhD, Cognitive and Neural Systems, Boston University
Anisotropic diffusion and nonlinear image filtering, space-variant vision,
computational models of early visual processing, and automated analysis of
magnetic resonance images.

Paolo Gaudiano
Associate Professor of Cognitive and Neural Systems
PhD, Cognitive and Neural Systems, Boston University
Computational and neural models of robotics, vision, adaptive sensory-motor
control, and behavioral neurobiology.

Jean Berko Gleason
Professor of Psychology
PhD, Harvard University
Psycholinguistics.

Sucharita Gopal
Associate Professor of Geography
PhD, University of California at Santa Barbara
Neural networks, computational modeling of behavior, geographical information
systems, fuzzy sets, and spatial cognition.

Stephen Grossberg
Wang Professor of Cognitive and Neural Systems
Professor of Mathematics, Psychology, and Biomedical Engineering
Chairman, Department of Cognitive and Neural Systems
Director, Center for Adaptive Systems
PhD, Mathematics, Rockefeller University
Theoretical biology, theoretical psychology, dynamical systems, and applied
mathematics.

Frank Guenther
Associate Professor of Cognitive and Neural Systems
PhD, Cognitive and Neural Systems, Boston University
Biological sensory-motor control, spatial representation, and speech production.

Catherine L. Harris
Assistant Professor of Psychology
PhD, Cognitive Science and Psychology, University of California at San Diego
Visual word recognition, psycholinguistics, cognitive semantics, second
language acquisition, computational models.

Thomas G. Kincaid
Professor of Electrical, Computer and Systems Engineering,
College of Engineering
PhD, Electrical Engineering, Massachusetts Institute of Technology
Signal and image processing, neural networks, non-destructive testing.

Mark Kon
Professor of Mathematics
PhD, Massachusetts Institute of Technology
Functional analysis, mathematical physics, partial differential equations.

Nancy Kopell
Professor of Mathematics
PhD, Mathematics, University of California at Berkeley
Dynamical systems, mathematical physiology, pattern formation in
biological/physical systems.

Gregory Lesher
Research Fellow, Department of Cognitive and Neural Systems
PhD, Cognitive and Neural Systems, Boston University
Modeling of visual processes, visual perception, statistical
language modeling, and augmentative communication.

Jacqueline A. Liederman
Associate Professor of Psychology
PhD, Psychology, University of Rochester
Dynamics of interhemispheric cooperation; prenatal correlates of
neurodevelopmental disorders.

Ennio Mingolla
Associate Professor of Cognitive and Neural Systems and Psychology
PhD, Psychology, University of Connecticut
Visual perception, mathematical modeling of visual processes.

Joseph Perkell
Adjunct Professor of Cognitive and Neural Systems
Senior Research Scientist, Research Lab of Electronics
and Department of Brain and Cognitive Sciences,
Massachusetts Institute of Technology
PhD, Massachusetts Institute of Technology
Motor control of speech production.

Alan Peters
Chairman and Professor of Anatomy and Neurobiology,
School of Medicine
PhD, Zoology, Bristol University, United Kingdom
Organization of neurons in the cerebral cortex, effects of aging on
the primate brain, fine structure of the nervous system.

Andrzej Przybyszewski
Research Fellow, Department of Cognitive and Neural Systems
PhD, Warsaw Medical Academy
Retinal physiology, mathematical and computer modeling of dynamical
properties of neurons in the visual system.

Adam Reeves
Adjunct Professor of Cognitive and Neural Systems
Professor of Psychology, Northeastern University
PhD, Psychology, City University of New York
Psychophysics, cognitive psychology, vision.

Mark Reinitz
Assistant Professor of Psychology
PhD, University of Washington
Cognitive psychology, attention, explicit and implicit memory,
memory-perception interactions.

Mark Rubin
Research Assistant Professor of Cognitive and Neural Systems
Research Physicist, Naval Air Warfare Center, China Lake, CA (on leave)
PhD, Physics, University of Chicago
Neural networks for vision, pattern recognition, and motor control.

Elliot Saltzman
Associate Professor of Physical Therapy, Sargent College
Assistant Professor, Department of Psychology
and Center for the Ecological Study of Perception and Action
University of Connecticut, Storrs
Research Scientist, Haskins Laboratories, New Haven, CT
PhD, Developmental Psychology, University of Minnesota
Modeling and experimental studies of human speech production.

Robert Savoy
Adjunct Associate Professor of Cognitive and Neural Systems
Scientist, Rowland Institute for Science
PhD, Experimental Psychology, Harvard University
Computational neuroscience; visual psychophysics of color, form, and motion
perception.

Eric Schwartz
Professor of Cognitive and Neural Systems; Electrical, Computer
and Systems Engineering; and Anatomy and Neurobiology
PhD, High Energy Physics, Columbia University
Computational neuroscience, machine vision, neuroanatomy, neural modeling.

Robert Sekuler
Adjunct Professor of Cognitive and Neural Systems
Research Professor of Biomedical Engineering, College of Engineering,
BioMolecular Engineering Research Center
Jesse and Louis Salvage Professor of Psychology, Brandeis University
PhD, Psychology, Brown University
Visual motion, visual adaptation, relation of visual perception, memory,
and movement.

Barbara Shinn-Cunningham
Assistant Professor of Cognitive and Neural Systems
and Biomedical Engineering
PhD, Electrical Engineering and Computer Science,
Massachusetts Institute of Technology
Psychoacoustics, audition, auditory localization, binaural hearing,
sensorimotor adaptation, mathematical models of human performance.

Malvin Teich
Professor of Electrical and Computer Systems Engineering
and Biomedical Engineering
PhD, Cornell University
Quantum optics, photonics, fractal stochastic processes, information
transmission in biological sensory systems.

Lucia Vaina
Professor of Biomedical Engineering
Research Professor of Neurology, School of Medicine
PhD, Sorbonne (France); Dres Science, National Politechnique Institute,
Toulouse (France)
Computational visual neuroscience, biological and computational learning,
functional and structural neuroimaging.

Takeo Watanabe
Assistant Professor of Psychology
PhD, Behavioral Sciences, University of Tokyo
Perception of objects and motion and effects of attention on perception
using psychophysics and brain imaging (fMRI).

Allen Waxman
Adjunct Associate Professor of Cognitive and Neural Systems
Senior Staff Scientist, MIT Lincoln Laboratory
PhD, Astrophysics, University of Chicago
Visual system modeling, mobile robotic systems, parallel computing,
optoelectronic hybrid architectures.

James Williamson
Research Assistant Professor of Cognitive and Neural Systems
PhD, Cognitive and Neural Systems, Boston University
Image processing and object recognition.  Particular interests: dynamic
binding, self-organization, shape representation, and classification.

Jeremy Wolfe
Adjunct Associate Professor of Cognitive and Neural Systems
Associate Professor of Ophthalmology, Harvard Medical School
Psychophysicist, Brigham & Women's Hospital, Surgery Dept.
Director of Psychophysical Studies, Center for Clinical Cataract Research
PhD, Massachusetts Institute of Technology
Visual attention, preattentive and attentive object representation.

Curtis Woodcock
Associate Professor of Geography; Chairman, Department of Geography
Director, Geographic Applications, Center for Remote Sensing
PhD, University of California, Santa Barbara
Biophysical remote sensing, particularly of forests and natural vegetation,
canopy reflectance models and their inversion, spatial modeling, and change
detection; biogeography; spatial analysis; geographic information systems;
digital image processing.


CNS DEPARTMENT COURSE OFFERINGS

CAS CN500  Computational Methods in Cognitive and Neural Systems
CAS CN510  Principles and Methods of Cognitive and Neural Modeling I
CAS CN520  Principles and Methods of Cognitive and Neural Modeling II
CAS CN530  Neural and Computational Models of Vision
CAS CN540  Neural and Computational Models of Adaptive Movement Planning
                        and Control
CAS CN550  Neural and Computational Models of Recognition, Memory and
Attention
CAS CN560  Neural and Computational Models of Speech Perception and Production
CAS CN570  Neural and Computational Models of Conditioning, Reinforcement,
           Motivation and Rhythm
CAS CN580  Introduction to Computational Neuroscience
GRS CN700  Computational and Mathematical Methods in Neural Modeling
GRS CN710  Advanced Topics in Neural Modeling
GRS CN720  Neural and Computational Models of Planning and Temporal Structure
           in Behavior
GRS CN730  Models of Visual Perception
GRS CN740  Topics in Sensory-Motor Control
GRS CN760  Topics in Speech Perception and Recognition
GRS CN780  Topics in Computational Neuroscience
GRS CN810  Topics in Cognitive and Neural Systems: Visual Event Perception
GRS CN811  Topics in Cognitive and Neural Systems: Visual Perception

GRS CN911,912
Research in Neural Networks for Adaptive Pattern Recognition

GRS CN915,916
Research in Neural Networks for Vision and Image Processing

GRS CN921,922
Research in Neural Networks for Speech and Language Processing

GRS CN925,926
Research in Neural Networks for Adaptive Sensory-Motor Planning
and Control

GRS CN931,932
Research in Neural Networks for Conditioning and Reinforcement Learning

GRS CN935,936
Research in Neural Networks for Cognitive Information Processing

GRS CN941,942
Research in Nonlinear Dynamics of Neural Networks

GRS CN945,946
Research in Technological Applications of Neural Networks

GRS CN951,952
Research in Hardware Implementations of Neural Networks

CNS students also take a wide variety of courses in related departments.
In addition, students participate in a weekly colloquium series, an informal
lecture series, and a student-run Journal Club, and attend lectures and
meetings
throughout the Boston area; and advanced students work in small research
groups.


LABORATORY AND COMPUTER FACILITIES

The department is funded by grants and contracts from federal
agencies that support research in life sciences, mathematics,
artificial intelligence, and engineering. Facilities include
laboratories for experimental research and computational
modeling in visual perception, speech and language processing,
and sensory-motor control and robotics. Data analysis and
numerical simulations are carried out on a state-of-the-art
computer network comprised of Sun workstations, Silicon Graphics
workstations, Macintoshes, and PCs.  All students have access to
X-terminals or UNIX workstation consoles, a selection of color
systems and PCs, a network of SGI machines, and standard modeling
and mathematical simulation packages such as Mathematica, VisSim,
Khoros, and Matlab.

The department maintains a core collection of books and
journals, and has access both to the Boston University libraries
and to the many other collections of the Boston Library
Consortium.

In addition, several specialized facilities and software are
available for use.  These include:

Computer Vision/Computational Neuroscience Laboratory

The Computer Vision/Computational Neuroscience Lab is comprised
of an electronics workshop, including a surface-mount
workstation, PCD fabrication tools, and an Alterra EPLD design
system; a light machine shop; an active vision lab including
actuators and video hardware; and systems for computer aided
neuroanatomy and application of computer graphics and image
processing to brain sections and MRI images.

Neurobotics Laboratory

The Neurobotics Lab utilizes wheeled mobile robots to study
potential applications of neural networks in several areas,
including adaptive dynamics and kinematics, obstacle avoidance,
path planning and navigation, visual object recognition, and
conditioning and motivation. The lab currently has three Pioneer
robots equipped with sonar and visual sensors; one B-14 robot
with a moveable camera, sonars, infrared, and bump sensors; and
two Khepera miniature robots with infrared proximity detectors.
Other platforms may be investigated in the future.

Psychoacoustics Laboratory

The Psychoacoustics Lab houses a newly installed, 8 ft. x 8 ft.
sound-proof booth.  The laboratory is  extensively equipped to
perform both traditional psychoacoustic experiments and
experiments using interactive auditory virtual-reality stimuli.
The major equipment dedicated to the psychoacoustics laboratory
includes two Pentium-based personal computers; two
Power-PC-based Macintosh computers; a 50-MHz array processor
capable of generating auditory stimuli in real time;
programmable attenuators; analog-to-digital and
digital-to-analog converters; a real-time head tracking system;
a special-purpose, signal-processing hardware system capable of
generating "spatialized" stereo auditory signals in real time; a
two-channel oscilloscope; a two-channel spectrum analyzer;
various cables, headphones, and other miscellaneous electronics
equipment; and software for signal generation, experimental
control, data analysis, and word processing.

Sensory-Motor Control Laboratory

The Sensory-Motor Control Lab supports experimental studies of
motor kinematics. An infrared WatSmart system allows measurement
of large-scale movements, and a pressure-sensitive graphics
tablet allows studies of handwriting and other fine-scale
movements.  Equipment includes a 40-inch monitor that allows
computer display of animations generated by an SGI workstation
or a Pentium Pro (Windows NT) workstation.  A second major
component is a helmet-mounted, video-based, eye-head tracking
system (ISCAN Corp, 1997).  The latter's camera samples eye
position at 240Hz and also allows reconstruction of what
subjects are attending to as they freely scan a scene under
normal lighting.  Thus the system affords a wide range of
visuo-motor studies.

Speech and Language Laboratory

The Speech and Language Lab includes facilities for
analog-to-digital and digital-to-analog software conversion.
Ariel equipment allows reliable synthesis and playback of
speech waveforms.  An Entropic signal processing package
provides facilities for detailed analysis, filtering, spectral
construction, and formant tracking of the speech waveform.
Various large databases, such as TIMIT and TIdigits, are
available for testing algorithms of speech recognition.  For
high speed processing, supercomputer facilities speed
filtering and data analysis.

Visual Psychophysics Laboratory

The Visual Psychophysics Lab occupies an 800-square-foot suite,
including three dedicated rooms for data collection, and houses
a variety of computer controlled display platforms, including
Silicon Graphics, Inc. (SGI) Onyx RE2, SGI Indigo2 High Impact,
SGI Indigo2 Extreme, Power Computing (Macintosh compatible)
PowerTower Pro 225, and Macintosh 7100/66 workstations.
Ancillary resources for visual psychophysics include a
computer-controlled video camera, stereo viewing glasses,
prisms, a photometer, and a variety of display-generation,
data-collection,  and data-analysis software.

Affiliated Laboratories

Affiliated CAS/CNS faculty have additional laboratories ranging
from visual and auditory psychophysics and neurophysiology,
anatomy, and neuropsychology to engineering and chip design.
These facilities are used in the context of faculty/student
collaborations.

*******************************************************************
DEPARTMENT OF COGNITIVE AND NEURAL SYSTEMS
GRADUATE TRAINING ANNOUNCEMENT

Boston University
677 Beacon Street
Boston, MA 02215

Phone: 617/353-9481
Fax:   617/353-7755
Email: inquiries at cns.bu.edu
Web: http://cns-web.bu.edu/
*******************************************************************

From esann at dice.ucl.ac.be  Tue Sep 22 12:57:19 1998
From: esann at dice.ucl.ac.be (ESANN)
Date: Tue, 22 Sep 1998 18:57:19 +0200
Subject: CFP: ESANN'99 European Symposium on Artificial Neural Networks
Message-ID: <3.0.3.32.19980922185719.006a7adc@ns1.dice.ucl.ac.be>

----------------------------------------------------
|                                                  |
|                     ESANN'99                     |
|                                                  |
|              7th European Symposium              |
|           on Artificial Neural Networks          |
|                                                  |
|      Bruges (Belgium) - April 21-22-23, 1999     |
|                                                  |
|      First announcement and call for papers      |
----------------------------------------------------

The call for papers for the ESANN 99 conference is now available on the Web:
       http://www.dice.ucl.ac.be/esann
For those of you who maintain WWW pages including lists of related ANN
sites: we would appreciate if you could add the above URL to your list;
thank you very much!

We try as much as possible to avoid multiple sendings of this call for
papers; however please apologize if you receive this e-mail twice, despite
our precautions.

You will find below a short version of this call for papers, without the
instructions to authors (available on the Web).  If you have difficulties
to connect to the Web please send an e-mail to
        esann at dice.ucl.ac.be
and we will send you a full version of the call for papers.

ESANN'99 is organised in collaboration with the UCL (Universite catholique
de Louvain, Louvain-la-Neuve) and the KULeuven (Katholiek Universiteit
Leuven), and is technically co-sponsored by the IEEE Neural Networks
Council, the IEEE Region 8, the IEEE Benelux section, and the INNS
(International Neural Networks Society).


Scope and topics
----------------

The aim of the ESANN series of conference is to provide an annual European
forum for the presentation and discussion of recent advances in artificial
neural networks.  ESANN focuses on fundamental aspects of ANNs: theory,
models, learning algorithms, mathematical aspects, approximation of
functions, classification, control, time-series prediction, statistics,
signal processing, vision, self-organization, vector quantization,
evolutive learning, psychological computations, biological plausibility,
etc.  Papers on links and comparisons between ANNs and other domains of
research (such as statistics, data analysis, signal processing, biology,
psychology, evolutive learning, bio-inspired systems, etc.) are also
encouraged.

Papers will be presented orally (no parallel sessions) and in poster
sessions; all posters will be complemented by a short oral presentation
during a plenary session. It is important to mention that it is the topics
of the paper which will decide if it better fits into an oral or a poster
session, not its quality. The quality of posters will be the same as the
quality of oral presentations, and both will be printed in the same way in
the proceedings. Nevertheless, authors have the choice to indicate on the
author submission form that they only accept to present their paper orally.

The following is a non-exhaustive list of topics which will be covered
during ESANN'99:
o theory
o models and architectures
o mathematics
o learning algorithms
o vector quantization
o self-organization
o RBF networks
o Bayesian classification
o recurrent networks
o approximation of functions
o time series forecasting
o adaptive control
o statistical data analysis
o independent component analysis
o signal processing
o natural and artificial vision
o cellular neural networks
o fuzzy neural networks
o hybrid networks
o identification of non-linear dynamic systems
o biologically plausible artificial networks
o bio-inspired systems
o formal models of biological phenomena
o neurobiological systems
o cognitive psychology
o adaptive behavior
o evolutive learning

Special sessions
----------------

Special sessions will be organised by renowned scientists in their
respective fields. Papers submitted to these sessions are reviewed
according to the same rules as any other submission. Authors who submit
papers to one of these sessions are invited to mention it on the author
submission form; nevertheless, submissions to the special sessions must
follow the same format, instructions and deadlines as any other submission,
and must be sent to the same address.

The special sessions organized during ESANN'99 are:
o      Information extraction using unsupervised neural networks
       Colin Fyfe, Univ. of Paisley (UK).
o      Spiking neurons
       Wulfram Gerstner, E.P.F. Lausanne (Switzerland).
o      Adaptive computation of structured information
       Marco Gori, Univ. di Siena (Italy)
o      Remote sensing spectral image analysis
       Erzsebet Merenyi, Univ. of Arizona (USA)
o      Large-scale recognition of sequential patterns
       Yves Moreau, K.U. Leuven (Belgium)
o      Support Vector Machines
       S. Canu, INSIA Rouen (France)
       Bernhard Schoelkopf, GMD FIRST Berlin (Germany)

Location
--------

The conference will be held in Bruges (also called "Venice of the North"),
one of the most beautiful medieval towns in Europe.  Bruges can be reached
by train from Brussels in less than one hour (frequent trains).  The town
of Bruges is world-wide known, and famous for its architectural style, its
canals, and its pleasant atmosphere.

The conference will be organised in an hotel located near the centre
(walking distance) of the town. There is no obligation for the participants
to stay in this hotel. Hotels of all level of comfort and price are
available in Bruges; there is a possibility to book a room in the hotel of
the conference, or in another one (50 m. from the first one) at a
preferential rate through the conference secretariat. A list of other
smaller hotels is also available.

The conference will be held at the Novotel hotel, Katelijnestraat 65B, 8000
Brugge, Belgium.

Call for contributions
----------------------

Prospective authors are invited to submit
- six original copies of their manuscript (including at least two originals
or very good copies without glued material, which will be used for the
proceedings)
- one signed copy of the author submission form
before December 7, 1998.  While this is not mandatory, authors are
encouraged to join a floppy disk or CD with their contribution in (generic)
PostScript or (preferred) PDF format. Sorry, electronic or fax submissions
are not accepted.

Working language of the conference (including proceedings) is English.  The
instructions to authors, together with the author submission form, are
available on the ESANN Web server:
       http://www.dice.ucl.ac.be/esann
A printed version of these documents is also available through the
conference secretariat (please use email if possible).  Authors are invited
to follow the instructions to authors.  A LaTeX style file is also
available on the Web. 

Authors must indicate their choice for oral or poster presentation on the
author submission form.  They must also sign a written agreement that they
will register to the conference and present the paper in case of
acceptation of their submission.  Authors of accepted papers will have to
register before February 28, 1999.  They will benefit from the advance
registration fee.

Submissions must be sent to:
       Michel Verleysen
       UCL - DICE
       3, place du Levant
       B-1348 Louvain-la-Neuve
       Belgium
       esann at dice.ucl.ac.be
All submissions will be acknowledged by fax or email before December 15, 1998.

Deadlines
---------
Submission of papers        December 7, 1998
Notification of acceptance  January 31, 1999
Symposium                   April 21-22-23, 1999

Registration fees
-----------------
              registration before         registration after
              February 28, 1999           February 28, 1999
Universities  BEF 15500                   BEF 16500
Industries    BEF 19500                   BEF 20500
The registration fee include the attendance to all sessions, the lunches
during the three days of the conference, the coffee breaks twice a day, the
conference dinner, and the proceedings.

Conference secretariat
----------------------
       Michel Verleysen		
       D facto conference services       phone: + 32 2 420 37 57
       27 rue du Laekenveld              Fax: + 32 2 420 02 55
       B - 1080 Brussels (Belgium)       E-mail: esann at dice.ucl.ac.be
                     http://www.dice.ucl.ac.be/esann

Steering and local committee
----------------------------

Fran?ois Blayo       Univ. Paris I (F)
Marie Cottrell       Univ. Paris I (F)
Jeanny Herault       INPG Grenoble (F)
Henri Leich          Fac. Polytech. Mons (B)
Bernard Manderick    Vrije Univ. Brussel (B)
Eric Noldus          Univ. Gent (B)
Jean-Pierre Peters   FUNDP Namur (B)
Joos Vandewalle      KUL Leuven (B)
Michel Verleysen     UCL Louvain-la-Neuve (B)

Scientific committee (to be confirmed)
--------------------

Edoardo Amaldi       Cornell Univ. (USA)
Agnes Babloyantz     Univ. Libre Bruxelles (B)
Herve Bourlard       IDIAP Martigny (CH)
Joan Cabestany       Univ. Polit. de Catalunya (E)
Holk Cruse           Universitat Bielefeld (D)
Eric de Bodt         Univ. Lille II (F) & UCL Louvain-la-Neuve (B)
Dante Del Corso      Politecnico di Torino (I)
Wlodek Duch          Nicholas Copernicus Univ. (PL)
Marc Duranton        Philips / LEP (F)
Jean-Claude Fort     Universite Nancy I (F)
Bernd Fritzke        Ruhr-Universitat Bochum (D)
Stan Gielen          Univ. of Nijmegen (NL)
Manuel Grana         UPV San Sebastian (E)
Anne Guerin-Dugue    INPG Grenoble (F)
Martin Hasler        EPFL Lausanne (CH)
Laurent Herault      CEA Grenoble (F)
Christian Jutten     INPG Grenoble (F)
Juha Karhunen        Helsinky Uniersity of Technology (FIN)
Vera Kurkova         Acad. of Science of the Czech Rep. (CZ)
Petr Lansky          Acad. of Science of the Czech Rep. (CZ)
Mia Loccufier        Univ. Gent (B)
Hans-Peter Mallot    Max-Planck Institut (D)
Eddy Mayoraz         IDIAP Martigny (CH)
Jean Arcady Meyer    Univ. Pierre & Marie Curie (F)
Jose Mira Mira       UNED (E)
Jean-Pierre Nadal    Ecole Normale Superieure Paris (F)
Gilles Pages         Universite Paris VI (F)
Thomas Parisini      University of Trieste (I)
Helene Paugam-Moisy  Univ. Lumiere Lyon 2 (F)
Alberto Prieto       Universitad de Granada (E)
Leonardo Reyneri     Politecnico di Torino (I)
Tamas Roska          Hungarian Academy of Science (H)
Jean-Pierre Rospars  INRA Versailles (F)
John Stonham         Brunel University (UK)
Johan Suykens        KUL Leuven (B)
John Taylor          King's College London (UK)
Claude Touzet        CESAR ONRL Oak Ridge (USA)
Marc Van Hulle       KUL Leuven (B)
Christian Wellekens  Eurecom Sophia-Antipolis (F)

==========================ESANN - European Symposium on Artificial Neural Networks
http://www.dice.ucl.ac.be/esann

* For submissions of papers, reviews,...
Michel Verleysen
Univ. Cath. de Louvain - Microelectronics Laboratory
3, pl. du Levant - B-1348 Louvain-la-Neuve - Belgium
tel: +32 10 47 25 51 - fax: + 32 10 47 25 98
mailto:esann at dice.ucl.ac.be

* Conference secretariat
D facto conference services
45 rue Masui - B-1000 Brussels - Belgium
tel: + 32 2 203 43 63 - fax: + 32 2 203 42 94
mailto:esann at dice.ucl.ac.be
==========================

From wolfskil at MIT.EDU  Thu Sep 24 01:55:27 1998
From: wolfskil at MIT.EDU (Jud Wolfskill)
Date: Thu, 24 Sep 1998 01:55:27 -0400
Subject: book announcement:  Brendan Frey, Graphical Models for Machine Learning and Digital Communication
Message-ID: <mailman.960.1149540385.24850.connectionists@cs.cmu.edu>

The following is a book which readers of this list might find of
interest.  For more information please visit
http://mitpress.mit.edu/promotions/books/FREGHF98


Graphical Models for Machine Learning and Digital Communication

Brendan J. Frey


A variety of problems in machine learning and digital communication
deal with complex but structured natural or artificial systems. In this
book, Brendan Frey uses graphical models as an overarching framework to
describe and solve problems of pattern classification, unsupervised
learning, data compression, and channel coding. Using probabilistic
structures such as Bayesian belief networks and Markov random fields,
he is able to describe the relationships between random variables in
these systems and to apply graph-based inference techniques to develop
new algorithms. Among the algorithms described are the wake-sleep
algorithm for unsupervised learning, the iterative turbodecoding
algorithm (currently the best error-correcting decoding algorithm), the
bits-back coding method, the Markov chain Monte Carlo technique, and
variational inference.


Brendan J. Frey is a Beckman Fellow, Beckman Institute for Advanced
Science and Technology, University of Illinois at Urbana-Champaign.


Adaptive Computation and Machine Learning series

A Bradford Book


August 1998

6 x 9, 216 pp., 65 illus. 

cloth 0-262-06202-X


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

     |       Jud Wolfskill                       
 |||||||     Associate Publicist                 Phone:  (617) 253-2079
 |||||||     MIT Press                           Fax:  (617) 253-1709
 |||||||     Five Cambridge Center               E-mail: wolfskil at mit.edu 
      |      Cambridge, MA  02142-1493           http://mitpress.mit.edu                

From barba at cvs.rochester.edu  Thu Sep 24 10:24:19 1998
From: barba at cvs.rochester.edu (Barbara Arnold)
Date: Thu, 24 Sep 1998 10:24:19 -0400
Subject: Faculty position University of Rochester
Message-ID: <l0310280db230030ebc6c@[128.151.80.12]>

Please post this job opening.


Two Assistant Professors in Visual Science.  The University of Rochester
has available two tenure-track positions for scientists working in the
broad domain of visual science, including psychophysical, physiological,
and computational approaches.  Especially encouraged to apply are
candidates whose research is multi-disciplinary. The positions will be in
the Department of Brain and Cognitive Sciences
(http://www.bcs.rochester.edu), one of six departments participating in the
Center for Visual Science (http://www.cvs.rochester.edu), an
interdisciplinary community of 27 faculty engaged in vision research.
Junior appointments are preferred, although appointments at a more senior
level may be considered.  Applicants should submit a curriculum vitae, a
brief statement of research and teaching interests, reprints and three
reference letters to:  David R. Williams, Director, Center for Visual
Science, University of Rochester, Rochester, NY 14627-0270.  Review of
applications will begin December 1, 1998.  Desired start date is September
99 to September 00.  The University of Rochester is an affirmative
action/equal opportunity employer.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Barbara N. Arnold
Administrator                            email: barba at cvs.rochester.edu
Center for Visual Science                phone: 716 275 8659
University of Rochester                    fax: 716 271 3043
Meliora Hall 274
Rochester NY 14627-0270
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 



From granger at uci.edu  Thu Sep 24 20:06:08 1998
From: granger at uci.edu (Richard Granger)
Date: Thu, 24 Sep 1998 17:06:08 -0700
Subject: The hierarchical hypothesis   (Re-send of lost final message; thanks, Dave.)
Message-ID: <v02140b01b2299b98634f@[128.200.120.16]>


We were long puzzled by our biological models' persistent tendency to
produce different cortical outputs over successive theta cycles, and
eventually recognized that what these outputs might be encoding was
sequential hierarchical information (the hypothesis whose formal statement
ultimately appears in the 1990 Science paper).  The hypothesis was novel
and exciting to us -- but perhaps the finding was already obvious to
everyone except us (and the reviewers and editors at Science).  We've
received many private messages indicating that most people in the field do
recognize the history as we've described it, and that it is easy to
misconstrue in hindsight (the continuing special investigation into our
lack of foresight notwithstanding).  Our thanks to the many writers of
those messages!   (One friend reminded us that Postmodernism has clearly
shown that authors know far less than their Text knows, and accordingly
admonished us to "shut up, sit down, and listen to the story of your life
as it Really Happened."  :-)

On the other hand, the discussion is shot through with useful threads
twining around understanding of the distinctions and relationships among
experimental findings, construction of simulations, observation of
simulations, and formal characterization and simplification of results,
much as in physics.  As we come to better understand the differential
nature of experiments, versus simulations, versus formal characterization,
our ability to talk constructively across the boundaries of computation and
biology correspondingly improves.

Finally, it's worth noting that this hypothesis (that cortical neurons
differentially respond over sequential cycles, yielding successive
hierarchical information) is readily differentiated from other hypotheses.
Perhaps, then, comfort can be taken from the realization that physiological
and behavioral evidence may one day demonstrate that some competing
hypothesis is correct after all.


-Rick Granger
 granger at uci.edu


[A number of writers have requested references to subsequent publications
of ours, so a partial list is appended.   (Those of you who requested
references to our research on ampakines, I'll send that list in a separate
message.)  ]

Selected topical bibliography since '90:

Ambros-Ingerson, J., Granger, R., and Lynch, G. (1990).  Simulation of
paleocortex performs hierarchical clustering.   Science, 247: 1344-1348.

Granger, R., Staubli, U., Powers, H., Otto, T., Ambros-Ingerson, J., and
Lynch, G. (1991).  Behavioral tests of a prediction from a cortical network
simulation.  Psychol. Sci., 2: 116-118.

McCollum, J., Larson, J., Otto, T., Schottler, F., Granger, R., and Lynch,
G. (1991).  Short-latency single-unit processing in olfactory cortex.  J.
Cog. Neurosci., 3: 293-299.

Anton, P., Lynch, G., and Granger, R. (1991).  Computation of
frequency-to-spatial transform by olfactory bulb glomeruli.  Biol. Cybern.,
65: 407-414.

Granger, R., and Lynch, G. (1991).  Higher olfactory processes: Perceptual
learning and memory.  Current Opin. Neurosci., 1: 209-214.

Coultrip, R., Granger, R., and Lynch, G. (1992).  A cortical model of
winner-take-all competition via lateral inhibition.  Neural Networks, 5:
47-54.

Lynch, G. and Granger, R. (1992).  Variations in synaptic plasticity and
types of memory in  cortico-hippocampal networks.  J. Cog. Neurosci., 4:
189-199.

Granger, R. and Lynch, G. (1993).  Cognitive modularity: Computational
division of labor in the brain.  In: The Handbook of Neuropsychology, New
York: Academic Press.

Gluck, M. and Granger, R. (1993).  Computational models of the neural bases
of learning and memory. Annual Review of Neurosci. 16: 667-706.

Anton, P., Granger, R., and Lynch, G. (1993).  Simulated dendritic spines
influence reciprocal synaptic strengths and lateral inhibition in the
olfactory bulb.  Brain Res., 628: 157-165.

Coultrip, R. and Granger, R. (1994).  LTP learning rules in sparse networks
approximate Bayes classifiers via Parzen's method.  Neural Networks, 7:
463-476.

Kowtha, V., Satyanarayana, P., Granger, R., and Stenger, D. (1994).
Learning and classification in a noisy environment by a simulated cortical
network.  Proceedings of the Third Annual Computation and Neural Systems
Conference,  Boston: Kluwer, pp. 245-250.

Granger, R., Whitson, J., Larson, J. and Lynch, G. (1994). Non-Hebbian
properties of LTP enable high-capacity encoding of temporal sequences.
Proc. Nat'l. Acad. Sci., 91: 10104-10108.

Myers, C., Gluck, M., and Granger, R. (1995).  Dissociation of hippocampal
and entorhinal function in associative learning: A computational approach.
Psychobiology, 23: 116-138.

Ozeki, T., Shouval, H., Intrator, N. and Granger, R. (1995).  Analysis of a
temporal sequence learning network based on the property of LTP induction.
In: Int'l Symposium on Nonlinear Theory, Las Vegas, 1995.

Kilborn, K., Granger, R., and Lynch, G. (1996).  Effects of LTP on response
selectivity of simulated cortical neurons.  J. Cog. Neurosci., 8: 338-353.

Granger, R., Wiebe, S., Taketani, M., Ambros-Ingerson, J., Lynch, G.
(1997).  Distinct memory circuits comprising the hippocampal region.
Hippocampus, 6: 567-578.

Hess, U.S., Granger, R., Lynch, G., Gall, C.M.  (1997).  Differential
patterns of c-fos mRNA expression in amygdala during sequential stages of
odor discrimination learning.  Learning and Memory, 4: 262-283.





From suem at soc.plym.ac.uk  Fri Sep 25 04:20:28 1998
From: suem at soc.plym.ac.uk (Sue McCabe)
Date: Fri, 25 Sep 1998 09:20:28 +0100
Subject: Job opportunities in the UK
Message-ID: <1.5.4.32.19980925082028.006f8e84@soc.plym.ac.uk>


Based in Plymouth, England, Neural Systems is a young and dynamic
company in the field of Neural Computing.  We are currently looking
for two key individuals to work on a new project.

Senior Research Engineer

Applicant requirements:

* Ph.D. in the field of neural computing
* Experience of the application of neural network technologies
* Strong programming skills using C++ and/or Java
* Self motivated individual with the ability to take responsibility for
project development

Applicant desirables:

* An understanding of process simulation
* Some statistical data analysis experience

Research Engineer

Applicant requirements:

* Post graduate in computer science, engineering or related discipline
* Excellent programming skills using C++ and/or Java
* Experience of the implementation of agent-based programming techniques
* Experience of the software implementation of advanced neural networks
* Microsoft NT operating system experience


Applicants for both positions should be able to demonstrate high levels of
professionalism and technical innovation.  In return Neural Systems, an
equal opportunity employer, are offering a competitive reward package and
the opportunity to work in a leading edge technology, with the chance to
play a major part in the companies' development.

Interested applicants should forward their CV and a covering letter to:

Human Resources
Neural Systems Limited
Tamar Science Park
1 Davy Road
Derriford
Plymouth PL6 8BX
E-mail: HR at neuralsys.com

Dr Sue McCabe
Centre for Neural and Adaptive Systems
School of Computing
University of Plymouth
Plymouth PL4 8AA
England

tel: +44 17 52 23 26 10
fax: +44 17 52 23 25 40
e-mail: sue at soc.plym.ac.uk
http://www.tech.plym.ac.uk/soc/research/neural/index.html


From mharm at CNBC.cmu.edu  Fri Sep 25 15:40:08 1998
From: mharm at CNBC.cmu.edu (Mike Harm)
Date: Fri, 25 Sep 1998 15:40:08 EDT
Subject: thesis available: Division of labor in visual word recognition
Message-ID: <199809251940.PAA04961@CNBC.CMU.EDU>

Hi.

My Ph.D. thesis is now publicly available.  



=================================================================

          DIVISION OF LABOR IN A COMPUTATIONAL MODEL OF 
                   VISUAL WORD RECOGNITION


                       Michael W. Harm
               University of Southern California
                Department of Computer Science
                        August, 1998


Abstract:

How do we compute the meanings of written words?  For decades, the
basic mechanisms underlying visual word recognition have remained
controversial.  The intuitions of educators and policy makers, and the
existing empirical evidence have resulted in contradictory
conclusions, particularly about the role of the sound structure of
language (phonology) in word recognition.  To explore the relative
contributions of phonological and direct information in word
recognition, a large scale connectionist model of visual word
recognition was created containing orthographic, semantic and
phonological representations. The behavior of the model is analyzed
and explained in terms of redundant representations, the development
of dynamic attractors in representational space, the time course of
activation and processing within such networks, and demands of the
reading task itself.  The different patterns of results that have been
obtained in previous behavioral studies are explained by appeal to
stimulus composition and properties of a common experimental paradigm.
A unified explanation of a wide range of empirical phenomena is
presented.

=================================================================

PDF version (you may need acrobat 3.0):

    ftp://siva.usc.edu/pub/coglab/mharm/thesis.pdf  (631 kb)

Compressed postscript version:

    ftp://siva.usc.edu/pub/coglab/mharm/thesis.ps.Z  (381 kb)


It's about 120 pages.


Cheers,

Mike Harm
mharm at cnbc.cmu.edu
Center for the Neural Basis of Cognition
Carnegie Mellon University
http://www.cnbc.cmu.edu/~mharm/
-----------------------------------------

    At midnight, all the agents,
    And the superhuman crew,
    Come out and round up everyone,
    That knows more than they do.

                  Bob Dylan,  "Desolation Row"

From mel at lnc.usc.edu  Fri Sep 25 21:13:25 1998
From: mel at lnc.usc.edu (Bartlett Mel)
Date: Fri, 25 Sep 1998 18:13:25 -0700
Subject: Preprint Available: Binding Problem
Message-ID: <360C3FB5.A08E09A@lnc.usc.edu>

The following preprint is now available via our web page:

http://lnc.usc.edu
25 pages, 230K gzipped postscript

=========================================================

"Seeing with Spatially-Invariant Receptive Fields: 
When the `Binding Problem' Isn't"

Bartlett W. Mel
Biomedical Engineering Department
University of Southern California

Jozsef Fiser
Department of Brain and Cognitive Sciences
University of Rochester


ABSTRACT

We have studied the design tradeoffs governing visual representations
based on complex spatially-invariant receptive fields (RF's), with an
emphasis on the susceptibility of such systems to false-positive
recognition errors---Malsburg's classical ``binding'' problem.  We begin
by deriving an analytical model that makes explicit how recognition
performance is affected by the number of objects that must be
distinguished, the number of features included in the representation,
the complexity of individual objects, and the clutter load, i.e. the
amount of visual material in the field of view in which multiple objects
must be simultaneously recognized, independent of pose, and without
explicit segmentation.  Using the domain of text as a convenient
surrogate for object recognition in cluttered scenes, we show that, with
corrections for the non-uniform probability and non-independence of
English text features, the analytical model achieves good fits to
measured recognition rates in simulations involving a wide range of
clutter loads, word sizes, and feature counts.  We then present a greedy
algorithm for feature learning, derived from the analytical model, which
grows a visual representation by choosing those features most likely to
distinguish objects from the cluttered backgrounds in which they are
embedded.  We show that the representations produced by this algorithm
are decorrelated, heavily weighted to features of low conjunctive order,
and remarkably compact.  Our results provide a quantitative basis for
understanding when, and under what conditions, spatially-invariant
RF-based representations can support veridical perception in
multi-object scenes, and lead to several insights regarding the
properties of visual representations optimized for specific visual
recognition tasks.


-- 

 Bartlett W. Mel                               (213)740-0334, -3397(lab)
 Assistant Professor of Biomedical Engineering (213)740-0343 fax
 University of Southern California, OHE 500     mel at lnc.usc.edu,
http://lnc.usc.edu
                             
     US Mail: BME Department, MC 1451, USC, Los Angeles, CA 90089
       Fedex: 3650 McClintock Ave, 500 Olin Hall, LA, CA 90089

From Eddy.Mayoraz at idiap.ch  Fri Sep 25 12:06:44 1998
From: Eddy.Mayoraz at idiap.ch (Eddy.Mayoraz@idiap.ch)
Date: Fri, 25 Sep 1998 18:06:44 +0200
Subject: Ph.D. position
Message-ID: <199809251606.SAA11198@bishorn.idiap.ch>

******** Uni-Lausanne & IDIAP-Martigny (Switzerland) *******

         PhD student position

We are seeking one outstanding  PhD candidate for an exciting research project,
the aim  of which  is the study  and the  adaptation of recent  developments in
machine learning (neural networks, mixture of experts, support vector machines)
for the resolution of some specific geostatistical tasks.

While  classical geostatistics deals  with spatial  interpolations, one  of the
focuses of this project is to predict,  not only a single expected value of the
observable in an arbitrary location, but also a probability distribution.  This
allows, among other  things, the computation of risk  estimates for being above
some  threshold, which  is critical,  for example,  in applications  related to
pollution.  Another objective of this project is to understand the correlations
between  several spatial  observables and  their exploitation  for  an improved
conditional estimate of one observable given the others.

This is  a joint project  with the Group  of Geostatistics at the  Institute of
Mineralogy and  Petrography, University of  Lausanne, and the  Machine Learning
Group of IDIAP -- Institute for Perceptual Artificial Intelligence at Martigny,
both in Switzerland.

The position is available as soon  as possible and for two years, renewable for
two more  years. Highly  qualified candidates are  sought with a  background in
computational  sciences,  statistics, mathematics,  physics  or other  relevant
areas.  Applicants should submit :  (i) Detailed curriculum vitae, (ii) List of
three references (and their email addresses), (ii) Transcripts of undergraduate
and  graduate (if  applicable) studies  and  (iii) Concise  statement of  their
research interests (two pages max). Please send all documents to:

Prof. Michel Maignan                                Michel.Maignan at imp.unil.ch
Institute for Mineralogy and Petrography        http://www-sst.unil.ch/geostat
Earth Sciences
University of Lausanne
1015 Lausanne, Switzerland

or

Dr. Eddy Mayoraz                                         Eddy.Mayoraz at idiap.ch
IDIAP                                             http://www.idiap.ch/learning
CP 592
1920 Martigny, Switzerland

Electronic applications (with WWW pointers to studies or papers, if available)
are encouraged.

Michel Maignan & Eddy Mayoraz


    /  _ \   /  _ \   _ \   Dr. Eddy Mayoraz, research director of the ML group
   /  /  /  /  /  /  /  /   IDIAP,  P.O. Box 592, CH-1920 Martigny, Switzerland
  /  /  /  /  _  /  ___/    voice: +41 27 721 77 29(11),  fax: +41 27 721 77 12
_/  ___/ _/ _/ _/ _/        Eddy.Mayoraz at idiap.ch, http://www.idiap.ch/~mayoraz


From harnad at coglit.soton.ac.uk  Sun Sep 27 13:16:57 1998
From: harnad at coglit.soton.ac.uk (Stevan Harnad)
Date: Sun, 27 Sep 1998 18:16:57 +0100 (BST)
Subject: Efference and Knowledge: Psyc Call for Commentators
Message-ID: <Pine.SGI.3.95.980927181449.367M-100000@mulberry.psy.soton.ac.uk>


                Jarvilehto: Efference and Knowledge

    The target article whose abstract appears below has just appeared
    in PSYCOLOQUY, a refereed journal of Open Peer Commentary sponsored
    by the American Psychological Association. Qualified professional
    biobehavioral, neural or cognitive scientists are hereby invited to
    submit Open Peer Commentary on it. Please email for Instructions if
    you are not familiar with format or acceptance criteria for
    PSYCOLOQUY commentaries (all submissions are refereed).

    To submit articles and commentaries or to seek information:

    EMAIL: psyc at pucc.princeton.edu 
    URL:   http://www.princeton.edu/~harnad/psyc.html
           http://www.cogsci.soton.ac.uk/psyc

    RATIONALE FOR SOLICITING COMMENTARY: On the basis of experimental
    data plus a simple thought experiment, it is argued that the senses
    should not be considered as transmitting environmental information
    to an organism. Rather, they are part of a dynamical
    organism-environment system in which efferent influences on the
    sensory receptors are especially critical. This view has both
    experimental and philosophical implications for understanding
    knowledge formation on which commentary is invited from
    psychophysicists, sensory physiologists, developmental
    neuroscientists, cognitive scientists, computational modelers,
    information theorists, Gibsonians, Gestaltists, and philosophers.

Full text of article available at:
    http://www.cogsci.soton.ac.uk/cgi/psyc/newpsy?9.41
or:
    ftp://ftp.princeton.edu/pub/harnad/Psycoloquy/1998.volume.9/psyc.98.9.41.efference-knowledge.1.jarvilehto

-----------------------------------------------------------------------
psycoloquy.98.9.41.efference-knowledge.1.jarvilehto     Sun Sep 27 1998
ISSN 1055-0143                (41 paragraphs, 28 references, 623 lines)
PSYCOLOQUY is sponsored by the American Psychological Association (APA)
                Copyright 1998 Timo Jarvilehto

        EFFERENT INFLUENCES ON RECEPTORS IN KNOWLEDGE FORMATION

                Timo Jarvilehto 
                Department of Behavioral Sciences,
                University of Oulu, 
                Finland 
                tjarvile at ktk.oulu.fi
                http://wwwedu.oulu.fi/ktleng/ktleng.htm

    ABSTRACT: This target article suggests a new interpretation of
    efferent influences on sensory receptor activity and the role of
    the senses in forming knowledge. Experimental data and a thought
    experiment about a hypothetical motor-only organism suggest that
    the senses are not transmitters of environmental information;
    rather, they create a direct connection between the organism and
    the environment that makes possible a dynamic organism-environment
    system.  In this system efferent influences on receptor activity
    are especially critical, because with their help the receptors can
    be adjusted in relation to the parts of the environment that are
    most important in achieving behavioral results. Perception joins
    new parts of the environment to the organism-environment system;
    thus knowledge is formed by perception through a reorganization (a
    widening and differentiation) of the organism-environment system
    rather than through the transmission of information from the
    environment.  With the help of efferent effects on receptors, each
    organism creates its own particular world. These considerations
    have implications for experimental work in the neurophysiology and
    psychology of perception as well as for the philosophy of knowledge
    formation.

    KEYWORDS: afference, artificial life, efference, epistemology,
    evolution, Gibson, knowledge, motor theory, movement, perception,
    receptors, robotics, sensation, sensorimotor systems, situatedness




From gyen at okway.okstate.edu  Sun Sep 27 15:57:35 1998
From: gyen at okway.okstate.edu (Gary Yen)
Date: Sun, 27 Sep 1998 13:57:35 -0600
Subject: Conference Announcement
Message-ID: <9809279069.AA906921992@okway.okstate.edu>

Contributed by: Gary G. Yen    gyen at master.ceat.okstate.edu



CALL FOR PAPERS

1999 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORK

RENAISSANCE HOTEL
WASHINGTON, D.C.
JULY 10-16, 1999

The premier conference on neural networks returns to Washington, D.C. in 
1999. Come celebrate the end of the Decade of the Brain and prepare to 
welcome the next millenium with a review of where we are and a preview of 
where we are going. IJCNN'99 spans the neural network field from neurons to 
consciousness, from learning algorithms to robotics, from chaos to control. 
This is an unparalleled opportunity to learn about cutting edge 
developments, to publicize your contributions, and to form new ties with 
your colleagues in industry, academia, and government from around the 
world. For details see our web site: www.inns.org or contact David Brown, 
IJCNN'99 General Chair: dgb at cdrh.fda.gov

CONFERENCE HIGHLIGHTS:
* Distinguished plenaries, highlighted by keynote speaker John Hopfield. 
* Strong technical program, presenting leading research in all 
  neural-network related fields.
* Focused special sessions on such subjects as Biomedical Applications and
  Chaos.
* Theme symposium: Modeling the Brain from Molecules to Mind.
* Forum on Global Competitiveness, including Congressional face-off on U.S.
  SBIR program.
* International workshops on "Getting Your Application to Market,"
  including patent and venture capital questions, and on "Getting Funding  
  for your Rsearch."
* Vigorous tutorial program, with courses taught by the leading experts in
  our field.
* Exhibits and demonstrations of real-world applications.
* Media fair-Show what you have achieved and help educate the public. 
* Awards for best posters and for student contributions.
* Job fair, matching up applicants with employment opportunities. 
* And much, much more: check out IJCNN'99 news on the web site:
  www.inns.org

CALL FOR PAPERS
(Deadline December 4, 1998)
Co-technical Program Committee Chairs: 
Daniel Alkon (NIH)  Clifford Lau (ONR)

IJCNN'99 review and acceptance will be based on a one-page summary, which 
will be distributed to Conference participants in a summary book. Detailed 
instructions for authors are given on the reverse side of this sheet and on 
the web site. Conference proceedings will be in CD form. Hard-copy 
proceedings may be available for an additional fee.

Use of illustrations is encouraged in the summaries, within the single 
allowed page. Use of audio and video segments in the CD proceedings will be 
considered. Poster presentations will be encouraged, with "single-slide" 
poster presentations interspersed with regular oral sessions. Monetary 
awards presented for best poster in several categories. Best student paper 
awards will also be given.

INSTRUCTIONS FOR AUTHORS:
Paper summary format information

The summary must be accompanied by a cover sheet listing the following 
information:
1. Paper title
2. Author information - full names and affiliations as they will appear in
   the program
3. Mailing address, telephone, fax, and email for each author 4. Request 
   for oral or poster presentation
5. Topic(s), selected from the list below:

Biological foundations       Neural systems 
Mathematical foundations     Architectures 
Learning algorithms          Intelligent control 
Artificial systems           Data analysis 
Pattern recognition          Hybrid systems 
Intelligent computation      Applications

  
The one-page, camera-ready summary must conform to the following 
requirements:
1. All text and illustrations must appear within a 7x9 in. (178x229mm)
   area. For US standard size paper (8.5x11 in.), set margins to 0.75 in.   
   (18mm) left and right and 1 in. top and bottom. For A4 size paper, set 
   margins to 17mm left and right, 25mm top, and 45 mm bottom.
2. Use 10-point Times Roman or equivalent typeface for the main text.
   Single space all text, allowing extra space between paragraphs.
3. Title (16 pt. Bold, centered) Capitalize only first word and proper
   names.
4. Authors names, affiliation (12 pt., centered) omit titles or degrees.
5. Five section headings (11 pt. bold). The following five headings MUST be
   used: Purpose, Method, Results, New or breakthrough aspect of work, and 
   Conclusions.

Three copies of the one-page summaries are due in camera-ready, hardcopy 
form to INNS by December 4, 1998. Sent to:

IJCNN'99/INNS
19 Mantua Road
Mt. Royal, NJ 08061 USA
Phone: 609-423-7222 ext. 350

Acceptance will be determined by February 8, 1999, and complete papers are 
due in digital form for CD publication by May 3, 1999.


From barba at cvs.rochester.edu  Mon Sep 28 12:32:44 1998
From: barba at cvs.rochester.edu (Barbara Arnold)
Date: Mon, 28 Sep 1998 12:32:44 -0400
Subject: ad posting
Message-ID: <l03102834b2356a7144f4@[128.151.80.12]>

Please post this ad as soon as possible.

Thank you,
Barbara Arnold

*******************************************************************
                  GRADUATE AND POSTDOCTORAL TRAINING
                   IN THE CENTER FOR VISUAL SCIENCE
                    AT THE UNIVERSITY OF ROCHESTER
*******************************************************************

The Center for Visual Science (CVS) at the University of Rochester
is among the largest research centers dedicated to the study of
visual perception at any university in the world.  Currently CVS
consists of more than 25 research laboratories.  These laboratories
are studying nearly all aspects of vision, from its earliest stages,
such as the encoding of spatial and temporal patterns of light by
neurons in the retina, to its latest stages, such as the interaction
between visual perception and memory.  These laboratories employ a
wide range of theoretical perspectives as well as a diversity of
neuroscientific, behavioral, and computational methodologies.

CVS is a research center that provides a number of services to its
members.  Most important, CVS provides a collegial community in which
vision scientists can meet with each other in order to discuss their
latest research projects and interests.  In addition, CVS provides its
members with a vast array of experimental and computational resources,
an extensive colloquium series, a bi-annual symposium, and other
amenities designed to promote the research activities of its members.


GRADUATE STUDY IN THE CENTER FOR VISUAL SCIENCE

Many students currently pursue graduate training in the Center for
Visual Science.  CVS offers a supportive environment in which students
receive training through coursework and research activities that are
supervised by one or more faculty members.  Due to its large size, CVS
can offer students a training program that is distinctive in its breadth
and depth.  Students can receive training in nearly all aspects of vision,
from its earliest stages in the retina to its latest stages where it
interacts with cognition.  Students are also exposed to a wide range of
theoretical perspectives as well as a diversity of neuroscientific,
behavioral, and computational methodologies.  Regardless of the nature
of a student's interests in visual perception, and regardless of how those
interests evolve during a student's graduate studies, the student can feel
confident that CVS provides an exceptional training environment for that
student.

Graduate study in the Center is undertaken through a degree program
administered by a collaborating academic unit, most often Brain and
Cognitive Sciences, Computer Science, Neuroscience, or Optics.  A
student chooses one of these degree programs, and satisfies its
requirements for a Ph.D. while specializing in visual science.
Because the program of study in the Center for Visual Science draws
students from a variety of backgrounds, and is integrated with the
other programs, the plan of study is flexible and easily tailored to
suit individual students' needs and interests, while ensuring a
thorough grounding in visual science.

The program of study emphasizes research that is supervised by one or
more members of the faculty, complemented by courses in visual perception
typically taken during the first two years of study. The sequence of
courses begins with the two-semester Mechanisms of Vision, which is team
taught by the faculty at the Center and covers a full range of topics in
vision.  This is followed by a series of more advanced courses, on topics
such as Color Vision, Spatial Vision, Motion Perception, Visual Space
Perception, Computational Problems in Vision, Computational Models of
Behavior, Real-time Laboratory Computing, and Instrumentation and Methods
for Vision Research.  Throughout the program students are actively engaged
in research; during their last two to three years of the four to five year
program students spend all of their time on the research that culminates
in the Ph.D.

Students contemplating graduate work at the Center should contact
Barbara Arnold (address below) who will be glad to provide additional
information and application materials.  Admission to the Center's
program includes a tuition waiver and a competitive 12-month stipend
that is guaranteed for at least four years, subject to satisfactory
progress.


POSTDOCTORAL STUDY IN THE CENTER FOR VISUAL SCIENCE

Many postdoctoral fellows currently receive training in the Center
for Visual Science.  The wide range of scientific disciplines
represented by faculty of the Center and the closeness of their
collegial contacts makes the Center a particularly attractive place
for interdisciplinary research.  Postdoctoral fellows often work with
more than one member of faculty, and the emphasis of the training is
on research methods (especially the conjunction of different methods
brought to bear on a single problem) that are characteristic of the
Center.

Scientists interested in postdoctoral study at the Center should contact
the faculty member(s) with whom they might wish to work. Postdoctoral
fellows are supported from a variety of sources: some receive support
through individual investigators' research grants; some receive stipends
from the Center's training grant, funded by the National Eye Institute;
some are supported by individual fellowships.

CVS is currently seeking new graduate students and postdoctoral
fellows to join our community.  To learn more about the Center for
Visual Science, please contact us at:

        Barbara Arnold
        Center for Visual Science
	Meliora Hall, River Campus
        University of Rochester
        Rochester, NY 14627
        Phone: (716) 275-2459
        Fax: (716) 271-3043
        e-mail: barba at cvs.rochester.edu
        WWW: http://www.cvs.rochester.edu


Faculty:
-------

Richard Aslin
   Perceptual development in infants

Dana Ballard
   Computer vision, computational neuroscience, visuomotor integration

Daphne Bavelier
   Brain imaging, visual attention, visuospatial representations

David Calkins
   Retinal neurophysiology, visual neuroscience

Robert Chapman
   Brain imaging, visual information processing

Charles Duffy
   Visual motion processing, visual neuroscience

Robert Emerson
   Spatial vision, visual neuroscience

Mary Hayhoe
   Visual perception and cognition, visuomotor integration

James Ison
   Audition, sensory reflexes, sensori-motor control

Robert Jacobs
   Visual perception and cognition, computational modeling

Carolyn Kalsow
   Clinical research in vision and eye care

Barrett Katz
   Clinical research in vision and eye care

Walter Makous
   Visual psychophysics, spatial vision

William Merigan
   Visual neural pathways, visual neuroscience

Gary Paige
   Vestibular and adaptive control of equilibrium, visual neuroscience

Tatiana Pasternak
   Neural mechanisms of motion and form perception, visual neuroscience

Alexandre Pouget
   Computational neuroscience, neural coding, visuospatial representations

James Ringo
   Neural mechanisms of memory and visual processing, visual neuroscience

Marc Schieber
   Neural control of finger movements, sensorimotor integration

Gail Seigel
   Retinal cell biology, visual neuroscience

Michael Weliky
   Neural development of the visual system, visual neuroscience

David Williams
   Spatial and color vision, retinal structure and visual perception

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Barbara N. Arnold
Administrator                            email: barba at cvs.rochester.edu
Center for Visual Science                phone: 716 275 8659
University of Rochester                    fax: 716 271 3043
Meliora Hall 274
Rochester NY 14627-0270
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 



From ormoneit at stat.Stanford.EDU  Mon Sep 28 15:50:21 1998
From: ormoneit at stat.Stanford.EDU (Dirk Ormoneit)
Date: Mon, 28 Sep 1998 12:50:21 -0700 (PDT)
Subject: Thesis on Density Estimation
Message-ID: <199809281950.MAA23100@rgmiller.Stanford.EDU>

Hi,

My PhD thesis on probability estimating neural networks
is now available from Shaker Verlag / Aachen (ISBN 3-8265-3723-8):

http://www.shaker.de/Online-Gesamtkatalog/Details.idc?ISBN=3-8265-3723-8

For more information on specific topics touched upon in the abstract below,
see also my Stanford homepage:

http://www-stat.stanford.edu/~ormoneit/

Best,

Dirk

===========================================================================

                   PROBABILITY ESTIMATING NEURAL NETWORKS

				Dirk Ormoneit
			Fakult"at f"ur Informatik
		     Technische Universit"at M"unchen

A central problem of machine learning is the identification of  
probability distributions that govern uncertain environments. 
A suitable concept for ``learning'' probability distributions from
sample data may be derived by employing neural networks. In this work 
I discuss several neural architectures that can be used to learn 
various kinds of probabilistic dependencies.
After briefly reviewing essential concepts from neural learning and 
probability theory, I provide an in-depth discussion of neural and other 
approaches to conditional density estimation. In particular, I  
introduce the ``Recurrent Conditional Density Estimation Network (RCDEN)'', 
a neural architecture which is particularly well-suited to 
identify the transition densities of time-series in the presence of 
latent variables. As a practical example, I consider the conditional 
densities of German stock market returns and compare the results of the 
RCDEN to those of ARCH-type models.
A second focus of the work is on the estimation of multivariate densities
by means of Gaussian mixture models. A severe problem for the practical 
application of Gaussian mixture estimates is their strong tendency to 
``overfit'' the training data. In my work I compare three regularization 
procedures that can be applied to deal with this problem. 
The first method consists of deriving EM 
update rules for maximum penalized likelihood estimation. In the second
approach, the ``full'' Bayesian inference is approximated by means of 
a Markov chain Monte Carlo algorithm. Finally, I apply ensemble averaging
to regularize the Gaussian mixture estimates, most prominently a variant of
the popular ``bagging'' algorithm. The three approaches are compared in
extensive experiments that involve the construction of Bayes classifiers
from the density estimates. The work concludes with considerations on
several practical applications of density estimating neural networks in
time-series analysis, data transmission, and optimal planning in 
multi-agent environments.


--------------------------------------------
Dirk Ormoneit
Department of Statistics, Room 206
Stanford University
Stanford, CA 94305-4065
 
ph.: (650) 725-6148
fax: (650) 725-8977
 
ormoneit at stat.stanford.edu
http://www-stat.stanford.edu/~ormoneit/


From smagt at dlr.de  Mon Sep 28 04:20:50 1998
From: smagt at dlr.de (Patrick van der Smagt)
Date: Mon, 28 Sep 1998 10:20:50 +0200
Subject: paper on conditioning and local minima in MLP
Message-ID: <360F46E2.42CFD049@robotic.dlr.de>

Dear connectionists:

the following ICANN'98 reprint is available via the web:
http://www.robotic.dlr.de/Smagt/papers/SmaHir98b.ps.gz

  "Why feed-forward networks are in a bad shape"
        P. van der Smagt and G. Hirzinger
    German Aerospace Center/DLR Oberpfaffenhofen

Abstract:
It has often been noted that the learning problem in feed-forward neural
networks is very badly conditioned.  Although, generally, the special form of
the transfer function is usually taken to be the cause of this condition, we
show that it is caused by the manner in which neurons are connected.  By
analyzing the expected values of the Hessian in a feed-forward network it is
shown that, even in a network where all the learning samples are well chosen
and the transfer function is not in its saturated state, the system has a
non-optimal condition.  We subsequently propose a change in the feed-forward
network structure which alleviates this problem.  We finally demonstrate the
positive influence of this approach.


Other papers available on http://www.robotic.dlr.de/Smagt/papers/
--
dr Patrick van der Smagt                phone +49 8153 281152, fax -34
DLR/Institute of Robotics and System Dynamics             smagt at dlr.de
P.O.Box 1116, 82230 Wessling, Germany     http://www.robotic.de/Smagt/

From gkk at neuro.informatik.uni-ulm.de  Mon Sep 28 18:57:22 1998
From: gkk at neuro.informatik.uni-ulm.de (Gerhard K. Kraetzschmar)
Date: Tue, 29 Sep 1998 00:57:22 +0200
Subject: Call for Interest in Participation in IJCAI-99 Workshop
Message-ID: <36101452.A96E5A97@neuro.informatik.uni-ulm.de>


Dear reader of this news group or list: 
(Our apologies, if you receive this multiple times)

We plan to organize a workshop at IJCAI-99 in Stockholm.
The topic is
        Adaptive Spatial Representations for Dynamic Environments 
and we believe it may be of interest to you. Please read the 
draft for the workshop proposal in the attachment for more information
on the workshop.

You can contribute to the workshop by submitting a paper, giving one of 
the survey talks, or one of the commenting statements in a session.

Note: IJCAI permits only active participants for workshops.

If you come to the conclusion that you are indeed interested in the
workshop and want to actively participate, please take the time to respond
with a short email that indicates your interest and kind of contribution.
Please send the email to 
	gkk at acm.org
and include "IJCAI-WORKSHOP" in the subject line. Thanks a lot.

Please do respond soon, because the due date for the proposal is in a 
few days, and we need to collect a list of tentative participants for it.

-- 
Sincerely Yours, Gerhard
---------------------------------------------------------------------------
Dr. Gerhard K. Kraetzschmar                   University of Ulm
Fon: intl.+49-731-502-4155	              Neural Information Processing
Fax: intl.+49-731-502-4156	              Oberer Eselsberg
Net: gkk at neuro.informatik.uni-ulm.de          89069 Ulm   
     gkk at acm.org                              Germany
WWW: http://www.informatik.uni-ulm.de/ni/staff/gkk.html
---------------------------------------------------------------------------

Proposal for IJCAI-99 Workshop
========================================================
Adaptive Spatial Representations of Dynamic Environments
========================================================
Workshop Description:
====================
Spatial representations of some sort are a necessary requirement for
any mobile robot in order to achieve tasks like efficient,
goal-oriented navigation, object manipulation, or interaction with
humans. A large number of approaches, ranging from CAD-like and
topological representations to metric, probabilistic methods and
biologically-oriented representations, has been developed in the past.
Most approaches were developed for solving robot navigation tasks 
and successfully applied in a wide variety of applications. In many
approaches, the spatial representation is a strong simplification of 
the environment (e.g. occupied and free space) and does not permit an 
easy representation of the spatial structure of task-relevant objects.
Furthermore, these approaches can model only static elements of the
environment in an adequate manner. Dynamic elements, such as doors,
changed locations of particular objects, moving obstacles, and humans,
are usually dealt with in one of two ways:
- A purely reactive level temporarily has a transient representation 
	for an anonymous object. The representation is present only as 
	long as the object can be actively sensed; it vanishes
	thereafter and does not settle into some kind of permanent
	representation. (Doors, moving obstacles, humans.)
- Repeated exposure of the robot to both the old and new location of
	an object that changed its position leads to a slow adaptation
	of (long-term) spatial memory. (Moved, relocated objects)

The current representations are sufficient for many tasks that have 
been researched in the past and led to many successful applications.
However, in order to achieve truely useful robots, e.g. for the office
domain, the robot will have to acquire AND MAINTAIN more complete 
models of its environment. It will have to know the precise locations 
(or have a good estimate of it) of objects and subjects it has seen,
if they are relevant for completing a task. Examples: Location of
tools (screwdriver), books, special equipment (video beamer), persons, 
doors, etc.

Thus, for any existing spatial representation, the following 
questions arise?
- Which structural aspects of the environment can be modeled? How?
- Can the representation model dynamic aspects at all? Could it be extended?
- Which kinds of dynamic aspects can be modeled?
- How are the spatial dynamics of an object modeled?
- How is uncertainty dealt with?
- How are the dynamics used to predict various spatial aspects of
	objects?
- Which methods can be used to update/maintain the representation
	based on sensory information?
- What computational effort is required for updating the representation?

For many of the current approaches, we have little knowledge about how 
these questions must be answered. The goal of the workshop is to bring 
together researchers who develop and use various kinds of spatial 
representations in mobile robots and make them think about how to answer 
the above questions for their approaches. A secondary goal is to provide 
a forum on which various kinds of representations can be compared.

Workshop Actuality and Target Audience:
======================================= 
Currently, various successful, though specialized applications of
mobile robots (RHINO, etc.) are known. However, improved adaptive
spatial representations will be needed to build service robots for
more complex tasks in more complex environments. We consider such
representations an essential step for making progress towards this
goal. Thus, the target audience includes 
- researchers working in mobile robots, especially map building, 
  spatial modelling, navigation, object manipulation, and human-robot
  interaction, 
- AI researchers working on topological and other symbolic methods, 
  metric and probabilistic representations, and uncertainty. 
The workshop also appeals to researchers that have 
- studied spatial data structures in CAD, GIS, and image processing or
- studied spatial representations in biological systems 
and are now applying their models in robotic systems.

Preliminary Workshop Agenda:
============================
Depending on submissions, available time, and IJCAI constraints on the
schedule, we plan four to five sessions, each one will most likely be 
centered around one of the following themes:
- CAD/GIS-Inspired Representations (Frank/?Samet)
- Topological Representations (Kuipers/Cohn/Nebel)
- Metric and Probabilistic Representations (Burgard/?Kaelbling)
- Biologically-Inspired Representations (Recce/Tani/?Mallot)

In each session (90 minutes) covering one of the above four
approaches, we plan to implement the following session program:
- Invited survey talk by an experienced research scientist (25+5 min)
- Two short talks selected from the workshop paper submissions (10+5 min)
- Two to three rebutting/commenting statements by representatives 
	from the other approaches (5 min each)
- Session discussion (15 min), moderated by session chair
A general discussion session (45 to 60 minutes) will try to 
summarize results, draw conclusions, and define future activities,
like workshops, definition of benchmark problems, and others.

Tentative Attendees:
====================
(will be collected from response after announcements of various 
 news groups and lists:
	comp.robotics
	comp.ai
	comp.ai.neural-nets
	comp.ai.nlang-know-rep
	connectionists list
	hybrid list
 Please add any relevant news group or list)
	
Workshop Organizing Committee:
=============================
* To be confirmed.

Andrew Frank (GIS/CAD representations)
Bernhard Nebel (AI, relational/topological representations)
Anthony Cohn (AI, relational/topological representations)
Gerhard Kraetzschmar (chair) (AI, robotics, hybrid representations)
Benjamin Kuipers (AI, robotics, hybrid representations)
Michael Beetz (AI, robotics, metric/probabilistic representations)
Wolfram Burgard (AI, robotics, metric/probabilistic representations)
Gunther Palm (neuroscience, neural networks)
Michael Recce (neuroscience, biologically-inspired robotics)
Jun Tani (biologically-inspired robotics, dynamics) 

*Leslie Kaelbling
*Hanspeter Mallot
*Hanan Samet

Primary Contact:
===============
Gerhard K. Kraetzschmar
University of Ulm, Neural Information Processing
James-Franck-Ring, 89081 Ulm, Germany
Fon: +49-731-50-24155
Fax: +49-731-50-24155
Net: gkk at acm.org  or gkk at neuro.informatik.uni-ulm.de

From ghaziri at aub.edu.lb  Tue Sep 29 17:59:40 1998
From: ghaziri at aub.edu.lb (Dr. Hassan Ghaziri)
Date: Tue, 29 Sep 1998 14:59:40 -0700
Subject: IFORS 99 CHINA, NN in OR
References: <36101452.A96E5A97@neuro.informatik.uni-ulm.de>
Message-ID: <3611584C.E650E291@aub.edu.lb>

Dear Colleagues,
I have been asked to organize a session on  neural networks and thewir
applications in operations research as part of the cluster on metaheuristics,
for which I would like to invite you to
  participate in presenting a paper at IFORS-99:


 {                     IFORS'99, 15th World Conference on Operational
  Research  Triennial Meeting of the International Federation of
  Operational Research Societies -- IFORS
               Hosted by the Operations Research Society of China
                      Beijing, China, August 16 - 20, 1999.
  http://www.IFORS.org/leaflet/triennial.html
  where you can find more details on the conference.
  }

  A typical session is 100 minutes long and has 3-5 papers.
  Submission of full papers is optional.  All submitted full papers
  will be considered for publication in the International Transactions
  in Operational Research (Peter Bell - Editor; Publisher Pergamon
  Press on behalf of IFORS).  Papers should not exceed 5,000 words.

  If you are interested please let me have a  title of your
paper and send them to my Lebanon  address with the following details.
  - title of the paper,
  - authors names and addresses,
  - 50-100 word abstracts.

  I would like to have these information not later than October 16, 1998.

         I am looking forward to hearing from you at your
  earliest.

         Best regards
Hassan Ghaziri
AUB, Business Scool, Beirut Lebanon
Tel: 00 961-1 352700
E-mail: ghaziri at aub.edu.lb






From niall.griffith at ul.ie  Tue Sep 29 08:41:51 1998
From: niall.griffith at ul.ie (Niall Griffith)
Date: Tue, 29 Sep 1998 13:41:51 +0100
Subject: Neural Networks and MultiMedia - IEE Colloquium
Message-ID: <9809291241.AA24335@zeus.csis.ul.ie>

Please pass this on to anyone or any group you think may be interested.

=========================================================
IEE Colloquium

"Neural Networks in Multimedia Interactive Systems"

Date:  Thursday 22 October 1998
Time:  10.30 - 17.00
Place: Savoy Place, London.

The Neural Networks Professional Group (A9) of the IEE is holding an
inaugural colloquium at Savoy Place, London, on the use of neural network
models in multimedia systems.

This colloquium will present a range of current neural network
applications in the area of interactive multimedia, and will cover:
learning, intelligent agents within multimedia systems, data mining, image
processing and intelligent application interfaces.

========================================================== 

For more information and registration details please contact 

The IEE Events Office:

Tel.   +44 171 240 1871    (Extension 2205/2206)
Email. events at iee.org.uk
URL.   http://www.iee.org.uk/Calendar/a22oct98.html

Colloquium organisers:
Niall Griffith, University of Limerick, niall.griffith at ul.ie
Nigel Allinson, UMIST, allinson at fs5.ee.umist.ac.uk
==========================================================

Provisional Timetable

10.00 - 10.25   Registration and Coffee

10.25 - 10.45   Welcome 
                Self-organising neural networks for multimedia
                Prof. N. Allinson (UMIST)

10.45 - 11.25   Searching image databases containing trademarks.
                Sujeewa Alwis and Dr. J. Austin (University of York)

11.25 - 12.05   Synthetic Characters: Behaving in Character
                Dr. B. Blumberg (MIT Media Lab)

12.05 - 12.45   Intelligent components for interactive multimedia
                Dr. R. Beale (University of Birmingham)

12.45 - 13.45   Lunch

13.45 - 14.15   Image Retrieval and Classification Using Affine 
                Invariant B-Spline Representation and Neural Networks
                Y. Xirouhakis  (National Technical University of Athens)

14.15 - 14.45   Hybrid Neural Symbolic Agent Architectures for Multimedia
                Prof. S. Wermter (University of Sunderland)

14.45 - 15.15   3D Reconstruction of Human Faces from Range Data 
                through HRBF Networks
                Dr. A. Borghese 
                (Istituto Neuroscienze e Biommagini, Milan)

15.15 - 15.30   Tea

15.30 - 16.00   A multi-agent framework for visual surveillance
                P. Remagnino & Dr. G Jones (Kingston University)

16.00 - 16.30   Discussion and Close
-----------------

From kehagias at egnatia.ee.auth.gr  Tue Sep 29 13:45:08 1998
From: kehagias at egnatia.ee.auth.gr (Thanasis Kehagias)
Date: Tue, 29 Sep 1998 10:45:08 -0700
Subject: New Book on modular NN and time series
Message-ID: <3.0.5.32.19980929104508.007a0a50@egnatia.ee.auth.gr>

NEW BOOK on Modular Neural Networks and Time Series

The following book has just been published by Kluwer Academic Publishers. 

TITLE:	Predictive Modular Neural Networks: Applications to Time Series 
AUTHORS:	V. Petridis and Ath. Kehagias 
PUBLISHER:	Kluwer Academic Publishers, Boston
YEAR:		1998
ISBN:		0-7923-8290-0

This book will be of interest to connectionists, machine learning
researchers, statisticians, control theorists and perhaps also to
researchers in biological and medical informatics, researchers in
econometrics and forecasting as well as psychologists. It can be ordered
from Kluwer's web site at 

http://www.wkap.nl/book.htm/0-7923-8290-0

The general subject of the book is the application of modular neural
networks (in another terminology: multiple models) to problems of time
series classification and  prediction. The problem of system identification
is also treated as a time series problem. We consider both supervised
learning of labelled TS data and unsupervised learning of unlabelled TS
data. We present a general framework for the design of PREDICTIVE MODULAR
algorithms and provide a rigorous convergence analysis for both the
supervised and unsupervised cases. We also present the application of the
above algorithms to three real world problems (encephalogram
classification, electric load prediction and waste water plant parameter
etsimation). Finally, we provide an extensive bibliography of modular and
multiple models methods and discuss the connections between such methods
which have appeared in the neural networks as well as in other research
communities.

More info about the book can be found at 

http://www.wkap.nl/book.htm/0-7923-8290-0

or at 

http://skiron.control.ee.auth.gr/~kehagias/thn/thn02b01.htm
-------------------------------------------------------------

TABLE OF CONTENTS

1. Introduction
1.1 Classification, Prediction and Identification: an Informal Description
1.2 Part I: Known Sources 
1.3 Part II: Applications 
1.4 Part III: Unknown Sources 
1.5 Part IV: Connections 

PART I Known Sources

2.  PREMONN Classification and Prediction
2.1 Bayesian Time Series Classification 
2.2 The Basic PREMONN Classification Algorithm 
2.3 Source Switching and Thresholding 
2.4 Implementation and Variants of the PREMONN Algorithm 
2.5 Prediction 
2.6 Experiments
2.7 Conclusions 

3.  Generalizations of the Basic PREMONN
3.1 Predictor Modifications
3.2 Prediction Error Modifications
3.3 Credit Assignment Modifications
3.4 Markovian Source Switching
3.5 Markovian Modifications of Credit Assignment Schemes
3.6 Experiments
3.7 Conclusions

4.  Mathematical Analysis
4.1 Introduction 
4.2 Convergence Theorems for Fixed Source Algorithms 
4.3 Convergence Theorem for a Markovian Switching Sources Algorithm 
4.4 Conclusions 

5.  System Identification by the Predictive Modular Approach
5.1 System Identification 
5.2 Identification and Classification 
5.3 Parameter Estimation: Small Parameter Set 
5.4 Parameter Estimation:\ Large Parameter Set
5.5 Experiments 
5.6 Conclusions 

PART II Applications

6.  Implementation Issues
6.1 PREMONN Structure
6.2 Prediction 
6.3 Credit Assignment 
6.4 Simplicity of Implementation 

7.  Classification of Visually Evoked Responses
7.1 Introduction 
7.2 VER Processing and Classification 
7.3 Application of PREMONN Classification 
7.4 Results 
7.5 Conclusions 

8.  Prediction of Short Term Electric Loads
8.1 Introduction 
8.2 Short Term Load Forecasting Methods 
8.3 PREMONN Prediction 
8.4 Results
8.5 Conclusions 

9.  Parameter Estimation for and Activated Sludge Process
9.1 Introduction 
9.2 The Activated Sludge Model 
9.3 Predictive Modular Parameter Estimation 
9.4 Results 
9.5 Conclusions 

PART III Unknown Sources

10.  Source Identification Algorithms
10.1 Introduction
10.2 Source Identification and Data Allocation 
10.3 Two Source Identification Algorithms 
10.4 Experiments 
10.5 A Remark about Local Models 
10.6 Conclusions 

11.  Convergence of Parallel Data Allocation
11.1 The Case of Two Sources 
11.2 The Case of Many Sources
11.3 Conclusions 

12.  Convergence of Serial Data Allocation
12.1 The Case of Two Sources 
12.2 The Case of Many Sources 
12.3 Conclusions 

PART IV Connections

13.  Bibliographic Remarks
13.1 Introduction 
13.2 Neural Networks 
		Combination of Specialized Networks252
		Ensembles of Networks
		Mixtures of Experts
		RBF and Related Networks
		Trees
13.3 Statistical Pattern Recognition 
13.4 Econometrics and Forecasting 
13.5 Fuzzy Systems 
13.6 Control Theory 
13.7 Statistics 

14.  Epilogue

Appendix: Mathematical Concepts
References
Index

-------------------------------------------------------------

			The book's PREFACE

The subject of this book is predictive modular neural networks and their
application to time series problems: classification, prediction and
identification. The intended audience is researchers and
graduate students in the fields of neural networks, computer science,
statistical pattern recognition, statistics, control theory and
econometrics. Biologists, neurophysiologists and medical engineers
may also find this book interesting. 

In the last decade the neural networks community has shown intense interest
in both modular methods and time series problems. Similar interest has been
expressed for many years in other fields as well, most notably in
statistics, control theory, econometrics etc. There is a considerable
overlap (not always recognized) of ideas and methods between these fields. 

Modular neural networks come by many other names, for instance multiple
models, local models and mixtures of experts. The basic idea is to
independently develop several ``subnetworks'' (modules), which may perform
the same or related tasks, and then use an ``appropriate'' method
for combining the outputs of the subnetworks. Some of the expected
advantages of this approach (when compared with the use of ``lumped'' or
``monolithic'' networks) are: superior performance, reduced development
time and greater flexibility. For instance, if a module is removed from the
network and replaced by a new module (which may perform the same task more
efficiently), it should not be necessary to retrain the aggregate network. 

In fact, the term ``modular neural networks'' can be rather vague. In its
most general sense, it denotes networks which consist of simpler
subnetworks (modules). If this point of view is taken to
the extreme, then every neural network can be considered to be modular, in
the sense that it consists of neurons which can be seen as elementary
networks. We believe, however, that it is more profitable to think of a
continuum of modularity, placing complex nets of very simple neurons
at one end of the spectrum, and simple nets of very complex neurons at the
other end. 

We have been working along these lines for several years and have developed
a family of algorithms for time series problems, which we call PREMONN's
(i.e. PREdictive MOdular Neural Networks). Similar algorithms and systems
have also been presented by other authors, under
various names. We will generally use the acronym PREMONN to refer to our
own work and retain ``predictive modular neural networks'' as a generic term. 

This book is divided in four parts. In Part I we present some of our work
which has appeared in various journals such as IEEE Transactions on Neural
Networks, IEEE Transactions on Fuzzy Systems, Neural Computation, Neural
Networks etc. We introduce the family of PREMONN algorithms. These
algorithms are appropriate for online time series classification,
prediction and identification. We discuss these algorithms at an informal
level and we also analyze mathematically their convergence properties. 

In Part II we present applications (developed by ourselves and other
researchers) of PREMONNs to real world problems. In both these parts a
basic assumption is that models are available to describe the input /
output behavior of the sources generating the time series of
interest. This is the known sources assumption. 

In Part III we remove this assumption and deal with time series generated
by completely unknown sources. We present algorithms which operate online
and discover the number of sources involved in the generation of a time
series and develop input/ output models for each source. These source
identification algorithms can be used in conjunction with the
classification and prediction algorithms of Part I. The results of Part III
have not been previously published. 

Finally, in Part IV we briefly review work on modular and multiple models
methods which has appeared in the literature of neural networks,
statistical pattern recognition, econometrics, fuzzy systems, control
theory and statistics. We argue that there is a certain unity of themes and
methods in all these fields and provide a unified interpretation of the
multiple models idea. We hope that this part will prove useful by pointing
out and elucidating similarities between the multiple models
methodologies which have appeared in several disparate fields. 

Indeed, we believe that there is an essential unity in the modular
approach, which cuts across disciplinary boundaries. A good example is the
work reported in this book. While we present our work in ``neural''
language, its essential characteristic is the combination of simple
processing elements which can be combined to form more complex (and
efficient) computational structures. There is nothing exclusively neural
about this theme; it has appeared in all the above mentioned
disciplines and this is why we believe that a detailed literature search
can yield rich dividends in terms of outlook and technique cross
fertilization. 

The main prerequisite for reading this book is the basics of neural network
theory (and a little fuzzy set theory). In Part I, the mathematically
involved sections are relegated to appendices, which may be left for a
second reading, or omitted altogether. The same is true of Part III:
convergence proofs (which are rather involved) are presented in appendices,
while the main argument can be followed quite independently of the
mathematics. Parts II and IV are nonmathematical. We have
also provided an appendix, which contains the basic mathematical concepts
used throughout the book.
___________________________________________________________________
Ath. Kehagias
--Assistant Prof. of Mathematics, American College of Thessaloniki 
--Research Ass., Dept. of Electrical and Computer Eng. Aristotle Univ.,
Thessaloniki, GR54006, GREECE

--email:	 kehagias at egnatia.ee.auth.gr, kehagias at ac.anatolia.edu.gr
--web:	http://skiron.control.ee.auth.gr/~kehagias/index.htm

From kehagias at egnatia.ee.auth.gr  Tue Sep 29 13:46:35 1998
From: kehagias at egnatia.ee.auth.gr (Thanasis Kehagias)
Date: Tue, 29 Sep 1998 10:46:35 -0700
Subject: new papers on modular NN and DATA ALLOCATION
Message-ID: <3.0.5.32.19980929104635.007a0600@egnatia.ee.auth.gr>

NEW PAPERS

The following papers can be obtained from my WEB site. 

-------------------------------------------------------------
1. "A General Convergence Result for Data Allocation in Online Unsupervised
Learning Methods". (With V. Petridis). Poster Presentation in the Second
International Conference on Cognitive and Neural Systems, Boston
University, 1998. 
(http://skiron.control.ee.auth.gr/~kehagias/thn/thn02c05.htm)

2. "Identification of Switching Dynamical Systems Using Multiple Models".
(With V. Petridis).  In Proceedings of CDC 98, 1998. 
(http://skiron.control.ee.auth.gr/~kehagias/thn/thn02c04.htm)

3. "Unsupervised Time Series Segmentation by Predictive Modular Neural
Networks". (With V. Petridis). In Proceedings of ICANN 98, 1998. 
(http://skiron.control.ee.auth.gr/~kehagias/thn/thn02c03.htm)

4. "Data Allocation for Unsupervised Decomposition of Switching Time Series
by Predictive Modular Neural Networks". (With V. Petridis). Accepted for
Publication in the Proccedings of IFAC Conference on Large Scale Systems,
Theory and Applications, Patras, Greece, 1998. 
(http://skiron.control.ee.auth.gr/~kehagias/thn/thn02c02.htm)
-------------------------------------------------------------

All these papers deal with a common problem for which we use the term DATA
ALLOCATION. Briefly, the setup is the following: suppose a collection of
data y(1), y(2), y(3), ... is generated by more than one SOURCES. Namely,
at time t one of the sources is selected (perhaps randomly) and then the
selected source generates the next datum y(t). Now, it is required to build
a model for each source, or estimate some of its parameters and so on. NO A
PRIORI INFORMATION IS AVAILABLE regarding the number, statistical behavior
etc. of the sources. 

If the observed data were split into groups, each group containing the data
generated by one source, it would be relatively easy to train a model (e.g.
a neural network) for each source. But the problem is that the data are
UNLABELLED: no information is available as to which source generated which
datum. So the main problem is DATA ALLOCATION, i.e. the grouping of the data.

The problem is as described above; furthermore we consider an online
version of it. (Hence EM and iterative clustering approaches cannot be
used). The results are of great generality: we provide some sufficient
conditions (which can reasonably be expected to hold for a large class of
algorithms)  which guarantee CORRECT (in a precise sense) data allocation.

Our newly published BOOK (announced in a separate message) also deals with
the same problem, in greater detail (i.e. all the proofs are included).
More info can be found at my WEB site:

http://skiron.control.ee.auth.gr/~kehagias/thn/thn02b01.htm

I will also post a separate message with some additional thoughts and
biblio on the DATA ALLOCATION problem.
___________________________________________________________________
Ath. Kehagias
--Assistant Prof. of Mathematics, American College of Thessaloniki 
--Research Ass., Dept. of Electrical and Computer Eng. Aristotle Univ.,
Thessaloniki, GR54006, GREECE

--email:	 kehagias at egnatia.ee.auth.gr, kehagias at ac.anatolia.edu.gr
--web:	http://skiron.control.ee.auth.gr/~kehagias/index.htm

From nnsp99 at neuro.kuleuven.ac.be  Mon Sep 21 12:17:57 1998
From: nnsp99 at neuro.kuleuven.ac.be (NNSP '99)
Date: Mon, 21 Sep 1998 18:17:57 +0200
Subject: First call for papers for NNSP'99
Message-ID: <36067C35.8161EDFE@neuro.kuleuven.ac.be>

Dear Colleague,

If you would like to be included in our mailing list,
and receive further announcements of the NNSP99 workshop,
let us know at NNSP99 at neuro.kuleuven.ac.be

    Sincerely,

    Marc M. Van Hulle

---

*******************************
**** FIRST CALL FOR PAPERS ****
*******************************

1999 IEEE Workshop on Neural Networks for Signal Processing
August 23-25, 1999, Madison, Wisconsin

NNSP'99 homepage: http://eivind.imm.dtu.dk/nnsp99

Thanks to the sponsorship of IEEE Signal Processing Society the ninth
of a series of IEEE workshops on Neural Networks for Signal Processing
will be held at the Concourse Hotel, Madison, Wisconsin, USA.
The workshop will feature keynote addresses, technical presentations,
panel discussions and special sessions. Papers are solicited for, but
not limited to, the following areas:

Paradigms:
Artificial neural networks, support vector machines, Markov models,
graphical models, dynamical systems, evolutionary computation,
nonlinear signal processing, and wavelets.

Application Areas:
Image/speech/multimedia processing, intelligent human computer
interfaces, intelligent agents, blind source separation, OCR, robotics,
adaptive filtering, communications, sensors, system identification,
issues related to RWC, and other general signal processing and pattern
recognition.

Theories:
Generalization, design algorithms, optimization, parameter estimation,
and network architectures.

Implementations:
Parallel and distributed implementation, hardware design, and other
general implementation technologies.


SPECIAL SESSIONS
The workshop features special sessions on
* Support vector machines
* Intelligent human computer interfaces


PAPER SUBMISSION PROCEDURE
Prospective authors are invited to submit 5 copies of extended
summaries of no more than 6 pages.  The top of the first page of the
summary should include a title, authors' names, affiliations, address,
telephone and fax numbers and email address.
Camera-ready full papers of accepted proposals will be published in
a hard-bound volume by IEEE and distributed at the workshop.

Please send paper submissions to:
Jan Larsen
NNSP'99, Department of Mathematical Modelling, Building 321
Technical University of Denmark
DK-2800 Lyngby, Denmark


SCHEDULE
Submission of extended summary:            February 1, 1999
Notification of acceptance:                March 31, 1999
Submission of photo-ready accepted paper:  April 29, 1999
Advanced registration, before:             June 30, 1999


ORGANIZATION

General Chair
Yu Hen HU
University of Wisonsin-Madison
email: hu at ece.wisc.edu

Finance Chair
Tulay ADALI
University of Maryland
Baltimore County
email: adali at umbc.edu

Proceedings Chair
Elizabeth J. WILSON
Raytheon Co.
email: bwilson at ed.ray.com

Proceedings Co-Chair
Scott C. DOUGLAS
Southern Methodist University
email: douglas at seas.smu.edu

Publicity Chair
Marc van HULLE
Katholieke Universiteit Leuven
email: marc at neuro.kuleuven.ac.be

Program Chair
Jan LARSEN
Technical University of Denmark
email: jl at imm.dtu.dk

Program Committee
Tulay ADALI
Amir ASSADI
Andrew BACK
Herve BOURLARD
Andrzej CICHOCKI
Anthony G. CONSTANTINIDES
Bert DE VRIES
Scott C. DOUGLAS
Kevin R. FARRELL
Hsin-Chia FU
Ling GUAN
Jenq-Neng HWANG
Shigeru KATAGIRI
Fa-Long LUO
David MILLER
Nelson MORGAN
Klaus-Robert MULLER
Mahesan NIRANJAN
Dragan OBRADOVIC
Volker TRESP
Marc VAN HULLE

From tp at ai.mit.edu  Tue Sep 29 23:36:39 1998
From: tp at ai.mit.edu (Tomaso Poggio)
Date: Tue, 29 Sep 1998 23:36:39 -0400
Subject: Computational Neuroscience Position (Note extension of
  deadline for application)
Message-ID: <3.0.5.32.19980929233639.00b78aa0@ai.mit.edu>


                MASSACHUSETTS INSTITUTE OF TECHNOLOGY
                DEPARTMENT OF BRAIN SCIENCES

The MIT Department of Brain  Sciences anticipates making another
tenure-track appointment in computational brain and cognitive science at
the Assistant Professor level.  Candidates should have a strong
mathematical background and an active research interest in the mathematical
modeling of specific biophysical, neural or cognitive phenomena.
Individuals whose research focuses on learning and memory at the level of
neurons and networks of neurons are especially encouraged to apply.
Responsibilities include graduate and undergraduate teaching and research
supervision.  Applications should include a brief cover letter stating the
candidate's research and teaching interests, a vita, three letters of
recommendation and representative reprints.  Qualified individuals should
send their dossiers by NOVEMBER 21, 1998 to:
               Chair, Faculty Search Committee/Computational Neuroscience
               Department of Brain & Cognitive Sciences, E25-406
               MIT                77 Massachusetts Avenue
Cambridge, MA 02139-4307
Previous applicants (last year) need not resubmit their dossiers.

MIT is an Affirmative Action/Equal Opportunity Employer.  Qualified women
and minority candidates are encouraged to apply.

Tomaso Poggio
Uncas and Helen Whitaker Professor
Brain Sciences Department and A.I. Lab
M.I.T., E25-218,
45 Carleton St
Cambridge, MA 02142

E-mail: tp at ai.mit.edu
Web: <http://www.ai.mit.edu/people/poggio/>
Phone: 617-253-5230
Fax: 617-253-2964





From COSCAVR at rivendell.otago.ac.nz  Tue Sep  1 12:15:44 1998
From: COSCAVR at rivendell.otago.ac.nz (ANTHONY ROBINS.)
Date: Tue, 01 Sep 1998 16:03:44 -0012
Subject: What have neural networks achieved?
Message-ID: <01J1APXR7F4I94E2TK@rivendell.otago.ac.nz>

> From: "Randall C. O'Reilly" <oreilly at grey.colorado.edu>
>
> Another angle on the hippocampal story has to do with the phenomenon
> of catestrophic interference (McCloskey & Cohen, 1989), and the notion
> that the hippocampus and the cortex are complementary learning systems


The catastrophic forgetting (catastrophic interference, serial
learning) problem has come up in this thread.  In most neural networks
most of the time, learning new information disrupts (even eliminates)
old information.  I want to quickly describe what we think is an
interesting and general solution to this problem.

First, a comment on rehearsal.  The catastrophic forgetting 
problem can be solved with rehearsal - relearning old items as
new items are learned.  A range of rehearsal regimes have been
explored (see for example Murre, 1992; Robins, 1995; and the
"interleaved learning" referred to earlier in this thread by Jay
McClelland from McClelland, McNaughton, & O'Reilly, 1995).

Rehearsal is an effective solution as long as the previously learned
items are actually available for relearning.  It may be, however, that
the old items have been lost, or it is not practical for some reason
to store them.  In any case, retaining old items for rehearsal in a
network seems somewhat artificial, as it requires that they be
available on demand from some other source, which would seem to make
the network itself redundant.

It is possible to achieve the benefits of rehearsal, however, even
when there is no access to old items.  This "pseudorehearsal"
mechanism, introduced in Robins (1995), is based on the relearning of
artificially constructed populations of "pseudoitems" instead of the
actual old items.

In MLP / backprop type networks a pseudoitem is constructed by
generating a new input vector at random, and passing it forward
through a network in the standard way.  Whatever output vector this
input generates becomes the associated target output.  Rehearsing
these pseudoitems during new learning protects the old items in the
same way that rehearsing the real old items does.  Why does it work?
The essence of preventing catastrophic forgetting is to localise
changes to the function instantiated by the network so that it changes
only in the immediate vicinity of the new item to be learned.
Rehearsal localises changes by relearning ("fixing") the original
training data points.  Pseudorehearsal localises change by relearning
("fixing") other points randomly chosen from the function (the
pseudoitems).  (Work in progress suggests that simply using a "local"
learning algorithm such as an RBF is not enough).

Pseudorehearsal is the generation of approximations of old knowledge
to be rehearsed as needed.  The method is very effective, and has been 
further explored in a number of papers (Robins, 1996; Frean & Robins, 
1998; Ans & Rousset, 1997; French, 1997; and as a part of work 
described in Silver & Mercer, 1998).  Pseudorehearsal enables sequential 
learning (the learning of new information at any time) in a neural 
network.

Extending these ideas to dynamical networks (such as Hopfield nets),
we can rehearse randomly chosen attractors to preserve previously
learned items / attractors during new learning (Robins & McCallum,
1998). Here the distinction between rehearsal and pseudorehearsal
starts to break down, as randomly chosen attractors naturally contain
a mixture of both real old items / learned attractors and pseudoitems
/ spurious attractors.

We have already linked pseudorehearsal in MLP networks to the
consolidation of information during sleep (Robins, 1996).  In the
context of Hopfield type nets another proposed solution to
catastrophic forgetting based on unlearning spurious attractors has
also been linked to sleep (eg Hopfield, Feinstein & Palmer, 1983;
Crick & Mitchison, 1983; Christos, 1996).  We are currently exploring
the relationship between this *unlearning* and our *relearning* based
accounts.  Details of the input patterns, architecture, and learning
algorithm are all significant in determining the efficacy of the two
approaches (we think our approach has advantages, but this is work in
progress!).


References

Ans,B. & Rousset,S. (1997) Avoiding Catastrophic Forgetting by Coupling
Two Reverberating Neural Networks.  Academie des Sciences, Sciences de la
vie, 320, 989 - 997.

Christos, G. (1996) Investigation of the Crick-Mitchison Reverse-Learning
Dream Sleep Hypothesis in a Dynamic Setting. Neural Networks, 9, 427 -
434.

Crick,F. & Mitchison,G (1983) The Function of Dream Sleep.  Nature, 304,
111 -114.

Frean,M.R. & Robins,A.V. (1998). Catastrophic forgetting and
"pseudorehearsal" in linear networks. In Downs T, Frean M & Gallagher M
(Eds) Proceedings of the Ninth Australian Conference on Neural Networks
Brisbane: University of Queensland (1998) 173 - 178.

French,R.M. (1997) Pseudo-recurrent Connectionist Networks:  An Approach
to the Sensitivity Stability Dilemma.  Connection Science, 9, 353 - 380.

Hopfield,J., Feinstein, D. & Palmer,R. (1983) 'Unlearning' has a
Stabilizing Effect in Collective Memories.  Nature, 304. 158 - 159.

McClelland,J., McNaughton,B. & O'Reilly,R. (1995) Why there are 
complementary learning systems in the hippocampus and neocortex: 
Insights from the successes and failures of connectionist models of 
learning and memory.  Psychological Review, 102, 419-457.

Murre,J.M.J. (1992) Learning and Categorization in Modular Neural
Networks.  Hillsdale, NJ:  Earlbaum.

Robins,A. (1995) Catastrophic Forgetting, Rehearsal, and Pseudorehearsal.
Connection Science, 7, 123 - 146.

Robins,A. (1996)  Consolidation in Neural Networks and in the Sleeping
Brain.  Connection Science, 8, 259 - 275.

Robins, A. & McCallum, S. (1998).  Pseudorehearsal and the Catastrophic
Forgetting Solution in Hopfield Type Networks.  Connection Science, 7 :
121 - 135.

Silver,D. & Mercer,R. (1998) The Task Rehearsal Method of Sequential
Learning.  Department of Computer Science University of Western Ontario
Technical Report # 517.



Anthony Robins  ----------------------------------------------------
Computer Science                                 coscavr at otago.ac.nz
University of Otago                             ph:    +64 3 4798314
Dunedin, NEW ZEALAND                            fax:   +64 3 4798529


From wahba at stat.wisc.edu  Tue Sep  1 15:00:30 1998
From: wahba at stat.wisc.edu (Grace Wahba)
Date: Tue, 1 Sep 1998 14:00:30 -0500 (CDT)
Subject: Gaussian statistical models, Hilbert spaces
Message-ID: <199809011900.OAA14197@hera.stat.wisc.edu>


Readers of 
...............
  http://www.santafe.edu/~zhuh/draft/edmc.ps.gz 

		Error Decomposition and Model Complexity

			     Huaiyu Zhu

  Bayesian information geometry provides a general error decomposition
  theorem for arbitrary statistical models and a family of information
  deviations that include Kullback-Leibler information as a special case.
  When applied to Gaussian measures it takes the classical Hilbert space
  (Sobolev space) theories for estimation (regression, filtering,
  approximation, smoothing) as a special case.  When the statistical and
  computational models are properly distinguished, the dilemmas of
  over-fitting and ``curse of dimensionality'' disappears, and the optimal
  model order disregarding computing cost is always infinity.
.............

  will do doubt be interested in the long history of the relationship 
  between reproducing kernel Hilbert spaces (rkhs), gaussian measures  
  and regularization,  -
    see 
    
    1962 Proccedings of the Symposium on Time Series Analysis 
    edited by Murray Rosenblatt, Wiley 1962, esp. the paper by Parzen
    1962 J. Hajek On linear statistical problems in stochastic processes
	 Czech Math J. v 87. 
    1971 Kimeldorf and Wahba, Some results on Tchebycheffian spline functions, 
    J. Math Anal. Applic. v 33.
    1990 G. Wahba, Spline Models for Observational Data, SIAM 
    1997 F. Girosi, An equivalence between sparse approximation and 
      support vector machines, to appear Neural Comp
    1997 G. Wahba, Support vector vachines, reproducing kernel Hilbert 
      spaces and the randomized GACV, to appear, Schoelkopf, Burges 
      and Smola, eds, forthcoming book on Support Vector Machines, MIT Press 
    1981 C. Micchelli and G. Wahba, Design problems for optimal
      surface interpolation, Approximation Theory and Applications, 
      Z. Ziegler ed, Academic press. Also numerous works by 
      L. Plaskota and others on optimal bases. First k eigenfunctions 
      of the reproducing kernel are well known to have certain 
      optimal properties under restricted circumstances, 
      see e.g. Ch 12 of Spline Models and references there, but 
      if there are n observations, then the Bayes estimates 
      are found in an at most  n-dimensional subspace of the rkhs 
      associated with the prior, KW 1971.   B. Silverman 1982
      `On the estimation of a probability density fuction by 
      the maximum penalized likelihood method', Ann. Statist 1982 
      will also be of interest - convergence rates are related
      to the rate of decay of the eigenvalues of the reproducing
      kernel. 
	     Grace Wahba http://www.stat.wisc.edu/~wahba/  


From ericr at ee.usyd.edu.au  Tue Sep  1 21:11:01 1998
From: ericr at ee.usyd.edu.au (Eric Ronco)
Date: Wed, 2 Sep 1998 11:11:01 +1000
Subject: Gated Modular Neural Networks for Control Oriented Modelling
Message-ID: <199809020111.LAA00815@merlot.ee.usyd.edu.au.usyd.edu.au>

Dear Connectionists,

A new technical report can be found in the on-line data base
of the Systems and Control Laboratory of Sydney University.
 
Its title is: Gated Modular Neural Networks for Control Oriented
Modelling.

The (gziped) PostScript file is accessible at:

       http://www.ee.usyd.edu.au/~ericr/pub

or alternatively at

       http://merlot.ee.usyd.edu.au/tech_rep/


The authors are: Eric Ronco, Peter J. Gawthrop and David J. Hill.

The keywords are: Non-linear modelling and control; Neural Network;
Modularity; 

The abstract:

This study is an attempt to review the main ``Gated Modular Neural
Networks'' (GMNNs) which are particularly suitable for modelling
oriented toward control. A GMNN consists of a network of computing
modules and a gating system. The computing modules are the operating
part of the architecture and corresponds to linear or simple
non-linear models or controllers. The simplicity of the computing
modules is the fundamental advantage of this approach as this often
clarifies and simplifies the modelling and control design. The gating
system is used for the selection of the computing module(s) valid for
the current state of the plant. Three types of GMNN are distinguished
according to the gating strategy they implement which are respectively
based on spatial clustering, modelling performance and probable
performance of the computing modules. The modelling oriented control
properties of these approaches are assessed in terms of adaptability,
performance, control design, implementation and analysis
properties. Conclusion are drawn to highlight required future works to
generalise the applicability of these approaches.

 
The number of this technical report is: EE-98009

Bye,

Eric
-------------------------------------------------------------------
Eric Ronco, PhD                    Tel: +61 2 9351 7680
Dt of Electrical Engineering       Fax: +61 2 9351 5132
Bldg J13, Sydney University        Email: ericr\@ee.usyd.edu.au
NSW 2006, Australia                http://www.ee.usyd.edu.au/~ericr
-------------------------------------------------------------------

From szepes at iserv.iki.kfki.hu  Wed Sep  2 04:57:40 1998
From: szepes at iserv.iki.kfki.hu (Csaba Szepesvari)
Date: Wed, 2 Sep 1998 10:57:40 +0200 (MET)
Subject: new publications
Message-ID: <Pine.A32.3.91.980902105239.22745A-100000@iserv.iki.kfki.hu>

Dear Connectionists,

I have updated my homepage which now contains a link to my
online publications. Below you can find a list of the publications
(title, abstract). They can be accessed from the page
http://sneaker.mindmaker.kfkipark.hu/~szepes/research/OnlinePubs.htm
in the form of gzipped postscript files.

Yours Sincerely,

  Csaba Szepesvari

====================================Reinforcement Learning & Markovian Decision Problems
-------------------------------------------------------------------------

Module-Based Reinforcement Learning: Experiments with a Real Robot

Zs. Kalmár, Cs. Szepesvári and A. Lorincz
Machine Learning 31:55-85, 1998. ps

Autonomous Robots 5:273-295, 1998. (this was a joint Special Issue of the
two journals..)

The behavior of reinforcement learning (RL) algorithms is best understood
in completely observable, discrete-time controlled Markov chains with
finite state and action spaces. In contrast, robot-learning domains are
inherently continuous both in time and space, and moreover are partially
observable. Here we suggest a systematic approach to solve such problems
in which the available qualitative and quantitative knowledge is used to
reduce the complexity of learning task. The steps of the design process
are to: i) decompose the task into subtasks using the qualitative
knowledge at hand; ii) design local controllers to solve the subtasks
using the available quantitative knowledge and iii) learn a coordination
of these controllers by means of reinforcement learning. It is argued
that the approach enables fast, semi-automatic, but still high quality
robot-control as no fine-tuning of the local controllers is needed. The
approach was verified on a non-trivial real-life robot task. Several RL
algorithms were compared by ANOVA and it was found that the model-based
approach worked significantly better than the model-free approach. The
learnt switching strategy performed comparably to a handcrafted version.
Moreover, the learnt strategy seemed to exploit certain properties of the
environment which were not foreseen in advance, thus supporting the view
that adaptive algorithms are advantageous to non-adaptive ones in complex
environments.

Non-Markovian Policies in Sequential Decision Problems

Cs. Szepesvári
Acta Cybernetica (to appear) 1998. ps

In this article we prove the validity of the Bellman Optimality Equation
and related results for sequential decision problems with a general
recursive structure. The characteristic feature of our approach is that
also non-Markovian policies are taken into account. The theory is
motivated by some experiments with a learning robot.

Convergence Results for Single-Step On-Policy Reinforcement-Learning
Algorithms

S. Singh, T. Jaakkola, M.L. Littman and Cs. Szepesvári
Machine Learning, to appear, 1998. ps

An important application of reinforcement learning (RL) is to
finite-state control problems and one of the most difficult problems in
learning for control is balancing the exploration/exploitation tradeoff.
Existing theoretical results for RL give very little guidance on
reasonable ways to perform exploration. In this paper, we examine the
convergence of single-step on-policy RL algorithms for control. On-policy
algorithms cannot separate exploration from learning and therefore must
confront the exploration problem directly. We prove convergence results
for several related on-policy algorithms with both decaying exploration
and persistent exploration. We also provide examples of exploration
strategies that can be followed during learning that result in
convergence to both optimal values and optimal policies.

Multi-criteria Reinforcement Learning

Z. Gábor, Zs. Kalmár and Cs. Szepesvári
In Proceedings of International Conference of Machine Learning, in press,
1998. ps

We consider multi-criteria sequential decision making problems where the
vector-valued evaluations are compared by a given, fixed total ordering.
Conditions for the optimality of stationary policies and the Bellman
optimality equation are given for a special, but important class of
problems when the evaluation of policies can be computed for the criteria
independently of each other. The analysis requires special care as the
topology introduced by pointwise convergence and the order-topology
introduced by the preference order are in general incompatible.
Reinforcement learning algorithms are proposed and analyzed. Preliminary
computer experiments confirm the validity of the derived algorithms.
These type of multi-criteria problems are most useful when there are
several optimal solutions to a problem and one wants to choose the one
among these which is optimal according to another fixed criterion.
Possible application in robotics and repeated games are outlined.

Multi-criteria Reinforcement Learning

Z. Gábor, Zs. Kalmár and Cs. Szepesvári
Technical Report TR-98-115, "Attila József" University, Research Group on
Artificial Intelligence Szeged, HU-6700, 1998 (submitted in 1997). ps

We consider multi-criteria sequential decision making problems where the
vector-valued evaluations are compared by a given, fixed total ordering.
Conditions for the optimality of stationary policies and the Bellman
optimality equation are given for a special, but important class of
problems when the evaluation of policies can be computed for the criteria
independently of each other. The analysis requires special care as the
topology introduced by pointwise convergence and the order-topology
introduced by the preference order are in general incompatible.
Reinforcement learning algorithms are proposed and analyzed. Preliminary
computer experiments confirm the validity of the derived algorithms.
These type of multi-criteria problems are most useful when there are
several optimal solutions to a problem and one wants to choose the one
among these which is optimal according to another fixed criterion.
Possible application in robotics and repeated games are outlined.

The Asymptotic Convergence-Rate of Q-learning

Cs. Szepesvári
In Proceedings of Neural Information Processing Systems 10, pp.
1064-1070, 1997. ps

In this paper we show that for discounted MDPs with discount factor
\gamma>1/2 the asymptotic rate of convergence of Q-learning is
O(1/t^{R(1-\gamma)}) if R(1-\gamma)<1/2 and O(\sqrt{\log\log t/ t})
otherwise provided that the state-action pairs are sampled from a fixed
probability distribution. Here R=\pmin/\pmax is the ratio of the minimum
and maximum state-action occupation frequencies. The results extend to
convergent on-line learning provided that \pmin>0, where \pmin and \pmax
now become the minimum and maximum state-action occupation frequencies
corresponding to the stationary distribution.

Learning and Exploitation do not Conflict Under Minimax Optimality

Cs. Szepesvári
In Proceedings of 9th European Conference of Machine Learning, pp.
242-249, 1997. ps

We show that adaptive real time dynamic programming extended with the
action selection strategy which chooses the best action according to the
latest estimate of the cost function yields asymptotically optimal
policies within finite time under the minimax optimality criterion. From
this it follows that learning and exploitation do not conflict under this
special optimality criterion. We relate this result to learning optimal
strategies in repeated two-player zero-sum deterministic games.

A unified analysis of value-function-based reinforcement-learning algorithms

Cs. Szepesvári and M. L. Littman

Submitted for review, 1997 ps

Reinforcement learning is the problem of generating optimal behavior in a
sequential decision-making environment given the opportunity of
interacting with it. Many algorithms for solving reinforcement-learning
problems work by computing improved estimates of the optimal value
function. We extend prior analyses of reinforcement-learning algorithms
and present a powerful new theorem that can provide a unified analysis of
value-function-based reinforcement-learning algorithms. The usefulness of
the theorem lies in how it allows the asynchronous convergence of a
complex reinforcement-learning algorithm to be proven by verifying that a
simpler synchronous algorithm converges. We illustrate the application of
the theorem by analyzing the convergence of Q-learning, model-based
reinforcement learning, Q-learning with multi-state updates, Q-learning
for Markov games, and risk-sensitive reinforcement learning.

Some basic facts concerning minimax sequential decision processes

Cs. Szepesvári
Technical Report TR-96-100, "Attila József" University, Research Group on
Artificial Intelligence Szeged, HU-6700, 1996. ps

It is shown that for discounted minimax sequential decision processes the
evaluation function of a stationary policy is the fixed point of the
so-called policy evaluation operator which is a contraction mapping.
Using this we prove that Bellman's principle of optimality holds. We also
prove that the asynchronous value iteration algorithm converges to the
optimal value function.

Adaptive (Neuro-)Control
---------------------------------------------------------------------
Uncertainty and Performance of Adaptive Controllers for Functionally
Uncertain Output Feedback Systems

M. French, Cs. Szepesvári and E. Rogers
In Proc. of 1998 IEEE Conference on Decision and Decision, 1998. ps

We consider nonlinear systems in an output feedback form which are
functionally known up to a L2 measure of uncertainty. The control task is
to drive the output of the system to some neighbourhood of the origin. A
modified L2 measure of transient performance (penalising both state and
control effort) is given, and the performance of a class of model based
adaptive controllers is studied. An upper performance bound is derived.

Uncertainty, Performance and Model Dependency in Approximate Adaptive
Nonlinear Control

M. French, Cs. Szepesvári and E. Rogers
In Proc. of 1997 IEEE Conference on Decision and Decision, San Diego,
California, 1997. ps

We consider systems satisfying a matching condition which are
functionally known up to a L2 measure of uncertainty. A modified L2
performance measure is given, and the performance of a class of model
based adaptive controllers is studied. An upper performance bound is
derived in terms of the uncertainty measure and measures of the
approximation error of the model. Asymptotic analyses of the bounds under
increasing model size are undertaken, and sufficient conditions are given
on the model that ensure the performance bounds are bounded independent
of the model size.

Uncertainty, Performance and Model Dependency in Approximate Adaptive
Nonlinear Control

M. French, Cs. Szepesvári and E. Rogers
Submitted for journal publication 1997. ps

We consider systems satisfying a matching condition which are
functionally known up to weighted L2 and L-infinity measures of
uncertainty. A modified LQ measure of control and state transient
performance is given, and the performance of a class of approximate model
based adaptive controllers is studied. An upper performance bound is
derived in terms of the uncertainty models (stability and the state
transient bounds require only the L2 uncertainty model; control effort
bounds require both L2 and L-infinity uncertainty models), and various
structural properties of the model basis. Sufficient conditions are
sgiven to ensure that the performance is bounded independent of the model
basis size.



From brander at csee.uq.edu.au  Wed Sep  2 14:28:01 1998
From: brander at csee.uq.edu.au (Rafael Brander)
Date: Thu, 3 Sep 1998 04:28:01 +1000 (EST)
Subject: What have neural networks achieved?
In-Reply-To: <01J1APXR7F4I94E2TK@rivendell.otago.ac.nz>
Message-ID: <Pine.GSO.3.95q.980903042457.15706A-100000@fife>


These look like achievements of neural networks (see detail below):

(1) Suggested by Randall O'Reilly,  "...the neural network
approach provides a principled basis for understanding why we have a
hippocampus, and what its functional characteristics should be."

The catastrophic interference literature has also given a plausible
explanation -- sparseness -- for why, given that our brains look very
much like neural networks, we have any memory at all.

(2) I think that the catastrophic interference and generalisation
literature suggests the possibility that in order to have generalisation
capability, the human weakness with discrete symbolic memory may be an
inevitability -- in artificial as well as biological computers. This
bolsters the common view that AI will only be achieved with
human-comparable computer hardware. It also explains the typical human
complaint "why do I have such a terrible memory..." (relative to
silicon chip computers).

Apologies for length of this email, see my Masters abstract at bottom.
Rafael Brander.

Randall O'Reilly wrote:

"Another angle on the hippocampal story has to do with the phenomenon
of catestrophic interference (McCloskey & Cohen, 1989), and the notion
that the hippocampus and the cortex are complementary learning systems
that each optimize different functional objectives (McClelland,
McNaughton, & O'Reilly, 1995).  In this case, the neural network
approach provides a principled basis for understanding why we have a
hippocampus, and what its functional characteristics should be.
Interestingly, one of the "sucesses" of neural networks in this case
was their dramatic failure in the form of the catestrophic
interference phenomenon.  This failure tells us something about the
limitations of the cortical memory system, and thus, why we might need
a hippocampus."


I agree. And further to this, research results involving sparse vectors
show that hippocampally realistic settings of certain of the parameters
of an otherwise standard MLP are sufficient to equal the
intermediary-term memory performance of human subjects. In other words,
current knowledge of sparse neural networks is actually consistent with
human intermediary-term memory performance, which is associated with the
hippocampus.
See bottom of email for my Masters abstract.

Jay McClelland wrote:

[text deleted]
"To allow rapid learning of the contents of a particular experience,
the arguement goes, a second learning system, complementary to the
first [neocortex..],
 is needed; such a system has a higher learning rate and recodes
inputs using what we call 'sparse, random conjunctive coding' to
minimize interference (while simultaneously reducing the adequacy of
generalization).  These characteristics are just the ones that appear
to characterize the hippocampal system: it is the part of the brain
known to be crucial for the rapid learning of the contents of specific
experiences; it is massively plastic; and neuronal recording studies
indicate that it does indeed use sparse, random conjunctive coding."


My research mentioned above, which found simple parameters allowing a
sparse network to equal human intermediary-term memory performance on
the tasks studied confirm Jay's comments. He also refers to sparseness
"simultaneously reducing the adequacy of generalization". I also studied
the issue of this generalisation/memory trade-off in my thesis. I found
that indeed generalisation can be wiped out by sparseness if the domain
is combinatorial. By *combinatorial*, I mean where any input features can
be combined freely in any combination to form an input vector. I also
found, affirming results of French and Lewandowsky, that generalisation
was not affected in simpler, noncombinatorial (actually classification)
domains.

This problem in combinatorial domains of sparseness damaging
generalisation, a classic utility of neural networks, is a bit
discouraging for applications. However, any working algorithm must find
a way to both sufficiently separate memories to avoid interference, and 
to combine them sensibly to perform tasks requiring recognition of shared
features. In this vein, current knowledge actually suggests that the
human brain has traded off detailed discrete memory against
generalisation capability. Relative to conventional silicon chip Von
Neumann computers, everyone is aware of human frustration with detailed
discrete symbolic memory. This, despite the colossal hardware available
in the human brain. Note that the "catastrophic interference" of naive
networks referred to in the literature is catastrophic relative to
humans; but humans are also catastrophic relative to computers. This
frustration relative to silicon chip computers may stem from
*partially* overlapping semantic feature based representations, which may
be necessary for generalisation and content addressability. According to
the literature on catastrophic interference in artificial networks,
overlap the representations too much and memory disappears; too little,
and generalisation vanishes (at least in a combinatorial domain).
Humans can perform generalisation on tasks spanning periods well under
the few months or years that memories take to become established
in the neocortex, hence the hippocampus may be involved. This line of
thinking, based on the catastrophic interference literature, suggests
the possibility that in order to have generalisation capability, the
human weakness with discrete symbolic memory may be an inevitability --
in artificial as well as biological computers. Most AI researchers
believe that they will need human-comparable computer hardware to achieve
human level performance; I think this is further evidence for that view.


Bryan Thompson wrote:

    Max writes,

    Think about the structure of this argument for a moment. It runs
    thus:

    1. Neural networks suffer from catastrophic interference.
    2. Therefore the cortical memory system suffers from catastrophic
    interference.  3. That's why we might need a hippocampus.

    Is everyone happy with the idea that (1) implies (2)?

    Max
                  max at currawong.bhs.mq.edu.au

  "I am not happy with the conclusion (1), above.  Catastrophic
  interference is a function of the global quality of the weights
  involved in the network.  More local networks are, of necessity, less
  prone to such interference as less overlapping subsets of the weights
  are used to maps the transformation from input to output space.  Modifying
  some weights may have *no* effect on some other predictions. In
  the extreme case of table lookups, it is clear that catastropic
  interference completely disappears (along with most options for
  generalization, etc.:) In many ways, it seems that this statement is
  true for supervised learning networks in which weights are more global
  than not.  Other, more practical counter examples would include
  (differentiable) CMACs and radial basis function networks.
  (text deleted..)"


This mostly ties in. But a couple of elaborations. Although sparser
networks in earlier research always reduced interference, the memory
performance of multilayered sparse networks was generally much below
what one might intuitively expect (see Masters abstract below). The
reasons for this I explain below, but if you set up the network
correctly then your comments about progressively more severe sparseness
reducing interference are correct. Regarding radial basis function
networks, I might expect an RBF net with narrow activation bases to be
analogous to sparse MLPs, although Robins' work in progress (just below)
seems pessimistic about its memory capabilities. Some of the unexpected
but correctable problems which I found with sparse networks might have
their analogues in narrowed RBFs.


Anthony Robins wrote:

[stuff deleted]...
In any case, retaining old items for rehearsal in a
network seems somewhat artificial, as it requires that they be
available on demand from some other source, which would seem to make
the network itself redundant.

[I agree. Retaining items for rehearsal requires memory overhead in a
system which is supposed to be trying to optimise memory...]

It is possible to achieve the benefits of rehearsal, however, even
when there is no access to old items.  This "pseudorehearsal"
mechanism, introduced in Robins (1995), is based on the relearning of
artificially constructed populations of "pseudoitems" instead of the
actual old items.

In MLP / backprop type networks a pseudoitem is constructed by
generating a new input vector at random, and passing it forward
through a network in the standard way.  Whatever output vector this
input generates becomes the associated target output.
[stuff deleted] ....
(Work in progress suggests that simply using a "local"
learning algorithm such as an RBF is not enough).

We have already linked pseudorehearsal in MLP networks to the
consolidation of information during sleep (Robins, 1996).
[stuff deleted]...


Considerations of some kind of rehearsal for consolidation of information
during sleep, and over the long-term in general sound very interesting.
Regarding shorter memory tasks, say over a number of minutes such
as the ones I studied in my Masters, it seems less likely to me that
the brain would have the time and memory capacity to implement
pseudorehearsal. From some of your data, rather large pseudopopulation
overhead seems to be needed to slow the swinging off-target (as shown by
dot products with target) of the original learned output vectors.


The abstract of my Masters thesis is appended below, but I mention a
few bits of extra relevant detail here. As mentioned above, it was
successfully demonstrated that hidden-layer sparseness, in combination
with three other hippocampally realistic network conditions, can
eliminate catastrophic interference. Some of the simulations were a
resolution of McCloskey and Cohen's [1989] expose of catastrophic
interference. The other three necessary anti-interference factors were
context dominance (for context-dependent tasks of course), initial
weightsize and the bias influence.
Context dominance refers to large (i.e. larger than 1) context unit
activation values.  It was set to 4 for the human-equal
context-dependent simulation, which actually matched the whole human
forgetting curve well. Comparing with the hippocampus, there is no easy
way to determine just how much "attention" it pays to list context. Much
larger than normal initial weightsizes -- at values typical of
in-training sizes -- were also found to be necessary for both tasks in
avoiding catastrophic interference. One would expect weight sizes in the
brain to always be at "in-training" sizes. Finally, a small bias
influence relative to the input layer, such as is typically found in
large networks (the hippocampus is a huge network), was required.

Here is a summarised explanation for how the four factors
influence interference. For sparseness, it's the obvious one given many
times by other researchers; sparseness reduces overlap, thus reduces
weight-learning interference. However, in my simulations sparseness by
itself only ever got first list retention just off the floor;
interference from the second list was still catastrophic. For the
context-dependent task, the most important other factor as context
dominance, which works as follows. If context activation values are too
small, switching list context is not going to change the total summed
inputs to the hidden layer by much. Consequently, the network will
train mostly the same weights for second list items as it previously
did for the first list, wiping out memory of list 1. Turning to initial
weightsizes. Firstly, training on list 1 naturally pushed the weights
most involved to the "intraining" sizes -- much greater than traditional
initialisations. During list 2 training, backpropagation of error
through these enlarged weights was far greater than through small
weights, which directly encouraged the new associations to be learned
using those same weights over again. Initialising weights around the
in-training range removed this quite substantial interference effect.
Regarding bias influence. In a small model network, the bias for each
node, traditionally set to 1, is significant in comparison to the
previous layer's fan-in to the node. When a hidden node is "switched
on" during early training (list 1), its bias is naturally increased to
speed up the training. The result is that the hidden node's activation
threshold is now low, and it is therefore much more likely to be
turned on during the learning of list 2. Thus early and late training
tends strongly to use the same hidden nodes, increasing interference.
This problem vanished for smaller bias settings.

It was not obvious that low initial weight-sizes and high bias influence
could cause substantial interference, and I point out that these factors
should obtain across a wide variety of commonly used networks.

I'll probably submit material from the Masters catastrophic interference
sections for publication in the near future (and probably some SOM
stuff too). If someone asks me about it, I can put the latest version of
the thesis on the web in a few weeks, I'm finishing some
examiner-initiated modifications. The thesis has a great deal more
references on catastrophic interference than I've appended below.

\title{On Sparse Neural Network Representations: Catastrophic Interference,
 Generalisation, and Combinatoriality}
\author{Rafael Antony Brander\\
 B.Sc.(Hons. Pure Math; Hons. Applied Math), G.Dip.(Comp.Sci.)

{\it A thesis submitted for the degree of Master of Science}

School of Information Technology\\
The University of Queensland\\
Australia}
\date{September 30, 1997}

Abstract:

  Memory is fundamental to human psychology, being necessary for any
thought processing. Hence the importance of research involving human
memory modelling, which aims to give us a better understanding of how the
human memory system works. Learning and forgetting are of course the most
important aspects of memory, and any worthwhile model of human memory
must be expected to exhibit these phenomena in a manner qualitatively
similar to that of a human. It has been claimed (see below) that standard
artificial neural networks cannot fulfil this elementary expectation,
suffering from {\it catastrophic interference}, and sparseness of neural
network representations has been employed, directly and indirectly, in
many of the attempts to answer this claim. Part of the motivation for the
employment of sparseness has been the fact that sparse vectors have been
observed in human neurological memory systems [Barnes, McNaughton,
Mizumori, Leonard and Lin, 1990]. In the broader field of neural
networks, sparse representations have recently become a popular tool of
research. This thesis aimed to discover what fundamental properties of
sparseness might justify the counter-claims alluded to above, and in so
doing uncovered a more general characterisation of the effects of sparse
vector representations in neural networks.

As yet, little formal knowledge of the concept of sparseness itself has
been reported in the literature; we developed some formal definitions and
measures, which served as foundational background theory for the thesis.
We also discussed several representative methods for implementing
sparsification.

We initially conjectured that the main problem of sparsification in the
case of a boolean space might be that of finding ``base'' clusters
without being concerned with orientation with respect to an origin. This
pointed us towards a clustering algorithm. We employed simulations and
theory to show that a particular sparse representation, which we derived
from a neural network cluster-based learning algorithm, the Self
Organising Map ({\it SOM}) [Kohonen 1982], is an ineffective basis for
even simple learning and generalisation tasks, {\it in combinatorial
domains}. The SOM is generally regarded as a good performer in
classification tasks, which is the noncombinatorial domain.

We then turned to the well known problem referred to earlier, where
neural networks are observed to fail to model a basic fundamental
property of human memory. Since McCloskey and Cohen [1989] and Ratcliff
[1990] first brought it to the attention of neural network researchers,
the problem of {\it catastrophic interference} in standard feedforward,
multilayer, backpropagation neural network models of human memory has
continued to generate research aimed at its resolution. Interference is
termed ``catastrophic'' when the learning of a second list almost
completely removes memory of a list learned earlier, and when forgetting
of this extreme nature does not occur in humans.

In previous research [French, 1991; McRae \& Hetherington, 1993],
sparseness at the hidden layer, either directly or indirectly induced,
has been shown to substantially reduce catastrophic interference in
memory tasks where no context is required to distinguish between the two
lists.

Our interference studies investigated the degree to which sparsification
algorithms can eliminate the serious problem of catastrophic
interference, by virtue of a comparison to the human performance data
[Barnes \& Underwood, 1959; Barnes-McGovern, 1964; Garskof, 1968], a
match to  which data had not yet been achieved with standard MLPs in the
literature. These studies investigated both the AB AC and AB CD list
learning paradigms, which represent instances of context dependent and
context independent tasks respectively.

It was successfully demonstrated that sparseness, in combination with
three other realistic network conditions, can eliminate catastrophic
interference. The other three necessary anti-interference factors were
context dominance, initial weightsize and the bias influence. Context
dominance was here definitionally implemented as setting the context
units to have large (i.e. larger than 1) activation values. Much larger
than normal initial weightsizes -- at values typical of in-training sizes
 -- were also found to be necessary in avoiding catastrophic
interference. Finally, a small bias influence relative to the input
layer, such as is typically found in large networks, was required. The
explanation for sparseness' removal of catastrophic interference was
argued to be that it reduces relative overlap between unrelated vectors.

However, it is believed [French, 1991; McRae \& Hetherington, 1993;
Lewandowsky, 1994; Sharkey \& Sharkey, 1995] that there is a trade-off
between sparseness and generalisation. We also addressed this trade-off
issue, and showed that the sparsification algorithms used above to
eliminate catastrophic interference concomitantly incur a great loss in
a neural network's generalisation capability {\it in combinatorial
domains}; although there was no such loss (as agreed by French [1992] and
Lewandowsky [1994]) if the domain was noncombinatorial.

Combining the results of the studies of the SOM and effects of sparseness
on generalisation, we suggested that sparseness has no effect on learning
or generalisation in noncombinatorial domains, and that in combinatorial
domains generalisation is removed while learning can only occur by
exhaustively training in a supervised scheme on all exemplars. Further,
it was shown that a more abstract result predicts -- and gives an
intuitive explanation for -- the specific results of all our experiments
discussed above. This more abstract, and intuitively expected result is
the following: {\it sparseness at the hidden layer of a standard network
has the effect, in combinatorial domains in particular, of reducing the
network's general operational similarity dependence on the similarity of
input vectors}.

The results of this thesis clarify understanding of the way the human
memory system works, which is of interest in psychology and neuroscience.
They also provide important insights into the functionality of the SOM
and the MLP.

[Barnes \& Underwood, 1959]{BU} Barnes, J. M. and Underwood, B.
 J. (1959).
``Fate'' of first-list associations in transfer theory.
 {\it Journal of Experimental Psychology}, {\bf 58}(2), 97-105.

[Barnes-McGovern, 1964] Barnes-McGovern, J. M. (1964).
Extinction of associations in four transfer paradigms.
 {\it Psychological Monographs: General and Applied}, Whole No. 593,
 {\bf 78}(16), 1-21.

[Barnes et al., 1990]{Ba} Barnes, C. A., McNaughton, B. L.,
 Mizumori, S. J.
 and Lim, L. H. (1990).
 Comparison of spatial and temporal characteristics of neuronal 
activity in sequential stages of hippocampal processing.
  {\it Progress in Brain Research}, {\bf 83}, 287-300.

[French, 1991]{F91} French, R. (1991).
 Using semi-distributed representations to overcome 
	catastrophic forgetting in connectionist networks.
 {\it Proceedings 
	of the 13th Annual Conference of the Cognitive Science Society},
 173-178. Hillsdale, NJ: Erlbaum.

[French, 1992]{F} French, R. M. (1992).
Semi-distributed representations and catastrophic
 forgetting in connectionist networks.                      
{\it Connection Science}, {\bf 4}, (3/4), 365-378.

[Garskof, 1968] Garskof, B. E. (1968).
Unlearning as a function of degree of interpolated learning and method of
testing in the A-B, A-C and A-B, C-D paradigms.
 {\it Journal of Experimental Psychology}, {\bf 76}(4), 579-583.

[Kohonen, 1982]{K} Kohonen, T. (1982).
 Self-organised formation of topologically correct feature maps.
{\it Biological Cybernetics}, {\bf 43}, 59-69.

[Lewandowsky, 1994]{L94} Lewandowsky, S. (1994).
 On the relation between catastrophic
	interference and generalization in connectionist networks.
Journal of Biological Systems, Vol. 2(3), 307-333.

[McCloskey \& Cohen, 1989]{MC} McCloskey, M. and Cohen, N. J.
 (1989).
  Catastrophic interference in connectionist networks: the
sequential learning problem.
In (Ed.), G. H. Bower,
{\it The Psychology of Learning and Motivation}, {\bf 24}, 109-165.

[McRae \& Hetherington, 1993]{MH}  McRae, K. and 
Hetherington, P. A. (1993).
 Catastrophic interference is eliminated in pretrained networks.
 {\it Proceedings of the
Fifteenth Annual Conference of the Cognitive Science Society}, 723-728.
Hillsdale, NJ: Erlbaum.

[Ratcliff, 1990]{R} Ratcliff, R. (1990).
  Connectionist models of recognition memory: constraints imposed
 by learning and forgetting functions.
{\it Psychological Review}, {\bf 97}(2), 285-308.

Sharkey, NE and Sharkey, AJC (1995).
 An analysis of Catastrophic Interference.
Connection Science, 7, 301-329 


From murre at psy.uva.nl  Wed Sep  2 15:12:13 1998
From: murre at psy.uva.nl (Jaap Murre)
Date: Wed, 02 Sep 98 21:12:13 0200
Subject: What have neural networks achieved?
Message-ID: <199809021909.AA19979@uvapsy.psy.uva.nl>

Max Coltheart wrote:

	Suppose a net learned Task A to criterion and then was trained on Tas
	B to criterion without any further exposure to A (no interleaving,
	nothing corresponding to rehearsal of A). Then retest on A will reveal
	catastrophic forgetting.

	What happens to people here? If I spend 1998 learning to play golf,
	and 1999 learning to play tennis and doing *nothing at all about golf*,
	I would not expect my golf game to be have been completely blown away
	when I try it again on Jan 1, 2000.

	Isn't this a big difference between how neural nets learn and how
	people learn?

	Circularity needs to be avoided here e.g. it would not be good to
	reply: If your golf game is still there, you must have been rehearsing
	it in 1999.

If one only focusses on one element of current hippocampal models this 
circularity may appear.

In fact, there are two independent problems here:

1. How neural networks are able to learn sequential tasks without much 
interference.

2. How the brain (e.g., hippocampus-cortex architecture) accomplishes this.

Neural networks that have very distributed architectures (such as 
backpropagation) will tend to show catastrophic interference. That is,
when compared to the human data (e.g., Osgood, 1949), they will either
show too much forgetting or--what is less well known--they will show 
too *much* learning (Murre, 1996a). As was pointed out in the debate, 
this effect can be reduced by interleaved learning of various kinds, 
bringing the network behavior in line with the human data. (It can also 
be reduced by using localist, modular or semi-distributed architectures.)

Hippocampus models that deal with the effects of hippocampal lesions,
must be able to explain why such lesions tend to obliterate *recent* rather
than old memories (called the Ribot effect). In normal forgetting these
recent memories are most readily accessible; under lesioning they are 
the first to go. This is typically explained by assuming (1) that 
memories are first stored in or via the hippocampus (or medial
temporal lobe complex) and (2) that there is a process of consolidation
whereby the memories are strengthened at a cortical level. At least
three models have been published with neural network simulations of such 
a process (Alvarez and Squire, 1994; McClelland, McNaughton, and O'Reilly,
1995; Murre, 1996b). Consolidation in these models is implemented by
selecting representations in the hippocampus by some random process and
giving them extra learning trials at a cortical level. (There is also
some process by which representations are gradually lost from the
hippocampus.) Some neurobiological data exists supporting such a
process (Wilson and McNaughton, 1994).

McClelland et al. use the catestrophic interference effect as an 
in-principle argument why this consolidation process exists, which
resembles interleaved learning. Other arguments have been put 
forward. We, for example, stress the fact that the cortex has a 
'connectivity problem' making it somewhat time-consuming to set up 
the long-range connections that underlie episodic memories (Murre 
and Sturdy, 1995). 

Evidence for the consolidation process is still a little thin 
at a neurobiological level. Some new data in neuropsychology has
recently emerged that seems to carry the thought experiment
"What would happen if consolidation could *not* take place, 
i.e., the case where the brain remains dependent primarily on the 
representations in the hippocampus (and some remnants in the 
cortex). This seems to be the case in a newly discovered form of 
dementia, called semantic dementia, whereby the semantic 
representations disappear but the episodic memory remains relatively 
preserved. On the basis of modelling work, we have predicted and 
found several new characteristics of semantic dementia(Graham and 
Hodges, 1997; Murre, Graham, and Hodges, submitted). 

Though there is clearly an enormous amount of work to be done, I
think that it is fair to say that neural network models have
contributed and continue to contribute towards our understanding 
of human (and animal)learning and memory and that one cannot rule 
out hippocampus/amnesia models on the basis of circularity.


References

Alvarez, P., & L.R. Squire (1994). Memory consolidation and the 
	medial temporal lobe: a simple network model. Proceedings 
	of National Academy of Sciences (USA), 91, 7041-7045.

Graham, K.S., & J.R. Hodges (1997). Differentiating the roles of 
	the hippocampal complex and the neocortex in long-term memory 
	storage: evidence from the study of semantic dementia and 
	Alzheimer's disease, Neuropsychology, 11, 1-13.

McClelland, J.L., B.L., McNaughton, & R.C. O'Reilly (1995). Why 
	there are complementary learning systems in the hippocampus 
	and neocortex: insights from the successes and failures of 
	connectionist models of learning and memory. Psychological 
	Review, 102, 419-457.

Murre, J.M.J. (1996a). Hypertransfer in neural networks. Connection 
	Science, 8, 225-234.

Murre, J.M.J. (1996b). TraceLink: a model of amnesia and consolidation 
	of memory. Hippocampus, 6, 675-684.

Murre, J.M.J., & D.P.F. Sturdy (1995). The connectivity of the brain: 
	multi-level quantitative analysis. Biological Cybernetics, 
	73, 529-545.

Murre, J.M.J., K.S. Graham and J.R. Hodges (submitted). Semantic 
	dementia: new constraints on connectionist models of long-term 
	memory. Submitted to Psychological Bulletin.

Osgood, C.E. (1949). The similarity paradox in human learning: a 
	resolution. Psychological Review, 56, 132-143.

Wilson, M.A., & B.L. McNaughton (1994). Reactivation of hippocampal 
	ensemble memories during sleep. Science, 255, 676-679.

From mike at deathstar.psych.ualberta.ca  Wed Sep  2 21:56:10 1998
From: mike at deathstar.psych.ualberta.ca (Michael R.W. Dawson)
Date: Wed, 2 Sep 1998 19:56:10 -0600 (MDT)
Subject: Job Openings At U. of A.
Message-ID: <Pine.LNX.3.96.980902195530.32579A-100000@emperor.psych.ualberta.ca>

Three Tenure-Track Assistant Professor Positions in Brain & Behaviour


The Department of Psychology, Faculty of Science at the University of
Alberta, is seeking to expand its development in the area of Brain and
Behaviour. Over the next two years, three tenure-track positions at the
level of assistant professor (salary floor $40,638) will be open to
competition. The first appointment, will be effective July 1, 1999; the
second and third appointments will be effective July 1, 2000.

Applicants should have expertise in any one of the following or related
approaches to the study of brain and behavior: neural plasticity and
development, brain dysfunction, computational or systems, and comparative
cognition.  Applicants must have completed their PhD or equivalent degree
by July 1, 1999.  The expectation is that the successful candidate will
secure NSERC, MRC, AHFMR, or equivalent funding.  Hiring decisions will be
made on the basis of demonstrated research capability, teaching ability,
the potential for interactions with colleagues, and fit with departmental
needs.

A curriculum vitae, a description of current and planned research, copies
of recent publications, and at least three letters of reference should be
sent to: Dr Charles H M Beck, Acting Chair, Department of Psychology, P220
Biological Sciences Building, University of Alberta, Edmonton, Alberta,
Canada  T6G 2E9. The closing date for applications for the position
available July 1, 1999, is November 15, 1998.  


In accordance with Canadian Immigration requirements, this competition is
directed at Canadian Citizens and permanent residents of Canada.  The
University of Alberta is committed to the principle of equity in
employment.  As an employer we welcome diversity in the workplace and
encourage applications from all qualified women and men, including
Aboriginal peoples, persons with disabilities, and members of visible
minorities.

Further information is available on the web at URL
http://web.psych.ualberta.ca/people.htmld.


--
Professor Michael R.W. Dawson | mike at bcp.psych.ualberta.ca | (403)-492-5175
Biological Computation Project, Dept. of Psychology, University of Alberta
Edmonton, AB, CANADA T6G 2E9 | http://www.bcp.psych.ualberta.ca/~mike/


From terry at salk.edu  Wed Sep  2 22:22:24 1998
From: terry at salk.edu (Terry Sejnowski)
Date: Wed, 2 Sep 1998 19:22:24 -0700 (PDT)
Subject: NEURAL COMPUTATION 10:7
Message-ID: <199809030222.TAA25746@helmholtz.salk.edu>

Neural Computation -  Contents Volume 10, Number 7 - October 1, 1998

VIEW

Analog Versus Digital:  Extrapolating from Electronics to Neurobiology
Rahul Sarpeshkar

NOTE

Employing The Z-Transform to Optimize the Calculation of the Synaptic 
Conductance of NMDA-and Other Synaptic Channels in Network Simulations
J. Kohn and F. Worgotter

LETTERS

Site-Selective Autophosphorylation of Ca2+/Calmodulin-
Dependent Protein Kinase II as a Synaptic Encoding Mechanism
C. J. Coomber

Ion Channel Stochasticity May Be Critical in Determining the 
Reliability and Precision of Spike Timing
Elad Schneidman, Barry Freedman, and Idan Segev

Fast Temporal Encoding and Decoding with Spiking Neurons
David Horn and Sharon Levanda

Linearization of F-I Curves by Adaptation
Bard Ermentrout

Mutual Information, Fisher Information and Population Coding
Nicolas Brunel and Jean-Pierre Nadal

Synaptic Pruning in Development:  A Computational Account
Gal Chechik, Isaac Meilijson and Eytan Ruppin

Spatial Decorrelation in Orientation-Selective Cortical Cells
Alexander Dimitrov and Jack D. Cowan

Receptive Field Formation in Natural Scene Environments:  
Comparison of Single-Cell Learning Rules
Brian S. Blais, N. Intrator, H. Shouval and Leon N. Cooper

Neural Feature Abstraction from Judgements of Similarity
Michael D. Lee

Classification of Temporal Patterns In Dynamic Biological Networks
Patrick D. Roberts

Kernel-Based Equiprobable Topographic Map Formation
Marc M. Van Hulle

An Energy Function and Continuous Edit Process for Graph Matching
Andrew M. Finch, Richard C. Wilson, and Edwin R. Hancock

Approximate Statistical Test for Comparing Supervised Classification 
Learning Algorithms
Thomas G. Dietterich

Probability Density Estimation Using Entropy Maximization
Gad Miller and David Horn

-----

ABSTRACTS -  http://mitpress.mit.edu/NECO/

SUBSCRIPTIONS - 1998 - VOLUME 10 - 8 ISSUES

                  USA    Canada*   Other Countries
Student/Retired   $50    $53.50      $78
Individual        $82    $87.74     $110
Institution      $285   $304.95     $318
 * includes 7% GST

(Back issues from Volumes 1-9 are regularly available for
$28 each to institutions and $14 each for individuals.
Add $5 for postage per issue outside USA and Canada.
Add +7% GST for Canada.)

MIT Press Journals, 5 Cambridge Center, Cambridge, MA 02142-9902.
Tel: (617) 253-2889  FAX: (617) 258-6779 mitpress-orders at mit.edu

-----

From elizondo at axone.u-strasbg.fr  Thu Sep  3 03:36:04 1998
From: elizondo at axone.u-strasbg.fr (David ELIZONDO)
Date: Thu, 3 Sep 98 09:36:04 +0200
Subject: catastrophic forgetting 
Message-ID: <9809030736.AA00658@axone.u-strasbg.fr>

The Recurcive Deterministic Perceptron (RDP) is an example of a  
neural network model that does not suffer from catastrophic  
interference. This feedforward multilayer neural network is a  
generalization of the single layer perceptron topology (SLPT) that  
can handle both linearly separable and non linearly separable  
problems.

Due to the incremental learning  nature of the RDP neural networks,  
the problem of catastrophic interference will not arise with this  
learning method. The latter because the topology is build one step at  
the time by adding an intermediate neuron (IN) to the topology. Once  
a new IN is added, its weights are frozen. 


Here are two references describing this model:

	M. Tajine and D. Elizondo.
	The recursive deterministic perceptron neural network. 

	Neural Networks (Pergamon Press). Acceptance date : 

	Mars 6, 1998

	M. Tajine and D. Elizondo. 

	Growing Methods for constructing Recursive eterministic 

	Perceptron Neural Networks and Knowledge extraction.   	
	Artificial Intelligence (Elsevier).
	Acceptance date : May 6, 1998

A limited number of pre-print hard copies are available.

From thorpe at cerco.ups-tlse.fr  Thu Sep  3 08:14:08 1998
From: thorpe at cerco.ups-tlse.fr (Simon Thorpe)
Date: Thu, 3 Sep 1998 13:14:08 +0100
Subject: Multiprocessor boards for neural network simulations...
Message-ID: <v03007825b21438635d45@[193.54.228.14]>

Hi,

	Over the past few months I have been in discussion with Neil Carson
from Causality Ltd. http://www.causality.com/ in the UK about the
possibility of developping hardware for doing neural network simulations.
Right now, the project is looking very promising,  and Neil has said that
he would be happy to go ahead and build a first batch of 25 boards as soon
as he can be reasonably confident that there will be enough buyers. I
myself will be buying 8-10 boards, but we need a few other interested
people to get the project off the ground. I was wondering whether anyone on
the comp-neuro mailing list might be interested.

	Briefly, what we are proposing is the following. The basic board
will be a standard PCI board that could be used in PCs, Macs or indeed any
other computer with PCI slots in it. Each board would be fitted with 6
processor modules, three on each side, which would plug into standard
SODIMM memory slots (Small Outline Dual In-Line Memory Modules), the same
type of slots that are used for memory expansion on laptop computers. Each
of these daughter boards would have a StrongARM  SA-110 micro-processor, a
Digital 21285-A PCI bridge circuit, and 32 Mbytes of SDRAM. In effect, each
board would be a computer in its own right, and would run a version of Unix
(Linux or NetBSD). It would also have an IP address, allowing messages to
be sent efficiently via the PCI bus from processor to processor. Neil and
the other programmers at Causality will look after the message passing
mechanisms, probably using I20 protocols (if you know what that is).

	In our case, we want to use these boards to run a parallel version
of SpikeNET, our asynchronous spiking neuronal network simulator. It turns
out that this particular program would parallelise very nicely on such a
system, because the communication bandwidth between processors is kept very
low. If you're interested in SpikeNET, just let me know - we're seriously
thinking about making the program available to whoever wants to use it. But
in fact, the same hardware could also be used for any sort of parallel
program that could be run using PVM or MPI message passing protocols,
including (I presume) Parallel Genesis.

	There are some limitations though. The StrongARM SA-110 doesn't
have a floating point unit, but as long as you only want to do integer
calculations, it goes like a rocket. Our own software (which is integer
only) runs as fast on the StrongARM as on a Pentium II of the same clock
speed. This is really impressive, since the StrongARM doesn't have a second
level cache (unlike the Pentium II) and the code has (as yet) not been
optimised for the StrongARM at all. I don't know what the situation is with
Genesis - my guess is that there's a lot of floating point in it, but I'm
not sure that this could be recoded for fixed point - after all, most
neurones don't have voltages that reach 10 to the power 200 volts ;-)

	The reasons for choosing the StrongARM are pretty straightforward.
It is very small, doesn't get hot (< I Watt) and is cheap. This means that
it becomes perfectly feasible to imagine packing large numbers of
StrongARMs in a very small space without having to worry about overheating
(imagine trying to do the same thing with Pentium IIs). In addition,
although the future of StrongARM was once in doubt (it was co-developped by
Digital and Advanced Risc Machines), it has now been bought up by Intel who
have recently announced that they will be investing heavilly in StrongARM
development. See http://developer.intel.com/design/strong/ for details.

	The current top-of-the-range StrongARM runs at 233 MHz, and this is
what Neil Carson is proposing to use in this first batch. However, in the
not to distant future there will be 400 MHz StrongARMs with 100 MHz SDRAM
memory busses. And there will be a new StrongARM processor (the SA-1500)
which will have a separate floating point unit for multimedia operations.
One of the nice features of this daughter board arrangement is that it
would be pretty simple and cost effective to do a new batch of boards using
whatever the best technology is at that moment. Another advantage of using
this sort of parallel hardware is that  even last years technology will
still be useful to you - not like conventional PCs where you feel that you
have to buy a new computer every six months if you don't want to be
obsolete.

	So, what about prices you may be saying. Well, if you are
interested it should be possible to do such a board for 1200 pounds ($2000)
on this first run.  Each board would only take one PCI slot, so with four
free PCI slots you could put up to 24 processors in a single PC! If we can
round up enough interested people, we should be able to get the boards done
in about 2 months. Please note that I am not personally going to making any
money on this, and Causality are only expecting to break even on it.
However, both Neil and I are confident that this could be a really
promising approach - we just need to get enough support to get the ball
rolling. Obviously, the more people that are interested, the cheaper it
gets....

	If you want more information, don't hesitate to contact either me
or Neil at neil at causality.com.

	Best wishes

	Simon Thorpe





__________________
Simon Thorpe
Centre de Recherche Cerveau et Cognition
133, route de Narbonne
31062 Toulouse France
Tel 33 (0)5 62 17 28 03. Fax 33 (0)5 62 17 28 09
__________________



From aminai at ececs.uc.edu  Thu Sep  3 10:19:09 1998
From: aminai at ececs.uc.edu (Ali Minai)
Date: Thu, 3 Sep 1998 10:19:09 -0400 (EDT)
Subject: function of hippocampus
Message-ID: <199809031419.KAA19536@holmes.ececs.uc.edu>

	From trengove at socs.uts.EDU.AU  Thu Sep  3 04:37:08 1998
	X-Sender: trengove at linus
	Subject: Re: function of hippocampus
	MIME-Version: 1.0

	  ...the above quotes concerning the role of hippocampus in categories of
	memory such as episodic memory as well as tasks such as spatial cognition
	suggests to me we consider from a top down, psychological standpoint why
	spatial cognition and episodic memory should be tied together
	functionally, and hence why we shouldn't be surprised that a single part
	of the brain is involved in both.
	  From my own subjective observations, if I am trying to remember a
	thought that I had, or a particular piece of information that came up in a
	conversation, often the best way to proceed is for me to remember where I
	was when I had the thought.  Once I have remembered the place where I had
	the thought I have access to a rich pool of cues that can help to trigger
	the particular thought I'm after.
	  It thus makes sense to me that spatial cognition should be involved in
	the laying down of new memories.  In the course of a day I will have
	perhaps hundreds of disctinct cognitive experiences to remember, but the
	number of distinct _places_ in which I dwell whilst having those
	experiences is likely to be at least an order of magnitude smaller.
	  Thus it makes good sense to organise memory around the memories of the
	places one has been during the course of a day.

There has been debate among hippocampal theorists (mainly in the context of
the rodent hippocampus) about whether the hippocampus is dedicated solely or
predominantly to spatial cognition. The alternative --- advanced most prominently
by Howard Eichenbaum and co-workers --- is that the hippocampus helps construct
memories involving complex relationships between cues, conditions, contexts, etc.
(relational memory). In this view, purely spatial representations, such as the
cognitive map of O'Keefe and Nadel, are special cases of relational memory of
complex episodes.

Let us think of an episode as a spatio-temporal structure in some very
high-dimensional space, where the dimensions correspond to sensory cues,
features, contexts, motivations, internal states, etc. In any particular
case, most of the possible dimensions will be irrelevant, and the episodic
representation will lie in a subspace of the full space of possibilities.
When this subspace is purely spatial, we see a place representation. When
the subspace is predominantly spatial but also includes other dimensions,
we see conditional place representations (e.g., context-dependent,
directional, reward-related, etc.) In particular cases --- e.g., experiments
designed to make place irrelevant --- we would see non-spatial representations
(e.g., in Eichenbaum's olfactory experiments). In general, as the quotation
above points out, place is an extremely important component of any episodic
experience, and it would not be surprising to find strong place-dependencies
in any neural representation of episodic memory. I believe that most
hippocampal theory now implicitly acknowledges this.

Theories which see the hippocampus as primarily involved in spatial cognition
are based on rodent data with its place and head-direction cells. There is no
question that, in many instances, one is hard pressed to find any correlate
of a hippocampal pyramidal cell's activity other than location. However, in
strongly directed environments (e.g., arm mazes), direction becomes a factor
too. And several reports have demonstrated that cells fire in response to
task contingencies when these are made important. It is tempting to think that,
given a rat's limited mental life (am I being speceist:-), place is almost
the entire default sub-space of experience, except at particular times. Thus,
episodic representations appear --- especially when measured one cell at a
time --- as place representations. In higher animals such as primates, episodes
have much richer content, and space is only a part of the picture --- hence
the absence of place cells. Interestingly, one does find view cells in primates
(e.g., in the work of Rolls and co-workers) which fire when the animal is looking
at a particular scene. Perhaps this means that, for primates, what is seen during
an episode is more significant than where one is located. Also, it is possible
that primates have a better ability to project experience to places that they
can see but where they are not currently located.


        ......  I believe one idea in
	circulation e.g. discussed by Rolls and coworkers, is that the hippocampus
	provides a short term 'buffer' for storing memories, which are later
	'transferred' to the neocortex for long term storage.

This idea has been around for a while in various forms. In terms of modeling,
I think the work by Gluck and Mayers, Squire and Alvarez, O'Reilly, McClelland
and McNaughton, and recently, by Redish and Touretzky, offers interesting
perspectives.


	  Back to the specific idea given above, I'm curious whether specialists
	of the hippocampus find it (a) highly dubious, implausible or naive; or
	(b) too obvious to be worth mentioning; or (c) a potentially useful way
	to look at the role of the hippocampus.

Without claiming specialist status, my answer is (c), provided we think of the
big picture that includes the whole constellation of available data. I believe
strongly that big theoretical ideas --- such as the cognitive mapping theory
--- are crucial to our understanding of neural function, even when the theories
turn out to be less than perfect in the end.

Ali Minai


-----------------------------------------------------------------------------
Ali A. Minai
Assistant Professor
Complex Adaptive Systems Laboratory
Department of Electrical
   & Computer Engineering
               and Computer Science
University of Cincinnati
Cincinnati, OH 45221-0030

Phone: (513) 556-4783
Fax:   (513) 556-7326
Email: Ali.Minai at uc.edu

Internet: http://www.ececs.uc.edu/~aminai/

From arbib at pollux.usc.edu  Thu Sep  3 11:53:55 1998
From: arbib at pollux.usc.edu (Michael Arbib)
Date: Thu, 03 Sep 1998 08:53:55 -0700
Subject: What have neural networks achieved?
Message-ID: <199809031555.IAA14614@pollux.usc.edu>


The following extract from the debate on what AI has achieved may be of
interest to connectionists engaged in the present discussion, and in the
one on
symbolic representation.

One might ask:  Does AI need neural networks to be really successful?


Subject: Re: What have neural networks achieved?
>Date: Wed, 2 Sep 1998 20:50:12 -0700 (PDT)
>From: John McCarthy <jmc at Steam.Stanford.EDU>
>Subject: challenges to AI
>
>Paul Rosenbloom asks David McAllester:
>
>   Can you elaborate a bit on what
>   you would find necessary before you would see "any evidence for the
>   feasibility of the grand goals"?
>
>David McAllester replied:
>
>     Some convincing ability to discuss, say, daily events in the
>     life of a child.  A human-level theorem proving machine for
>     UNRESTRICTED conceptual mathematics would also be evidence
>     for me.  I am quite familiar with the current state of the
>     art in theorem proving and I feel like I have climbed a tree
>     while staring at the moon.  While current formal methods do
>     have applications, I do not believe that current
>     applications constitute evidence for the grand goals.  It
>     seems popular among AI researchers to take a "sour grapes"
>     attitude toward grand-goal problems like the Turing test and
>     human-level understanding of general conceptual mathematics
>     --- "oh those can't be done but they're not important
>     anyway".
>
>       David
>
>David McAllester's complaints are similar in some respects to those
>Hubert Dreyfus and Lotfi Zadeh presented at the recent Wonderfest in
>Berkeley.  The difference is that McAllester knows a lot more about
>AI.  At the Berkeley meeting, I tried to get Dreyfus and Zadeh to say
>what was the *easiest* task they considered infeasible to AI.  After
>some discussion, this came down to what task would not be accomplished 
>in the next ten years.  I think I got something out of each of them.
>
>Dreyfus gave the example of "Jane saw the puppy in the window of the
>pet store.  She pressed her nose against it."  The problem is to
>get the referent of "it" to be the window in an honest way, i.e. not
>building in too much.  This requires returning to the task of using
>world knowledge to get the referents of pronouns.  Mere scripts
>wouldn't do it.
>
>Zadeh's example was to get a car out of a parking garage with columns
>and lots of other cars some of which would have to be moved.  That one 
>doesn't seem very hard.  Zadeh thinks AI without fuzzy logic
>won't work.
>
>I have some sympathy with McAllester's point of view.  Much
>present work in AI uses methodologies that are limited in what
>they can ultimately do.  I discuss this in my
>
>FROM HERE TO HUMAN-LEVEL AI
>
>http://www-formal.stanford.edu/jmc/human.html.
>
>That paper lists some of the problems that must be solved to
>reach human-level AI.  I reread it and plan to improve it.
>
>I would like David McAllester to list some of the problems that
>he sees.  If there are any relatively concrete problems that he
>thinks can't be solved in the next ten years, this would serve as 
>a worthwhile challenge to AI research.  He should list the
>*easiest* problem he thinks won't be solved in ten years.
>
>It hasn't helped AI much to be challenged only by the ignorant,
>but McAllester isn't ignorant, so his challenges, if he can make
>them more concrete than in his message, will be helpful.
>
>To give an example, I don't think the methods that have been
>moderately successful at chess will succeed with Go.
>I say "moderately successful", because I think the amount of
>computer power used by Deep Blue is disgracefully large and
>conceals a lack of understanding.  Almost all of those 200
>million positions examined each second would be rightfully
>ignored by a more sophisticated program.
>
>See http://www-formal.stanford.edu/jmc/newborn.html which
>appeared in _Science_.
>
>
>
> 
> 
*************************
Michael Arbib
Director, USC Brain Project
University of Southern California
Los Angeles, CA 90089-2520
(213) 743-6452     FAX (213) 740-5687
arbib at pollux.usc.edu
http://www-hbp.usc.edu/HBP/ 

From trengove at socs.uts.EDU.AU  Thu Sep  3 04:36:53 1998
From: trengove at socs.uts.EDU.AU (Chris Trengove)
Date: Thu, 3 Sep 1998 18:36:53 +1000 (EST)
Subject: function of hippocampus
In-Reply-To: <199808270518.BAA06571@holmes.ececs.uc.edu>
Message-ID: <Pine.GSO.4.02A.9809031737480.26360-100000@linus>

As a non-expert in regards to the hippocampus I would like to offer a
thought triggered by the following remarks of Minai, to see what people
think:

On Thu, 27 Aug 1998, Ali Minai wrote:
> That having been said, I do think (and others can marshall the evidence
> better than I can) that a preponderance of evidence favors a hippocampal
> involvement in episodic memory and, at least in rodents, spatial cognition.
..
> The issue of hippocampal involvement in spatial cognition in rodents...
> is given overwhelming credibility ... by the undeniable existence of
> place cells and head-direction cells. 
> ... provides
> convincing evidence that the hippocampus ``knows'' a great deal about
> the animal's spatial environment, is very sensitive to it, and responds
> robustly to disruptions of landmarks, etc. 
.. 
> I do not think we really understand what role the rodent hippocampus plays
> in spatial cognition, but it is hard to dispute that it plays some ---
> possibly many --- important roles. I think that, as theories about
> hippocampal function begin to place the hippocampus in the larger context
> of other interconnected systems (e.g., in the work of Redish and Touretzky),
> we will move away from the urge to say, ``Here! This is what the hippocampus
> does'' and towards the recognition that it is probably an important part
> in a larger system for spatial cognition.
.. 
> Finally, one issue that is particularly relevant to hippocampal theories
> is the possibility that the categories of memory (e.g., episodic, declarative,
> etc.) or task (DNMS, spatial memory, working memory, etc.) that
> we use in our theorizing may not match up with the categories relevant to
> actual hippocampal functionality. Perhaps we are trying to build a science
> of chemistry based on air, water, fire, and earth. 

  So, the above quotes concerning the role of hippocampus in categories of
memory such as episodic memory as well as tasks such as spatial cognition
suggests to me we consider from a top down, psychological standpoint why
spatial cognition and episodic memory should be tied together
functionally, and hence why we shouldn't be surprised that a single part
of the brain is involved in both.
  From my own subjective observations, if I am trying to remember a
thought that I had, or a particular piece of information that came up in a
conversation, often the best way to proceed is for me to remember where I
was when I had the thought.  Once I have remembered the place where I had
the thought I have access to a rich pool of cues that can help to trigger
the particular thought I'm after.
  It thus makes sense to me that spatial cognition should be involved in
the laying down of new memories.  In the course of a day I will have
perhaps hundreds of disctinct cognitive experiences to remember, but the
number of distinct _places_ in which I dwell whilst having those
experiences is likely to be at least an order of magnitude smaller.
  Thus it makes good sense to organise memory around the memories of the
places one has been during the course of a day.  I believe one idea in
circulation e.g. discussed by Rolls and coworkers, is that the hippocampus
provides a short term 'buffer' for storing memories, which are later
'transferred' to the neocortex for long term storage.  This idea is in a
similar vein.
  On a more general note, this kind of thinking suggests that eventually
we will find that there is a harmonious correspondence between the
functional interrelationships of certain, various aspects of cognition on
the one hand and the manner in which various brain structures (areas and
pathways) are especially involved in these aspects of cognition.  Thus the
feedback between neuroscience and psychology ought to give us insights
into the functional organisation of cognition which could not otherwise be
found; i.e. to help us to find the right categories, to go beyond 'air
water fire and earth' c.f. Minai, above.
  Back to the specific idea given above, I'm curious whether specialists
of the hippocampus find it (a) highly dubious, implausible or naive; or
(b) too obvious to be worth mentioning; or (c) a potentially useful way
to look at the role of the hippocampus.

Chris Trengove
School of Mathematical Sciences,
University of Technology, Sydney.


From ucganlb at ucl.ac.uk  Mon Sep  7 13:06:25 1998
From: ucganlb at ucl.ac.uk (Neil Burgess - Anatomy UCL London)
Date: Mon, 07 Sep 98 17:06:25 +0000
Subject: Hippocampus, spatio-temporal context and episodic memory
Message-ID: <68743.9809071606@link-1.ts.bcc.ac.uk>

Several of the recent contributions to this list have considerd the 
relationship between the hippocampus and episodic and spatial memory. 
These appear to be converging on idea of the hippocampus providing a spatial
(and perhaps temporal) context in which events are embedded, so as
to form an episodic memory and facilitate their subsequent recall. 

This view of the role of the (right) human hippocampus was presented as 
part of O'Keefe and Nadel's idea of the hippocampus as a cognitve map
(for a concise discussion see O'Keefe and Nadel, 1979; pp 493 and 527).
Further discussion of this point of view, and the relationship between
hippocampal and parietal regions in spatial and mnemonic tasks can be 
found in the introduction and several of the chapters of the forthcoming
book (see below).

Best wishes,

Neil


O'Keefe and Nadel (1979) The Behavioural and Brain Sciences 2, 487-533
(Precis of The hippocampus as a cognitive map, and peer commentary).

The Hippocampal and Parietal Foundations of Spatial Cognition
(Eds: N Burgess, KJ Jeffery and J O'Keefe) Oxford University Press 
(1998 - should be out in time for the Soc. Neurosci. conference).


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Dr Neil Burgess                        
Institute of Cognitive Neuroscience & Dept. of Anatomy
University College London              
London WC1E 6BT, U.K.                  
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

From erdi at rmki.kfki.hu  Mon Sep  7 12:54:13 1998
From: erdi at rmki.kfki.hu (Erdi Peter)
Date: Mon, 7 Sep 1998 18:54:13 +0200 (MDT)
Subject: Computational Neuroscience in Europe
Message-ID: <Pine.SGI.3.95.980907185232.19954A-100000@sgiserv.rmki.kfki.hu>


COMPUTATIONAL NEUROSCIENCE in EUROPE:  Where we are and where to go?


	Within the framework of the Forum of European Neuroscience held in
Berlin (27 June - 1 July 1998) a special workshop "Spectrum of
Computational Neuroscience in Europe" was organized (see:
http://www.hirn.uni-duesseldorf.de/~rk/ena98sw5.htm).

	In addition to the Workshop, an informal discussion on
Computational Neuroscience was held. This report summarizes the central
ideas: 

	The fast growing field of Computational Neuroscience plays an
important role in integrating the results of structural,
functional and dynamic approaches to the nervous system. In Europe, an
active community is emerging investigating structure/function
relationships of the nervous system with computational tools at
subcellular, cellular, network, and system levels.  

	To enhance the growth of this field in Europe it was thought that
the following specific undertakings should be considered:
	
1. To make a tentative list of the European Computational Neuroscience
(cns) labs and of their webpages;

2. To hear about the position of the cns labs within the national
neuroscience communities;
	
3. To search for funding programs;
	
4. To think on the formation of a network of European laboraories (i) to
apply for common grants; (ii) to organize a regular series of cns
meetings.
	
5. To consider the creation and utilization of Neuroscience databases as a
joint effort in collaboration with experimentalists. 	 
	The participants were well aware of not representing the field of
Computational Neuroscience in Europe and of not having any mandate. It was
felt, however, that this initiative could be a first step towards the
creation of a more representative voice. 

	The participants agreed that the creation of a network
among the European computational neuroscientists would be useful. 
We would highly appreciate your input at this point.  In the first
step we should like to make a tentative list of Computational
Neuroscience laboratories as mentioned in 1. Any pointers to
relevant web sites or email contacts are most welcome.  

	Concerning the second step we should like to hear if and how the
field of Computational Neuroscience is publicly represented in different
countries and whether it is considered useful to initiate the foundation
of such sections within the national (and thereby the European)
neuroscience societies. 	
	

	Prof. Peter Erdi
	Dept. Biophysics
	KFKI Research Institute for 
	Particle and Nuclear Physics
	of the Hungarian Academy of Sciences
	H-1525 Budapest, P.O. Box 49, Hungary
	Tel: (36-1)-395-92-20/2505 ext.

Fax: (36-1)-395-9151; 
	http://www.rmki.kfki.hu/biofiz/biophysics.html
	
	Dr. Rolf Kotter
	Center for Anatomy and Brain Research
	Heinrich Heine University, 
	Moorenstr. 5, D-40225 Dusseldorf, FRG 
	Tel./Fax: +49-211-81-12095
	http://www.hirn.uni-duesseldorf.de/~rk/
	


From austin at minster.cs.york.ac.uk  Mon Sep  7 14:22:00 1998
From: austin at minster.cs.york.ac.uk (Jim Austin)
Date: Mon, 7 Sep 1998 19:22:00 +0100
Subject: Connectionist symbol processing: any progress? 
Message-ID: <9809071922.ZM17338@minster.cs.york.ac.uk>



Its time to add one more (!) line of work on the symbolic neural networks
debate. We have been working on these systems for about 5 years, in the
context of binary neural networks. We have concentrated on the use
of outer-product methods for storing and accessing information in
many (commercially relevant) problems. We have not concentrated on the
cognitive significance (although I believe what we have done has some). We
exploit them for the following reasons;

1) The use of outer-product based methods are very fast in both learning and
access, this comes from the sparce representations we use, along with the
simple learning rules.

2) They are very memory efficient if used with distributed representations.

3) They have the ability to offer any-time properties (in this I mean
they can give a sensible result at any time after a fixed initial
processing time has passed).

4) They can be used in a modular way.

5) They are fault tolerant (probably)

6) They map to hardware very efficiently.

Our main work has been in there use in search engines, rule based systems
and image analysis.

The approach is based on the use of tensor products for binding data
before presentation to a CMM (an outer product based memory), the other
features are;
The use of binary distributed representations of data.
The use of threshold logic to select interesting matches.
The use of pre-processing to orthoganalise data for efficient representation.
The use of superposition to maintain fixed length representations.

Our work has looked at the use of large numbers of CMM systems for solving
hard problems. For example, we have done work using them for
molecule databases, where they are used for structure matching. We
have been using them to recognise images (simple ones), where they can
be made to generalise to translations and scale varying images.

We have built hardware to perform binary outer-products and other operations,
and are now scaling up the architecture to support large numbers of CMMs
using these cards.

The largest CMM we have used (i.e. an outer product based representation) is
750Mb on a text database.

Details of this work can be found on our web pages and in the following
papers;

The basic methods of tensor binding;
%A    J Austin
%E    N Kasabov
%J    International Journal on Fuzzy Sets and Systems
%T    Distributed Associative Memories for High Speed Symbolic Reasoning
%P 223-233
%V 82
%D 1996


Some more of the same;
%A J Austin
%B Conectionist Symbolic Integration
%D 1997
%E Ron Sun
%E Frederic Alexander
%I Lawrence Erlbaum Associates
%C 15
%P 265-278
%T A Distributed Associative Memory for High Speed Symbolic Reasoning
%K BBBB+
%O ISBN 0-8058-2348-4


The properties of a CMM that we use to store the bindings;
%A Mick Turner
%A Jim Austin
%T Matching Performance of Binary Correlation Matrix Memories
%J Neural Networks
%D 1997
%I Elsevier Science
%P 1637-1648
%N 9
%V 10

The use of the symbolic methods to do molecular database searching;
(A new paper just sent to Neural networks is also on this)
%A M Turner
%A J Austin
%T A neural network technique for chemical graph matching
%D July 1997
%E M Niranjan
%I IEE
%B Fifth International Conference on Artificial Neural Networks, Cambridge, UK

Image understanding architecture;
%A C Orovas
%A J Austin
%D 1997
%J 5th IEEE Int. Workshop on Cellular Neural Networks and their application.
%D 14-17 April 1998
%T A cellular system for pattern recognition using associative neural networks

The hardware;
%A J Austin
%A J Kennedy
%T PRESENCE, a hardware implementation of binary neural networks
%J International Conference on Artificial Neural Networks
%C Sweden
%D 1998





Jim

-- 

Dr. Jim Austin, Senior Lecturer, Department of Computer Science, University of York, York,
YO1 5DD, UK.
Tel : 01904 43 2734
Fax : 01904 43 2767
web pages: http://www.cs.york.ac.uk/arch/


From reggia at cs.umd.edu  Tue Sep  8 14:44:15 1998
From: reggia at cs.umd.edu (James A. Reggia)
Date: Tue, 8 Sep 1998 14:44:15 -0400 (EDT)
Subject: Faculty Position in Neural Modeling
Message-ID: <199809081844.OAA05767@avion.cs.umd.edu>


      FACULTY POSITION AVAILABLE

The position is intended for an entry level, full time
faculty member at the Maryland Psychiatric Research Center, 
a dedicated research facility, part of the University of Maryland's
Department of Psychiatry, in Baltimore.

This funded position is meant to provide imaging, computational, and
research assistant resources.  We are looking for someone who wants to
use fMRI, possibly supplemented by evoked response potential data, to
develop and test models of brain-behavior relationships.

Using the high spatial and temporal relationships provided by fMRI and ERP
studies, the investigator should be able to generate mathematical
models to account for observed and predicted phenomena.  

The investigator should have a Ph.D. in a field that is directly
concerned with biological modeling or system engineering.  Applied 
mathematics, computer science, electrical engineering, and neuroscience
(with an emphasis on system modeling) are appropriate areas of 
concentration.  

Substantial resources for fMRI and ERP research are available for
the investigator.  

A competitive salary will be offered.

Interested applicants should contact
Henry H Holcomb, M.D.
email = hholcomb at mprc.umaryland.edu
phone = 410 719 6817
fax = 410 719 6882
address = MPRC, P.O. Box 21247, Baltimore, MD 21228-0247.

From ehartman at pav.com  Tue Sep  8 15:49:00 1998
From: ehartman at pav.com (Eric Hartman)
Date: Tue, 08 Sep 98 14:49:00 CDT
Subject: NN commercial successes
Message-ID: <35F58AC9@pav.com>


>Michael Arbib wrote:
>
> b) What are the "big success stories" (i.e., of the kind the general
> public could understand) for neural networks contributing to the
> construction of "artificial" brains, i.e., successfully fielded
> applications of NN hardware and software that have had a major
> commercial or other impact?

Pavilion Technologies, Inc., is a very strong NN-based commerical   
 success story.
See Pavilion's web site
   http://www.pavtech.com
for extensive information about Pavilion.

Pavilion produces NN-based software for modeling, optimizating, and   
controling
  continuous process manufacturing processes.
Hundreds of on-line applications are in operation across a number of   
major industries
  world-wide.

Pavilion's many software products include:
 - Process Insights, primarily geared toward steady-state modeling and   
optimization
 - Process Perfecter, a (nonlinear/linear) dynamic, closed-loop   
controller/optimizer
 - Software CEM
 - Virtual OnLine Analyzer
 - BOOST

Pavilion's primary customer base spans blue chip corporations in the   
petrochemical,
power generation, refining, pulp and paper, and food processing markets.   
   

The technology is widely applicable to all continuous manufacturing   
processes.
Pavilion maintains its headquarters in Austin, with offices in New   
Jersey, Brussels,
Frankfurt, and Tokoyo. Pavilion currently employs over 100 people   
worldwide.

Pavilion was incorporated in Austin, Texas in September 1991. Pavilion   
has been honored
with numerous awards for both technical and business accomplishments,   
including:
 - Process Insights received a second consecutive Reader's Choice Award
      for "Best Neural Network Software" from Control Magazine.
 - Process Perfecter received Product of the Year title at the
      1997 KPMG Austin ICE High Tech Awards program.
 - Pavilion has twice been recognized as one of the 20 fastest growing
      companies in Austin by the Austin Business Journal and Ernst &   
Young.
 - Pavilion has been awarded 8 patents for technology by the U.S. Patent   
Office,
      with 10 additional patents pending.

Application Example:
  Chevron Chemical s Experience with Pavilion's Process Perfecter

"The installation of Pavilion s Perfecter controllers on our BP-licensed,   

gas-phase high density polyethylene (HDPE) and linear low density
 polyethylene (LLDPE) reactors has been a very rewarding project," says
 a Chevron spokesman. "The controllers have been well received by the
operating group.  We chose Pavilion for the ease of modeling from   
historical
data, nonlinear capabilities of their dynamic multivariable controller,   
and the
polymer experience and competence of the Pavilion engineers."

"Results from this project have been impressive. Our reactor regulatory,
quality and product transition control have been significantly improved
using Pavilion s dynamic multivariable controller and neural net based
virtual on-line analyzers. The results have exceeded our expectations,"
says the Chevron spokesman.

"We have been working with Pavilion for three or four years, using their
Process Insights modeling software," says Russ Clinton, Supervisor of
Automated Systems for the Chevron site.  "Pavilion s expertise and unique   

controller capabilities were well matched to the needs of ourapplication.   

We have recently decided to extend Pavilion s control technology to our
autoclave low density polyethylene (LDPE) process."
   

For more information about Pavilion, please visit Pavilion's web site:
  http://www.pavtech.com

====================================================

Dr. Eric Hartman
Chief Scientist
hartman at pav.com

(512) 438-1534
(512) 438-1401 fax

Pavilion Technologies, Inc.
11100 Metric Blvd., #700
Austin, TX 78758-4018
USA

From granger at uci.edu  Tue Sep  8 20:52:38 1998
From: granger at uci.edu (Richard Granger)
Date: Tue, 8 Sep 1998 17:52:38 -0700
Subject: What have neural networks achieved?
Message-ID: <v02140b01b21b8169d34e@[128.200.120.16]>


Michael Arbib wrote:
>>  So: I would like to see responses of the form:
>>  "Models A and B have shown the role of brain regions C and D in functions E
>>  and F - see specific references G and H".

Models of the induction and expression (storage and retrieval) mechanisms
of synaptic long-term potentiation (LTP), the actual biological change in
connection strength that underlies at least some forms of real
telencephalic memory in humans, have led to novel hypotheses of the
functions LTP gives rise to in the actual circuitries in which it occurs.

One instance is the olfactory bulb-cortex system:  LTP in this system
(which was shown by Kanter and Haberly, Brain Research, 525:175-179, 1990;
and Jung et al., Synapse, 6: 279-283, 1990), endogenously induced by the 5
Hz "theta" rhythm that occurs during exploration and learning (Komisaruk,
J.Comp.Physiol.Psychol., 70: 482-492, 1970; Macrides, Behav.Biol., 14:
295-308, 1975; Otto et al., Hippocampus, 1991), was shown in models to lead
to an unexpected function of not just remembering odors but organizing
those memories hierarchically and producing successively finer-grained
recognition of an odor over iterative (theta) cycles of operation
(Ambros-Ingerson et al., Science, 247: 1344-1348, 1990; Kilborn et al.,
J.Cog.Neurosci., 8: 338-353, 1996).

This is an instance in which modeling of physiological activity in
anatomical circuitry gave rise to an operation that was unexpected from
behavioral studies, and had been little-studied in the related
psychological and behavioral literatures.


>>  The real interest comes when claims appear to conflict.

Other studies of the olfactory system have yielded quite different
predictions; this raises the question of whether animals cluster and
subcluster odors behaviorally, and whether paleocortical cells respond
selectively to different sampling cycles of clusters of similar odors.
These important issues are far from resolved; some relevant experimental
evidence on behavioral learning of odors is found in (Granger et al.,
Psychol. Sci., 2: 116-118, 1991); and on unit cell activity in cortex
during learning in (McCollum et al., J.Cog.Neurosci., 3: 293-299, 1991; and
see Granger & Lynch, Curr.Biol., 1: 209-214, 1991, for a review).


>>  What about the role of hippocampus in both spatial navigation and
>>  consolidation of short term memory?

Many studies begin with observed behaviors linked to medial temporal
regions by lesion studies and some chronic recording; it has been pointed
out that the connection specifically to hippocampus, as opposed to
surrounding perirhinal cortical regions, is difficult.  Moreover, the range
of behaviors, from spatial navigation to short-term memories, are
suggestive of emergent operations arising from combinations of more
fundamental mechanisms that may be occurring within the various modules of
these brain areas.

Not only is the medial temporal region composed of hippocampus, subiculum,
and overlying cortical structures, but these naming conventions occlude the
richness of circuitries within.  The "hippocampus" is composed of three
extraordinarily distinct structures (dentate, CA3 and CA1), each of which
consists of very different cell types, synaptic connections and local
circuits, and which are strongly connected with neighboring structures
(subiculum, pre- and parasubiculum, and superficial and deep entorhinal
cortex).  Of interest from a modeling point of view are the distinct
functions that emerge from the physiological operations of these disparate
circuits as well as composite functions arising from interactions among
them.  It will be interesting to uncover not only how these circuits
participate in well-studied behavioral circumstances such as navigation and
memory consolidation, but also what heretofore unexpected functions may be
found to arise from their action and interaction (Lynch & Granger,
J.Cog.Neurosci., 4: 189-199, 1992; Granger et al., Hippocampus, 6:
567-578).

It's worth mentioning that a special issue of the journal "Hippocampus"
dedicated to "computational models of hippocampal function in memory"
appeared as Volume 6, number 6 (1996); it may be a useful reference for
this part of the discussion.

- Rick Granger




From becker at curie.psychology.mcmaster.ca  Wed Sep  9 11:04:39 1998
From: becker at curie.psychology.mcmaster.ca (Sue Becker)
Date: Wed, 9 Sep 1998 11:04:39 -0400 (EDT)
Subject: correction to NIPS*98 workshop announcements
Message-ID: <Pine.SGI.3.96.980909110335.29540G-100000@meitner.psychology.mcmaster.ca>

Michael Kearn's name was inadvertently left off the organizing list
for the NIPS*98 workshop on Integrating Supervised and Unsupervised 
Learning. A corrected announcement for that workshop appears below. 

**************************************************************

TITLE: Integrating Supervised and Unsupervised Learning

This workshop will debate the relationship between supervised and
unsupervised learning.  The discussion will run the gamut from
examining the view that supervised learning can be performed by
unsupervised learning of the joint distribution between the inputs and
targets, to discussion of how natural learning systems do supervised
learning without explicit labels, to the presentation of practical
methods of combining supervised and unsupervised learning by using
unsupervised clustering or unlabelled data to augment a labelled
corpus.  The debate should be fun because some attendees believe
supervised learning has clear advantages, while others believe
unsupervised learning is the only game worth playing in the long run.

More information (including a call for abstracts) can be found at
www.cs.cmu.edu/~mccallum/supunsup.

ORGANIZERS:   Rich Caruana
              Virginia de Sa
              Michael Kearns
              Andrew McCallum




From bert at mbfys.kun.nl  Wed Sep  9 04:11:48 1998
From: bert at mbfys.kun.nl (Bert Kappen)
Date: Wed, 9 Sep 1998 10:11:48 +0200 (MET DST)
Subject: PhD position available
Message-ID: <199809090811.KAA23643@bertus.mbfys.kun.nl>

PhD position for neural network research at SNN, University of 
Nijmegen, the Netherlands.

Background:
The SNN neural networks research group at the university of Nijmegen 
consists of 10 researchers and PhD students and conducts
theoretical and applied research on neural networks and graphical
models. The group is part
of the Laboratory of Biophysics which is involved in experimental brain
science.

Recent research of the group has focussed on theoretical description of
learning processes using the theory of stochastic processes and the design
of efficient learning rules for Boltzmann machines using techniques from
statistical mechanics; the extraction of rules from data and the
integration of knowledge and data for modeling; the design of robust
methods for confidence estimation with neural networks; applications in
medical diagnosis and prediction of consumer behaviour. 

Research project:
The modern view on AI, neural networks as well as parts of statistics, is to
describe learning and reasoning using a probabilistic framework. A particular
advantage of the probabilistic framework is that domain knowledge in the form
of rules and data can be easily combined in model construction. The main
drawback is that inference and learning in large probabilistic networks 
is intractible.
Therefore, robust approximation schemes are needed to apply this technology to
large real world applications. 
The topic of research is to develop learning rules for neural networks
and graphical models using techniques from statistical mechanics.

Requirements:
The candidate should have a strong background in theoretical physics or
mathematics.

The PhD position:
Appointment will be full-time for four years. Gross salary will be NLG 2184 
per month in the first year, increasing to NLG 3899 in the fourth year. 

More information:
Details about the research can be found at
http://www.mbfys.kun.nl/SNN or by contacting dr. H.J.
Kappen (bert at mbfys.kun.nl, ++31243614241). 

Applications (three copies) should include a curriculum vitae and a
statement of the candidate's professional interests and goals, and one
copy of recent work (e.g., thesis, article). Applications 
should be sent before October 10 to the
Personnel Department of the Faculty of Natural Sciences, University of
Nijmegen, Toernooiveld 1, 6525 ED Nijmegen, vacancy number 98-52.


From dhw at santafe.edu  Wed Sep  9 14:28:27 1998
From: dhw at santafe.edu (dhw@santafe.edu)
Date: Wed, 9 Sep 1998 12:28:27 -0600
Subject: New Journal Announcement
Message-ID: <199809091828.MAA05358@santafe.santafe.edu>


We apologize if you receive multiple copies of this message.
 

****************************************
	NEW JOURNAL ANNOUNCEMENT
****************************************


                   JOURNAL OF COMPLEX SYSTEMS, Vol. 1, # 1

 
 
CONTENTS

Editorial						9
E. Bonabeau 

Modelling migration and economic agglomeration with 
active brownian particles				11
F. Schweitzer 

Amplitude spectra of fitness landscapes			39
W. Hordijk and P. F. Stadler 


From halici at rorqual.cc.metu.edu.tr  Thu Sep 10 06:20:29 1998
From: halici at rorqual.cc.metu.edu.tr (Ugur Halici)
Date: Thu, 10 Sep 1998 13:20:29 +0300
Subject: CFP: HYBRID APPROACHES ON SOFT COMPUTING TECHNIQUES
Message-ID: <35F7A7ED.16F0@rorqual.cc.metu.edu.tr>

CALL FOR PAPERS 
--------------------------------------------------------------------------------
Special Session on 
HYBRID APPROACHES FOR SOFT COMPUTING TECHNIQUES
--------------------------------------------------------------------------------
in SOCO'99, SOFT COMPUTING, June 1-4, 1999 at the Palazzo Ducale in
Genova, Italy


High quality research papers are sought on the  hybrid approaches or
comparative studies  using at least two of Neural Networks, Genetic
Algorithms or Fuzzy Logic. If you are interested in presenting a paper
in this special session in SOCO'99 please send me
(halici at rorqual.cc.metu.edu.tr)

1. A draft title of your paper                       now
2. One page abstract of the paper and a short CV     by September 20

Then the deadlines are:
3. Draft paper   (at most 7 pages)                   October   15
4. Full paper                                        January   31

You may find detailed information on SOCO  at site
SOCO'99 http://www.icsc.ab.ca/soco99.htm

------------------------------------------------
Ugur Halici, Prof. Dr., (Session Organizer)
Dept of Electrical and Electronics Eng.
Middle East Technical University,
06531, Ankara, Turkey

email: halici at rorqual.cc.metu.edu.tr or ugur-halici at metu.edu.tr
http://www.metu.edu.tr/~wwwnng/ugur/halici.html
Tel: (+90) 312 210 2333
Fax: (+90) 312 210 1261

From austin at minster.cs.york.ac.uk  Thu Sep 10 13:15:55 1998
From: austin at minster.cs.york.ac.uk (Jim Austin)
Date: Thu, 10 Sep 1998 18:15:55 +0100
Subject: Connectionist symbol processing: any progress?
Message-ID: <9809101815.ZM19721@minster.cs.york.ac.uk>


Another outline of symbolic/neural work taking place at York, UK, that
may be of interest to the debate.

Jim Austin


\author{Victoria J. Hodge and Jim Austin}
e-mail: vicky,austin at cs.york.ac.uk

We are proposing a unified-connectionist-distributed system.  The system is
currently theoretical, proposing a logical architecture that we aim to map onto
the AURA \cite{Austin_96}, \cite{AURA_web} modular, distributed neural network
methodology.

We posit a flexible, generic hierarchical topology with three fundamental
layers: features, case, and classes.  There is no repetition, each concept is
represented by a single node in the hierarchy, maintaining consistency and
allowing multifarious data types to be represented thus permitting generic
domains to be accommodated.

The front-end is an implicit, self-organising, unsupervised approach
similar to the Growing Cell Structures of Fritzke
\cite{Fritzke_93:TR} but constructing a hierarchy on top of the clusters and
generating feature descriptions and weighted connections for all nodes.  This
hierarchy will be mapped onto the binary representations of AURA and input to a
hierarchically arranged CMM topology representing features, cases and classes;
all partitioned into CMMs according to the natural partitions inherent in the
hierarchy.

A constraint elimination process will be implemented initially to eliminate
implausible concepts and context-sensitively reduce the search space.  A
spreading activation (SA)-type process (purely theoretical at present) will be
initiated on the required features (i.e., sub-conceptually) and allowed to
spread via the weighted links throughout the hierarchy.  SA is postulated as
psychologically plausible and can implement context effects (semantically
focussing  retrieval) and priming of recently retrieved concepts.  The highest
activated case(s) and class(es) will be retrieved as the best match.

New classes, cases and features can be aggregated into the hierarchy anytime,
simply and efficiently merely by incorporating new node and connections.  We
also aim to implement a deletion procedure that will ensure our hierarchy
remains within finite bounds (i.e., is asymptotically limited).  When a
predetermined size is reached, nodes are generalised that have least utility,
i.e., are covered by other nodes and least frequently accessed.  This allows
forgetting as a new addition results in the generalisation of older concepts.

Future investigation includes: structured concepts; more complexity for classes
(including hierarchically divided classes); weight adaptation where the weights
in the hierarchy are adjusted if the retrieved case(s) or class(es) are a poor
match; solution adaptation allowing solutions to be generated for new cases and
possibly generated from subsections of other solutions and aggregated together;
and, an explanation procedure.

@misc{AURA_web,
	title	= {{The AURA Homepage:
	\emph{http://www.cs.york.ac.uk/arch/nn/aura.html}}},
	}

@Article{Austin_96,
	author 	= {Austin, J. },
	title 	= {{Distributed associative memories for high speed symbolic
				reasoning}},
	journal = {Fuzzy Sets and Systems},
	volume 	= 82,
	pages 	= {223--233},
	year 	= 1996
	}

@Techreport{Fritzke_93:TR,
	author 	= {Fritzke, Bernd},
	title 	= {{Growing Cell Structures - a Self-organizing Network for
		Unsupervised and  Supervised Learning}},
	institution	= {International Computer Science Institute},
	address = {Berkeley, CA},
	number 	= {TR-93-026},
	year 	= 1993,
	}

--


-- 

Dr. Jim Austin, Senior Lecturer, Department of Computer Science, University of York, York,
YO1 5DD, UK.
Tel : 01904 43 2734
Fax : 01904 43 2767
web pages: http://www.cs.york.ac.uk/arch/


From bernabe at cnmx4-fddi0.imse.cnm.es  Fri Sep 11 04:44:00 1998
From: bernabe at cnmx4-fddi0.imse.cnm.es (Bernabe Linares B.)
Date: Fri, 11 Sep 1998 10:44:00 +0200
Subject: ART Microchips
Message-ID: <199809110844.KAA08430@cnm12.cnm.es>


Book Announcement:

ADAPTIVE RESONANCE THEORY MICROCHIPS, Circuit Design Techniques
Authors: T. Serrano-Gotarredona, B. Linares-Barranco and A. G. Andreou

is now available for purchase from Kluwer Academic Publishers
at "http://www.wkap.nl/book.htm/0-7923-8231-5". Table of content and
preface can be copied from "http://www.imse.cnm.es/~bernabe".


ADAPTIVE RESONANCE THEORY MICROCHIPS, Circuit 
Design Techniques, describes circuit strategies resulting in 
efficient and functional adaptive resonance theory (ART) 
hardware systems. While ART algorithms have been developed 
in software by their creators, this is the first book that addresses 
efficient VLSI design of ART systems. All systems described in 
the book have been designed and fabricated (or are nearing 
completion) as VLSI microchips in anticipation of the 
impending proliferation of ART applications to autonomous 
intelligent systems. To accomodate these systems, the book not 
only provides circuit design techniques, but also validates them 
through experimental measurements. The book also includes a 
chapter tutorially describing four ART architectures (ART1, 
ARTMAP, Fuzzy-ART and Fuzzy-ARTMAP) while providing 
easily understandable MATLAB code examples to implement 
these four algorithms in software. In addition, an entire chapter 
is devoted to other potential applications in real-time data 
clustering and category learning. 


From uwe.zimmer at gmd.de  Fri Sep 11 09:15:15 1998
From: uwe.zimmer at gmd.de (Uwe R. Zimmer)
Date: Fri, 11 Sep 1998 15:15:15 +0200
Subject: PostDoc Pos. at GMD Japan Research Lab. (Robotics)
Message-ID: <35F92261.4783DF31@gmd.de>

PosDoc-Pos-Announcement


     --------------------------------------------------------    
             Post-Doctoral Research Positions in

           Autonomous Robotics in Open Environments
   --------------------------------------------------------



Two new post-doctoral positions are open at GMD Japan Research 
Laboratory, Kitakyushu, Japan and will be filled at the earliest 
convenience. The new laboratory (starting officially at first of 
November with a team of 8 scientists and 3 support staff, where 
these first 8 positions should be filled completely until March 
'99) is based on long term cooperations with the Japanese research
community and focuses on the robotics and the telecommunications research fields.

Investigated questions (in the area of robotics) are:

   - How to localize and move in many DoF without global 
     correlation? 
   - Interpretation / integration / abstraction / compression of 
     complex sensor signals? 
   - How to build models of previously unknown environments? 
   - Direct sensor-actuator prediction 
   - How to coordinate multiple loosely coupled robots?

Underwater robotics is regarded as one of the most promising
experimental and application environment in this context. Real six
degrees of freedom, real dynamic environments and real autonomy 
(which is required in most setups here), settle these questions in
a fruitful area. The overall goal is of course not 'just' 
implementing prototype systems, but to get a better understanding 
of autonomy, and situatedness. Modeling, adaptation, clustering, 
prediction, communication, or - from the perspective of robotics -
spatial and behavioral modeling, localization, navigation, and 
exploration are cross-topics addressed in most questions.

Although we are an independent research group, there are of course
close connections to the robotics activities in the institute of 
Thomas Christaller at GMD (German National Research Center for 
Information Technology) in Sankt Augustin, Germany.

Techniques employed and developed up to now include dynamical 
systems, connectionist approaches, behavior-based techniques, rule
based systems, and systems theory. Experiments are based on 
physical robots (not yet underwater!). Thus the discussion of 
experimental setups and particularly the meaning of embodiment 
became topics in itself.

If the above challenges rose your interest, please proceed to our 
expectations regarding the ideal candidate:

   - Ph.D. / doctoral degree in computer sciences, electrical 
     engineering, physics, mathematics, biology, or related 
     disciplines. 
   - Experiences in experimenting with autonomous systems 
   - Theoretical foundations in mathematics, control, 
     connectionism, dynamical systems, or systems theory 
   - Interest in joining an international team of motivated 
     researchers

Furthermore it is expected that the candidate evolves/introduces 
her/his own perspective on the topic, and pushes the goals of the 
whole group at the same time.

Salary starts at 8 Mill. Yen per year depending on experience.

For any further information, and applications (including addresses
of referees, two recent publications, and a letter of interest!) 
please contact:

Uwe R. Zimmer (address below)

link to related activities: http://www.gmd.de/AutoSys/

___________________________________________ 
                                     ____________________________|
    Dr. Uwe R. Zimmer  -  GMD                                 ___|
                    Schloss Birlinghoven                         |
                    53754 St. Augustin, Germany                  |
  _______________________________________________________________.
          Voice: +49 2241 14 2373 - Fax: +49 2241 14 2384        |
             http://www.gmd.de/People/Uwe.Zimmer/                |

From schierwa at informatik.uni-leipzig.de  Fri Sep 11 07:07:46 1998
From: schierwa at informatik.uni-leipzig.de (Andreas Schierwagen)
Date: Fri, 11 Sep 1998 13:07:46 +0200
Subject: Call for Participation: FNS '99
Message-ID: <35F90482.7713@informatik.uni-leipzig.de>

Dear Colleagues,

Below is a brief annoucement of the 6th International Workshop
"Fuzzy-Neuro Systems '99" taking place in Leipzig, Germany on
March 18-19, 1999.
We have published a web page with further details. See

http://www.informatik.uni-leipzig.de/~brewka/FNS/

Andreas Schierwagen

------------------------------------------------------------------------


                        6th International Workshop

                         Fuzzy-Neuro Systems '99


Fuzzy-Neuro Systems `99 is the sixth event of a well established series
of workshops with international participation. Its aim is to give an
overview of the state of the art in research and development of fuzzy
systems and artificial neural networks. Another aim is to highlight
applications of these methods and to forge innovative links between
theory and application by means of creative discussions.

Fuzzy-Neuro Systems `99 is being organized by the Research Committee 1.2
"Inference Systems" (Fachausschuss 1.2 "Inferenzsysteme") of the German
Society of Computer Science GI (Gesellschaft f?r Informatik e. V.) and
Universit?t Leipzig, March 18 - 19, 1999

Workshop Chairs: Gerhard Brewka and Siegfried Gottwald
Local Organization: Ralf Der and Andreas Schierwagen

Topics of interest include:

theory and principles of multivalued logic and fuzzy logic
representation of fuzzy knowledge
approximate reasoning
fuzzy control in theory and practice
fuzzy logic in data analysis, signal processing and pattern recognition
fuzzy classification systems
fuzzy decision support systems
fuzzy logic in non-technical areas like business administration,
	management etc.
fuzzy databases
theory and principles of artificial neural networks
hybrid learning algorithms
neural networks in pattern recognition, classification, process
	monitoring and production control
theory and principles of evolutionary algorithms: genetic algorithms and
	evolution strategies
discrete parameter and structure optimization
hybrid systems like neuro-fuzzy systems, connectionist expert systems
	etc.
special hardware and software

Submissions should be extended abstracts of 4-6 pages. Please send 5
copies of your contribution to Gerhard Brewka, Universit?t Leipzig,
Institut f?r Informatik, Augustusplatz 10-11, 04109 Leipzig, Germany.
Proceedings containing full versions of accepted papers will be
published.

Preliminary schedule:


      Submission of papers: Nov. 8th, 1998
      Notification of authors: Dec. 15th, 1998
      Final version of accepted papers: Jan. 16th, 1999
      Workshop: March 18-19, 1999

The conference language will be English.

From cmbishop at microsoft.com  Fri Sep 11 10:20:57 1998
From: cmbishop at microsoft.com (Christopher Bishop)
Date: Fri, 11 Sep 1998 07:20:57 -0700
Subject: Cambridge-Microsoft Postdoctoral Research Fellowship
Message-ID: <3FF8121C9B6DD111812100805F31FC0D06C00E25@RED-MSG-59>


                 Darwin College Cambridge

               Microsoft Research Fellowship

          http://research.microsoft.com/cambridge  
                http://www.dar.cam.ac.uk/ 

The Governing Body of Darwin College Cambridge, and Microsoft Research
Limited (MSR), jointly invite applications for a stipendary
post-doctoral Research Fellowship supporting research in the field of
adaptive computing (including topics such as pattern recognition,
probabilistic inference, handwriting recognition, statistical learning
theory, computer vision and speech recognition). Applicants should
hold a PhD or should be expecing to have completed their thesis prior
to commencement of the Fellowship.

The Fellowship, which is funded by MSR, will be tenable for two years
commencing on 1 January 1999 (or any other mutually convenient date).
The successful candiate will work closely with Professor C M Bishop at
the MSR laboratory in Cambridge. In addition to a salary, the
Fellowship provides funding for conference participation.

College accommodation will be provided, subject to availability, or an
accommodation allowance will be paid in lieu.

Applicants should send their curriculum vitae, a list of publications,
and the names and addresses of three referees, via email (with the
subject line Darwin-Microsoft Research Fellowship) to
jmg39 at hermes.cam.ac.uk

Hard copies may be sent by surface mail to the Master's Secretary,
Darwin College, Cambridge, CB3 9EU.

   * The closing date for applications is 28 September 1998. *


From jjameson at bayarea.net  Sat Sep 12 19:22:39 1998
From: jjameson at bayarea.net (John Jameson)
Date: Sat, 12 Sep 1998 16:22:39 -0700
Subject: could you post this?
Message-ID: <35FB023E.77DFD006@bayarea.net>

Autonomous Systems, a small startup located in the Bay Area, CA, is
seeking one person to join us.  Our focus is shifting to computer
vision and the consumer PC market.  This person should have a good
background in C++, machine learning (especially neural networks), and
mathematical analysis.  Experience in vision, as well as an
entrepreneurial blood type, are plusses.

Email us if you would like more details.  If you prefer, attach your
resume and what kinds of problems interest you (Word 97 or prior,
postscript, text, pdf).

Best regards,

John Jameson
CTO
Autonomous Systems
San Carlos, CA
jjameson at bayarea.net
http://cdr.stanford.edu/~jameson/autonomous/public_html



From jbower at bbb.caltech.edu  Mon Sep 14 14:05:06 1998
From: jbower at bbb.caltech.edu (James M. Bower)
Date: Mon, 14 Sep 1998 10:05:06 -0800
Subject: Neural networks and neuroscience
Message-ID: <v0401171db22307d6cd8d@[131.215.137.89]>

I would point out that Michael Arbib's request:

>>  "Models A and B have shown the role of brain regions C and D in functions E
>>  and F - see specific references G and H".

does not necessarily involve "neural networks" in the strict sense at all.
My understanding is that the original question raised by Michael involved
the value of "neural network" research to brain science.

There are many models of brain function that have no relationship to what
are generally accepted as "neural network" forms (connectionist models,
backprop, etc).

Further, I would claim that there are very few examples where "neural
network" type models have had much to say at all about neurobiology.  A
quick look at the NN component of the table of contents of the NIPS
proceedings (NIPS being the long running gold standard for neural
network/connectionist research) should make clear the lack of connection
(or real interest) of most NN practisioners in real neurobiology.
Similarly, a survey of the usual traffic on this mailing list reveals the
same.

+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


With respect to Richard Granger's post,  the statement:

(Various neuro-biological results)  was shown in models to lead
to an unexpected function of not just remembering odors but organizing
those memories hierarchically and producing successively finer-grained
recognition of an odor over iterative (theta) cycles of operation
(Ambros-Ingerson et al., Science, 247: 1344-1348, 1990)

Was not, in fact, unexpected, as the model was specifically designed to do
just this.


Jim Bower



From gary at cs.ucsd.edu  Mon Sep 14 20:37:40 1998
From: gary at cs.ucsd.edu (Gary Cottrell)
Date: Mon, 14 Sep 1998 17:37:40 -0700 (PDT)
Subject: What have neural networks achieved?
Message-ID: <199809150037.RAA20208@gremlin.ucsd.edu>

(Hi Jay)

I like the Seidenberg & McClelland account of reading, as
well as PMSP. However, the PMSP result (nonword reading)
really relies on a very well engineered representation of
the outputs. That is, it is very difficult to get a poor
pronunciation with that representation, which already encodes
many of the rules of English pronunciation. Thus, it seems
like one of Lachter and Bever's TRICS (The Representation It
Crucially Supposes). This does not contradict the fact that
a single mechanism account has been demonstrated. But one of
Jay's "loose ends" is: How would a network develop such a
representation in the first place? Which I think is a crucial
question.

g.

From jbower at bbb.caltech.edu  Mon Sep 14 20:11:57 1998
From: jbower at bbb.caltech.edu (James M. Bower)
Date: Mon, 14 Sep 1998 16:11:57 -0800
Subject: nn and neuroscience
Message-ID: <v04011742b2235f566544@[131.215.137.89]>

I should probably say, to somewhat temper my earlier email, that the one
area where NNs and neuroscience HAVE interacted in a more direct and
potentially useful way, is in the general area of thinking about memory
storage.  This includes not only the work on the hippocampus being
discussed at present in this mail group, but also the work of Mike Hasselmo
on neuromodulation of olfactory and hippocampal networks.

However, the vast majority of those in NNs have little direct interest in
the details of brain function -- accordingly, it is not particularly
surprising that much of the work is not particularly related to this
subject.


Jim Bower



From dario at cns.nyu.edu  Tue Sep 15 09:21:45 1998
From: dario at cns.nyu.edu (Dario Ringach)
Date: Tue, 15 Sep 1998 09:21:45 -0400
Subject: Postdoctoral position
Message-ID: <980915092145.ZM25831@alberich.cns.nyu.edu>


Applications are invited for a postdoctoral position to investigate
mechanisms of cortical processing in primate visual cortex using
single and multi-electrode techniques.  The emphasis of the research
is on the measurement and mathematical modeling of neural dynamics in
assemblies of visual cortical neurons.  The Departments of
Neurobiology and Psychology at UCLA provide an excellent research
environment to combine psychophysical, computational and experimental
studies of vision.  Details of past research may be found at
http://cns.nyu.edu/home/dario.  Candidates with previous experience in
primate cortical electrophysiology are preferred.  Candidates with
mathematical or engineering background are also encouraged to apply.
The position is available starting Jan 1st, 1999.  Applicants should
send a CV, the names of three references, and a summary of research
interests and experience to:

Before Dec 15th, 1998:

	Dario Ringach
	Center for Neural Science, Rm 809
	New York University
	4 Washington Place
	New York, NY 10003

	email: dario at cns.nyu.edu

After Dec 15th, 1998:

	Dario Ringach
	Dept of Psychology & Neurobiology
	Franz Hall
	405 Hilgard Ave
	Los Angeles, CA 90095-1563

From mharm at CNBC.cmu.edu  Tue Sep 15 13:07:13 1998
From: mharm at CNBC.cmu.edu (Mike Harm)
Date: Tue, 15 Sep 1998 13:07:13 EDT
Subject: Paper available: Phonology, Reading Acquisition, and Dyslexia
Message-ID: <199809151707.NAA24361@CNBC.CMU.EDU>

Hi.

The following paper has been accepted for publication in Psychological
Review.  It bears on much of the recent discussion on connectionist
models of reading.

=================================================================

         Phonology, Reading Acquisition, and Dyslexia:
             Insights from Connectionist Models


            Michael W. Harm, Mark S. Seidenberg

             University of Southern California


Abstract:

The development of reading skill and the bases of developmental
dyslexia were explored using a connectionist model of word
recognition.  Four issues were examined: the acquisition of
phonological knowledge prior to reading, how this knowledge
facilitates learning to read, the bases of phonological and
non-phonological types of dyslexia, and the effects of literacy on
phonological representation. Representing phonological knowledge in an
attractor network yielded improved acquisition and generalization
compared to simple feedforward networks.  Phonological and surface
forms of developmental dyslexia, which are usually attributed to
impairments in distinct lexical and nonlexical processing routes, were
derived from different types of damage to the network. The results
provide a computationally explicit account of the role of phonological
representations in normal and disordered reading and how they are in
turn shaped by their participation in the reading task. They also show
that connectionist principles that have been applied to skilled
reading and reading impairments following brain injury account for
many aspects of reading acquisition.

========================================================

Send me email at mharm at cnbc.cmu.edu for directions where to download
electronic (postscript or pdf) versions of the paper from.

I apologize that I have not provided a URL for the paper, and that you
must contact me for such information.  My understanding is that the
APA has rules against making such things available in a direct form:

    "If a paper is accepted for publication and the copyright has been
     transferred to the American Psychological Association, the author
     must remove the full text of the article from the Web site. They
     author may leave an abstract up and, on request, the author may 
     send a copy of the full article (electronically or by other means) 
     to the requestor."

     (from http://www.apa.org/journals/posting.html)


cheers,

Mike Harm
mharm at cnbc.cmu.edu
http://www.cnbc.cmu.edu/~mharm/
-----------------------------------------

  Big Science.  Every man, every man for himself.
  Big Science.  Hallelujah.  Yodellayehoo.

                                 -Laurie Andersen


From granger at uci.edu  Tue Sep 15 15:16:15 1998
From: granger at uci.edu (Richard Granger)
Date: Tue, 15 Sep 1998 12:16:15 -0700
Subject: Unexpected hypotheses arising from brain circuit simulation
Message-ID: <v02140b01b2233a42ccab@[128.200.120.16]>


jbower wrote that a set of results of ours, published in Science (1990), ...

> "... Was not, in fact, unexpected, as the model was specifically designed
>to do
> just this."

Thanks for giving us the credit for this design.  Since nothing like this
had ever been suggested as a hypothesis of olfactory function (indeed, it's
still studied, and still controversial), if we designed it, it would be a
testament to our inventiveness.

Nonetheless, we admit that the findings actually were quite surprising to
us.  The result emerged only after considerable observation and analysis of
a series of brain network simulations of the anatomical circuits of the
olfactory bulb and cortex (as characterized by Price, Haberly, Shepherd,
and others), operating under normal physiological conditions (as identified
by both in vivo and in vitro work from many labs, including Macrides,
Kauer, Freeman, Haberly,  Eichenbaum, Otto, Komisaruk, Staubli, et al.) and
in response to the physiological induction and expression rules for
synaptic long-term potentiation (LTP; Kanter & Haberly, '90; Jung et al.,
'90).  Extensive references can be found in the many published papers on
the topic; a glance at Medline will readily find most of our papers,
containing these references.  Anyone interested in more detail is welcome
to contact us:  granger at uci.edu

Back to the science of the thing: the hypothesis in question is that the
operation of the bulb-cortex system not only acts to 'remember' odors but
operates (via feedback inhibition over iterative samples of an odor) to
produce first recognition of the general 'category' (or cluster) of an
odor, followed by successively finer-grained recognition over sequential
(theta) cycles of operation, thereby re-using cortical cells over
successive samples to effectively read out a hierarchical description of
the odor.  It's interesting to note that this remains an intriguing
candidate hypothesis that continues to be cited and studied both
behaviorally and physiologically, since its initial publication
(Ambros-Ingerson et al., Science, 247: 1344-1348, 1990).

Many relevant articles from our lab and others' have appeared over the
years; we can send (or post) a list to any interested parties.

-Rick Granger




From uipodola at jetta.if.uj.edu.pl  Tue Sep 15 11:53:05 1998
From: uipodola at jetta.if.uj.edu.pl (Igor T. Podolak)
Date: Tue, 15 Sep 1998 17:53:05 +0200 (MET DST)
Subject: Connectionist symbol processing: any progress?
In-Reply-To: <199808231449.AAA02154@numbat.cs.rmit.edu.au>
Message-ID: <Pine.SOL.3.90.980915173548.16805A-100000@jetta>

Arun Jagota and B. Garner wrote:

> * 
> * It would be nice if some sort of a record of the "Connectionist
> * Symbol Processing" debate were to be produced and archived for
> * the benefit of the community.
> * 
> I think this would be a good idea.. because there were so many 
> interesting ideas expressed.

  I have done some homework towards it.  You can find a first 
approach towards an archive of the discussion at my www home page

	http://www.ii.uj.edu.pl/~podolak

  With help of a Perl program I have moved all the emails into 
separate html files, and the discussion is organized as a time 
sorted liste, name sorted list, and a list of consecutive 
postings starting with Dave Touretzky's email.  For ease of use 
all the email addresses are turned into 'mailto:' links, and, 
most important, all the given web adresses of various pages and 
documents are turned into 'http:' links ready to use.

  In some places, where algorithms were described, the Perl 
program made them harder to read, but I shall work on.  

  Please email me if you find it usable.

					igor


Igor T. Podolak, PhD               phone (48 12) 6323355 ext 320
Computer Science Department        fax   (48 12) 6341865
Jagiellonian University            home  (48 12) 6331324
Nawojki 11, 30 072 Krakow, Poland
e-mail:uipodola at if.uj.edu.pl       http://www.ii.uj.edu.pl/~podolak/index.html


From Otto_Schnurr-A11505 at email.mot.com  Tue Sep 15 14:42:40 1998
From: Otto_Schnurr-A11505 at email.mot.com (Otto Schnurr-A11505)
Date: Tue, 15 Sep 1998 13:42:40 -0500
Subject: What have neural networks achieved?
Message-ID: <35FEB520.DFBC8AAC@ccrl.mot.com>

Michael Arbib wrote:
> b) What are the "big success stories" (i.e., of the kind the general public
> could understand) for neural networks contributing to the construction of
> "artificial" brains, i.e., successfully fielded applications of NN hardware
> and software that have had a major commercial or other impact?
>
>  *********************************
>  Michael A. Arbib
>  USC Brain Project
>  University of Southern California
>  Los Angeles, CA 90089-2520, USA
>  arbib at pollux.usc.edu
 
 
While our application does not address brain function, it does
represent a successful example of how neural networks are able learn
and synthesize human behavior.
 
Motorola has developed a text-to-speech synthesizer that utilizes
multiple cooperating neural networks, each specializing in a
particular area of human language ability.  This use of neural
networks for both linguistic and acoustic processing produces speech
with exceptional naturalness.  Speech produced by our system has been
found to be more acceptable to listeners than that of other commercial
systems [11].
 
The system excels in learning the specific characteristics of a given
speaker and allows us to develop new dialects and languages rapidly
when compared to other methods.
 
To date, we have developed four voices: two male speakers of American
English, one female speaker of American English and one male speaker
of Mandarin.  Additional linguistic processing has also produced
speech in Spanish, Greek and Turkish with an American accent.
 
 
Regards,
 
Otto Schnurr
 
Speech Processing Research Lab
Chicago Corporate Research Laboratories
Motorola
schnurr at ccrl.mot.com
 
--
 
  [1] Karaali, O., Corrigan, G., Massey, N., Miller, C., Schnurr, O.,
      & Mackie, A. (1998).  A High Quality Text-To-Speech System Composed
      of Multiple Neural Networks.  International Conference on
      Acoustics, Speech and Signal Processing.  Seattle.
 
  [2] Miller, Corey (to appear).  Individuation of Postlexical
      Phonology for Speech Synthesis.  ESCA/COCOSDA Third
      International Workshop on Speech Synthesis, Jenolan Caves
      Australia.
 
  [3] Miller, Corey (1998).  Pronunciation Modeling in Speech Synthesis.
      Doctoral dissertation, University of Pennsylvania.  Philadelphia,
      Pennsylvania.  Published as Technical Report 98-09, Institute for
      Research in Cognitive Science, University of Pennsylvania.
 
  [4] Miller, C., Karaali, O., Massey, N., (1998).  Learning
      Postlexical Variation in an Individual.  Paper presented at the
      Linguistics Society of America Annual Meeting, New York.
 
  [5] Miller, C., Massey, N., Karaali, O. (1998).  Exploring the Nature
      of Postlexical Processes.  Paper presented at the Penn Linguistics
      Colloquium.
 
  [6] Corrigan, G., Massey, N., & Karaali, O. (1997).  Generating Segment
      Durations in a Text-to-Speech System: A Hybrid Rule-Based/Neural
      Network Approach.  In Proceedings of Eurospeech '97.
      pp. 2675-2678.  Rhodes, Greece.
 
  [7] Karaali, O., Corrigan, G., Gerson, I., & Massey, N., (1997).
      Text-to-Speech Conversion with Neural Networks: A Recurrent TDNN
      Approach.  In Proceedings of Eurospeech '97.  pp. 561-564.  Rhodes,
      Greece.
 
  [8] Miller, C., Karaali, O., & Massey, N. (1997).  Variation and
      Synthetic Speech.  Paper presented at NWAVE 26, Quebec, Canada.
 
  [9] Gerson, I., Karaali, O., Corrigan, G., & Massey, N. (1996).  Neural
      Network Speech Synthesis.  Speech Science and Technology (SST-96).
      Australia.
 
 [10] Karaali, O., Corrigan, G., & Gerson, I. (1996).  Speech Synthesis
      with Neural Networks.  Invited paper, World Congress on Neural
      Networks (WCNN-96).  pp. 40-50.  San Diego.
 
 [11] Nusbaum, H., & Luks, T. (1995).  Comparative Evaluation of the
      Quality of Synthetic Speech Produced at Motorola.  Technical Report
      1, University of Chicago.  Chicago, Illinois.

From gary at cs.ucsd.edu  Tue Sep 15 19:29:50 1998
From: gary at cs.ucsd.edu (Gary Cottrell)
Date: Tue, 15 Sep 1998 16:29:50 -0700 (PDT)
Subject: Blended memory for faces: preprint
Message-ID: <199809152329.QAA12129@gremlin.ucsd.edu>

The following paper has been accepted for publication
in NIPS-11. It is available from my web page (given
below):

Dailey, Matthew N., Cottrell, Garrison W. and Busey, Thomas A. (1999)
Facial memory is kernel density estimation (almost).

To appear in Advances in Neural Information Processing Systems
11, MIT Press, Cambridge, MA.

We compare the ability of three exemplar models, each
using three different face stimulus representations, to
account for the probability a human subject responded
``old'' in an old/new facial memory experiment.  The
models are 1) the Generalized Context Model, 2) a
probabilistic sampling model, and 3) a novel model related
to kernel density estimation that explicitly encodes
stimulus distinctiveness.  The representations are 1)
positions of stimuli in MDS ``face space,'' 2) projections
of test faces onto the eigenfaces of the study set, and 3)
a representation based on response to a grid of Gabor
filters.  Of the 9 model/representation combinations, only
the distinctiveness model in MDS space predicts the
observed ``morph familiarity inversion'' effect, in which
subjects' false alarm rate for morphs between similar
parents is higher than their hit rate for the studied
parents of the morphs.  This evidence is consistent with
the hypothesis that human memory for faces is a kernel
density estimation task, with the caveat that distinctive
faces require larger kernels.


Gary Cottrell 619-534-6640 FAX: 619-534-7029 
Faculty Assistant Joy Gorback: 619-534-5948
Computer Science and Engineering 0114
IF USING FED EX INCLUDE THE FOLLOWING LINE:          "Only connect"
3101 Applied Physics and Math Building
University of California San Diego			-E.M. Forster
La Jolla, Ca. 92093-0114

Email: gary at cs.ucsd.edu or gcottrell at ucsd.edu
Home page: http://www-cse.ucsd.edu/~gary/


From backhaus at zedat.fu-berlin.de  Wed Sep 16 04:44:46 1998
From: backhaus at zedat.fu-berlin.de (PD Dr. Backhaus)
Date: Tue, 15 Sep 1998 23:44:46 -0900 (PDT)
Subject: Naples/Ischia Course: Final Call for Abstracts  
In-Reply-To: <Pine.SUN.3.91.980913111408.27536A-100000@berg> 
Message-ID: <Pine.PCW.3.91.980915233921.23310P-100000@Theorie1.biologie.fu-berlin.de>


See:    http://www.fu-berlin.de/backhaus/circul.html

Call for Abstracts (deadline: 19.9.98)

ISTITUTO ITALIANO PER GLI STUDI FILOSOFICI
STUDY PROGRAM ON "FROM NEURONAL CODING TO CONSCIOUSNESS"

INTERNATIONAL SCHOOL OF BIOCYBERNETICS
NEURONAL CODING OF PERCEPTUAL SYSTEMS

Isle of Ischia (Naples), Italy
October 12-17, 1998

OPENING CEREMONY AND INTRODUCTORY LECTURE:
Naples, morning of October 12, 1998

TOPICS:

[1] Vision: Neuronal Coding of Colour, Space, Motion, and Polarized Light
    Perception

[2] Hearing and Touch: Neuronal Coding of Auditory and
    Mechano Perception,
[3] Taste and Smell: Neuronal Coding of Chemical Perception,
[4] Neuronal Coding of Temperature, Pain, Electro, and Magneto Perception
[5] Neuronal Coding, Internal Representations, Qualia and Sensations
    (Consciousness)

ADVISORY BOARD:
A. Clark (USA), M. Kavaliers (C), L. Maffei (I), T. Radil, (CZ), U. Thurm,
(D), G. Tratteur (I), R. de Valois, (USA), R. Wehner, (CH), J. S. Werner, (USA),


SCHOOL DIRECTOR
Werner Backhaus
Freie Universit?t Berlin

For program and registration form see:

http://www.fu-berlin.de/backhaus/circul.html
   


W. Backhaus
homepage:
http://www.fu-berlin.de/backhaus
with a link to our book "Color Vision -
Perspectives from Different Disciplines, eds.
W. Backhaus, R. Kliegl, and J.S. Werner. De Gruyter,
Berlin - New York, 1998.

New Address:
W. Backhaus
Theoretical and Experimental Biology
Freie Universitaet Berlin
Villa, Koenigin-Luise-Str. 29
14195 Berlin

Tel./Fax.: +49-30-838 2692
e-mail: backhaus at zedat.fu-berlin.de




From ken at phy.ucsf.EDU  Wed Sep 16 05:22:21 1998
From: ken at phy.ucsf.EDU (Ken Miller)
Date: Wed, 16 Sep 1998 02:22:21 -0700 (PDT)
Subject: Paper Available: Model of V1 Development
Message-ID: <13823.33613.834512.602479@coltrane.ucsf.edu>

FTP-host:	ftp.keck.ucsf.edu
FTP-filename:   pub/ken/jn-erwin.ps.gz
URL:		ftp://ftp.keck.ucsf.edu/pub/ken/jn-erwin.ps.gz
Uncompressed version is available by omitting the '.gz'. 

The following paper is available by anonymous ftp.  It can also be
obtained from my web page:

http://www.keck.ucsf.edu/~ken
  (click on 'Publications';
   or alternatively, go directly to
   http://www.keck.ucsf.edu/~ken/miller.htm#references)

--------------------------------------------------------------------------

E. Erwin and K.D. Miller (1998).  ``Correlation-Based Development of
Ocularly-Matched Orientation and Ocular Dominance Maps: Determination
of Required Input Activities.''  In press, Journal of Neuroscience.

ABSTRACT:
We extend previous models for separate development of ocular dominance
and orientation selectivity in cortical layer 4 by exploring
conditions permitting combined organization of both properties.  These
conditions are expressed in terms of functions describing the degree
of correlation in the firing of two inputs from the lateral geniculate
nucleus (LGN), as a function of their retinotopic separation and their
``type'' (ON-center or OFF-center, left-eye or right-eye).

The development of ocular dominance requires that an input's
correlations with other inputs from the same eye be stronger than or
equal to its correlations with inputs of the opposite eye, and
strictly stronger at small retinotopic separations.  This must be true
after summing correlations with inputs of both center types.  The
development of orientation-selective simple cells requires that (1) an
input's correlations with other inputs of the same center type be
stronger than its correlations with inputs of the opposite center type
at small retinotopic separation; and (2) this relationship reverse at
larger retinotopic separations within an arbor radius (the radius over
which LGN cells can project to a common cortical point).  This must be
true after summing correlations with inputs serving both eyes.

For orientations to become matched in the two eyes, correlated
activity within the receptive fields must be maximized by specific
between-eye alignments of ON and OFF subregions.  Thus the
correlations between the eyes must differ depending on center type,
and this difference must vary with retinotopic separation within an
arbor radius.

These principles are satisfied by a wide class of correlation
functions.  Combined development of ocularly matched orientation maps
and ocular dominance maps can be achieved either simultaneously or
sequentially.  In the latter case, the model can produce a correlation
between the locations of orientation map singularities and local
ocular dominance peaks similar to that observed physiologically.

The model's main prediction is that the above correlations should
exist among inputs to cortical layer 4 simple cells before vision.  In
addition, mature simple cells are predicted to have certain
relationships between the locations of the ON and OFF subregions of
the left- and right-eyes' receptive fields.

--------------------------------------------------------------------

Ken Miller
 
        Kenneth D. Miller               telephone: (415) 476-8217
        Dept. of Physiology		fax: (415) 476-4929
        UCSF                            internet: ken at phy.ucsf.edu
        513 Parnassus			www: http://www.keck.ucsf.edu/~ken
        San Francisco, CA 94143-0444    


From wahba at stat.wisc.edu  Wed Sep 16 17:09:51 1998
From: wahba at stat.wisc.edu (Grace Wahba)
Date: Wed, 16 Sep 1998 16:09:51 -0500 (CDT)
Subject: Bias-Variance, GACV paper
Message-ID: <199809162109.QAA15339@hera.stat.wisc.edu>

The following paper has been accepted for oral
presentation at NIPS*98:  Available as University
of Wisconsin-Madison Statistics Dept TR997 in 

http://www.stat.wisc.edu/~wahba -> TRLIST
..................................................
The Bias-Variance Tradeoff and the Randomized GACV

Grace Wahba*, Xiwu Lin, Fangyu Gao,  Dong Xiang, 
Ronald Klein MD and Barbara Klein MD


We propose a new in-sample cross validation based 
method (randomized GACV) for choosing smoothing 
or bandwidth parameters that govern the bias-variance 
or fit-complexity tradeoff in `soft' classification. 
Soft classification refers to a learning procedure 
which estimates the probability
that an example with a given attribute vector is in 
class 1 {\it vs} class 0. The target for optimizing the 
the tradeoff is the Kullback-Liebler distance between 
the estimated probability distribution and the `true' 
probability distribution, representing knowledge 
of an infinite population. The method uses a 
randomized estimate of the trace of a Hessian and mimics 
cross validation at the cost of a single relearning with 
perturbed outcome data.

*corresponding author wahba at stat.wisc.edu 
.................................................


From jbower at bbb.caltech.edu  Wed Sep 16 19:36:19 1998
From: jbower at bbb.caltech.edu (James M. Bower)
Date: Wed, 16 Sep 1998 15:36:19 -0800
Subject: which came first the idea or the model?
Message-ID: <v04011785b225afa8cd49@[131.215.137.89]>

Not to pick nits, but,

In response to Richard Granger:

In his original email he stated:

>This (iterative finer grained representation of an odor) is an instance in
>which modeling of physiological activity in anatomical circuitry gave rise to
>an operation that was unexpected from behavioral studies,

However, the idea that cortical (olfactory) processing involved iterative
response refinement and specificity was actually a central thesis of the
monograph published by Lynch in 1986.  This monograph includes a discussion
of supporting behavioral data.

Reference:  G. Lynch, Synapses, Circuits, and the beginnings of memory. MIT
Press. 1986

It was clearly the objective of the subsequent model by Granger and Lynch
to see if this specific idea could be incorporated into a "cortical like"
structure.  Thus, as I indicated earlier, the model essentially served to
demonstrate a particular idea, which as Richard points out is still
controversial.

Second, the first reference I know for the Granger model was actually in
1988, two years before publication of the physiological studies claimed to
serve as its foundation:

Reference:  Granger et al., Partitioning of sensory data by a cortical
network, Neural Information Processing Systems. D. Anderson Ed. AIP, 1988}.

To quote previous email:


>"physiological induction and expression rules for
>synaptic long-term potentiation (LTP; Kanter & Haberly, '90; Jung et al.,
>'90)."

In fact, as I remember, the Granger model assumed that the only LTP was in
the synaptic connections made by the Lateral olfactory tract (LOT) not in
the association fiber system.  Kanter and Haberly (1990) actually showed
that the association fiber system is the major source of LTP in olfactory
cortex.

Thus, in summary, models can indeed generate novel ideas about brain
function.  And I agree completely with Richard that physiologically and
anatomically based models are much more likely to do so.  We ourselves have
built and "mined" many such models.  However, it is very important that a
clear distinction be made between models of this type, and those intended
to demonstrate a previously proposed functional idea using a mix of
convenient "neurobiological-like" structures and mechanisms.  There is
nothing wrong with such demonstration models, it is just not appropriate to
claim that they originated the ideas that they were actually designed to
demonstrate.


Jim Bower

           ***************************************
                       James M. Bower
                     Division of Biology
                     Mail code:  216-76
                           Caltech
                     Pasadena, CA 91125
                      (626) 395-6817
                      (626) 795-2088 FAX

      WWW addresses for:
        laboratory                 http://www.bbb.caltech.edu/bowerlab
        GENESIS:                   http://www.bbb.caltech.edu/GENESIS
        science education reform   http://www.caltech.edu/~capsi and
				   http://www.nas.edu/rise/examp81.htm

	J. Computational Neuroscience  http://www.bbb.caltech.edu/JCNS/
	Annual CNS meetings	http://www.bbb.caltech.edu/cns-meetings

From reza at bme.jhu.edu  Thu Sep 17 09:30:08 1998
From: reza at bme.jhu.edu (Reza Shadmehr)
Date: Thu, 17 Sep 1998 09:30:08 -0400 (EDT)
Subject: a paper on human adaptive control
Message-ID: <199809171330.JAA04456@bme.jhu.edu>


Dear Connectionists:

An abridged version of the following paper on human adaptive
control will be presented at NIPS this year.  It is available from  

http://www.bme.jhu.edu/~reza/nb_paper.pdf


   Computational Nature of Human Adaptive Control During
       Learning of Reaching Movements in Force Fields

                Nikhil Bhushan and Reza Shadmehr

Learning to make reaching movements in force fields was used as a
paradigm to explore the system architecture of the biological adaptive
controller.  We compared the performance of a number of candidate
control systems that acted on a model of the neuromuscular system of
the human arm and asked how well the dynamics of the candidate system
compared with the behavior of the biological controller.  We found
that control via a supra-spinal system that utilized an adaptive
inverse model resulted in dynamics that were similar to that observed
in our subjects, but lacked essential characteristics.  These
characteristics pointed to a different architecture where descending
commands were influenced by an adaptive forward model.  However, we
found that control via a forward model alone also resulted in dynamics
that did not match the behavior of the human arm.  We considered a
third control architecture where a forward model was used in
conjunction with an inverse model and found that the resulting
dynamics were remarkably similar to that observed in the experimental
data.  The essential property of this control architecture was that it
predicted a complex pattern of near-discontinuities in hand trajectory
in the novel force field. A nearly identical pattern was observed in
our subjects, suggesting that generation of descending motor commands
was likely through a control system architecture that included both
adaptive forward and inverse models.  We further demonstrate that as
subjects learned to make reaching movements, adaptation rates for the
forward and inverse models could be independently estimated and the
resulting changes in performance of subjects from movement to movement
could be accurately accounted for.  It appeared that in learning to
make reaching movements, adaptation of the forward model played a very
significant role in reducing the errors in performance.  Finally, we
found that after a period of consolidation, the rates of adaptation in
the models were significantly larger than those observed before the
memory had consolidated.  This suggested that consolidation of motor
memory may have coincided with freeing of certain computational
resources for subsequent learning.

From rafal at idsia.ch  Thu Sep 17 10:04:18 1998
From: rafal at idsia.ch (Rafal Salustowicz)
Date: Thu, 17 Sep 1998 16:04:18 +0200 (MET DST)
Subject: Prediction and Automatic Task Decomposition
Message-ID: <Pine.GSO.3.92.980917160123.4324C-100000@broccolo>


      LEARNING TO PREDICT THROUGH PROBABILISTIC INCREMENTAL
      PROGRAM  EVOLUTION  AND  AUTOMATIC TASK DECOMPOSITION

      Rafal Salustowicz                 Juergen Schmidhuber

                 Technical Report IDSIA-11-98

Analog gradient-based  recurrent  neural  nets  can learn complex
prediction tasks.  Most,   however,  tend to fail in case of long
minimal time lags between relevant training events.  On the other
hand, discrete methods such as search in a space of event-memori-
zing programs are not  necessarily  affected  at all by long time
lags:  we show that  discrete  "Probabilistic Incremental Program
Evolution" (PIPE) can solve several long time lag tasks that have
been successfully solved by only one analog method ("Long Short-
Term Memory" - LSTM).  In fact,  sometimes  PIPE even outperforms
LSTM. Existing discrete methods, however, cannot easily deal with
problems  whose solutions  exhibit comparatively high algorithmic
complexity.   We overcome this drawback by introducing filtering,
a novel,  general,  data-driven  divide-and-conquer technique for
automatic  task decomposition that is not limited to a particular
learning method. We compare PIPE plus filtering to various analog
recurrent net methods.

      ftp://ftp.idsia.ch/pub/rafal/TR-11-98-filter_pipe.ps.gz
      http://www.idsia.ch/~rafal/research.html

Rafal & Juergen, IDSIA, Switzerland                  www.idsia.ch


From Bill_Warren at Brown.edu  Thu Sep 17 08:57:51 1998
From: Bill_Warren at Brown.edu (Bill Warren)
Date: Thu, 17 Sep 1998 08:57:51 -0400 (EDT)
Subject: Please post -- thanks!
Message-ID: <v03010d02b2267ee6a8d6@[128.148.208.31]>


FACULTY POSITION IN VISUAL PERCEPTION, BROWN UNIVERSITY:  The Department of
Cognitive and Linguistic Sciences invites applications for a position in
visual perception beginning July 1, 1999.  An appointment will be made
either as a three-year renewable tenure-track Assistant Professor, or a
tenured Associate Professor. Applicants must have a strong experimental
research program combined with strong computational or theoretical
interests in vision, a broad teaching ability in cognitive science at both
the undergraduate and graduate levels, and an interest in contributing to
an interdisciplinary vision group spanning the departments of applied
mathematics, neuroscience, psychology, engineering, and computer science.
Applicants should have completed all Ph.D. requirements by no later than
July 1, 1999.  Women and minorities are especially encouraged to apply.
Send curriculum vitae, three letters of reference, reprints, and preprints
of publications, and a one-page statement of research interests to
Perception Search Committee, Dept. Of Cognitive and Linguistic Sciences,
Brown University, Providence, R.I. 02912, by January 1, 1999.

Brown University is an Equal Opportunity/Affirmative Action Employer.

                        -- Bill

                        William H. Warren, Professor
                        Dept. of Cognitive & Linguistic Sciences
                        Box 1978
                        Brown University
                        Providence, RI  02912
                        (401) 863-3980 ofc, 863-2255 FAX
                        Bill_Warren at brown.edu



From M.Usher at ukc.ac.uk  Thu Sep 17 11:24:55 1998
From: M.Usher at ukc.ac.uk (M.Usher@ukc.ac.uk)
Date: Thu, 17 Sep 1998 16:24:55 +0100
Subject: article on LATERAL INTERACTIONS
Message-ID: <199809171524.QAA17844@snipe.ukc.ac.uk>


The following article, to appear in SPATIAL VISION
(Special Issue on "Long Range Spatial Interactions in Vision"),
can now be accessed from:
http://ukc.ac.uk/psychology/people/usherm/ (at recent  publications)

The article addresses psychophysical data that indicate facilitatory
lateral interaction in visual processing, and presents a computational
model based on principles from neural information processing and signal
detection theory, to explain those interactions.

-Marius Usher

Department of Psychology
University of Kent

--------------------------------------------------------------

     MECHANISMS FOR SPATIAL INTEGRATION IN VISUAL DETECTION:
        A model based on lateral interactions

    Marius Usher, Yoram Bonneh, Dov Sagi & Michael Herrmann

                   Abstract

Studies of visual detection of multiple targets  show a weak
improvement of thresholds with the number of targets, which
corresponds to a fourth-root power law. We find this result to 
be inconsistent with probability summation models, and account
for it by a model of ``physiological'' integration that is based 
on excitatory lateral interactions in the visual cortex.
The model explains several phenomena which are confirmed by the
experimental data, such as the absence of spatial and temporal
uncertainty effects, temporal summation curves, and facilitation
by a pedestal in 2AFC tasks. The summation exponents are dependent 
on the strength of the lateral interactions, and on the distance 
and orientation relationship between the elements.


From cmerz at saanen.ics.uci.edu  Thu Sep 17 12:48:43 1998
From: cmerz at saanen.ics.uci.edu (Chris Merz)
Date: Thu, 17 Sep 1998 09:48:43 -0700
Subject: Articles on combining multiple models
Message-ID: <9809170948.aa29866@paris.ics.uci.edu>


I am announcing the availability of several articles related to
classification and regression by combining models. The first list
below contains the titles, url's and FEATURES of each article. The
second list contains the abstracts.

Please contact me at cmerz at ics.uci.edu with any questions.

Thanks,
Chris Merz

================== Titles, URL's and *** FEATURES *** =================

1. My dissertation:  

       "Classification and Regression by Combining Models"
       Merz, Christopher J. (1998) 
       http://www.ics.uci.edu/~cmerz/thesis.ps

       *** DESCRIBES TWO ROBUST METHODS FOR COMBINING LEARNED MODELS ***
       *** USING TECHNIQUES BASED ON SINGULAR VALUE DECOMPOSITION.   ***
       *** CONTAINS COMPREHENSIVE BACKGROUND AND SURVEY CHAPTERS.    ***

2. Preprints of two accepted Machine Learning Journal articles:

       "A Principal Components Approach to Combining Regression Estimates", 
       Merz, C. J., Pazzani, M. J. (1997) 
       To appear in the Special Issue of Machine Learning on Integrating 
       Multiple Learned Models. 
       http://www.ics.uci.edu/~cmerz/jr.html/mlj.pcr.ps

       *** SHOWS HOW PCA MAY BE USED TO SYSTEMATICALLY EXPLORE  ***
       *** WEIGHT SETS WITH VARYING DEGREES OF REGULARIZATION.  ***

       "Using Correspondence Analysis to Combine Classifiers", 
       Merz, C. J. (1997) 
       To appear in the Special Issue of Machine Learning on Integrating 
       Multiple Learned Models. 
       http://www.ics.uci.edu/~cmerz/jr.html/mlj.scann.ps

       *** SHOWS THAT THE SCANN METHOD COMBINES BOOSTED MODEL  ***
       *** SETS BETTER THAN BOOSTING DOES.                     ***

3. A bibtex file of the references in my dissertation survey:

       http://www.ics.uci.edu/~cmerz/bib.html/survey.bib

       *** COMPREHENSIVE BIBLIOGRAPHY - MANY ABSTRACTS INCLUDED ***

======================== Abstracts ==========================

1.      "Classification and Regression by Combining Models"

     Two novel methods for combining predictors are introduced in this
thesis; one for the task of regression, and the other for the task of
classification. The goal of combining the predictions of a set of
models is to form an improved predictor. This dissertation
demonstrates how a combining scheme can rely on the stability of the
consensus opinion and, at the same time, capitalize on the unique
contributions of each model.

     An empirical evaluation reveals that the new methods consistently
perform as well or better than existing combining schemes for a
variety of prediction problems. The success of these algorithms is
explained empirically and analytically by demonstrating how they
adhere to a set of theoretical and heuristic guidelines.

     A byproduct of the empirical investigation is the evidence that
existing combining methods fail to satisfy one or more of the
guidelines defined. The new combining approaches satisfy these
criteria by relying upon Singular Value Decomposition as a tool for
filtering out the redundancy and noise in the predictions of the learn
models, and for characterizing the areas of the example space where
each model is superior. The SVD-based representation used in the new
combining methods aids in avoiding sensitivity to correlated
predictions without discarding any learned models. Therefore, the
unique contributions of each model can still be discovered and
exploited. An added advantage of the combining algorithms derived in
this dissertation is that they are not limited to models generated by
a single algorithm; they may be applied to model sets generated by a
diverse collection of machine learning and statistical modeling
methods.

     The three main contributions of this dissertation are: 
        1. The introduction of two new combining methods capable of 
           robustly combining classification and regression estimates, and
           applicable to a broad range of model sets. 
        2. An in-depth analysis revealing how the new methods address the 
           specific problems encountered in combining multiple learned
           models. 
        3. A detailed account of existing combining methods and an 
           assessment of where they fall short in the criteria for 
           combining approaches. 

----------------

2. Preprints of two accepted Machine Learning Journal articles:

"A Principal Components Approach to Combining Regression Estimates"
Christopher J. Merz and Michael J. Pazzani

  Abstract

  The goal of combining the predictions of multiple learned 
  models is to form an improved estimator.  A combining strategy must be 
  able to robustly handle the inherent correlation, or 
  multicollinearity, of the learned models while identifying the unique 
  contributions of each.  A progression of existing approaches and their 
  limitations with respect to these two issues are discussed.  A new 
  approach, PCR*, based on principal components regression is proposed 
  to address these limitations.  An evaluation of the new approach on a 
  collection of domains reveals that 1) PCR* was the most robust 
  combining method, 2) correlation could be handled without eliminating 
  any of the learned models, and 3) the principal components of the 
  learned models provided a continuum of ``regularized'' weights from 
  which PCR* could choose.


"Using Correspondence Analysis to Combine Classifiers"
Christopher J. Merz

  Abstract

  Several effective methods have been developed recently for improving
  predictive performance by generating and combining multiple learned
  models.  The general approach is to create a set of learned models
  either by applying an algorithm repeatedly to different versions of
  the training data, or by applying different learning algorithms to the
  same data.  The predictions of the models are then combined according
  to a voting scheme.  This paper focuses on the task of combining the
  predictions of a set of learned models.  The method described uses the
  strategies of stacking and Correspondence Analysis to model the
  relationship between the learning examples and their classification by
  a collection of learned models.  A nearest neighbor method is then
  applied within the resulting representation to classify previously
  unseen examples.  The new algorithm does not perform worse than, and
  frequently performs significantly better than other combining
  techniques on a suite of data sets.

----------------

3. The bibtex file contains all of the references in my dissertation,
   including the survey. I've managed to paste in the abstracts of 
   many of the articles. I am willing to update this bibliography
   if any authors want to contribute references, abstracts and/or URL's.


From jbower at bbb.caltech.edu  Thu Sep 17 14:36:16 1998
From: jbower at bbb.caltech.edu (James M. Bower)
Date: Thu, 17 Sep 1998 10:36:16 -0800
Subject: plausibility
Message-ID: <v040117b6b227065373b9@[131.215.137.89]>

A non-text attachment was scrubbed...
Name: not available
Type: multipart/alternative
Size: 2160 bytes
Desc: not available
Url : https://mailman.srv.cs.cmu.edu/mailman/private/connectionists/attachments/00000000/aa92f5db/attachment-0001.bin

From jagota at cse.ucsc.edu  Thu Sep 17 15:26:37 1998
From: jagota at cse.ucsc.edu (Arun Jagota)
Date: Thu, 17 Sep 1998 12:26:37 -0700
Subject: new survey-type publication
Message-ID: <199809171926.MAA15141@arapaho.cse.ucsc.edu>



New refereed e-publication (action editor: Risto Miikkulainen)

comprehensive area bibliography with thematic and keyword indices

Samuel Kaski, Jari Kangas, Teuvo Kohonen, 
Bibliography of Self-Organizing Map (SOM) Papers: 1981--1997, 
Neural Computing Surveys, 1, 102--350, 1998, 3343 references.
http://www.icsi.berkeley.edu/~jagota/NCS

Abstract:

The Self-Organizing Map (SOM) algorithm has attracted an ever
increasing amount of interest among researchers and practitioners in
a wide variety of fields. The SOM and a variant of it, the LVQ, have
been analyzed extensively, a number of variants of them have been
developed and, perhaps most notably, they have been applied
extensively within fields ranging from engineering sciences to
medicine, biology, and economics. We have collected a comprehensive
list of 3343 scientific papers that use the algorithms, have
benefited from them, or contain analyses of them. The list is
intended to serve as a source for literature surveys. We have
provided both a thematic and a keyword index to help finding
articles of interest.


From joe at cs.caltech.edu  Thu Sep 17 17:28:59 1998
From: joe at cs.caltech.edu (Joe Sill)
Date: 17 Sep 1998 21:28:59 GMT
Subject: Special issue on VC dimension
Message-ID: <6truur$h4m@gap.cco.caltech.edu>

Machine learning theorists may be interested in a recent issue
of the journal Discrete Applied Mathematics (Vol 86, Number 1, 
August 18, 1998). This special issue, edited by John Shawe-Taylor, is 
devoted entirely to the VC dimension.

Contents:

"Combinatorial variability of Vapnik-Chervonenkis classes with 
applications to sample compression schemes"
S. Ben-David and A. Litman

"A graph-theoretic generalization of the Sauer-Shelah lemma"
N. Cesa-Bianchi and D. Haussler

"Scale-sensitive dimensions and skeleton estimates for classification"
M. Horvath and G. Lugosi

"Vapnik-Chervonenkis dimension of recurrent neural networks"
P. Koiran and E.D. Sontag

"The degree of approximation of sets in euclidean space using sets
with bounded Vapnik-Chervonenkis dimension"
V. Maiorov and J. Ratsaby

"The capacity of monotonic functions"
J. Sill

"Fluctuation bounds for sock-sorting and other stochastic processes"
D. Steinsaltz

From granger at uci.edu  Thu Sep 17 20:20:39 1998
From: granger at uci.edu (Richard Granger)
Date: Thu, 17 Sep 1998 17:20:39 -0700
Subject: Unexpected hypotheses arising from brain circuit simulation
Message-ID: <v02140b00b226dfa12849@[128.200.120.16]>


  Jim writes:
> the idea that cortical (olfactory) processing involved iterative
> response refinement and specificity was actually a central thesis of the
> monograph published by Lynch in 1986.
>
>Reference:  G. Lynch, Synapses, Circuits, and the beginnings of memory. MIT
>Press. 1986

If that's so, then its author doesn't know it.  The monograph neither
contains nor presages the hypothesis that appears in our 1990 Science
paper.  (Perhaps this is being confused with operations of excitatory
associational feedback fibers, which Lynch in 1986 hypothesized might cycle
repeatedly in response to a single input, rapidly building a
"representation" of an odor.  Such an operation is of course utterly
unrelated to the finding being discussed: there's no mention of bulb, no
mention of theta cycles, no mention of inhibitory feedback, no hierarchy.
The author states that the hypothesis that appears in the Science paper was
not even conceived of at the time of the 1986 monograph.)

The 1986 monograph was an early and fruitful step in the field of modeling
of real biological systems.  It is replete with attempts at identifying
computational concomitants of a range of biological phenomena, and with
compiling and integrating data related to research on LTP, the olfactory
system, and the hippocampus.  In particular, a central phenomenon is the
(4-8 Hz) theta rhythm, which:
  i) entrains cells throughout the olfactory system, hippocampus and much
of neocortex exclusively during learning and exploration (not during sleep,
or in a home cage, or passive restraint, etc.), (Macrides, 1975; Macrides
et al., 1982; Komisaruk, 1970; Otto et al., 1991; Kauer 1991), and
  ii) has been shown to be the optimal stimulation pattern for induction of
LTP (Larson et al., 1986; Diamond et al., 1987) due to an endogenously
occurring time-dependent gaba-b inactivation of gaba terminals (Mott &
Lewis, 1991).

For historical interest, after the monograph, what one finds is a
succession of papers by us, all on studies of this system (two papers in
1988 and four in '89), all attempting to identify various aspects of
function from the structure and operation of the system.  Our studies
focused in particular on the theta rhythm (such as the marvelous fact that
in small mammals the rhythm literally drives overt behavior (sniffing,
moving whiskers, etc., all at 5 Hz) during exploration).  Those papers make
the historical case glaringly:  they contain some interesting early
findings on the computational advantages of the synchrony itself provided
by theta, and on local lateral inhibition, and on refinement of recognition
with episodes of training and incremental steps of LTP, but nothing at all
on hierarchical clustering (the topic of the 1990 Science paper).   In
particular, the 1988 papers made the point that the initial sniff clustered
inputs (as was expected from work by other researchers), but nothing on
later sniffs subclustering.

Then we took on the puzzling observation that the cortical model was
producing different outputs over time, as it sampled a single input.  We
showed that not only do the initial cortical responses tend to empirically
cluster inputs, as mentioned, but also that later cortical responses,
paradoxically, tend to differentiate those same inputs.  We observed that
the system's inhibitory feedback (Price, 1973), and the long-lasting nature
of the resulting bulb granule cell IPSPs (Nicoll, 1969; Mori, 1987) tended
to selectively remove the same portions of the inputs that were giving rise
to the initial responses, and that therefore the system might not simply be
first clustering and then differentiating, but actually clustering,
sub-clustering, sub-sub-clustering, etc., over iterative cycles of the
theta rhythm.  This constituted a new hypothesis; not successive episodes
of learning; not steps of LTP; not cycling associational pathways; but
successive iterative feedforward excitation and feedback inhibition, giving
rise to a sequence of distinct different outputs on successive theta
cycles, traversing a hierarchical tree from general (clusters) to specific
(subclusters).  We wrote this up in 1989 and submitted it to Science, where
it was accepted and appeared in 1990.  It turned out that a relationship
could be shown between this circuit behavior and a nonstandard method of
hierarchical clustering; and it further turned out that this method was
unusually efficient with respect to time and space complexity.

Thus a puzzling operation that arose from interactions among a large number
of features in a biological simulation, turned out to yield an unusual
computational function, and did so in an unusually efficient manner.  It
was this finding that was surprising (to us, and to the reviewers and
editors of Science).

>It was clearly the objective of the subsequent model by Granger and Lynch
>to see if this specific idea could be incorporated into a "cortical like"
>structure.

Clearly not, given the above.  It was clearly our objective to explore the
model for its behaviors and functions, and to attempt to identify and
characterize the emergent properties of its many, many parts.


Our hypothesis is that a function of the olfactory system is the iterative
decomposition of inputs over successive rhythmic cycles of operation:
specifically, that sequential outputs of the cortex (every 200 msec) give
different information, corresponding to successive hierarchical clusters
and subclusters.  This is a novel hypothesis (or was, in 1990), and has
been much cited and studied for its behavioral, neurobiological,
psychological, and computational consequences.  It was derived directly
from, and incorporates, the detailed characteristics of the biological
system itself: theta, LTP induction and expression rules, sparse
connectivity, feedforward excitation, LOT and associational pathways,
feedback inhibition, mitral, tufted and granule cells, local lateral
inhibition, differential time courses of EPSPs and IPSPs in different cell
types, axonal arborization radius of inhibitory interneurons, etc.  We'd be
glad simply to take the credit for having come up with it by inspection, if
that were true.  The fact that we actually came up with it only after
extensive construction and observation of anatomically and physiologically
realistic models is a notable methodological point.  That this new idea has
given rise to a fruitful series of subsequent behavioral, physiological and
theoretical studies (in our labs and others) is a notable consequence.
That the hypothesis will undoubtedly turn out to be wrong in many ways is
the point of ongoing scientific inquiry.  We should continue to doubt, and
to study, and to counter-hypothesize, and to experiment.

-Rick Granger



[Footnote:

>In fact, as I remember, the Granger model assumed that the only LTP was in
>the synaptic connections made by the Lateral olfactory tract (LOT) not in
>the association fiber system.  Kanter and Haberly (1990) actually showed
>that the association fiber system is the major source of LTP in olfactory
>cortex.

No, actually, our models have LTP both in the LOT and the association fiber
system.  We have even studied the individual effects of these two pathways,
and their possible differential contributions to the function of the
overall system.]




From M.Usher at ukc.ac.uk  Fri Sep 18 09:40:58 1998
From: M.Usher at ukc.ac.uk (M.Usher@ukc.ac.uk)
Date: Fri, 18 Sep 1998 14:40:58 +0100
Subject: web-site correction
Message-ID: <199809181340.OAA05419@snipe.ukc.ac.uk>

There was an error in the web-site address I posted yesterday for
our article, to appear in SPATIAL VISION
(Special Issue on "Long Range Spatial Interactions in Vision"),

The correct address is:

http://www.ukc.ac.uk/psychology/people/usherm/public.html

I appologize for the error

-Marius Usher
 
Department of Psychology
University of Kent
 
--------------------------------------------------------------
 
     MECHANISMS FOR SPATIAL INTEGRATION IN VISUAL DETECTION:
        A model based on lateral interactions
 
    Marius Usher, Yoram Bonneh, Dov Sagi & Michael Herrmann
 
                   Abstract
 
Studies of visual detection of multiple targets  show a weak
improvement of thresholds with the number of targets, which
corresponds to a fourth-root power law. We find this result to
be inconsistent with probability summation models, and account
for it by a model of ``physiological'' integration that is based
on excitatory lateral interactions in the visual cortex.
The model explains several phenomena which are confirmed by the
experimental data, such as the absence of spatial and temporal
uncertainty effects, temporal summation curves, and facilitation
by a pedestal in 2AFC tasks. The summation exponents are dependent
on the strength of the lateral interactions, and on the distance
and orientation relationship between the elements.
 

From xw3f at avery.med.virginia.edu  Fri Sep 18 10:39:29 1998
From: xw3f at avery.med.virginia.edu (Xiangbao Wu)
Date: Fri, 18 Sep 1998 10:39:29 -0400
Subject: Postdoctoral Position Available
Message-ID: <199809181439.KAA213090@avery.med.Virginia.EDU>

            COMPUTATIONAL NEUROSCIENCE POSTDOCTORAL RESEARCH ASSOCIATE 
		         UNIVERSITY OF VIRGINIA
		       CHARLOTTESVILLE, VIRGINIA

A postdoctoral research associate position is available in the
laboratory of Dr. William B Levy.

The applicant will work on a research project in at least one of
four areas: (1) quantitative neural network theory of hippocampal
function using network simulations; (2) computational analysis
of neural network solutions to cognitive tasks; (3) effects of activity
fluctuations and noise to hippocampal function by computational simulations;
(4) a theory of hippocampal neocortical interactions.

Applicants should have a strong background in quantitative research and some
basic knowledge of neuroscience or cognitive science.  Candidates will also
be judged on their relevant research experience and communication skills.

The starting date is flexible. The position is available for 1-2 years
depending on accomplishment.

Please send a CV, a letter describing research interests and background,
and three (3) references by post or email to:

William B Levy
Department of Neurosurgery
Health Sciences Center Box 420
University of Virginia
Charlottesville, Va 22908
Email: wbl at virginia.edu

From jbower at bbb.caltech.edu  Thu Sep 17 14:36:16 1998
From: jbower at bbb.caltech.edu (James M. Bower)
Date: Thu, 17 Sep 1998 10:36:16 -0800
Subject: plausibility
Message-ID: <mailman.956.1149591460.29955.connectionists@cs.cmu.edu>

In response to my recent post, I received the following note:


>> actually designed to demonstrate.
>                          ^
> You surely meant to say test.
>


In fact I meant "demonstrate", and this is a very important distinction and
issue in brain-like modeling.  It is my view that the majority of the
models generated in this field to date (especially those of the NN type)
are actually demonstration type models, and not in any real sense "tests".
In order to be a test, there must be some mechanism for formally evaluating
the plausibility of a particular model, given the available neurobiological
data.  We have recently published a paper suggesting one (some would say
the only) formal approach to this problem:

Baldi, P.,  Vanier, M.C., and Bower, J.M.  (1998)  On the use of Bayesian
methods for evaluating compartment neural models.  J. Computational
Neurosci. 5: 285-314.

However, at present there are no accepted standards for such an evaluation
(in fact there is almost no discussion of this issue).  Instead, far too
much modeling involves twiddling the right knobs to get the functional
results you want.  Those few experimentalists interested in modeling
usually evaluate a models plausibility based mostly on intuition.

It is for this reason that the question of prior functional assumptions is
so important, and why I continue to try to draw a strong distinction
between modeling based first on anatomy and physiology and efforts intended
to demonstrate the plausibility of a particular preconceived idea (c.f.
Bower, J.M. (1995)  Reverse engineering the nervous system: an in vito, in
vitro, and in computo  approach to understanding the mammalian olfactory
system.  In: An Introduction to Neural and Electronic Networks, Second
Edition. S. Zornetzer, J. Davis, and C. Lau, editors.  Academic Press.  pp.
3-28.).  If the functional idea truly comes about as a result of the
modeling, and not vice versa, then it is more likely (although still far
from certain) that the revealed mechanisms have something to do with the
real brain.  Of course, this is the same reason that some modelers try to
blur this distinction.


Jim Bower

================ Message 2 of 4 ================


From jbower at bbb.caltech.edu  Fri Sep 18 15:55:56 1998
From: jbower at bbb.caltech.edu (James M. Bower)
Date: Fri, 18 Sep 1998 11:55:56 -0800
Subject: iterative processing
Message-ID: <mailman.957.1149591461.29955.connectionists@cs.cmu.edu>

Read carefully, Richard's response to my previous email indicates pretty
clearly the degree to which prior thinking about how the olfactory system
works influenced the development of the olfactory model.  This is the sole
point I have been trying to make.

In particular he states that:

"The 1986 monograph was an early and fruitful step in the field of modeling
of real biological systems."

Of course there was no model in the monograph -- the discussion involved
speculations based on biological data and certain assumptions concerning
olfactory processing (e.g. iterative refinement of olfactory response).
Richard's long history makes pretty clear that it was those assumptions
that drove the subsequent modeling.  This, I presume, is what Richard meant
by a "fruitful step".  The "fruit" in this case, being the model.

Richard's recounting of the history also makes it clear that the original
claim that the model was based on a wide range of biological data,
including data published in 1990 and after is somewhat difficult to
reconcile with the statement that:

"two papers (were written) in 1988 and four in '89, all attempting to
identify various aspects of function from the structure and operation of
the system."


Finally, there are many technical and biological issues that could be
raised concerning the assumptions and conclusions of this particular
modeling effort (The location of LTP, or evidence that rats can apparently
recognize odors on a single sniff (theta cycle), or the issue of whether
olfactory perception space is really heirarchically clustered), however, it
was never my intent to argue about the model itself.  Instead, I was trying
to make the point that one has to be very careful to distinguish between
models that assume a particular function, and then try to identify a
biologically plausible structure that might provide it, and models that
start with structure, and try to infer function.   It may be that this
distinction is blurry to some -- however, if one has done the later, the
distinction is obvious and important.

Jim Bower


================ Message 3 of 4 ================


From jbower at bbb.caltech.edu  Fri Sep 18 17:08:19 1998
From: jbower at bbb.caltech.edu (James M. Bower)
Date: Fri, 18 Sep 1998 13:08:19 -0800
Subject: cerebellum
Message-ID: <mailman.958.1149591461.29955.connectionists@cs.cmu.edu>

Just catching up on the NN/Brain discussions.

And another cautionary note not unrelated to my previous emails:  There is
growing evidence that the cerebellum may not be a "motor control system" in
the classical sense.  If correct then models that were constructed under
this assumption will need to be revisited.


Jim Bower


================ Message 4 of 4 ================


From jbower at bbb.caltech.edu  Fri Sep 18 17:32:48 1998
From: jbower at bbb.caltech.edu (James M. Bower)
Date: Fri, 18 Sep 1998 13:32:48 -0800
Subject: Chickens and eggs again -- the sequence matters
Message-ID: <mailman.959.1149591461.29955.connectionists@cs.cmu.edu>

Sorry everyone for being so tight about this.

DeLiang Wang wrote:


>His theory and prediction led to the two first
>confirmative reports by Echorn et al. (1988) and Gray et
>al. (1989).

However, in fact, at least Charlie Gray did this work because of his
interest in oscillations arising out of his work in the olfactory system
with Walter Freeman.  I believe that Echorn's group also did the work
without knowing about the theory.



>Since then numerous experiments have been conducted that
>confirm the theory (not without some controversy),

this is vastly too strong a statement.  There are major issues outstanding
about the basic idea and its evidence.  While it is true that Christof's
theory has generated a lot of interest and discussion, it remains to be
seen whether, in the long run, that discussion was useful in figuring out
the significance of cortical oscillations.  It may have been a distraction.



>But one would not dispute that his neural network theory has
>generated major impact on neuroscience.
>


This can not be disputed.  One can wish or not that it was otherwise.


Jim Bower

From hagai at phy.ucsf.EDU  Sun Sep 20 11:17:24 1998
From: hagai at phy.ucsf.EDU (Hagai Attias)
Date: Sun, 20 Sep 98 08:17:24 -0700
Subject: Paper available -- Independent Factor Analysis
Message-ID: <199809201517.IAA08579@phy.ucsf.EDU>


A new paper on a simple graphical model approach to the problem of 
blind separation of independent sources, using exact and variational EM, 
is available at

   http://keck.ucsf.edu/~hagai/papers.html


- ---------------------------------------------------


	  INDEPENDENT FACTOR ANALYSIS

	       Hagai Attias, UCSF

         (Neural Computation, in press)


We introduce the independent factor analysis (IFA) method for recovering 
independent hidden sources from their observed mixtures. IFA generalizes 
and unifies ordinary factor analysis (FA), principal component analysis (PCA), 
and independent component analysis (ICA), and can handle not only square
noiseless mixing, but also the general case where the number of mixtures
differs from the number of sources and the data are noisy. 

IFA is a two-step procedure. In the first step, the source densities, mixing 
matrix and noise covariance are estimated from the observed data by maximum 
likelihood. For this purpose we present an expectation-maximization (EM) 
algorithm, which performs unsupervised learning of an associated probabilistic 
model of the mixing situation. Each source in our model is described by a 
mixture of Gaussians, thus all the probabilistic calculations can be performed 
analytically. In the second step, the sources are reconstructed from the 
observed data by an optimal non-linear estimator. 

A variational approximation of this algorithm is derived for cases with a large 
number of sources, where the exact algorithm becomes intractable.

Our IFA algorithm reduces to the one for ordinary FA when the sources become 
Gaussian, and to an EM algorithm for PCA in the zero-noise limit. We derive an 
additional EM algorithm specifically for noiseless IFA. This algorithm is shown 
to be superior to ICA since it can learn arbitrary source densities from the 
data. Beyond blind separation, IFA can be used for modeling multi-dimensional 
data by a highly constrained mixture of Gaussians, and as a tool for non-linear 
signal encoding.

From tgd at iiia.csic.es  Mon Sep 21 12:01:51 1998
From: tgd at iiia.csic.es (Thomas Dietterich)
Date: Mon, 21 Sep 1998 18:01:51 +0200 (MET DST)
Subject: A Computing Research Repository
Message-ID: <199809211601.SAA25899@sinera.iiia.csic.es>

Annoucing A Computing Research Repository

Researchers have made their papers available by putting them on personal
web pages, departmental pages, and on various ad hoc sites known only
to cognoscenti.  Until now, there has not been a single repository to 
which researchers from the whole field of computing can submit reports.  

This is about to change.  Through a partnership of ACM, the Los Alamos 
e-Print archive, and NCSTRL (Networked Computer Science Technical
Reference Library), an online Computing Research Repository (CoRR) is 
being established.  The Repository has been integrated into the 
collection of over 20,000 computer science research reports and other 
material available through NCSTRL (http://www.ncstrl.org) and will be 
linked with the ACM Digital Library.  Most importantly, the Repository 
will be available to all members of the community at no charge.

We encourage you to start using the Repository right away.  For more details,
see http://xxx.lanl.gov/archive/cs/intro.html.  That site provides 
information on how to submit documents, browse, search, and subscribe to 
get notification of new articles of interest.  Please spread the word 
among your colleagues and students.  CoRR will only gain in value as
more researchers use it.  

See http://www.acm.org/repository for a more detailed description of CoRR.    

From aminai at ececs.uc.edu  Mon Sep 21 16:20:47 1998
From: aminai at ececs.uc.edu (Ali Minai)
Date: Mon, 21 Sep 1998 16:20:47 -0400 (EDT)
Subject: ICCS'98 Focus Session
Message-ID: <199809212020.QAA19332@holmes.ececs.uc.edu>


                               ANNOUNCEMENT
                               ------------
   

      Focus Session on Neural Computation, Cognition, and Complex Systems
      -------------------------------------------------------------------

          Second International Conference on Complex Systems (ICCS'98)
                               Nashua, NH
                           October 25-30, 1998
                 

A focus session on neural computation, cognition, and complex systems will
be held on Oct. 29, 7:00 - 10:00 p.m., as part of the Second International
Conference on Complex Systems. This invited session brings together a
group of researchers who are working at the active interface of complex
systems and neuroscience, and who have thought very deeply about these issues.
The session will be chaired by Prof. Walter Freeman, University of California,
Berkeley, who will give a keynote talk on the afternoon of Wednesday,
October 28. The speakers include:

Walter Freeman (University of California, Berkeley) - CHAIR
Steve Bressler (Center for Complex Systems, Florida Atlantic Univ.)
Michael Hasselmo (Boston University)
Jorge Jose (Northeastern University)
John Lisman (Volen Center for Complex Systems - Brandeis University)
Randy McIntosh (Rotman Research Institute, Toronto)
John Symons (Boston University)

The session will include presentations and a panel discussion. The
focus of discussion will be:


    How, and to what extent, can the study of spatio-temporal
         dynamics in the brain help explain cognition?


with the following specific issues:


1. What sort of dynamic processes and structures in the brain underlie
    the processes of cognition? At what scales do they occur, and are
    they subject to global principles of organization such as
    cooperation, competition, synchronization, etc. across scales?

2. What types of experiments do we need to probe these dynamic processes
    and construct useful neurobiologically grounded theories of cognitive
    function?

3. Is the theoretical/mathematical framework in which we model neural
    systems (e.g., compartmental models, local learning rules, patterns
    of activity, spike train statistics, etc.) sufficient to capture the
    processes of interest?

4. Are the emerging sciences of complexity, with ideas like self-organization,
    self-similarity, chaos, and scaling, likely to provide a useful paradigm
    for relating biology to cognition? How successful have attempts to
    apply these ideas been so far?

5. Is the information processing metaphor for the brain still a viable one,
    or should it be expanded/modified in some way?



The International Conference on Complex Systems is organized by the
New England Complex Systems Institute (NECSI), and is an important effort
to establish complex systems as a research area in its own right. The first
conference last year (also in Nashua) brought together several hundred people
and produced very animated discussions. In addition to the speakers at the
neural computation session, this year's conference speakers include Stephen
Kosslyn, Scott Kelso, Per Bak, Doyne Farmer, Phillip Anderson, Matt Wilson
and many others whose ideas speak to issues of interest to neural systems
researchers.

I will post more information on this as it becomes available. Interested
readers should check out the ICCS'98 website at http://necsi.org/html/iccs2.html
or send mail to iccs at necsi.org for information on registration, etc.


Ali A. Minai
Complex Adaptive Systems Laboratory
Department of Electrical
   & Computer Engineering
               and Computer Science
University of Cincinnati
Cincinnati, OH 45221-0030

Phone: (513) 556-4783
Fax:   (513) 556-7326
Email: Ali.Minai at uc.edu

Internet: http://www.ececs.uc.edu/~aminai/

From jagota at cse.ucsc.edu  Mon Sep 21 19:57:55 1998
From: jagota at cse.ucsc.edu (Arun Jagota)
Date: Mon, 21 Sep 1998 16:57:55 -0700
Subject: Call for volunteers: NIPS*98
Message-ID: <199809212357.QAA28989@arapaho.cse.ucsc.edu>


		NIPS*98 Call For Volunteers
		===========================

NIPS*98 needs student volunteers to assist onsite at the tutorials, 
main conference, and workshops. In exchange for approximately 9 hours 
of work a volunteer will receive free registration to the component 
of NIPS (tutorials, conference, or workshops) that (s)he volunteers 
time towards. Volunteers can look forward to this being an educational 
and fun experience!

To apply check out the procedure at

	http://www.cse.ucsc.edu/~jagota/

	
For questions (but first try the web site) contact me by e-mail,

Arun Jagota 			jagota at cse.ucsc.edu
Local arrangements, NIPS*98

From hochreit at informatik.tu-muenchen.de  Mon Sep 21 11:06:47 1998
From: hochreit at informatik.tu-muenchen.de (Josef Hochreiter)
Date: Mon, 21 Sep 1998 17:06:47 +0200
Subject: paper on ICA and LOCOCODE
Message-ID: <98Sep21.170648+0200_met_dst.7646-25738+67@papa.informatik.tu-muenchen.de>



                 Feature extraction through LOCOCODE                

   Sepp Hochreiter,  TUM                Juergen Schmidhuber, IDSIA       

   Neural Computation, in press   (28 pages, 0.7MB, 5MB gunzipped) 

LOw-COmplexity COding and DEcoding  (LOCOCODE)  is a novel approach to 
sensory coding and unsupervised learning.   Unlike previous methods it
explicitly takes into account the  information-theoretic complexity of 
the code generator:      it computes lococodes that convey information 
about the input data and can be computed and decoded by low-complexity 
mappings.  We implement LOCOCODE by training autoassociators with Flat
Minimum Search (Neural Computation 9(1):1-42, 1997),  a general method
for discovering low-complexity neural nets.     It turns out that this
approach can unmix an  unknown number of  independent  data sources by 
extracting a minimal number of  low-complexity features  necessary for
representing the data.   Experiments show:  unlike codes obtained with
standard autoencoders, lococodes are based on feature detectors, never
unstructured, usually sparse, sometimes factorial or local  (depending 
on statistical properties of the data).       Although LOCOCODE is not 
explicitly designed to enforce sparse or factorial codes,  it extracts 
optimal codes for  difficult  versions of the  bars benchmark problem, 
whereas ICA and PCA do not.        It produces familiar,  biologically 
plausible  feature  detectors  when applied to real world images,  and 
codes with fewer bits per pixel than ICA and PCA.   Unlike ICA it does 
not need to know the number of independent sources.  As a preprocessor 
for a  vowel  recognition  benchmark  problem it  sets  the  stage for 
excellent classification performance.            Our results reveil an 
interesting,  previously  ignored  connection  between  two  important 
fields: regularizer research, and ICA-related research.       They may 
represent  a  first  step  towards  unification  of regularization and 
unsupervised learning.

   ftp://ftp.idsia.ch/pub/juergen/lococode.ps.gz
   ftp://flop.informatik.tu-muenchen.de/pub/articles-etc/
   hochreiter.lococode.ps.gz

   http://www7.informatik.tu-muenchen.de/~hochreit/pub.html
   http://www.idsia.ch/~juergen/onlinepub.html

   Conference spin-offs:
1. Low-complexity coding and decoding. In K. M. Wong, I. King, D. 
   Yeung, eds., Proc. TANC'97, 297-306, Springer, 1997.
2. Unsupervised coding with LOCOCODE. In W. Gerstner, A. Germond, M. 
   Hasler, J.-D.  Nicoud, eds., ICANN'97, 655-660, Springer, 1997.
3. LOCOCODE versus PCA and ICA. In L. Niklasson, M. Boden, T. Ziemke,
   eds., ICANN'98, 669-674, Springer, 1998.
4. Source separation as a by-product of regularization.  
   To be presented at NIPS'98, 1998.

Sepp & Juergen




From sylee at eekaist.kaist.ac.kr  Tue Sep 22 21:13:29 1998
From: sylee at eekaist.kaist.ac.kr (sylee)
Date: Wed, 23 Sep 1998 10:13:29 +0900
Subject: Several Post Doc Positions in Korea
Message-ID: <199809230133.KAA08525@eekaist.kaist.ac.kr>

                   Immediate Opening:
           Post Doc Positions Available
     at Brain Science Research Center

Korean Ministry of Science and Technology just initiated a 10 year national
research program on Braintech'21, which consists of neuroscience,
cognitive science, and artificial neural networks. The Brain Science
Research Center (BSRC) is selected as the main research organization
for the Braintech'21. Although the BSRC is located at Korea Advanced
Institute of
Science and Technology (KAIST) at Taejon, Korea (South), it supports the
whole
Korean brain research community. More than 70 professors and researchers
from all over the Korea are affiliated to the BSRC.

Several Post Doc positions are available starting from November 1998 or
later.
The research areas cover all aspects of neuroscience, cognitive science,
and artificial neural networks. Currently we have 3 inter-disciplinary
projects and 6 basic-research projects.

o Interdicsiplinary Projects
   - artifical vision and speech recognition system
     (from biological models to hardware implementations, i.e.
      artificial retina and artificial cochlea chips)
   - inferrence system
     (from biology to neural network models)
   - EEG classification system
o Basic-research projects
   - Neuroscience: molecular level
   - Neuroscience: system level
   - Cognitive science
   - Artificial neural network models
   - Neuro-chip hardware impementation
   - Neural network applications (control and telecommunications)

Applicants should have a strong background in quantitative research and
some
basic knowledge of neuroscience, cognitive science, artificial neural
networks,
or VLSI design.  Interests in multidisciplinary researches are
advantageous.

In colllaobation with many academic and research organizations in Korea,
the BSRC will provide excellent research environments. Researchers from
diversified research backgrounds join together to stimulate each other
and come up with excellent multidisciplinary researches.

The starting date is flexible. The position is available for 1-3 years
depending on accomplishment.

Applicants should send a CV, the names and e-mail addresses of three
references, 
and a summary of research interests and experience to:

Prof. Soo-Young Lee
Director, Brain Science Research Center
3rd Floor, LG Semicon Hall
KAIST
373-1 Kusong-dong, Yusong-gu
Taejon 305-701
Korea (South)
Tel: +82-42-869-3431
Fax: +82-42-869-8570
E-mail: sylee at ee.kaist.ac.kr




From friess at acse.shef.ac.uk  Wed Sep 23 08:09:01 1998
From: friess at acse.shef.ac.uk (Thomas Friess)
Date: Wed, 23 Sep 1998 13:09:01 +0100 (BST)
Subject: new RR on support vector neural networks 
Message-ID: <Pine.SOL.3.94.980923125756.6731A-100000@puffin>



a new research report on support vector neural networks is
available at:
http://www.brunner-edv.com/friess/index.html


Support Vector Neural Networks: The Kernel Adatron with Bias 
and Soft Margin


Abstract: The kernel Adatron with bias and soft margin (KAb) is a new 
neural network alternative to support vector (SV) machines. It can learn
large-margin decision functions in kernel feature spaces in an iterative
"on-line" fashion which are identical to support vector machines.
Support vector learning is batch learning and is strongly based on solving
constrained quadratic programming problems which are nontrivial to
implement and may be subject to stability problems.
The kernel Adatron algorithm (KA), which has been developed as a joint
project, has been introduced recently. So far it has been assumed that the
bias parameter of the plane in feature space is always zero, and that all
patterns can be correctly classified by the learning machine. These
assumptions cannot always be made.
At first perceptrons in the data dependent representation, support vector
machines, and the kernel Adatron will be reviewd. Then the kernel Adatron
with bias and soft margin will be introduced.
The algorithm is conceptually simple and implements an iterative form of
unconstrained quadratic programming.
Experimental results using benchmarks and real data are provided which
allow to compare the performance and speed of kernel Adatrons and
SV machines.







From cns-cas at cns.bu.edu  Wed Sep 23 11:08:43 1998
From: cns-cas at cns.bu.edu (Boston University - Cognitive and Neural Systems)
Date: Wed, 23 Sep 1998 11:08:43 -0400
Subject: Graduate Training in CNS at Boston University
Message-ID: <199809231508.LAA20573@mattapan.bu.edu>

*******************************************************************
GRADUATE TRAINING IN THE
DEPARTMENT OF COGNITIVE AND NEURAL SYSTEMS (CNS)
AT BOSTON UNIVERSITY
*******************************************************************

The Boston University Department of Cognitive and Neural Systems
offers comprehensive graduate training in the neural and computational
principles, mechanisms, and architectures that underlie human and
animal behavior, and the application of neural network architectures
to the solution of technological problems.

Applications for Fall, 1999, admission and financial aid are now being
accepted for both the MA and PhD degree programs.

To obtain a brochure describing the CNS Program and a set of application
materials, write, telephone, or fax:

DEPARTMENT OF COGNITIVE AND NEURAL SYSTEMS
Boston University
677 Beacon Street
Boston, MA 02215

617/353-9481 (phone)
617/353-7755 (fax)

or send via e-mail your full name and mailing address to the attention
of Mr. Robin Amos at:

inquiries at cns.bu.edu

Applications for admission and financial aid should be received by the
Graduate School Admissions Office no later than January 15.  Late
applications will be considered until May 1; after that date
applications will be considered only as special cases.

Applicants are required to submit undergraduate (and, if applicable,
graduate) transcripts, three letters of recommendation, and Graduate
Record Examination (GRE) scores. The Advanced Test should be in the
candidate's area of departmental specialization. GRE scores may be
waived for MA candidates and, in exceptional cases, for PhD
candidates, but absence of these scores will decrease an applicant's
chances for admission and financial aid.

Non-degree students may also enroll in CNS courses on a part-time
basis.

Stephen Grossberg, Chairman
Gail A. Carpenter, Director of Graduate Studies

Description of the CNS Department:

The Department of Cognitive and Neural Systems (CNS) provides
advanced training and research experience for graduate students
interested in the neural and computational principles,
mechanisms, and architectures that underlie human and animal
behavior, and the application of neural network architectures to
the solution of outstanding technological problems. Students are
trained in a broad range of areas concerning cognitive and
neural systems, including vision and image processing; speech
and language understanding; adaptive pattern recognition;
cognitive information processing; self-organization; associative
learning and long-term memory; cooperative and competitive
network dynamics and short-term memory; reinforcement,
motivation, and attention; adaptive sensory-motor control and
robotics; and biological rhythms; as well as the mathematical
and computational methods needed to support modeling research
and applications. The CNS Department awards MA, PhD, and BA/MA
degrees.

The CNS Department embodies a number of unique features. It has
developed a curriculum that consists of interdisciplinary
graduate courses, each of which integrates the psychological,
neurobiological, mathematical, and computational information
needed to theoretically investigate fundamental issues
concerning mind and brain processes and the applications of
neural networks to technology. Additional advanced courses,
including research seminars, are also offered. Each course is
typically taught once a week in the afternoon or evening to make
the program available to qualified students, including working
professionals, throughout the Boston area. Students develop a
coherent area of expertise by designing a program that includes
courses in areas such as biology, computer science, engineering,
mathematics, and psychology, in addition to courses in the CNS
curriculum.

The CNS Department prepares students for thesis research with
scientists in one of several Boston University research centers
or groups, and with Boston-area scientists collaborating with
these centers. The unit most closely linked to the department is
the Center for Adaptive Systems.  Students interested in neural
network hardware work with researchers in CNS, at the College
of Engineering, and at MIT Lincoln Laboratory.  Other research
resources include distinguished research groups in neurophysiology,
neuroanatomy, and neuropharmacology at the Medical School and
the Charles River Campus; in sensory robotics, biomedical
engineering, computer and systems engineering, and neuromuscular
research within the College of Engineering; in dynamical systems
within the Mathematics Department; in theoretical computer science
within the Computer Science Department; and in biophysics and
computational physics within the Physics Department.

In addition to its basic research and training program, the
department conducts a seminar series, as well as conferences and
symposia, which bring together distinguished scientists from
both experimental and theoretical disciplines.

The department is housed in its own new four-story building
which includes ample space for faculty and student offices and
laboratories, as well as an auditorium, classroom and seminar
rooms, a library, and a faculty-student lounge.

Below are listed departmental faculty, courses and labs.


1998-99 CAS MEMBERS and CNS FACULTY:

Thomas J. Anastasio
Visiting Scholar, Department of Cognitive and Neural Systems (9/1/98-6/30/99)
Associate Professor, Molecular & Integrative Physiology,
Univ. of Illinois, Urbana/Champaign
PhD, McGill University
Computational modeling of vestibular, oculomotor, and other sensorimotor
systems.

Jelle Atema
Professor of Biology
Director, Boston University Marine Program (BUMP)
PhD, University of Michigan
Sensory physiology and behavior.

Aijaz Baloch
Adjunct Assistant Professor of Cognitive and Neural Systems
PhD, Electrical Engineering, Boston University
Neural modeling of role of visual attention in recognition, learning and
motor control, computational vision, adaptive control systems, reinforcement
learning.

Helen Barbas
Associate Professor, Department of Health Sciences
PhD, Physiology/Neurophysiology, McGill University
Organization of the prefrontal cortex, evolution of the neocortex.

Jacob Beck
Research Professor of Cognitive and Neural Systems
PhD, Psychology, Cornell University
Visual perception, psychophysics, computational models.

Daniel H. Bullock
Associate Professor of Cognitive and Neural Systems and Psychology
PhD, Psychology, Stanford University
Real-time neural systems, sensory-motor learning and control, evolution of
intelligence, cognitive development.

Gail A.Carpenter
Professor of Cognitive and Neural Systems and Mathematics
Director of Graduate Studies, Department of Cognitive and Neural Systems
PhD, Mathematics, University of Wisconsin, Madison
Pattern recognition, machine learning, differential equations, technology
transfer.

Gert Cauwenberghs
Visiting Scholar, Department of Cognitive and Neural Systems (6/1/98-8/31/99)
Associate Professor of Electrical And Computer Engineering,
The Johns Hopkins University
PhD, Electrical Engineering, California Institute of Technology
VLSI circuits, systems and algorithms for parallel analog signal processing
and adaptive neural computation.

Laird Cermak
Director, Memory Disorders Research Center, Boston Veterans Affairs Medical
Center
Professor of Neuropsychology, School of Medicine
Professor of Occupational Therapy, Sargent College
PhD, Ohio State University
Memory disorders.

Michael A. Cohen
Associate Professor of Cognitive and Neural Systems and Computer Science
PhD, Psychology, Harvard University
Speech and language processing, measurement theory, neural modeling,
dynamical systems.

H. Steven Colburn
Professor of Biomedical Engineering
PhD, Electrical Engineering, Massachusetts Institute of Technology
Audition, binaural interaction, signal processing models of hearing.

Howard Eichenbaum
Professor of Psychology
PhD, Psychology, University of Michigan
Neurophysiological studies of how the hippocampal system is involved in
reinforcement learning, spatial orientation, and declarative memory.

William D. Eldred III
Associate Professor of Biology
PhD, University of Colorado, Health Science Center
Visual neural biology.

Gil Engel
Research Fellow, Department of Cognitive and Neural Systems
Chief Engineer, Vision Applications, Inc.
Senior Design Engineer, Analog Devices, CTS Division
MS, Polytechnic University, New York
Space-variant active vision systems for use in human-computer interactive
control.

Bruce Fischl
Research Fellow, Department of Cognitive and Neural Systems
PhD, Cognitive and Neural Systems, Boston University
Anisotropic diffusion and nonlinear image filtering, space-variant vision,
computational models of early visual processing, and automated analysis of
magnetic resonance images.

Paolo Gaudiano
Associate Professor of Cognitive and Neural Systems
PhD, Cognitive and Neural Systems, Boston University
Computational and neural models of robotics, vision, adaptive sensory-motor
control, and behavioral neurobiology.

Jean Berko Gleason
Professor of Psychology
PhD, Harvard University
Psycholinguistics.

Sucharita Gopal
Associate Professor of Geography
PhD, University of California at Santa Barbara
Neural networks, computational modeling of behavior, geographical information
systems, fuzzy sets, and spatial cognition.

Stephen Grossberg
Wang Professor of Cognitive and Neural Systems
Professor of Mathematics, Psychology, and Biomedical Engineering
Chairman, Department of Cognitive and Neural Systems
Director, Center for Adaptive Systems
PhD, Mathematics, Rockefeller University
Theoretical biology, theoretical psychology, dynamical systems, and applied
mathematics.

Frank Guenther
Associate Professor of Cognitive and Neural Systems
PhD, Cognitive and Neural Systems, Boston University
Biological sensory-motor control, spatial representation, and speech production.

Catherine L. Harris
Assistant Professor of Psychology
PhD, Cognitive Science and Psychology, University of California at San Diego
Visual word recognition, psycholinguistics, cognitive semantics, second
language acquisition, computational models.

Thomas G. Kincaid
Professor of Electrical, Computer and Systems Engineering,
College of Engineering
PhD, Electrical Engineering, Massachusetts Institute of Technology
Signal and image processing, neural networks, non-destructive testing.

Mark Kon
Professor of Mathematics
PhD, Massachusetts Institute of Technology
Functional analysis, mathematical physics, partial differential equations.

Nancy Kopell
Professor of Mathematics
PhD, Mathematics, University of California at Berkeley
Dynamical systems, mathematical physiology, pattern formation in
biological/physical systems.

Gregory Lesher
Research Fellow, Department of Cognitive and Neural Systems
PhD, Cognitive and Neural Systems, Boston University
Modeling of visual processes, visual perception, statistical
language modeling, and augmentative communication.

Jacqueline A. Liederman
Associate Professor of Psychology
PhD, Psychology, University of Rochester
Dynamics of interhemispheric cooperation; prenatal correlates of
neurodevelopmental disorders.

Ennio Mingolla
Associate Professor of Cognitive and Neural Systems and Psychology
PhD, Psychology, University of Connecticut
Visual perception, mathematical modeling of visual processes.

Joseph Perkell
Adjunct Professor of Cognitive and Neural Systems
Senior Research Scientist, Research Lab of Electronics
and Department of Brain and Cognitive Sciences,
Massachusetts Institute of Technology
PhD, Massachusetts Institute of Technology
Motor control of speech production.

Alan Peters
Chairman and Professor of Anatomy and Neurobiology,
School of Medicine
PhD, Zoology, Bristol University, United Kingdom
Organization of neurons in the cerebral cortex, effects of aging on
the primate brain, fine structure of the nervous system.

Andrzej Przybyszewski
Research Fellow, Department of Cognitive and Neural Systems
PhD, Warsaw Medical Academy
Retinal physiology, mathematical and computer modeling of dynamical
properties of neurons in the visual system.

Adam Reeves
Adjunct Professor of Cognitive and Neural Systems
Professor of Psychology, Northeastern University
PhD, Psychology, City University of New York
Psychophysics, cognitive psychology, vision.

Mark Reinitz
Assistant Professor of Psychology
PhD, University of Washington
Cognitive psychology, attention, explicit and implicit memory,
memory-perception interactions.

Mark Rubin
Research Assistant Professor of Cognitive and Neural Systems
Research Physicist, Naval Air Warfare Center, China Lake, CA (on leave)
PhD, Physics, University of Chicago
Neural networks for vision, pattern recognition, and motor control.

Elliot Saltzman
Associate Professor of Physical Therapy, Sargent College
Assistant Professor, Department of Psychology
and Center for the Ecological Study of Perception and Action
University of Connecticut, Storrs
Research Scientist, Haskins Laboratories, New Haven, CT
PhD, Developmental Psychology, University of Minnesota
Modeling and experimental studies of human speech production.

Robert Savoy
Adjunct Associate Professor of Cognitive and Neural Systems
Scientist, Rowland Institute for Science
PhD, Experimental Psychology, Harvard University
Computational neuroscience; visual psychophysics of color, form, and motion
perception.

Eric Schwartz
Professor of Cognitive and Neural Systems; Electrical, Computer
and Systems Engineering; and Anatomy and Neurobiology
PhD, High Energy Physics, Columbia University
Computational neuroscience, machine vision, neuroanatomy, neural modeling.

Robert Sekuler
Adjunct Professor of Cognitive and Neural Systems
Research Professor of Biomedical Engineering, College of Engineering,
BioMolecular Engineering Research Center
Jesse and Louis Salvage Professor of Psychology, Brandeis University
PhD, Psychology, Brown University
Visual motion, visual adaptation, relation of visual perception, memory,
and movement.

Barbara Shinn-Cunningham
Assistant Professor of Cognitive and Neural Systems
and Biomedical Engineering
PhD, Electrical Engineering and Computer Science,
Massachusetts Institute of Technology
Psychoacoustics, audition, auditory localization, binaural hearing,
sensorimotor adaptation, mathematical models of human performance.

Malvin Teich
Professor of Electrical and Computer Systems Engineering
and Biomedical Engineering
PhD, Cornell University
Quantum optics, photonics, fractal stochastic processes, information
transmission in biological sensory systems.

Lucia Vaina
Professor of Biomedical Engineering
Research Professor of Neurology, School of Medicine
PhD, Sorbonne (France); Dres Science, National Politechnique Institute,
Toulouse (France)
Computational visual neuroscience, biological and computational learning,
functional and structural neuroimaging.

Takeo Watanabe
Assistant Professor of Psychology
PhD, Behavioral Sciences, University of Tokyo
Perception of objects and motion and effects of attention on perception
using psychophysics and brain imaging (fMRI).

Allen Waxman
Adjunct Associate Professor of Cognitive and Neural Systems
Senior Staff Scientist, MIT Lincoln Laboratory
PhD, Astrophysics, University of Chicago
Visual system modeling, mobile robotic systems, parallel computing,
optoelectronic hybrid architectures.

James Williamson
Research Assistant Professor of Cognitive and Neural Systems
PhD, Cognitive and Neural Systems, Boston University
Image processing and object recognition.  Particular interests: dynamic
binding, self-organization, shape representation, and classification.

Jeremy Wolfe
Adjunct Associate Professor of Cognitive and Neural Systems
Associate Professor of Ophthalmology, Harvard Medical School
Psychophysicist, Brigham & Women's Hospital, Surgery Dept.
Director of Psychophysical Studies, Center for Clinical Cataract Research
PhD, Massachusetts Institute of Technology
Visual attention, preattentive and attentive object representation.

Curtis Woodcock
Associate Professor of Geography; Chairman, Department of Geography
Director, Geographic Applications, Center for Remote Sensing
PhD, University of California, Santa Barbara
Biophysical remote sensing, particularly of forests and natural vegetation,
canopy reflectance models and their inversion, spatial modeling, and change
detection; biogeography; spatial analysis; geographic information systems;
digital image processing.


CNS DEPARTMENT COURSE OFFERINGS

CAS CN500  Computational Methods in Cognitive and Neural Systems
CAS CN510  Principles and Methods of Cognitive and Neural Modeling I
CAS CN520  Principles and Methods of Cognitive and Neural Modeling II
CAS CN530  Neural and Computational Models of Vision
CAS CN540  Neural and Computational Models of Adaptive Movement Planning
                        and Control
CAS CN550  Neural and Computational Models of Recognition, Memory and
Attention
CAS CN560  Neural and Computational Models of Speech Perception and Production
CAS CN570  Neural and Computational Models of Conditioning, Reinforcement,
           Motivation and Rhythm
CAS CN580  Introduction to Computational Neuroscience
GRS CN700  Computational and Mathematical Methods in Neural Modeling
GRS CN710  Advanced Topics in Neural Modeling
GRS CN720  Neural and Computational Models of Planning and Temporal Structure
           in Behavior
GRS CN730  Models of Visual Perception
GRS CN740  Topics in Sensory-Motor Control
GRS CN760  Topics in Speech Perception and Recognition
GRS CN780  Topics in Computational Neuroscience
GRS CN810  Topics in Cognitive and Neural Systems: Visual Event Perception
GRS CN811  Topics in Cognitive and Neural Systems: Visual Perception

GRS CN911,912
Research in Neural Networks for Adaptive Pattern Recognition

GRS CN915,916
Research in Neural Networks for Vision and Image Processing

GRS CN921,922
Research in Neural Networks for Speech and Language Processing

GRS CN925,926
Research in Neural Networks for Adaptive Sensory-Motor Planning
and Control

GRS CN931,932
Research in Neural Networks for Conditioning and Reinforcement Learning

GRS CN935,936
Research in Neural Networks for Cognitive Information Processing

GRS CN941,942
Research in Nonlinear Dynamics of Neural Networks

GRS CN945,946
Research in Technological Applications of Neural Networks

GRS CN951,952
Research in Hardware Implementations of Neural Networks

CNS students also take a wide variety of courses in related departments.
In addition, students participate in a weekly colloquium series, an informal
lecture series, and a student-run Journal Club, and attend lectures and
meetings
throughout the Boston area; and advanced students work in small research
groups.


LABORATORY AND COMPUTER FACILITIES

The department is funded by grants and contracts from federal
agencies that support research in life sciences, mathematics,
artificial intelligence, and engineering. Facilities include
laboratories for experimental research and computational
modeling in visual perception, speech and language processing,
and sensory-motor control and robotics. Data analysis and
numerical simulations are carried out on a state-of-the-art
computer network comprised of Sun workstations, Silicon Graphics
workstations, Macintoshes, and PCs.  All students have access to
X-terminals or UNIX workstation consoles, a selection of color
systems and PCs, a network of SGI machines, and standard modeling
and mathematical simulation packages such as Mathematica, VisSim,
Khoros, and Matlab.

The department maintains a core collection of books and
journals, and has access both to the Boston University libraries
and to the many other collections of the Boston Library
Consortium.

In addition, several specialized facilities and software are
available for use.  These include:

Computer Vision/Computational Neuroscience Laboratory

The Computer Vision/Computational Neuroscience Lab is comprised
of an electronics workshop, including a surface-mount
workstation, PCD fabrication tools, and an Alterra EPLD design
system; a light machine shop; an active vision lab including
actuators and video hardware; and systems for computer aided
neuroanatomy and application of computer graphics and image
processing to brain sections and MRI images.

Neurobotics Laboratory

The Neurobotics Lab utilizes wheeled mobile robots to study
potential applications of neural networks in several areas,
including adaptive dynamics and kinematics, obstacle avoidance,
path planning and navigation, visual object recognition, and
conditioning and motivation. The lab currently has three Pioneer
robots equipped with sonar and visual sensors; one B-14 robot
with a moveable camera, sonars, infrared, and bump sensors; and
two Khepera miniature robots with infrared proximity detectors.
Other platforms may be investigated in the future.

Psychoacoustics Laboratory

The Psychoacoustics Lab houses a newly installed, 8 ft. x 8 ft.
sound-proof booth.  The laboratory is  extensively equipped to
perform both traditional psychoacoustic experiments and
experiments using interactive auditory virtual-reality stimuli.
The major equipment dedicated to the psychoacoustics laboratory
includes two Pentium-based personal computers; two
Power-PC-based Macintosh computers; a 50-MHz array processor
capable of generating auditory stimuli in real time;
programmable attenuators; analog-to-digital and
digital-to-analog converters; a real-time head tracking system;
a special-purpose, signal-processing hardware system capable of
generating "spatialized" stereo auditory signals in real time; a
two-channel oscilloscope; a two-channel spectrum analyzer;
various cables, headphones, and other miscellaneous electronics
equipment; and software for signal generation, experimental
control, data analysis, and word processing.

Sensory-Motor Control Laboratory

The Sensory-Motor Control Lab supports experimental studies of
motor kinematics. An infrared WatSmart system allows measurement
of large-scale movements, and a pressure-sensitive graphics
tablet allows studies of handwriting and other fine-scale
movements.  Equipment includes a 40-inch monitor that allows
computer display of animations generated by an SGI workstation
or a Pentium Pro (Windows NT) workstation.  A second major
component is a helmet-mounted, video-based, eye-head tracking
system (ISCAN Corp, 1997).  The latter's camera samples eye
position at 240Hz and also allows reconstruction of what
subjects are attending to as they freely scan a scene under
normal lighting.  Thus the system affords a wide range of
visuo-motor studies.

Speech and Language Laboratory

The Speech and Language Lab includes facilities for
analog-to-digital and digital-to-analog software conversion.
Ariel equipment allows reliable synthesis and playback of
speech waveforms.  An Entropic signal processing package
provides facilities for detailed analysis, filtering, spectral
construction, and formant tracking of the speech waveform.
Various large databases, such as TIMIT and TIdigits, are
available for testing algorithms of speech recognition.  For
high speed processing, supercomputer facilities speed
filtering and data analysis.

Visual Psychophysics Laboratory

The Visual Psychophysics Lab occupies an 800-square-foot suite,
including three dedicated rooms for data collection, and houses
a variety of computer controlled display platforms, including
Silicon Graphics, Inc. (SGI) Onyx RE2, SGI Indigo2 High Impact,
SGI Indigo2 Extreme, Power Computing (Macintosh compatible)
PowerTower Pro 225, and Macintosh 7100/66 workstations.
Ancillary resources for visual psychophysics include a
computer-controlled video camera, stereo viewing glasses,
prisms, a photometer, and a variety of display-generation,
data-collection,  and data-analysis software.

Affiliated Laboratories

Affiliated CAS/CNS faculty have additional laboratories ranging
from visual and auditory psychophysics and neurophysiology,
anatomy, and neuropsychology to engineering and chip design.
These facilities are used in the context of faculty/student
collaborations.

*******************************************************************
DEPARTMENT OF COGNITIVE AND NEURAL SYSTEMS
GRADUATE TRAINING ANNOUNCEMENT

Boston University
677 Beacon Street
Boston, MA 02215

Phone: 617/353-9481
Fax:   617/353-7755
Email: inquiries at cns.bu.edu
Web: http://cns-web.bu.edu/
*******************************************************************

From esann at dice.ucl.ac.be  Tue Sep 22 12:57:19 1998
From: esann at dice.ucl.ac.be (ESANN)
Date: Tue, 22 Sep 1998 18:57:19 +0200
Subject: CFP: ESANN'99 European Symposium on Artificial Neural Networks
Message-ID: <3.0.3.32.19980922185719.006a7adc@ns1.dice.ucl.ac.be>

----------------------------------------------------
|                                                  |
|                     ESANN'99                     |
|                                                  |
|              7th European Symposium              |
|           on Artificial Neural Networks          |
|                                                  |
|      Bruges (Belgium) - April 21-22-23, 1999     |
|                                                  |
|      First announcement and call for papers      |
----------------------------------------------------

The call for papers for the ESANN 99 conference is now available on the Web:
       http://www.dice.ucl.ac.be/esann
For those of you who maintain WWW pages including lists of related ANN
sites: we would appreciate if you could add the above URL to your list;
thank you very much!

We try as much as possible to avoid multiple sendings of this call for
papers; however please apologize if you receive this e-mail twice, despite
our precautions.

You will find below a short version of this call for papers, without the
instructions to authors (available on the Web).  If you have difficulties
to connect to the Web please send an e-mail to
        esann at dice.ucl.ac.be
and we will send you a full version of the call for papers.

ESANN'99 is organised in collaboration with the UCL (Universite catholique
de Louvain, Louvain-la-Neuve) and the KULeuven (Katholiek Universiteit
Leuven), and is technically co-sponsored by the IEEE Neural Networks
Council, the IEEE Region 8, the IEEE Benelux section, and the INNS
(International Neural Networks Society).


Scope and topics
----------------

The aim of the ESANN series of conference is to provide an annual European
forum for the presentation and discussion of recent advances in artificial
neural networks.  ESANN focuses on fundamental aspects of ANNs: theory,
models, learning algorithms, mathematical aspects, approximation of
functions, classification, control, time-series prediction, statistics,
signal processing, vision, self-organization, vector quantization,
evolutive learning, psychological computations, biological plausibility,
etc.  Papers on links and comparisons between ANNs and other domains of
research (such as statistics, data analysis, signal processing, biology,
psychology, evolutive learning, bio-inspired systems, etc.) are also
encouraged.

Papers will be presented orally (no parallel sessions) and in poster
sessions; all posters will be complemented by a short oral presentation
during a plenary session. It is important to mention that it is the topics
of the paper which will decide if it better fits into an oral or a poster
session, not its quality. The quality of posters will be the same as the
quality of oral presentations, and both will be printed in the same way in
the proceedings. Nevertheless, authors have the choice to indicate on the
author submission form that they only accept to present their paper orally.

The following is a non-exhaustive list of topics which will be covered
during ESANN'99:
o theory
o models and architectures
o mathematics
o learning algorithms
o vector quantization
o self-organization
o RBF networks
o Bayesian classification
o recurrent networks
o approximation of functions
o time series forecasting
o adaptive control
o statistical data analysis
o independent component analysis
o signal processing
o natural and artificial vision
o cellular neural networks
o fuzzy neural networks
o hybrid networks
o identification of non-linear dynamic systems
o biologically plausible artificial networks
o bio-inspired systems
o formal models of biological phenomena
o neurobiological systems
o cognitive psychology
o adaptive behavior
o evolutive learning

Special sessions
----------------

Special sessions will be organised by renowned scientists in their
respective fields. Papers submitted to these sessions are reviewed
according to the same rules as any other submission. Authors who submit
papers to one of these sessions are invited to mention it on the author
submission form; nevertheless, submissions to the special sessions must
follow the same format, instructions and deadlines as any other submission,
and must be sent to the same address.

The special sessions organized during ESANN'99 are:
o      Information extraction using unsupervised neural networks
       Colin Fyfe, Univ. of Paisley (UK).
o      Spiking neurons
       Wulfram Gerstner, E.P.F. Lausanne (Switzerland).
o      Adaptive computation of structured information
       Marco Gori, Univ. di Siena (Italy)
o      Remote sensing spectral image analysis
       Erzsebet Merenyi, Univ. of Arizona (USA)
o      Large-scale recognition of sequential patterns
       Yves Moreau, K.U. Leuven (Belgium)
o      Support Vector Machines
       S. Canu, INSIA Rouen (France)
       Bernhard Schoelkopf, GMD FIRST Berlin (Germany)

Location
--------

The conference will be held in Bruges (also called "Venice of the North"),
one of the most beautiful medieval towns in Europe.  Bruges can be reached
by train from Brussels in less than one hour (frequent trains).  The town
of Bruges is world-wide known, and famous for its architectural style, its
canals, and its pleasant atmosphere.

The conference will be organised in an hotel located near the centre
(walking distance) of the town. There is no obligation for the participants
to stay in this hotel. Hotels of all level of comfort and price are
available in Bruges; there is a possibility to book a room in the hotel of
the conference, or in another one (50 m. from the first one) at a
preferential rate through the conference secretariat. A list of other
smaller hotels is also available.

The conference will be held at the Novotel hotel, Katelijnestraat 65B, 8000
Brugge, Belgium.

Call for contributions
----------------------

Prospective authors are invited to submit
- six original copies of their manuscript (including at least two originals
or very good copies without glued material, which will be used for the
proceedings)
- one signed copy of the author submission form
before December 7, 1998.  While this is not mandatory, authors are
encouraged to join a floppy disk or CD with their contribution in (generic)
PostScript or (preferred) PDF format. Sorry, electronic or fax submissions
are not accepted.

Working language of the conference (including proceedings) is English.  The
instructions to authors, together with the author submission form, are
available on the ESANN Web server:
       http://www.dice.ucl.ac.be/esann
A printed version of these documents is also available through the
conference secretariat (please use email if possible).  Authors are invited
to follow the instructions to authors.  A LaTeX style file is also
available on the Web. 

Authors must indicate their choice for oral or poster presentation on the
author submission form.  They must also sign a written agreement that they
will register to the conference and present the paper in case of
acceptation of their submission.  Authors of accepted papers will have to
register before February 28, 1999.  They will benefit from the advance
registration fee.

Submissions must be sent to:
       Michel Verleysen
       UCL - DICE
       3, place du Levant
       B-1348 Louvain-la-Neuve
       Belgium
       esann at dice.ucl.ac.be
All submissions will be acknowledged by fax or email before December 15, 1998.

Deadlines
---------
Submission of papers        December 7, 1998
Notification of acceptance  January 31, 1999
Symposium                   April 21-22-23, 1999

Registration fees
-----------------
              registration before         registration after
              February 28, 1999           February 28, 1999
Universities  BEF 15500                   BEF 16500
Industries    BEF 19500                   BEF 20500
The registration fee include the attendance to all sessions, the lunches
during the three days of the conference, the coffee breaks twice a day, the
conference dinner, and the proceedings.

Conference secretariat
----------------------
       Michel Verleysen		
       D facto conference services       phone: + 32 2 420 37 57
       27 rue du Laekenveld              Fax: + 32 2 420 02 55
       B - 1080 Brussels (Belgium)       E-mail: esann at dice.ucl.ac.be
                     http://www.dice.ucl.ac.be/esann

Steering and local committee
----------------------------

Fran?ois Blayo       Univ. Paris I (F)
Marie Cottrell       Univ. Paris I (F)
Jeanny Herault       INPG Grenoble (F)
Henri Leich          Fac. Polytech. Mons (B)
Bernard Manderick    Vrije Univ. Brussel (B)
Eric Noldus          Univ. Gent (B)
Jean-Pierre Peters   FUNDP Namur (B)
Joos Vandewalle      KUL Leuven (B)
Michel Verleysen     UCL Louvain-la-Neuve (B)

Scientific committee (to be confirmed)
--------------------

Edoardo Amaldi       Cornell Univ. (USA)
Agnes Babloyantz     Univ. Libre Bruxelles (B)
Herve Bourlard       IDIAP Martigny (CH)
Joan Cabestany       Univ. Polit. de Catalunya (E)
Holk Cruse           Universitat Bielefeld (D)
Eric de Bodt         Univ. Lille II (F) & UCL Louvain-la-Neuve (B)
Dante Del Corso      Politecnico di Torino (I)
Wlodek Duch          Nicholas Copernicus Univ. (PL)
Marc Duranton        Philips / LEP (F)
Jean-Claude Fort     Universite Nancy I (F)
Bernd Fritzke        Ruhr-Universitat Bochum (D)
Stan Gielen          Univ. of Nijmegen (NL)
Manuel Grana         UPV San Sebastian (E)
Anne Guerin-Dugue    INPG Grenoble (F)
Martin Hasler        EPFL Lausanne (CH)
Laurent Herault      CEA Grenoble (F)
Christian Jutten     INPG Grenoble (F)
Juha Karhunen        Helsinky Uniersity of Technology (FIN)
Vera Kurkova         Acad. of Science of the Czech Rep. (CZ)
Petr Lansky          Acad. of Science of the Czech Rep. (CZ)
Mia Loccufier        Univ. Gent (B)
Hans-Peter Mallot    Max-Planck Institut (D)
Eddy Mayoraz         IDIAP Martigny (CH)
Jean Arcady Meyer    Univ. Pierre & Marie Curie (F)
Jose Mira Mira       UNED (E)
Jean-Pierre Nadal    Ecole Normale Superieure Paris (F)
Gilles Pages         Universite Paris VI (F)
Thomas Parisini      University of Trieste (I)
Helene Paugam-Moisy  Univ. Lumiere Lyon 2 (F)
Alberto Prieto       Universitad de Granada (E)
Leonardo Reyneri     Politecnico di Torino (I)
Tamas Roska          Hungarian Academy of Science (H)
Jean-Pierre Rospars  INRA Versailles (F)
John Stonham         Brunel University (UK)
Johan Suykens        KUL Leuven (B)
John Taylor          King's College London (UK)
Claude Touzet        CESAR ONRL Oak Ridge (USA)
Marc Van Hulle       KUL Leuven (B)
Christian Wellekens  Eurecom Sophia-Antipolis (F)

==========================ESANN - European Symposium on Artificial Neural Networks
http://www.dice.ucl.ac.be/esann

* For submissions of papers, reviews,...
Michel Verleysen
Univ. Cath. de Louvain - Microelectronics Laboratory
3, pl. du Levant - B-1348 Louvain-la-Neuve - Belgium
tel: +32 10 47 25 51 - fax: + 32 10 47 25 98
mailto:esann at dice.ucl.ac.be

* Conference secretariat
D facto conference services
45 rue Masui - B-1000 Brussels - Belgium
tel: + 32 2 203 43 63 - fax: + 32 2 203 42 94
mailto:esann at dice.ucl.ac.be
==========================

From wolfskil at MIT.EDU  Thu Sep 24 01:55:27 1998
From: wolfskil at MIT.EDU (Jud Wolfskill)
Date: Thu, 24 Sep 1998 01:55:27 -0400
Subject: book announcement:  Brendan Frey, Graphical Models for Machine Learning and Digital Communication
Message-ID: <mailman.960.1149591461.29955.connectionists@cs.cmu.edu>

The following is a book which readers of this list might find of
interest.  For more information please visit
http://mitpress.mit.edu/promotions/books/FREGHF98


Graphical Models for Machine Learning and Digital Communication

Brendan J. Frey


A variety of problems in machine learning and digital communication
deal with complex but structured natural or artificial systems. In this
book, Brendan Frey uses graphical models as an overarching framework to
describe and solve problems of pattern classification, unsupervised
learning, data compression, and channel coding. Using probabilistic
structures such as Bayesian belief networks and Markov random fields,
he is able to describe the relationships between random variables in
these systems and to apply graph-based inference techniques to develop
new algorithms. Among the algorithms described are the wake-sleep
algorithm for unsupervised learning, the iterative turbodecoding
algorithm (currently the best error-correcting decoding algorithm), the
bits-back coding method, the Markov chain Monte Carlo technique, and
variational inference.


Brendan J. Frey is a Beckman Fellow, Beckman Institute for Advanced
Science and Technology, University of Illinois at Urbana-Champaign.


Adaptive Computation and Machine Learning series

A Bradford Book


August 1998

6 x 9, 216 pp., 65 illus. 

cloth 0-262-06202-X


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

     |       Jud Wolfskill                       
 |||||||     Associate Publicist                 Phone:  (617) 253-2079
 |||||||     MIT Press                           Fax:  (617) 253-1709
 |||||||     Five Cambridge Center               E-mail: wolfskil at mit.edu 
      |      Cambridge, MA  02142-1493           http://mitpress.mit.edu                

From barba at cvs.rochester.edu  Thu Sep 24 10:24:19 1998
From: barba at cvs.rochester.edu (Barbara Arnold)
Date: Thu, 24 Sep 1998 10:24:19 -0400
Subject: Faculty position University of Rochester
Message-ID: <l0310280db230030ebc6c@[128.151.80.12]>

Please post this job opening.


Two Assistant Professors in Visual Science.  The University of Rochester
has available two tenure-track positions for scientists working in the
broad domain of visual science, including psychophysical, physiological,
and computational approaches.  Especially encouraged to apply are
candidates whose research is multi-disciplinary. The positions will be in
the Department of Brain and Cognitive Sciences
(http://www.bcs.rochester.edu), one of six departments participating in the
Center for Visual Science (http://www.cvs.rochester.edu), an
interdisciplinary community of 27 faculty engaged in vision research.
Junior appointments are preferred, although appointments at a more senior
level may be considered.  Applicants should submit a curriculum vitae, a
brief statement of research and teaching interests, reprints and three
reference letters to:  David R. Williams, Director, Center for Visual
Science, University of Rochester, Rochester, NY 14627-0270.  Review of
applications will begin December 1, 1998.  Desired start date is September
99 to September 00.  The University of Rochester is an affirmative
action/equal opportunity employer.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Barbara N. Arnold
Administrator                            email: barba at cvs.rochester.edu
Center for Visual Science                phone: 716 275 8659
University of Rochester                    fax: 716 271 3043
Meliora Hall 274
Rochester NY 14627-0270
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 



From granger at uci.edu  Thu Sep 24 20:06:08 1998
From: granger at uci.edu (Richard Granger)
Date: Thu, 24 Sep 1998 17:06:08 -0700
Subject: The hierarchical hypothesis   (Re-send of lost final message; thanks, Dave.)
Message-ID: <v02140b01b2299b98634f@[128.200.120.16]>


We were long puzzled by our biological models' persistent tendency to
produce different cortical outputs over successive theta cycles, and
eventually recognized that what these outputs might be encoding was
sequential hierarchical information (the hypothesis whose formal statement
ultimately appears in the 1990 Science paper).  The hypothesis was novel
and exciting to us -- but perhaps the finding was already obvious to
everyone except us (and the reviewers and editors at Science).  We've
received many private messages indicating that most people in the field do
recognize the history as we've described it, and that it is easy to
misconstrue in hindsight (the continuing special investigation into our
lack of foresight notwithstanding).  Our thanks to the many writers of
those messages!   (One friend reminded us that Postmodernism has clearly
shown that authors know far less than their Text knows, and accordingly
admonished us to "shut up, sit down, and listen to the story of your life
as it Really Happened."  :-)

On the other hand, the discussion is shot through with useful threads
twining around understanding of the distinctions and relationships among
experimental findings, construction of simulations, observation of
simulations, and formal characterization and simplification of results,
much as in physics.  As we come to better understand the differential
nature of experiments, versus simulations, versus formal characterization,
our ability to talk constructively across the boundaries of computation and
biology correspondingly improves.

Finally, it's worth noting that this hypothesis (that cortical neurons
differentially respond over sequential cycles, yielding successive
hierarchical information) is readily differentiated from other hypotheses.
Perhaps, then, comfort can be taken from the realization that physiological
and behavioral evidence may one day demonstrate that some competing
hypothesis is correct after all.


-Rick Granger
 granger at uci.edu


[A number of writers have requested references to subsequent publications
of ours, so a partial list is appended.   (Those of you who requested
references to our research on ampakines, I'll send that list in a separate
message.)  ]

Selected topical bibliography since '90:

Ambros-Ingerson, J., Granger, R., and Lynch, G. (1990).  Simulation of
paleocortex performs hierarchical clustering.   Science, 247: 1344-1348.

Granger, R., Staubli, U., Powers, H., Otto, T., Ambros-Ingerson, J., and
Lynch, G. (1991).  Behavioral tests of a prediction from a cortical network
simulation.  Psychol. Sci., 2: 116-118.

McCollum, J., Larson, J., Otto, T., Schottler, F., Granger, R., and Lynch,
G. (1991).  Short-latency single-unit processing in olfactory cortex.  J.
Cog. Neurosci., 3: 293-299.

Anton, P., Lynch, G., and Granger, R. (1991).  Computation of
frequency-to-spatial transform by olfactory bulb glomeruli.  Biol. Cybern.,
65: 407-414.

Granger, R., and Lynch, G. (1991).  Higher olfactory processes: Perceptual
learning and memory.  Current Opin. Neurosci., 1: 209-214.

Coultrip, R., Granger, R., and Lynch, G. (1992).  A cortical model of
winner-take-all competition via lateral inhibition.  Neural Networks, 5:
47-54.

Lynch, G. and Granger, R. (1992).  Variations in synaptic plasticity and
types of memory in  cortico-hippocampal networks.  J. Cog. Neurosci., 4:
189-199.

Granger, R. and Lynch, G. (1993).  Cognitive modularity: Computational
division of labor in the brain.  In: The Handbook of Neuropsychology, New
York: Academic Press.

Gluck, M. and Granger, R. (1993).  Computational models of the neural bases
of learning and memory. Annual Review of Neurosci. 16: 667-706.

Anton, P., Granger, R., and Lynch, G. (1993).  Simulated dendritic spines
influence reciprocal synaptic strengths and lateral inhibition in the
olfactory bulb.  Brain Res., 628: 157-165.

Coultrip, R. and Granger, R. (1994).  LTP learning rules in sparse networks
approximate Bayes classifiers via Parzen's method.  Neural Networks, 7:
463-476.

Kowtha, V., Satyanarayana, P., Granger, R., and Stenger, D. (1994).
Learning and classification in a noisy environment by a simulated cortical
network.  Proceedings of the Third Annual Computation and Neural Systems
Conference,  Boston: Kluwer, pp. 245-250.

Granger, R., Whitson, J., Larson, J. and Lynch, G. (1994). Non-Hebbian
properties of LTP enable high-capacity encoding of temporal sequences.
Proc. Nat'l. Acad. Sci., 91: 10104-10108.

Myers, C., Gluck, M., and Granger, R. (1995).  Dissociation of hippocampal
and entorhinal function in associative learning: A computational approach.
Psychobiology, 23: 116-138.

Ozeki, T., Shouval, H., Intrator, N. and Granger, R. (1995).  Analysis of a
temporal sequence learning network based on the property of LTP induction.
In: Int'l Symposium on Nonlinear Theory, Las Vegas, 1995.

Kilborn, K., Granger, R., and Lynch, G. (1996).  Effects of LTP on response
selectivity of simulated cortical neurons.  J. Cog. Neurosci., 8: 338-353.

Granger, R., Wiebe, S., Taketani, M., Ambros-Ingerson, J., Lynch, G.
(1997).  Distinct memory circuits comprising the hippocampal region.
Hippocampus, 6: 567-578.

Hess, U.S., Granger, R., Lynch, G., Gall, C.M.  (1997).  Differential
patterns of c-fos mRNA expression in amygdala during sequential stages of
odor discrimination learning.  Learning and Memory, 4: 262-283.





From suem at soc.plym.ac.uk  Fri Sep 25 04:20:28 1998
From: suem at soc.plym.ac.uk (Sue McCabe)
Date: Fri, 25 Sep 1998 09:20:28 +0100
Subject: Job opportunities in the UK
Message-ID: <1.5.4.32.19980925082028.006f8e84@soc.plym.ac.uk>


Based in Plymouth, England, Neural Systems is a young and dynamic
company in the field of Neural Computing.  We are currently looking
for two key individuals to work on a new project.

Senior Research Engineer

Applicant requirements:

* Ph.D. in the field of neural computing
* Experience of the application of neural network technologies
* Strong programming skills using C++ and/or Java
* Self motivated individual with the ability to take responsibility for
project development

Applicant desirables:

* An understanding of process simulation
* Some statistical data analysis experience

Research Engineer

Applicant requirements:

* Post graduate in computer science, engineering or related discipline
* Excellent programming skills using C++ and/or Java
* Experience of the implementation of agent-based programming techniques
* Experience of the software implementation of advanced neural networks
* Microsoft NT operating system experience


Applicants for both positions should be able to demonstrate high levels of
professionalism and technical innovation.  In return Neural Systems, an
equal opportunity employer, are offering a competitive reward package and
the opportunity to work in a leading edge technology, with the chance to
play a major part in the companies' development.

Interested applicants should forward their CV and a covering letter to:

Human Resources
Neural Systems Limited
Tamar Science Park
1 Davy Road
Derriford
Plymouth PL6 8BX
E-mail: HR at neuralsys.com

Dr Sue McCabe
Centre for Neural and Adaptive Systems
School of Computing
University of Plymouth
Plymouth PL4 8AA
England

tel: +44 17 52 23 26 10
fax: +44 17 52 23 25 40
e-mail: sue at soc.plym.ac.uk
http://www.tech.plym.ac.uk/soc/research/neural/index.html


From mharm at CNBC.cmu.edu  Fri Sep 25 15:40:08 1998
From: mharm at CNBC.cmu.edu (Mike Harm)
Date: Fri, 25 Sep 1998 15:40:08 EDT
Subject: thesis available: Division of labor in visual word recognition
Message-ID: <199809251940.PAA04961@CNBC.CMU.EDU>

Hi.

My Ph.D. thesis is now publicly available.  



=================================================================

          DIVISION OF LABOR IN A COMPUTATIONAL MODEL OF 
                   VISUAL WORD RECOGNITION


                       Michael W. Harm
               University of Southern California
                Department of Computer Science
                        August, 1998


Abstract:

How do we compute the meanings of written words?  For decades, the
basic mechanisms underlying visual word recognition have remained
controversial.  The intuitions of educators and policy makers, and the
existing empirical evidence have resulted in contradictory
conclusions, particularly about the role of the sound structure of
language (phonology) in word recognition.  To explore the relative
contributions of phonological and direct information in word
recognition, a large scale connectionist model of visual word
recognition was created containing orthographic, semantic and
phonological representations. The behavior of the model is analyzed
and explained in terms of redundant representations, the development
of dynamic attractors in representational space, the time course of
activation and processing within such networks, and demands of the
reading task itself.  The different patterns of results that have been
obtained in previous behavioral studies are explained by appeal to
stimulus composition and properties of a common experimental paradigm.
A unified explanation of a wide range of empirical phenomena is
presented.

=================================================================

PDF version (you may need acrobat 3.0):

    ftp://siva.usc.edu/pub/coglab/mharm/thesis.pdf  (631 kb)

Compressed postscript version:

    ftp://siva.usc.edu/pub/coglab/mharm/thesis.ps.Z  (381 kb)


It's about 120 pages.


Cheers,

Mike Harm
mharm at cnbc.cmu.edu
Center for the Neural Basis of Cognition
Carnegie Mellon University
http://www.cnbc.cmu.edu/~mharm/
-----------------------------------------

    At midnight, all the agents,
    And the superhuman crew,
    Come out and round up everyone,
    That knows more than they do.

                  Bob Dylan,  "Desolation Row"

From mel at lnc.usc.edu  Fri Sep 25 21:13:25 1998
From: mel at lnc.usc.edu (Bartlett Mel)
Date: Fri, 25 Sep 1998 18:13:25 -0700
Subject: Preprint Available: Binding Problem
Message-ID: <360C3FB5.A08E09A@lnc.usc.edu>

The following preprint is now available via our web page:

http://lnc.usc.edu
25 pages, 230K gzipped postscript

=========================================================

"Seeing with Spatially-Invariant Receptive Fields: 
When the `Binding Problem' Isn't"

Bartlett W. Mel
Biomedical Engineering Department
University of Southern California

Jozsef Fiser
Department of Brain and Cognitive Sciences
University of Rochester


ABSTRACT

We have studied the design tradeoffs governing visual representations
based on complex spatially-invariant receptive fields (RF's), with an
emphasis on the susceptibility of such systems to false-positive
recognition errors---Malsburg's classical ``binding'' problem.  We begin
by deriving an analytical model that makes explicit how recognition
performance is affected by the number of objects that must be
distinguished, the number of features included in the representation,
the complexity of individual objects, and the clutter load, i.e. the
amount of visual material in the field of view in which multiple objects
must be simultaneously recognized, independent of pose, and without
explicit segmentation.  Using the domain of text as a convenient
surrogate for object recognition in cluttered scenes, we show that, with
corrections for the non-uniform probability and non-independence of
English text features, the analytical model achieves good fits to
measured recognition rates in simulations involving a wide range of
clutter loads, word sizes, and feature counts.  We then present a greedy
algorithm for feature learning, derived from the analytical model, which
grows a visual representation by choosing those features most likely to
distinguish objects from the cluttered backgrounds in which they are
embedded.  We show that the representations produced by this algorithm
are decorrelated, heavily weighted to features of low conjunctive order,
and remarkably compact.  Our results provide a quantitative basis for
understanding when, and under what conditions, spatially-invariant
RF-based representations can support veridical perception in
multi-object scenes, and lead to several insights regarding the
properties of visual representations optimized for specific visual
recognition tasks.


-- 

 Bartlett W. Mel                               (213)740-0334, -3397(lab)
 Assistant Professor of Biomedical Engineering (213)740-0343 fax
 University of Southern California, OHE 500     mel at lnc.usc.edu,
http://lnc.usc.edu
                             
     US Mail: BME Department, MC 1451, USC, Los Angeles, CA 90089
       Fedex: 3650 McClintock Ave, 500 Olin Hall, LA, CA 90089

From Eddy.Mayoraz at idiap.ch  Fri Sep 25 12:06:44 1998
From: Eddy.Mayoraz at idiap.ch (Eddy.Mayoraz@idiap.ch)
Date: Fri, 25 Sep 1998 18:06:44 +0200
Subject: Ph.D. position
Message-ID: <199809251606.SAA11198@bishorn.idiap.ch>

******** Uni-Lausanne & IDIAP-Martigny (Switzerland) *******

         PhD student position

We are seeking one outstanding  PhD candidate for an exciting research project,
the aim  of which  is the study  and the  adaptation of recent  developments in
machine learning (neural networks, mixture of experts, support vector machines)
for the resolution of some specific geostatistical tasks.

While  classical geostatistics deals  with spatial  interpolations, one  of the
focuses of this project is to predict,  not only a single expected value of the
observable in an arbitrary location, but also a probability distribution.  This
allows, among other  things, the computation of risk  estimates for being above
some  threshold, which  is critical,  for example,  in applications  related to
pollution.  Another objective of this project is to understand the correlations
between  several spatial  observables and  their exploitation  for  an improved
conditional estimate of one observable given the others.

This is  a joint project  with the Group  of Geostatistics at the  Institute of
Mineralogy and  Petrography, University of  Lausanne, and the  Machine Learning
Group of IDIAP -- Institute for Perceptual Artificial Intelligence at Martigny,
both in Switzerland.

The position is available as soon  as possible and for two years, renewable for
two more  years. Highly  qualified candidates are  sought with a  background in
computational  sciences,  statistics, mathematics,  physics  or other  relevant
areas.  Applicants should submit :  (i) Detailed curriculum vitae, (ii) List of
three references (and their email addresses), (ii) Transcripts of undergraduate
and  graduate (if  applicable) studies  and  (iii) Concise  statement of  their
research interests (two pages max). Please send all documents to:

Prof. Michel Maignan                                Michel.Maignan at imp.unil.ch
Institute for Mineralogy and Petrography        http://www-sst.unil.ch/geostat
Earth Sciences
University of Lausanne
1015 Lausanne, Switzerland

or

Dr. Eddy Mayoraz                                         Eddy.Mayoraz at idiap.ch
IDIAP                                             http://www.idiap.ch/learning
CP 592
1920 Martigny, Switzerland

Electronic applications (with WWW pointers to studies or papers, if available)
are encouraged.

Michel Maignan & Eddy Mayoraz


    /  _ \   /  _ \   _ \   Dr. Eddy Mayoraz, research director of the ML group
   /  /  /  /  /  /  /  /   IDIAP,  P.O. Box 592, CH-1920 Martigny, Switzerland
  /  /  /  /  _  /  ___/    voice: +41 27 721 77 29(11),  fax: +41 27 721 77 12
_/  ___/ _/ _/ _/ _/        Eddy.Mayoraz at idiap.ch, http://www.idiap.ch/~mayoraz


From harnad at coglit.soton.ac.uk  Sun Sep 27 13:16:57 1998
From: harnad at coglit.soton.ac.uk (Stevan Harnad)
Date: Sun, 27 Sep 1998 18:16:57 +0100 (BST)
Subject: Efference and Knowledge: Psyc Call for Commentators
Message-ID: <Pine.SGI.3.95.980927181449.367M-100000@mulberry.psy.soton.ac.uk>


                Jarvilehto: Efference and Knowledge

    The target article whose abstract appears below has just appeared
    in PSYCOLOQUY, a refereed journal of Open Peer Commentary sponsored
    by the American Psychological Association. Qualified professional
    biobehavioral, neural or cognitive scientists are hereby invited to
    submit Open Peer Commentary on it. Please email for Instructions if
    you are not familiar with format or acceptance criteria for
    PSYCOLOQUY commentaries (all submissions are refereed).

    To submit articles and commentaries or to seek information:

    EMAIL: psyc at pucc.princeton.edu 
    URL:   http://www.princeton.edu/~harnad/psyc.html
           http://www.cogsci.soton.ac.uk/psyc

    RATIONALE FOR SOLICITING COMMENTARY: On the basis of experimental
    data plus a simple thought experiment, it is argued that the senses
    should not be considered as transmitting environmental information
    to an organism. Rather, they are part of a dynamical
    organism-environment system in which efferent influences on the
    sensory receptors are especially critical. This view has both
    experimental and philosophical implications for understanding
    knowledge formation on which commentary is invited from
    psychophysicists, sensory physiologists, developmental
    neuroscientists, cognitive scientists, computational modelers,
    information theorists, Gibsonians, Gestaltists, and philosophers.

Full text of article available at:
    http://www.cogsci.soton.ac.uk/cgi/psyc/newpsy?9.41
or:
    ftp://ftp.princeton.edu/pub/harnad/Psycoloquy/1998.volume.9/psyc.98.9.41.efference-knowledge.1.jarvilehto

-----------------------------------------------------------------------
psycoloquy.98.9.41.efference-knowledge.1.jarvilehto     Sun Sep 27 1998
ISSN 1055-0143                (41 paragraphs, 28 references, 623 lines)
PSYCOLOQUY is sponsored by the American Psychological Association (APA)
                Copyright 1998 Timo Jarvilehto

        EFFERENT INFLUENCES ON RECEPTORS IN KNOWLEDGE FORMATION

                Timo Jarvilehto 
                Department of Behavioral Sciences,
                University of Oulu, 
                Finland 
                tjarvile at ktk.oulu.fi
                http://wwwedu.oulu.fi/ktleng/ktleng.htm

    ABSTRACT: This target article suggests a new interpretation of
    efferent influences on sensory receptor activity and the role of
    the senses in forming knowledge. Experimental data and a thought
    experiment about a hypothetical motor-only organism suggest that
    the senses are not transmitters of environmental information;
    rather, they create a direct connection between the organism and
    the environment that makes possible a dynamic organism-environment
    system.  In this system efferent influences on receptor activity
    are especially critical, because with their help the receptors can
    be adjusted in relation to the parts of the environment that are
    most important in achieving behavioral results. Perception joins
    new parts of the environment to the organism-environment system;
    thus knowledge is formed by perception through a reorganization (a
    widening and differentiation) of the organism-environment system
    rather than through the transmission of information from the
    environment.  With the help of efferent effects on receptors, each
    organism creates its own particular world. These considerations
    have implications for experimental work in the neurophysiology and
    psychology of perception as well as for the philosophy of knowledge
    formation.

    KEYWORDS: afference, artificial life, efference, epistemology,
    evolution, Gibson, knowledge, motor theory, movement, perception,
    receptors, robotics, sensation, sensorimotor systems, situatedness




From gyen at okway.okstate.edu  Sun Sep 27 15:57:35 1998
From: gyen at okway.okstate.edu (Gary Yen)
Date: Sun, 27 Sep 1998 13:57:35 -0600
Subject: Conference Announcement
Message-ID: <9809279069.AA906921992@okway.okstate.edu>

Contributed by: Gary G. Yen    gyen at master.ceat.okstate.edu



CALL FOR PAPERS

1999 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORK

RENAISSANCE HOTEL
WASHINGTON, D.C.
JULY 10-16, 1999

The premier conference on neural networks returns to Washington, D.C. in 
1999. Come celebrate the end of the Decade of the Brain and prepare to 
welcome the next millenium with a review of where we are and a preview of 
where we are going. IJCNN'99 spans the neural network field from neurons to 
consciousness, from learning algorithms to robotics, from chaos to control. 
This is an unparalleled opportunity to learn about cutting edge 
developments, to publicize your contributions, and to form new ties with 
your colleagues in industry, academia, and government from around the 
world. For details see our web site: www.inns.org or contact David Brown, 
IJCNN'99 General Chair: dgb at cdrh.fda.gov

CONFERENCE HIGHLIGHTS:
* Distinguished plenaries, highlighted by keynote speaker John Hopfield. 
* Strong technical program, presenting leading research in all 
  neural-network related fields.
* Focused special sessions on such subjects as Biomedical Applications and
  Chaos.
* Theme symposium: Modeling the Brain from Molecules to Mind.
* Forum on Global Competitiveness, including Congressional face-off on U.S.
  SBIR program.
* International workshops on "Getting Your Application to Market,"
  including patent and venture capital questions, and on "Getting Funding  
  for your Rsearch."
* Vigorous tutorial program, with courses taught by the leading experts in
  our field.
* Exhibits and demonstrations of real-world applications.
* Media fair-Show what you have achieved and help educate the public. 
* Awards for best posters and for student contributions.
* Job fair, matching up applicants with employment opportunities. 
* And much, much more: check out IJCNN'99 news on the web site:
  www.inns.org

CALL FOR PAPERS
(Deadline December 4, 1998)
Co-technical Program Committee Chairs: 
Daniel Alkon (NIH)  Clifford Lau (ONR)

IJCNN'99 review and acceptance will be based on a one-page summary, which 
will be distributed to Conference participants in a summary book. Detailed 
instructions for authors are given on the reverse side of this sheet and on 
the web site. Conference proceedings will be in CD form. Hard-copy 
proceedings may be available for an additional fee.

Use of illustrations is encouraged in the summaries, within the single 
allowed page. Use of audio and video segments in the CD proceedings will be 
considered. Poster presentations will be encouraged, with "single-slide" 
poster presentations interspersed with regular oral sessions. Monetary 
awards presented for best poster in several categories. Best student paper 
awards will also be given.

INSTRUCTIONS FOR AUTHORS:
Paper summary format information

The summary must be accompanied by a cover sheet listing the following 
information:
1. Paper title
2. Author information - full names and affiliations as they will appear in
   the program
3. Mailing address, telephone, fax, and email for each author 4. Request 
   for oral or poster presentation
5. Topic(s), selected from the list below:

Biological foundations       Neural systems 
Mathematical foundations     Architectures 
Learning algorithms          Intelligent control 
Artificial systems           Data analysis 
Pattern recognition          Hybrid systems 
Intelligent computation      Applications

  
The one-page, camera-ready summary must conform to the following 
requirements:
1. All text and illustrations must appear within a 7x9 in. (178x229mm)
   area. For US standard size paper (8.5x11 in.), set margins to 0.75 in.   
   (18mm) left and right and 1 in. top and bottom. For A4 size paper, set 
   margins to 17mm left and right, 25mm top, and 45 mm bottom.
2. Use 10-point Times Roman or equivalent typeface for the main text.
   Single space all text, allowing extra space between paragraphs.
3. Title (16 pt. Bold, centered) Capitalize only first word and proper
   names.
4. Authors names, affiliation (12 pt., centered) omit titles or degrees.
5. Five section headings (11 pt. bold). The following five headings MUST be
   used: Purpose, Method, Results, New or breakthrough aspect of work, and 
   Conclusions.

Three copies of the one-page summaries are due in camera-ready, hardcopy 
form to INNS by December 4, 1998. Sent to:

IJCNN'99/INNS
19 Mantua Road
Mt. Royal, NJ 08061 USA
Phone: 609-423-7222 ext. 350

Acceptance will be determined by February 8, 1999, and complete papers are 
due in digital form for CD publication by May 3, 1999.


From barba at cvs.rochester.edu  Mon Sep 28 12:32:44 1998
From: barba at cvs.rochester.edu (Barbara Arnold)
Date: Mon, 28 Sep 1998 12:32:44 -0400
Subject: ad posting
Message-ID: <l03102834b2356a7144f4@[128.151.80.12]>

Please post this ad as soon as possible.

Thank you,
Barbara Arnold

*******************************************************************
                  GRADUATE AND POSTDOCTORAL TRAINING
                   IN THE CENTER FOR VISUAL SCIENCE
                    AT THE UNIVERSITY OF ROCHESTER
*******************************************************************

The Center for Visual Science (CVS) at the University of Rochester
is among the largest research centers dedicated to the study of
visual perception at any university in the world.  Currently CVS
consists of more than 25 research laboratories.  These laboratories
are studying nearly all aspects of vision, from its earliest stages,
such as the encoding of spatial and temporal patterns of light by
neurons in the retina, to its latest stages, such as the interaction
between visual perception and memory.  These laboratories employ a
wide range of theoretical perspectives as well as a diversity of
neuroscientific, behavioral, and computational methodologies.

CVS is a research center that provides a number of services to its
members.  Most important, CVS provides a collegial community in which
vision scientists can meet with each other in order to discuss their
latest research projects and interests.  In addition, CVS provides its
members with a vast array of experimental and computational resources,
an extensive colloquium series, a bi-annual symposium, and other
amenities designed to promote the research activities of its members.


GRADUATE STUDY IN THE CENTER FOR VISUAL SCIENCE

Many students currently pursue graduate training in the Center for
Visual Science.  CVS offers a supportive environment in which students
receive training through coursework and research activities that are
supervised by one or more faculty members.  Due to its large size, CVS
can offer students a training program that is distinctive in its breadth
and depth.  Students can receive training in nearly all aspects of vision,
from its earliest stages in the retina to its latest stages where it
interacts with cognition.  Students are also exposed to a wide range of
theoretical perspectives as well as a diversity of neuroscientific,
behavioral, and computational methodologies.  Regardless of the nature
of a student's interests in visual perception, and regardless of how those
interests evolve during a student's graduate studies, the student can feel
confident that CVS provides an exceptional training environment for that
student.

Graduate study in the Center is undertaken through a degree program
administered by a collaborating academic unit, most often Brain and
Cognitive Sciences, Computer Science, Neuroscience, or Optics.  A
student chooses one of these degree programs, and satisfies its
requirements for a Ph.D. while specializing in visual science.
Because the program of study in the Center for Visual Science draws
students from a variety of backgrounds, and is integrated with the
other programs, the plan of study is flexible and easily tailored to
suit individual students' needs and interests, while ensuring a
thorough grounding in visual science.

The program of study emphasizes research that is supervised by one or
more members of the faculty, complemented by courses in visual perception
typically taken during the first two years of study. The sequence of
courses begins with the two-semester Mechanisms of Vision, which is team
taught by the faculty at the Center and covers a full range of topics in
vision.  This is followed by a series of more advanced courses, on topics
such as Color Vision, Spatial Vision, Motion Perception, Visual Space
Perception, Computational Problems in Vision, Computational Models of
Behavior, Real-time Laboratory Computing, and Instrumentation and Methods
for Vision Research.  Throughout the program students are actively engaged
in research; during their last two to three years of the four to five year
program students spend all of their time on the research that culminates
in the Ph.D.

Students contemplating graduate work at the Center should contact
Barbara Arnold (address below) who will be glad to provide additional
information and application materials.  Admission to the Center's
program includes a tuition waiver and a competitive 12-month stipend
that is guaranteed for at least four years, subject to satisfactory
progress.


POSTDOCTORAL STUDY IN THE CENTER FOR VISUAL SCIENCE

Many postdoctoral fellows currently receive training in the Center
for Visual Science.  The wide range of scientific disciplines
represented by faculty of the Center and the closeness of their
collegial contacts makes the Center a particularly attractive place
for interdisciplinary research.  Postdoctoral fellows often work with
more than one member of faculty, and the emphasis of the training is
on research methods (especially the conjunction of different methods
brought to bear on a single problem) that are characteristic of the
Center.

Scientists interested in postdoctoral study at the Center should contact
the faculty member(s) with whom they might wish to work. Postdoctoral
fellows are supported from a variety of sources: some receive support
through individual investigators' research grants; some receive stipends
from the Center's training grant, funded by the National Eye Institute;
some are supported by individual fellowships.

CVS is currently seeking new graduate students and postdoctoral
fellows to join our community.  To learn more about the Center for
Visual Science, please contact us at:

        Barbara Arnold
        Center for Visual Science
	Meliora Hall, River Campus
        University of Rochester
        Rochester, NY 14627
        Phone: (716) 275-2459
        Fax: (716) 271-3043
        e-mail: barba at cvs.rochester.edu
        WWW: http://www.cvs.rochester.edu


Faculty:
-------

Richard Aslin
   Perceptual development in infants

Dana Ballard
   Computer vision, computational neuroscience, visuomotor integration

Daphne Bavelier
   Brain imaging, visual attention, visuospatial representations

David Calkins
   Retinal neurophysiology, visual neuroscience

Robert Chapman
   Brain imaging, visual information processing

Charles Duffy
   Visual motion processing, visual neuroscience

Robert Emerson
   Spatial vision, visual neuroscience

Mary Hayhoe
   Visual perception and cognition, visuomotor integration

James Ison
   Audition, sensory reflexes, sensori-motor control

Robert Jacobs
   Visual perception and cognition, computational modeling

Carolyn Kalsow
   Clinical research in vision and eye care

Barrett Katz
   Clinical research in vision and eye care

Walter Makous
   Visual psychophysics, spatial vision

William Merigan
   Visual neural pathways, visual neuroscience

Gary Paige
   Vestibular and adaptive control of equilibrium, visual neuroscience

Tatiana Pasternak
   Neural mechanisms of motion and form perception, visual neuroscience

Alexandre Pouget
   Computational neuroscience, neural coding, visuospatial representations

James Ringo
   Neural mechanisms of memory and visual processing, visual neuroscience

Marc Schieber
   Neural control of finger movements, sensorimotor integration

Gail Seigel
   Retinal cell biology, visual neuroscience

Michael Weliky
   Neural development of the visual system, visual neuroscience

David Williams
   Spatial and color vision, retinal structure and visual perception

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Barbara N. Arnold
Administrator                            email: barba at cvs.rochester.edu
Center for Visual Science                phone: 716 275 8659
University of Rochester                    fax: 716 271 3043
Meliora Hall 274
Rochester NY 14627-0270
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 



From ormoneit at stat.Stanford.EDU  Mon Sep 28 15:50:21 1998
From: ormoneit at stat.Stanford.EDU (Dirk Ormoneit)
Date: Mon, 28 Sep 1998 12:50:21 -0700 (PDT)
Subject: Thesis on Density Estimation
Message-ID: <199809281950.MAA23100@rgmiller.Stanford.EDU>

Hi,

My PhD thesis on probability estimating neural networks
is now available from Shaker Verlag / Aachen (ISBN 3-8265-3723-8):

http://www.shaker.de/Online-Gesamtkatalog/Details.idc?ISBN=3-8265-3723-8

For more information on specific topics touched upon in the abstract below,
see also my Stanford homepage:

http://www-stat.stanford.edu/~ormoneit/

Best,

Dirk

===========================================================================

                   PROBABILITY ESTIMATING NEURAL NETWORKS

				Dirk Ormoneit
			Fakult"at f"ur Informatik
		     Technische Universit"at M"unchen

A central problem of machine learning is the identification of  
probability distributions that govern uncertain environments. 
A suitable concept for ``learning'' probability distributions from
sample data may be derived by employing neural networks. In this work 
I discuss several neural architectures that can be used to learn 
various kinds of probabilistic dependencies.
After briefly reviewing essential concepts from neural learning and 
probability theory, I provide an in-depth discussion of neural and other 
approaches to conditional density estimation. In particular, I  
introduce the ``Recurrent Conditional Density Estimation Network (RCDEN)'', 
a neural architecture which is particularly well-suited to 
identify the transition densities of time-series in the presence of 
latent variables. As a practical example, I consider the conditional 
densities of German stock market returns and compare the results of the 
RCDEN to those of ARCH-type models.
A second focus of the work is on the estimation of multivariate densities
by means of Gaussian mixture models. A severe problem for the practical 
application of Gaussian mixture estimates is their strong tendency to 
``overfit'' the training data. In my work I compare three regularization 
procedures that can be applied to deal with this problem. 
The first method consists of deriving EM 
update rules for maximum penalized likelihood estimation. In the second
approach, the ``full'' Bayesian inference is approximated by means of 
a Markov chain Monte Carlo algorithm. Finally, I apply ensemble averaging
to regularize the Gaussian mixture estimates, most prominently a variant of
the popular ``bagging'' algorithm. The three approaches are compared in
extensive experiments that involve the construction of Bayes classifiers
from the density estimates. The work concludes with considerations on
several practical applications of density estimating neural networks in
time-series analysis, data transmission, and optimal planning in 
multi-agent environments.


--------------------------------------------
Dirk Ormoneit
Department of Statistics, Room 206
Stanford University
Stanford, CA 94305-4065
 
ph.: (650) 725-6148
fax: (650) 725-8977
 
ormoneit at stat.stanford.edu
http://www-stat.stanford.edu/~ormoneit/


From smagt at dlr.de  Mon Sep 28 04:20:50 1998
From: smagt at dlr.de (Patrick van der Smagt)
Date: Mon, 28 Sep 1998 10:20:50 +0200
Subject: paper on conditioning and local minima in MLP
Message-ID: <360F46E2.42CFD049@robotic.dlr.de>

Dear connectionists:

the following ICANN'98 reprint is available via the web:
http://www.robotic.dlr.de/Smagt/papers/SmaHir98b.ps.gz

  "Why feed-forward networks are in a bad shape"
        P. van der Smagt and G. Hirzinger
    German Aerospace Center/DLR Oberpfaffenhofen

Abstract:
It has often been noted that the learning problem in feed-forward neural
networks is very badly conditioned.  Although, generally, the special form of
the transfer function is usually taken to be the cause of this condition, we
show that it is caused by the manner in which neurons are connected.  By
analyzing the expected values of the Hessian in a feed-forward network it is
shown that, even in a network where all the learning samples are well chosen
and the transfer function is not in its saturated state, the system has a
non-optimal condition.  We subsequently propose a change in the feed-forward
network structure which alleviates this problem.  We finally demonstrate the
positive influence of this approach.


Other papers available on http://www.robotic.dlr.de/Smagt/papers/
--
dr Patrick van der Smagt                phone +49 8153 281152, fax -34
DLR/Institute of Robotics and System Dynamics             smagt at dlr.de
P.O.Box 1116, 82230 Wessling, Germany     http://www.robotic.de/Smagt/

From gkk at neuro.informatik.uni-ulm.de  Mon Sep 28 18:57:22 1998
From: gkk at neuro.informatik.uni-ulm.de (Gerhard K. Kraetzschmar)
Date: Tue, 29 Sep 1998 00:57:22 +0200
Subject: Call for Interest in Participation in IJCAI-99 Workshop
Message-ID: <36101452.A96E5A97@neuro.informatik.uni-ulm.de>


Dear reader of this news group or list: 
(Our apologies, if you receive this multiple times)

We plan to organize a workshop at IJCAI-99 in Stockholm.
The topic is
        Adaptive Spatial Representations for Dynamic Environments 
and we believe it may be of interest to you. Please read the 
draft for the workshop proposal in the attachment for more information
on the workshop.

You can contribute to the workshop by submitting a paper, giving one of 
the survey talks, or one of the commenting statements in a session.

Note: IJCAI permits only active participants for workshops.

If you come to the conclusion that you are indeed interested in the
workshop and want to actively participate, please take the time to respond
with a short email that indicates your interest and kind of contribution.
Please send the email to 
	gkk at acm.org
and include "IJCAI-WORKSHOP" in the subject line. Thanks a lot.

Please do respond soon, because the due date for the proposal is in a 
few days, and we need to collect a list of tentative participants for it.

-- 
Sincerely Yours, Gerhard
---------------------------------------------------------------------------
Dr. Gerhard K. Kraetzschmar                   University of Ulm
Fon: intl.+49-731-502-4155	              Neural Information Processing
Fax: intl.+49-731-502-4156	              Oberer Eselsberg
Net: gkk at neuro.informatik.uni-ulm.de          89069 Ulm   
     gkk at acm.org                              Germany
WWW: http://www.informatik.uni-ulm.de/ni/staff/gkk.html
---------------------------------------------------------------------------

Proposal for IJCAI-99 Workshop
========================================================
Adaptive Spatial Representations of Dynamic Environments
========================================================
Workshop Description:
====================
Spatial representations of some sort are a necessary requirement for
any mobile robot in order to achieve tasks like efficient,
goal-oriented navigation, object manipulation, or interaction with
humans. A large number of approaches, ranging from CAD-like and
topological representations to metric, probabilistic methods and
biologically-oriented representations, has been developed in the past.
Most approaches were developed for solving robot navigation tasks 
and successfully applied in a wide variety of applications. In many
approaches, the spatial representation is a strong simplification of 
the environment (e.g. occupied and free space) and does not permit an 
easy representation of the spatial structure of task-relevant objects.
Furthermore, these approaches can model only static elements of the
environment in an adequate manner. Dynamic elements, such as doors,
changed locations of particular objects, moving obstacles, and humans,
are usually dealt with in one of two ways:
- A purely reactive level temporarily has a transient representation 
	for an anonymous object. The representation is present only as 
	long as the object can be actively sensed; it vanishes
	thereafter and does not settle into some kind of permanent
	representation. (Doors, moving obstacles, humans.)
- Repeated exposure of the robot to both the old and new location of
	an object that changed its position leads to a slow adaptation
	of (long-term) spatial memory. (Moved, relocated objects)

The current representations are sufficient for many tasks that have 
been researched in the past and led to many successful applications.
However, in order to achieve truely useful robots, e.g. for the office
domain, the robot will have to acquire AND MAINTAIN more complete 
models of its environment. It will have to know the precise locations 
(or have a good estimate of it) of objects and subjects it has seen,
if they are relevant for completing a task. Examples: Location of
tools (screwdriver), books, special equipment (video beamer), persons, 
doors, etc.

Thus, for any existing spatial representation, the following 
questions arise?
- Which structural aspects of the environment can be modeled? How?
- Can the representation model dynamic aspects at all? Could it be extended?
- Which kinds of dynamic aspects can be modeled?
- How are the spatial dynamics of an object modeled?
- How is uncertainty dealt with?
- How are the dynamics used to predict various spatial aspects of
	objects?
- Which methods can be used to update/maintain the representation
	based on sensory information?
- What computational effort is required for updating the representation?

For many of the current approaches, we have little knowledge about how 
these questions must be answered. The goal of the workshop is to bring 
together researchers who develop and use various kinds of spatial 
representations in mobile robots and make them think about how to answer 
the above questions for their approaches. A secondary goal is to provide 
a forum on which various kinds of representations can be compared.

Workshop Actuality and Target Audience:
======================================= 
Currently, various successful, though specialized applications of
mobile robots (RHINO, etc.) are known. However, improved adaptive
spatial representations will be needed to build service robots for
more complex tasks in more complex environments. We consider such
representations an essential step for making progress towards this
goal. Thus, the target audience includes 
- researchers working in mobile robots, especially map building, 
  spatial modelling, navigation, object manipulation, and human-robot
  interaction, 
- AI researchers working on topological and other symbolic methods, 
  metric and probabilistic representations, and uncertainty. 
The workshop also appeals to researchers that have 
- studied spatial data structures in CAD, GIS, and image processing or
- studied spatial representations in biological systems 
and are now applying their models in robotic systems.

Preliminary Workshop Agenda:
============================
Depending on submissions, available time, and IJCAI constraints on the
schedule, we plan four to five sessions, each one will most likely be 
centered around one of the following themes:
- CAD/GIS-Inspired Representations (Frank/?Samet)
- Topological Representations (Kuipers/Cohn/Nebel)
- Metric and Probabilistic Representations (Burgard/?Kaelbling)
- Biologically-Inspired Representations (Recce/Tani/?Mallot)

In each session (90 minutes) covering one of the above four
approaches, we plan to implement the following session program:
- Invited survey talk by an experienced research scientist (25+5 min)
- Two short talks selected from the workshop paper submissions (10+5 min)
- Two to three rebutting/commenting statements by representatives 
	from the other approaches (5 min each)
- Session discussion (15 min), moderated by session chair
A general discussion session (45 to 60 minutes) will try to 
summarize results, draw conclusions, and define future activities,
like workshops, definition of benchmark problems, and others.

Tentative Attendees:
====================
(will be collected from response after announcements of various 
 news groups and lists:
	comp.robotics
	comp.ai
	comp.ai.neural-nets
	comp.ai.nlang-know-rep
	connectionists list
	hybrid list
 Please add any relevant news group or list)
	
Workshop Organizing Committee:
=============================
* To be confirmed.

Andrew Frank (GIS/CAD representations)
Bernhard Nebel (AI, relational/topological representations)
Anthony Cohn (AI, relational/topological representations)
Gerhard Kraetzschmar (chair) (AI, robotics, hybrid representations)
Benjamin Kuipers (AI, robotics, hybrid representations)
Michael Beetz (AI, robotics, metric/probabilistic representations)
Wolfram Burgard (AI, robotics, metric/probabilistic representations)
Gunther Palm (neuroscience, neural networks)
Michael Recce (neuroscience, biologically-inspired robotics)
Jun Tani (biologically-inspired robotics, dynamics) 

*Leslie Kaelbling
*Hanspeter Mallot
*Hanan Samet

Primary Contact:
===============
Gerhard K. Kraetzschmar
University of Ulm, Neural Information Processing
James-Franck-Ring, 89081 Ulm, Germany
Fon: +49-731-50-24155
Fax: +49-731-50-24155
Net: gkk at acm.org  or gkk at neuro.informatik.uni-ulm.de

From ghaziri at aub.edu.lb  Tue Sep 29 17:59:40 1998
From: ghaziri at aub.edu.lb (Dr. Hassan Ghaziri)
Date: Tue, 29 Sep 1998 14:59:40 -0700
Subject: IFORS 99 CHINA, NN in OR
References: <36101452.A96E5A97@neuro.informatik.uni-ulm.de>
Message-ID: <3611584C.E650E291@aub.edu.lb>

Dear Colleagues,
I have been asked to organize a session on  neural networks and thewir
applications in operations research as part of the cluster on metaheuristics,
for which I would like to invite you to
  participate in presenting a paper at IFORS-99:


 {                     IFORS'99, 15th World Conference on Operational
  Research  Triennial Meeting of the International Federation of
  Operational Research Societies -- IFORS
               Hosted by the Operations Research Society of China
                      Beijing, China, August 16 - 20, 1999.
  http://www.IFORS.org/leaflet/triennial.html
  where you can find more details on the conference.
  }

  A typical session is 100 minutes long and has 3-5 papers.
  Submission of full papers is optional.  All submitted full papers
  will be considered for publication in the International Transactions
  in Operational Research (Peter Bell - Editor; Publisher Pergamon
  Press on behalf of IFORS).  Papers should not exceed 5,000 words.

  If you are interested please let me have a  title of your
paper and send them to my Lebanon  address with the following details.
  - title of the paper,
  - authors names and addresses,
  - 50-100 word abstracts.

  I would like to have these information not later than October 16, 1998.

         I am looking forward to hearing from you at your
  earliest.

         Best regards
Hassan Ghaziri
AUB, Business Scool, Beirut Lebanon
Tel: 00 961-1 352700
E-mail: ghaziri at aub.edu.lb






From niall.griffith at ul.ie  Tue Sep 29 08:41:51 1998
From: niall.griffith at ul.ie (Niall Griffith)
Date: Tue, 29 Sep 1998 13:41:51 +0100
Subject: Neural Networks and MultiMedia - IEE Colloquium
Message-ID: <9809291241.AA24335@zeus.csis.ul.ie>

Please pass this on to anyone or any group you think may be interested.

=========================================================
IEE Colloquium

"Neural Networks in Multimedia Interactive Systems"

Date:  Thursday 22 October 1998
Time:  10.30 - 17.00
Place: Savoy Place, London.

The Neural Networks Professional Group (A9) of the IEE is holding an
inaugural colloquium at Savoy Place, London, on the use of neural network
models in multimedia systems.

This colloquium will present a range of current neural network
applications in the area of interactive multimedia, and will cover:
learning, intelligent agents within multimedia systems, data mining, image
processing and intelligent application interfaces.

========================================================== 

For more information and registration details please contact 

The IEE Events Office:

Tel.   +44 171 240 1871    (Extension 2205/2206)
Email. events at iee.org.uk
URL.   http://www.iee.org.uk/Calendar/a22oct98.html

Colloquium organisers:
Niall Griffith, University of Limerick, niall.griffith at ul.ie
Nigel Allinson, UMIST, allinson at fs5.ee.umist.ac.uk
==========================================================

Provisional Timetable

10.00 - 10.25   Registration and Coffee

10.25 - 10.45   Welcome 
                Self-organising neural networks for multimedia
                Prof. N. Allinson (UMIST)

10.45 - 11.25   Searching image databases containing trademarks.
                Sujeewa Alwis and Dr. J. Austin (University of York)

11.25 - 12.05   Synthetic Characters: Behaving in Character
                Dr. B. Blumberg (MIT Media Lab)

12.05 - 12.45   Intelligent components for interactive multimedia
                Dr. R. Beale (University of Birmingham)

12.45 - 13.45   Lunch

13.45 - 14.15   Image Retrieval and Classification Using Affine 
                Invariant B-Spline Representation and Neural Networks
                Y. Xirouhakis  (National Technical University of Athens)

14.15 - 14.45   Hybrid Neural Symbolic Agent Architectures for Multimedia
                Prof. S. Wermter (University of Sunderland)

14.45 - 15.15   3D Reconstruction of Human Faces from Range Data 
                through HRBF Networks
                Dr. A. Borghese 
                (Istituto Neuroscienze e Biommagini, Milan)

15.15 - 15.30   Tea

15.30 - 16.00   A multi-agent framework for visual surveillance
                P. Remagnino & Dr. G Jones (Kingston University)

16.00 - 16.30   Discussion and Close
-----------------

From kehagias at egnatia.ee.auth.gr  Tue Sep 29 13:45:08 1998
From: kehagias at egnatia.ee.auth.gr (Thanasis Kehagias)
Date: Tue, 29 Sep 1998 10:45:08 -0700
Subject: New Book on modular NN and time series
Message-ID: <3.0.5.32.19980929104508.007a0a50@egnatia.ee.auth.gr>

NEW BOOK on Modular Neural Networks and Time Series

The following book has just been published by Kluwer Academic Publishers. 

TITLE:	Predictive Modular Neural Networks: Applications to Time Series 
AUTHORS:	V. Petridis and Ath. Kehagias 
PUBLISHER:	Kluwer Academic Publishers, Boston
YEAR:		1998
ISBN:		0-7923-8290-0

This book will be of interest to connectionists, machine learning
researchers, statisticians, control theorists and perhaps also to
researchers in biological and medical informatics, researchers in
econometrics and forecasting as well as psychologists. It can be ordered
from Kluwer's web site at 

http://www.wkap.nl/book.htm/0-7923-8290-0

The general subject of the book is the application of modular neural
networks (in another terminology: multiple models) to problems of time
series classification and  prediction. The problem of system identification
is also treated as a time series problem. We consider both supervised
learning of labelled TS data and unsupervised learning of unlabelled TS
data. We present a general framework for the design of PREDICTIVE MODULAR
algorithms and provide a rigorous convergence analysis for both the
supervised and unsupervised cases. We also present the application of the
above algorithms to three real world problems (encephalogram
classification, electric load prediction and waste water plant parameter
etsimation). Finally, we provide an extensive bibliography of modular and
multiple models methods and discuss the connections between such methods
which have appeared in the neural networks as well as in other research
communities.

More info about the book can be found at 

http://www.wkap.nl/book.htm/0-7923-8290-0

or at 

http://skiron.control.ee.auth.gr/~kehagias/thn/thn02b01.htm
-------------------------------------------------------------

TABLE OF CONTENTS

1. Introduction
1.1 Classification, Prediction and Identification: an Informal Description
1.2 Part I: Known Sources 
1.3 Part II: Applications 
1.4 Part III: Unknown Sources 
1.5 Part IV: Connections 

PART I Known Sources

2.  PREMONN Classification and Prediction
2.1 Bayesian Time Series Classification 
2.2 The Basic PREMONN Classification Algorithm 
2.3 Source Switching and Thresholding 
2.4 Implementation and Variants of the PREMONN Algorithm 
2.5 Prediction 
2.6 Experiments
2.7 Conclusions 

3.  Generalizations of the Basic PREMONN
3.1 Predictor Modifications
3.2 Prediction Error Modifications
3.3 Credit Assignment Modifications
3.4 Markovian Source Switching
3.5 Markovian Modifications of Credit Assignment Schemes
3.6 Experiments
3.7 Conclusions

4.  Mathematical Analysis
4.1 Introduction 
4.2 Convergence Theorems for Fixed Source Algorithms 
4.3 Convergence Theorem for a Markovian Switching Sources Algorithm 
4.4 Conclusions 

5.  System Identification by the Predictive Modular Approach
5.1 System Identification 
5.2 Identification and Classification 
5.3 Parameter Estimation: Small Parameter Set 
5.4 Parameter Estimation:\ Large Parameter Set
5.5 Experiments 
5.6 Conclusions 

PART II Applications

6.  Implementation Issues
6.1 PREMONN Structure
6.2 Prediction 
6.3 Credit Assignment 
6.4 Simplicity of Implementation 

7.  Classification of Visually Evoked Responses
7.1 Introduction 
7.2 VER Processing and Classification 
7.3 Application of PREMONN Classification 
7.4 Results 
7.5 Conclusions 

8.  Prediction of Short Term Electric Loads
8.1 Introduction 
8.2 Short Term Load Forecasting Methods 
8.3 PREMONN Prediction 
8.4 Results
8.5 Conclusions 

9.  Parameter Estimation for and Activated Sludge Process
9.1 Introduction 
9.2 The Activated Sludge Model 
9.3 Predictive Modular Parameter Estimation 
9.4 Results 
9.5 Conclusions 

PART III Unknown Sources

10.  Source Identification Algorithms
10.1 Introduction
10.2 Source Identification and Data Allocation 
10.3 Two Source Identification Algorithms 
10.4 Experiments 
10.5 A Remark about Local Models 
10.6 Conclusions 

11.  Convergence of Parallel Data Allocation
11.1 The Case of Two Sources 
11.2 The Case of Many Sources
11.3 Conclusions 

12.  Convergence of Serial Data Allocation
12.1 The Case of Two Sources 
12.2 The Case of Many Sources 
12.3 Conclusions 

PART IV Connections

13.  Bibliographic Remarks
13.1 Introduction 
13.2 Neural Networks 
		Combination of Specialized Networks252
		Ensembles of Networks
		Mixtures of Experts
		RBF and Related Networks
		Trees
13.3 Statistical Pattern Recognition 
13.4 Econometrics and Forecasting 
13.5 Fuzzy Systems 
13.6 Control Theory 
13.7 Statistics 

14.  Epilogue

Appendix: Mathematical Concepts
References
Index

-------------------------------------------------------------

			The book's PREFACE

The subject of this book is predictive modular neural networks and their
application to time series problems: classification, prediction and
identification. The intended audience is researchers and
graduate students in the fields of neural networks, computer science,
statistical pattern recognition, statistics, control theory and
econometrics. Biologists, neurophysiologists and medical engineers
may also find this book interesting. 

In the last decade the neural networks community has shown intense interest
in both modular methods and time series problems. Similar interest has been
expressed for many years in other fields as well, most notably in
statistics, control theory, econometrics etc. There is a considerable
overlap (not always recognized) of ideas and methods between these fields. 

Modular neural networks come by many other names, for instance multiple
models, local models and mixtures of experts. The basic idea is to
independently develop several ``subnetworks'' (modules), which may perform
the same or related tasks, and then use an ``appropriate'' method
for combining the outputs of the subnetworks. Some of the expected
advantages of this approach (when compared with the use of ``lumped'' or
``monolithic'' networks) are: superior performance, reduced development
time and greater flexibility. For instance, if a module is removed from the
network and replaced by a new module (which may perform the same task more
efficiently), it should not be necessary to retrain the aggregate network. 

In fact, the term ``modular neural networks'' can be rather vague. In its
most general sense, it denotes networks which consist of simpler
subnetworks (modules). If this point of view is taken to
the extreme, then every neural network can be considered to be modular, in
the sense that it consists of neurons which can be seen as elementary
networks. We believe, however, that it is more profitable to think of a
continuum of modularity, placing complex nets of very simple neurons
at one end of the spectrum, and simple nets of very complex neurons at the
other end. 

We have been working along these lines for several years and have developed
a family of algorithms for time series problems, which we call PREMONN's
(i.e. PREdictive MOdular Neural Networks). Similar algorithms and systems
have also been presented by other authors, under
various names. We will generally use the acronym PREMONN to refer to our
own work and retain ``predictive modular neural networks'' as a generic term. 

This book is divided in four parts. In Part I we present some of our work
which has appeared in various journals such as IEEE Transactions on Neural
Networks, IEEE Transactions on Fuzzy Systems, Neural Computation, Neural
Networks etc. We introduce the family of PREMONN algorithms. These
algorithms are appropriate for online time series classification,
prediction and identification. We discuss these algorithms at an informal
level and we also analyze mathematically their convergence properties. 

In Part II we present applications (developed by ourselves and other
researchers) of PREMONNs to real world problems. In both these parts a
basic assumption is that models are available to describe the input /
output behavior of the sources generating the time series of
interest. This is the known sources assumption. 

In Part III we remove this assumption and deal with time series generated
by completely unknown sources. We present algorithms which operate online
and discover the number of sources involved in the generation of a time
series and develop input/ output models for each source. These source
identification algorithms can be used in conjunction with the
classification and prediction algorithms of Part I. The results of Part III
have not been previously published. 

Finally, in Part IV we briefly review work on modular and multiple models
methods which has appeared in the literature of neural networks,
statistical pattern recognition, econometrics, fuzzy systems, control
theory and statistics. We argue that there is a certain unity of themes and
methods in all these fields and provide a unified interpretation of the
multiple models idea. We hope that this part will prove useful by pointing
out and elucidating similarities between the multiple models
methodologies which have appeared in several disparate fields. 

Indeed, we believe that there is an essential unity in the modular
approach, which cuts across disciplinary boundaries. A good example is the
work reported in this book. While we present our work in ``neural''
language, its essential characteristic is the combination of simple
processing elements which can be combined to form more complex (and
efficient) computational structures. There is nothing exclusively neural
about this theme; it has appeared in all the above mentioned
disciplines and this is why we believe that a detailed literature search
can yield rich dividends in terms of outlook and technique cross
fertilization. 

The main prerequisite for reading this book is the basics of neural network
theory (and a little fuzzy set theory). In Part I, the mathematically
involved sections are relegated to appendices, which may be left for a
second reading, or omitted altogether. The same is true of Part III:
convergence proofs (which are rather involved) are presented in appendices,
while the main argument can be followed quite independently of the
mathematics. Parts II and IV are nonmathematical. We have
also provided an appendix, which contains the basic mathematical concepts
used throughout the book.
___________________________________________________________________
Ath. Kehagias
--Assistant Prof. of Mathematics, American College of Thessaloniki 
--Research Ass., Dept. of Electrical and Computer Eng. Aristotle Univ.,
Thessaloniki, GR54006, GREECE

--email:	 kehagias at egnatia.ee.auth.gr, kehagias at ac.anatolia.edu.gr
--web:	http://skiron.control.ee.auth.gr/~kehagias/index.htm

From kehagias at egnatia.ee.auth.gr  Tue Sep 29 13:46:35 1998
From: kehagias at egnatia.ee.auth.gr (Thanasis Kehagias)
Date: Tue, 29 Sep 1998 10:46:35 -0700
Subject: new papers on modular NN and DATA ALLOCATION
Message-ID: <3.0.5.32.19980929104635.007a0600@egnatia.ee.auth.gr>

NEW PAPERS

The following papers can be obtained from my WEB site. 

-------------------------------------------------------------
1. "A General Convergence Result for Data Allocation in Online Unsupervised
Learning Methods". (With V. Petridis). Poster Presentation in the Second
International Conference on Cognitive and Neural Systems, Boston
University, 1998. 
(http://skiron.control.ee.auth.gr/~kehagias/thn/thn02c05.htm)

2. "Identification of Switching Dynamical Systems Using Multiple Models".
(With V. Petridis).  In Proceedings of CDC 98, 1998. 
(http://skiron.control.ee.auth.gr/~kehagias/thn/thn02c04.htm)

3. "Unsupervised Time Series Segmentation by Predictive Modular Neural
Networks". (With V. Petridis). In Proceedings of ICANN 98, 1998. 
(http://skiron.control.ee.auth.gr/~kehagias/thn/thn02c03.htm)

4. "Data Allocation for Unsupervised Decomposition of Switching Time Series
by Predictive Modular Neural Networks". (With V. Petridis). Accepted for
Publication in the Proccedings of IFAC Conference on Large Scale Systems,
Theory and Applications, Patras, Greece, 1998. 
(http://skiron.control.ee.auth.gr/~kehagias/thn/thn02c02.htm)
-------------------------------------------------------------

All these papers deal with a common problem for which we use the term DATA
ALLOCATION. Briefly, the setup is the following: suppose a collection of
data y(1), y(2), y(3), ... is generated by more than one SOURCES. Namely,
at time t one of the sources is selected (perhaps randomly) and then the
selected source generates the next datum y(t). Now, it is required to build
a model for each source, or estimate some of its parameters and so on. NO A
PRIORI INFORMATION IS AVAILABLE regarding the number, statistical behavior
etc. of the sources. 

If the observed data were split into groups, each group containing the data
generated by one source, it would be relatively easy to train a model (e.g.
a neural network) for each source. But the problem is that the data are
UNLABELLED: no information is available as to which source generated which
datum. So the main problem is DATA ALLOCATION, i.e. the grouping of the data.

The problem is as described above; furthermore we consider an online
version of it. (Hence EM and iterative clustering approaches cannot be
used). The results are of great generality: we provide some sufficient
conditions (which can reasonably be expected to hold for a large class of
algorithms)  which guarantee CORRECT (in a precise sense) data allocation.

Our newly published BOOK (announced in a separate message) also deals with
the same problem, in greater detail (i.e. all the proofs are included).
More info can be found at my WEB site:

http://skiron.control.ee.auth.gr/~kehagias/thn/thn02b01.htm

I will also post a separate message with some additional thoughts and
biblio on the DATA ALLOCATION problem.
___________________________________________________________________
Ath. Kehagias
--Assistant Prof. of Mathematics, American College of Thessaloniki 
--Research Ass., Dept. of Electrical and Computer Eng. Aristotle Univ.,
Thessaloniki, GR54006, GREECE

--email:	 kehagias at egnatia.ee.auth.gr, kehagias at ac.anatolia.edu.gr
--web:	http://skiron.control.ee.auth.gr/~kehagias/index.htm

From nnsp99 at neuro.kuleuven.ac.be  Mon Sep 21 12:17:57 1998
From: nnsp99 at neuro.kuleuven.ac.be (NNSP '99)
Date: Mon, 21 Sep 1998 18:17:57 +0200
Subject: First call for papers for NNSP'99
Message-ID: <36067C35.8161EDFE@neuro.kuleuven.ac.be>

Dear Colleague,

If you would like to be included in our mailing list,
and receive further announcements of the NNSP99 workshop,
let us know at NNSP99 at neuro.kuleuven.ac.be

    Sincerely,

    Marc M. Van Hulle

---

*******************************
**** FIRST CALL FOR PAPERS ****
*******************************

1999 IEEE Workshop on Neural Networks for Signal Processing
August 23-25, 1999, Madison, Wisconsin

NNSP'99 homepage: http://eivind.imm.dtu.dk/nnsp99

Thanks to the sponsorship of IEEE Signal Processing Society the ninth
of a series of IEEE workshops on Neural Networks for Signal Processing
will be held at the Concourse Hotel, Madison, Wisconsin, USA.
The workshop will feature keynote addresses, technical presentations,
panel discussions and special sessions. Papers are solicited for, but
not limited to, the following areas:

Paradigms:
Artificial neural networks, support vector machines, Markov models,
graphical models, dynamical systems, evolutionary computation,
nonlinear signal processing, and wavelets.

Application Areas:
Image/speech/multimedia processing, intelligent human computer
interfaces, intelligent agents, blind source separation, OCR, robotics,
adaptive filtering, communications, sensors, system identification,
issues related to RWC, and other general signal processing and pattern
recognition.

Theories:
Generalization, design algorithms, optimization, parameter estimation,
and network architectures.

Implementations:
Parallel and distributed implementation, hardware design, and other
general implementation technologies.


SPECIAL SESSIONS
The workshop features special sessions on
* Support vector machines
* Intelligent human computer interfaces


PAPER SUBMISSION PROCEDURE
Prospective authors are invited to submit 5 copies of extended
summaries of no more than 6 pages.  The top of the first page of the
summary should include a title, authors' names, affiliations, address,
telephone and fax numbers and email address.
Camera-ready full papers of accepted proposals will be published in
a hard-bound volume by IEEE and distributed at the workshop.

Please send paper submissions to:
Jan Larsen
NNSP'99, Department of Mathematical Modelling, Building 321
Technical University of Denmark
DK-2800 Lyngby, Denmark


SCHEDULE
Submission of extended summary:            February 1, 1999
Notification of acceptance:                March 31, 1999
Submission of photo-ready accepted paper:  April 29, 1999
Advanced registration, before:             June 30, 1999


ORGANIZATION

General Chair
Yu Hen HU
University of Wisonsin-Madison
email: hu at ece.wisc.edu

Finance Chair
Tulay ADALI
University of Maryland
Baltimore County
email: adali at umbc.edu

Proceedings Chair
Elizabeth J. WILSON
Raytheon Co.
email: bwilson at ed.ray.com

Proceedings Co-Chair
Scott C. DOUGLAS
Southern Methodist University
email: douglas at seas.smu.edu

Publicity Chair
Marc van HULLE
Katholieke Universiteit Leuven
email: marc at neuro.kuleuven.ac.be

Program Chair
Jan LARSEN
Technical University of Denmark
email: jl at imm.dtu.dk

Program Committee
Tulay ADALI
Amir ASSADI
Andrew BACK
Herve BOURLARD
Andrzej CICHOCKI
Anthony G. CONSTANTINIDES
Bert DE VRIES
Scott C. DOUGLAS
Kevin R. FARRELL
Hsin-Chia FU
Ling GUAN
Jenq-Neng HWANG
Shigeru KATAGIRI
Fa-Long LUO
David MILLER
Nelson MORGAN
Klaus-Robert MULLER
Mahesan NIRANJAN
Dragan OBRADOVIC
Volker TRESP
Marc VAN HULLE

From tp at ai.mit.edu  Tue Sep 29 23:36:39 1998
From: tp at ai.mit.edu (Tomaso Poggio)
Date: Tue, 29 Sep 1998 23:36:39 -0400
Subject: Computational Neuroscience Position (Note extension of
  deadline for application)
Message-ID: <3.0.5.32.19980929233639.00b78aa0@ai.mit.edu>


                MASSACHUSETTS INSTITUTE OF TECHNOLOGY
                DEPARTMENT OF BRAIN SCIENCES

The MIT Department of Brain  Sciences anticipates making another
tenure-track appointment in computational brain and cognitive science at
the Assistant Professor level.  Candidates should have a strong
mathematical background and an active research interest in the mathematical
modeling of specific biophysical, neural or cognitive phenomena.
Individuals whose research focuses on learning and memory at the level of
neurons and networks of neurons are especially encouraged to apply.
Responsibilities include graduate and undergraduate teaching and research
supervision.  Applications should include a brief cover letter stating the
candidate's research and teaching interests, a vita, three letters of
recommendation and representative reprints.  Qualified individuals should
send their dossiers by NOVEMBER 21, 1998 to:
               Chair, Faculty Search Committee/Computational Neuroscience
               Department of Brain & Cognitive Sciences, E25-406
               MIT                77 Massachusetts Avenue
Cambridge, MA 02139-4307
Previous applicants (last year) need not resubmit their dossiers.

MIT is an Affirmative Action/Equal Opportunity Employer.  Qualified women
and minority candidates are encouraged to apply.

Tomaso Poggio
Uncas and Helen Whitaker Professor
Brain Sciences Department and A.I. Lab
M.I.T., E25-218,
45 Carleton St
Cambridge, MA 02142

E-mail: tp at ai.mit.edu
Web: <http://www.ai.mit.edu/people/poggio/>
Phone: 617-253-5230
Fax: 617-253-2964