No subject
shawn@helmholtz.sdsc.edu
shawn at helmholtz.sdsc.edu
Sat Aug 31 15:30:16 EDT 1991
Several months ago I asked about connectionist efforts in modeling
chemotaxis and animal orientation. Response was quite good. Here is
a compilation of what I received. Thanks to all those who kindly
responded.
OUTGOING QUERY:
"
I am a neurobiologist interested in training neural networks to
perform chemotaxis, and other feats of simple animal navigation. I'd
be very interested to know what has been done by connectionists in
this area. The only things I have found so far are: Mozer and Bachrach
(1990) Discovering the Structure of a Reacative nvironment by
Exploration, and Nolfi et al. (1990) Learning and Evolution in Neural
Networks
Many thanks,
Shawn Lockery
CNL
Salk Institute
Box 85800
San Diego, CA 92186-5800
(619) 453-4100 x527
shawn at helmholtz.sdsc.edu
"
_____________________________________________________________________
THE REPLIES
_____________________________________________________________________
From: mlittman at breeze.bellcore.com (Michael L. Littman)
Hi,
Dave Ackley and Michael Littman (me) did some work where we used a
combination of natural selection and reinforcement learning to train
simulated creatures to survive in a simulated environment.
%A D. H. Ackley
%A M. L. Littman
%T Interactions between learning and evolution
%B Artificial Life 2.
%I Addison-Wesley
%D 1990
%E Langton, Chris
%O (in press)
%A D. H. Ackley
%A M. S. Littman
%T Learning from natural selection in an artificial environment
%B Proceedings of the International Joint Conference on Neural Networks
%C Washington, D.C.
%D January 1990
There is also some neat work by Stewart Wilson as well as Rich Sutton
and friends. I'm not sure exactly what sort of things you are looking
for so I'm having trouble knowing exactly where to point you. If you
describe the problem you have in mind I might be able to indicate some
other relevant work.
-Michael
----------------------------------------------------------------------------
From: David Cliff <davec at cogs.sussex.ac.uk>
Re. your request for chemotaxis and navigation: do you know about Randy Beer's
work on a simulated cockroach? He did studies of locomotion control using
hardwired network models (ie not much training involved) but the simulated bug
performed simple navigation tasks. I think it had chemoreceptors in
it's antennae, so I think there was some chemotaxis involved. He's written a
book: R.D.Beer "Intelligence as Adaptive Behavior: An Experiment in
Computational Neuroethology" Academic Press, 1990.
davec at cogs.susx.ac.uk
COGS 5C17
School of Cognitive and Computing Sciences
University of Sussex
Brighton BN1 9QH
England UK
----------------------------------------------------------------------------
From: Ronald L Chrisley <chrisley at ws.oxford.ac.uk>
You might take a look at my modest efforts in a paper in the Proc. of
the 1990 CMSS. Not biologically motivated at all, though.
Ron Chrisley
----------------------------------------------------------------------------
From: beer at cthulhu.ces.cwru.edu (Randy Beer)
Hello Shawn!
I'm not sure that this is what you're looking for, but as I mentioned to
you at Neurosciences, we've been using genetic algorithms to evolve
dynamical NNs. One of our experiments involved gradient following. A
simple circular "animal" with two chemosensors and two motors was placed
in an environment with a patch of food emitting an odor whose intensity
decreased as the inverse square of the distance from the patch's center.
The animal's behavior was controlled by a bilaterally symmetric, six node,
fully interconnected dynamical NN (2 sensory neurons, 2 motor neurons, and
two interneurons). The time constants (3, due to bilateral symmetry) and
the weights (18) were encoded on a bit string genome. The performance
function was simply the average distance from the food patch that the
animal was found after a fixed amount of time for a variety of initial
starting positions. We evolved several different solutions to this problem,
including one less than optimal but interesting "wiggler". This animal
oscillated from side to side until it happened to come near the food patch,
then the relatively strong signal from the nearby food damped out the
oscillations and it turned toward the food. Most of the other solutions
simply compared the signals in each chemosensor and turned toward the
stronger side, as you would expect. These more obvious solutions still
varied in the overall gain of their response. Low gain solutions performed
very well near the food patch, but had a great deal of trouble finding it
if they started too far away. High gain solutions rarely had any trouble
finding the food patch, but their behavior at the patch was often more
erratic and sometimes they would fly back off of it.
Randy
----------------------------------------------------------------------------
From: wey at psyche.mit.edu (Wey Fun)
You may look into Christ Watkins, ANdrew Barto & Richard Barto's work on
TD algo. My colleague at Univ of Edinburgh, Peter Dayan, ahs also done a lot
of work on the simulation of rats swimming in milky water and finding a fast
shortest route after trials to a platform. His email address is :
dayan at cns.ed.ac.uk
Wey
----------------------------------------------------------------------------
From: Jordan B Pollack <pollack at cis.ohio-state.edu>
Im pretty sure that Andy Barto at cs.umass.edu and his students worked on
"A-RP" reinforcement learning in a little creature navigating through
an environment of smell gradients. This is the only reference
in my list:
%A A. G. Barto
%A C. W. Anderson
%A R. S. Sutton
%T Synthesis of Nonlinear Control Surfaces by a layered Associative Search Network
%J Biological Cybernetics
%V 43
%P 175-185
%D 1982
%K R12
jordan
----------------------------------------------------------------------------
From: Peter Dayan <dayan at cns.edinburgh.ac.uk>
[This is in response to your direct note to me - you should also have
received a reply to your connectionists at cmu posting from me.]
In that, I neglected to mention:
Barto, AG (1989). From chemotaxis to cooperativity: Abstract exercises
in neuronal learning strategies. In R Durbin, C Miall \& G Mitchison,
editors, {\it The Computing Neuron.\/} Wokingham, England:
Addison-Wesley.
and
Watkins, CJCH (1989). {\it Learning from Delayed Rewards.\/} PhD
Thesis. University of Cambridge, England.
which is one of my `source' texts, and contains interesting
discussions of TD learning from the viewpoint of dynamical
programming.
Regards,
Peter
Randy
-----------------------------------------------------------------------------
From: "Vijaykumar Gullapalli (413) 545-1596" <VIJAYKUMAR at cs.umass.EDU>
Andy Barto wrote a nice paper discussing learning issues that might be of
interest. It appeared as a tech report and as a book chapter. The ref is
@techreport{Barto-88a,
author="Barto, A. G.",
title="From Chemotaxis to Cooperativity: {A}bstract
Exercises in Neuronal Learning Strategies",
institution="University of Massachusetts",
address="Amherst, MA", number="88-65", year=1988,
note="To appear in {\it The Computing Neurone},
R. Durbin and R. Maill and G. Mitchison (eds.),
Addison-Wesley"}.
A copy of the tech report can be obtained by writing to Connie Smith
at smith at cs.umass.edu.
Vijay
__________________________________________________________________________________
From: nin at cns.brown.edu (Nathan Intrator)
Could you give me more information on the task, is the input binary
and is the dimensionality of the input large. I have an unsupervised
network that is supposed to discover structure in HIGH DIMENSIONAL
spaces in an unsupervised way which may be of interest to you.
---------------------------------------------------------------------
From: meyer%frulm63.bitnet at Sds.sdsc.edu (Jean-Arcady MEYER)
I'm interested in the simulation of adaptive behavior and I have written a
Technical Report on the subject, in which I think you could find several
interesting references. In particular, various works have been made in the
spirit of Nolfi et al. I'm sending this report to you today.
Let me add that I have organized - together with Stewart Wilson - the
conference SAB90 (Simulation of adaptive behavior: from animals to animats)
which has been held in Paris in September 1990. The corresponding proceedings
are about to be published by The MIT Press/Bradford Books. I'm also sending
you a booklet of the papers'summaries.
Finally, I don't know the paper from Mozer and Bachrach you are mentioning
in your mail. Could you be kind enough to send me its reference?
Hope this will be helpful to you.
Jean-Arcady Meyer
Groupe de BioInformatique
URA686. Ecole Normale Superieure
46 rue d'Ulm
75230 PARIS Cedex05
FRANCE
---------------------------------------------------------------------
From: barto at envy.cs.umass.edu
We have done a number of papers over the years that
relate to chemotaxis. Chemotaxic behavior of single
cells has inspired a lot of our thinking about
learning. Probably the most relevant are:
Barto, From Chemotaxis to Cooperativity, in The Computing
Neuron, edited by Durbin, Miall, Mitchison. Addison Wesley, 1989
Barto and Sutton, Landmark Learning: An Illustration of
Associative Search, Biol. Cyb. 42, 1981
Andy Barto
Dept. of Computer and Information Science
University of Massachusetts
Amherst MA 01003
---------------------------------------------------------------------
From: dmpierce at cs.utexas.edu
I had a paper myself at a recent Paris conference (September 1990) which
might be relevant to you:
Pierce, D.M., \& Kuipers, B.J. (1991). Learning hill-climbing
functions as a strategy for generating behaviors in a mobile robot. {\em
From Animals to Animats: Proceedings of The First International Conference
on Simulation of Adaptive Behavior}, J.-A. Meyer \& S.W. Wilson, eds.,
Cambridge, MA: The MIT Press/Bradford Books, pp.~327-336.
This is also available as University of Texas AI Lab. Tech. Report
AI90-137. Here is the abstract:
We consider the problem of learning, in an unknown environment, behaviors
(i.e., sequences of actions) which can be taken to achieve a given goal.
This general problem involves a learning agent interacting with a
reactive environment: the agent produces actions that affect the
environment and in turn receives sensory feedback from the environment.
The agent must learn, through experimentation, behaviors that
consistently achieve the goal.
In this paper, we consider the particular problem of a mobile robot in a
spatial two-dimensional world whose goal is to find a target location
which contains a ``food'' source.
The robot has access to incomplete information about the state of the
world via a set of senses and is able to detect when it has achieved the
goal. Its task is to learn to use its motor apparatus to reliably move
to the food. The catch is that the robot does not know a priori what its
sensors mean, nor what effects its motor apparatus has on the world.
We propose a method by which the robot may analyze its sensory
information in order to derive (when possible) a function defined in
terms of the sensory data which is maximized at the food and which is
suitable for hill-climbing. Given this function, the robot solves its
problem by learning a behavior that maximizes the function thereby
resulting in motion to the food.
-Dave Pierce
------------------------------------------------------------------------
From: KDBG100%bgunve.bgu.ac.il at BITNET.CC.CMU.EDU
I may not have understood the specifics of what you require, but about
spatial environments, there is a paper in PDP II 1986 which I suppose
you must know about. So -- what is the issue you are pursuing?
David leiser, Jerusalem
-------------------------------------------------------------------------
From: Steve Hampson <hampson at ics.uci.edu>
I was just going over old mail and found your request for refs on
animal navigation. I was planning on replying, but probably never did.
My book "Connecitonistic Problem Solving" Birkhauser, Boston, is
an attempt at general problem solving, but almost all of the examples
are maze-like. Several approaches are discussed and implemented.
Sorry for the delay.
Steven Hampson
ICS Dept. UCI.
------------------------------------------------------------------------
More information about the Connectionists
mailing list