Preprint: building sensory-motor hierarchies

[Mark Ring]

Recently there's been some interest on this mailing list regarding
neural net hierarchies for sequence "chunking".  I've placed a
relevant paper in the Neuroprose Archive for public ftp.  This is a
(very slightly extended) copy of a paper to be published in the
Proceedings of the Eighth International Workshop on Machine Learning.

The paper summarizes the results to date of work begun a year and a
half ago to create a system that automatically and incrementally
constructs hierarchies of behaviors in neural nets.  The purpose of
the system is to develop continuously through the encapsulation, or
"chunking," of learned behaviors.

----------------------------------------------------------------------
				   
    INCREMENTAL DEVELOPMENT OF COMPLEX BEHAVIORS THROUGH AUTOMATIC
	      CONSTRUCTION OF SENSORY-MOTOR HIERARCHIES
				   
			      Mark Ring
		    University of Texas at Austin

      This paper addresses the issue of continual, incremental
      development of behaviors in reactive agents. The reactive
      agents are neural-network based and use reinforcement
      learning techniques.
      
      A continually developing system is one that is constantly
      capable of extending its repertoire of behaviors. An agent
      increases its repertoire of behaviors in order to increase
      its performance in and understanding of its environment.
      Continual development requires an unlimited growth
      potential; that is, it requires a system that can
      constantly augment current behaviors with new behaviors,
      perhaps using the current ones as a foundation for those
      that come next.  It also requires a process for organizing
      behaviors in meaningful ways and a method for assigning
      credit properly to sequences of behaviors, where each
      behavior may itself be an arbitrarily long sequence.
      
      The solution proposed here is hierarchical and bottom up.
      I introduce a new kind of neuron (termed a ``bion''),
      whose characteristics permit it to be automatically
      constructed into sensory-motor hierarchies as determined
      by experience.  The bion is being developed to resolve the
      problems of incremental growth, temporal history
      limitation, network organization, and credit assignment
      among component behaviors.


A longer, more detailed paper will be announced shortly.

----------------------------------------------------------------------
Instructions to retrieve paper by ftp, (no hard copies available at
this time):

         % ftp cheops.cis.ohio-state.edu (or 128.146.8.62)
         Name: anonymous
         Password: neuron
         ftp> cd pub/neuroprose
         ftp> binary
         ftp> get ring.ml91.ps.Z
         ftp> bye
         % uncompress ring.ml91.ps.Z
         % lpr -P(your_postscript_printer) ring.ml91.ps.Z

----------------------------------------------------------------------

DO NOT "reply" DIRECTLY TO THIS MESSAGE!

If you have any questions or difficulties, please send e-mail to:
ring at cs.utexas.edu.

or send mail to:

Mark Ring				    
Department of Computer Sciences
Taylor 2.124
University of Texas at Austin
Austin, TX 78712