task consolidation

[Lisa Meeden]

Danny Silver asked whether others had tried methods of task
consolidation at a meta level.  In my dissertation I used an
Elman-style recurrent network trained with reinforcement learning to
control a simple robot with a few goals.  At the time of goal
achievement, the hidden layer potentially reflects a consolidated
history of the perceptual states encountered during the process of
solving the task.  I argued that these hidden layer activations could
serve as a sort of plan for achieving the goal and called them
protoplans. 

To investigate the efficacy of protoplans, a transfer of learning
experiment was done.  The protoplans learned in one controller network
were saved in an associative memory and used to guide a second
controller network as it learned the same task from scratch.  The
associative memory mapped the precursor sensor states of a protoplan
to the protoplan itself.  Controllers trained with protoplans instead
of goals as input converged more quickly on good solutions than the
original controllers trained with goals.  Protoplans were able to
guide the robot's behavior by marking the important moments in the
interaction with the environment when a switch in behavior should
occur.  This kind of timing information was indirect--no specific
action was indicated--but knowing when to change from a particular
strategy to a new one can be very important information.

For more details on these experiments see chapter 5 of my thesis which
is available at:

FTP-host: archive.cis.ohio-state.edu
FTP-filename: /pub/neuroprose/Thesis/meeden.thesis.ps.Z

-- 
Lisa Meeden
Computer Science Program
Swarthmore College
500 College Ave
Swarthmore, PA 19081

(610) 328-8565
meeden at cs.swarthmore.edu