PhD thesis: Continual Learning in Reinforcement Environments.

[Mark Ring]

FTP-host:     ftp.gmd.de
FTP-filename: /Learning/rl/papers/ring.thesis.ps.Z
URL:          ftp://ftp.gmd.de/Learning/rl/papers/ring.thesis.ps.Z

  also

URL:          http://borneo.gmd.de:80/~ring/Diss
              http://www.cs.utexas.edu/users/ring/Diss

138 pages total, 624 kbytes compressed postscript.



My dissertation from last year is now available publicly in book format.
It can be retrieved via ftp, is accessible in sections by WWW, or can be
ordered in any book store from Oldenbourg Verlag (publishers) with the
following ISBN number: ISBN 3-486-23603-2.

    ----------------------------------------------------------------------

Title:        Continual Learning in Reinforcement Environments

                                 August, 1994

Abstract:

*Continual learning* is the constant development of complex behaviors with no
final end in mind.  It is the process of learning ever more complicated skills
by building on those skills already developed.  In order for learning at one
stage of development to serve as the foundation for later learning, a
continual-learning agent should learn hierarchically.  CHILD, an agent capable
of Continual, Hierarchical, Incremental Learning and Development is proposed,
described, tested, and evaluated in this dissertation.  CHILD accumulates
useful behaviors in reinforcement environments by using the Temporal
Transition Hierarchies learning algorithm, also derived in the dissertation.
This constructive algorithm generates a hierarchical, higher-order neural
network that can be used for predicting context-dependent temporal sequences
and can learn sequential-task benchmarks more than two orders of magnitude
faster than competing neural-network systems.  Consequently, CHILD can quickly
solve complicated non-Markovian reinforcement-learning tasks and can then
transfer its skills to similar but even more complicated tasks, learning these
faster still.  This continual-learning approach is made possible by the unique
properties of Temporal Transition Hierarchies, which allow existing skills to
be amended and augmented in precisely the same way that they were constructed
in the first place.

Contents: 

   Leading pages (pp. iv - xiv)
   
   Chapters:

   1. Introduction (pp. 1 - 7)
   2. Robotics Environments and Learning Tasks (pp. 8 - 16).
   3. Neural-Network Learning (pp. 17 - 24).
   4. Solving Temporal Problems with Neural Networks (pp. 25 - 33).
   5. Reinforcement Learning (pp. 34 - 44).
   6. The Automatic Construction of Sensorimotor Hierarchies (pp. 45 - 71).
      6.1 Behavior Hierarchies (pp. 45 - 52).
      6.2 Temporal Transition Hierarchies (pp. 52 - 69).
      6.3 Conclusions (pp. 70 - 71).
   7. Simulations (pp. 72 - 95).
      7.1  Description of Simulation System (pp. 72 - 73).
      7.2  Supervised-Learning Tasks (pp. 73 - 82).
      7.3  Continual Learning Results (pp. 82 - 95).
   8. Synopsis, Discussion, and Conclusions (pp. 96 - 107).
   
   Appendices A-E (pp. 108 - 117).
   Bibliography (pp. 118 - 127).



 ----------
Mark Ring
Research Group for Adaptive Systems
GMD - German National Research Center for Information Technology
Schloss Birlinghoven
D-53 754 Sankt Augustin
Germany

Mark.Ring at gmd.de
http://borneo.gmd.de:80/~ring