paper available: incremental self-improvement

[Juergen Schmidhuber]

          ON LEARNING HOW TO LEARN LEARNING STRATEGIES

              Technical Report FKI-198-94 (20 pages)               
                       Juergen Schmidhuber 
                    Fakultaet fuer Informatik 
                 Technische Universitaet Muenchen  
                     80290 Muenchen, Germany 

                         November 24, 1994
	             Revised January 31, 1995

This paper introduces the ``incremental self-improvement paradigm''.
Unlike previous methods,  incremental self-improvement  encourages a
reinforcement learning  system to improve  the way it learns, and to
improve the way  it improves the way it learns,  without significant 
theoretical limitations -- the system  is able to ``shift its induc-
tive bias'' in  a universal way.  Its major features are:  (1) There 
is no  explicit difference  between ``learning'', ``meta-learning'', 
and other kinds of  information  processing.  Using a Turing machine 
equivalent  programming  language,  the system  itself  occasionally 
executes  self-delimiting,  initially  highly  random ``self-modifi-
cation programs'' which modify  the context-dependent  probabilities 
of  future  programs (including future  self-modification programs).
(2) The system keeps  only those probability  modifications computed 
by ``useful'' self-modification  programs:  those which  bring about 
more payoff per time  than all previous  self-modification programs.
(3) The computation  of payoff per time  takes into account  all the 
computation time required for learning -- the entire system  life is 
considered: boundaries between learning trials are ignored (if there
are any). A particular implementation based on the novel paradigm is 
presented.  It is  designed to  exploit  what  conventional  digital 
machines are good at: fast storage addressing, arithmetic operations  
etc. Experiments illustrate the system's mode of operation.


-------------------------------------------------------------------

FTP-host: flop.informatik.tu-muenchen.de (131.159.8.35)
FTP-filename: /pub/fki/fki-198-94.ps.gz (use gunzip to uncompress)
Alternatively, retrieve the paper from my home page:

http://papa.informatik.tu-muenchen.de/mitarbeiter/schmidhu.html

If you don't have gzip/gunzip,  I can mail  you an uncompressed
postscript version (as a  last resort). There will be a future 
revised version of the tech report. Comments are welcome.
    
Juergen Schmidhuber