continual learning etc.

[Juergen Schmidhuber]

Concerning the recent messages on "transfer learning", "incremental 
learning" etc: Mark Ring has been working on this subject for many 
years now, specifically on bottom-up, hierarchical behavior learning 
and skill transfer in reinforcement-learning agents (1991, 1993c), 
and with time-dependent context-sensitive neural networks (1993a, 
1993b) that keep adding new units in order to learn longer and more 
complicated sequences.  In his dissertation on "continual learning", 
he described a hierarchical mechanism for learning non-Markovian 
reinforcement tasks where hierarchy construction was done bottom-up 
as learning progressed.  He tested it on "continual learning" tasks, 
where the behaviors his learning agent acquired for simple tasks were 
used for learning more difficult tasks with much less effort (skill 
transfer).  Even after learning much more complicated tasks the agent 
could still generally solve the simpler ones (avoiding catastrophic 
forgetting).



Juergen Schmidhuber

Fakultaet fuer Informatik 
Technische Universitaet Muenchen 
80290  Muenchen, Germany 


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++=

References:

@InProceedings{Ring:1991,
  author = 	"Ring, Mark B.",
  title = 	"Incremental Development of Complex Behaviors through
		 Automatic Construction of Sensory-motor Hierarchies",
  booktitle = 	"Machine Learning: Proceedings of the Eighth
		 International Workshop (ML91)",
  year = 	1991,
  editor = 	"Birnbaum, Lawrence A. and Collins, Gregg C.",
  pages = 	"343--347",
  publisher = 	"Morgan Kaufmann Publishers",
  month = 	"June",
}
@InProceedings{Ring:1993a,
  author = 	"Ring, Mark B.",
  title = 	"Learning Sequential Tasks by Incrementally Adding
		 Higher Orders", 
  booktitle = 	"Advances in Neural Information Processing Systems 5",
  year = 	1993,
  editor = 	"Giles, C. L. and Hanson, S. J. and Cowan, J. D.",
  pages = 	"115--122",
  publisher = 	"Morgan Kaufmann Publishers",
  address = 	"San Mateo, California",
}
@InProceedings{Ring:1993b,
  author = 	"Ring, Mark B.",
  title = 	"Two Methods for Hierarchy Learning in Reinforcement
		 Environments", 
  booktitle = 	"From Animals to Animats 2: Proceedings of the Second
		 International Conference on Simulation of Adaptive Behavior",
  year = 	1993,
  editor = 	"Meyer, J. A. and Roitblat, H. and Wilson, S.",
  pages = 	"148--155",
  publisher = 	"MIT Press",
}
@TechReport{Ring:1993c,
  author = 	"Ring, Mark B.",
  title = 	"Sequence Learning with Incremental Higher-Order
		 Neural Networks",
  institution = 	"Artificial Intelligence Laboratory,
		 University of Texas at Austin",
  year = 	1993,
  number = 	"AI 93--193",
  month = 	"January",
}
@PhDThesis{Ring:1994,
  author = 	"Ring, Mark B.",
  title = 	"Continual Learning in Reinforcement Environments",
  school = 	"University of Texas at Austin",
  year = 	1994,
  address = 	"Austin, Texas  78712",
  month = 	"August",
}