Transfer/Interference in control problems...

[Satinder Singh]

	This is another note on the topic of transfer and interference
across problems. 

	I have investigated this issue *a bit* in the reinforcement
learning setting of an agent required to solve a SET of hierarchically
structured control problems in the *same* environment. Reinforcement
learning (RL) algorithms solve control problems by using control
experience to update utility functions. With experience, the utility
function improves and therefore so does the resulting ``greedy''
decision strategy.

	I studied the case of compositionally structured control
tasks, where complex tasks are sequences of simpler tasks. I showed
how a modular RL architecture (that is based on Jacobs, Jordan, Nowlan
and Hinton's mixture of experts architecture) is able to reuse utility
functions learned for simple tasks to quickly construct utility
functions for more complex tasks in the hierarchy. The composition of
a complex task is not available to the agent, but has to be learned. 
The use of ``shaping'' to encourage transfer is also illustrated. 

	Anonymous ftp instructions for the above work follow -- the
filename is singh-MLJ-1.ps.Z (another file of possible interest is
singh-ML92.ps.Z).

	Several other RL researchers have since worked on this topic.
See Dayan and Hinton's paper on ``Feudal Reinforcement Learning'' in
NIPS-5 for more recent work and uptodate references.
================================================================

unix> ftp envy.cs.umass.edu

Name: anonymous
Password: [your ID]
ftp> cd pub/singh
ftp> binary
ftp> get <filename>.ps.Z
ftp> bye

unix> uncompress <filename>.ps.Z
unix> [your command to print PostScript file] <filename>.ps