Paper on RL, exploration, hidden state

[Andrew McCallum]

The following paper on reinforcement learning, hidden state and
exploration is available by FTP.  Comments and suggestions are
welcome.

 "Efficient Exploration in Reinforcement Learning with Hidden State"

		       Andrew Kachites McCallum

			(submitted to NIPS)

			       Abstract

  Undoubtedly, efficient exploration is crucial for the success of a
  learning agent.  Previous approaches to directed exploration in
  reinforcement learning exclusively address exploration in Markovian
  domains, i.e. domains in which the state of the environment is fully
  observable.  If the environment is only partially observable, they
  cease to work because exploration statistics are confounded between
  aliased world states.

  This paper presents Fringe Exploration, a technique for efficient
  exploration in partially observable domains.  The key idea,
  (applicable to many exploration techniques), is to keep statistics
  in the space of possible short-term memories, instead of in the
  agent's current state space.  Experimental results in a partially
  observable maze and in a difficult driving task with visual routines
  show dramatic performance improvements.
 

Retrieval information:

FTP-host:      ftp.cs.rochester.edu
FTP-pathname:  /pub/papers/robotics/96.mccallum-nips.ps.gz
URL: ftp://ftp.cs.rochester.edu/pub/papers/robotics/96.mccallum-nips.ps.gz