Reinforcement Learning Paper

[guzman@gluttony.cs.umass.edu]

Reinforcement Learning in Partially Markovian Decision Processes
   
            submitted to NIPS*95 (8 pages)


FTP-HOST: ftp.cs.umass.edu
FTP-FILENAME: /pub/anw/pub/guzman/guzman-lara.PMDP.ps.Z

ABSTRACT

	We define a subclass of Partially Observable Markovian 
Decision Processes (POMDPs), which we call {\em Partially 
Markovian Decision Processes} (PMDPs), and propose a novel 
approach to solve this kind of problem. In contrast to traditional 
methods for POMDPs, our method does not involve estimation of the state 
of an underlying Markovian problem; its goal is to find an optimal 
observation-based policy (an action-selection rule that uses only 
the information immediately available to the agent). We 
show that solving this non-Markovian problem is equivalent to 
solving multiple Markovian Decision Processes (MDPs). 
We argue that this approach opens new possibilities 
for distributed systems, and we support this claim with 
some preliminary results 
where the use of an observation-based policy yielded a 
good solution in a complex stochastic environment. 



  Sorry, no hard copies

 

  Sergio Guzman-Lara
  Computer Science Department
  LGRC, University of Massachusetts 
  Amherst, MA 01003
  guzman at cs.umass.edu