Connectionists: paper available on efficient stochastic control

[Bert Kappen]

Dear all,

I would like to announce  the following paper, that can be downloaded
from arxiv.org/physics/0505066

Path integrals and symmetry breaking for optimal control theory
H.J. Kappen

This paper considers linear-quadratic control of a non-linear dynamical
system subject to arbitrary cost.  I show that for this class of
stochastic control problems the non-linear Hamilton-Jacobi-Bellman
equation can be transformed into a linear equation. The transformation is
similar to the transformation used to relate the classical Hamilton-Jacobi
equation to the Schr\"odinger equation.  As a result of the linearity,
the usual backward computation can be replaced by a forward diffusion
process, that can be computed by stochastic integration or by the
evaluation of a path integral. It is shown, how in the deterministic
limit the PMP formalism is recovered. The significance of the path
integral approach is that it forms the basis for a number of efficient
computational methods, such as MC sampling, the Laplace approximation
and the variational approximation. We show the effectiveness of the
first two methods in number of examples.  Examples are given that show
the qualitative difference between stochastic and deterministic control
and the occurrence of symmetry breaking as a function of the noise.



Bert Kappen             SNN           Radboud University Nijmegen
URL: www.snn.kun.nl/~bert             The Netherlands			
tel: +31 24 3614241                   fax: +31 24 3541435
B.Kappen at science.ru.nl