adaptive natural gradient papers

[Shunichi Amari]

      Two papers concerning natural gradient on-line learning are
                           
                            available!


     Natural gradient learning has attracted much attention because of
its excellent dynamical behaviors.  When it is applied to multilayer
perceptrons, its superior properties have been proved by statistical-
physical methods.  It is not only locally Fisher efficient but avoids
plateaus or quickly gets rid of them.  Although theoretically good,
it is believed that its practical implementation is difficult.  This
is because calculation of the Fisher information matrix and its
inversion are very difficult.  In order to avoid this difficulty,
we have developed an adaptive method of directly calculating the
inverse of the Fisher information.

     The natural gradient method works surprisingly well with this
adaptive estimate of the inverse.  The first paper proposes the
method itself, which has been accepted for publication in Neural
Computation.  The second paper generalizes the idea to be applicable
to a wider class of network models and loss functions.  This has been
submitted to Neural Networks.

     S.Amari, H.Park and K.Fukumizu,  Adaptive method of realizing
         natural gradient learning for multilayer perceptrons,

     H.Park, S.Amari, and K.Fukumizu,  Adaptive natural gradient
         learning algorithms for various stochastic models.

You can copy the papers from

     http://www.bsis.brain.riken.go.jp/

Shun-ichi Amari
　
Wako-shi, Hirosawa 2-1, Saitama 351-0198, Japan
RIKEN Brain Science Institute
Director of Brain-Style Information Systems Research Group
Laboratory for Information Synthesis, Head

tel: +81-(0)48-467-9669
fax: +81-(0)48-467-9687
e-mail: amari at brain.riken.go.jp
home page: http://www.bsis.brain.riken.go.jp/