ON-LINE LEARNING IN NEURAL NETWORKS

[saadd@helios.aston.ac.uk]

The following book is available from Cambridge University Press; see

http://www.cup.cam.ac.uk/Scripts/webbook.asp?isbn=0521652634

ON-LINE LEARNING IN NEURAL NETWORKS

David Saad, Ed.


Series: Publications of the Newton Institute

ISBN: 0 521 65263 4


DESCRIPTION

On-line learning is one of the most powerful and
commonly used techniques for training large layered
networks, and has been used successfully in many
real-world applications. Traditional analytical
methods have been recently complemented by ones
from statistical physics and Bayesian statistics.
This powerful combination of analytical methods
provides more insight and deeper understanding of
existing algorithms, and leads to novel and
principled proposals for their improvement. This
book presents a coherent picture of the
state-of-the-art in the theoretical analysis of
on-line learning. An introduction relates the subject
to other developments in neural networks and
explains the overall picture. Surveys by leading
experts in the field combine new and established
material and enable non-experts to learn more
about the techniques and methods used. This book,
the first in the area, provides a comprehensive view
of the subject and will be welcomed by mathematicians,
scientists and engineers, whether in industry or
academia.



CONTENTS

Foreward - Christopher M. Bishop

1.  Introduction
    David Saad
2.  On-line Learning and Stochastic Approximations
    Leon Bottou
3.  Exact and Perturbative Solutions for the Ensemble Dynamics
    Todd K. Leen
4.  A Statistical Study on On-line Learning
    Noboru Murata
5.  On-line Learning in Switching and Drifting Environments
    with Application to Blind Source Separation
    Klaus-Robert Mueller, Andreas Ziehe, Noboru Murata and
    Shun-ichi Amari
6.  Parameter Adaptation in Stochastic Optimization
    Luis B. Almeida, Thibault Langlois, Jose D. Amaral 
    and Alexander Plakhov
7.  Optimal On-line Learning in Multilayer Neural Networks
    David Saad and Magnus Rattray
8.  Universal Asymptotics in Committe Machines with Tree Architecture  
    Mauro Copelli and Nestor Caticha
9.  Incorporating Curvature Information into On-line Learning
    Magnus Rattray and David Saad
10. Annealed On-line Learning in Multilayer Neural Networks
    Siegfried Boes and Shun-ichi Amari
11. On-line Learning of Prototypes and Principal Components
    Michael Biehl, Ansgar Freking, Matthias Hoelzer, Georg Reents
    and  Enno Schloesser
12. On-line Learning whith Time-Correlated Examples
    Tom Heskes and Wim Wiegerinck
13. On-line Learning from Finite Training Sets
    David Barber and Peter Sollich
14. Dynamics of Supervised Learning with Restricted Training Sets 
    Anthony C.C. Coolen and David Saad
15. On-line Learning of a Decision Boundary with and without Queries 
    Yoshiyuki Kabashima and Shigeru Shinomoto
16. A Bayesian Approach to On-line Learning
    Manfred Opper
17. Optimal perceptron learning: an on-line Bayesian approach
    Sara A. Solla and Ole Winther

See:
http://www.ncrg.aston.ac.uk/books/OLNN/index.html