Dear researchers,

[P.Refenes@cs.ucl.ac.uk]

YOU MAY BE INTERESTED IN THIS:

          CONSTRUCTIVE LEARNING by SPECIALISATION
 
                A. N. REFENES & S. VITHLANI
               Department of Computer Science
                 University College London
                    Gower Street London
                        WC1 6BT, UK
ABSTRACT


This  paper  describes  and  evaluates  a   procedure   for
constructing  and  training  multi-layered  perceptrons  by
adding units to the network  dynamically  during  training.
The  ultimate aim of the learning procedure is to construct
an architecture which is sufficiently large  to  learn  the
problem but necessarily small to generalise well. Units are
added as they are needed.  By showing that the newly  added
unit  makes fewer mistakes than before, and by training the
unit  not  to  disturb  the  earlier   dynamics,   eventual
convergence  to zero-errors is guaranteed. Other techniques
operate in the opposite direction by  pruning  the  network
and  removing "redundant" connections.  Simulations suggest
that this method is efficient in terms of  learning  speed,
the   number  of  free  parameters  in  the  network  (i.e.
the   number  of  free  parameters  in  the  network  (i.e.
connections), and it compares  well  to  other  methods  in
terms of generalisation performance.