Incremental Learning

[rachida]

Using incremental neural networks procedures to perform learning tasks is
certainely a very attractive idea. These methods allow automatic tuning of
the network size what one generally does empirically with an important risk
of over-estimation or under-estimation, implying an untractable computation
consuming trial/error procedure.The main questions to address when dealing
with evolutive architectures are:

1. How to estimate the new unit(s) parameters ?

2. How to connect this(these) unit(s) to the previous network so that is
possible to carry on learning without restarting ?

3. When to stop the adding process ?

We recently published in NPL (Neural Processing letters in January 1995) a
paper presenting our new incremental procedure for supervised learning with
noisy data. Each step consists in adding to the current network a new unit
which is trained to learn the error of the network. The incremental step is
repeated until the error of the current network reduce to the noise in the
data. The stopping criterion is very simple and can be directly deduced
from a statistical test on the estimated parameters of the new unit. Some
experimental results on function approximation tasks point out the efficacy
of this new incremental scheme especially to avoid spurious minima and to
design a network with a well-suited size. The number of basic operations is
also decreased and gives an average gain on convergence speed of about 20%.

For more information, consult:
=============================

C.Jutten and R.Chentouf. A New Scheme for Incremental Learning. Neural
Processing Letters, Vol. 2, 1, pp. 1-4, 1995.

R.Chentouf and C.Jutten. Incremental Learning with a Stopping Criterion:
experimental results. In IWANN'95: From Natural to artificial Neural
Computation, J. Mira and F. Sandoval (Eds.), Lecture Notes in Computer
Science 930, Springer, pp. 519-526, June 7-9, 1995


+++++++++++++++++++++++
Mrs CHENTOUF Rachida
LTIRF-INPG
46 AV Felix Viallet
38000 Grenoble France
Tel : (+33) 76 57 45 50
Fax : (+33) 76 57 47 90