About sequential learning (or interference)

[Ping Li]

Most of the previous efforts to reduce catastrophic intereference
seem to have focused on modifying the network archetecture (though
with some exceptions, e.g., Sharkey's and McRae's work). I wonder 
to what extent catastrophic intereference may be reduced if one
manipulates the training data in some way. For example, my early
study (CRL TR 9203) found if the network is presented with the
full data, catastrophic intereference occurs. Some of my preliminary
results now suggest that if one uses an "incremental learning" 
schedule (input data enters into training piece by piece; there are 
many reasons for a developmental psychologist why this kind of increments 
are necessary --- see Elman, 1993 Cognition, Plunkett & Marchman, 1993 
Cognition), then catastrophic intereference may be reduced.  This also
seems to go well with what Jay McClelland suggests earlier in his message:

> According to this view, cortical (and some other non-hippocampal)
> systems learn slowly, using what I call 'interleaved learning'.
> Weights are adjusted a small amount after each experience, so that
> the overall direction of weight change is governed by the structure
> present in the ensemble of events and experiences.  New material can
> be added to such a memory without catastrophic intereference if it
> is added slowly, interleaved with ongoing exposure to other events and
> experiences.


Happy New Year! 

**********************************************************************
Ping LI					     Email: pingli at cuhk.hk
Department of Psychology                     Phone: (852)609-6576
The Chinese University of Hong Kong          Fax:   (852)603-5019
**********************************************************************