About sequential learning (or interference)

[James L. McClelland]

DeLiang Wang <dwang at cis.ohio-state.edu> writes:

> It was reported some time ago that multilayer perceptrons suffer the problem
> of so-called "catastrophic interference", meaning that later training will
> destroy previously acquired "knowledge" (see McCloskey & Cohen, Psychol. of
> Learning and Motivat. 24, 1990; Ratcliff, Psychol. Rev. 97, 1990). This seems
> to be a serious problem, if we want to use neural networks both as a stable
> knowledge store and a long-term problem solver.

The brain appears to have solved this problem by storing new
information in a special associative memory system in the hippocampus.
According to this view, cortical (and some other non-hippocampal)
systems learn slowly, using what I call 'interleaved learning'.
Weights are adjusted a small amount after each experience, so that the
overall direction of weight change is governed by the structure
present in the ensemble of events and experiences.  New material can
be added to such a memory without catastrophic intereference if it is
added slowly, interleaved with ongoing exposure to other events and
experiences.  This is too slow for the demands placed on memory by the
world.  To allow rapid learning of new material without catastrophic
interference, new material is initially stored in the hippocampus,
where sparse, conjunctive representations are used to minimize
interference with other memories.  Reinstatement of these patterns
occurs in task-relevant situations via bi-directional connections
between hippocampus and cortex together with pattern completion within
the hippocampal formation.  Interestingly, it appears that
reinstatement occurs in off-line situations as well -- most notably,
during sleep.  Each reinstatement provides an opportunity for the
neocortex to learn; but these reinstatements occur interleaved with
other ongoing events and experiences, and weight changes are small on
each reinstatement, so that the neocortex learns the new material via
interleaved learning.  This theory is consistent with a lot of data at
this point, including of course the basic fact that bilateral removal
of the hippocampus leads to a profound deficit in the ability to form
arbitrary new memories rapidly.  Among the most important additional
findings are 1) the phenomenon of temporally graded retrograde amnesia
--- the finding that damage to the hippocampus can produce a selective
deficit for recently acquired memories leaving memories acquired
several weeks or months earlier intact --- and 2) the finding that
neurons in the hippocampus that are coactive during a particular
behavior appear to be coactive during sleep following the behavior, as
if the patterns of activity that were present during behavior are
reactivated during sleep.

I announced a technical report discussing the above ideas last March.
In that report, we focused on why it makes sense from a connectionist
point of view that the system should be organized as described above.
A reprint of the announcement (sans abstract) appears below.  This TR
is currently under revision for publication; the revised version will
contain a fuller presentation of the physiological evidence than the
present version, and I will announce it on connectionists when it
becomes available.

========================================================================
Technical report announcement:
------------------------------------------------------------------------

             Why there are Complementary Learning Systems
                  in the Hippocampus and Neocortex:
             Insights from the Successes and Failures of
             Connectionist Models of Learning and Memory

    James L. McClelland, Bruce L. McNaughton & Randall C. O'Reilly
        Carnegie Mellon University & The University of Arizona

                    Technical Report PDP.CNS.94.1
                             March, 1994

=======================================================================

Retrieval information:

unix> ftp 128.2.248.152                 # hydra.psy.cmu.edu
Name: anonymous
Password: <email address>
ftp> cd pub/pdp.cns
ftp> binary
ftp> get pdp.cns.94.1.ps.Z
ftp> quit
unix> zcat pdp.cns.94.1.ps.Z | lpr      # or however you print postscript

NOTE:  

The compressed file is 306994 bytes long.
Uncompressed, the file is 840184 byes long.

The printed version is 63 total pages long.

For those who do not have FTP access, physical copies can be requested from
Barbara Dorney <bd1q+ at andrew.cmu.edu>.  Ask for the report by title or
pdp.cns technical report number.