TR: Complementary Learning Systems in Hippocampus and Neocortex

[James L. McClelland]

The following Technical Report is available both electronically from
our own FTP server or in hard copy form.  Instructions for obtaining 
copies may be found at the end of this post.

========================================================================


             Why there are Complementary Learning Systems
                  in the Hippocampus and Neocortex:
             Insights from the Successes and Failures of
             Connectionist Models of Learning and Memory

    James L. McClelland, Bruce L. McNaughton & Randall C. O'Reilly
        Carnegie Mellon University & The University of Arizona

                    Technical Report PDP.CNS.94.1
                             March, 1994

The influence of prior experience on some forms of behavior and
cognition is drastically affected by damage to the hippocampal system.
However, if the hippocampal system is left intact both during the
experience and for a period of time thereafter, subsequent damage can
have much less or even no effect.  Such findings suggest that memory
traces change over time in a way that makes them less dependent on the
hippocampal system.  This process of change has often been called
consolidation.  Consolidation is a very gradual process; in humans, it
appears to span up to 15 years.  This article asks what consolidation
is and why it occurs.  We take as our point of departure the view that
the initial memory trace that results from a relevant experience
consists of changes to the strengths of the connections among neurons
in the hippocampal system.  Bidirectional connections between the
neocortex and the hippocampus allow these initial traces to mediate
the reinstatement of representations of events or experiences in the
neocortex.  Consolidation results from the cumulative effects of
small, incremental changes to connections among neurons in the
neocortex that occur each time such a representation is reinstated.
This view leads to two key questions: 1) Why are plastic changes made
initially in the hippocampus, if ultimately the substrate of a
consolidated memory lies in the neocortex?  2) Why does consolidation
span such an extended period of time?

Insights from connectionist network models of learning and memory
provide one set of possible answers to these questions.  These models
consist of networks of simple processing units and weighted
connections among the units, and they offer procedures for discovering
what weights or values to use on the connections so that the network
can capture the structure present in ensembles of events and
experiences drawn from some domain.  These connection weights then
provide the basis for appropriate generalization to novel examples
from the same domain.  Crucially, the success of these procedures
depends on interleaved learning: making only very small changes to the
connection weights on each learning trial, so that the overall
direction of weight change can be governed by the structure of the
domain rather than the individual examples.  The sequential
acquisition of new data is incompatible with the gradual discovery of
structure and can lead to catastrophic interference with what has
previously been learned.  In the light of these observations, we
suggest that the neocortex may be optimized for the gradual discovery
of the shared structure of events and experiences, and that the
hippocampal system is there to provide a mechanism for rapid
acquisition of new information without interference with previously
discovered regularities.  After this initial acquisition, the
hippocampal system serves as teacher to the neocortex: That is, it
allows for the reinstatement in the neocortex of representations of
past events, so that they may be gradually acquired by the cortical
system via interleaved learning.  We equate this interleaved learning
process with consolidation, and we suggest that it is necessarily slow
so that new knowledge can be integrated effectively into the
structured knowledge contained in the neocortical system.

=======================================================================

Retrieval information for pdp.cns TRs:

unix> ftp 128.2.248.152                 # hydra.psy.cmu.edu
Name: anonymous
Password: <email address>
ftp> cd pub/pdp.cns
ftp> binary
ftp> get pdp.cns.94.1.ps.Z
ftp> quit
unix> zcat pdp.cns.94.1.ps.Z | lpr      # or however you print postscript

NOTE:  

The compressed file is 306994 bytes long.
Uncompressed, the file is 840184 byes long.

The printed version is 63 total pages long.

For those who do not have FTP access, physical copies can be requested from
Barbara Dorney <bd1q+ at andrew.cmu.edu>.