Sequential learning and interactive tandem networks: paper available

[Bob French]

FTP-host: cogsci.indiana.edu 
FTP-filename: /pub/french.tandem-stm-ltm.ps.Z
Total no. of pages: 6


The following paper is now available by anonymous ftp from the CRCC 
archive at Indiana University, Bloomington. 


                     Interactive Tandem Networks 
                and the Problem of Sequential Learning

                           Robert M. French
                       CRCC, Indiana University
                        Bloomington, IN 47408
                      french at cogsci.indiana.edu


                               Abstract

This paper presents a novel connectionist architecture to handle the
"sensitivity-stability" problem and, in particular, an extreme
manifestation of the problem, catastrophic interference.  This
architecture, called an interactive tandem-network (ITN) architecture,
consists of two continually interacting networks, one -- the LTM
network -- dynamically storing "prototypes" of the patterns learned,
the other -- the STM network -- being responsible for "short-term"
learning of new patterns.  Prototypes stored in the LTM network
influence hidden-layer representations in the STM network and,
conversely, newly learned representations in the STM network gradually
modify the more stable LTM prototypes.  As prototypes are learned by
the LTM network, they are dynamically constrained to maximize mutual
orthogonality.  This system of tandem networks performs particularly
well on the problem of catastrophic interference.  It also produces
"long-term" representations that are stable in the face of new input
and "short-term" representations that remain sensitive to new input.
Justification for this type of architecture is similar to that given
recently by McClelland, McNaughton, & O'Reilly (1994) in arguing for
the necessary complementarity of the hippocampal (short-term memory)
and neocortical (long-term memory) systems. 

                                 * * *

This paper has been submitted to The 1995 Cognitive Science Society
Conference.  A longer version of this paper is in preparation and, 
consequently, comments are welcome.  I am currently visiting at the 
Univeristy of Wisconsin.  Any snail-mail should be sent there.

Robert M. French
Department of Psychology
University of Wisconsin
Madison, Wisconsin 53706
Tel: (608) 243-8026
FAX: (608) 262-4029
email: french at cogsci.indiana.edu
       french at merlin.psych.wisc.edu