sequential learning

[Phil A. Hetherington]

A couple notes on interfernce. First, most of Jay's idea is still simply
theory, a good theory, but there is still little direct evidence that the
hippocampus does either pattern separation or pattern completion.  Most of
the pattern completion idea comes from Marr's conjectures on the
architecture of CA3 in hippocampus.  Second (to Hanson), interference is
not necessarily the flip side of generalization.  This is intuitive, but you
can reduce one without affecting the other.  See Lewandowsky's stuff on
this .  Next, whether the interference observed in connectionist nets is a
problem or not depends on your needs.  Is the net to model human learning
or to solve a particular engineering problem.  If the former, then see
McRae and Hetherington (1993)--it is not clear that realisticly designed
nets suffer from catastrophic interference.  Finally, it appears the the
bigger problem is to reduce interference without eliminating the
distinction between old and new items.  This was Ratcliff's main concern,
and is addressed by Sharkey. 

See also:

French, R.M. (1991). Proceedings of the Thirteenth Annual Conference of
  the Cognitive Science Society, 173-178. 
French, R.M. (1992). Connection Science, 4, 3-4, 365-377. 
Hetherington, P.A. (1990).  Neural Network Review, 4(1), 27-29. 
Hetherington, P.A. (1991).  The Sequential Learning Problem in 
  Connectionist Networks.  Unpublished Masters Thesis, McGill
  University, Montreal, Quebec. 
Hetherington, P.A., & Seidenberg, M.S. (1989).  Proceedings of the 
  Eleventh Annual Conference of the Cognitive Science Society, 26-33. 
Hinton, G.E., & Plaut, D.C. (1987).  Proceedings of the Ninth Annual 
  Conference of the Cognitive Science Society, 177-186.
Kortge, C.A. (1990). Proceedings of the Twelfth Annual Conference of the
  Cognitive Science Society, 764-771. 
Krushke, J.K. (1992).  Psychological Review, 99, 22-44. 
Krushke, J.K. (1993).  Connection Science, 5, 3-36.
Lewandowsky, S. (1991).  In W.E. Hockley & S, Lewandowsky (Eds.), Relating
  theory and data: Essays on human memory in honor of Bennet B. Murdock.
McRae, K., & Hetherington, P.A. (1993). Proceedings of the Fifteenth
  Annual Conference of the Cognitive Science Society, 723-728. 
Murre, J.M.J. (1992).  Learning and categorization in modular neural 
  networks.  Lawrence Erlbaum. 
Sharkey, N.E., & Sharkey, A.J.C. Understanding Catastrophic
  Interference in neural Nets.  Technical Report, Department of Computer
  Science, University of Sheffield, U.K. 
Sloman, S.A., & Rumelhart, D.E. (1992).  In A. Healy, S.M. Kosslyn, & 
  R.M. Shiffrin (Eds.), From learning theory to cognitive processes: Essays 
  in honor of William K. Estes.