The following paper is now available in pub/neuroprose.
Catastrophic Interference is Eliminated in Pretrained Networks
Ken McRae
University of Rochester
&
Phil A. Hetherington
McGill University
When modeling strictly sequential experimental memory tasks, such as
serial list learning, connectionist networks appear to experience
excessive retroactive interference, known as catastrophic interference
(McCloskey & Cohen,1989; Ratcliff, 1990). The main cause of this
interference is overlap among representations at the hidden unit layer
(French, 1991; Hetherington,1991; Murre, 1992). This can be alleviated by
constraining the number of hidden units allocated to representing each
item, thus reducing overlap and interference (French, 1991; Kruschke,
1992). When human subjects perform a laboratory memory experiment, they
arrive with a wealth of prior knowledge that is relevant to performing the
task. If a network is given the benefit of relevant prior knowledge, the
representation of new items is constrained naturally, so that a sequential
task involving novel items can be performed with little interference.
Three laboratory memory experiments (ABA free recall, serial list, and ABA
paired-associate learning) are used to show that little or no interference
is found in networks that have been pretrained with a simple and relevant
knowledge base. Thus, catastrophic interference is eliminated when
critical aspects of simulations are made to be more analogous to the
corresponding human situation.
Thanks again to Jordan Pollack for maintaining this electronic library.
An example of how to retrieve mcrae.pretrained.ps.Z:
your machine> ftp archive.cis.ohio-state.edu
Connected to archive.cis.ohio-state.edu.
220 archive FTP server (Version 6.15 Thu Apr 23 15:28:03 EDT 1992) ready.
Name (archive.cis.ohio-state.edu:kenm): anonymous
331 Guest login ok, send e-mail address as password.
Password:
230 Guest login ok, access restrictions apply.
ftp> cd pub/neuroprose
250-Please read the file README
250- it was last modified on Mon Feb 17 15:51:43 1992 - 357 days ago
250-Please read the file README~
250- it was last modified on Wed Feb 6 16:41:29 1991 - 733 days ago
250 CWD command successful.
ftp> binary
200 Type set to I.
ftp> get mcrae.pretrained.ps.Z
200 PORT command successful.
150 Opening BINARY mode data connection for mcrae.pretrained.ps.Z (129046 bytes).
226 Transfer complete.
local: mcrae.pretrained.ps.Z remote: mcrae.pretrained.ps.Z
129046 bytes received in 30 seconds (4.2 Kbytes/s)
ftp> quit
221 Goodbye.
your machine> uncompress mcrae.pretrained.ps.Z
your machine> then print the file