catastrophic interference of BP

[Phil A. Hetherington]

Neil Burgess wrote:

> Studies of catastrophic interference in BP networks are
> interesting when considering such a network as a model of some human
> (or animal) memory system. 
> Is there any reason for doing that?

Of course.  Network models have properties such as distributed
representations, generalization, interference, content addressability,
etc., that are also true of animal and human memory.  They provide an
alternative framework for construing memory processes that is superior to
box and arrow modeling primarily because they can be 'lesioned' as in
animal experiments or as found in human patients and because they are
'executable'.  Because they are executable (i.e., perform a function) the
effects of these lesions and other parameter manipulations on the
performance of the network can be observed.  There are now many
alternative supervised learning algorithms available, but there are still 
many reasons to continue to study this one. It is most certainly
true that both humans and animals learn via feedback provided by the
effect of erroneous behavior--a process analagous to back prop. 
Unsupervised learning algorithms such as competetive learning do not give
you this.  Unsupervised learning algorithms are not easily executable--you
can't get them to do many simple 'behaviors', like plot trajectories from
starting locations to multiple goals, as in Neil's models.  Of course, any
supervised learning algorithm will confer the ability to train the net to
perform, but there are a couple, mostly pragmatic, reasons to stick with
back prop for now.  Primarily because of the availablity of the
McClelland & Rumelhart books and program disks, back prop is most easily
available and most commonly used.  Back prop already provides the engine
in countless models already published.  Gaining an understanding of the
behavior of this algorithm will enable a better understanding of the flaws
of these already published models.  Thus, its not so much that you *would
want to* use the algorithm, it is that it has already been used.  Lets
understand why we are discarding it before we do so. 

Phil Hetherington
pah at unixg.ubc.ca