A summary on Catastrophic Interference
Phil A. Hetherington
pah at unixg.ubc.ca
Fri Feb 3 12:57:00 EST 1995
> * Reduce overlapping in hidden unit representations. French (1991,
>1994), Kruschke (1992), Murre (1992), Fahlman (1991).
>
> * Prior training. Assuming later patterns are drawn from the same
>underlying function as earlier patterns, prior training of the general task
>can reduce RI (McRae and Hetherington, 1993). This proposal will not
>work if later patterns have little to do with previously trained patterns.
Actually, prior training is a method of reduction of overlap at the
hidden layer, as it achieves the same goal naturally, and will work if
later patterns have little to do with previously trained patterns. McRae
and Hetherington demonstrated that the method was sucessful with
autoencoders and then replicated this with a network that computed
arbitrary associations (i.e., random). Given that prior training also
consisted of random associations, there was no general function the net
could derive (i.e., later patterns had little to do with previous
patterns). The training merely had the effect of 'sharpening' the
hidden unit responses so that later items would not be distributed across
all or most units, as is the case in a naive net. I would make an
empirical prediction that in a net trained sequentially to recall faces,
prior training on white-noise would probably suffice to reduce CI.
Cheers,
Phil Hetherington
More information about the Connectionists
mailing list