Paper on Generalization in Interactive Networks

[Randall C. O'Reilly]

The following preprint is now available for downloading:

ftp://grey.colorado.edu/pub/oreilly/papers/oreilly00_gen_nc.pdf *or*
ftp://grey.colorado.edu/pub/oreilly/papers/oreilly00_gen_nc.ps

  Generalization in Interactive Networks: The Benefits of Inhibitory
		   Competition and Hebbian Learning

			 Randall C. O'Reilly

		       Department of Psychology
		  University of Colorado at Boulder

		    In press at Neural Computation

Abstract:

Computational models in cognitive neuroscience should ideally use
biological properties and powerful computational principles to produce
behavior consistent with psychological findings.  Error-driven
backpropagation is computationally powerful, and has proven useful for
modeling a range of psychological data, but is not biologically
plausible.  Several approaches to implementing backpropagation in a
biologically plausible fashion converge on the idea of using
bidirectional activation propagation in interactive networks to convey
error signals.  This paper demonstrates two main points about these
error-driven interactive networks: (a) they generalize poorly due to
attractor dynamics that interfere with the network's ability to
systematically produce novel combinatorial representations in response
to novel inputs; and (b) this generalization problem can be remedied
by adding two widely used mechanistic principles, inhibitory
competition and Hebbian learning, that can be independently motivated
for a variety of biological, psychological and computational reasons.
Simulations using the Leabra algorithm, which combines the generalized
recirculation (GeneRec) biologically-plausible error-driven learning
algorithm with inhibitory competition and Hebbian learning, show that
these mechanisms can result in good generalization in interactive
networks.  These results support the general conclusion that cognitive
neuroscience models that incorporate the core mechanistic principles
of interactivity, inhibitory competition, and error-driven and Hebbian
learning satisfy a wider range of biological, psychological and
computational constraints than models employing a subset of these
principles.

				- Randy

+----------------------------------------------------------------+
| Dr. Randall C. O'Reilly   |                                    |
| Assistant Professor       | Phone: (303) 492-0054              |
| Department of Psychology  | Fax:   (303) 492-2967              |
| Univ. of Colorado Boulder | Home:  (303) 448-1810              |
| Muenzinger D251C          | Cell:  (720) 839-7751              |
| 345 UCB                   | email: oreilly at psych.colorado.edu  |
| Boulder, CO 80309-0345    | www:   psych.colorado.edu/~oreilly |
+----------------------------------------------------------------+