neural binding

[stephen j. hanson]

We recently published a paper in Neural Computation on a related kind of
binding problem:


In our case we were able to show transfer to NOVEL vocabularies (fixing
DFAs) after training on other vocabularies using the same grammar.
In effect the RNN learns to factor the vocabulary from the grammar and
then quickly recruit new vocabularies that merely have syntactic
similarity to already learned vocabularies.

Examination of the hidden space indicates the RNN learns a type of
"spatial metaphor" to recruit the unseen vocabulary in nearby parts of
state (DFA)space. The network creates a hierarchical state structure
that allows arbitrary and unique binding of new instances.

Similarly for decades Pinker and others have maintained that this sort
of transfer was impossible for "associationist" networks.

Cheers,

Steve

----------

"On the Emergence of Rules in Neural Networks"
Stephen Jose Hanson and Michiro Negishi

Neural Computation - Contents - Volume 14, Number 9 - September 1, 2002




On Mon, 2002-08-05 at 12:18, Jerome Feldman wrote:
>  
>  In his new book, Foundations of Language, Ray Jackendoff poses four challenges
> to "cognitive neuroscience", all related to the binding problem. He starts with
> the simple sentence:
> 
>         The big star's beside a little star.
> 
> and asks how neural computation could model the two different stars without
> cross talk. He then goes on to more complex problems like variables, memory 
> levels and learning. 
> 
>  Various groups, including ours, have been working on these issues for decades and
> Ray's challenges provide a nice focus for assessing the current state of play. I
> will be happy to collect responses and post a summary back to the group. The
> relevant section of the book is pages 58-67 and is self contained.
> 
> -- 
> Jerome Feldman
> ICSI & EECS UC Berkeley		"O wad some Pow'r the giftie gie us
> 1947 Center St.	     		  To see oursels as other see us!"
> Berkeley CA 94704		 Robert Burns - To a Louse
>