Connectionist symbol processing: any progress?

[Jerry Feldman]

Dave Touretsky asks how well we are doing at making 
neurally plausible models of human symbolic processes 
like natural language. Let's start at CMU. One of the 
driving sources of the "new connectionism" was the
interactive activation model of McClelland and Rumelhart; 
Jay McC continues to work on this as well as other 
things. John Anderson's spreading activation ACT* models 
continue to attract (annual?) workshops.

 More broadly, the various basic ideas of connectionist 
modeling are playing an important (sometimes dominant) 
role in several fields that deal with language and 
symbolic behavior. For example, Elman nets continue
to be a standard way to do models in Cognitive 
Psychology. The text and workbook on "Rethinking 
Innateness" by Elman, et.al. is a major force in 
Developmental Psychology. Spreading activation 
models underlie all priming work in Cognitive Psychology 
and Psycholinguistics. In Neuropsychology, Damasio's 
convergence zones continue to attract serious attention.

 Paul Smolensky's Harmony Theory (in a simplified form) 
has become a dominant paradigm in phonology and is 
beginning to play a large role in discussions of grammar 
- witness the long invited article in Science this year. 
Shastri's note lists a number of other relevant efforts.

 In Artificial Intelligence, Belief Networks have 
arguably replaced Symbolic Logic as the leading paradigm. 
The exact relation between Belief Networks and structured 
connectionist models remains to be worked out and this 
would be a good topic for discussion on this list. For a 
good recent example, see the (prize) paper by Srini 
Narayanan and Dan Jurafsky at CogSci98.
 
 It is true that none of this is much like Touretsky's 
early attempt at a holographic LISP and that there has 
been essentially no work along these lines for a decade. 
There are first order computational reasons for this. 
These can be (and have been) spelled out technically
but the basic idea is straightforward - PDP (Parallel 
Distributed Processing) is a contradiction in terms. To 
the extent that representing a concept involves all of 
the units in a system, only one concept can be active at 
a time. Dave Rumelhart says this is stated somewhere in 
the original PDP books, but I forget where. The same 
basic point accounts for the demise of the physicists' 
attempts to model human memory as a spin glass. 
Distributed representations do occur in the brain and
are useful in many tasks, conceptual representation just 
isn't one of them.

 The question of how the connectionist brain efficiently 
realizes (and learns) symbolic processes like language is 
one of the great intellectual problems of our time. I 
hope that people on this list will continue to contribute 
to its solution.

-- 
Jerry Feldman