Connectionist symbol processing: any progress?

[Douglas Blank]

Dave_Touretzky at cs.cmu.edu wrote:

> I'd like to start a debate on the current state of connectionist
> symbol processing?  Is it dead?  Or does progress continue?

For me, it is dead. 

Implementing symbol processing in networks was a good first step in
solving many problems that plagued symbolic systems. Tony Plate's HRR as
applied to analogy is a great example (Plate, 1993). Using connectionist
representations and methodologies, an expensive symbolic similarity
estimation process was eliminated in the analogy-making MAC/FAC system
(Gentner and Forbus, 1991). 

The bad news is that, in my opinion, the entire MAC/FAC model (like many
symbolic models) has a fatal flaw and will never lead to an autonomous,
flexible, creative, intelligent (analogy-making) machine. Even if
Gentner's entire model were implemented completely in a network (or even
real neurons), their problem would remain: the overall system
organization is still "symbolic". Their method requires that analogies
be encoded as symbols and structures, which leaves no room for
perception or context effects during the analogy making process (for a
detailed description of this problem, see Hofstadter, 1995).

I believe that in order to solve the big AI/cognitive problems ahead
(like making analogies), we, as modelers, will have to face a radical
idea: we will no longer understand how our models solve a problem
exactly. I mean that, for many complex problems, systems that solve them
won't be able to be broken down into symbols and modules, and,
therefore, there may not be a description of the solution more abstract
than the actual solution itself. 

Some researchers have been focusing on solving high-level problems via a
purely connectionist framework rather than augmenting a symbolic one.
Meeden's planning system comes to mind, as does (warning:
self-promotion) my own work in analogy-making (Meeden, 1994; Blank,
1997). Rather than focusing on some assumed-necessary symbolically-based
process (say, variable binding) these models look at a bigger goal:
modeling a complex behavior. 

Building and manipulating structured representations or binding
variables via networks should not be our goals.* Neither should creating
a model such that we can understand its inner workings.** Rather, we
should focus on the techniques that allow a system to self-organize such
that it can solve The Bigger Problems. I think much of the discussion on
"learning to learn" has been related to this issue.

> I'd love to hear some good news.

For me, "connectionist symbol processing" was a very useful stage I went
through as a cognitive scientist. Now I see that networks can do the
equivalent of processing symbols, and not have anything to do with
symbols. In addition, I learned that I can feel ok about not
understanding exactly how they do it.

-Doug Blank

*Of course, building and manipulating structured representations or
binding variable via nets is still useful for some problems, just not
all of them.

**The DOD is not interested in these types of systems. 


References

Blank, D.S. (1997). "Learning to see analogies: a connectionist
exploration." Unpublished PhD Thesis, Indiana University, Bloomington.
http://dangermouse.uark.edu/~dblank/thesis.html

Gentner, D., and Forbus, K. (1991). MAC/FAC: a model of similarity-based
access and mapping. In "Proceedings of the Thirteenth Annual Cognitive
Science Conference," 504-9. Hillsdale, NJ: Lawrence Erlbaum.

Hofstadter, D., and FARG (1995). "Fluid concepts and creative
analogies." Basic Books, new York, NY.

Meeden, L. (1994) "Towards planning: incremental investigations into
adaptive robot control." Unpublished PhD Thesis, Indiana University,
Bloomington. http://www.cs.swarthmore.edu/~meeden/

Plate, T.A. (1991). Holographic reduced representations: convolution
algebra for compositional distributed representations. In "Proceedings
of the Twelfth International Joint Conference on Artificial
Intelligence." Myopoulos, J. and Reiter, R. (Eds.), pp. 30-35. Morgan
Kaufmann.

-- 
=====================================================================
dblank at comp.uark.edu            Douglas Blank, University of Arkansas
Assistant Professor                                  Computer Science
==================== http://www.uark.edu/~dblank ====================