Connectionist symbol processing: any progress?

[Dr. Istvn S. N. Berkeley]

Hi there,
I am afraid that I can no longer resist adding my 2 cents worth to this
debate.

Douglas Blank wrote:

> I believe that in order to solve the big AI/cognitive problems ahead
> (like making analogies), we, as modelers, will have to face a radical
> idea: we will no longer understand how our models solve a problem
> exactly. I mean that, for many complex problems, systems that solve them
> won't be able to be broken down into symbols and modules, and,
> therefore, there may not be a description of the solution more abstract
> than the actual solution itself. 

It seems to me that there is something fundamentally wrong about the
proposal here. As McCloskey (1991) has argued, unless we can develop an
understanding of how network models (or any kind of model for that
matter) go about solving problems, they will not have any useful impact
upon cognitive theorizing. Whilst this may not be a problem for those
who wish to use networks merely as a technology, it surely must be a
concern to those who wish to deploy networks in the furtherment of
cognitive science. If we follow the suggestion made above then even
successful attempts at modelling will be theoretically sterile, as we
will be creating nothing more than 'black boxes'.

This much having been said, the problem of interpreting and analysing
trained network systems is not a trivial one, especially for large scale
models. Although there are a variety of techniques which have been
deployed (See Berkeley et al. 1995, Hanson and Burr 1990 and Elman 1990,
for examples), none of them are entirely satisfactory, or universally
applicable. Indeed, there has been some skepticism about the feasibility
of trained network analysis in the literature (See Hecht-Nielsen 1990,
Moser and Smolensky 1989 and Robinson, 1992). Nonetheless, if
connectionist networks are to prove useful to cognitive science,
continuing efforts to better understand mature networks are going to be
crucial.

A further point which needs to be raised (and sorry, this is where the
self-advertising begins) is that some efforts at trained network
analysis have turned up surprising results, which seem highly germaine
to the topic of connectionist symbol processing. Some years ago myself
and a number of members of The Biological Computation Project undertook
the analysis of a network trained upon a logic problem originally
studied by Bechtel and Abrahamsen (1991). Much to our surprise, our
analysis showed that the network had developed stable patterns of hidden
unit activation which closely mirrored the standard rules of inference
from traditional sentential calculus, such as Modus Ponens. Moreover, we
were able to make a number of useful and abstract generalizations about
network functioning which were novel and informative. These results
directly challenged the conclusions orignially drawn about the task by
Bechtel and Abrahasen (1991). This work is described in detail in
Berkeley et al. (1995).

What this work suggests is that, rather than abandoning attempts at
understanding mature networks, a more rational and productive path is to
attempt to analyse in detail trained network function and then use the
empirical results from such studies to inform judgements about
connectionist symbol processing. 

All the best,

Istvan


Bibliography

Bechtel, W. and Abrahamsen, A. (1991), *Connectionism and the Mind*,
Basil Blackwell (Cambridge, MA).

Berkeley, I., Dawson, M., Medler, D. Schopflocher, D. and Hornsby, L.
(1995) "Density Plots of Hidden Value Unit Activations Reveal
Interpretable Bands" in *Connection Science* 7/2, pp. 167-186.

Elman, J. (1990), "Finding Structure in Time", in *Cognitive Science*
14, pp. 179-212.

Hanson, S. and Burr, D. (1990), "What Connectionist Models Learn:
Learning and Representation in Connectionist Networks" in Behavioral and
Brain Sciences 13, pp. 471-518.

Hecht-Nielsen, R. (1990), *Neurocomputation* Addison-Wesley Pub. Co.
(New York).

McCloskey, M. (1991), "Networks and Theories: The Place of Connectionism
in Cognitive Science", *Psychological Science* 2/6, pp. 387-395.

Mozer, M. and Smolensky, P. (1989), "Using Relevance to Reduce Network
Size Automatically", in *Connection Science* 1, pp. 3-16.

Robinson, D. (1992) "Implications of Neural Networks for How we Think
about Brain Function" in Behavioral and Brain Sciences, 15, pp. 644-655.
-- 
Istvan S. N. Berkeley Ph.D,     E-mail: istvan at USL.edu,  
Philosophy, The University of Southwestern Louisiana, 
USL P. O. Box 43770, Lafayette, LA 70504-3770, USA.
Tel:(318) 482 6807, Fax: (318) 482 6195, http://www.ucs.usl.edu/~isb9112