On blurring the gap between NN and AI

[Dave Rumelhart]

It seems to me that a number of issues are being confused in this
discussion.  One has to do with what AI is and another has to do with
what "connectionism" is and why we might be interested it.

First, with regard to AI it seems to me that there are at least three
aspects. (1) Theoretical AI, (2) Experimental AI and (3) Applied AI.  I
take it that Theoretical AI is the development and analysis of
algorithms and computational models which draw their inspiration from
considerations of biological (especially) human intelligence.  This
strikes me as a special kind of applied mathematics.  On this count,
connectionist approaches, search based approaches, logic based
approaches, problem space based approaches and many others all fall
squarely in the realm of theoretical AI.  These various approaches
simply draw their inspiration from different aspects of natural
intelligence and therefore find different mathematical formalisms more
useful.   I take it that experimental AI is the study of the algorithms
and computational models developed in theoretical AI by experimental
means -- that is through the development of computer simulations and
implementations of the algorithms.  If our models and algorithms could
be fully analytically developed there would be no need for an
experimental approach.  Since most algorithms and computational models
seem to be too complex and to interact with a world that is itself not
easy to characterize we often resort to an experimental approach.  Here
AI differes from most (but not all) classical applied mathematical
approaches.  On the whole connectionist approaches employ the
experimental method to about the same degree as most other AI
approaches.  To the degree that different AI approaches can be applied
to the same problems it is certainly possible to compare different
algorithms and computational approaches in terms of efficiency (on a
machine of a particular type) quality of performance and other
dimensions, but the effort is primarly one of analysis -- what are the
properties of the systems under study.  The third activity within AI,
applied AI, is simply the applications of the AI techniques mentioned
above to a real world problem.  In this case, many practical issues
intervene.   Here we ask simply how well does the algorithm and
procedure do on the area of application.  The measurement criteria may
involve difficulty of development as well as the quality of the
performance.   In this case, I would be suprised if a single approach
was always better for all areas of application. At this level it is an
empirical question.

To summarize, as far as AI is concerned, it strikes me that the
connectionist approach is one among many and partakes of most of the
features of the other approaches.  It may turn out that the
connectionist approach is particularly well suited to particular kinds
of applications, may be particularly elegant and may be nice in certain
other ways, but beauty is often in the eye of the beholder.  The
question of whether there is a great divide between the symbolic and
"sub-symbolic" approaches is one that I would rather leave to the
philosophers.   In any case, it has nothing to do with wether or not
connectionist AI is a kind of AI.  I simply can't think of any
reasonable definition of AI that would exclude it.


The second major point concerns the nature of "connectionism" itself.
It should be noted that there are connectionist approaches to fields
other than AI and in this case the connectionist approach cuts across
several fields.  In particular, there are connectionist approaches to
psychology, to neuroscience, to linguistics and to other scientific
domains.   In these cases, the criteria for evaluation is rather
different than for AI.  In these fields we are interested in the degree
to which models developed within the connectionist paradigm are useful
in understanding, explaining or predicting empirical phenomena.  These
phenomena may involve explaining the behavior of people or other animals
or in explaining the observations made by a neurobiologist when studying
the brain.  One of the hopes for the connectionist approach is that it
will be able to provide a formalism for explaining the relatioinship
between neural activity and behavior someday.  The evidence that this
will occur is, of course, not yet in.  It is the job of connectionist
researchers to do the necessary research, develop the necessary ideas
and make the case to the scientific community at large that this is
possible.  Finally, I should say that this attempt to develop AI
formalisms that have applicability to scientific model building is not
unique to the connectionist approach.  Many AI formalisms have been
proposed as useful for explaining psychological or linguistic (but rarely
neurobiological) phenomena.  For a good example see Newell's new book on
so-called Unified Theories.


Sorry for the wordiness of this message.


        D. E. Rumelhart