What is a connectionist net? Here's what it's not.

[KELLY%BROWNCOG.BITNET@mitvma.mit.edu]

        What is a connectionist model, you ask?  Well, I don't think I can
answer that specifically, but I can tell you what it's not.  In the first
place it *is* a member of a larger class of models called complex systems.
But that doesn't help us either, because nobody really knows what a complex
system is.  The generally conceived definition has something to do with large
numbers of simple, interconnecting units which can perform some type of
"cooperative computation".  That is, individually the units are so dumb that
they can't do anything, but together they can do alot.
        Well, then my claim (I'm really out on a limb here), is that systems
with large numbers of very complex, interconnecting units really aren't
connectionist models (or even complex systems) at all, no matter how many
connections there are or what type of amazing results they achieve.  In
particular I am referring to the result that Hecht-Nielson reports in his
paper on "Kolmogorov's Mapping Neural Network Theorem" [1987 INNS proceedings?].
There he describes a way of proving that a 2-layered net (one hidden layer)
is capable of solving any mapping problem.  However, the units in the
network are incredibly complex.  No longer are we dealing with units that
compute threshold functions.  The hidden layer units must be able to compute
any real, continuous monotonically increasing function, and the output layer
units must be able to compute any *arbitrary* real continuous function.
While the fact that a system like this can do some serious computation is
interesting (neat, even), it really tells us nothing about connectionist
networks.