Another Point of View on what Connectionism REALLY is ...

[thanasis kehagias]

I am sorry that I show up a little late in this "what connectionism
really is (should be)?" debate - I want to present another point of  view
and this really is a different discussion. But it got started in my mind
by reading the latest debate and at the same time writing a paper.

It is my impression that historically connectionism started as
something and developed into something quite different. Researchers
in the late 50's to (even) early 80's were mostly focusing on building
intelligent/thinking networks. However, for many reasons, not least among
them being the proverbial bandwagon effect and the increase of funds
available for connectionist research, many "computationally" oriented
researchers started conducting connectionist research.

I repeat that all this is personal opinion. I should also say that
I consider myself to be one of the more "computationally" oriented
researchers. I will now describe why neural nets are an interesting
subject to me, I suspect this is the case for other researchers.

My interest in neural nets is NOT biologically motivated. It is an
interesting  conjecture that systems that look "like" the brain will
behave "intelligently", but I do not feel qualified to pursue this.
It is a good thing that there are people pursuing this direction, as well
as (other ?) people trying to model  actual brains.

But neural nets are also parallel, distributed computers. Some advantages
of PDP are obvious, e.g.  faster  computing, and other somewhat  not so
obvious, e.g. robustness of computing properties with respect to local
damage. This is what I find really interesting in connectionism, and I
believe  some people agree with me.

This point of view has an important implication. We no longer need
to insist that a network develops its own internal representations without
any external guidance. It's OK to try to incorporate in the network/algorithm
as much knowledge and design as we can. It's also OK to look into other
disciplines (statistics, stochastic control) to see what methods they
have developed and try to incorporate them into our networks. What really
counts is to make the algorithm fast and efficient.

Is this really connectionism? Is it any different from, say, the
art of parallel algorithms? I believe the answer is yes to both
questions. The Book has "Parallel Distributed Computing" in its title
and we are all looking at parallel and distributed networks. And, even if
boundaries are blurry, connectionism  has the distinct flavor of building
things up from small, simple units, in a way that is different from,
say, a parallel implementation of Kalman filtering.

In conclusion, I repeat that the boundaries between connectionism and
a number of other disiplines, e.g. parallel algorithms, cellular
automata, statistics etc. are blurry. This is not a disadvantage, on
the contrary, I think, we would benefit from taking a look at these
disciplines and comparing their point of view at problems similar to
the ones we are examining.