Distributed Representations

[Steve Lehar]

I think the essence of this debate is in the nature of the input data.
If your  input is  boolean in  nature  and reliably correct,  then the
processing  performed  on it can be  similarly boolean and  sequential
with a great saving in time and space.  It is when the input is fuzzy,
ambiguous and  distributed that  the  sequential logical  boolean type
of processing runs into problems.

A perfect example is  image understanding.  No  single local region of
the image  is sufficient   for  reliable  identification.    Try  this
yourself- punch a little hole in a big piece of  paper and lay it on a
randomly selected photograph    and see  how  much  you  can recognize
through that one local aperture.  You have no way of  knowing what the
local feature is without the global context, but  how do  you know the
global context  without   building   it up  out of   the local pieces?
Studies of  the visual system  suggest that  in nature this problem is
solved by a parallel optimization of all  the local pieces in parallel
together with many levels  of  global representations, such  that  the
final   interpretation is a   kind of relaxation due    to all  of the
constraints felt at  all of  the different representations all  at the
same time.   This is the basic  idea of Grossberg's BCS/FCS algorithm,
and is in contrast to a more  sequential "AI" approach where the local
pieces are each evaluated independantly, and the  results passed on to
the next stage.  I would  claim that such an  approach can never  work
reliably with natural images.

I would  be happy to  provide more  information on the BCS/FCS  and my
implementations of it to interested parties.