Distributed vs. local representation

[Bo Xu]

Following I would like to state my views on the distributed and local
representations.  All comments are more than welcome.
 
I think that if we define a strict local representation as: "one object
(or item, entity, etc.) is represented by one node (or unit, neuron, etc.)
only, and one node represents only one object", then all the other situations
probably can be classified as distributed representation (either semi- or
full-distributed).  In other words, only the one-to-one representation
belongs to local representation.  The others, multiple-to-one, one-to-
multiple, and multiple-to-multiple representations all belong to distributed
representation.  Therefore, the distributed representation has more senses
than local.  This may help reduce the confusion regarding these definitions.
 
Because distributed representation covers more range than local, there are
many different appearance in distributed representation.
 
One point unnoticed up to now is the difference between the "binary
representation" (the node takes binary values only) and the "analog
representation" (the node takes analog values).  In NETtalk and many other
examples, the distributed representation used seems to be the binary one.
However, the world seems favoring and is taking the analog form.  Therefore,
analog distributed representation probably is the one that is working and
dominating our cognitive processes.  I met one such problem in our work on
parabolic problem.  We found that if it was not impossible, it would be very
difficult to use (strict) local or binary distributed representation to
solve the parabolic problem.  It was only the analog distributed representation
that worked well.  We concluded that from the practical application viewpoint,
local and the distributed representation all would work if the training and
test patterns were discrete and finite.  However, if the training and/or test
patterns were continuous and infinite, only distributed representation worked.
 
-Bo