No subject

[Lina Massone]

About distributed representations

The concept of distributed representation is intimately related to the
concept of redundancy. The central nervous system makes a great use of
redundant representations in the way receptive/projective fields are
organized.
I do not agree on the fact that distributed/redundant
representations are primarily a protection against possible injuries
or failures of the components; I'd rather consider that as a useful
side-effect. To me the main values of redundancy are: greater sensitivity,
higher resolution, improvement of signal-to-noise ratio, reduction of
demand for stability of performance and for precision in ontogenesis.
In general a comparison between the activity of a population of neurons and
the activity of a single neuron will show that the population is sensitive
to lower stimulus intensities, smaller increments, briefer events, higher
frequencies, wider dynamic ranges than a single neuron and is less
disturbed by independent drift and instability.
As far as the amount of redundancy, there is some physiological evidence
that the coding of information in the CNS is a compromise between fully
distributed and fully localized. Given that the available number of neurons
is limited, an entity (a piece of information) cannot be represented over a
very large population of neurons that overlaps almost completely with the
population activated by a different entity; this would cause a high degree
of interference and would correspond to a very inefficient memory storage
system. To maintain some degree of orthogonality within a limited number of
neurons, the CNS makes the number of neurons - active for each stimulus -
low. In other words each entity is represented across an ensemble of
neurons but the ensemble is of limited size. 
As far as coarse coding, Ken Laws raised the issue of matching the
structure of data with the code. I agree on that. The CNS 
does that by having neighboring receptors stimulated by neighboring fractions
of the impinging world, i.e. by means of a topological principle. An example
of the computational advantages of this idea for control problems is given in

L. Massone, E. Bizzi (1990) On the role of input representations in
sensorimotor mapping, Proc. IJCNN, Washington D.C.


Lina Massone