Connectionists: AI at 50 videos...

[Barak A. Pearlmutter]

> Plate (2002): “Another equivalent property is that in a distributed representation one cannot interpret the meaning of activity on a single neuron in isolation: the meaning of activity on any particular neuron is dependent on the activity in other neurons (Thorpe, 1995).”

> Thorpe (1995, p. 550): “With a local representation, activity in individual units can be interpreted directly … with distributed coding individual units cannot be interpreted without knowing the state of other units in the network.”

> Elman (1995, p. 210): “These representations are distributed, which typically has the consequence that interpretable information cannot be obtained by examining activity of single hidden units.”

People blithely toss out statements like that, but local
uninterpretability does not follow from the definition of a
distributed representation.

It is very easy to construct distributed representations whose
components *can* be understood in isolation (the bits of a standard
binary representation of a natural number ≤2³², say) or whose
components cannot be understood in isolation (a code for a natural
number ≤2³² whose bits each have an associated set of half the numbers
in the domain picked at random, say).

The "big regions" distributed representation for points in 2D space
from the PDP books is locally interpretable.

Naturally, all other things being equal, given the choice we'd choose
distributed representations whose parts are interpretable in
isolation. Does the brain do that? Well, deep learning networks seem
to, and experimental data seems consistent with that notion.

--Barak Pearlmutter.