Distributed Representations

[Ali Ahmad Minai]

We connectionists never tire of talking about "distributed representations",
and with good reason. However, I have never come across a rigorous definition
of the concept. Now, I realize that this notion, like most powerful ones,
will necessarily be diminished in any process of definition, however inclusive
that might be. That has not fazed us in trying to define entropy, information,
complexity, learnability --- and probability! My question is: has anyone
rigorously, or even empirically, tried to come up with a definition for
distributed representations --- especially a way to quantify distributed-ness?
I suppose high-order statistics represent a way to look at this, but have
there been any attempts to develop a definition specifically in the context
of connectionist networks? And would that be such a bad thing?


Ali Minai
Dept of EE
University of Virginia
aam9n at Virginia.EDU