Connectionist symbol processing: any progress?

[Bryan B. Thompson]

Tony Plate's response is interesting and I, for one, will have to give
it some thought.  I am not certain that

> concepts, respectively.  For example, one can build a distributed
> representation for a shape configuration#33 of "circle above
> triangle" as: config33 = vertical + circle + triangle > +
> ontop*circle +
>   below*triangle > > By using an appropriate multiplication
>   operation (I used > circular, or wrapped, convolution), the
>   reduced > representation of the compositional concept (e.g.,
>   config33) > has the same dimension as its components, and can
>   readily be > used as a component in other higher-level relations.
>   Quite

is inherently different from a spatial approach and, hence, a localist
approach itself.  You need to have enough dimensionality to represent
the key features as well as enough to multiply them out by the key
relational features -- quite a few dimensions, even if some of that
dimensionality is pushed off into numerical precision.  It sounds
suspiciously like a localist (i.e., locally spatial) encoding.
Frankly, I imagine that even a temporal encoding must be localist if
it is to show "symbolic processing" behavior.  That is, the temporal
encoding must be striated with patterned regions that are, themselves,
interpretable elements -- compositionality in time vs space.

If I am willing to call both temporal encoding and spatial encoding
schemes localist, then what would I consider "distributed?"  To the
extent which this is a meaningful distinction, I would have to say
that "distributed" refers to the equi-presence of the encoding of an
entity or compositional relation among all elements of the
representation, e.g., equally present in all internal variables in a
recurrent network.  This is perhaps the intent of people who point to
"distributed" representations and say that they can only encode a
single entity at a time.  When such systems are forced to encode
compositional representations, they are also forced to develop
decidedly non-equal distributions of the information across the
elements of the representation.  That is, they *must* become localist
in time or in space to encode things compositionally.  If this line of
conjecture is correct, then localist and distributed are simple the
ordinate directions on an axis of representation that reflects the
compositionality of information, and spatial / temporal are the
ordinate directions of an orthogonal axis reflecting how information
is encoded within a fixed set of resources.  Clearly this sense of
distributed vs localist is directly tied to the connectivity of the
network and the degree to which weights are global vs local.  Another
"upside" of localism, however achieved, is that is results in
structured credit assignment -- weight or dynamics changes exert only
a localized influence on the network behavior and do not disturb
unrelated dynamics.


My challenge for spatial encoding schemes is that they seem profoundly
challenged by metaphor.  For example, "Life is like a garden."  This
saying, when considered, immediately enacts a deep correspondence, an
*invariance*, between two different *sets* of systematic relations
(each defined over a different set of entities).  If relations are
spatially encoded, then it is beyond me how such systematic
correspondences can be enacted by the dynamic activation of a single
new relation.  As I consider the ways in which I relate to a garden,
the metaphor expands for me systematically parallel ways in which I
may relate to life as well.  For example, you sow seeds, tend them,
and harvest nourishing rewards.  The seeds become metaphorical, e.g.,
as new beginnings, and the parallel yields an interpretation in
"life".  (Other inferences which can be systematically drawn -- it
takes a lot of "fertilizer" to grow anything :} and sometimes I can't
tell which is the weed and which is the seedling.)

If we allocate spatial encoding to systematic relations, then how can
we apply those systematic relations to new semantics -- both
"instantly" and without loss of the original interpretations?  In
fact, our understanding typically grows for both domains illuminated
by the metaphor.  For me, a temporal (vs spatial) encoding does not
help.  I would expect a temporal encoding to have developed a
topology, upon whose relative stasis the system is equally dependent
to draw out meanings.  It seems, to me, that another level of
indirection may be required to map onto one another such previously
distinct systematic relations.

On the other hand, perhaps such inferences "by metaphor" are not as
automatic as I might believe.  It that case it becomes more plausible
to see these as a metalevel in which systematic correspondences are
established between bindings in the two realms of metaphor.  Then,
within those binding legitimizing invariances, systematic relations
from one domain may readily apply to the other and our directed
attention, or wandering gaze, is used to draw out new inferences from
within one domain or the other.

--bryan thompson

PS: I will be on vacation next week (Aug 17-21) and will be unable to
answer any replies until I return.