Connectionist symbol processing: any progress?

[Tony Plate]

Bryan B. Thompson writes:

> ... I am not certain that
>
>    [snip description of my scheme ...]
>
>is inherently different from a spatial approach and, hence, a localist
>approach itself.  You need to have enough dimensionality to represent
>the key features as well as enough to multiply them out by the key
>relational features -- quite a few dimensions [snip ...]
>
>If I am willing to call both temporal encoding and spatial encoding
>schemes localist, then what would I consider "distributed?"  To the
>extent which this is a meaningful distinction, I would have to say
>that "distributed" refers to the equi-presence of the encoding of an
>entity or compositional relation among all elements of the
>representation, e.g., equally present in all internal variables in a
>recurrent network.  [snip ...]

Actually, Holographic Reduced Representations (HRRs) are an
"equi-present" code -- everything represented is represented
over all of the units.  Suppose you have a vector X which represents
some structure.  Then you can take just the first half of X, and
it will also represent that structure, though it will be noisier.
This same property is shared by Kanerva's binary spatter-code and
may be shared by some of the codes Ross Gayler has been developing.

The dimensionality required is high -- for HRRs it's in the
hundreds to thousands of elements.  But, HRRs have an interesting
scaling property -- toy problems involving a just a couple dozen
relations might require a dimensionality of 1000, but the
dimensionality doesn't need to increase much (to 2 or 4 thousand)
to handle problems involving tens of thousands of relations.

Yes, I agree fully that metaphor and analogy are intriguing
examples of structural processing, and I believe it could be very
fruitful to investigate connectionist processing for them.

Tony Plate