No subject

[Graham Smith]

Subject: Dynamic Binding
Cc: neuron-request at edu.upenn.psych.cattell graham


It strikes me that an obvious solution to the binding problem has been
overlooked in our rush to study phase locking in oscillatory networks. ;-)

I have recently trained a simple multi-layer feed-forward network using
back-propagation, to auto-associate patterns which simultaneously describe
two items. The patterns consist of four features (red, blue, square,
triangle) which are enumerated over the two items. The domain allows 16 two
item patterns (e.g. "red square and blue triangle" or "blue triangle and
blue square") and 4 single item patterns. The network had 8 input units, 4
hidden units and 8 output units. It was successfully trained to
auto-associate 15 of the 20 patterns and was able to correctly
auto-associate the 5 previously unseen patterns. This result is
unsurprising, the network has simply learned the regularities of the set of
bit patterns. But I will argue that the network can be described at the
symbolic level as performing dynamic binding.

Binding does not occur at either the input or output layer as both of these
representations are enumerated. However, a hidden layer activation pattern
is a transformation of the input pattern which contains sufficient
information to allow its transformation back to the original by the hidden
to output layer of weights. Such descriptions are dubbed holistic
representations by RAAM enthusiasts. Furthermore, van Gelder argues that
holistic representations are functionally compositional and that truely
connectionist representations have functional rather than concatenative
compositionality. Phase synchrony is a concatenative compositional putative
binding mechanism and is a hybrid approach rather than connectionist.

The holistic representation of "red square and blue triangle" is not
ambiguous. It could not be confused for the holistic representation of
"blue square and red triangle".  The holistic representation is performing
binding. To be more accurate, binding does not literally take place at the
subsymbolic level. No variables are concatenatively bound to constants.
Subsets of features are not "glued" together. Rather dynamic binding is a
symbolic level approximate description of the subsymbolic process and
subsymbolic "binding" is a functionally compositional state-space
representation.

I hope to publish the above-mentioned work but before doing so I shall be
grateful for some feedback either to reassure myself that there is
something here worth publishing or to spare my blushes with a wider
audience.

Graham Smith
Centre for Intelligent Systems
University of Plymouth
England