Thesis available for ftp

[Tony Plate]

Ftp-host: ftp.cs.utoronto.ca
Ftp-filename: /pub/tap/plate.thesis.2up.ps.Z
Ftp-filename: /pub/tap/plate.thesis.ps.Z

The following thesis is available for ftp.  There are two
versions: plate.thesis.ps.Z prints on 216 sheets of
paper, and plate.thesis.2up.ps.Z prints on 108 sheets.
The compressed files are around 750Kb each.

	Distributed Representations and
	Nested Compositional Structure

		    by

		Tony A. Plate*
	Department of Computer Science,
	   University of Toronto,
		   1994

A thesis submitted in conformity with the requirements for
the degree of Doctor of Philosophy in the Graduate Department
of Computer Science at the University of Toronto.


		 Abstract

Distributed representations are attractive for a number of
reasons.  They offer the possibility of representing
concepts in a continuous space, they degrade gracefully with
noise, and they can be processed in a parallel network of
simple processing elements.  However, the problem of
representing nested structure in distributed representations
has been for some time a prominent concern of both
proponents and critics of connectionism
\cite{fodor-pylyshyn-88,smolensky-90,hinton-90}.  The lack
of connectionist representations for complex structure has
held back progress in tackling higher-level cognitive tasks
such as language understanding and reasoning.

In this thesis I review connectionist representations and
propose a method for the distributed representation of
nested structure, which I call ``Holographic Reduced
Representations'' (HRRs).  HRRs provide an implementation of
Hinton's~\shortcite{hinton-90} ``reduced descriptions''.
HRRs use circular convolution to associate atomic items,
which are represented by vectors.  Arbitrary variable
bindings, short sequences of various lengths, and
predicates can be represented in a fixed-width vector.
These representations are items in their own right, and can
be used in constructing compositional structures.  The noisy
reconstructions extracted from convolution memories can be
cleaned up by using a separate associative memory that has
good reconstructive properties.

Circular convolution, which is the basic associative
operator for HRRs, can be built into a recurrent neural
network.  The network can store and produce sequences.  I
show that neural network learning techniques can be used
with circular convolution in order to learn representations
for items and sequences.

One of the attractions of connectionist representations of
compositional structures is the possibility of computing
without decomposing structures.  I show that it is possible
to use dot-product comparisons of HRRs for nested structures
to estimate the analogical similarity of the structures.
This demonstrates how the surface form of connectionist
representations can reflect underlying structural similarity
and alignment.


* New Address:

Tony Plate,
Department of Chemical Engineering
University of British Columbia
2216 Main Mall, Vancouver,
BC, Canada V6T 1Z4
tap at chml.ubc.ca