Connectionist symbol processing---any progress?

[Whitney Tabor]

David Touretzky wrote:

>    I concluded that connectionist symbol processing had reached a
> plateau, and further progress would have to await some revolutionary
> new insight about representations. 

and mentioned Tony Plate's inspiring work on Holographic Reduced
Representations (HRRs). 

I don't think that anybody has yet mentioned the related line of work
which sets aside the learning problem for the moment and focuses on
the geometry of the trajectories of metric (and vector) space
computers, including many connectionist networks.  One essential idea
(proposed in embryonic form in Pollack, 1987, 1991, Siegelmann and
Sonntag, 1991, Wiles and Elman, 1995, Rodriguez, Wiles, and Elman, to
appear) is to use fractals to organize recursive computations in a
bounded metric space.  Cris Moore (Moore, 1996) provides the first
substantial development of this idea, relating it to the traditional
practice of classifying machines based on their computational power.
He shows, for example, that every context free language can be
recognized by some "dynamical recognizer" that moves around on an
elaborated, one-dimensional Cantor Set.  I have described a similar
method which operates on high-dimensional Cantor sets and thus leads
to an especially natural implementation in neural hardware (Tabor,
1998, submitted).

	This approach sheds some new light on the symbolic vs. metric
space computation question by showing how we can use the structured
entities recognized by traditional computational theory (e.g.
particular context free grammars) as bearing points in navigating the
larger set (Siegelman, 1996; Moore, 1996) of computing devices
embodied in many analog computers.  To my knowledge, no one has tried
to use this kind of computational/geometric perspective to interpret
HRRs and related outer product representations---I think this would be
a very rewarding project.

Whitney Tabor

University of Connecticut
http://www.cs.cornell.edu/home/tabor/tabor.html


@UNPUBLISHED{Pollack:87,
  AUTHOR = {Jordan B. Pollack},
  TITLE = {On Connectionist Models of Natural Language Processing},
  NOTE = {Ph.D. Thesis, Department of Computer Science, University of Illinois},
  YEAR = {1987},
  }

@ARTICLE{S&S:91,
  AUTHOR = {H. T. Siegelmann and E. D. Sontag},
  TITLE = {Turing computability with neural nets},
  JOURNAL = {Applied Mathematics Letters},
  YEAR = {1991},
  VOLUME = {4},
  NUMBER = {6},
  PAGES = {77-80},
  }

@ARTICLE{Pollack:91,
  AUTHOR = {Jordan B. Pollack},
  TITLE = {The Induction of Dynamical Recognizers},
  JOURNAL = {Machine Learning},
  YEAR = {1991},
  VOLUME = {7},
  PAGES = {227-252},
  }

@INCOLLECTION{W&E:95,
  AUTHOR = {Janet Wiles and Jeff Elman},
  TITLE = {Landscapes in Recurrent Networks},
  BOOKTITLE = {Proceedings of the 17th Annual Cognitive Science Conference},
  EDITOR = {Johanna D. Moore and Jill Fain Lehman},
  PUBLISHER = {Lawrence Erlbaum Associates},
  YEAR = {1995},
  }

@ARTICLE{R&W&E:ta,
  AUTHOR = {Paul Rodriguez and Janet Wiles and Jeffrey Elman},
  TITLE = {How a Recurrent Neural Network Learns to Count},
  JOURNAL = {Connection Science},
  YEAR = {ta},
  VOLUME = {},
  NUMBER = {},
  PAGES = {},
  }

@UNPUBLISHED{Moore:96b,
  AUTHOR = {Christopher Moore},
  TITLE = {Dynamical Recognizers:  Real-time Language Recognition by
		  Analog Computers},
  NOTE = {TR No. 96-05-023, Santa Fe Institute},
  YEAR = {1996},
  }

@ARTICLE{Siegelmann:96,
  AUTHOR = {Hava Siegelmann},
  TITLE = {The simple dynamics of super {T}uring theories},
  JOURNAL = {Theoretical Computer Science},
  YEAR = {1996},
  VOLUME = {168},
  PAGES = {461-472},
  }
@UNPUBLISHED{Tabor:98,
  AUTHOR = {Whitney Tabor},
  TITLE = {Dynamical Automata},
  NOTE = {47 pages.  Tech Report \# 98-1694.  Department of Computer Science,
Cornell University.  Download from http://cs-tr.cs.cornell.edu/},
  YEAR = {1998},
  }

@UNPUBLISHED{Tabor:subb,
  AUTHOR = {Whitney Tabor},
  TITLE = {Context Free Grammar Representation in Neural Networks},
  NOTE = {7 pages.  Draft version available at 
http://simon.cs.cornell.edu/home/tabor/papers.html},
  YEAR = {submitted to NIPS},
  }