Thesis anouncement, again

[john kolen]

This is a reposting of my original announcement.  The orignal announcement
had the wrong ftp address.  (One of the benefits of being at OSU is that I
can cd to the neuroprose archive, hence, I don't bother with ftp addresses.)
In addition to the address change, I have added a long abstract to the
the announcement.  As still is the case, those with old PS printers and
previewers will have difficulties (see warning below.)

Ftp-site: ftp.cis.ohio-state.edu
          ^^^^
	  (archive still works, too)
Ftp-directory:  pub/neuroprose/Thesis/kolen.thesis.ps.Z
PrintedPages: 191


			  EXPLORING THE COMPUTATIONAL
		   CAPABILITIES OF RECURRENT NEURAL NETWORKS

				 DISSERTATION

				      By

				 John F. Kolen

		       Prof. Jordan B. Pollack, Adviser

				Short Abstract


While many researchers have successfully organized neural networks into
structures displaying universal computational capability, most have ignored the
more daunting endeavor of identifying ongoing computation, or information
processing, as it occurs.  My thesis addresses this problem as it relates to
the understanding of the information processing capabilities of recurrent
neural networks.  I have isolated three important facets of recurrent neural
networks that contribute to their computational power: internal dynamics, input
modulation, and output generation.  Theories of dynamical systems and iterated
function systems have proven crucial in developing an understanding of these
facets.  With respect to my original question of identifying ongoing
computation, the evidence suggests that the any observed computational powers
of neural networks arise not from the network itself, but our application of it
as a symbol processing device.  If we consider recurrent neural networks as the
a cognitive e. coli, this dissertation is a step toward understanding the
nature cognition and intelligence.


				 Long Abstract


This dissertation addresses the issues surrounding the computational
capabilities of recurrent neural networks. My results apply not only to simple
recurrent networks, Jordan networks, and higher order recurrent networks, but
many other networks implemented as input-parameterized iterated functions.

The following reasons have driven my efforts to understand their computational
capabilities. First, the question of knowledge content arises whenever we
attempt to understand how a given network produces its behavior. Second,
knowing the range of what is computable by a recurrent network can guide us in
their intelligent application.  Finally, this knowledge may also help us to
develop new training strategies which bias the network towards desirable
solutions.

While we already know that recurrent networks can perform complex computation
by simulating machine tapes and stacks, one problem still remains: someone
designed each universal-computing network by hand. We know the function
decomposition because the designer can tell us what they intended each part to
do. Unfortunately, weak learning methods, like back-propagation, that discover
operable network weights cannot explain the internal functionality of final
product. Thus, we are forced to externally determine the recurrent network's
computation process by observing its structure and behavior.

To this end, I identify three facets of recurrent networks that directly affect
their emergent computational descriptions: system dynamics, input modulation of
state dynamics, and output generation. System dynamics, the mapping of current
state to next state, have been traditionally considered the source of complex
behavior. Input modulation occurs as a finite set of input vectors and induces
beiterated function system-like behavior from the recurrent network. This
selection creates state space representations for information processing states
which display recursive structure. I show that the mechanism producing discrete
outputs has dramatic effects on the resulting system complexity by imposing
information processing regularities in the output stream strong enough to
manipulate both complexion (number of states) and generative class of the
observed computation.

As for new training methods, I outline a method of network training called
entrainment learning which offers a novel explanation of the transmission of
grammatical behavior structures between agents.


********* WARNING ***********
This document contains Level 2 PostScript commands that will not work on
Level 1 printers (i.e. old LaserWriters and old versions of ghostview).