Neural Network Seminar

[noordewi@cs.rutgers.edu]

			  RUTGERS UNIVERSITY
	    Dept. of Computer Science/Dept. of Mathematics

	  Neural Networks Colloquium Series --- Spring 1991

			     C. L. Giles
			NEC Research Institute

		  Teaching Recurrent Neural Networks
		to be Finite State Machines (Digraphs)

			       Abstract

Recurrent neural networks are natural models for encoding and learning
temporal sequences. If these temporal sequences are strings from the
languages of formal grammars, then teaching a neural network to learn
these sequences is a form of grammatical inference. We demonstrate how
to train second-order recurrent networks with real-time learning
algorithms to be finite state machines. In particular, we present
extensive simulation results which show that simple regular grammars
are easy to learn. We devise and use heuristic clustering algorithms
which extract finite state machines or digraphs from recurrent neural
networks during and after training. The resultant finite state
machines usually have large numbers of states and can be reduced in
complexity to minimal finite state machines using a standard
minimization algorithm. Depending on the training method and type of
training set, different minimal finite state machines emerge. If the
grammar is well learned, then identical finite state machines are
produced in the minimization process. These finite state machines
constitute an equivalence class of neural networks which covers
different numbers of neurons and different initial conditions. This
can be interpreted as a measure of how well a set of strings and its
generative grammar are learned. We present a video of the learning
process and show the emergent finite state machines during and after
training.


			    April 17, 1991
	       Busch Campus --- 4:30 p.m., room 217 SEC

		 host: Mick Noordewier (201/932-3698)
   finger noordewi at cs.rutgers.edu for further schedule information