paper available in neuroprose

[arun maskara spec lec cis]

Paper availble:

 
Forced Simple Recurrent Neural Networks and Grammatical Inference

		Arun Maskara
	New Jersey Institute of Technology
	Department of Computer and Information Sciences
	University Heights, Newark, NJ 07102
	arun at hertz.njit.edu

		Andrew Noetzel
	The William Paterson College
	Department of Computer Science
	Wayne, NJ 07470


			ABSTRACT

A simple recurrent neural network (SRN) introduced by Elman can be trained to
infer a regular grammar from the positive examples of symbol sequences generated
by the grammar. The network is trained, through the back-propagation of error,
to predict the next symbol in each sequence, as the symbols are presented 
successively as inputs to the network. The modes of prediction failure of 
the SRN architecture are investigated. The SRN's internal encoding of the
context (the previous symbols of the sequence) is found to be insufficiently
developed when a particular aspect of context is not required for the immediate 
prediction at some point in the input sequence, but is required later. It is
shown that this mode of failure can be avoided by using the auto-associative
recurrent network (AARN). The AARN architecture contains additional output
units, which are trained to show the current input and the current context. 

The effect of the size of the training set for grammatical inference is also
considered.  The SRN has been shown to be effective when trained on an infinite
(very large) set of positive examples. When a finite (small) set of positive
training data is used, the SRN architectures demonstrate a lack of
generalization capability. This problem is solved through a new training
algorithm that uses both positive and negative examples of the sequences. 
Simulation results show that when there is restriction on the number of nodes
in the hidden layers, the AARN succeeds in the cases where the SRN fails.

----------------------------------------------------------------------------


This paper will appear in Proceedings of 14th Annual Coggnitive Science
Conference, 1992

It is FTPable from archive.cis.ohio-state.edu 
  in: pub/neuroprose (Courtesy of Jordan Pollack)

Sorry, no hardcopy available.


FTP procedure:
        unix> ftp archive.cis.ohio-state.edu (or 128.146.8.52)
        Name: anonymous
        Password: (your email address)
        ftp> cd pub/neuroprose
        ftp> binary
        ftp> get maskara.cogsci92.ps.Z
        ftp> quit
        unix> uncompress maskara.cogsci92.ps.Z
        unix> lpr -Pxxx maskara.cogsci92.ps (or however you print postscript)


Arun Maskara
arun at hertz.njit.edu