Reprint: Rule Refinement with Recurrent Neural Networks

[Lee Giles]

The following reprint is available via the NEC Research
Institute ftp archive external.nj.nec.com. Instructions for
retrieval from the archive follow the summary.

----------------------------------------------------------------------------------


                 "Rule Refinement with Recurrent Neural Networks"

                    C. Lee Giles(a,b) and Christian W. Omlin(a,c)

       (a) NEC Research Institute, 4 Independence Way, Princeton, NJ 08540
(b) Institute for Advanced Computer Studies, U. of Maryland, College Park, MD 20742
 (c) Computer Science Department, Rensselaer Polytechnic Institute, Troy, NY 12180



                                  ABSTRACT
Recurrent neural networks can be trained to behave like deterministic finite-state 
automata (DFA's) and methods have been developed for extracting grammatical rules from 
trained networks. Using a simple method for inserting prior knowledge of a subset of 
the DFA state transitions into recurrent neural networks, we show that recurrent neural 
networks are able to perform rule refinement. The results from training a recurrent neural 
network to recognize a known non-trivial, randomly generated regular grammar show that not 
only do the networks preserve correct prior knowledge, but that they are able to correct 
through training inserted prior knowledge which was wrong. (By wrong, we mean that the 
inserted rules were not the ones in the randomly generated grammar.)

-------------------------------------------------------------------------------------

                          FTP INSTRUCTIONS

                unix> ftp external.nj.nec.com (138.15.10.100)
                Name: anonymous
                Password: (your_userid at your_site)
                ftp> cd pub/giles/papers
                ftp> binary
                ftp> get rule_refinement.ps.Z
                ftp> quit
                unix> uncompress rule_refinement.ps.Z


----------------------------------------------------------------------------------------

--                                 
C. Lee Giles / NEC Research Institute / 4 Independence Way
Princeton, NJ 08540 / 609-951-2642 / Fax 2482
==