TR Available: Recurrent Neural Networks and Dynamical Systems

[Lee Giles]

The following Technical Report is available via the University of Maryland 
Department of Computer Science and the NEC Research Institute archives:

_____________________________________________________________________________



             "Finite State Machines and Recurrent Neural Networks --
                 Automata and Dynamical Systems Approaches"


    UNIVERSITY OF MARYLAND TECHNICAL REPORT UMIACS-TR-95-1 and CS-TR-3396

            Peter Tino[1,2], Bill G. Horne[2], C. Lee Giles[2,3]  
   [1] Dept. of Informatics and Computer Systems, Slovak Technical University,
                Ilkovicova 3, 812 19 Bratislava, Slovakia
    [2] NEC Research Institute, 4 Independence Way, Princeton, NJ  08540
         [3] UMIACS, University of Maryland, College Park, MD 20742
          
                   {tino,horne,giles}@research.nj.nec.com
                

We present two approaches to the analysis of the relationship between a
recurrent neural network (RNN) and the finite state machine M the network 
is able to exactly mimic. First, the network is treated as a state machine 
and the relationship between the RNN and M is established in the context of
algebraic theory of automata. In the second approach, the RNN is viewed as 
a set of discrete-time dynamical systems associated with input symbols of M.
In particular, issues concerning network representation of loops and cycles 
in the state transition diagram of M are shown to provide a basis for the
interpretation of learning process from the point of view of bifurcation
analysis. The circumstances under which a loop corresponding to an input 
symbol x is represented by an attractive fixed point of the underlying 
dynamical system associated with x are investigated. For the case of two 
recurrent neurons, under some assumptions on weight values, bifurcations can 
be understood in the geometrical context of intersection of increasing and 
decreasing parts of curves defining fixed points. The most typical bifurcation 
responsible for the creation of a new fixed point is the saddle node bifurcation. 


-------------------------------------------------------------------------

http://www.neci.nj.nec.com/homepages/giles.html
http://www.cs.umd.edu/TRs/TR-no-abs.html

or

ftp://ftp.nj.nec.com/pub/giles/papers/UMD-CS-TR-3396.fsm-rnn.dynamics.systems.ps.Z

--------------------------------------------------------------------------


--                                 
C. Lee Giles / NEC Research Institute / 4 Independence Way
Princeton, NJ 08540, USA / 609-951-2642 / Fax 2482
URL  http://www.neci.nj.nec.com/homepages/giles.html
==