technical report announcement - rule extraction

[Christian Omlin]

Dear Connectionists,

The following technical report (see abstract attached below)

	A. Vahed, C.W. Omlin, "Rule Extraction from Recurrent
        Neural Networks using a Symbolic Machine Learning Algorithm"
         
is available from 

	http://www.cs.sun.ac.za/~omlin/papers/iconip_99.paper.ps.gz

This paper contains preliminary results and we welcome any comments
you may have.

Thank you.

Best regards,

Christian

Christian W. Omlin                 e-mail: omlin at cs.sun.ac.za
Department of Computer Science     phone (direct): +27-21-808-4210
University of Stellenbosch         phone (secretary): +27-21-808-4232
Private Bag X1                     fax: +27-21-808-4416
Stellenbosch 7602                  http://www.cs.sun.ac.za/people/staff/omlin
SOUTH AFRICA                       http://www.neci.nj.nec.com/homepages/omlin 


------------------------------- cut here --------------------------------


	       Rule Extraction from Recurrent Neural Networks
                using a Symbolic Machine Learning Algorithm


                  A. Vahed                         C.W. Omlin
       Department of Computer Science    Department of Computer Science
       University of the Western Cape      University of Stellenbosch
               7535 Bellville                   7600 Stellenbosch 
                South Africa                       South Africa
               avahed at uwc.ac.za                 omlin at cs.sun.ac.za


       This  paper  addresses the extraction of knowledge from recurrent
       neural networks trained to behave like deterministic finite-state
       automata  (DFAs). To date, methods used to extract knowledge from
       such networks have relied on the hypothesis that networks  states
       tend to cluster and that clusters of network states correspond to
       DFA states.  The computational complexity of such a cluster anal-
       ysis has led to heuristics which either limit the number of clus-
       ters that may form during training or limit  the  exploration  of
       the output space of hidden recurrent state neurons. These limita-
       tions,  while  necessary, may  lead to  decreased  fidelity, i.e.
       the  extracted  knowledge  may  not  model the true behavior of a
       trained network, perhaps not even  for  the  training  set.   The
       method  proposed  here  uses a polynomial-time, symbolic learning
       algorithm to infer DFAs solely from the observation of a  trained
       network's input/output behavior. Thus, this method has the poten-
       tial to increase the fidelity of the extracted knowledge.