Technical Report Available

[Jordan Holt]

Technical Report Available:



		    Finite Precision Error Analysis of Neural
			Network Hardware Implementations

			Jordan Holt, Jenq-Neng Hwang

The high speed desired in the implementation of many neural network algorithms,
such as back-propagation learning in a multilayer perceptron (MLP), may be  
attained through the use of finite precision hardware.  This finite precision 
hardware; however, is prone to errors.  A method of theoretically deriving and 
statistically evaluating this error is presented and could be used as a guide 
to the details of hardware design and algorithm implementation.  The paper is 
devoted to the derivation of the techniques involved as well as the details of 
the back-propagation example.  The intent is to provide a general framework by 
which most neural network algorithms under any set of hardware constraints may 
be evaluated.

Section 2 demonstrates the sources of error due to finite precision computation
and their statistical properties.  A general error model is also derived by 
which an equation for the error at the output of a general compound operator 
may be written.  As an example, error equations are derived in Section 3 for 
each of the operations required in the forward retrieving and error back-
propagation steps of an MLP.  Statistical analysis and simulation results of 
the resulting distribution of errors for each individual step of an MLP are 
also included in this section.  These error equations are then integrated, in 
Section 4, to predict the influence of finite precision computation on several
stages (early, middle, final stages) of back-propagation learning.  Finally,  
concluding remarks are given in Section 5.

----------------------------------------------------------------------------

The report can be received by ftp:

             unix>         ftp cheops.cis.ohio-state.edu

             Name:         anonymous
             Guest Login ok, send ident as password
             Password:     neuron
             ftp>          binary
             ftp>          cd pub
             ftp>          cd neuroprose
             ftp>          get holt.finite_error.ps.Z
             ftp>          bye
             
             unix>         uncompress holt.finite_error.ps
             unix>         lpr holt.finite_error.ps