neuroprose paper on 2nd derivatives and their use in BP

[Wray Buntine]

The following paper is available by public ftp from Jordan Pollack's 
wonderful neuroprose collection.  Details below.

------------------

	Computing Second Derivatives in Feed-Forward Networks: a Review

	Wray L. Buntine 	 		Andreas S. Weigend
	RIACS & NASA Ames Research Center	Xerox PARC

	To appear in IEEE Trans. of Neural Networks

				Abstract.  

The calculation of second derivatives is required by recent training
and analyses techniques of connectionist networks, such as the
elimination of superfluous weights, and the estimation of confidence
intervals both for weights and network outputs.  We here review and
develop exact and approximate algorithms for calculating second
derivatives.  For networks with $|w|$ weights, simply writing the full
matrix of second derivatives requires $O(|w|^2)$ operations.  For
networks of radial basis units or sigmoid units, exact calculation of
the necessary intermediate terms requires of the order of $2h+2$
backward/forward-propagation passes where $h$ is the number of hidden
units in the network.  We also review and compare three approximations
(ignoring some components of the second derivative, numerical
differentiation, and scoring).  Our algorithms apply to arbitrary
activation functions, networks, and error functions (for instance,
with connections that skip layers, or radial basis functions, or
cross-entropy error and Softmax units, etc.).

-----------------------------

The paper is buntine.second.ps.Z in the neuroprose archives.
The INDEX sentence is
	A review of computing second derivatives in feed-forward networks.

To retrieve this file from the neuroprose archives:

unix> ftp cheops.cis.ohio-state.edu
Name (cheops.cis.ohio-state.edu:wray): anonymous
Password: (use your email address)
ftp> cd pub/neuroprose
ftp> get buntine.second.ps.Z
200 PORT command successful.
150 Opening BINARY mode data connection for buntine.second.ps.Z
.
ftp> quit
221 Goodbye.
unix> uncompress buntine.second.ps.Z
unix> lpr buntine.second.ps

----------------------

If you cannot ftp or print the postscript file, please
send email to   silva at parc.xerox.com   or write to

Nicole Silva
Xerox PARC		
3333 Coyote Hill Rd	
Palo Alto, CA 94304	
USA