preprint

[Benny Lautrup]

The following preprint is now available:

FTP-host: archive.cis.ohio-state.edu
FTP-file: pub/neuroprose/hertz.nonlin.ps.Z

Authors:  J. Hertz, A. Krogh, B. Lautrup and T. Lehmann

Title: Non-Linear Back-propagation: Doing Back-Propagation without
       Derivatives of the Activation Function.

Size:  13 pages

Abstract:

The conventional linear back-propagation algorithm is replaced by a  non-linear
version, which avoids the necessity  for  calculating  the  derivative  of  the
activation function. This may be exploited in hardware realizations  of  neural
processors. In this paper we derive the non-linear back-propagation  algorithms
in the framework of  recurrent  back-propagation  and  present  some  numerical
simulations of feed-forward networks on the NetTalk problem.  A  discussion  of
implementation in analog VLSI electronics concludes the paper.