Paper available

[Georg Thimm]

FTP-host: archive.cis.ohio-state.edu
FTP-file: pub/neuroprose/thimm.gain.ps.Z

6 pages, compressed file size: 63685 KB

The file thimm.gain.ps.Z is now available for copying from the
Neuroprose archive:


	  The Interchangeability of Learning Rate and  Gain
		  in Backpropagation Neural Networks

		G. Thimm, P. Moerland, and E. Fiesler

			      Abstract:
The backpropagation algorithm is widely used for training multilayer
neural networks. In this publication the gain of its activation
function(s) is investigated. In specific, it is proven that changing
the gain of the activation function is equivalent to changing the
learning rate and the weights. This simplifies the backpropagation
learning rule by eliminating one of its parameters.  The theorem can
be extended to hold for some well-known variations on the
backpropagation algorithm, such as using a momentum term, flat spot
elimination, or adaptive gain.  Furthermore, it is successfully
applied to compensate for the non-standard gain of optical sigmoids
for optical neural networks.


Keywords: 
neural network, neural computation, neural computing, connectionism,
neurocomputing, multilayer neural network, backpropagation, (sigmoid)
steepness, gain, slope, temperature, adaptiv e gain, (steep)
activation function, (adaptive) learning rate, initial weight,
momentum, flat spot elimination, weight discretization, threshold,
bias, optical implementation.


Please do not reply directly to this message.

FTP INSTRUCTIONS:

unix> ftp archive.cis.ohio-state.edu (or 128.146.8.52)
    Name: anonymous
    Password: <your e-mail address>
    ftp> cd pub/neuroprose
    ftp> binary
    ftp> get thimm.gain.ps.Z 
    ftp> quit
unix> zcat thimm.gain.ps.Z |lpr

This paper is also available from the URL
http://www.idiap.ch/pub/papers/neural/thimm.gain.ps.Z

Sorry, no paper copies available.

Regards,
	Georg Thimm


--------------------------------------------------------------
Georg Thimm                            E-mail: thimm at idiap.ch
Institut Dalle Molle d'Intelligence    Fax:  ++41 26 22 78 18
Artificielle Perceptive (IDIAP)        Tel.: ++41 26 22 76 64
Case Postale 592		       WWW: http://www.idiap.ch
1920 Martigny / Suisse
--------------------------------------------------------------