Preprint about scaled conjugate gradient available.

[Martin Moller]

*************************************************************
******************** PREPRINT announcement: ****************

    A Scaled Conjugate Gradient Algorithm for Fast Supervised Learning.


		Martin F. Moller
		Computer Science Dept.
		University of Aarhus
		Denmark
		e-mail: fodslett at daimi.aau.dk


Abstract-- 

A supervised learning algorithm (Scaled Conjugate Gradient, SCG)
with superlinear convergence rate is introduced. SCG uses second order 
information from the neural network, but requires only O(N) memory usage.
The performance of SCG is benchmarked against the performance of the standard
backpropagation algorithm (BP) and several recently proposed standard conjugate
gradient algorithms. SCG yields a speed-up at least an order of magnitude 
relative to BP. The speed-up depends on the convergence criterion,i.e., the
bigger demand for reduction in error the bigger the speed-up. SCG is fully
automated including no user dependent parameters and avoids a time consuming 
line search, which other conjugate gradient algorithms use in order to 
determine a good step size.
Incorporating problem dependent structural information in the architecture of 
a neural network often lowers the overall complexity. The smaller the 
complexity of the neural network relative to the problem domain, the bigger the
possibility that the weights space contains long ravines characterized by sharp
curvature. While BP is inefficient on these ravine phenomena, SCG handles them 
effectively.



The paper is available by ftp in the neuroprose directory under the name:

	moller.conjugate-gradient.ps.Z

Any question or comments on this short writing or on the preprint would 
be very much appriciated.


Martin M.