Tech Report available

[Scott.Fahlman@B.GP.CS.CMU.EDU]

The following CMU Computer Science Dept. Tech Report is now available.  If
you want a copy, please send your request by computer mail to
"catherine.copetas at cs.cmu.edu", who handles our tech report distribution.
Indicate that you want "CMU-CS-88-162" and be sure to include a physical
mail address.

Try not to send your request to the whole connectionists mailing list --
people who do that look really stupid.

Copies of this report have already been sent to students and faculty of the
recent Connectionist Models Summer School at CMU, except for CMU people who
can easily pick up a copy.

---------------------------------------------------------------------------
Technical Report CMU-CS-88-162
"An Empirical Study of Learning Speed in Back-Propagation Networks"

Scott E. Fahlman
Computer Science Department
Carnegie-Mellon University
Pittsburgh, PA 15213

Abstract:

Most connectionist or "neural network" learning systems use some form of
the back-propagation algorithm.  However, back-propagation learning is too
slow for many applications, and it scales up poorly as tasks become larger
and more complex.  The factors governing learning speed are poorly
understood.  I have begun a systematic, empirical study of learning speed
in backprop-like algorithms, measured against a variety of benchmark
problems.  The goal is twofold: to develop faster learning algorithms and
to contribute to the development of a methodology that will be of value in
future studies of this kind.

This paper is a progress report describing the results obtained during the
first six months of this study.  To date I have looked only at a limited
set of benchmark problems, but the results on these problems are
encouraging: I have developed a new learning algorithm called "quickprop"
that, on the problems tested so far, is faster than standard backprop by an
order of magnitude or more.  This new algorithm also appears to scale up
very well as the problem size increases.