NIPS 92 Workshop on Training Issues

[Scott A. Markel x2683]

****************************   NIPS 92  Workshop   ****************************

                 "Computational Issues in Neural Network Training"
                 
                                       or

                   Why is Back-Propagation Still So Popular?

*******************************************************************************

Roger Crane and I are are leading a NIPS '92 workshop on "Computational Issues
in Neural Network Training".  Our workshop will be on Saturday, 5 December, the
second of two days of workshops in Vail.

The discussion will focus on optimization techniques currently used by neural
net researchers, and include some other techniques that are available.  Back-
propagation is still the optimization technique of choice even though there are
obvious problems in training with BP: speed, convergence, ... .  Several
innovative algorithms have been proposed by the neural net community to improve
upon BP, e.g., Scott Fahlman's QuickProp.  We feel that there are classical
optimization techniques that are superior to back-propagation.  In fact,
gradient descent (BP) fell out of favor with the mathematical optimization
folks way backin the 60's!  So why is BP still so popular?

Topics along these lines include:

   *  Why are classical methods generally ignored?

   *  Computational speed

   *  Convergence criteria (or lack thereof!)

Broader issues to be discussed include:

   *  Local minima

   *  Selection of starting points

   *  Conditioning (for higher order methods)

   *  Characterization of the error surface

If you would like to present something on any of these or similar topics,
please contact me by e-mail and we can discuss details.

Workshops are scheduled for a total of four hours.  We're allowing for approxi-
mately 8 presentations of 10-20 minutes each, since we want to make sure that
ample time is reserved for discussion and informal presentations.  We will
encourage (incite) lively audience participation.  By the way, none of the NIPS
workshops are limited to presenters only.  People who want to show up and just
listen are more than welcome.

Scott Markel                   Computational Science Research
smarkel at sarnoff.com            David Sarnoff Research Center
Tel. 609-734-2683              CN 5300
FAX  609-734-2662              Princeton, NJ 08543-5300