Survey of Second Order Techniques

[Eric A. Wan]

I am compiling a study on the extent to which researches have gone
beyond simple gradient descent (back-propagation) for training
layered neural networks by applying more sophisticated classical
techniques in non-linear optimization (e.g. Newton, Quasi-Newton,
Conjugate-Gradient methods, etc.)?  Please e-mail me any comments
and/or references that you have on the subject.  I will summarize 
the responses.

Thanks in advance.

Eric Wan  
wan at isl.stanford.edu