No subject

[Thomas W. Karjala]

Dear connectionist researchers,

The members of our research group have be applying neural networks to
fault detection, data reconciliation, and control in the field of
chemical engineering for several years.  Recently, we have been using
nonlinear programming techniques for training of feedforward networks.
We have found these techniques to be more successful than
backpropagation and have been able to train larger networks more
quickly without external tuning of parameters such as learning rate and
momentum.

Training methods based on nonlinear programming share one drawback
with backpropagation.  Selection of the wrong starting point
in the weight space can often lead to local minima where the network
has learned the training data only poorly.   

In our readings, several of us have come across vague mention of
methods of choosing initial starting weights other than picking small
random values.  We are now searching for more substantial references
in this area.  We would welcome any suggestions, comments, or pointers
to papers on this subject and will be glad to share any information we
find.  Please contact us directly via email.

Thanks!

Thomas Karjala
Department of Chemical Engineering
The University of Texas at Austin
Ausin, TX  78712

tom at faulty.che.utexas.edu