paper on conditioning and local minima in MLP

[Patrick van der Smagt]

Dear connectionists:

the following ICANN'98 reprint is available via the web:
http://www.robotic.dlr.de/Smagt/papers/SmaHir98b.ps.gz

  "Why feed-forward networks are in a bad shape"
        P. van der Smagt and G. Hirzinger
    German Aerospace Center/DLR Oberpfaffenhofen

Abstract:
It has often been noted that the learning problem in feed-forward neural
networks is very badly conditioned.  Although, generally, the special form of
the transfer function is usually taken to be the cause of this condition, we
show that it is caused by the manner in which neurons are connected.  By
analyzing the expected values of the Hessian in a feed-forward network it is
shown that, even in a network where all the learning samples are well chosen
and the transfer function is not in its saturated state, the system has a
non-optimal condition.  We subsequently propose a change in the feed-forward
network structure which alleviates this problem.  We finally demonstrate the
positive influence of this approach.


Other papers available on http://www.robotic.dlr.de/Smagt/papers/
--
dr Patrick van der Smagt                phone +49 8153 281152, fax -34
DLR/Institute of Robotics and System Dynamics             smagt at dlr.de
P.O.Box 1116, 82230 Wessling, Germany     http://www.robotic.de/Smagt/