Does backprop need the derivative ??

[john kolen]

When I used the term stable in my previous posting, I did not entail the
mathematical notion of stability when applied to a control system.  What I
meant was the apparent behavior of the network, learning a set of
associations of patterns, was unaffected by quantitative changes in these
operations.  An analogy I often use is the symbolic dynamics of unimodal
iterated function systems.  As long as small number of qualitative
conditions are true, then the system will exhibit the same symbol dynamics
as other functions for which the conditions hold regardless of the
numerical differences between functions.  Thus the bifurcation diagrams of
rx(1-x) and a bump made up of sigmoids will exhibit the same type of period
doubling cascaded.  

Even if it wasn't mathematically stable, but was guaranteed to pass through
a region of weight space with usable weights, most of the NN community
would find it useful.

John