Does backprop need the derivative?

[Christian Darken]

>Other posters have discussed, regarding backprop...  
>
>> ... the question whether the derivative can be replaced by a constant, 
>
>To clarify, I believe the intent is that the "constant" have the same
>sign as the derivative, but have constant magnitude.


I haven't been following this thread, but the following reference
may be helpful to those that are.

Blum (Annals of Math. Statistics vol. 25 1954 p.385) shows that if the
"constant magnitude" is going to zero (so that the system is convergent)
the convergence is not to a minimum of the expected error (this is usually
what we want backprop to do), but to a minimum of the *median* of the error.


Chris Darken
darken at learning.scr.siemens.com