Does backprop need the derivative?

john kolen kolen-j at cis.ohio-state.edu
Tue Feb 9 07:51:53 EST 1993


The sign of the derivative is always positive ( remember o(1-o) and 0<o<1).
What is important is the general shape of derivative:  maximumal at 0.5,
minimal at the extremes. I have replaced the derivative with other
functions ( d(x)=c, d(x)=cx, d(x)=min(x,1-x), d(x)=x(1-x)+c ) and bp works
when there is a bump.

John Kolen



More information about the Connectionists mailing list