Does backprop need the derivative ??

[Scott_Fahlman@SEF-PMAX.SLISP.CS.CMU.EDU]

    In his paper, 'An Empirical Study of Learning Speed in Back-Propagation
    Networks', Scott E. Fahlmann shows that with the encoder/decoder problem
    it is possible to replace the derivative of the transfer function by
    a constant. I have been able to reproduce this example. However, for
    several other examples, it was not possible to get the network 
    converged using a constant for the derivative.
    
Interesting.  I just tried this on encoder problems and a couple of other
simple things, and leapt to the conclusion that it was a general
phenomenon.  It seems plausible to me that any "derivative" function that
preserves the sign of the error and doesn't have a "flat spot" (stable
point of 0 derivative) would work OK, but I don't know of anyone who has
made an extensive study of this.

I'd be interested in hearing more about the problems you've encountered and
about any results others send to you.

-- Scott

===========================================================================
Scott E. Fahlman			Internet:  sef+ at cs.cmu.edu
Senior Research Scientist		Phone:     412 268-2575
School of Computer Science              Fax:       412 681-5739
Carnegie Mellon University		Latitude:  40:26:33 N
5000 Forbes Avenue			Longitude: 79:56:48 W
Pittsburgh, PA 15213
===========================================================================