Does backprop need the derivative ??

[john kolen]

   From: Scott_Fahlman at sef-pmax.slisp.cs.cmu.edu

   Of course, a learning system implemented in analog hardware might have only
   a few bits of accuracy due to noise and nonlinearity in the circuits, but
   it wouldn't suffer from this quantization effect, since you get a sort of
   probabilistic dithering for free.

This assumes, of course, that the mechanism is actually "computing" using
the available bits.  Bits are the result of binary measurements.  An analog
device does not normally convert voltages or currents into a binary
representation and then operate on it.  An analog mechanism sloppilly
implementing backprop should be able to tweak the weights in the general
direction, but not necessarily the same direction as theoretical backprop.

John Kolen