Robustness ?

[Terry Sejnowski]

Another paper that addresses fault tolerance in feedforward nets:

Neti, C, Schneider, MH and Young, ED, Maximally fault-tolerant
neural networks and nonlinear programming.  Vol II, p 483
IJCNN San Diego, June 1990.

Comparisons between the brain and BP nets may be misleading since a unit
should not be equated with a single neuron.  If one unit represents the
average firing rate in thousands of neurons then random loss of neurons
would correspond more closely with randomly perturbing the weights
rather than cutting out units.  Cutting out a unit is closer to
the damage that occurs with lesions of many neurons, which often leads to
unusual deficits.

The performance of BP-derived feedforward nets is remarkably resistant
to adding random noise to the weights, as Charlie Rosenberg and I
showed using NETtalk.  It took us a while to realize that our
random number generator was really working.

Terry

-----