Adding noise to training data

[HOLMSTROM@csc.fi]

A note to a comment by Tom English on this topic:

He writes

> At first blush, it seems there's a close relationship between Parzen
> estimation (Duda & Hart 1973) and training with noise added to the
> samples.  If we were to use the noise function as the window function
> in Parzen estimation of the distribution from which the training set
> was drawn, wouldn't we would obtain precisely the noisy-sample
> distribution?

Yes, this is correct.

> And wouldn't a network minimizing squared error for the noisy training
> set asymptotically realize (i.e., as the number of noisy sample
> presentations approaches infinity) the Parzen estimator?  The
> results of Hampshire and Perlmutter (1990) seem to be relevant here.

As I said in my earlier note to John Hampshire's comment, there will be
a paper in the January issue of IEEE Transactions on Neural Networks that
gives a statistical analysis of using additive noise in training.
Several asymptotic results are given there.


Lasse Holmstrom