Paper available by ftp

[bishopc]

FTP-host: archive.cis.ohio-state.edu
FTP-filename: /pub/neuroprose/bishop.noise.ps.Z 

The following technical report is available by anonymous ftp.

------------------------------------------------------------------------


       TRAINING WITH NOISE IS EQUIVALENT TO TIKHONOV REGULARIZATION

                          Chris M Bishop


                    Neural Computing Research Group
                          Aston University
                       Birmingham, B4 7ET, U.K.
                    email: c.m.bishop at aston.ac.uk


          Neural Computing Research Group Report: NCRG/4290

          (Accepted for publication in Neural Computation)



                               Abstract

It is well known that the addition of noise to the input data of 
a neural network during training can, in some circumstances, lead 
to significant improvements in generalization performance. Previous 
work has shown that such training with noise is equivalent to a 
form of regularization in which an extra term is added to the error 
function. However, the regularization term, which involves second 
derivatives of the error function, is not bounded below, and so can 
lead to difficulties if used directly in a learning algorithm based 
on error minimization. In this paper we show that, for the purposes 
of network training, the regularization term can be reduced to a 
positive definite form which involves only first derivatives of the 
network mapping. For a sum-of-squares error function, the 
regularization term belongs to the class of generalized Tikhonov 
regularizers. Direct minimization of the regularized error function 
provides a practical alternative to training with noise.


-------------------------------------------------------------------- 
ftp instructions: 

% ftp archive.cis.ohio-state.edu
Name: anonymous
password: your full email address
ftp> cd pub/neuroprose
ftp> binary
ftp> get bishop.noise.ps.Z
ftp> bye
% uncompress bishop.noise.ps.Z
% lpr bishop.noise.ps

-------------------------------------------------------------------- 

  Professor Chris M Bishop           Tel. +44 (0)21 359 3611 x4270
  Neural Computing Research Group    Fax. +44 (0)21 333 6215
  Dept. of Computer Science          c.m.bishop at aston.ac.uk
  Aston University               
  Birmingham B4 7ET, UK

--------------------------------------------------------------------