NIPS paper in Neuroprose

[Anders Krogh]

The following paper has been placed in the Neuroprose archive

Title:   A Simple Weight Decay Can Improve Generalization
Authors: Anders Krogh and John A. Hertz

Filename: krogh.weight-decay.ps.Z

(To appear in proceedings from NIPS 91)

Abstract:
It has been observed in numerical simulations that a weight decay can
improve generalization in a feed-forward neural network.  This paper
explains why.  It is proven that a weight decay has two effects in a
linear network.  First, it suppresses any irrelevant components of the
weight vector by choosing the smallest vector that solves the learning
problem.  Second, if the size is chosen right, a weight decay can
suppress some of the effects of static noise on the targets, which
improves generalization quite a lot.  It is then shown how to extend
these results to networks with hidden layers and non-linear units.
Finally the theory is confirmed by some numerical simulations using the
data from NetTalk.


----------------------------------------------------------------
FTP INSTRUCTIONS

     unix> ftp archive.cis.ohio-state.edu (or 128.146.8.52)
     Name: anonymous
     Password: anything
     ftp> cd pub/neuroprose
     ftp> binary
     ftp> get krogh.weight-decay.ps.Z
     ftp> bye
     unix> zcat krogh.weight-decay.ps.Z | lpr
                  (or however you uncompress and print postscript)