TR on Bayesian backprop by Hybrid Monte Carlo

[Radford Neal]

                 *** DO NOT FORWARD TO OTHER LISTS ***


The following paper has been placed in the neuroprose archive:


             BAYESIAN TRAINING OF BACKPROPAGATION NETWORKS BY

                     THE HYBRID MONTE CARLO METHOD


                           Radford M. Neal

                     Department of Computer Science
                         University of Toronto
 
                        radford at cs.toronto.edu


  It is shown that Bayesian training of backpropagation neural networks
  can feasibly be performed by the ``Hybrid Monte Carlo'' method. This
  approach allows the true predictive distribution for a test case given
  a set of training cases to be approximated arbitrarily closely, in
  contrast to previous approaches which approximate the posterior weight
  distribution by a Gaussian. In this work, the Hybrid Monte Carlo
  method is implemented in conjunction with simulated annealing, in
  order to speed relaxation to a good region of parameter space. The
  method has been applied to a test problem, demonstrating that it can
  produce good predictions, as well as an indication of the uncertainty
  of these predictions. Appropriate weight scaling factors are found
  automatically. By applying known techniques for calculation of ``free
  energy'' differences, it should also be possible to compare the merits
  of different network architectures. The work described here should
  also be applicable to a wide variety of statistical models other than
  neural networks.


This paper may be retrieved and printed on a PostScript printer as follows:

  unix> ftp archive.cis.ohio-state.edu
  (log on as user 'anonymous')
  ftp> cd pub/neuroprose
  ftp> binary
  ftp> get neal.hmc.ps.Z
  ftp> quit
  unix> uncompress neal.hmc.ps.Z
  unix> lpr neal.hmc.ps


For those unable to do this, hardcopies may be requested from:

  The CRG Technical Report Secretary 
  Department of Computer Science
  University of Toronto 
  10 King's College Road
  Toronto  M5S 1A4
  CANADA

  INTERNET: maureen at cs.toronto.edu 
  UUCP: uunet!utai!maureen
  BITNET: maureen at utorgpu