New Bayesian work

[David MacKay]

Two new papers available
------------------------

	The papers that I presented at Snowbird this year are 
now available in the neuroprose archives. 

The titles:

	[1] Bayesian interpolation 		(14 pages)
	[2] A practical Bayesian framework 
		for backprop networks		(11 pages)

	The first paper describes and demonstrates recent 
developments in Bayesian regularisation and model comparison. 
The second applies this framework to backprop. The first paper 
is a prerequisite for understanding the second. 

	Abstracts and instructions for anonymous ftp follow. 

	If you have problems obtaining the files by ftp, feel 
free to contact me. 

        David MacKay              Office:      (818) 397 2805
                                  Fax:         (818) 792 7402
                                  Email:  mackay at hope.caltech.edu
                                  Smail:  Caltech 139-74,
                                          Pasadena, CA 91125

Abstracts
---------
		Bayesian interpolation
		----------------------
		Although Bayesian analysis has been in use since Laplace, 
	the Bayesian method of {\em model--comparison} has only 
	recently been developed in depth.
		In this paper, the Bayesian approach to  
	regularisation and model--comparison is demonstrated by 
	studying the inference problem of interpolating noisy data. 
	The concepts and methods described are quite general and can 
	be applied to many other problems. 
		Regularising constants are  set by examining their  
	posterior probability distribution. Alternative regularisers  
	(priors) and alternative basis sets are objectively compared  
	by evaluating the {\em evidence} for them. `Occam's razor' is  
	automatically embodied by this framework.
		The way in which Bayes infers the values of regularising   
	constants and noise levels has an elegant interpretation in terms  
	of the effective number of parameters  determined by the data set. 
	This  framework is due to Gull and Skilling.

	   A practical Bayesian framework for backprop networks
	   ----------------------------------------------------
		A quantitative and practical Bayesian framework is described  
	for learning of mappings in feedforward networks.  
	The framework makes possible: 

	(1) objective comparisons between solutions using 
		alternative network architectures; 
	(2) objective stopping rules for deletion of weights; 
	(3) objective choice of magnitude and type of weight decay terms or
		additive regularisers  (for penalising large weights, etc.); 
	(4) a measure of the effective number of well--determined parameters 
		in a model; 
	(5) quantified estimates of the error bars on network parameters 
		and on network output; 
	(6) objective comparisons with alternative learning and interpolation 
		models such as splines and radial basis functions.  

	The Bayesian `evidence' automatically embodies `Occam's razor,' 
	penalising over--flexible and over--complex architectures. 
	The Bayesian approach helps detect poor underlying assumptions in 
	learning models. For  learning models well--matched to a problem, 
	a good correlation between generalisation ability and the Bayesian 
	evidence is obtained.

Instructions for obtaining copies by ftp from neuroprose:
---------------------------------------------------------

unix> ftp cheops.cis.ohio-state.edu		# (or ftp 128.146.8.62)

      Name: anonymous
      Password: neuron
      ftp> cd pub/neuroprose
      ftp> binary
      ftp> get mackay.bayes-interpolation.ps.Z
      ftp> get mackay.bayes-backprop.ps.Z
      ftp> quit

unix> [then `uncompress' files and lpr them.]