new paper

[David Wolpert]

***************************************************************
        DO NOT FORWARD TO OTHER BOARDS OR LISTS
***************************************************************


The following paper has been placed in neuroprose, under the name
wolpert.nips92.ps.Z. It is a major revision of an earlier preprint
on the same topic. An abbreviated version (2 fewer pages) will
appear in the proceedings of NIPS 92.




0N THE USE OF EVIDENCE IN NEURAL NETWORKS.


David H. Wolpert, Santa Fe Institute


Abstract: The Bayesian evidence approximation, which is closely related
to generalized maximum likelihood, has recently been employed to
determine the noise and weight-penalty terms for training neural nets.
This paper shows that it is far simpler to perform the exact calculation
than it is to set up the evidence approximation. Moreover, unlike that
approximation, the exact result does not have to be re-calculated for
every new data set. Nor does it require the running of complex
numerical computer code (the exact result is closed form). In addition,
it turns out that for neural nets, the evidence procedure's MAP
estimate is *in toto* approximation error. Another advantage of the exact
analysis is that it does not lead to incorrect intuition, like the claim
that one can "evaluate different priors in light of the data". This paper
ends by discussing sufficiency conditions for the evidence approximation
to hold, along with the implications of those conditions. Although couched
in terms of neural nets, the analysis of this paper holds for any Bayesian
interpolation problem.


Recover the file in the usual way:

unix> ftp cheops.cis.ohio-state.edu
Connected to cheops.cis.ohio-state.edu.
220 cheops.cis.ohio-state.edu FTP server ready.
Name: anonymous
331 Guest login ok, send ident as password.
Password: {your address}
230 Guest login ok, access restrictions apply.
ftp> binary
200 Type set to I.
ftp> cd pub/neuroprose
250 CWD command successful.
ftp> get wolpert.nips92.ps.Z
200 PORT command successful.
150 Opening BINARY mode data connection for rosenblatt.reborn.ps.Z
226 Transfer complete.
100000 bytes sent in 3.14159 seconds
ftp> quit
221 Goodbye
unix> uncompress wolpert.nips92.ps.Z
unix> lpr wolpert.nips92.ps (or however you print postscript