new paper in neuroprose

[dhw@santafe.edu]

                **DO NOT FORWARD TO OTHER GROUPS**



The following file has been placed in neuroprose, under the name
wolpert.overfitting.ps.Z. Thanks to Jordan Pollack for
maintaining this very useful system.




                ON OVERFITTING AVOIDANCE AS BIAS

                                by

             David H. Wolpert, The Santa Fe Institute




Abstract: In supervised learning it is commonly believed that Occam's
razor works, i.e., that penalizing complex functions helps one avoid
"overfitting" functions to data, and therefore improves
generalization. It is also commonly believed that cross-validation is
an effective way to choose amongst algorithms for fitting functions to
data. In a recent paper, Schaffer (1993) presents experimental
evidence disputing these claims. The current paper consists of a
formal analysis of these contentions of Schaffer's. It proves that his
contentions are valid, although some of his experiments must be
interpreted with caution.


Keywords: overfitting avoidance, cross-validation, decision tree
pruning, inductive bias, extended Bayesian analysis, uniform priors.



To retrieve the file:

unix> ftp cheops.cis.ohio-state.edu
Connected to cheops.cis.ohio-state.edu.
220 cheops.cis.ohio-state.edu FTP server ready.
Name: anonymous
331 Guest login ok, send ident as password.
Password:neuron
230 Guest login ok, access restrictions apply.
ftp> binary
200 Type set to I.
ftp> cd pub/neuroprose
250 CWD command successful.
ftp> get wolpert.overfitting.ps.Z
200 PORT command successful.
150 Opening BINARY mode data connection for rosenblatt.reborn.ps.Z
226 Transfer complete.
100000 bytes sent in 3.14159 seconds
ftp> quit
221 Goodbye.
unix> uncompress wolpert.overfitting.ps.Z
unix> lpr wolpert.overfitting.ps (or however you print postscript)