No subject

Applications}, V. Cherkassky, J.H.  Friedman and H. Wechsler (eds.),
NATO ASI Series F, Springer-Verlag 1994.

-------------------------------------------------------------------------

Prediction Risk and Architecture Selection for Neural Networks

John Moody 


Abstract:

We describe two important sets of tools for neural network modeling:
prediction risk estimation and network architecture selection.
Prediction risk is defined as the expected performance of an
estimator in predicting new observations.  Estimated prediction
risk can be used both for estimating the quality of model predictions
and for model selection.  Prediction risk estimation and model
selection are especially important for problems with limited data.
Techniques for estimating prediction risk include data resampling
algorithms such as {\em nonlinear cross--validation (NCV)} and
algebraic formulae such as the {\em predicted squared error (PSE)}
and {\em generalized prediction error (GPE)}.  We show that exhaustive
search over the space of network architectures is computationally
infeasible even for networks of modest size.  This motivates the
use of {\em heuristic} strategies that dramatically reduce the
search complexity.  These strategies employ directed search
algorithms, such as selecting the number of nodes via {\em sequential
network construction (SNC)} and pruning inputs and weights via {\em
sensitivity based pruning (SBP)} and {\em optimal brain damage
(OBD)} respectively.


Keywords:

prediction risk, network architecture selection, cross--validation
(CV), nonlinear cross--validation (NCV), predicted squared error
(PSE), generalized prediction error (GPE), effective number of
parameters, heuristic search, sequential network construction (SNC),
pruning, sensitivity based pruning (SBP), optimal brain damage
(OBD).

=========================================================================

Retrieval instructions are:

unix> ftp neural.cse.ogi.edu
login: anonymous
password: name at email.address

ftp> cd pub/neural
ftp> cd papers
ftp> get INDEX
ftp> binary
ftp> get moody94.predictionrisk.ps.Z
ftp> quit

unix> uncompress *.Z
unix> lpr *.ps