Cross-Val: Summary of Lit Survey and Request for References

[Mark Plutowksi]

Hello,

This is a follow-on to recent postings on using
cross-validation to assess neural network models.
It is a request for further references,
after an exhausting literature survey of my own
which failed to find the results I seek. 
A summary of my findings follows the request,
followed by an informative response from Grace Wahba, 
and finally, a list of the references I looked at.

Thanks for any leads or tips,

=================
== Mark Plutowski   pluto at cs.ucsd.edu
Computer Science and Engineering 0114
University of California, San Diego
La Jolla,  California, USA. 




			THE REQUEST:
			------------
Do you know of convergence/consistency results for justifying 
cross-validatory model assessment for nonlinear compositions 
of basis functions, such as the usual sigmoided feedforward network?


		  SUMMARY OF MY LIT SURVEY:
		  -------------------------
While the use of cross-validation to assess nonlinear neural network
models CAN be justified to a certain degree, 
(e.g., [Stone 76,77]) the really nice theoretical results exist
for other estimators, e.g., kernel density, histograms, linear
models, and splines (see references below.)

These results are not directly applicable to neural nets.
They all exploit properties of the particular estimators 
which are not shared by neural networks, in general.
In short, the proofs for linear models exploit linear 
reductions, and the other (nonlinear) estimators for
which optimality results have been published have the 
property that deleting a single example has negligible 
effect on the estimate outside a bounded region surrounding 
the example (e.g., kernel density estimators and splines.) 
In comparison, a single example can affect every weight 
of a neural network - deleting it can have global effect 
on the estimate.


		   GRACE WAHBA SAYS:
		   ------------------

Thanks to Grace Wahba for her informative response
to my request to her for information after I was
unable to get hold of a copy of her relevant book:
============================================================