new paper on Asymptotic Statistical Theory of Overtraining and Cross-Validation

[klaus@prosun.first.gmd.de]

FTP-host: archive.cis.ohio-state.edu
FTP-file: pub/neuroprose/amari.overtraining.ps.Z

The following paper is now available for copying from the Neuroprose 
repository: amari.overtraining.ps.Z


amari.overtraining.ps.Z klaus at first.gmd.de
(128151 bytes) 32 pages. 
 
S. Amari, N. Murata, K.-R. M\"uller, M. Finke, H. Yang:

"Asymptotic Statistical Theory of Overtraining and Cross-Validation"


  A statistical theory for overtraining is proposed. The analysis
  treats general realizable stochastic neural networks, trained with
  Kullback-Leibler loss in the  asymptotic case of a large number
  of training examples.
  It is shown that the asymptotic gain in the generalization error is
  small if we perform early stopping, even if we have access to the
  optimal stopping time.  Considering cross-validation stopping we
  answer the question: In what ratio the examples should be divided
  into training and testing sets in order to obtain the optimum
  performance.  However, cross-validated early stopping is useless in
  the asymptotic region, while it decreases the generalization error
  only in the non-asymptotic region.  Our large scale simulations done
  on a CM5 are in nice agreement with our analytical findings.

(University of Tokyo Technical Report METR 06-95 and submitted to IEEE
Transactions on NN)


Best regards,

Klaus


&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&

Dr. Klaus-Robert M\"uller
GMD First (Forschungszentrum Informationstechnik)
Rudower Chaussee 5, 12489 Berlin
Germany

mail: klaus at first.gmd.de
Tel:  +49 30 6392 1860
Fax: +49 30 6392 1805

web-page:
http://www.first.gmd.de/persons/Mueller.Klaus-Robert.html

&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&