Report on NN learning algorithm evaluation practices

[Lutz Prechelt]

A technical report titled

  A Study of Experimental Evaluations
  of Neural Network Learning Algorithms:
  Current Research Practice

is now available for anonymous ftp (binary mode!) from

  archive.cis.ohio-state.edu  /pub/neuroprose/prechelt.eval.ps.Z

The file is 70 Kb, the paper has 9 pages.

This is the abstract:

113 journal articles about neural network learning algorithms published in
1993 and 1994 are examined for the amount of experimental evaluation they
contain. Every third of them does employ not even a single realistic or real
learning problem. Only 6\% of all articles present results for more
than one problem using real world data.
Furthermore, one third of all articles does not present any
quantitative comparison with a previously known algorithm.
These results indicate that the quality of research in the
area of neural network learning algorithms needs improvement.
The publication standards should be raised and easily accessible
collections of example problems be built.

Here is a bibtex entry for the report:

@techreport{Prechelt94d,
   author    = {Lutz Prechelt},
   title     = {A Study of Experimental Evaluations of Neural Network
                Learning Algorithms: Current Research Practice},
   institution = {Fakult\"at f\"ur Informatik, Universit\"at Karlsruhe},
   year      = {1994},
   number    = {19/94},
   address   = {D-76128 Karlsruhe, Germany},
   month     = aug,
   note      = {anonymous ftp: /pub/papers/techreports/1994/1994-19.ps.Z
                on ftp.ira.uka.de},
}

Comments are welcome.

  Lutz
  
Lutz Prechelt   (email: prechelt at ira.uka.de)            | Whenever you 
Institut fuer Programmstrukturen und Datenorganisation  | complicate things,
Universitaet Karlsruhe;  76128 Karlsruhe;  Germany      | they get
(Voice: ++49/721/608-4068, FAX: ++49/721/694092)        | less simple.