Techreport on Proben1 Neural Network benchmark collection

[Lutz Prechelt]

FTP-host: archive.cis.ohio-state.edu
FTP-filename: /pub/neuroprose/prechelt.bench.ps.Z
URL: ftp://archive.cis.ohio-state.edu/pub/neuroprose/prechelt.bench.ps.Z
        
        
The technical report 

   Proben1 --- A Set of Neural Network Benchmark Problems
   and Benchmarking Rules

is now available for anonymous ftp as

  ftp.ira.uka.de  /pub/papers/techreports/1994/1994-21.ps.Z
and as
  archive.cis.ohio-state.edu  /pub/neuroprose/prechelt.bench.ps.Z

The report has 38 pages, the file is 158 Kb.
The report is the documentation of a benchmark collection that I have
prepared. This collection is the first closed and exactly documented
benchmark collection specifically made for neural network research.
All of its problems are 'real' problems in the sense that the data has
not been generated artificially. Most of the problems were taken from
the UCI machine learning databases archive.
Particular emphasis lies on achieving reproducibility of results,
which is difficult with most existing real world data benchmarks.
Here is the abstract:

Proben1 is a collection of problems for neural network learning in the
realm of pattern classification and function approximation plus a set
of rules and conventions for carrying out benchmark tests with these
or similar problems. Proben1 contains 15 data sets from 12 different
domains. All datasets represent realistic problems which could be
called diagnosis tasks and all but one consist of real world data. The
datasets are all presented in the same simple format, using an
attribute representation that can directly be used for neural network
training.  Along with the datasets, Proben1 defines a set of rules for
how to conduct and how to document neural network benchmarking.
The purpose of the problem and rule collection is to give researchers
easy access to data for the evaluation of their algorithms and
networks and to make direct comparison of the published results feasible.
This report describes the datasets and the benchmarking rules. It
also gives some basic performance measures indicating the difficulty
of the various problems. These measures can be used as baselines for
comparison.
--

Here is a bibtex entry of the report:

@techreport{Prechelt94c,
   author    = {Lutz Prechelt},
   title     = {{PROBEN1} --- {A} Set of Benchmarks and Benchmarking
                Rules for Neural Network Training Algorithms},
   institution = {Fakult\"at f\"ur Informatik, Universit\"at Karlsruhe},
   year      = {1994},
   number    = {21/94},
   address   = {D-76128 Karlsruhe, Germany},
   month     = sep,
   note      = {Anonymous FTP: /pub/pa\-pers/tech\-reports/1994/1994-21.ps.Z
                on ftp.ira.uka.de},
}


The benchmark collection itself (including the report) is available
for anonymous ftp from the directories

  ftp.ira.uka.de  /pub/neuron
and
  ftp.cs.cmu.edu  /afs/cs/project/connect/bench/contrib/prechelt

in both cases the file name is  proben1.tar.gz  (ca. 2 Mb)
  
 Lutz


Lutz Prechelt   (email: prechelt at ira.uka.de)            | Whenever you 
Institut fuer Programmstrukturen und Datenorganisation  | complicate things,
Universitaet Karlsruhe;  76128 Karlsruhe;  Germany      | they get
(Voice: ++49/721/608-4068, FAX: ++49/721/694092)        | less simple.