Preprint available

[Michael P. Perrone]

          FTP-host: archive.cis.ohio-state.edu
          FTP-filename: perrone.MSE-averaging.ps.Z

The following paper is now available in neuroprose.  It was presented
at the 1992 CAIP Conference at Rutger University.  It will appear in 

   Neural Networks for Speech and Image processing,
   R. J. Mammone (ed.), Chapman-Hall, 1993.

The paper is relevant to the recent discussion on Connectionists about
multiple neural network estimators.

Enjoy!
Michael
--------------------------------------------------------------------------------
Michael P. Perrone                                      Email: mpp at cns.brown.edu
Institute for Brain and Neural Systems                  Tel:   401-863-3920
Brown University                                        Fax:   401-863-3934
Providence, RI 02912
--------------------------------------------------------------------------------

   When Networks Disagree: Ensemble Method for Neural Networks
       M. P. Perrone and L. N Cooper

Abstract:

This paper presents a general theoretical framework for ensemble methods
of constructing significantly improved regression estimates.  Given a
population of regression estimators, we construct a hybrid estimator which
is as good as or better than, in the MSE sense, any estimator in the population.
We argue that the ensemble method presented has several properties:

   1) It efficiently uses all the networks of a population - none of the
      networks need be discarded.
   2) It efficiently uses all the available data for training without over-
      fitting.
   3) It inherently performs regularization by smoothing in functional space
      which helps to avoid over-fitting.
   4) It utilizes local minima to construct improved estimates whereas other
      neural network algorithms are hindered by local minima.
   5) It is ideally suited for parallel computation.
   6) It leads to a very useful and natural measure of the number of distinct
      estimators in a population.
   7) The optimal parameters of the ensemble estimator are given in closed form.

Experimental results are provided which show that the ensemble method
dramatically improves neural network performance on difficult real-world
optical character recognition tasks.

--------------------------------------------------------------------------------

Retrieval instructions:

The paper is found in the neuroprose archive under

	perrone.MSE-averaging.ps.Z		15 pages

To retrieve these files from the neuroprose archives:

unix> ftp cheops.cis.ohio-state.edu
Name (cheops.cis.ohio-state.edu:username): anonymous
Password: (use your email address)
ftp> cd pub/neuroprose
ftp> binary
ftp> get perrone.MSE-averaging.ps.Z
ftp> bye