Thesis available: Optimal Linear Combinations of Neural Networks

[Sherif Hashem]

FTP-host: archive.cis.ohio-state.edu
FTP-file: pub/neuroprose/Thesis/hashem.thesis.ps.Z

The file hashem.thesis.ps.Z is now available for copying from the
Neuroprose archive:

      
          OPTIMAL LINEAR COMBINATIONS OF NEURAL NETWORKS

                         Sherif Hashem

                          Ph.D. Thesis
                        Purdue University
      
ABSTRACT: Neural network (NN) based modeling often involves trying multiple
networks with different architectures, learning techniques, and
training parameters in order to achieve ``acceptable'' model accuracy.
Typically, one of the trained networks is chosen as ``best,'' while
the rest are discarded.

In this dissertation, using optimal linear combinations (OLCs) of the 
corresponding outputs of a number of NNs is proposed as an alternative to 
using a single network.  Modeling accuracy is measured by mean squared 
error (MSE) with respect to the distribution of random inputs to the NNs.
Optimality is defined by minimizing the MSE, with the resultant combination
referred to as MSE-OLC.

MSE-OLCs are investigated for four cases: allowing (or not) a constant
term in the combination and requiring (or not) the combination-weights to 
sum to one.  In each case, deriving the MSE-OLC is straightforward and the 
optimal combination-weights are simple, requiring modest matrix 
manipulations.

In practice, the optimal combination-weights need to be estimated from 
observed data: observed inputs, the corresponding true responses, and the
corresponding outputs of each component network.  Given the data, 
estimating the optimal combination-weights is straightforward.

Collinearity among the outputs and/or the approximation errors of the 
component NNs sometimes degrades the generalization ability of the 
estimated MSE-OLC.  To improve generalization in the presence of degrading 
collinearity, six algorithms for selecting subsets of the NNs for the 
MSE-OLC are developed and tested.

Several examples, including a real-world problem and an empirical
study, are discussed.  The examples illustrate the importance of
addressing collinearity and demonstrate significant improvements in model 
accuracy as a result of employing MSE-OLCs supported by the NN selection 
algorithms.

---------------------------

The thesis is 126 Pages (10 preamble + 116 text). 

To obtain a copy of the Postscript file:

  %ftp archive.cis.ohio-state.edu
  >Name: anonymous
  >Password: <Your email address>
  >cd pub/neuroprose/Thesis
  >binary
  >get hashem.thesis.ps.Z
  >quit

Then:
 
  %uncompress hashem.thesis.ps.Z  
            (The size of the uncompressed file is about 1.1Mbyte)
  %lpr -s -P<Printer-name> hashem.thesis.ps

---------------------------

Hard copies may be requested from the School of Industrial Engineering,
1287 Grissom Hall, Purdue University, West Lafayette, IN 47907-1287, USA. 
(Refer to Technical Report SMS 94-4.)


--Sherif Hashem

===================================================================
  Pacific Northwest Laboratory        E-mail: s_hashem at pnl.gov
  906 Battelle Boulevard              Tel. (509) 375-6995
  P.O. Box 999, MSIN K1-87            Fax. (509) 375-6631
  Richland, WA 99352
  USA
===================================================================