2 Papers available on combining estimators and missing data

[Volker Tresp]

The  -2- files  tresp.combining.ps.Z and tresp.effic_miss.ps.Z 
can now be copied from Neuroprose.

The papers are  8 and 9  pages long.
Hardcopies copies are not available.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
FTP-host: archive.cis.ohio-state.edu
FTP-filename: /pub/neuroprose/tresp.combining.ps.Z


      COMBINING ESTIMATORS USING NON-CONSTANT WEIGHTING FUNCTIONS

              by Volker Tresp and Michiaki Taniguchi
 

Abstract:
This paper discusses the linearly weighted combination of estimators in
which the weighting functions are  dependent on  the input.  We show
that the weighting functions can be derived either by evaluating the
input dependent variance of each estimator or by estimating how likely
it is that a given estimator has seen data in the region of the input
space close to the input pattern.  The latter solution is closely
related to the mixture of experts approach and we show how  learning
rules for the mixture of experts can be derived from the theory about
learning with missing features.  The presented approaches are  modular
since the weighting functions can easily be modified  (no retraining)
if more estimators are added.  Furthermore, it is easy to incorporate
estimators which were not derived from data such as expert systems or
algorithms.



%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
FTP-host: archive.cis.ohio-state.edu
FTP-filename: /pub/neuroprose/tresp.effic_miss.ps.Z



EFFICIENT METHODS FOR DEALING WITH MISSING DATA IN SUPERVISED LEARNING

     by Volker Tresp, Ralph Neuneier, and Subutai Ahmad 



Abstract:
We present efficient algorithms for dealing with the problem of missing
inputs (incomplete feature vectors) during training and recall. Our
approach is based on the approximation of the input data distribution
using Parzen windows.  For recall,   we obtain closed form solutions
for arbitrary feedforward networks.  For training, we show how the
backpropagation step for an  incomplete pattern can be approximated by
a weighted averaged backpropagation step.  The complexity of the
solutions for training and recall is independent of the number of
missing features.  We verify  our theoretical results using one
classification and one regression  problem.




The papers will appear in
G. Tesauro, D. S. Touretzky and T. K. Leen, eds.,
"Advances in Neural Information Processing Systems 7",
MIT Press, Cambridge MA, 1995.


________________________________________
 
Volker Tresp

Siemens AG
ZFE T SN4
81730 Munich
Germany

email: Volker.Tresp at zfe.siemens.de
Phone: +49 89 636 49408
Fax:   +49 89 636 3320
________________________________________