PhD thesis available: Combining Predictors ...

[Jakob Vogdrup Hansen]

Dear Connectionists,

Some people have had problems downloading my PhD thesis. I therefore
give four different links to the PhD thesis in postscript and pdf
format. The two last links are identical to the two first except some
(unnecessary) pictures have been removed.

http://www.daimi.au.dk/~vogdrup/diss.ps 
http://www.daimi.au.dk/~vogdrup/diss.pdf
http://www.daimi.au.dk/~vogdrup/diss2.ps
http://www.daimi.au.dk/~vogdrup/diss2.pdf

Comments are welcome.

regards,

Jakob

Title:

Combining Predictors.
Meta Machine Learning Methods and
Bias/Variance & Ambiguity Decompositions

Abstract:


The most important theoretical tool in connection with machine
learning is the bias/variance decomposition of error
functions. Together with Tom Heskes, I have found the family of error
functions with a natural bias/variance decomposition that has target
independent variance. It is shown that no other group of error
functions can be decomposed in the same way. An open problem in the
machine learning community is thereby solved. The error functions are
derived from the deviance measure on distributions in the
one-parameter exponential family. It is therefore called the deviance
error family.

A bias/variance decomposition can also be viewed as an ambiguity
decomposition for an ensemble method. The family of error functions
with a natural bias/variance decomposition that has target independent
variance can therefore be of use in connection with ensemble methods.

The logarithmic opinion pool ensemble method has been developed
together with Anders Krogh. It is based on the logarithmic opinion
pool ambiguity decomposition using the Kullback-Leibler error
function. It has been extended to the cross-validation logarithmic
opinion pool ensemble method. The advantage of the cross-validation
logarithmic opinion pool ensemble method is that it can use unlabeled
data to estimate the generalization error, while it still uses the
entire labeled example set for training.

The cross-validation logarithmic opinion pool ensemble method is
easily reformulated for another error function, as long as the error
function has an ambiguity decomposition with target independent
ambiguity. It is therefore possible to use the cross-validation
ensemble method on all error functions in the deviance error family.


-- 
Jakob V. Hansen			Tlf: 86 750618
Rydevnget 87, 1. th.   	Kontor: B2.15 Lokal: (8942)3355
8210 Aarhus V			E-mail: Vogdrup at daimi.au.dk