PhD thesis available

[Carl Edward Rasmussen]

My PhD thesis is now available on the net. It is entitled

         EVALUATION OF GAUSSIAN PROCESSES AND
       OTHER METHODS FOR NON-LINEAR REGRESSION

The thesis is 138 pages long, occupies 460Kb in compressed postscript
and is formatted for double-sided printing. You can obtain a copy via
the web at

  http://www.cs.toronto.edu/~carl/pub.html

or via anonymous ftp to

  ftp.cs.toronto.edu

where the file "thesis.ps.gz" is placed in the directory "pub/carl".


ABSTRACT:

This thesis develops two Bayesian learning methods relying on Gaussian
processes and a rigorous statistical approach for evaluating such
methods.  In these experimental designs the sources of uncertainty in
the estimated generalisation performances due to both variation in
training and test sets are accounted for. The framework allows for
estimation of generalisation performance as well as statistical tests
of significance for pairwise comparisons. Two experimental designs are
recommended and supported by the DELVE software environment.

Two new non-parametric Bayesian learning methods relying on Gaussian
process priors over functions are developed.  These priors are
controlled by hyperparameters which set the characteristic length
scale for each input dimension. In the simplest method, these
parameters are fit from the data using optimization.  In the second,
fully Bayesian method, a Markov chain Monte Carlo technique is used to
integrate over the hyperparameters. One advantage of these Gaussian
process methods is that the priors and hyperparameters of the trained
models are easy to interpret.

The Gaussian process methods are benchmarked against several other
methods, on regression tasks using both real data and data generated
from realistic simulations. The experiments show that small datasets
are unsuitable for benchmarking purposes because the uncertainties in
performance measurements are large. A second set of experiments
provide strong evidence that the bagging procedure is advantageous for
the Multivariate Adaptive Regression Splines (MARS) method.

The simulated datasets have controlled characteristics which make them
useful for understanding the relationship between properties of the
dataset and the performance of different methods. The dependency of
the performance on available computation time is also investigated. It
is shown that a Bayesian approach to learning in multi-layer
perceptron neural networks achieves better performance than the
commonly used early stopping procedure, even for reasonably short
amounts of computation time. The Gaussian process methods are shown to
consistently outperform the more conventional methods.


--
                                                                \
Carl Edward Rasmussen       Email: carl at cs.toronto.edu          o/\_
Dept of Computer Science    Phone: +1 (416) 978 7391            <|__,\
University of Toronto,      Home : +1 (416) 531 5685             ">   |
Toronto, ONTARIO,           FAX  : +1 (416) 978 1455              `   |
Canada, M5S 1A4             web  : http://www.cs.toronto.edu/~carl