2 papers: Application of Bayesian inference in NN

[Georg Dorffner]

Dear connectionists,

the following two papers are available at the WWW sites listed below.


----------

Experiences with bayesian learning in a real world application

Sykacek P., Dorffner G., Rappelsberger P., Zeitlhofer J.
to appear in: Advances in Neural Information Processing Systems, Vol.10, 
1998. 

http://www.ai.univie.ac.at/~georg/papers/nips97.ps.Z

Abstract:

This paper reports about an application of Bayes' inferred neural
network classifiers to the field of automatic sleep staging. The reason
for using Bayesian learning for this task is two-fold. First,
Bayesian inference is known to embody regularization automatically. 
Second, a side effect of Bayesian learning leads to larger variance of
network outputs in regions without training data. This results in well
known moderation effects, which can be used to detect outliers.
In a 5 fold cross-validation experiment the full Bayesian solution was
not better than a single maximum a-posteriori (MAP) solution found
with D.J. MacKay's evidence approximation (see MacKay 1992). In a second
experiment we studied the properties of both solutions in rejecting
classification of movement artefacts.


-----------

Evaluating confidence measures in a neural network based sleep stager

Sykacek P., Dorffner G., Rappelsberger P., Zeitlhofer J.
Austrian Research Institute for Artificial Intelligence, Vienna,
Technical Report TR-97-21, 1997; submitted for publication

http://www.ai.univie.ac.at/~georg/papers/ieee.ps.Z

Abstract:

In this paper we report about an extensive investigation on neural
networks -- multilayer perceptrons (MLP), in particular -- in the task
of automatic sleep staging based on electroencephalogram (EEG) and
electrooculogram (EOG) signals. After the important first step of
preprocessing and feature selection (for which, a search-based selection
technique could reduce the large number of features to a feature vector
of size ten), the main focus was on evaluating the used of so-called
``doubt-levels'' and ``confidence intervals'' (``error bars'') in
improving the results by rejecting uncertain cases and patterns not
well represented by the training set. The main technique used here is
that of Bayesian inference to arrive at probability distributions of
network weights based on training data. We compare the results of the
full-blown Bayesian method with a reduced method calculating only the
maximum posterior solution and with an MLP trained with the more
common gradient descent technique for minimizing an error measure
(``backpropagation''). The results show that, while both the
full-blown Bayesian technique and the maximum posterior solution
significantly outperform the simpler backpropagation technique, only
the application of doubt-levels to reject uncertain cases can lead to
an improvement of results. Our conclusion is that the data set of five
independent all-night recordings from five normal subjects represents
the possible data space well enough. At the same time, we show that
Bayesian inference, for which we have developed a useful extension for
reliable calculation of error bars, can still be used for the
rejection of artefacts.


-------------------


This work was done in the context of the concerted action ANNDEE
(http://www.ai.univie.ac.at/oefai/nn/anndee), investigating neural
networks in EEG processing, sponsored by the European Union and the
Austrian Federal Ministry of Science and Transport.


--------------------

Georg Dorffner
Austrian Research Institute for Artificial Intelligence
Neural Networks Group
Schottengasse 3, A-1010 Vienna, Austria

http://www.ai.univie.ac.at/oefai/nn