Thesis and paper available: variational methods

[Tommi Jaakkola]

The following Ph.D. thesis is available on the web at

 ftp://psyche.mit.edu/pub/tommi/thesis.ps.gz

--------------------------------------------------------------------

Variational Methods for Inference and Estimation in Graphical Models

                     Tommi S. Jaakkola
                            MIT 

Graphical models enhance the representational power of probability
models through qualitative characterization of their properties. This
also leads to greater efficiency in terms of the computational
algorithms that empower such representations. The increasing complexity
of these models, however, quickly renders exact probabilistic
calculations infeasible. We propose a principled framework for
approximating graphical models based on variational methods.

We develop variational techniques from the perspective that unifies and
expands their applicability to graphical models. These methods allow the
(recursive) computation of upper and lower bounds on the quantities of
interest. Such bounds yield considerably more information than mere
approximations and provide an inherent error metric for tailoring the
approximations individually to the cases considered. These desirable
properties, concomitant to the variational methods, are unlikely to
arise as a result of other deterministic or stochastic approximations.

The thesis consists of the development of this variational methodology
for probabilistic inference, Bayesian estimation, and towards efficient
diagnostic reasoning in the domain of internal medicine.



================================================================

The following technical report is now available via ftp:

 ftp://psyche.mit.edu/pub/tommi/varqmr.ps     (~400kb)
 ftp://psyche.mit.edu/pub/tommi/varqmr.ps.Z   (~150kb)
 ftp://psyche.mit.edu/pub/tommi/varqmr.ps.gz  (~ 95kb)

-------------------------------------------------------------

      Variational methods and the QMR-DT database

        Tommi S. Jaakkola and Michael I. Jordan

                          MIT

We describe variational approximation methods for efficient
probabilistic reasoning, applying these methods to the problem of
diagnostic inference in the QMR-DT database.  The QMR-DT database is a
large-scale belief network based on statistical and expert knowledge
in internal medicine.  The size and complexity of this network render
exact probabilistic diagnosis infeasible for all but a small set of
cases.  This has hindered the development of the QMR-DT network as a
practical diagnostic tool and has hindered researchers from exploring
and critiquing the diagnostic behavior of QMR.  In this paper we
describe how variational approximation methods can be applied to the
QMR network, resulting in fast diagnostic inference.  We evaluate the
accuracy of our methods on a set of standard diagnostic cases and
compare to stochastic sampling methods.


 MIT Computational Cognitive Science Technical Report 9701.