paper available: Mean Field Theory for Sigmoid Belief Networks
Lawrence Saul
lksaul at psyche.mit.edu
Tue May 9 18:01:25 EDT 1995
FTP-host: psyche.mit.edu
FTP-file: pub/lksaul/belief.ps.Z
The following paper is now available by anonymous ftp.
==================================================================
Mean Field Theory for Sigmoid Belief Networks (12 pages)
Lawrence K. Saul, Tommi Jaakkola, and Michael I. Jordan
Center for Biological and Computational Learning
Massachusetts Institute of Technology
79 Amherst Street, E10-243
Cambridge, MA 02139
Abstract:
Bayesian networks (a.k.a. belief networks) are stochastic
feedforward networks of discrete or real-valued units.
In this paper we show how to calculate a rigorous lower
bound on the likelihood of observed activities in sigmoid
belief networks. We view these networks in the framework
of statistical mechanics and derive a mean field theory
for the average activities of the units. The advantage of
this framework is that the mean field free energy gives a
rigorous lower bound on the log-likelihood of any partial
instantiation of the network's activity. The feedforward
directionality of belief networks gives rise to terms that
do not appear in the mean field theory for symmetric networks
of binary units. Nevertheless, the mean field equations have
a simple closed form and can be solved by iteration to yield
a lower bound on the likelihood. Empirical results suggest
that this bound may be tight enough to serve as a basis for
inference and learning.
==================================================================
More information about the Connectionists
mailing list