Boltzmann Machine learning using mean field theory ...

[Bert Kappen]

Dear Connectionists,

The following article
will apear in the proceedings NIPS of 1998 ed. Micheal Kearns.
This version contains some significant improvements over the 
earlier version.

Boltzmann Machine learning using mean field 
theory and linear response correction 

written by (Hil)bert Kappen and Paco Rodrigues

We present a new approximate learning algorithm
for Boltzmann Machines, using a systematic expansion
of the Gibbs free energy to second order in the weights.
The linear response correction to the correlations
is given by the Hessian of the Gibbs free energy.
The computational complexity of the
algorithm is cubic in the number of neurons.
We compare the performance of the exact BM learning
algorithm with first order (Weiss) mean field theory and second order
(TAP) mean field theory. The learning task consists of a fully connected
Ising spin glass model on 10 neurons. We conclude that 1) the method
works well for paramagnetic problems
2) the TAP correction gives a
significant improvement over the Weiss mean field theory, both for
paramagnetic and spin glass problems
and 3) that the inclusion of diagonal weights
improves the Weiss approximation for paramagnetic problems, but not for
spin glass problems.

This article can now be downloaded from 
ftp://ftp.mbfys.kun.nl/snn/pub/reports/Kappen.LR_NIPS.ps.Z

Best regards,
                        Hilbert Kappen

FTP INSTRUCTIONS

unix% ftp ftp.mbfys.kun.nl
Name: anonymous
Password: (use your e-mail address)
ftp> cd snn/pub/reports/
ftp> binary
ftp> get Kappen.LR_NIPS.ps.Z
ftp> bye
unix% uncompress Kappen.LR_NIPS.ps.Z
unix% lpr Kappen.LR_NIPS.ps