paper available learning in BMs with linear response

[Bert Kappen]

Dear Connectionists,

The following article

Title: Efficient learning in Boltzmann Machines using linear response theory
Authors: H.J. Kappen and F.B. Rodriguez

can now be downloaded from as ftp://ftp.mbfys.kun.nl/snn/pub/reports/Kappen.LR_NC.ps.Z

This article has been accepted for publication in the journal Neural Computation.

Abstract: The learning process in Boltzmann Machines is computationally very
expensive. The computational complexity of the exact
algorithm is exponential in the number of neurons.
We present a new approximate learning algorithm
for Boltzmann Machines, which is based on mean field theory and
the linear response theorem. The computational complexity of the
algorithm is cubic in the number of neurons.
 
In the absence of hidden units, we show how the weights can be directly
computed from the fixed point equation of the learning rules.
Thus, in this case we do not need to use a
gradient descent procedure for the learning process.
We show that the solutions of this method are close to the optimal
solutions and give a significant improvement when correlations
play a significant role.
Finally, we apply the method to a pattern completion task and
show good performance for networks up to 100 neurons.

Best Regards,

Bert Kappen


FTP INSTRUCTIONS

unix% ftp ftp.mbfys.kun.nl
Name: anonymous
Password: (use your e-mail address)
ftp> cd snn/pub/reports/
ftp> binary
ftp> get Kappen.LR_NC.ps.Z
ftp> bye
unix% uncompress Kappen.LR_NC.ps.Z
unix% lpr Kappen.LR_NC.ps