Detailed Balance

[aboulang@BBN.COM]

   The property (2) is called detailed balance resulting in a Gibbs
   distribution for the probability to find the system in a particular
   state. The rule (1) is an update procedure for the spin Sk which
   ensure detailed balance provided that E is an energy. Both principles
   are fundamental facts of statistical mechanics of neural networks
   (or if you prefer result from an maximum entropy analysis of neural
   nets). The book by Hertz Krogh and Palmer summerizes all that in
   a nice way. The book title is "Introduction to Neural Computation".


We really should be saying that detailed balance in sampling implies a
Gibbs distribution, but that the Gibbs distribution does not imply the
use of a sampling procedure with detailed balance. There is some new
work on this:

J. Marroquin & A. Ramerez
"Stochastic Cellular Automata with Gibbsian Invariant Measures"
IEEE Trans Information Theory May(*), 1991

* I can't find the paper so I may have the month wrong.

This is potentially good news to people trying to get annealing-type
algorithms to work for fine-grained MIMD parallelism.

Regrads,
Albert Boulanger
aboulanger at bbn.com