Technical Report available

[Geoffrey Hinton]

Please do not reply to this message. To order a copy of the TR described
below, please send email to carol at ai.toronto.edu

_________________________________________________________________________

		       DETERMINISTIC BOLTZMANN LEARNING
		   IN NETWORKS WITH ASYMMETRIC CONNECTIVITY
				       
		   Conrad C. Galland and Geoffrey E. Hinton
			Department of Computer Science
			    University of Toronto
			    10 Kings College Road
			   Toronto M5S 1A4, Canada

			Technical Report CRG-TR-89-6
	

The simplicity and locality of the "contrastive Hebb synapse" (CHS) used in
Boltzmann machine learning makes it an attractive model for real biological
synapses.  The slow learning exhibited by the stochastic Boltzmann machine can
be greatly improved by using a mean field approximation and it has been shown
(Hinton, 1989) that the CHS also performs steepest descent in these
deterministic mean field networks.  A major weakness of the learning
procedure, from a biological perspective, is that the derivation assumes
detailed symmetry of the connectivity.  Using networks with purely asymmetric
connectivity, we show that the CHS still works in practice provided the
connectivity is grossly symmetrical so that if unit i sends a connection to
unit j, there are numerous indirect feedback paths from j to i. So long as the
network settles to a stable state, we show that the CHS approximates steepest
descent and that the proportional error in the approximation can be expected
to scale as 1/sqrt(N), where N is the number of connections.

________________________________________________________________________
PS: The research described in this TR uses a different kind of network and a
different analysis than the research described in the TR by Allen and
Alspector that was recently advertised on the connectionists mailing list.
However, the general conclusion of both TR's is the same.