technical reports available

[Geoffrey Hinton]

Two new technical reports are now available at
http://www.gatsby.ucl.ac.uk/hinton/chronological.html

_______________________________

Training Products of Experts by Maximizing Contrastive Divergence
                        Geoffrey Hinton
               Technical Report GCNU TR 2000-004

                            ABSTRACT
It is possible to combine multiple probabilistic models of the same
data by multiplying their probability distributions together and then
renormalizing.  This is a very efficient way to model high-dimensional
data which simultaneously satisfies many different low-dimensional
constraints because each individual expert model can focus on giving
high probability to data vectors that satisfy just one of the
constraints.  Data vectors that satisfy this one constraint but
violate other constraints will be ruled out by their low probability
under the other experts.  Training a product of experts appears
difficult because, in addition to maximizing the probability that each
individual expert assigns to the observed data, it is necessary to
make the experts be as different as possible. This ensures that the
product of their distributions is small which allows the
renormalization to magnify the probability of the data under the
product of experts model. Fortunately, if the individual experts are
tractable there is an efficient way to train a product of experts.

__________________________________

         Learning Distributed Representations of Concepts 
               Using Linear Relational Embedding

             Alberto Paccanaro and Geoffrey Hinton
               Technical Report GCNU TR 2000-002

                             ABSTRACT

In this paper we introduce Linear Relational Embedding as a means of
learning a distributed representation of concepts from data consisting
of binary relations between concepts. The key idea is to represent
concepts as vectors, binary relations as matrices, and the operation
of applying a relation to a concept as a matrix-vector multiplication
that produces an approximation to the related concept. A repesentation
for concepts and relations is learned by maximizing an appropriate
discriminative goodness function using gradient ascent.  On a task
involving family relationships, learning is fast and leads to good
generalization.