Paper on mixtures of probabilistic PCA

[Prof. Chris Bishop]

        Mixtures of Probabilistic Principal Component Analysers
        =======================================================

             Michael E. Tipping and Christopher M. Bishop

                   Neural Computing Research Group
            Dept. of Computer Science & Applied Mathematics
                 Aston University, Birmingham B4 7ET


                  Technical Report NCRG/97/003

                  Submitted to Neural Computation


Abstract
--------

Principal component analysis (PCA) is one of the most popular
techniques for processing, compressing and visualising data, although
its effectiveness is limited by its global linearity. While nonlinear
variants of PCA have been proposed, an alternative paradigm is to
capture data complexity by a combination of local linear PCA
projections. However, conventional PCA does not correspond to a
probability density, and so there is no unique way to combine PCA
models. Previous attempts to formulate mixture models for PCA have
therefore to some extent been ad hoc. In this paper, PCA is formulated
within a maximum-likelihood framework, based on a specific form of
Gaussian latent variable model. This leads to a well-defined mixture
model for probabilistic principal component analysers, whose
parameters can be determined using an EM algorithm. We discuss the
advantages of this model in the context of clustering, density
modelling and local dimensionality reduction, and we demonstrate its
application to image compression and handwritten digit recognition.


Available as a postscript file:

  http://neural-server.aston.ac.uk/Papers/postscript/NCRG_97_003.ps.Z

To access other publications by the Neural Computing Research Group, go
to the group home page:

  http://www.ncrg.aston.ac.uk/

and click on `Publications' -- you can then obtain a list of all 
online NCRG publications, or search by author, title or abstract.