Paper available -- Independent Factor Analysis

[Hagai Attias]

A new paper on a simple graphical model approach to the problem of 
blind separation of independent sources, using exact and variational EM, 
is available at

   http://keck.ucsf.edu/~hagai/papers.html


- ---------------------------------------------------


	  INDEPENDENT FACTOR ANALYSIS

	       Hagai Attias, UCSF

         (Neural Computation, in press)


We introduce the independent factor analysis (IFA) method for recovering 
independent hidden sources from their observed mixtures. IFA generalizes 
and unifies ordinary factor analysis (FA), principal component analysis (PCA), 
and independent component analysis (ICA), and can handle not only square
noiseless mixing, but also the general case where the number of mixtures
differs from the number of sources and the data are noisy. 

IFA is a two-step procedure. In the first step, the source densities, mixing 
matrix and noise covariance are estimated from the observed data by maximum 
likelihood. For this purpose we present an expectation-maximization (EM) 
algorithm, which performs unsupervised learning of an associated probabilistic 
model of the mixing situation. Each source in our model is described by a 
mixture of Gaussians, thus all the probabilistic calculations can be performed 
analytically. In the second step, the sources are reconstructed from the 
observed data by an optimal non-linear estimator. 

A variational approximation of this algorithm is derived for cases with a large 
number of sources, where the exact algorithm becomes intractable.

Our IFA algorithm reduces to the one for ordinary FA when the sources become 
Gaussian, and to an EM algorithm for PCA in the zero-noise limit. We derive an 
additional EM algorithm specifically for noiseless IFA. This algorithm is shown 
to be superior to ICA since it can learn arbitrary source densities from the 
data. Beyond blind separation, IFA can be used for modeling multi-dimensional 
data by a highly constrained mixture of Gaussians, and as a tool for non-linear 
signal encoding.