Paper available on factorial hidden Markov models

[Zoubin Ghahramani]

FTP-host: psyche.mit.edu
FTP-filename: /pub/zoubin/facthmm.ps.Z

URL: ftp://psyche.mit.edu/pub/zoubin/facthmm.ps.Z

This technical report is 13 pages long [102K compressed].


		    Factorial hidden Markov models

	       Zoubin Ghahramani and Michael I. Jordan
	       Department of Brain & Cognitive Sciences
		Massachusetts Institute of Technology
			 Cambridge, MA 02139

We present a framework for learning in hidden Markov models with
distributed state representations. Within this framework, we derive a
learning algorithm based on the Expectation--Maximization (EM)
procedure for maximum likelihood estimation. Analogous to the standard
Baum-Welch update rules, the M-step of our algorithm is exact and can
be solved via a set of linear equations. However, due to the
combinatorial nature of the hidden state representation, the exact
E-step is intractable. A simple and tractable mean field approximation
is derived. Promising empirical results on a small time series
modeling problem are presented for both the mean field approximation
and Gibbs sampling.


      MIT COMPUTATIONAL COGNITIVE SCIENCE TECHNICAL REPORT 9502