Connection between Hidden Markov Models and Connectionist Networks

[thanasis kehagias]

the following paper explores the connection between Hidden Markov Models
and Connectionist networks. anybody interested in a copy, email me.
if you have a TeX setup i will send you the dvi file. else give me your
physical mail address.

           OPTIMAL CONTROL FOR TRAINING
            THE MISSING LINK BETWEEN
             HIDDEN MARKOV MODELS
           AND CONNECTIONIST NETWORKS

                 by Athanasios Kehagias
               Division of Applied Mathematics
                 Brown University
                Providence, RI 02912



                       ABSTRACT

For every Hidden Markov Modl there is a set of forward probabilities
that need to be computed for both the recognition and training problem .
These probabilties are computed recursively and hence the computation
can be performed by a multistage , feedforward network that we will
call Hidden Markov Model Network (HMMN). This network has exactly
the same architecture as the standard Connectionist Network(CN).
Furthermore, training a Hidden Markov Model is equivalent to optimizing
a function of the HMMN; training a CN is equivalent to minimizing
a function of the CN. Due to the multistage feedforward architecture,
both problems can be seen as Optimal Control problems. By applying
standard Optimal Control techniques, we discover in both problems
that certain back propagating quantities (backward probabilities
for HMMN, backward propogated errors for CN) are of crucial
importance for the solution. So HMMN's and CN's are similar both
in architecture and training.

          **************

i was influenced in this research by the work of H. Bourlard and C.
C. Wellekens (the HMM- CN connection) and Y. leCun (Optimal Control
applications in CN's). as i was finishing my aper i received a message
by J.N. Hwang saying that he and  S.Y. Kung have written a paper that
includes similar results.




                         Thanasis Kehagias