Layered hidden Markov model

The layered hidden Markov model (LHMM) is a statistical model derived from the hidden Markov model (HMM).

A layered hidden Markov model consists of N levels of HMMs, where the HMMs on level i + 1 correspond to observation symbols or probability generators at level i.

Every level i of the LHMM consists of Ki HMMs running in parallel.

Background

LHMMs are sometimes useful in specific structures because they can facilitate learning and generalization. For example, even though a fully connected HMM could always be used if enough training data were available, it is often useful to constrain the model by not allowing arbitrary state transitions. In the same way it can be beneficial to embed the HMM in a layered structure which, theoretically, may not be able to solve any problems the basic HMM cannot, but can solve some problems more efficiently because less training data is needed.

The layered hidden Markov model

A layered hidden Markov model (LHMM) consists of

N

{\displaystyle N}

levels of HMMs where the HMMs on level

N

+

1

{\displaystyle N+1}

corresponds to observation symbols or probability generators at level

N

{\displaystyle N}

.

Every level

i

{\displaystyle i}

of the LHMM consists of

K

i

{\displaystyle K_{i}}

HMMs running in parallel.

At any given level

L

{\displaystyle L}

in the LHMM a sequence of

T

L

{\displaystyle T_{L}}

observation symbols

o

L

=

{

o

1

,

o

2

,

…

,

o

T

L

}

{\displaystyle \mathbf {o} _{L}=\{o_{1},o_{2},\dots ,o_{T_{L}}\}}

can be used to classify the input into one of

K

L

{\displaystyle K_{L}}

classes, where each class corresponds to each of the

K

L

{\displaystyle K_{L}}

HMMs at level

L

{\displaystyle L}

. This classification can then be used to generate a new observation for the level

L

−

1

{\displaystyle L-1}

HMMs.