Phase transitions in Hidden Markov Models

In \textit{Hidden Markov Models} (HMMs), a Markov process is not directly accessible. In the simplest case, a two-state Markov model “emits” one of two “symbols” at each time step. We can think of these symbols as noisy measurements of the underlying state. With some probability, the symbol implies that the system is in one state when it is actually in the other. The ability to judge which state the system is in sets the efficiency of a Maxwell demon that observes state fluctuations in order to extract heat from a coupled reservoir. The \textit{state-inference problem} is to infer the underlying state from such noisy measurements at each time step. We show that there can be a phase transition in such measurements:\footnote{John Bechhoefer, \textit{New J. Phys.} \textbf{17}, 075003 (2015).} for measurement error rates below a certain threshold, the inferred state always matches the observation. For higher error rates, there can be continuous or discontinuous transitions to situations where keeping a memory of past observations improves the state estimate. We can partly understand this behavior by mapping the HMM onto a 1d random-field Ising model at zero temperature. We also present more recent work that explores a larger parameter space and more states.