ABSTRACT
This chapter describes hidden Markov model (HMM) methods for various problems in language processing. HMMs provide a powerful and flexible formalism for modeling sequences of words. They allow us to estimate the posterior probability that a document would be relevant to a user, given the user’s query or to compute the probability that a document discusses a particular set of topics. They allow us to determine automatically which words are related to which topics, even though each document is annotated with multiple topics. They even allow us to decompose an unannotated corpus of documents into its component set of topic basis functions. While some of the HMMs used are extremely simple, they afford a paradigm for model parameter estimation and offer the possibility of using more powerful models in the future. In information extraction, we can use HMMs to estimate the probability that a sequence of words in a particular context is a name of a particular type or that two entities in a text are related in a particular way. We describe simple HMMs used for these and other language processing tasks.
