ABSTRACT
Markov chain models and hidden Markov models have been widely used to build models and make inferences of sequential data patterns. This chapter describes Markov chain models and hidden Markov models. It provides a list of data mining software packages that support the learning and inference of Markov chain models and hidden Markov models. Some applications of Markov chain models and hidden Markov models are given with references. Markov chain models can be used to learn and classify sequential data patterns. In the applications of Markov chain models to cyber attack detection, computer audit data under the normal use condition and under various attack conditions on computers are collected. The Markov chain model for a target class is learned from the training data under the condition of the target class. For each test sequence of audit events in an observation window, the joint probability of the test sequence is computed under each Markov chain model.
