ABSTRACT
CONTENTS 10.1 Hidden Markov Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238 10.2 Mathematical Framework of HMMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238 10.3 Non-Deterministic Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 10.4 Hidden Markov Models (HMMs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240 10.5 Hidden Markov Models: Mathematical Definition . . . . . . . . . . . . . . 241 10.6 HMM Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243 10.7 Hidden Markov Models: Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244 10.8 Hidden Markov Models: Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247 10.9 HMMs: Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252 10.10 ExampleMatlab HMM Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 10.11 Hidden Markov Models for Speech . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 10.12 HMM Emission Probability Estimation for ASR . . . . . . . . . . . . . . . . 254 10.13 Single-Word Speech Recognition System . . . . . . . . . . . . . . . . . . . . . . . 255 10.14 HMMs for ASR: Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255 10.15 Emission Probability Estimation using Gaussian Mixture
Models (GMMs) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256 10.16 ASR using Deep Neural Networks (DNNs) . . . . . . . . . . . . . . . . . . . . . 257 10.17 Moving Forward: HMM-Free ASR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259 10.18 HMM, GMMs and DNNs: Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . 262 10.19 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
This chapter firstly introduces Hidden Markov Models (HMMs) as a key underlying method for the analysis of speech. HMMs have been extremely important for current and historical ASR systems. Until fairly recently the state of the art performance of ASR systemswas achieved through the combi-
More recent have replaced GMMswith DeepNeural Networks (DNNs). For large training datasets, the current state of the art ASR systems have moved away from usingHMMs and nowdirectly use DNNs such as RNNs in so-called end-to-end DNNs for ASR.
