ABSTRACT
Contents Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320 15.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320 15.2 Audio signal processing (audio, speech, text, contextual
events, interaction modeling) . . . . . . . . . . . . . . . . . . . . . . . . . . 322 15.2.1 Audio acquisition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323 15.2.2 Automatic speech recognition and synthesis . . . . . . . . 324 15.2.3 Affective speech processing . . . . . . . . . . . . . . . . . . . . . 325 15.2.4 Speech prosody. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326 15.2.5 Modeling interaction with speech cues . . . . . . . . . . . . 329 15.2.6 Speech: Future directions and open challenges . . . . . . 331
15.3 Visual signal processing (facial expressions, eye tracking, gestures) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331 15.3.1 Facial processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332
15.4 Physiological signal processing ( heart rate and skin conductance) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333 15.4.1 Signal fundamentals . . . . . . . . . . . . . . . . . . . . . . . . . . . 334 15.4.2 Collecting physiological signals . . . . . . . . . . . . . . . . . . 334 15.4.3 Physiological signal-processing techniques . . . . . . . . . 335 15.4.4 Behavioral signal processing for physiological data. . . 336
15.5 Meta-data processing with machine learning. . . . . . . . . . . . . . 338 15.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 340
Abstract Human communication is an intricate exchange of information occurring at different timescales through the coordination of several modalities; facial expressions, body language, and intonation coincide with spoken words for effective transferal of information. For children with autism spectrum disorder (ASD), the subtle coordination between modalities, which seems so natural to the experienced communicator, can be dif-cult to ascertain and master. In this chapter, we discuss the ways in which signal processing of behavioral data can aid in our understanding and treatment of autism and other behavioral disorders.
