ABSTRACT
This chapter contributes a comprehensive theory of text and picture comprehension that is embedded in a broader framework of human cognition and that considers the active, constructive nature of the human mind. It develops an integrated model of text and picture comprehension that is based on a semiotic and cognitive analysis of verbal and pictorial information processing. The model explains why the form of visualization used in a picture affects the structure of the mental model created during picture comprehension. According to the theoretical model, text comprehension and picture comprehension are to be considered as complementary ways of creating mental representations. The chapter considers three alternative hypotheses about the use of verbal and pictorial information: the independence hypothesis, the substitution hypothesis, and the stimulation hypothesis. It focuses on different hypotheses about how learning is affected by adding pictures, when the pictures use different forms of visualization: the dual coding hypothesis, the structure support hypothesis, and the structure interference hypothesis.
