ABSTRACT

It would be difficult to find a topic associated with human behavior representations (HBRs) that brings forth more spirited and lively discussion than the topic of validation. Many assertions on this topic elicit almost violent agreement. For example, it is generally agreed that validation is tremendously important, and the risk of drawing erroneous conclusions from unvalidated models is unacceptable (e.g., U.S. Department of Defense, 2001a). Authors of one recent report (Committee on Technology for Future Naval Forces, National Research Counsel [NRC], 2003) stated that the need to ensure valid model content is so critical it is worth an investment of $20 to $30 million per year. In addition, there is general agreement that HBR validation is a difficult and costly process (e.g., Ritter & Larkin, 1994; U.S. Department of Defense, 2001b). Finally, most in the community would probably agree that HBR validation is rarely, if ever, done (e.g., Harmon, Hoffman, Gonzalez, Knauf, & Barr, 1999). The issue that elicits disagreement, on the other hand, is exactly what activities and results constitute a demonstration of satisfactory HBR validation. The goal of this chapter is to focus on this issue and attempt to incorporate insights from several disciplines, including software engineering, mathematics, statistics, and psychology, bringing in examples from the AMBR project whenever appropriate.