Tactical scenarios for user-based performance evaluation
While laboratory-based experiments are critical to research, development of new systems also requires context-driven evaluation of technology to assure generalizability to realistic operator task demands. The capture of Soldier performance demands is particularly challenging in this regard, as many factors converge to impact performance "in the wild", such as interactions with other soldiers also performing operational tasks, tactical conditions and additional workload that is multifaceted, dynamic, and often uncertain and stressful. Thus, equipment or principles that work well in laboratory conditions need follow-up evaluations within controlled scenario-based situations that have critical characteristics of core work demands. Scenarios having well-defined boundary conditions (e.g., realistic and well articulated task demands, user experience, etc.) thus yield assessments of performance and operational relevance that are arguably valid. This report describes issues and outcomes related to development of scenarios and measures for the assessment of devices for dismount soldier teams. Procedures and results are discussed within the context of cognitive task analytic techniques
Keywords: Cognitive task analysis; Multisensory systems; Scenario validation
U.S. military operations involve ever-increasing interactions with advanced technology-from vehicle consoles and wearable displays to robot control interfaces. Soldiers are now expected to master an array of sensors, controls, and displays. These capabilities can enable Soldiers to see in the dark, maintain mapbased situation awareness, control semi-autonomous air and ground vehicles, and network up and down communication channels. However, it should be noted that the Soldier/user can also easily be overcome with technology complexity, high workload, information overload, and distraction from critical mission events. It is critical that these systems be evaluated in situ for effectiveness and ease of use. Lab-based performance criteria are not sufficient. Engineering developers focused on a particular capability may indeed succeed to performance criteria (e.g., ability to detect and identify a human threat at 200 meters using night vision) and yet still produce a device that is unsuited for operational use.