ABSTRACT
The development of automated decision support systems requires a real-time metric of decision quality. Historical links between decision quality and mission effectiveness suggest that the former is predictive of the latter, and thus the outcome is what needs to be measured. The purpose of the present paper is to describe an experiment designed to separate the measurement of decision quality from that of mission effectiveness. Mission effectiveness and decision quality were examined using the RoboFlag simulation environment. Environmental uncertainty and knowledge of the opponent’s strategies were manipulated factorially. Measures of mission effectiveness and decision quality were obtained along with subjective assessments of situation awareness and workload. Results are discussed in terms of decision quality as expressed by the tactics assigned to the operator’s robots, and with respect to the mission outcome for each trial. Further, the number and appropriateness of re-tasking orders and the effects of unreliable information are addressed.
