ABSTRACT
The artificial intelligence black-box problem refers to the phenomenon that with most artificial intelligence and its tools one does not know how they do what they do. Evaluating black boxes is familiar territory for evaluators. However, realist, theory-driven evaluators have rarely applied this approach to black boxes when big data/artificial intelligence are involved in policy making and implementation. This essay outlines an approach to do that. First, several components of the black box problem are mentioned. Next, a six-step approach to unpack artificial intelligence black boxes is presented.
