ABSTRACT
In a match against Lee Sedol, one of the greatest contemporary Go players, DeepMind’s AI programme AlphaGo played a move that stunned commentators at the time, who described it as ‘unthinkable’, ‘surprising’, ‘a big shock’, and ‘bad’. Move 37 turned out to be key to AlphaGo’s victory in that game, and it displays what in this chapter is referred to as the property of ‘unpredictable brilliance’. Unpredictable brilliance also poses a challenge for a central use-case for AI in the military, namely in AI-enabled decision-support systems. Advanced versions of these systems can be expected to display unpredictable brilliance, while also posing risks, both to the safety of blue force personnel and to a military’s likelihood of success in its campaign objectives. The central task of this chapter is to show how the management of these risks will result in the redistribution of responsibility for performance in combat away from commanders, and toward the institutions that design, build, authorize, and regulate these AI-enabled systems. Surprisingly, this redistribution of responsibility is structurally akin for systems in which humans are ‘in the loop’ as it is for those in which humans are ‘out’ of it.
