ABSTRACT
Meaningful human control (MHC) of safety-critical systems is an important goal as digitalisation, automation/artificial intelligence (AI) are implemented, and remote oversight is introduced. In the EU/AI regulation, the concept of human oversight is introduced, especially for safety-critical operations. MHC and human oversight are challenging because they depend on human strengths and weaknesses, system design, knowledge and training, and organisational factors like responsibilities, staffing, and work processes. MHC is more useful than human oversight because it ensures that systems and organisational structures are designed to keep humans in control of safety-critical operations, thereby preventing disasters. However, to be useful, MHC needs to be defined and specified. This chapter aims to define MHC by addressing three key areas: design, operations, and learning. 1) Design issues include adopting a system approach, using human-centred design best practices, conducting task analysis to manage cognitive workload, creating consistent interfaces for quick situational understanding, designing alarms to support situational awareness (SA), and establishing work processes that promote shared SA across teams. 2) Operational issues include ensuring safety, managing change, addressing error traps and training, and maintaining physical and mental conditions to enable MHC in all situations. In a critical situation, we observe that it can take ten minutes to observe, understand and act correctly in crises. 3) Main issues in learning from accidents must be to identify root causes, including poor concepts/design, and to understand reasons for human SA and actions. We have used ‘Human Error’ as a starting point for analysis. Learning and understanding should drive change and improvement in governing values, prioritising learning over blame.
