ABSTRACT
The designer of an autonomous system chooses system goals and the system itself searches for and selects at least some aspects of actions that will best achieve those goals. In complex situations, the designer cannot afford to examine all possible operating conditions and the system’s response. This kind of autonomous system is rare today but will become much more common in the near future. Today, failures often arise from systems which were intended to be preprogrammed but whose designers neglected
certain operating conditions. These systems can have unintended bugs or security holes.
