ABSTRACT
At first glance, the oracle problem for systems with extensive cal-culations and complex conditions may seem almost insoluble: how can we check its correctness without implementing equally complex so- ware, whose correctness must also be checked, leading to an innite string of verication exercises? Assertions and self-checking soware can help (Chapter 11), but they are not always su cient. In other cases, previous versions of the soware may be available to check at least the old functionality, or the code may be implementing a formal standard (e.g., for network protocols or cryptography) and other implementations may exist to compare against. In most cases though, the soware is doing something new, and we need to verify that it is working correctly for a large set of possible inputs. e di culty of devising a set of complete tests with inputs and expected results is one of the reasons why ad hoc approaches such as “use cases” are widespread. Testers use formal or informal requirements to determine anticipated system uses, plus inputs and outputs for each such use, a slow and expensive way to develop a test oracle. To make thorough testing practical, more automated approaches are needed.
