ABSTRACT
At the present time, expert system technology has no widely recognized and accepted methodology by which the development of expert systems is achieved. The most commonly used method of development is a trial-and-error method, called iterative refine ment or prototyping. Simplistically stated, this method consists of obtaining a set of example test cases, determining from the domain expert how he/she would solve each of these test cases, and encoding this expertise into a knowledge base. This process is repeated until an expert system evolves which satisfies the re search being conducted or which meets the customer’s needs. When the iteration stops, the expert system is validated on a small set of test cases. These test cases are often the same ones upon which the knowledge base is built or scenarios encountered in the operational environment. In both situations, the verification and validation testing and the analysis of the results from the testing are performed in an informal, arbitrary manner. This testing and analysis entails the knowledge engineer or customer manually executing test cases and checking the results against the expected results. While this empirical approach to development was satis factory for the development of expert systems within the research laboratory environment, more systematic, structured approaches to development are needed to produce quality expert systems for commercial applications. The empirical development of iterative refinement is “not supported by sound and general methodolo gies. It is more like handicraft than engineering, and it lacks several of the desirable features of an industrial process (reliability,
repeatability, work-sharing, cost [estimation], quality assurance, etc.).” [Guida, 1989b]
Much work has been done in developing the methods or techniques needed to produce quality expert systems for commer cial applications. The process model for expert system develop ment, described in Section III below, illustrates many of the technical and managerial activities needed for the production of quality expert systems. Some of these activities ate adapted from the concepts of the software engineering life cycle for conven tional software. This model, like many recently proposed process models for expert system development, supports the belief that the “commercial development and implementation of expert systems necessitates a synthesis of the life-cycle and prototyping strategies as prototypes must eventually be transformed into usable produc tion systems.” [Agarwal, 1990] This process model represents the future in the development of expert systems; it does not represent common practice in expert system development. The common practice is iterative refinement. Research is still needed on many of the concepts presented in this example process model. Verifi cation and validation are some of the concepts which require more definition, methods, techniques, and tools in order to be an integral part of the production of quality expert systems. The validation needed for expert system development, its relationship to the verification and evaluation of expert systems, and tech niques for providing verification and validation of expert systems are described in the following sections.
