ABSTRACT
Artificial neural networks (ANNs) have been successfully applied to real-world problems as varied as steering a motor vehicle [10] and pronouncing English text [13]. In addition to these practical successes, several empirical studies have concluded that neural networks provide performance comparable to, and in some cases, better than common symbolic learning algorithms [1, 3, 6]. A distinct advantage of symbolic learning algorithms, however, is that the concept representations they form are usually more easily understood by humans than the representations formed by neural networks. In this paper we describe and investigate an approach for extracting symbolic rules from trained ANNs. Our approach uses the NofM algorithm [16] to extract rules from networks that have been trained using Nowlan and Hinton’s method of soft weight-sharing [7]. Although soft weight-sharing was designed as a technique for improving generalization in neural networks, we explore it here as a means for facilitating rule extraction. We present experiments that demonstrate that our method is able to learn rules that are more accurate than rules induced using a common symbolic learning algorithm – Quinlan’s C4.5 system [11]. Furthermore, the rules that are extracted from our trained networks are comparable to rules induced by C4.5 in terms of complexity and understandability. We also present a method that simplifies extracted rules by pruning antecedents from them. Our experiments show that this technique improves both the comprehensibility and the generalization performance of extracted rules.
