inputs to RBF classifiers. The attribute subsets with the lowest classification error rates and the least numbers of attributes are selected. Rules are extracted based on feature subsets selected. Compared to other methods, more concise and accurate rules are extracted for the Iris and Breast cancer data sets.
Although, for the Monk3 data set, the rule accuracy is slightly lower, the number of antecedents per rule is smaller than other methods. For the Thyroid data set, compared with the rule method based on GAs in Sect. 7.4, the rule accuracy is higher and fewer rules are needed after data dimensionality reduction. For the Mushroom data set, a high rule accuracy is obtained with fewer premises compared to other methods.
In general, DDR results lead to a less complicated RBF neural network architecture. As a decompositional algorithm, in this rule extraction method, one hidden unit corresponds to one initial rule. Hence, compact rules can be extracted from compact RBF neural networks. Experimental results show that our rule extraction method is simple for implementation and can lead to concise rules and high rule accuracies.
7.7 Rule Extraction Based on Class-dependent Features
7.7.1 The Procedure of Rule Extraction
In this section, rule extraction is carried out for concise rules based on class-dependent features. We demonstrate our approach using computer simula-tions. The rule extraction algorithm described here is based on the trained RBF neural network classifier with class-dependent features. For each class, a subset of features is selected in order to discriminate the class from other classes. A group of kernel functions is generated for the class based on the selected feature subset. Each hidden neuron of the RBF neural network is responsive to a subset of input patterns (instances).
7.7.2 Experimental Results
The Thyroid data set and the Wine data set from the UCI Repository of Machine Learning Databases [223] are used in this section to demonstrate our algorithm.
Thyroid Data Set
It is shown in the feature masks (Table 6.3) that feature 1 does not play an important role in discriminating classes. Hence, the T3-resin uptake test can be unnecessary in this type of Thyroid diagnosis. For class 3, feature 2 can discriminate class 3 from other classes. Features 2 and 3 are used to
186 7 Rule Extraction from RBF Neural Networks
Table 7.14. Rule accuracy for the Thyroid data set based on class-dependent fea-tures.
Rule accuracy Full features Class-dependent features
Training set 94.57% 95.54%
Validation set 95.35% 94.6%
Testing set 90.7% 95.48%
discriminate class 2 from other classes. Features 2, 3, 4, and 5 are used to discriminate class 1 from other classes.
Two rules are extracted for the Thyroid data set based on class-dependent features. The rule accuracy (in Table 7.14) is 95.54% for the train-ing data set, 94.6% for the validation data set, and 95.48% for the test data set. With full features as inputs, two rules are obtained, and the rule accuracy is 94.57% for the test data set, 95.35% for the training data set, and 90.7%
for the validation set. Thus, higher rule accuracy and more concise rules are obtained when using class-dependent features compared to full features.
Rules for the Thyroid data set based on class-dependent features are:
Rule 1:
IF attribute 2 is within the interval (12.9, 25.3 ) AND attribute 3 is within the interval (1.5, 10) THEN the class label is hyper-thyroid.
Rule 2:
IF attribute 2 is within the interval (0, 5.67) THEN the class label is hypo-thyroid.
Default rule:
the class label is normal.
Wine Data Set
It is shown in the feature masks (Table 6.4) that the feature subset{2,4,5,6,7,9,11,12} plays an important role in discriminating class 1 from other classes, the
fea-ture subset{3,4,5,6,7,10,11,12,13}is used to discriminate class 2 from other classes, and the feature subset{2,3,11,12,13}is used to discriminate class 3 from other classes.
Seven rules are extracted for the Wine data set based on class-dependent features. The rule accuracy (in Table 7.15) is: 88.7% for the training data set, 83.4% for validation data set, and 86.1% for the test data set. With full fea-tures as inputs, seven rules are obtained, and the rule accuracy is 90.6% for the training data set, 77.8% for the validation set, and 86.1% for the test set.
Thus, the same rule accuracy and more concise rules with fewer premises are obtained when using class-dependent features compared to using full features.
7.7 Rule Extraction Based on Class-dependent Features 187 Table 7.15.Rule accuracy for the Wine data set based on class-dependent features.
Rule accuracy Full features Class-dependent features
Training set 90.6% 88.7%
Validation set 77.8% 83.4%
Test set 86.1% 86.1%
7.7.3 Summary
In this section, we have described a rule extraction method from our RBF classifier based on class-dependent features. The discriminatory power of each feature for discriminating classes is considered for each class. Different feature subsets are selected for different classes individually based on their ability in discriminating the class from other classes, which show the relationship be-tween the feature subset and the class concerned. The class-dependent feature selection results obtained above provide a new way for rule extraction. The Thyroid and Wine data sets are used to demonstrate the algorithm. Experi-mental results show that our algorithm is effective in reducing the number of feature inputs and leads to compact and accurate rules simultaneously.