The purpose of this set of experiments is to reveal the contribution of the second (multi-class) classification level in the overall recommendation ability of the Cascade Content-based Recommender. Equations6.41and6.42 provide the minimum and maximum values for the average MAE over all users, given the classification perfor-mance of the first (one-class) classification level. Having in mind that these lower and upper bounds on the average MAE concern the overall performance of the cascade recommender at both levels, they reflect the impact of the second (multi-class)
Fig. 7.5 MSE (mean for all users)
classification component. The lower bound on the average MAE corresponds to the best case scenario in which the second (multi-class) classification level performs inerrably. On the other hand, the upper bound on the average MAE corresponds to the worst case scenario, in which the second (multi-class) classification level fails completely. In this context, if we measure the actual value of the average MAE over all users, we can assess the influence of the second classification level on the overall recommendation accuracy of our system. Thus, if the actual value of the average MAE is close to the lower bound, this implies that the second classification level operated close to the highest possible performance level. On the other hand, if the actual value of the average MAE is closer to its upper bound, this implies that the second classification level did not contribute significantly to the overall performance of our recommender (Fig.7.5).
Figure7.7shows the actual average MAE relative to its corresponding lower and upper bound curves. Each curve is generated by parameterizing the one-class SVM classifier with respect to the fraction of the positive data that should be rejected during the training process.
The relative performance of one-class SVM-based classifier was measured in terms of precision, recall, F1-measure and MAE, which are defined in the following.
7.3 One-Class SVM—Fraction: Analysis 117
Fig. 7.6 Ranked Scoring (mean for all users)
The precision is defined as an average over all users and folds in relation to the average values for the true positives and the false positives:
Pr eci si on= T P
T P+F P. (7.1)
On the other hand, therecall is defined as the average over all users and folds in relation to the average values for the true positives and the false negatives:
Recall= T P
T P+F N. (7.2)
Fig. 7.7 MAE Boundaries for one-class SVM
Fig. 7.8 Hybrid Recommender 2nd level personality diagnosis: Fraction analysis
7.3 One-Class SVM—Fraction: Analysis 119
Fig. 7.9 Hybrid Recommender 2nd level Pearson correlation: Fraction analysis
Fig. 7.10 Hybrid Recommender 2nd level vector similarity: Fraction analysis
Finally, theF1-measureis defined as the average value for the F1-measure over all users and folds.
F1= 2×Pr eci si on×Recall
Pr eci si on+Recall (7.3)
The precision quantifies the amount ofinformationthat is not lost, while the recall expresses the amount of data that is not lost. Higher precision and recall values indicate superior classification performance. The F1-measure is a combination of precision and recall which ranges within the[0,1]interval. The minimum value(0) indicates the worst possible performance, while the maximum value(1)indicates the highest possible performance.
The MAE is a measure related to the overall classification performance of the Cascade Recommender. MAE values closer to zero indicate higher recommendation accuracy. It is very important to note that in the context of the highly unbalanced classification problem related to recommendation, the quality that dominates the level of the MAE is the number of the correctly classified negative patterns, i.e.
the true negatives. Since the vast majority of patterns belong to the negative class, correctly identifying them reduces the overall classification error. Thus, a lower MAE value for the one-class SVM classifier indicates that this classifier performs better in filtering out non-desirable patterns. On the other hand, the F1-measure, that specifically relates to precision and recall according to Eq.7.3, is dominated by the amount of positive patterns that are correctly classified (i.e., true positives), according to Eqs.7.1and7.2. The F1-measure quantifies the amount of true (thus, useful) positive recommendations that the system provides to the user.
Fig. 7.11 One class SVM (precision, recall, F1)
7.3 One-Class SVM—Fraction: Analysis 121
The previous findings are characteristic of the behavior of the one-class classifiers with respect to the fraction of positive and negative patterns that they identify during their testing process. Our experiments indicate the following:
• The precision performance of the one-class SVM classifier involves increasing true negative rates as the fraction of positive patterns rejected during training approaches 95 %.
• On the other hand, the recall performance of the one-class SVM classifier involves increasing true positive rates as the fraction of positive patterns rejected during training approaches 5 %.
An efficient one-class classifier attempts to achieve one of two goals: (1) to mini-mize the fraction of false positives and (2) to minimini-mize the fraction of false negatives.
Thus, it is a matter of choice whether the recommendation process will focus on in-creasing the true positive rate or inin-creasing the true negative rate. Inin-creasing the true negative rate results in lower MAE levels, while increasing the true positive rate results in higher F1-measure levels. Specifically, the fact that the non-desirable patterns are significantly higher in number than the desirable ones, suggests that the quality of recommendation is crucially influenced by the number of the correctly identified negative patterns. In other words, constraining the amount of the false pos-itive patterns that pass to the second level of the RS increases the reliability (quality) of the recommended items. The most appropriate measure to describe the quality of recommendation is given by the RSC, as the RSC illustrates the amount of true posi-tive items that are placed at the top of the ranked list. This fact is clearly demonstrated in Fig.7.6, where the RSC for the Cascade Content-based RS of the one-class SVM classifier outperforms the other recommendation approaches (Figs.7.7,7.8,7.9,7.10 and7.11).
Conclusions and Future Work
Abstract Recommender Systems (RS) attempt to provide information in a way that will be most appropriate and valuable to its users and prevent them from being overwhelmed by huge amounts of information that, in the absence of RS, they should browse or examine. In this book, we presented a number of innovative RS, which are summarized in this chapter. Conclusions are drawn and avenues of future research are identified.