Classification
k Nearest-Neighbors
Ana Karina Fermin
ISEFAR
1 k Nearest-Neighbors
2 Generative Modeling (Naive Bayes, LDA, QDA)
3 Logistic Modeling
4 SVM
5 Tree Based Methods (M. Zetlaoui course, Apprentissage)
Methods
1 k Nearest-Neighbors (knn)
Example: kNearest-Neighbors (withk= 3)
1 2
3 4
Example: kNearest-Neighbors (withk= 4)
kNearest-Neighbors
Neighborhood Vx ofx: k closest from x learning samples.
k-NN as local conditional density estimate
bp+1(x) = P
xi∈Vx1{yi=+1}
|Vx|
KNN Classifier:
fbKNN(x) =
(+1 ifpb+1(x)≥pb−1(x)
−1 otherwise
Example: TwoClass Dataset
Synthetic Dataset
Two features/covariates.
Two classes.
Dataset from Applied Predictive Modeling, M. Kuhn and K.
Johnson, Springer
Numerical experiments withR.
Example: KNN
Example: KNN
Example: KNN
Application : model selection using cross validation
Example: KNN (bk= 25 using cross-validation)
