KNN

Question 1

Can we determine the significance of the predictor in KNN?

Answer 1

No

Question 2

In which case KNN outperforms LDA and Logistic regression?

Answer 2

KNN is a completely non-parametric approach, no assumptions are made about the shape of the decision boundary. So, when the decision boundary is highly non linear, it dominates LDA and Logistic regression.

Question 3

In case of non-linear data, does KNN always outperforms other classification techniques?

Answer 3

No, even when the data exhibits the complex non-linear relationship, KNN can still give poor result if the level of smoothness is not chosen correctly

Question 4

How to determine the value of K in KNN?

Answer 4

Cross-validation

However, it is also sometime taken as Sqrt(n)

Question 5

Disadvantage of KNN?

Answer 5

1. Lazy learning technique - There is no such thing like training a KNN model. So if the number of data set is too large, it is too computationally expensive.
2. For high dimensional data, distance calculation using Euclidean distance is not reliable.
3. Affected by outliers
4. Poor performance in Imbalance dataset
5. Not interpretable - Black box model
6. Non-homogeneous dataset (need scaling)