4. Classification

Question 1

Purpose of Classification (and by extension Purpose of Logistic Regression)

Answer 1

Predicting qualitative or categorical outcome of Y

(Linear would be possible with dummy variable, but interpretation difficult…)
(Linear regression assumes an outcome variable that is continuous or quantitative, therefore any implied ordering is not real )

Question 2

Classification Techniques

Answer 2

• Decision Tree
• Bagging, RF, Boosting
• Neural Networks
• Logistic Regression (continuous, discrete variables)
• Support Vector Machines (SVM)

Question 3

Logistic function and log odds function

Answer 3

p(x)
and
log (odds) = logistic regression coefficients

Question 4

How to estimate coefficients in logistic regression

Answer 4

Maximum Likelihood method
1. Produce S-shaped curve
2. Product ( individual likelihoods pi ) * Product ( (1 - pi) )

( or alternatively add the log-likelihoods )

Question 5

Classification model performance measures w/ formula

Answer 5

• (P) Precision: TP / TP + FP
• (TP) True positive/ Recall/ Sensitivity: TP / TP + FN
• (TN) True negative rate (Specificity): TN / TN + FP
• (FP) False positive rate: FP / FP + TN
• F-measure: (2RecallPrecision)/(Recall+Precision)