L8: Classification Problems

Question 1

What are the three important approaches for classification problems?

Answer 1

Question 2

SVM: what is the high-level intuition behind this?

Answer 2

• Imagine each column of our design matrix X formed a hyperplane –> in which we end up with two characteristics we want to separate the data on –>
• Ideally, these would be linearly separable data and see on different sides of the hyperplane –> and we are trying to find out how to split this plane to categorise our data correctly.

Question 3

SVM what is the Affine function for our hyperplane?

Answer 3

Question 4

SVM: What is our support vector and the notion of the “best hyperplane”?

Answer 4

Question 5

SVM: How do you actually form the support vector?

Answer 5

Question 6

SVM: What do we do when our features are generally not linearly separable?

Answer 6

Question 7

SVM: What is our dual optimisation problem when we account for soft margins?

Answer 7

Question 8

What are some non-linearly separable problems?

Answer 8

Question 9

What are Kernels and the Kernel trick?

Answer 9

Question 10

What is the RBF Kernel?

How does SVM look on a graph with a linear kernel vs an RBF kernel?

Answer 10

Question 11

What is the Summary of SVM?

Answer 11

Question 12

What is the Logistic Regression? What is the output of the model?

What does it estimate?

What is the log-loss function and what are we trying to do with it?

Answer 12

Question 13

Optimisation 1?

What is a convex and non-convex loss function?

What is the difference between a local minimum/global minimum, and a unique minimum?

Answer 13

Question 14

What is the optimisation problem in our linear regression?

Answer 14

Question 15

What 3 pieces of terminology is used interchangeably within optimisation problems?

Do they have any nuanced differences?

Answer 15

Question 16

Generally, what does constrained optimisation look like?

Answer 16

Question 17

What is the ordinal encoding of categorical features?

Answer 17

Question 18

What is one-hot and dummy encoding of categorical data?

Answer 18

Question 19

What are performance measures?

What are the common ones for classifications?

What are the common ones for regression?

Answer 19

Question 20

What is a confusion matrix?

Answer 20

Imagine we are testing for something bad e.g. if someone has an illness)

• False Positive = false alarm –> you say something is bad when its actually fine (waste resources and trust)
• False negative = missed detection –> you say something is fine when it is actually bad (cause real-world harm and risk)

Question 21

How do we calculate accuracy as a performance measure?

What is it importance and it limitations?

Answer 21

Question 22

What is precision and recall as performance measures?

Answer 22

Question 23

What is F₁ or, more generally, F as a performance measure?

Answer 23

Question 24

What is the Receiver Operating Characteristic Curve?

What is the Area Under the Curve (AUC)?

Answer 24

Question 25

What is the summary of all the performance metrics?

Answer 25

Question 26

Examples of when we would want different F_b measures?

Answer 26

Example 1: Medical Cancer Screening (False negative is worse!) so B >1 True Positive --> Patient has cancer and test detects False Positive --> Patient doesnt have cancer but tests positive for it (worrisome but not life threating) False Negative --> Patient has cancer but tests negative --> Thinks they are find but could die Example 2: Emall Spam Filter (False positives are worse) so B< 1 True Positive --> detects spam and deletes email and was spam False Positive --> detects spam and deletes but was an important personal email --> missed legal document this is bad! False negative --> allows spam email in which is annoying but not as bad as a false positive Example 3: Credit Card Fraud Detection --> beta=1 as only dataset will be unbalanced to normal people using their credit cards day to day True Positive --> detects someone is committing credit card fraud and they are False Positive --> Shut down normal peoples accounts --> angry customers and loss of business False Negative --> direct financial loss and fraud growth