! Supervised Learning: Regression

Question 1

Regression

Answer 1

• finding & predicting relationship between indepent & continuous output/dependent variables
• e.g. predict price of car given set of features (age, brand)
• supervised ML

Question 2

Regression - Tasks

Answer 2

• Linear Regression
• Neural Networks

Question 3

Linear Regression - Goal

Answer 3

• learn a linear model that we can use to predict a new y given a previously unseen x with as little error as possible
• parametic

Question 4

Linear Regression - Method

Answer 4

• Linear equation: y’ = ß0 + ß1x1 + … + ßixi
• estimating coefficient(s) (weights wi, bias b) so that
• predicts continous variable (y’) based on other variable(s) (xi) in best way
• Finding straight line minimizing distance between predicted & actual output

Question 5

Linear Regression - Steps

Answer 5

• 1. Define cost / loss function: measures inaccuracy of models prediction
• 2. Find parameters minimizing loss function: make model as accurate as possible -> Gradient Descent

Question 6

Linear Regression - Steps - Gradient Descent

Answer 6

• method to find minimum of model’s (y’ = ß0 + ß1x1 + … + ßixi) loss function by iteratively process
• Function f(ß0, ß1) = z <- focus on ß1 & ß0 (other variables of cost f given)
• Guess ß0 & ß1
• [dz/dβ0, dz/dβ1] <- Get partial derivatives of loss function with respect to each beta (= how much total loss is increased or decreased if increase ß0 or ß1 by very small amount)
• Adjust ß0 & ß1 accordingly: if partial derivative < 0 -> increase ß1; > 0 -> decrease ß2
• Repeat 3 & 4 until partial derivative = 0

Question 7

Linear Regression - Steps - define cost function

Answer 7

• a) Take average of
• b) Square (-> no negative numbers penalizing large differences)
• c) the sum of all differences between each data point (yi) & model predictor (ß1xi + ßo)

Question 8

Linear Regression - Solution over- & underfitting

Answer 8

• 1. Use more training data
• 2. Use regularization = penalty to loss function when model assigns too much power to 1 feature or to too many features (c + Lambda (sum of ßi^2) = hyperparameter -> higher: more penalty)

Question 9

Regression - Types

Answer 9

simple = 1 independent variable <- > Multiple > 1 independent variable

Question 10

Linear Regression - Assumptions / Dissdvantages

Answer 10

• variables = quantitative (Categorical -> binary); measure at continous level
• No significant outliers
• Residuals (errors) of best-fit regression line = normal distribution
• relationship between dependent & independent variables = linear
• all observations = independent

Question 11

Supervised Learning Goal

Answer 11

• find relation btw input & output data based on already knows answers (labeled data)
• apply to predict outcomes of new variables

Question 12

Supervised application

Answer 12

image classification (dog or cat?), fraud detection, spam filtering

Question 13

Logistic Regression

Answer 13

• predict discrete class based on probability
• put linear regression f(x) in sigmoid function P(Y=1) = 1 / (1+e^-lf(x)) -> result = probability btw 0 & 1
• threshold: calss decision based on tolerance false + / -

Question 14

Logistic Regression - Logg-odds-ratio

Answer 14

• gonna die ln(p to (1-p) ) log-ods

Question 15

Logistic Regression - Con

Answer 15

• risk of overfitting if large number of independent variables
• need sample of even categories