What are predictors (features)?
The input variables x1,x2,…,xp. (Ideally) Independent variables used to predict y.
What is the response variable?
The output y. Also called the target or dependent variable. The thing we are trying to predict.
What is the goal of statistical learning?
Estimate a function f(x) that relates predictors to response.
What is the general statistical model?
Y=f(X)+ε where ε is random error.
Why can’t we perfectly predict Y?
Because of irreducible error ε.
What type of response variable gives a regression problem?
Quantitative (numeric).
What type of response variable gives a classification problem?
Qualitative (categorical).
Give 2 examples of regression problems.
House price prediction, life expectancy prediction.
Give 2 examples of classification problems.
Fraud detection, disease diagnosis.
What is a linear regression model with p predictors?
Y=β0+β1X1+⋯+βpXp
What do the coefficients βj represent?
The change in Y for a 1-unit increase in X (slope of each feature)j, holding others fixed.
What assumption does linear regression make?
The relationship between predictors and response is linear.
What is polynomial regression?
A regression model that includes powers of a predictor (e.g. Y=β0+β1X1+⋯+βp(X_p)^p).
What is Mean Squared Error (MSE)?
average of difference b/w outcomes and prediction, errors, squared. u = (u_1, … , u_p) and v =(v_1, … , v_p)
MSE=(1/p) * ∑(u_i−v_i)^2
Why square the errors in MSE?
Penalizes large errors more heavily and ensures no negative errors.
What is MAE?
Mean Absolute Error. Average of absolute difference b/w outcomes and predictions.. u = (u_1, … , u_p) and v =(v_1, … , v_p)
MSE=(1/p) * ∑|u_i−v_i|
Which metric penalizes large errors more, MSE or MAE?
MSE. bc squared
What is training data used for?
to train y=f(x) to obtain a learned model y=f_hat(x)
What is testing data used for?
Evaluating accuracy on the learned model
what is training error used for?
quantify difference b/w true response & models predicted response for the data that the model was fit to.
Why is testing error important?
It reflects how the model performs on data/information it has never seen before
We always expect training error to be ___ testing error.
Less than
What is overfitting?
when the model matches the training so well that there ends up being Low training error, high testing error
Why does overfitting happen?
Model is too flexible and captures noise. Picking up random trends in data rather than just an underlying pattern
