Regression vs classification
- Regression: when we predict quantitative outputs
- Classification: when we predict categorical (qualitative) outputs
Regression
determine the relationship between the
dependent variable ๐ and a set of independent variables ๐
Dependent and independent values
X are independent, Y changes as a consequence of other variables
How do we fit linear model
We fit it through m inimizing the difference between the actual and predicted Y value.
Multiple Linear Regression
multiple input variables
Ordinary Least Squares
Used to fit regression line
How to measure the fit of the model
R2 measure.
Gradient Descent
Itโs an optimization algorithm that finds the linear regression coefficients
iteratively
Problems with fitting the data
Over-fitting: the model models the training data
too well. Under-fitting: the model that can neither model the
training data nor generalize to new data
Problems with OLS
Low performance (such as over-fitting and Interpretation: to get the bigger picture. Solution is regularizing the coefficient estimates (Shrinkage Methods) which can be done with RIdge and lasso regression
Ridge regression
Shrinks the regression coefficients by imposing a
penalty on their size
Lasso Regression
Lasso regression is a shrinkage method like ridge. The only difference
is instead of taking the square of the coefficients, magnitudes are taken
into account:
Assumptions in linear regression. what is it?
โassumptionsโ are statements that we take to be true about our data and the model in order for the mathematical properties of linear regression to hold.
What are the assumptions?
The observations are independent (random sampling)
The relationship of ๐ with ๐ and the error term is linear
๐ is normally distributed at each value of ๐
The error term is normally distributed with mean zero and constant
variance
The ๐ variables are independent(only multiple linear regression)
Evaluation of a linear regression model
The performance of the model must be reported as an error for the predictions. Some methods are MSE, RMSE, MAE, MAPE
MSE
Is calculated as the mean or average of the squared differences
between predicted and expected target values in a dataset
RMSE
RMSE is calculated as the square root of the MSE, which means that
the units of the error are the same as the units of the target value that is
being predicted:
MAE
MAE is calculated as the average of the absolute error values, and like
RMSE, the units of the error score match the units of the target value
that is being predicted
MAPE
MAPE is the percentage equivalent of MAE.
