What is a predictor in regression?
An observed variable used as input to explain or predict the response variable.
What is the model (design) matrix?
A numeric matrix whose rows are observations and whose columns correspond to model parameters.
What do the rows of the model matrix represent?
Individual observations.
What do the columns of the model matrix represent?
Model parameters (one per column).
What equation defines fitted values using the model matrix?
ŷ = Xβ̂
What is Mean Squared Error (MSE)?
The average squared difference between observed values and fitted values.
What is the formula for prediction MSE?
MSE = (1/n) Σ (yᵢ − ŷᵢ)²
What is the regression MSE used to estimate error variance?
MSE = RSS / (n − p)
What does p represent in regression MSE?
The number of estimated parameters (columns of X).
What does regression MSE estimate?
The variance of the error term σ².
What are the units of MSE?
Squared units of the response variable.
What is RMSE?
The square root of MSE, returning error to original response units.
What is Mean Absolute Error (MAE)?
The average absolute difference between observed and fitted values.
What is the formula for MAE?
MAE = (1/n) Σ |yᵢ − ŷᵢ|
How does MAE differ from MSE?
MAE uses absolute errors; MSE squares errors and penalizes large errors more.
Is MAE more robust to outliers than MSE?
Yes, because it does not square large errors.
What is R²?
The proportion of variability in the response explained by the model.
What is the formula for R²?
R² = 1 − RSS / TSS
What does TSS represent?
Total Sum of Squares, the total variability in y.
What are the bounds of R²?
Typically between 0 and 1 for models with intercepts.
Can R² decrease when adding predictors?
No, R² never decreases when predictors are added.
Does a high R² guarantee a good model?
No, it does not guarantee good prediction or valid assumptions.
What does MSE measure?
Average squared prediction error magnitude.
What does MAE measure?
Average absolute prediction error magnitude.
