QA9 - Regression Diagnostics

Question 1

Explain how to test whether a regression is affected by heteroskedasticity

Answer 1

Heteroskedasticity is where the variance of error systematically varies with one or more explanatory variables

1. Estimate the model and compute the residuals ei
2. Regress ei against constant, all variables, and cross product of all variables
   (ei = v0 + v1 * Xi1 + v2 * Xi2 + v3 * Xi1^2 + v4 * Xi1 * Xi2 + v5 * Xi2^2 + h)
3. If homoscedastic , W0: v1 = v2 = .. = vn = 0

Question 2

Describe approaches to using heteroskedastic data

Answer 2

1. ignore them and use heteroskedastic robust residuals for hypothesis testing
2. transform data to remove (log for example)
3. use weighted least squares for parameter estimation

Question 3

Characterise multicollinearity and its consequences; distinguish between multicollinearity and perfect collinearity

Answer 3

Multicollinearity is where one or more explanatory variables can be substantially explained by the others. When modelling this means jointly significant variables may have very small individual f-stats.

Perfect collinearity is where one of the variables is perfectly described by another

Question 4

Describe the consequences of excluding a relevant explanatory variable from a model and contrast those with the consequences of including a relevant regressor

Answer 4

Omitting important variable means increasing bias in the model

Including irrelevant variable reduces adjusted R^2 due to penalty factor

Question 5

Explain two model selection procedures and how these relate to the bias variance tradeoff

Answer 5

Bias-variance is balancing large models with low bias but less precise parameters, with high bias but less estimation error models

1. General-to-specific:
   - start with model with lots of parameters
   - remove variable with smallest T stat that is insignificant
   - repeat until all variables significant
2. M-fold cross-validation
   - split data into m blocks, fit all candidate models on m - 1 blocks
   - calculate the residuals on the unused block
   - take the model with smallest residual

Question 6

Describe methods for identifying outliers and their impact

Answer 6

Found using cooks distance, find residuals of model with potential outlier dropped, if Dj > 1 then outlier