10 Simple Linear Regression

Question 1

Predicted Values

Answer 1

Values of the dependent variable based upon estimated regression coefficients and the prediction about the value of the independent variable.

Y = b0 + b1*X

Question 2

Confidence interval

Answer 2

Estimate a prediction interval around a predicted value

Question 3

Standard error of the estimate (SEE)

Answer 3

An estimated regression does not describe the relationship between the dependent and independent variables perfectly.

The SEE is the standard deviation of the error term.

Question 4

Confidence interval of Predicted Y

Answer 4

Y +- (t * s)

Where
s^2 = SEE^2 [ 1 + 1/n + (X-\bar(X))/(n-1)s^2

Degrees of freedom is n - 2

Question 5

Functional forms

Answer 5

When the relationship between X and Y is not linear, fitting a linear model would result in biased predictions.

Question 6

Natural log transformation examples.

Answer 6

Log-lin: Log(Y) = bX
Lin-log: Y = bLog(X)
Log-Log = Log(Y) = bLog(X)

Question 7

Simple linear regression

Answer 7

The variation in a dependent variable in terms of the variation in a single independent variable.

Question 8

Dependent variable

Answer 8

The variable whose variation is explained by the independent variable. Sometimes also referred to as the explained variable, endogenous variable, or the predicted variable.

Question 9

Independent variable

Answer 9

The variable used to explain the variation of the dependent variable. Sometimes referred to as the explanatory variable, exogenous variable, or the predicting variable.

Question 10

Ordinary Least of Squares formula

Answer 10

Y = b0 + b1X + e

Question 11

Regression Coefficients

Answer 11

b1 = Cov(X,Y)/\sigma^2
b0 = Y - \hat(b1)*X

Question 12

Assumptions of Linear Regression

Answer 12

1) There is a linear relationship between the dependent and independent variables.
2) Variance of the error terms is constant (homoskedasticity)
3) Error terms are independently distributed.
4) Error terms are normally distributed.

Question 13

Homoskedasticity

Answer 13

The case where prediction errors all have the same constant variance

Question 14

Heteroskedasticity

Answer 14

The variance of the error terms not being constant.

Question 15

Analysis of Variance (ANOVA)

Answer 15

SSE + SSR = SST
SST = \sum(Y -\bar(Y))^2
SSE = \sum(Y -\hat(Y))^2
SSR = \sum(\hat(Y) -\bar(Y))^2

Question 16

Mean Square Regression (MSR)

Answer 16

MSR = SSR/ K

Question 17

Mean Square Error (MSE)

Answer 17

MSE = SSE (n-k-1)

Question 18

Coefficient of Determination (R^2)

Answer 18

R^2 measure the percentage of total variation in Y explained by the variation in X (SSR/SST)

Question 19

Standard Error of the Estimate (SEE)

Answer 19

Measures the accuracy of predicted values from the regression equation.

SEE = (SEE/(n -2))^.5 = MSE^.5

Question 20

The F-Statistic

Answer 20

Tests whether the independent variables explain the variation in the dependent variable.

Notes:
1) One-tailed test
2) 2 degrees of freedom

Question 21

F-Statistic Formula

Answer 21

F = MSR/MSE = (SSR/k)/(SSE/(n-k-1))

Question 22

Regression Coefficient t-Test

Answer 22

t = (\hat(b) - b)/s

where,
s= SEE/\sum(X-\bar(X))