Midterm

Question 1

SSTO

Answer 1

SSTO=SSE+SSR
SSTO=Observed-Mean
Degree of Freedom=n-1
Distance from observed point to mean

𝑆𝑆𝑇=Σ(𝑌𝑖−𝑌̅ )^2

SSTO=Y’(I-1/n J)Y (Quadratic Formula)
SSTO=Y’Y- (1/n)Y’JY

Question 2

SSE

Answer 2

Error Sum of Squares
SSE=Observed - Predicted

Degree of Freedom=n-p
Simple Linear Degree of freedom is n-2
Distance from observed point to predicted 𝑆𝑆𝐸=Σ(𝑌^𝑖−𝑌𝑖 )^2

SSE=Y’Y-Y’HY
=Y’(I-H)Y

Question 3

SSR

Answer 3

Residual Sum of Square
SSR=Predicted-mean
Degree of freedom=p-1

Simple Linear Degree of Freedom=1
Distance from Predicted line to mean
𝑆𝑆𝑅=Σ(𝑌^𝑖−𝑌̅ )^2

SSR=Y’HY-(1/n)Y’JY
=Y’(H-(1/n)J)Y

Question 4

What are the general assumptions of the linear model? How to assess it?

Answer 4

1. Linearity: The response and covariates are related in linear way
   - Assess with scatter plot of Y vs X and residuals
2. Normality: Error terms (Ei) or response Yi are normally distributed
   -Assess with qq plot and Shapiro Wilk of residuals or semistudentized residuals
   -Assess with histogram of residulas
   E.g., QQplot shows a trend at the tails away from the diagonal line
3. Constant Variance: Variance of error is constant. var(Ei)=σ2
   - residuals (r)
   - absolute value of residuals
   - residuals squared (r2)
   - semistudentized residuals
   - semistudentized squared
   - absolute value

• Assess by Scatter Plot (semi-resi vs Y, or squared-resi vs predictors), megaphone shape imply non-constant variance
• Breusch Pagan test (Heteroscedasticity Test (H0=constant variance, Ha=Nonconstant variance)

4. Independence: Subjects (Errors or responses) are independent

Question 5

MSR

Answer 5

SSR/p-1

Variation that is explained by the fitted regression line

Question 6

MSE

Answer 6

SSE/n-p
MSE is estimate of sigma σ2
√(MSE) is estimate of σ
Variation NOT explained by the fitted regression line

Question 7

F statistic

Answer 7

F=MSR/MSE

If 𝛽=0→𝐹=1, else 𝛽≠0→𝐹>1

Question 8

R2

What is R2=0.1885 means

What is R2 adjusted =0.1817 means

Is Adjusted R2 better than R2?

Answer 8

R2=SSR/SSTO
Coefficient of determination
Proportion of variance of Y explained (linearly) by the variation in X
√R2 with the appropriate sign is equal to the correlation coefficient (r) between X and Y (only in simple linear regression

R2=0.18 means, 19% variation in inverse BP is explained by the predictors in the model

R2=0.18 means, 18% variation in inverse BP is explained by the predictors in the model after adjusting for the number of variables in the model

Adjusted R2 is better because it accounts for cost of number of variables

Question 9

Diagnostics using Raw Residuals

Answer 9

1. Linearity
   - if linear, residuals form an even band around 0 across the graph of residuals vs. predicted values, and each x from the model. Symmetry around zero, then linear
   - Systematic pattern means non-linear, or an important predictor was forgotten.
2. Normality of the error terms
   - qq plot of the residuals, and shapiro wilk
3. Constant variance
   - Look for megaphone in residuals
   - increasing/decreasing trend in absolute value of residuals or squared residuals versus the predicted values, or versus any X.
   - Can also be seen on scatter plot of Y vs each X.

Question 10

Limitation of R2

Answer 10

1. High R2 does not necessarily imply useful prediction (better to look at width of prediction interval)
2. High R2 does not necessarily mean good fit
3. R2 close to 0 does not mean no relationship. It only refers to the weak linear relationship

Question 11

Box Cox Transformation

Answer 11

Method to find a useful transformation

Chose the best λ

Question 12

Modeling Strategy

Answer 12

1. Look at data
2. Fit preliminary model
3. Perform diagnostics (normality, constant variance, linearity)
4. Fix problems by Box Cox Transformation
5. Fit new model
6. Diagnose new model
7. Repeat steps 4-6 until all problems are fixed
8. Final inference (p-values, CI, etc)

Question 13

Boxplot

Answer 13

Boxplot is used to describe the symmetry of the data, can’t directly draw the conclusion of normality from boxplot

Question 14

Collinearity is tested by

Answer 14

Correlation Matrix

Question 15

Outlier can be tested by

Answer 15

Plotting semistudentized residual against X or predicted Y

Question 16

Omission of important predictors can be tested by

Answer 16

Plotting residuals against omitted variables

Question 17

Non linearity can be fixed by

Answer 17

Transformation

• Transform X if error terms are normally distributed and have constant variance
• Transform Y when unequal error variance and nonnormality of the error terms
• Box Cox Transformation

Question 18

Non variance consistency can be fixed by

Answer 18

Variance stabilizing transformation

Weighted least squares

Question 19

Non independence of the error terms can be fixed by

Answer 19

Adding a time covariates to the model

Question 20

Omission of important predictors can be fixed by

Answer 20

Adding them

Question 21

Transformation

Answer 21

1. Can linearize non-linear relationships
2. Can stabilize non-constant variance
3. Can reduce non-normality

Question 22

Interpret: 1/SBP=0.00985-0.0000416*AGE

Answer 22

For every 1 year increase in age, we expect the mean inverse of SBP decrease by 00.0000416 mmHG-1 (while considering other variables holding it constant)

Question 23

What is the Null Hypothesis and Alternative Hypothesis for F test for multiple variable.

Decision Rule

F critical value

F test equation

Answer 23

H0: 𝛽1=𝛽2=0→
H1: Not all 𝛽s are equal to 0

Decision rule:
If F^≤F(1-α;p-1,n-p) conclude H0
If F^>F(1-α;p-1,n-p) conclude H1

F critical value= F(1-α;p-1,n-p)

F test equation=MSR/MSE

Question 24

What is null hypothesis for the Spearman correlation?

Answer 24

H0=There is no relationship between two variables

Question 25

Degree of freedom | dfSSTO=dfSSR+dferror

Answer 25

dfsst=dferror+dfreg | n-1=n-p+p-1

Question 26

Interpret Breusch-Pagan test results P=0.3

Answer 26

Null: Constant Variance Alter: Non-constant Variance E.g., Age with p value 0.3, we fail to reject the null hypothesis at the 0.05 level and claim there is NOT enough evidence to conclude non-constant variance E.g., Even though BMI has borderline significant p-value, the overall test suggests constant variance. We claim that no remedial actions are necessary at this point

Question 27

How to test outlier?

Answer 27

You can test outlier by scatter plotting semi-studentized residual Any observation with greater than plus or minus 4 semi-studentized residuals suggest outlier.

Question 28

``` Write equation Independent variable SYSBP-1 Intercept=0.01276 Dependent variables -BMI =0.00007775 -AGE=-0.00003659 -TOTCHOL=-0.000001463 ```

Answer 28

SYSBP^(-1)=0.01276-0.00007775(BMI)-0.00003659(AGE)-0.000001463(TOTCHOL)

Question 29

Interpret Shapiro-Wilk test results | p=0.624

Answer 29

Null: Residuals are normal Alt: Residuals are NOT normal At the 0.05 level, we fail to reject the null hypothesis and claim there NOT enough evidence to conclude the residuals are NOT normal

Question 30

What is the decision rule for F(0.95,3,336)=2.631

Answer 30

Decision Rule If F*≤2.63149, conclude H0 If F*＞2.63149 conclude HA (One predictor: There is a linear relationship, Multiple predictors=At least one of the coefficient is non-zero)

Question 31

Does β0 have an interpretation in this model? Why or why not (no more than 2 sentence answer).

Answer 31

No, BMI, age, total cholesterol of 0 do not make any clinical sense and is biologically impossible. The range of data does not include 0 for any of the independent variables, so we cannot draw inference there

Question 32

How to report the significance of EACH coefficient for each predictor in the model? - State statistic - State hypotheses - interpret the results (Age, test statistics -5.60, p<0.0001)

Answer 32

-Statistic = t test -Hypotheses H0=B1=0 (Parameter is zero) HA=B1=/ 0 (Parameter is NOT equal to zero) - We reject the null hypothesis that the coefficient of age is significantly different from 0

Question 33

Interpret BMI coefficient SYSBP^(-1)=0.01276-0.00007775(BMI)-0.00003659(AGE)-0.000001463(TOTCHOL)

Answer 33

For every 1 unit increase in BMI, the average inverse systolic blood pressure decrease by 0.00007775 mmHg^(-1) while considering age and serum cholesterol total and holding them constant.

Question 34

True or False | Boxplot is designed to assess Normality

Answer 34

False: Boxplot is designed for examining quantiles, and describe symmetry of the data so that it doesn't reveal whole information of distribution.

Question 35

Interpret Age coefficient SYSBP^(-1)=0.01276-0.00007775(BMI)-0.00003659(AGE)-0.000001463(TOTCHOL)

Answer 35

For every 1 year increase in Age, the average inverse systolic blood decrease by 0.00003659 mmHG^-1 while considering BMI and serum cholesterol total and holding them constant

Question 36

Interpret TOTCHOL coefficient SYSBP^(-1)=0.01276-0.00007775(BMI)-0.00003659(AGE)-0.000001463(TOTCHOL)

Answer 36

For every 1 unit increase in serum cholesterol total, average inverse systolic blood will decrease by 0.000001463 mmHG-1 while considering BMI and Age and holding them constant.

Question 37

``` Interpret 90% confidence interval of BMI (-0.00010325 -0.00005225) ```

Answer 37

We are 90% confident that the true parameter of BMI falls between -0.00010325 -0.00005225

Question 38

``` Interpret 95% confidence interval of AGE (-0.00004738 -0.00002581) ```

Answer 38

We are 95% confident that the true parameter of Age falls between -0.00004738 -0.00002581

Question 39

Predict the average expected inverse systolic blood pressure of a person with BMI 25, 54 years old, 200 cholesterol level, and 95% confidence interval. And interprete the results SYSBP^(-1)=0.01276-0.00007775(BMI)-0.00003659(AGE)-0.000001463(TOTCHOL)

Answer 39

95% confidence interval Y ̂h±t(1-α⁄2;n-p) s{Y ̂h } S(Y ̂h )=MSE(Xh^' (X^' X)^(-1) Xh )=Xh^' s^2 {b} Xh SYSBP^(-1)=0.01276-0.00007775(25)-0.00003659(54)-0.000001463(200) =0.007914 The predicted average expected INVERSE systolic blood pressure is 0.007914 with a 95% prediction interval of (0.007362; 0.000.008092)mmHg-1

Question 40

Steps in diagnostics

Answer 40

1. Visualize your data using Scatter plot, Histogram | 2.

Question 41

ei

Answer 41

ei=Yi-Y^i | ei is best estimate of Ei

Question 42

Least square method

Answer 42

Fine the line through the data that has the smallest sum of squared perpendicular distance from the line to each point

Question 43

Semi-studentized residual

Answer 43

residual (ei)√(MSE) | MSE is in the table

Question 44

MSE is a good estimator of ?

Answer 44

σ2

Question 45

What are the 6 problems to diagnose?

Answer 45

1. Non linear regression function 2. Non constant variance of the error term (ei) 3. Non-independence of error terms (Ei) 4. Identify outliers 5. Non-normality of errors (Ei) 6. Omission of other important predictors from the model

Question 46

Histogram

Answer 46

Check normal distribution | BUT sample size can affect

Question 47

qqplot

Answer 47

plot the quantile, if data line on equator line, it means residuals have normal distribution

Question 48

Linearity

Answer 48

Check via scatterplot Plot residual vs predicted or Plot residual vs xi

Question 49

Which one is better to plot? Plot residual vs Xi or Plot residual vs Y^ (predicted value)

Answer 49

Plot residual vs Y^ (predicted value) is preferred for multiple regression

Question 50

Residual can use to check what?

Answer 50

Non-constant variance -No pattern (megaphone shape) Check for outlier (plot semi resid vs predicted Yi^) (Outliers are >4 or

Question 51

Why is it better to check normality last?

Answer 51

It can be affected by many things such as non-constant variance

Question 52

Non constant variance of the error term (Ei)

Answer 52

ei vs x e2i vs x (it magnify the relationship, it is preferred) Ieil vs x

Question 53

Non-independence of error terms

Answer 53

ei vs predicted

Question 54

Identify outliers

Answer 54

semi-stud residuals vs predicted

Question 55

Breusch Pagan test

Answer 55

- Large sample test - Assumes the error terms Ei are independent - Assume the error terms Ei are normally distributed

Question 56

When you identify Non-linearity, then?

Answer 56

- Adding nonlinear terms (like X2) | * Use a transform of Y and/or X

Question 57

When you identify non-constancy of the error variance, then?

Answer 57

- Variance stabilizing transformation | - Weighted least squares

Question 58

When you found non independence of error terms, then?

Answer 58

-Adding time covariate to the model

Question 59

When you found important predictors are omitted, then?

Answer 59

-Add them

Question 60

When you identify outlier, then?

Answer 60

- Check if they are errors in the data and correct them | - Robust linear regression methods

Question 61

Transformation can:

Answer 61

1. Can linearize non-linear relationships 2. Can stabilize non-constant variance 3. Can reduce non-normality

Question 62

Transform X or Y?

Answer 62

1. Start transform Y 2. If you see non-linear trend (quadratic, logarithmic) in scatter plot, add terms like X2 or log(X) in the model form start

Question 63

Box Cox Transformation

Answer 63

Used to find best transformation from the family of power transformations on Y Chooses the best ramda from the data

Question 64

LOWESS

Answer 64

LOWESS (locally weighted regression scatter plot smoother) is non-parametric regression curve that can check linearity. -If LOWESS curve is inside of confidence bands, then linearity is satisfied Smoothing technique, Non-parametric regression curves can help: -Fit the data with a smooth curve -Does not assume the shape of the curve

Question 65

Write the Linear, First order regression model

Answer 65

Yi=B0+B1Xi1+Bp-1Xip-1+Ei

Question 66

What is the Matrix format?

Answer 66

Y=XB+E E(Y)=XB Var (Y)=σ2I

Question 67

What is the most mathematically right way to write model?

Answer 67

E(Y)=XB

Question 68

How to write coefficient estimate using Matrix

Answer 68

b=(X'X)-1X'Y

Question 69

How to write variance estimate using Matrix

Answer 69

var(b)=σ2(X'X)-1

Question 70

Short Multivariate Normal Density Function

Answer 70

Y~MVN (M, ∑)

Question 71

Y=B0+B1X1+B2X2+E 1. How many paramters in this equation? 2. B0? 3. B1 4. B2

Answer 71

1. 3 paramters 2. Only meaningful if X1 and X2 include 0. It means the mean response, E(Y) at X1=0 and X2=0 3. B1 is change in the mean response, E(Y) per unit increase in X1 while holding X2 constant 4. B2 is the change in the mean response, E(Y) per unit increase in X2 while holding X1 constant

Question 72

Root MSE

Answer 72

Estimate for σ