LEC 11a Simple Linear Regression

Question 1

How to decide on appropriate statistical test for regression?

Answer 1

Depends on the type of dependent variable

1. Continuous variable
   - linear regression
2. Ordinal variable
   - ordinal regression
3. Nominal variable
   - logistic regression

Question 2

Simple vs Multiple regression

Answer 2

Applies to continuous, ordinal and nominal variables

Simple regression
- only 1 independent variable

Multiple regression
- more than 1 independent variables

Question 3

Correlation vs Simple linear regression (2)

• definition
• symmetry

Answer 3

Correlation

• quantifies the degree to which 2 random variables are related, provided that the relationship is linear
• makes no distinction between the 2 variables (symmetrical)

Simple linear regression

• determines the best-fitting straight line for a dataset to investigate the change in 1 variable (dependent variable Y) that corresponds to a given change in the other variable (independent variable X), provided that there is significant correlation
• X and Y are asymmetrical

Question 4

Applications of simple linear regression (2)

Answer 4

1. Describe the linear relationship between the 2 variables

2. Predict or estimate the value of Y associated with a fixed value of X

Question 5

Can extrapolate values of Y beyond the observed range?

Answer 5

Cannot extrapolate beyond the observed range as the relationship between X and Y may not be linear

Question 6

Simple linear regression model

Answer 6

Y = alpha + beta(X)

alpha = y-intercept
beta = slope

Question 7

Alpha meaning

Answer 7

Mean value of Y when X=0

Question 8

Beta meaning

Answer 8

The change in the mean value of Y that corresponds to a one-unit change in X

Question 9

Does linear regression test for linear relationship between the 2 variables?

Answer 9

No.
- it assumes linear relationship
- finds the best-fitting straight line with the y-intercept and slope
Hence, always plot scatter plot to determine if there is any linear relationship

Linear relationship : linear regression
Non-linear relationship : non-linear regression

Question 10

Scatter plot to determine use of linear regression

Answer 10

Scatter plot must suggest :

1. Linear relationship
2. Significant correlation

Question 11

Assumptions of simple linear regression model (4)

Answer 11

1. There is linear relationship between the variables
2. Each observations are independent of one another
3. For any specified values of X, the distribution of the Y values is normal
4. For any set of values of X, the variance is constant (equal variance)

Question 12

How to determine the best-fitting straight line?

Answer 12

Method of Least Squares

- best-fitting line = line with the smallest residual sum of squares

Question 13

Residual plot

Answer 13

Residual against Y values

Each residual data is randomly scattered above and below ei=0

Question 14

Test statistics for beta (slope)

Ho & H1

Answer 14

Two-tailed test

Ho :

• there is no effect of the independent variable X on the dependent variable Y
• beta = 0
• equivalent to testing correlation = 0

H1 :

• there is an effect of the independent variable X on the dependent variable Y
• beta =/ 0

Question 15

Test statistics for alpha (constant)

Answer 15

Seldom done cos not really important

Two-tailed test

Ho :
- alpha = 0

H1 :
- alpha =/ 0

Question 16

Evaluation of the goodness-of-fit regression model

Answer 16

Coefficient of determination (R^2)

In simple linear regression, R^2 = r^2,
r = Pearson product-moment correlation coefficient

Question 17

R^2

• meaning
• range

Answer 17

• the proportion of the variability among the observed values of Y that is explained by the linear regression of Y on X
• range : 0 =< R^2 =< 1

Question 18

R^2 = 1

Answer 18

All data points lie exactly on the best-fitting line

Question 19

R^2 = 0

Answer 19

There is no linear relationship between X and Y

Question 20

R^2

Answer 20

Coefficient of determination

Question 21

Significance level for constant value in statistical report

Answer 21

Not important

Question 22

Significance level for ANOVA report (2)

Answer 22

• p-value for overall significance of regression model

- same significance level value for Coefficients report

Question 23

Sum of squares (regression) in ANOVA report

Answer 23

• variability in Y that is explained by the regression model

Question 24

Sum of squares (residual) in ANOVA report

Answer 24

• variability in Y that is unexplained by the regression model

Question 25

Sum of squares (total) in ANOVA report

Answer 25

- total variability in Y

Question 26

Steps to analyse linear regression (4)

Answer 26

1. Check if assumptions of linear regression fulfilled - independent observations - for each set of X values, there is equal variance - for each set of X values, the distribution of Y values is normal - linear relationship between both variables 2. Scatter plot to determine linear relationship 3. Correlation - Pearson Product Moment Correlation - Spearman Rank Correlation - must show significant correlation (proceed to step 4) 4. Conduct linear regression analysis