linear regression

Question 1

regression

Answer 1

a method to understand how an outcome Y changes as predictor X1, X2,…, Xp vary

Regression finds the best-fitting line through the cloud of points

Question 2

single regression formula

Answer 2

Y = B0+ B1X1

B is beta

Question 3

Y =

Answer 3

outcome variable (DV)

Question 4

X =

Answer 4

predictor variables (IV)

Question 5

B0 =

Answer 5

intercept (model’s predicted value of Y when X = 0)

Question 6

Bi =

Answer 6

Slope coefficients (change in Y per unit change in X)

How much the outcome changes for each 1-unit increase in 𝑋𝑖, holding all else constant

Question 7

E =

Answer 7

Error term (unexplained variance)

Question 8

R^2

Answer 8

coefficient of determination

represents the proportion of the variance in a dependent variable that is predictable from the independent variables in a regression model.

It indicates how well the regression line fits the data, with a value of (1) meaning all data points fall perfectly on the line and (0) meaning the line explains none of the variability.

ex: R^2 = 0.215; About 21.5% of burnout (Y) variation is explained by age (X)

Question 9

residuals

Answer 9

the distance between a point and the reference line (linear) – error

added up, squared, know how much error it present → want them to be around 0, want the least amount of error

Question 10

The more slope (steeper slope)….

Answer 10

the more of a relationship the variables have

Large beta

Question 11

assumptions of regression

Answer 11

1. linearity
2. independence
3. Homoscedasticity
4. Normality of residuals
5. No multicollinearity - only for multiple regression

Question 12

1. linearity

Answer 12

Relationship between predictors and outcome is learn

Check: scatterplots, residual plots

Violation: curved relationships, U-shaped effects

Question 13

2. independence

Answer 13

Observations are independent of each other

Violation: clustered data, repeated measures

Solution: multilevel modeling

Question 14

3. Homoscedasticity

Answer 14

Constant variance of residuals across predictor values

Check: residual vs fitted plots

Violation: funnel-shaped patterns

Question 15

4. Normality of residuals

Answer 15

Residuals are normally distributed

Check: Q-Q plots, histograms

Violation: skewed distributions

Question 16

5. No multicollinearity - only for multiple regression

Answer 16

Predictors are not highly correlated

Check: correlation matrix, VIF value

Rule: VIF < 5 (or < 10)

Question 17

key takeaways

Answer 17

Regression quantifies relationships between variables

Coefficients tell us the size and direction of effects

Confidence intervals show uncertainty in estimates

P-values indicate statistical significance

Question 18

multiple regression

Answer 18

Multiple regression is an extension of simple regression — we’re simply adding more predictors to the model.

Question 19

multiple regression formula

Answer 19

Y = B0+ B1X1+ B2BX2 + … + E

B is beta

Question 20

How to interpret the coefficients

Answer 20

Each coefficient represents the expected change in the outcome for a one-unit change in that predictor, holding all other variables constant.

In multiple regression, each regression coefficient (β) represents the unique contribution of its predictor to the outcome, after controlling for the effects of the other predictors in the model

Question 21

which predictor has the strongest effect

Answer 21

We can use standardized coefficients to compare the strength of the effects of the predictors

we need to know if it has a scale function

without it, we don’t know if they are standardized – can draw conclusions about effect size