Linear Regression

Question 1

interpret constant & coeffcient

Answer 1

constant = when education equals 0, income is 457
coeffcient = with every year of education the mean income increases by 104

Question 2

What is a prediction error?

Answer 2

Question 3

Lin regression assumptions:

Answer 3

1. Linear relationship
2. Multivariate normality (all variables need to be normally distributed –> When the data is not normally distributed a non-linear transformation e.g., log-transformation might fix this issue)
3. No or little multicollinearity (If multicollinearity is found in the data, centering the data (that is deducting the mean of the variable from each score) might help to solve the problem. However, the simplest way to address the problem is to remove independent variables with high VIF values)
4. No auto-correlation (Autocorrelation occurs when the residuals are not independent from each other. For instance, this typically occurs in stock prices, where the price is not independent from the previous price)
5. Homoscedasticity (The scatter plot is good way to check whether the data are homoscedastic (meaning the residuals are equal across the regression line).

Question 4

How does OLS work?

Answer 4

Question 5

What is r^2?

Answer 5

Question 6

How is R^2 calculated?

Answer 6

Question 7

What is the loss function for linear regression?

Answer 7

Question 8

What does a lin regression model predict?

Answer 8

The mean value of y for a given value of x
(No probabilities, it’s a model of the mean)

Question 9

How can we make a constant more meaningful? (1)

Answer 9

centering: usually mean centered (subtract -12,5 years from years of education)

Question 10

How can we make a constant more meaningful? (2)

Answer 10

standardizing: subtract mean / by SD

Question 11

Answer 11

1) For every 1-SD of education, mean of income rises by 402
2) For every 1-SD of education, mean of income rises by 0,3 SD of mean income

Question 12

Why would you want to standardize?

Answer 12

Allows comparison

Question 13

How are the standardized coeffcients also called?

Answer 13

Question 14

What is true about correlations?

Answer 14

1) Standardizing gets rid of scale –> whole point
3) Perfect correlation = 0 error
4) just not linear -> just a measure for linear relationships!

Question 15

Why would we even need a regression, why not only calculate the conditional means?

Answer 15

1) reduce noise -> virtue of abstraction
2) prediction even for data that is not there
3) allows for more control i.e. mediation, moderation, controls, etc.

Question 16

Why do we square residuals in r^2?

Answer 16

1) prevent cancelling out
2) bigger penalty for large residuals

Question 17

Answer 17

• no threshold will do
• similar rationale with alpha value
• highly noisy data in SS how could we possibly achieve it? or should even want to?

Question 18

Is my R² too low?

Answer 18

Low R-Squared is often good BUT also a limitation

Question 19

Is my R² too high?

Answer 19

High R-Squared is often not good
BUT can be

Question 20

Why a Low R-Squared is often good

Answer 20

Question 21

Why a Low R-Squared is also a limitation

Answer 21

Question 22

Why a High R-Squared is often not good

Answer 22

Question 23

Why a High R-Squared can be good

Answer 23

very accurate prediction if really captures the relationship

Question 24

What do we need to control for?

Answer 24

Question 25

How can Parent's SES influence education --> income?

Answer 25

Confounding as well as Mediation, interlaced

Question 26

Income achieved or inhereted?

Answer 26

Reduction of FISEI much more than reduction of education --> rather achieved

Question 27

When do we need to put in a control variable?

Answer 27

1) not a good idea --> kitchen sink approach leads to overfitting (unless you want a really good in-sample prediction) 2) not enough - not clear in which direction the correlation works

Question 28

What is the collider bias?

Answer 28

When an exposure and an outcome independently cause a third variable --> ‘collider’. Inappropriately controlling for a collider variable, can induce a distorted association between the exposure and outcome, when in fact none exists.

Question 29

Answer 29

Question 30

What is happening here?

Answer 30

Expectation: predictor variable will get smaller when other variables are added to the regression model BUT sometimes a coefficient gets larger when other variables are added --> Special case of confounding: Surpression Usuallyoccurs when there's an inconsistency of signs: - younger people more education (edu expansion) - older people more income (curvilinear rel) --> Age neg rel to edu BUT pos rel to income

Question 31

Answer 31

Question 32

Which of these var could be overcontrolled?

Answer 32

First three could be mediators, gender needs to be controlled bc moderator

Question 33

Guidelines for selecting explanatory variables:

Answer 33

Question 34

Gander Pay Gap - Argumentation makes sense?

Answer 34

Conditioning on a mediator NOT a control Why would you look at the gap after cancelling out reasons why it exists?

Question 35

What could be other mediators for gender --> income?

Answer 35

Question 36

How do you assess a mediator?

Answer 36

Question 37

How can you interpret the child effect theoretically?

Answer 37

Having child maybe relates to a higher income group as costly decision also, usually older when having a child could also be usually shared income when having a child

Question 38

How does an interaction effect look like for gender --> income with having a child or not

Answer 38

Symmetrical effect of child effect differed by gender OR gender effect differed by child status

Question 39

Answer 39

Main effects are defined for the interacting variable equalling zero

Question 40

Answer 40

Question 41

How would this reg table look in a normal table?

Answer 41

Question 42

Why are margins larger at the beginning/end?

Answer 42

fewer people in sample with 0 or 30 years of education

Question 43

Why is different with standardizing?

Answer 43

more meaningful

Question 44

Answer 44

Question 45

Why is the effect more important than the statistical significance?

Answer 45

Can be not significant at 0 but highly significant at other ages

Question 46

Key takeaways from visualizations

Answer 46

Question 47

Three types of plots

Answer 47

1. Coefficient plot 2. Profile plot 3. Conditional effect plot

Question 48

What is a coeffcient plot?

Answer 48

Question 49

What is a profile plot?

Answer 49

Question 50

What is a conditional effects plot?

Answer 50

Often used for interactions