6.1 Linear Regression

Question 1

If two variables, x and y, have a very strong linear relationship

Answer 1

there might not be any causal relationship between x and y

Correlation does not imply causation.

Question 2

In the following figure, which point will affect the LR model the most if added to the dataset?

Answer 2

Linear regression is sensitive to outliers in the data. Point d is the top outlier in this plot, which can be measured by the vertical distances between approximated/predicted y and actual y.

Question 3

If a linear regression fits the training data perfectly, which of the following is true?

Answer 3

None of the above

Fitness of the test data depends on how well the model fits the training data, as well as how well the test data is represented in the training data. If test data is represented perfectly, in another word, there is no noise in the test data, the model can fit test set perfectly. Otherwise, it won’t.

Question 4

Which of the following are correct about linear regression?

Answer 4

• The model should represent a line which minimises the MSE.
• Linear regression captures changes in one variable linearly with changes in another variable

In linear regression models,

• the predictors are features, which are accurate measurements,
• outcome/response values are estimations.
• The true relationship between the outcome variable and predictors may not be linear.