L7: Machine Learning

Question 1

Generally, what is Machine Learning?

Answer 1

Question 2

What are the Key categories of ML?

Answer 2

Question 3

What are the two main tasks of ML?

What are some other tasks?

Answer 3

Question 4

What are the common ways to measure the performance for classifications?

Answer 4

• accuracy
• Precision
• Recall
• F₁ or more generally F

Question 5

What are the common ways to measure the performance for classifications?

Answer 5

• Mean Squared Error MSE
• Root Mean Squared Error RMSE
• Mean Absolute Error
• R-squared

Question 6

What is our basic notation?

• a single example
• a feature or attribute
• dataset
• sample or example
• label or target
• design matrix X

Answer 6

Question 7

What does a design matrix look like? What is the notation?

With this notation, what is the goal of supervised learning?

Answer 7

Question 8

What is a train-test split?

Answer 8

Question 9

Standard pipeline for machine learning?

Answer 9

Question 10

What model will we generally use for regression ML?

Answer 10

Question 11

In our regression models, what do we seek to minimise?

Answer 11

Question 12

How do we go about minimising the MSE in practice?

Answer 12

Question 13

How can I visualise this algebra?

Answer 13

Question 14

What is the intercept in linear regression models referred to?

Answer 14

Question 15

What are training errors, and what are test/generalisation errors?

What are underfitting and overfitting, and how do they relate to these errors?

Answer 15

Question 16

What is a way to control the capacity?

Answer 16

• while the data appears non-linearly(x’s) it is a linear model in the weights
• There is an optimal place in which we minimise the generalisation gap.

Question 17

What is Regularisation?

Answer 17

Question 18

How does Regularisation apply to linear regression?

Answer 18

• Ridge regression is the baseline standard (like using Gaussian)
• Lasso is useful as it will set some weights exactly to zero –.> could allow you to throw out those variables from the model to make your algorithm more efficient

Question 19

What are hyperparameters?

Answer 19

Question 20

What is the standard training cross-validation protocol?

Answer 20

Question 21

What is Cross-validation? How do we use it to pick the best Hyperparameters for our model?

Answer 21

• WE dont know which are the best hyperparameters to use for the model that minimise our errors or maximise our performance scores - hence the need for cross validation –> test how my data performance on the hyperparameters (using grid search and then use the best one to train my model on).

Question 22

What is the k-fold cross-validation method?

What is grid search for hyperparameter search?

Answer 22

• SPlit the data in k folds and and then split the data again into 5, I will then use k-1 on the fold to train the data for our hyperparameter, then I would tell on the remaining fold in our split. THen I would repeat k number of times aross each split testing on each of the folds throughout the split
• There is usually just one best hyperparameter so we use grid search to test what value or order of magnitude of the hyperparameter we want, this can be done logarithm me on even spacing and we plot the hyperparameter against the score on a graph (grid search is more efficient in low dimensions).
• or you can do a random shot gun approach through random search