Untitled Deck

Question 1

What is the definition of Labels (y ∈ Y) in supervised learning?

Answer 1

The target variable or output representing the quantity or category to be predicted

Y = R^k is referred to as regression. For classification, Y is a finite set.

Question 2

What is the purpose of Labels (y ∈ Y) in supervised learning?

Answer 2

Provides the ‘ground truth’ for training models and evaluating predictions

Does not appear in unsupervised learning.

Question 3

What are Features (x ∈ X) in the context of machine learning?

Answer 3

The input data used to make predictions, represented as a feature vector x ∈ R^d

Each component represents a measurable attribute of the data.

Question 4

What is the purpose of Features (x ∈ X)?

Answer 4

Encodes the information from which the model learns patterns to predict labels.

Question 5

Define Feature Map (ϕ : X → H).

Answer 5

A transformation that maps input data to a potentially alternative feature representation.

Question 6

What is the purpose of a Feature Map?

Answer 6

Enables non-linear relationships or convenient features to be captured.

Question 7

What is a Predictor (fθ : X → Y)?

Answer 7

A function that maps x ∈ X to predicted labels f(x) = ˆy ∈ Y.

Question 8

What is the purpose of a Predictor?

Answer 8

Learns the relationship between features and labels to make accurate predictions.

Question 9

Name a key example of a Predictor.

Answer 9

• Support Vector Machine (SVM)
• Logistic Regression
• Discriminative Models
• Generative Models
• Likelihood Ratio Tests

Question 10

Define Loss (l(y, z)).

Answer 10

A function quantifying the penalty for predicting z when the true label is y.

Question 11

What is the purpose of Loss?

Answer 11

Guides the optimisation process by measuring prediction errors.

Question 12

What is Empirical Risk (Rˆ)?

Answer 12

The average loss computed over the training data.

Question 13

What is the purpose of Empirical Risk?

Answer 13

Measures how well the model fits the training data.

Question 14

Define Risk (R[fθ]).

Answer 14

The expected loss over the true data distribution.

Question 15

What is the purpose of Risk?

Answer 15

Represents the model’s generalisation performance.

Question 16

What is the Generalisation Gap?

Answer 16

The difference between the true risk and the empirical risk.

Question 17

What is the purpose of the Generalisation Gap?

Answer 17

Quantifies how much worse the model performs on unseen data compared to training data.

Question 18

Define Bayes Predictor.

Answer 18

The risk-minimising predictor that achieves the lowest possible risk.

Question 19

What is the purpose of the Bayes Predictor?

Answer 19

Serves as the theoretical optimal benchmark for any predictive model.

Question 20

What is Excess Risk?

Answer 20

The difference between the risk of the learned model and the risk of the Bayes predictor.

Question 21

What is the purpose of Excess Risk?

Answer 21

Measures how far the learned model is from the optimal predictor.

Question 22

Define Overfitting.

Answer 22

When a model learns overly complex patterns in the training data.

Question 23

What is the purpose of Overfitting?

Answer 23

Highlights the need for regularisation, cross-validation, or simpler models.

Question 24

What are Hyperparameters?

Answer 24

Parameters that specify the learning process and are tuned but not learned directly.

Question 25

What is the **purpose** of Hyperparameters?

Answer 25

Optimises model performance and generalisation.

Question 26

Define **k-Fold Cross-Validation**.

Answer 26

A technique where the data is split into k folds for training and validation.

Question 27

What is the **purpose** of k-Fold Cross-Validation?

Answer 27

Provides a robust estimate of generalisation performance.

Question 28

What is **Gradient Descent**?

Answer 28

An iterative optimisation algorithm that updates θ using the gradient of the loss.

Question 29

What is the **purpose** of Gradient Descent?

Answer 29

Efficiently minimises the empirical risk.

Question 30

Define **Learning Rate (η)**.

Answer 30

A hyperparameter controlling the step size at each iteration of gradient descent.

Question 31

What is the **purpose** of Learning Rate?

Answer 31

Balances convergence speed and stability during optimisation.

Question 32

What is **Regularisation**?

Answer 32

Techniques to reduce overfitting and improve generalisation.

Question 33

What is the **purpose** of Regularisation?

Answer 33

Aims to bias the learning algorithm towards preferred solutions.

Question 34

Define **Reproducing Kernel**.

Answer 34

A function that computes inner products in the feature space H.

Question 35

What is the **purpose** of a Reproducing Kernel?

Answer 35

Enables efficient computation of similarities in high-dimensional spaces.

Question 36

What are **Neural Networks**?

Answer 36

A class of models composed of layers that process input data.

Question 37

What is the **purpose** of Neural Networks?

Answer 37

To learn complex patterns and representations from data.

Question 38

What is a **Residual Layer**?

Answer 38

A layer that adds its input to its output.

Question 39

What is the **purpose** of a Residual Layer?

Answer 39

Mitigates the vanishing gradient problem.

Question 40

What is **Backpropagation**?

Answer 40

An algorithm for computing gradients of the loss with respect to the weights.

Question 41

What is the **purpose** of Backpropagation?

Answer 41

Enables efficient gradient computation for updating weights.

Question 42

Define **Universal Approximators**.

Answer 42

Neural networks that can approximate any continuous function on compact subsets.

Question 43

What is the **purpose** of Universal Approximators?

Answer 43

To represent complex functions through neural networks.

Question 44

What is the difference between **Underparameterised** and **Overparameterised Models**?

Answer 44

* Underparameterised: Lacks capacity to fit training data well * Overparameterised: Sufficient capacity to interpolate training data exactly