Supervised learning: concepts and applications

Question 1

What is machine learning

Answer 1

a set of methods that can automatically detect patterns in data

Question 2

What does machine learning do with these patterns

Answer 2

they are used to predict future data or to perform other kinds of decision making under uncertainty

Question 3

What is a key premise of machine learning

Answer 3

the learning problem

Question 4

what is the learning problem

Answer 4

learning from data is used in situations where we don’t have any analytic solutions but we do have data that can construct an empirical solution

Question 5

what does the learning problem use

Answer 5

a machine learning method

Question 6

What kind of inputs does a model have

Answer 6

feature, attribute, predictor, independent variable

Question 7

what kind out outputs does a model have

Answer 7

response, dependant variable, label growth

Question 8

When is regression used in supervised learning

Answer 8

where Y is continuous (quantitative)

Question 9

Where is classification used in supervised learning

Answer 9

covers situations where Y is categorical

Question 10

How do you minimise the least square error

Answer 10

the gradient decent method

Question 11

what are the main two types of supervised learning

Answer 11

regression and classification

Question 12

Give an example of regression

Answer 12

predicting house prices based on size, location and number of bedrooms

Question 13

Give an example of classification

Answer 13

spam/non-spam

Question 14

What is classification used for

Answer 14

assigning instance to discrete categories

Question 15

How is the “best fit” defined

Answer 15

the line that minimises the sum of squared errors between actual and predicted values

Question 16

What algorithms are supervised and can solve both regression and classification problems

Answer 16

Decision trees
Random Forest
K-nearest neighbours

Question 17

What are strengths of linear regression

Answer 17

• Simple and easy to implement
• works well for linearly separable data
• Computationally efficient

Question 18

What are the limitations of linear regression

Answer 18

• Assumes linearity between features and the target variable
• Sensitive to outliers
• Struggles with multi collinearity
• poor performance with complex, non-linear data

Question 19

What is the process of the decision tree

Answer 19

• Determine attribute to select as the root (according to some goodness measure… information gain) two split on
• Partition inout examples into subsets according to values order the root attribute
• construct DT recursively for each subset
• Connect the roots for the subtrees to the root of the whole tree via labelled links

Question 20

What is entropy

Answer 20

a measure of uncertainty or impurity of a dataset

Question 21

What does information gain measure?

Answer 21

A: The reduction in entropy after a dataset is split on an attribute.

Question 22

Why Is information gain important

Answer 22

It determines the best attribute to split on.

Question 23

What are the strengths of the decision tree

Answer 23

• easy to visualise and interpret
• handles non-linear relationships well
• No need for data normalisation or scaling
• can handle categorical and continuous data

Question 24

What are the limitations of the decision tree

Answer 24

• Prone to overfitting (without running or max depth constraints)
• Sensitive to small change in data (instability)

Question 25

How to do random forest

Answer 25

1. Train each tree on a random subset of data with replacement (Bootstrapping). 2. Use a random subset of features at each split (Feature Subsampling). 3. Grow individual decision trees independently (Tree Building). 4. Combine results using majority vote (classification) or averaging (regression) – (Aggregation) 5. Output the aggregated result from all trees (Final Prediction).

Question 26

what is bootstrapping

Answer 26

Each decision tree is trained on a random sample of the dataset: - Sampling is with replacement - So some data points appear multiple times - Some data points are left out

Question 27

Example in random forest

Answer 27

each trees uses a different ML method to make predictions, the majority vote by each model (tree) is then combined to find the majority and the output is the aggregated result from all trees

Question 28

random forest definition

Answer 28

A Random Forest trains many different decision trees, lets each one make a prediction, and then combines their answers to get a more reliable final result.

Question 29

What are the strengths of random forest

Answer 29

- Reduces overfitting compared to individual decision trees as it only looks a t subset of the data - Handles high dimensional data effectively

Question 30

What are the limitations of random forest

Answer 30

- Less interpretable than individual trees - Computationally intensive for large datasets - May require tuning for hyper parameters (e.g. number of trees)

Question 31

What is the nearest neighbour classification

Answer 31

1. user inputs k: an integer representing the number of nearest neighbours (instances) to search for 2. with each unlabelled instance: calculate the distance between it and all the instances in the data set 3. 4. 5. find the k nearest neighbours count the assigned class labels in k nearest neighbour for each class the class with the highest count (majority vote) is the output

Question 32

What are the strengths of k-nearest neighbours

Answer 32

- Simple to implement and understand - Makes no assumptions about data distribution - effective for non-linear and multi-class problems - adaptive to changes in the dataset

Question 33

What are the limitations of K-nearest Neighbours

Answer 33

- Computationally expensive during prediction (lazy learning) - Performance depends heavily on the choice of k and distance metric - sensitive to irrelevant or noisy features

Question 34

What are the key issues of nearest neighbour classification

Answer 34

- no model is built! all the data is retained - training could take up to O(np) per observation - will not do well when number of features is large