Unsupervised Learning (Cluster Analysis)

Question 1

What is the one line definition of Unsupervised Learning?

Answer 1

A type of algorithm that learns hidden patterns from unlabelled data

Question 2

What are two primary aspects of Unsupervised Learning?

Answer 2

Cluster Analysis - Divide data into meaningful groups
Dimensionality Reduction - PCA, Autoencoders, Generative Models

Question 3

What are the 4 key aspects of Cluster Analysis?

Answer 3

• Features that describe the subjects
• Similarity Functions
• Basic Clustering Algorithms
• Cluster Validation

Question 4

What 4 common Similarity Functions are in Unsupervised Learning?

Answer 4

Euclidean Distance
Cosine Distance
Manhattan Distance
Jaccard Distance

Question 5

What is the equation for Euclidean Distance?

Answer 5

Euclidean Distance = Square Root of ((A - B)^T * (A - B))
Where:
A, B: Column Vectors that contain features of two data samples

Question 6

What is the equation for Cosine Distance?

Answer 6

Cosine Distance: cos(theta) = (A * B) / (||A|| * ||B||)
Where:
A, B: Column Vectors that contain features of two data samples

Question 7

What is the equation for the Manhattan Distance?

Answer 7

Manhattan Distance = 1/n (n sigma) |A - B|
Where:
A, B: Column Vectors that contain features of two data samples

Question 8

What is the equation for the Jaccard Distance?

Answer 8

Jaccard Distance = |A intersection B| / |A union B|
Where:
A, B: Column Vectors that contain features of two data samples

Question 9

What is the step-by-step method of K-Means Algorithm?

Answer 9

1. Randomly select K points as the initial centroids (centres) for each of the K groups
2. Repeat this process:
   a - Assign each point to its closest centroid (centre)
   b - Re-compute the centroid (centre) of each cluster
3. Until centroids (centre) do not change

Question 10

What are the advantages of K-Means Algorithm?

Answer 10

Simple
Efficient

Question 11

What are the disadvantages of K-Means Algorithm?

Answer 11

Solution dependent on the initialisation

Need to specify number of clusters

Sensitive to Outliers

Question 12

What is the step-by-step method of Agglomerative Hierarchical Clustering?

Answer 12

1. Treat each data as a cluster, and compute the similarity matrix between each pair of data
2. Repeat this process:
   a - Merge the closest two clusters
   b - Update the similarity matrix
3. Repeat the above until only one cluster remains

Question 13

What are the advantages of Agglomerative Hierarchical Clustering?

Answer 13

Flexible with number of clusters

Can capture hierarchical relationship/s

Question 14

What are the disadvantages of Agglomerative Hierarchical Clustering?

Answer 14

Solution is local optimum, dependent on subject functions e.g. minimum, maximum, group average, etc…
Requires larger memory and longer computational time

Question 15

What is the step-by-step method for Density-Based Spatial Clustering of Applications with Noise (DBSCAN)?

Answer 15

1. Find the neighbourhood points of every point with distance E
2. Identify the core points with more than minimum number of points worth of neighbours
3. Connect the core points if they are within distance E
4. Make each group of connected core points into a separate cluster
5. Assign each non-core point to a nearby cluster if they are within distance R, called border points
6. Unassigned points are noise points

Question 16

What are the advantages of DBSCAN?

Answer 16

Robust to outliers

Learn non-regular population density patterns

Automatically determine the number of clusters

Question 17

What are the disadvantages of DBSCAN?

Answer 17

Not robust to variable density clusters (due to single E being used)

Computationally expensive

Sensitive to parameter settings

Question 18

What is the step-by-step method for Expectation Maximisation?

Answer 18

1. Assume a distribution model that describes the data e.g. Gaussian
2. Initialise the model parameters (mean and variance)
3. Repeat this process:
   a - E step: Calculate the expected value (which class each data belongs to) of the log likelihood function with the current model parameters
   b - M step: Estimate model parameters that maximise the log likelihood function
4. Until Convergence

Question 19

What are the advantages of Expectation Maximisation?

Answer 19

Soft clustering

Question 20

What are the disadvantages of Expectation Maximisation?

Answer 20

Restricted by the distribution model

Sensitive to initialisation

Need to specificity the number of clusters

Question 21

What is a sign of a good Clustering Analysis?

Answer 21

If the clustering algorithm separates dissimilar data samples apart and similar data samples together, then it has performed well.