8. Unsupervised Learning

Question 1

Purpose of PCA

Answer 1

( Principal Component Analysis is ordered )
-> finds a sequence of linear combinations of the variables that have maximal variance, are mutually uncorrelated

1. Data visualization / pre-processing before supervised techniques are applied.
2. Dimensionality reduction

Question 2

Purpose of clustering

Answer 2

Discovering unknown sub-groups (homogenous clusters) in data

Question 3

Unsupervised Learning methods

Answer 3

PCA
Clustering

( Data observation and low-complexity data description )

Question 4

SVM vs. Logistic Regression for (almost) separable classes

Answer 4

SVM

Question 5

SVM vs. Logistic Regression for non-separable classes

Answer 5

SIMILAR
SVM and Logistic regression (with ridge penalty)

Question 6

SVM vs. Logistic Regression for estimating probabilites

Answer 6

Logistic regression

Question 7

SVM vs. Logistic Regression for fast and interpretable model

Answer 7

Logistic regression

Question 8

SVM vs. Logistic Regression for non-linear boundaries

Answer 8

kernel SVM’s

( kernel Logistic regression expensive )

Question 9

K-means clustering partition requirements

Answer 9

1. Each value belongs to at least one cluster
2. Each value belongs to only 1 cluster

(no-overlap)

Question 10

Clustering potential issues

Answer 10

1. Standardize observations first?
2. How many clusters?
3. What type of linkage / dissimilarity measure?

Question 11

Within-Cluster-Variation formula

Answer 11

-

( usually a cumulative value is reported )

Question 12

K-means clustering Vorgehensweise

Answer 12

1. Initial clustering - Randomly assign a number from 1 to K to each of the observations.
2. Compute cluster centroids
3. Assign observations to closest centroid (using eucledian distance)

Question 13

Hierarchical clustering Vorgehensweise

Answer 13

1. Treat each observation as own cluster
2. Measure pairwise dissimilarities (i.e. eucledian distance)
3. Fuse most similar clusters, and re-compute

(benefit is

Question 14

Linkage / Dissimilarity measures

Answer 14

1. Complete: maximal intercluster dissimilarity
2. Single: minimal intercluster dissimilarity
3. Average: mean intercluster dissimilarity
4. Centroid: dissimilarity between the centroid for the cluster

Question 15

Vorgehensweise of Hyperplane / SVM

Answer 15

Among all separating hyperplanes, find the one that makes the biggest gap or margin between the two classes == Maximal Margin Classifier

If not possible:
loosen „separate“ requirement (slack variables)
enlarge the feature space so that separation becomes possible (e.g. feature expansion woth transformed variables -> non-linear boundaries)

Question 16

Popular kernel functions

Answer 16

• Linear kernel (standard for linear classification)
• Radial Basis Function
• Polynomial