Linear Classifiers

Question 1

What does P(X,Y) represent

Answer 1

The joint distribution that describes the joint statistics of 2 random variables

Question 2

Define a joint cumulative distribution function

Answer 2

F(x,y) = P(X <= x, Y <= y)

Question 3

What do the parameters of the univariate gaussian distribution control

Answer 3

μ controls the location of the peak
σ controls the width of the peak

Question 4

Why is the euclidean distance not suitable for higher dimensions

Answer 4

It ignores covariance between features

Question 5

Give the formula for the Mahalanobis distancce

Answer 5

Multiply inverse covariance matrix by (x-y)^T

Then dot with (x-y)

That answer then ^1/2

Question 6

Describe the Mahalanobis distance in words

Answer 6

The distance between two vectors when we account for covariance

Question 7

What are eigenvectors

Answer 7

The eigenvectors (e) of the covariance matrix describe the direction of the contours

Question 8

What are eigenvalues

Answer 8

The eigenvalues (λ) describe the spread of the contours in these principal directions defined by the eigenvectors

Question 9

What is the length of a contour of equal probability

Answer 9

The length will be proportional to the square root of the eigenvalue

Question 10

How do you sketch the approximate contours for a guassian, given the mean column vector, the covariance matrix, the eigenvalues and the eigenvectors

Answer 10

The centre point is the mean’s co-ordinates

Draw ‘x and y’ arrows in the direction of the eigenvectors, starting at mean

Stretch the circle by the factors of the square root of the eigenvalues in their respective direction

Distance from mean to edge of ‘circle’ is the sqrt of eigenvalue in the direction

Question 11

Sketch the contours for this guassian

Answer 11

Question 12

Sketch the contours for this guassian

Answer 12

Question 13

What are the 2 parameters of the multivariate guassian distribution

Answer 13

Mean vector and a covariance matrix

Question 14

If there are L features, how many values do we have to estimate for the parameters

Answer 14

L values of the mean vector

L(L+1)/2 values for the covariance