Parameter Estimation (needs practice (mixups))

Question 1

Describe the parametric approach to building a classifier

Answer 1

Choose distribution for P(x|ωi) with parameters θ

For each class ωi, find the parameters that best fit the training data

Determine the prior probabilities P(wi), determine what share of the training data this class makes up

Compute P(ωi|x) for each class and make a classification

Question 2

What does p(x) means

Answer 2

a function of x, same as f(x)

Question 3

What is the parameter (θ) likelihood function with respect to X

Answer 3

p(x1,x2, … ,xN:θ)

We can think of this as the probability that all these datapoints came from the distribution with parameters θ

Question 4

What is the maximum likelihood estimate

Answer 4

A function that chooses the parameter values 𝜃 that make the observed data 𝑋
as likely as possible under the model.

The function chooses parameters that maximises the likelihood

likelihood: probability of the training data given the class

Question 5

Why maximise the likelihood

Answer 5

By maximising the likelihood function, we find the parameter values that best explain the observed data

Question 6

In many cases the data is independent of each other (peoples heights are independent of one another), what do we do to ensure that Max likelihood function only has one input

Answer 6

Factorise the probabilities

P(A,B) = P(A)P(B)

Question 7

How is the product of all the max likelihood probabilities shown

Answer 7

Question 8

How do you find the minimum and maximum of f(x)

Answer 8

Find the turning points

df(x)/dx = 0

Question 9

Using a cunning trick, instead of maximising the likelihood, what should we maximise

Answer 9

the log likelihood

ln p(X; θ)

Question 10

How do we find the best parameters using the log likelihood

Answer 10

Start with the likelihood

Take the log of the likelihood, this turns it into a sum

Take the derivative of the log likelihood and set it to zero

Calculate θ

Question 11

What is the advantage of taking the log likelihood

Answer 11

It turns the product of all the likelihoods into the sum of all the log likelihoods, which is easier to deal with

Question 12

What is the negative log likelihood

Answer 12

the minus of the negative log likelihood, you minimise it rather than maximise it

Question 13

What is the maximum of the gaussian function for the mean and variance and therefore the parameters

Answer 13