Week 6 - Maximum likelihood

Question 1

Statistical modelling

Answer 1

Trying to decide with distribution fits best

Question 2

Statistical modelling process

Answer 2

1. Choose a model for the data - family of probability distributions (NEGB) - we assuming a model here
2. Fit the model - estimating parameters using a sample, which selects a specific distribution from the chosen family (gives us the parameters)
3. Assess adequacy of model

Question 3

Maximum likelihood

Answer 3

Find parameter values that maximise the probability of the data

Question 4

Maximum likelihood estimate

Answer 4

The estimate of the parameter that maximises the likelihood

Obtain L(p) - likelihood function
Convert to l(p) - log-likelihood function
Solve derivative

Question 5

Maximum likelihood: general procedure

Answer 5

Likelihood function - L(θ) = Product(from i to n, in relation to xi) * pdf/pmf

MLE - θ^ = arg max (of θ) * L(θ)

Question 6

Invariance property

Answer 6

A consistent estimator for a parameter, then any transformation of that parameter will also be a consistent estimator for the transformed parameter

Question 7

MLE properties

Answer 7

• Asymptotically smallest variance (‘efficient’)
• Asymptotically unbiased
• Asymptotically normally distributed

Question 8

CI for MLE

Answer 8

Similar approach
- Can use: estimate +- quantile * se

OR
- Bootstrap method

Question 9

Quantile-Quantile (QQ) plots

Answer 9

Help assess if plausible that data came from specified distribution
(e.g. a distribution from MLE fit)

Create scatter plot of ordered data (y-axis) against theoretical quantiles (x-axis)

If both sets of quantiles from same distribution ⇒ points should lie on a straight line

Use a “thick marker” judgment approach - thick marker approach highlights the outliers as it covers up the close values and only showcases the outliers

Note that a bootstrap can be applied -> bootstrap on sample and then do QQ plot for each sample against chosen distribution