Week 7 - Bayesian inference

Question 1

Frequentist probability

Answer 1

Probability is based on how often something happens in the long run

Recall to the Law of Large Numbers (LLN)

Question 2

Bayesian probability

Answer 2

Probability as a measure of belief or uncertainty about an event, based on both prior knowledge and new evidence

Question 3

Bayesian inference

Answer 3

Approach that uses Bayes’ theorem to update the probability estimate for a hypothesis as more evidence or data becomes available

It provides a flexible way to model uncertainty by combining prior beliefs with observed data, giving a posterior distribution—the updated belief after observing data

P(θ∣data) = [P(data∣θ) ⋅ P(θ)] / P(data)

• P(θ∣data): Posterior -> the updated belief about the parameter 𝜃 after observing the data
• P(data∣θ): Likelihood – the probability of the observed data given the parameter 𝜃
• P(θ): Prior – the initial belief or information about the parameter
  𝜃 before observing the data
• P(data): Evidence – the total probability of observing the data, used for normalisation

Question 4

Bayesian inference steps

Answer 4

1. Specify a Prior Distribution
2. Collect Data and Specify the Likelihood
3. Apply Bayes’ Theorem to Combine Prior and Likelihood
4. Determine the conjugate pair
5. Workout posterior distribution using conjugate pair
6. Summarize the Posterior Distribution using the prior and data

Question 5

Credible interval

Answer 5

Simply take probability intervals from the POSTERIOR, which will represent the uncertainty

The posterior probability of the parameter being in the credible interval is 95%

Question 6

Conjugate prior

Answer 6

A prior distribution that, when combined with a specific type of likelihood function, results in a posterior distribution belonging to the same family as the prior

This pairing is known as a conjugate pair

Question 7

Prior–likelihood conjugate pairs

Answer 7

Beta-Binomial
Beta-Bernoulli
Gamma-Poisson
Gamma-Exponential
Normal-Normal (mean)