Common Distributions

Question 1

What is a Bernoulli random variable?

Answer 1

A discrete random variable that takes value 1 with probability p and 0 with probability 1 − p, modeling a single success/failure trial.

Question 2

What is a typical use case for a Bernoulli distribution?

Answer 2

Modeling outcomes of yes/no events such as click vs no click, success vs failure, or positive vs negative label.

Question 3

What are the parameters of a Bernoulli distribution?

Answer 3

A single parameter p, the probability of success (X=1).

Question 4

What are the mean and variance of a Bernoulli(p) random variable?

Answer 4

Mean is p and variance is p(1 − p).

Question 5

What is a Binomial random variable?

Answer 5

A discrete random variable that counts the number of successes in n independent Bernoulli trials with success probability p.

Question 6

What are the parameters of a Binomial distribution?

Answer 6

Two parameters: n (number of trials) and p (success probability per trial).

Question 7

What are the mean and variance of a Binomial(n,p) random variable?

Answer 7

Mean is np and variance is np(1 − p).

Question 8

What is a typical ML-related use case of the Binomial distribution?

Answer 8

Modeling counts of positive events in a fixed number of trials, such as number of clicks in n impressions.

Question 9

What is a Geometric random variable?

Answer 9

A discrete random variable that represents the number of trials needed to get the first success in repeated Bernoulli trials.

Question 10

What is the key property of the Geometric distribution?

Answer 10

The memoryless property: the probability of needing k more trials does not depend on how many failures occurred previously.

Question 11

What is a common use of the Geometric distribution?

Answer 11

Modeling waiting times for the first occurrence of an event, such as first success or first failure.

Question 12

What is a Poisson random variable?

Answer 12

A discrete random variable that counts the number of events occurring in a fixed interval when events happen independently at a constant average rate.

Question 13

What is the parameter of a Poisson distribution?

Answer 13

A single parameter λ, the average rate or expected count over the interval.

Question 14

What are the mean and variance of a Poisson(λ) random variable?

Answer 14

Both mean and variance are equal to λ.

Question 15

What is a typical ML-related use case for Poisson distributions?

Answer 15

Modeling count data such as number of events, calls, or arrivals per time window.

Question 16

How is the Poisson distribution related to the Binomial distribution?

Answer 16

When the number of trials n is large and p is small with np fixed, a Binomial(n,p) can be approximated by a Poisson with λ = np.

Question 17

What is a discrete uniform distribution on a finite set?

Answer 17

A distribution where each of a finite set of k outcomes has equal probability 1/k.

Question 18

What is a continuous uniform distribution on an interval [a,b]?

Answer 18

A distribution where the random variable is equally likely to take any value in [a,b], with constant density 1/(b − a).

Question 19

What is the mean of a Uniform(a,b) continuous distribution?

Answer 19

The mean is (a + b)/2.

Question 20

What is the variance of a Uniform(a,b) distribution?

Answer 20

The variance is (b − a)² / 12.

Question 21

What is a Normal (Gaussian) random variable?

Answer 21

A continuous random variable whose distribution has the bell-shaped curve defined by a mean μ and variance σ².

Question 22

What are the parameters of a Normal distribution?

Answer 22

Mean μ and variance σ² (or equivalently standard deviation σ).

Question 23

What is the mean and variance of a Normal(μ,σ²) distribution?

Answer 23

Mean is μ and variance is σ² by definition.

Question 24

Why is the Normal distribution important in ML and statistics?

Answer 24

Because many aggregated effects approximate a normal distribution (via CLT), and many models assume Gaussian noise or priors.

Question 25

What is a standard Normal distribution?

Answer 25

A Normal distribution with mean 0 and variance 1, often denoted Z ~ N(0,1).

Question 26

How can any Normal(μ,σ²) variable be standardized to a standard Normal?

Answer 26

By subtracting the mean and dividing by the standard deviation: Z = (X − μ)/σ.

Question 27

What is the Exponential distribution?

Answer 27

A continuous distribution describing the time between events in a Poisson process, with density decreasing exponentially.

Question 28

What is the parameter of an Exponential distribution?

Answer 28

Rate parameter λ > 0, where mean waiting time is 1/λ.

Question 29

What is the mean and variance of an Exponential(λ) distribution?

Answer 29

Mean is 1/λ and variance is 1/λ².

Question 30

What key property does the Exponential distribution share with the Geometric distribution?

Answer 30

The memoryless property: future waiting time does not depend on how long you have already waited.

Question 31

What is a common modeling use of the Exponential distribution?

Answer 31

Modeling waiting times or lifetimes when events occur at a constant rate and independently over time.

Question 32

What is a Gamma distribution (at a high level)?

Answer 32

A continuous distribution that generalizes the Exponential, modeling the sum of multiple independent Exponential waiting times.

Question 33

What are typical parameters of a Gamma distribution?

Answer 33

Shape parameter k (or α) and rate parameter θ⁻¹ or λ; conventions vary.

Question 34

What is a common use of the Gamma distribution in ML?

Answer 34

As a prior over positive quantities like rates or variances in Bayesian models.

Question 35

What is a Beta distribution (at a high level)?

Answer 35

A continuous distribution defined on [0,1] with flexible shapes controlled by two parameters, often used to model probabilities.

Question 36

What are the parameters of a Beta distribution?

Answer 36

Two positive shape parameters α and β.

Question 37

Why is the Beta distribution often used as a prior for probabilities?

Answer 37

It is defined on [0,1], can represent many shapes (uniform, skewed, peaked), and is conjugate to the Bernoulli/Binomial likelihood.

Question 38

What is a log-normal distribution?

Answer 38

A distribution where the logarithm of the variable is normally distributed.

Question 39

When might log-normal distributions appear in ML contexts?

Answer 39

For modeling positive, multiplicative quantities such as incomes, traffic, or response times that have right-skewed distributions.

Question 40

Why are heavy-tailed distributions important to recognize?

Answer 40

They indicate higher probability of extreme values than the Normal, affecting risk, outlier handling, and model robustness.

Question 41

What does it mean to choose a 'likelihood' in a statistical model?

Answer 41

Selecting a distribution family that describes how observed data are generated from underlying parameters or latent variables.

Question 42

Why should the choice of distribution be guided by the data?

Answer 42

Using an inappropriate distribution can lead to biased estimates, poor uncertainty quantification, and mis-specified models.

Question 43

What is the difference between parametric and nonparametric distributions (roughly)?

Answer 43

Parametric distributions are described by a finite set of parameters (e.g., Normal with μ,σ²); nonparametric methods make fewer assumptions and allow more flexible shapes.

Question 44

In one sentence, what is the role of common distributions in ML?

Answer 44

They provide building blocks for modeling noise, counts, waiting times, and probabilities, and underpin many loss functions and inference procedures.