What is a likelihood function?
It is a special type of distribution that specifies how likely different potential values of a parameter are given the type of sampling strategy, data, and estimate collected.
Define parameters
Characteristics about an entire population of interest
What is the interval estimate?
The range of numbers from the entire middle 95% of a likelihood function.
How do you compute an interval estimate?
1) Conduct a simulation to find estimate/standard error
2) Find the margin of error (2*estimation error)
3) Compute the lower limit of the interval estimate (point estimate - margin of error)
4) Compute the upper limit of the interval estimate (point estimate + margin of error)
5) Report lower and upper limits together
How would you format reporting lower and upper limits together?
“The middle 95% of the likelihood function lies between a value of [lower limit] and [upper limit]. Therefore, we believe that the true value of a [parameter]of [attribute of interest] for [population of interest] is somewhere between [lower limit] and [upper limit].”D
Define bias
The systematic way in which data has been collected has fundamental problems that will prevent statistical methods from producing a correct estimate
Define measurement bias
Any bias in the data due to problems with the measurement process
Define sample bias
Any bias in the data due to problems with the representativeness of the sample
Define causal bias
Any bias in the data due to differences between the observational units in the control group versus in the treatment group
Define estimate precision
How narrow the interval estimate can be created based on the collected data
Name two factors that affect precision
1) Amount of data collected
2) Standard deviation of the observational units
What is evidence-based testing?
A form of statistical testing that begins with observed evidence in the form of data. It then uses the likelihood function for a parameter before evaluating hypotheses by comparing them to said likelihood function.
Define null model
A representation of your expectation for what the data will be
Define likelihood ratio test
Computes the ratio of two likelihoods for two hypotheses
Three or more means that one hypothesis is much more likely than the other
ROOM FOR ISH-NESS ONE IS LESS CRAPPY THAN THE OTHER
Define decision test
First, two null models are plotted on the same graph.
Next, the point at which the two null models intersect is located
Finally, we compare the observed evidence to this intersection and discard the hypothesis furthest from the observed evidence
NO ROOM FOR ISH-NESS or GREY AREA
What is power and accuracy?
Refers to the ability of a decision test to distinguish between competing hypotheses.
The less the overlap the better the test, because it’s clearer.
What does it mean if the P-value is large (e.g. 0.374)?
The data is consistent with what was expected
What does it mean if the P-value is small (e.g. 0.002)?
The data is not consistent with what was expected
Hypothesis-based Testing
A form of statistical testing that begins with a hypothesis, creates a probability distribution based on the hypothesis and the estimation error, and subsequently evaluates estimates from data by comparing them to the probability distribution.
