Lecture 14; Two-Sample Testing:

Question 1

What are the 5 steps in statistical hypothesis testing?

Answer 1

1) How the research hypothesis should be transformed into a statistical question.
2) State the null and alternative hypotheses.
3) Compute the observed value for a particular metric of interest
4) Compute the P-value
5) Draw a conclusion by contrasting the p-value against the significance level (𝛼). If the p-value is greater than 𝛼, then do not reject H0; if P-value is smaller or equal than 𝛼, then reject H0.

Question 2

In the following example: define the null and alternative hypotheses:
Example; Normal human body temperature, as kids are taught in North America, is 98.6oF. But how well is this supported by data?:

Answer 2

1) H0 (null hypothesis): the mean human body temperature is 98.6oF
2) HA (alternative hypothesis): the true population is different from 98.6oF.

Question 3

What are the effects of increasing sample size on hypothesis testing?

Answer 3

By increasing the sample size, the probability of the p-value of observing a value that is equal or more extreme is much smaller because the t-distribution becomes more narrow
–> increasing sample size likely provided more statistical power such that we may be able to reject the null hypothesis

Question 4

Can we say with 100% confidence that increasing sampling size increased statistical power (precision)?

Answer 4

NO: Statistical power is defined relative to a true effect, and since the truth is unknown in real data, we cannot directly observe or confirm power; we can only infer that, if a real effect exists, a larger sample size would make it more likely to detect it.

Question 5

What occurs to the p-values are sample size increases?

Answer 5

Sample size decreases the standard error, which makes the t value (test statistic) increase, which in turn leads to smaller p-values.
–> Smaller P values allows rejecting the null hypothesis

Question 6

What is the power of a test (1-β) and what occurs to it as sample size increases?

Answer 6

It is the probability of rejecting the null hypothesis when it is truly false.
–> This probability increases as sample size increases.

Question 7

What are the 2 important assumptions of the one-sample t-test?

Answer 7

1) The data are assumed to represent a random and independent sample from the population = unbiased
2) The variable of interest (e.g., human body temperature) is assumed to follow an approximately normal distribution in the population. This assumption is particularly important for small sample sizes, as it justifies the use of the t-distribution.

Question 8

Scientific question: Does clear-cutting a forest affect the number of salamanders present: is this a one or two-sample hypothesis testing and why?

Answer 8

Statistical hypothesis testing for comparing two sample means: Because there are two treatments: clear cutting / no clearcutting (control).

Question 9

In the following example, what are the 2 types of variables seen?

Scientific question: Does clear-cutting a forest affect the number of salamanders present?

Answer 9

1) Treatment is a categorical variable
2) Number of salamanders is a numerical variable.

Question 10

Scientific question: Does clear-cutting a forest affect the number of salamanders present?

There are two main alternative study designs that affect the choice of statistical test, what are they?

Answer 10

1) In the two-sample design, each treatment group is composed of an independent, random sample unit.
2) In the paired design, both treatments are applied to every sampled unit (here - forest plots)

Question 11

What is the advantage of a paired design when doing the comparison of 2 means?

Answer 11

The advantage of a paired design is that it minimizes the impact of variability among sampling units that is unrelated to the treatment, thereby increasing the precision of the results (e.g., local environmental differences among observational units). It reduces confounder variables.

Question 12

What is an example of a confounder variable that could be present in the following example?

Scientific question: Does clear-cutting a forest affect the number of salamanders present?

Answer 12

Notice that clear-cutting occurred more frequently in wet soils, whereas areas without clear-cutting were predominantly dry. If soil moisture plays a critical role for salamanders, this non-random distribution of sampling units could bias the results and affect the conclusions.
–> By doing paired tests, we can eliminate confounding effects since the effects will likely affect either treatment areas

Question 13

What is a confounding variable?

Answer 13

A confounding variable is an unmeasured third variable that influences both the supposed cause and the supposed effect

Question 14

What are 5 other examples of paired study designs?

Answer 14

1) Comparing patient weight before and after hospitalization.
2) Comparing fish species diversity in lakes before and after heavy metal contamination.
3) Testing effects of sunscreen applied to one arm of each subject compared with a placebo applied to the other arm.
4) Testing effects of smoking in a sample of smokers, each of which is compared with a non-smoker closely matched by age, weight, and ethnic background.
5) Testing effects of socioeconomic condition on dietary preferences by comparing identical twins raised in separate adoptive families that differ in their socioeconomic conditions.

Question 15

What is a previously seen example of paired design?

Answer 15

The same spider male that self amputates, speed before and speed after = paired design

Question 16

What is the H0 and HA in the following example:

Are males with high testosterone paying a cost for this extra mating success in other ways (trade-offs)? OR Is avian humoral immunocompetence (i.e., ability of the immune system to produce antibodies to defend against pathogens) suppressed by testosterone?

Answer 16

1) H0: The mean change in antibody production in the population after testosterone implants is zero.
2) HA: The mean change in antibody production in the population after testosterone implants is different from zero.

Question 17

What is 𝜇d?

Answer 17

𝜇d: is the population mean difference between treatments

Question 18

What is the difference between paired observations between two samples equal to (property of the means)?

Answer 18

Differences between paired observations between two samples is equal to the differences between means (this is a property of means)

Question 19

Differences between paired observations between two samples is equal to the differences between means (this is a property of means): what does this property allow for?

Answer 19

This property allows us to analyze within-pair differences directly using a one-sample t-test, which properly accounts for the pairing and reduces variability.
–> Importantly, if we instead treated the data as two independent samples, we would lose this pairing information and use fewer degrees of freedom (df), resulting in a less powerful test.

Question 20

What are the 3 assumptions for paired t-tests (paired comparison of 2 means?

Answer 20

1) The observational units are randomly and independently sampled from the population because that’s how the t-distribution was built
2) The observations are paired (i.e., measurements are meaningfully linked within units = have the same units).
3) The differences within pairs are approximately normally distributed in the population (especially important for small sample sizes).

Question 21

What is an example doing a comparison of 2 independent sample means?

Answer 21

Do the spikes of horned lizards provide protection against predation from loggerhead shrikes?

Question 22

In the following example, what are the HA and H0?

Do the spikes of horned lizards provide protection against predation from loggerhead shrikes?

Answer 22

1) H0: Lizards killed by shrikes and living lizard do not differ in mean horn length (i.e., 𝜇1 = 𝜇2).
2) HA: Lizards killed by shrikes and living lizards differ in mean horn length (i.e., 𝜇1 ≠ 𝜇2).

Question 23

What is the quantity Sp^2?

Answer 23

The quantity 𝑠p2 is called the pooled sample variance and is the average of the sample variances weighted by their degrees of freedom (related to sample sizes).

Question 24

What is pooled sample variance?

Answer 24

Pooled sample variance is a weighted average of individual sample variances, where groups with larger sample sizes (higher degrees of freedom) receive greater weight.

Question 25

Do the spikes of horned lizards provide protection against predation from loggerhead shrikes? If the results are P = 0.000023, what is the decision based on alpha and the statistical conclusion and scientific conclusion?

Answer 25

Decision based on alpha = 0.05: reject H0 - STASTISTICAL CONCLUSION: Evidence shows that the mean horn length statistically differs between lizards killed by shrikes and those that survive. - SCIENTIFIC CONCLUSION: We generated evidence suggesting that larger horn size may provide protection against predation.

Question 26

What are the 4 important assumptions of 2 sample (means) t-test?

Answer 26

1) Both samples are independent random samples drawn from their respective populations 2) The observations are independent within and between samples. 3) The variable of interest is approximately normally distributed in each population (especially important for small sample sizes). 4) *The population variances are equal across groups (homoscedasticity).

Question 27

What occurs if the population variances are not equal across groups?

Answer 27

When population variances are not equal across groups (a condition known as heteroscedasticity), the actual alpha level (Type I error rate, the probability of rejecting a true null hypothesis) deviates from the chosen nominal alpha (e.g., 0.05).

Question 28

What are the 2 statistics of a normally distributed population?

Answer 28

Mean and variance

Question 29

Why is it especially important for small sample sizes that a population be normally distributed?

Answer 29

Because with small samples, the sampling distribution of the mean follows the shape of the underlying population rather than becoming normal automatically. When sample sizes are small (typically), the Central Limit Theorem (CLT) does not "kick in" to correct for skewed or non-normal data, meaning that statistical inferences (like t-tests or confidence intervals) are only valid if the population itself is normal.