Wk 9: Comparing Two Groups

Question 1

If we increase the size of our sample, we would expect that the sample standard deviation will be the ______ (same/different). Why?

Answer 1

same

because the population is fixed.

Question 2

If we increase the size of our sample, we would expect that the standard deviation of the sample mean will be _______ (larger/smaller). Why?

Answer 2

smaller

As we get more people in the sample, they will be closer together - smaller standard deviation
x4 sample size, then standard deviation will 1/2

Question 3

If we increase the size of our sample, we would expect that the standard deviation of the sample mean will be ______. As we get more people in the sample, they will be closer together - smaller standard deviation x4 sample size, then standard deviation will be _________.

Answer 3

smaller; 1/2

Question 4

What are 2 things that normal distributions describe?

Answer 4

1. Describes the distribution of observations
2. Describes the distribution of statistics, such as the sample mean and sample proportion
   
   • Sample proportion is a type of sample mean

Question 5

In any normal distribution, _____% of data fall within _____ standard deviations of the mean.

Answer 5

95; 2

Question 6

Why do we have standard error?

Answer 6

In practice, we usually do not know the population standard deviation, so we have to estimate it using the sample standard de viation.

Question 7

What is standard error?

Answer 7

The estimated standard deviation of a statistic

Question 8

Why do we use Student’s T Distribution?

Answer 8

• Using the sample standard deviation instead of the population standard deviation adds more uncertainty to our estimation.
• We use Student’s T distribution instead of a normal distribution as it accounts for the extra variability introduced.
  
  • We use Student’s T distribution based on the degrees of freedom of our estimate.

Question 9

If mean difference is outside the 95% confidence interval, then it ______ (supports/rejects) the “effect” hypothesis.

Answer 9

supports

Question 10

What are the circles, vertical lines, half length of vertical line, grey and red on this graph? What changes the length of the vertical lines?

Answer 10

• Circles are sample mean
• Margin of error is the vertical lines
• Half the length of vertical line is standard error of mean x 1.96
• Grey = the population mean is within confidence interval (94% are in, which is close to 95% confidence)
• Red = the population mean is not within confidence interval
• Extra variability changes the length of vertical lines

Question 11

What is Significance? What supports the “effect” hypothesis and what supports the null hypothesis?

Answer 11

Significance is the P value

• If P value is outside the 95% confidence interval, then it supports the “effect” hypothesis.
• If P value is within the 95% confidence interval, then it supports the null hypothesis.

Question 12

What is the P value?

Answer 12

If a decision is required then a threshold for evidence needs to be set.

Question 13

What is the natural suspicion level?

Answer 13

The natural suspicion level is α = 0.05.

Question 14

If we find a P value <0.05, we would ______ (support/reject ) H0 null hypothesis and say that the results were significant at the 5% level.

Answer 14

reject

Question 15

What is the P value a transformation of?

Answer 15

Data > Mean > T value > P value

• Random the whole way through, so P-value is a random variable like the sample mean.

Question 16

Similarly, a confidence interval procedure generates _____ intervals.

Answer 16

random

Question 17

If the null hypothesis is true, the shape of the P-value distribution will be ________.

Answer 17

uniform (flat)

Question 18

Usually it is worthwhile to do the experiment if the probability of P <0.05 is ≥_____ % - this means you have _____ % chance that the “effect” hypothesis is true.

Answer 18

80; 80

Question 19

What are type 1 errors?

Answer 19

Rejecting a true null hypothesis.

• The probability of making a Type 1 error is the significance level α that we choose for making decisions

Question 20

What are type 2 errors?

Answer 20

Retaining a false null hypothesis.

• The probability of making a Type 1 error is β

Question 21

What is power?

Answer 21

The power of an experiment is the probability of detecting an effect when there is indeed an effect.

Question 22

More power is _____ (better/worst)

Answer 22

better

Question 23

What are 4 ways that power can be improved?

Answer 23

1. Increasing the effect size
2. Decreasing the variability: Stricter protocol, more accurate measurement etc. to account for other things that affect variability.
3. Increasing the sample size
4. Increasing the significance threshold α

Question 24

What is the Effect of signal to noise?

Answer 24

μ/σ

Where:

• μ = effect size (signal)
• σ = variability (noise)
• μ/σ = 1 means signal equals to noise
• μ/σ = 0.5 means signal is half as much as noise

Question 25

What does more power mean for effect size and variability?

Answer 25

Higher effect size, lower variability = more power

Question 26

What does this show?

Answer 26

1. μ/σ=1 2. Larger sample size = more power

Question 27

What does this show?

Answer 27

1. μ/σ = 0.5 2. Larger sample size = more power, but the power increases slower this time because the noise is double as much as the signal.

Question 28

What are 3 things when choosing sample size?

Answer 28

1. Typically we want to design our study to have a power of ≥80%. 2. This involves estimating the effect size and variability, and then choosing a sample size accordingly.

Question 29

What are independent samples T-test?

Answer 29

We take the difference between the two sample means and compare it to the standard error of the difference.

Question 30

What is the t statistic in independent samples T Test?

Answer 30

the number of standard errors that the difference between our groups is away from a hypothesised difference of 0.

Question 31

What are the 4 assumptions that the independent samples T test is based on?

Answer 31

1. The two groups are independent 2. The populations have normal variability 3. The variances are equal (optional) 4. Outliers can dilute the results by in inflating the standard error.

Question 32

What are equal variances?

Answer 32

the same amount of variability (spread) between two groups.

Question 33

If we can assume equal variances then we use a \_\_\_\_\_\_\_. Why?

Answer 33

pooled t test * Slightly more powerful and also generalises easily to comparing more than two groups.

Question 34

If we can't assume equal variances then we use a \_\_\_\_\_\_\_.

Answer 34

Welch t test

Question 35

What is the Levene's Test?

Answer 35

It has the null hypothesis that the populations have equal variances.

Question 36

If Levene's test outcome is signifcant (P _____ (\>/\<) 0.05 ), then it suggests there is ____ (equal/different) variances \> use _____ t test.

Answer 36

\>; different; Welch

Question 37

If Levene's test outcome is not significant (P _____ (\>/\<) 0.05 ), then it suggests there is ____ (equal/different) variances \> use the _____ t test.

Answer 37

\<; equal; pooled t test

Question 38

How should you analysis this data for P value?

Answer 38

1. First, look at Levene's Test P value (0.239). * If P \>0.05, then use the first row of data. * If P\<0.05, then use the second row of data. 2. Second, look at T test P value and see whether it falls between the mean differences and standard error difference. * If it falls within, then it supports the "effect" hypothesis. * If it does not fall within, then it supports the null hypothesis.