Research Methods - Inferential Testing

Question 1

When are statistical tests used?

Answer 1

A statistical test is used to determine whether a difference or association/correlation found in a particular investigation is statistically significant (i.e. whether the result could have occurred by chance or there is a real effect).

Question 2

What are the three criteria when choosing a statistical test?

Answer 2

1. Looking for a difference or a correlation/association?
2. Is experimental design related (repeated measures/matched pairs) or unrelated (independent groups)?
3. What is the level of measurement?

Question 3

What are the tests of difference?

Answer 3

(Related) Repeated
Nominal: Sign test
Ordinal: Wilcoxon
Interval: Related t-test

(Unrelated) Independent
Nominal: Chi-Squared
Ordinal: Mann-Whitney U
Interval: Unrelated t-test

Question 4

What are the tests of correlation?

Answer 4

Nominal: Chi-Squared
Ordinal: Spearman’s rank/rho
Interval: Pearson’s product-moment/r

Question 5

What are parametric tests?

Answer 5

Parametric tests are those that make assumptions about the parameters of the population distribution from which the sample is drawn.

They include the related t-test, the unrelated t-test, and Pearson’s product-moment/r.

Question 6

What is nominal data?

Answer 6

• This is the most level of measurement,
• Used when data is put into tally charts/categories. For this reason, it is sometimes referred to as category data.
• Gives very little information as it is basically a headcount, it only tells us how many people are in each group.
• Each item can only appear in one category. There is no order.

Question 7

What is ordinal data?

Answer 7

• This is used when data can be put into order, e.g. 1st, 2nd and 3rd.
• If there is a scale, it’s ordinal data.
• It cannot tell us what gap is between 1st and 2nd, or between 4th and 5th (intervals are variable).
• Intervals are subjective.
• Usually based on opinion therefore tend to be subjective rather than objective, and so lacks precision.

Question 8

What is interval data?

Answer 8

• The most precise level of measurement.
• Interval data is based on numerical scales that include units of equal, precisely defined size.
• e.g. the gap between 1 and 3 seconds is exactly double the gap between 1 and 2 seconds.
• e.g. the gap between 2 and 3cm is exactly the same as the gap between 10 and 11cm.
• Public units of measurement.
• Interval data is ‘better’ than ordinal data because more detail is preserved as the scores are not converted to ranks.

Question 9

What is significance?

Answer 9

The difference/association between two sets of data is greater than what would occur by chance - coincidence or fluke. To find out if the difference/association is significant, we need to use a statistical test.

Question 10

What happens if the statistical test is not significant?

Answer 10

If the statistical test is not significant, the null hypothesis is accepted. The null hypothesis states there is ‘no difference’ or ‘no correlation’ between the conditions. The statistical test determines which hypothesis (null or alternative) is ‘true’ and thus which we accept and reject.

Question 11

What is probability?

Answer 11

Probability (p) is a numerical measure of the likelihood that certain events will occur, where 0 is statistical impossibility and 1 is a statistical certainty.

There are no statistical certainties in psychology but there is a significance level - the point at which the null hypothesis is accepted or rejected.

Question 12

What is the usual level of significance?

Answer 12

The accepted level of probability in psychology is 0.05 (or 5%). This is the level at which the researcher decides to accept the research hypothesis or not.

If the research hypothesis is accepted, there is less than 5% probability that the results occurred by chance.

This is a compromise between too lenient (10%) or too stringent (1%).

Question 13

What is the calculated and critical values?

Answer 13

The calculated value is compared with a critical value to decide whether the result is significant or not. The critical values for a particular test are given in a table of critical values based on probabilities.

Question 14

How do you find the critical value?

Answer 14

To find the critical value, you need to know:

• The significance level (usually 0.05 or 5%).
• The number of participants in the investigation (the N value) or the degrees of freedom (df).
• Whether the hypothesis is directional or non-directional.

Question 15

What does one-tailed mean?

Answer 15

When there’s a directional hypothesis.

Question 16

What does two-tailed mean?

Answer 16

When there’s a non-directional hypothesis.

Question 17

What is a type one error?

Answer 17

A type one error refers to a situation in which we assume that our findings show something when they don’t.

• rejecting a null hypothesis that is true
• wrongly accepting the alternative hypothesis
• assuming the results were due to the IV when they were in fact due to chance

Therefore we should have kept the null hypothesis rather than rejecting it.

This is an optimistic error or false positive as a significant difference or correlation is found when one does not exist.

Question 18

What is a type two error?

Answer 18

A type two error refers to a situation in which we may miss something that is actually happening.

• accepting a null hypothesis that is false
• wrongly rejecting the alternative hypothesis
• assuming the results were due to chance when they were in fact due to the IV

Therefore we should have rejected the null hypothesis rather than accepting it.

This is a pessimistic error or false negative.

Question 19

What is wrong with type one error?

Answer 19

too lenient

Question 20

What is wrong with type two error?

Answer 20

too strict

Question 21

What is the standard p-value?

Answer 21

Setting the p-value correctly is a balancing act, as we want to avoid both type one and two errors but they can never be entirely avoided. The “standard” used by the social sciences is P < (equal to or less than) 0.05.

Question 22

What are the conditions of use for the sign test?

Answer 22

Used to analyse the difference in scores between related items, e.g. the same participant is tested twice. Can be used with nominal data (or better).

Question 23

What is the calculation for the sign test?

Answer 23

1. Convert the data to nominal data if necessary.
2. The score for condition B is subtracted from condition A to produce the sign of difference (either a plus or a minus).
3. The total number of pluses and the total number of minuses should be calculated.
4. Participants who achieved the same score in condition A and condition B should be disregarded, and deducted from the N value.
5. The S value is the total of the less frequent sign.

Question 24

What is the critical value for the sign test?

Answer 24

If S is equal to or less than critical value, then S is significant and the experimental hypothesis is retained.