Reliability (Paper 2)

Question 1

What is the rhyme about reliability

Answer 1

It ain’t great unless it’s +.8 (Statisticians usually write it as .80 though)

Question 2

What is reliability

Answer 2

How consistent results are. If the researchers replicate their study exactly, they should get similar results.

Question 3

What is external reliability

Answer 3

The extent to which a measure is consistent when repeated.

(e.g. the results of a study are consistent with an exact replication at a different time and/or with different participants)

Question 4

What is internal reliability

Answer 4

The extent to which different parts of a measure are consistent with itself.

(e.g. if a 100-question IQ test is divided into two 50-question tests, the results for each set of questions with the same participant would be similar)

Question 5

How can you assess internal reliability

Answer 5

Split half method: Assess measures that test one variable with multiple questions, for example, questionnaires or tests of IQ.

Question 6

How do you perform the split half method

Answer 6

Split the test into two parts.

Participants complete both parts.

Test the strength of the correlation between the two parts of the measure.

A strong correlation indicates internal reliability

Question 7

How can you assess external reliability

Answer 7

Test - Retest and Inter Observer / inter-rater reliability

Question 8

How do you perform a test retest

Answer 8

Repeat the study using the same procedures / measuring devices at different times and test the correlation between the two versions.

Question 9

How do you perform inter - observer / inter-rater reliability

Answer 9

Two (or more) observers record behaviours during thesameobservation using the same behavioural categories; then, they test the correlation between each tally of behaviour to identify if the behavioural categories are appropriately operationalised.

Question 10

How can you compare the original test and the retest or the observations by the 2 observers

Answer 10

The level of correlation is assessed using a test of correlation such as Pearson’s R or Spearman’s Rho. A correlation of 0.8 or higher is usually accepted as a strong correlation.

Question 11

How can psychologists improve reliability of observations

Answer 11

Improving the training given to observers can improve their accuracy in assigning a particular observed behaviour to the correct behavioural category. Pilot studies can identify poorly defined behavioural categories; operationalisation is the process of clearly defining a behavioural category.

Question 12

How can psychologists improve reliability of interviews

Answer 12

Use structured interviews rather than unstructured interviews. The interview will then include a script the interviewer can follow, ensuring each participant has a similar experience and provides answers. The researcher can then compare responses from each participant.

Question 13

How can psychologists improve reliability of questionnaires

Answer 13

Use closed questions to reduce the range of possible responses, and if there is an established questionnaire that tests for what you need to measure, use that rather than creating a new test.

Question 14

How can psychologists improve reliability of experiments

Answer 14

Use standardised procedures for each participant, so they all have the same experience. For example, keep what is said to the participants the same (by using a script) and environmental conditions the same. Use established tests as measures rather than creating a new test.