Reliability

Question 1

what is reliability?

Answer 1

a measure of consistency

Question 2

what are the ways of assessing reliability?

Answer 2

• test-retest
• inter-observer reliability
• measuring reliability

Question 3

what is the process of test-retest?

Answer 3

• involves administering the same test or questionnaire to the same person or people on different occasions
• if the test or questionnaire is reliable, the results obtained should be the same or at least very similar, each time they are administered

Question 4

what is test-retest most commonly used for?

Answer 4

questionnaires and psychological tests(e.g. IQ tests)
- can be applied to interviews

Question 5

what is the precaution of test-retest?

Answer 5

• there must be sufficient time between the test and retest to ensure that the ppt/ respondent cannot recall their answers
• but not so long so that their attitudes, opinions or abilities may have changed

Question 6

what should be done when two scores can be collected in test-retest?

Answer 6

• they are correlated
• if the correlation is significant then the reliability is good

Question 7

what is inter-observer reliability?

Answer 7

• observers watch the same event, or sequence of events, but record their data independently
• data collected by observers should be correlated to assess its reliability

Question 8

what is inter-observer reliability relevant for?

Answer 8

observational research
- one observer’s interpretation of events may differ widely from someone else’s
- pilot studies can be used to train observers to see if they apply behavioural categories in the same way

Question 9

what is one way to establish inter-observer reliability?

Answer 9

• pilot study to check that observers are applying behavioural categories in the same way

Question 9

what is measuring reliability?

Answer 9

• reliability is measured using correlational analysis
• in test-retest and inter-observer reliability, two test scores are correlated

Question 10

what is the correlation coefficient?

Answer 10

exceed +0.80 for reliability

Question 11

how can we improve reliability in questionnaires?

Answer 11

• test-retest method (comparing two sets of data should produces correlation that exceeds +0.80)

Question 12

if a questionnaire produces a low test-retest reliability what should be done?

Answer 12

• some of the items to be ‘deselected’ or rewritten

Question 13

how can we improve reliability in interviews?

Answer 13

• use the same interviewer each time
• interviewers should not ask leading or ambiguous questions
• use structured interviews = fixed questions

Question 14

what type is interview is the least reliable?

Answer 14

unstructured interviews

Question 15

how can we improve reliability in observations?

Answer 15

• making behavioural categories properly operationalised, measurable and self evident
• categories should not overlap

Question 16

what should be done if the reliability of an observation is low?

Answer 16

• observers may need further training in using behavioural categories
• discuss their decisions with each other

Question 17

how can we improve reliability in experiments?

Answer 17

• procedures are the focus of reliability
• to compare performance of ppts, the procedure must be the same each time
  = standardised procedures