Reliability And Validity

Question 1

What is reliability?

Answer 1

The consistency of a measuring instrument/results

Question 2

How is reliability measured?

Answer 2

By assessing:
- inter-rater reliability
- split-half reliability
- test re-test reliability

Question 3

What is inter-rater reliability?

Answer 3

• the extent to which the is rn agreement between 2 or more raters
• often used for observation research and content analysis

Question 4

How do you assess inter-rater reliability?

Answer 4

• 2 (or more) observers/laters independently observe/assess/rate the same thing and record their data
• the data from 2 armore laters are compared and correlated
• positive correlation means there’s inter-rater reliability, correlation coefficient of 0.80 or more = good inter-rater reliability

Question 5

What is split-half reliability measuring?

Answer 5

Measures internal reliability/internal consistency of a measuring instrument

Question 6

How do you assess split-half reliability?

Answer 6

• Split test in half (first half and second half)
• positive correlation indicates consistency

Question 7

What is test re-test reliability?

Answer 7

• External reliability
• measures consistency over time/ on different occasions

Question 8

How do you assess test re-test reliability?

Answer 8

• Same participants complete same test on 2 separate occasions
• positive correlation between first and second testing indicates consistency
  Same test + same people= reliability, same test + different people = validity

Question 9

What is validity?

Answer 9

Whether me research measures what it set out to measure
- measuring room cannot be valid if it isn’t reliable

Question 10

What is internal validity?

Answer 10

• Whether the researcher rested what may intended to test
• is concerned with what goes on within the study

Question 11

What factors reduce internal validity?

Answer 11

Observer/researcher bias
Demand characteristics and participant reactivity
Investigator effects
Social desirability bias
Confounding variables
Poorly operationalised behavioural categories

Question 12

What is external validity?

Answer 12

The extent at which the findings of the study can be generalised beyond the study to:
- other situations (ecological)
- other people (population)
- other cultures (cross-cultural)
- other times (temporal)

Question 13

What is ecological validity?

Answer 13

Ability to generalise a research effect beyond the particular setting which its demonstrated in another setting

Question 14

What is mundane realism?

Answer 14

• how a study numbers the real world
• research environment is realistic to experiences in the real world
  -Non real it feels for participants

Question 15

What is temporal validity?

Answer 15

Whether research from one time period can be generalised to another time period

Question 16

How can you assess/improve validity?

Answer 16

• Face (or content) validity
• Concurrent validity
• Predictive validity

Question 17

What is face (or content) validity?

Answer 17

Extent to which the test looks like they measure what the test claims to measure

Question 18

How do you assess face validity?

Answer 18

Ask independent judges whether they think the measuring tool measures what it set out to measure

Question 19

What is concurrent validity?

Answer 19

Establishing validity by comparing your newly designed test with an existing previously validated one assessing he same factor/ variable

Question 20

How do you assess concurrent validity?

Answer 20

• Compare performance on new test with another previously established test assessing the same thing
• participants complete both tests
• positive correlation between the 2 tests = concurrent validity