validity and reliability

Question 1

validity definition

Answer 1

the extent to which a study/test/measuring instrument measures what it intends to measure

Question 2

internal validity definition

Answer 2

whether the effects observed in a study are due to the manipulation of the independent variable or any other factor

Question 3

external validity definition

Answer 3

how well you can generalise the results from research participants to people,places,times outside of the study

Question 4

population validity

Answer 4

• type of external validity
• the extent to which we can apply the findings of the study to different types of people

Question 5

temporal validity

Answer 5

-type of external validity
- the extent to which we can apply the findings of the study to different type periods/eras

Question 6

ecological validity

Answer 6

• type of external validity
• the extent to which we can apply the findings of the study to other settings/situations

Question 7

cultural validity

Answer 7

• type of external validity
• the extent to which we can apply the findings to other cultures

Question 8

how to determine whether a study is ecologically valid

Answer 8

• participants’ awareness of the experimenter (investigator effects)
• mundane realism of the environment
• mundane realism of the task

Question 9

what is face validity

Answer 9

whether a test, scale or measure appears to measure what it is supposed to measure ‘on the face of it’. This is done by eyeballing the measuring instrument or getting an expert to check

Question 10

what is concurrent validity

Answer 10

demonstrated when the results obtained are very close to those obtained on another recognised and well-established test measuring the same concept

Question 11

how to improve validity in experimental research

Answer 11

• use a control group (researcher can assess whether change in the DV was due to the IV or not)
• standardise procedures (minimise investigator effects)
• use single blind or double blind procedures (minimise investigator effects and demand characteristics)

Question 12

how to improve validity of questionnaires

Answer 12

• include a likert scale (assess the consistency of a participant’s response, controls social desirability bias)
• assure participants their results are anonymous (minimises social desirability bias)

Question 13

how to improve the validity of observations

Answer 13

• use covert, non-participant observations (minimal intervention by researcher leads to high ecological validity)
• use specific, non-overlapping unambiguous behavioural categories

Question 14

how to improve the validity of qualitative methods

Answer 14

• make sure the researcher’s report is coherent and matches the participant’s intentions (include direct quotes in a report)
• triangulation

Question 15

what is reliability

Answer 15

how consistent the findings from an investigation or measuring device are.

Question 16

what is external reliability

Answer 16

the extent to which results from a research method can be replicated over time or with different participants

Question 17

how to test reliability

Answer 17

• test re-test reliability
• inter-observer reliability

Question 18

test re-test reliability

Answer 18

• assesses reliability of questionnaires, psychological tests, interviews
• administering the same test to the same person on different occasions
• enough time between tests to ensure that participants cannot recall their answers
• the results obtained should be the same each time they are administered, if it is reliable
• 2 sets of scores are correlated with the appropriate statistical test (+0.8+ reliable)

Question 19

inter-observer reliability

Answer 19

• assesses reliability of observations
• observations are conducted in teams of 2+
• observers discuss and agree on their interpretation of the behavioural categories
• they watch the same event but record the data independently, using the same categories
• data collected is correlated using the appropriate statistical test (+0.8+ reliable)

Question 20

how to improve the reliability of questionnaires

Answer 20

• if test-retest reliability is low, some items are deselected or rewritten
• open, ambiguous questions replaced with closed, fixed choice, less ambiguous questions

Question 21

how to improve the reliability of interviews

Answer 21

• use the same interviewer each time
• interviewers properly trained using structured interviews

Question 22

how to improve the reliability of observations

Answer 22

• behavioural categories operationalised
• categories should not overlap
• train observers in the use of observational checklists and behavioural categories (using time/event sampling)