C2 - Reliability and Validity

Question 1

What is reliability?

Answer 1

The extent to which a test produces consistent results

Question 2

what are the two types of reliability?

Answer 2

internal reliability & external reliability

Question 3

what is internal reliability?

Answer 3

Whether the test is consistent in itself

Question 4

What is an example of internal reliability?

Answer 4

Whether the different questions in a questionnaire are all measuring the same thing

Question 5

what is external reliability?

Answer 5

the extent to which a test produces consistent results over several occasions

Question 6

What is an example of external reliability?

Answer 6

If we repeated an IQ test on a different day, we would expect the same result

Question 7

What are causes of reliability issues? (4 points)

Answer 7

• Lack of operationalisation of the variables
• Order effects
• Lack of standardised procedures
• Inconsistency of measuring tools

Question 8

What are examples of lack of standardised procedures?

Answer 8

Instructions aren’t given/questions aren’t worded in the same way to each participant

Question 9

why is a lack of operationalised variables a reliability issue?

Answer 9

If you don’t state how you are defining and measuring your variables, it makes replication impossible and makes inter-rater reliability issues more likely

Question 10

why are order effects a reliability issue?

Answer 10

order effects occur in a repeated measures design. Whilst completing the second condition the participant can become better practised at the task the second time around, or underperform due to boredom

Question 11

how do we deal with reliability issues?
(2 points)

Answer 11

• using a standardised procedure, instructions and scientific measuring equipment
• using a laboratory environment

Question 12

how do we deal with order effects?

Answer 12

Using counterbalancing :
get half of the participants to complete condition a then condition b and the other half vice versa.

Question 13

what are the three ways of assessing reliability?

Answer 13

• split half reliability/ testing
• test retest reliability
• inter-rater reliability

Question 14

What type of reliability does split half reliability test?

Answer 14

Internal reliability

Question 15

what is split half reliability/ testing?

Answer 15

splitting a participants test answers in half and seeing whether they got the same or similar scores scores on the two halves. if they are similar, internal reliability is high, if not, the questions either need to be reordered or redesigned.

Question 16

Which type of reliability does test-retest reliability test?

Answer 16

External reliability

Question 17

what is test-retest reliability?

Answer 17

testing and retesting the same participants over time with the same test, and comparing their scores. if the scores are the same, the test has external reliability

Question 18

what is inter-rater reliability?

Answer 18

When you get 2 or more observers to observe the participants and they record what they see. If their results are similar then there is high internal reliability.

Question 19

What does inter-rater reliability get rid of?

Answer 19

Observer bias

Question 20

What are the two types of validity?

Answer 20

• Internal validity
• External validity

Question 21

What is internal validity? (2)

Answer 21

• Whether the study measures what it intends to measure
• Whether we can be certain that the IV is what is causing the change in the DV

Question 22

What is external validity?

Answer 22

Whether the study paints a true picture of real-life behaviours (mundane realism) and would apply to different places, times or people

Question 23

What are the 2 main validity issues?

Answer 23

• Extraneous and confounding variables
• Low mundane realism

Question 24

What is researcher bias?

Answer 24

The researcher can directly or indirectly influence the result of a study, through the process of designing the study or through the way the research was conducted or analysed

Question 25

What are demand characteristics?

Answer 25

When participants unconsciously work out the aim of the study and act differently than how they would normally

Question 26

What is social desirability bias (SDB)?

Answer 26

When participants give the response that they think will show them in the best possible light, meaning they are not a true reflection of their thoughts or feelings

Question 27

How can researcher bias be dealt with?

Answer 27

Using a double blind technique where neither the participant nor the research knows which condition the participant is in

Question 28

How can SDB and demand characteristics be reduced?

Answer 28

Using a single blind procedure where the participant does not know which condition they are in

Question 29

What are the 5 ways of assessing validity?

Answer 29

- Face validity - Predictive validity - Content validity - Concurrent validity - Construct validity

Question 30

What is face validity?

Answer 30

Whether the test appears to measure what it claims to, so is therefore objective

Question 31

What is predictive validity?

Answer 31

The degree to which a test predicts a future outcome on a more broadly related topic

Question 32

What is content validity?

Answer 32

When you use a panel of experts to check whether you are measuring what you intend to measure

Question 33

What is concurrent validity?

Answer 33

Comparing your test to a pre-established test. If results are similar then it is valid

Question 34

What is construct validity?

Answer 34

When you clearly define what you are aiming to measure and then you make sure that all parts of the DV are being measured

Question 35

What can internal validity be affected by? (2)

Answer 35

- Lack of mundane realism - Extraneous variables and confounding variables

Question 36

What are examples of EVs and CVs that affect the internal validity of an experiment? (4)

Answer 36

- Environment (e.g. lighting) - Differences in participants (participant variables) - The way the researcher acts to each participant (investigator effects) - Demand characteristics

Question 37

How can EVs and CVs that cause issues with validity be fixed?

Answer 37

- Using standardised procedures - Using repeated measures or matched pairs design - Using double blind technique - Creating a situation with higher mundane realism