reliability

Question 1

What is reliability?

Answer 1

How consistent something is

Question 2

What is internal reliability?

Answer 2

Whether a test is consistent within itself a.k.a. are all questions of equal difficulty
A test is said to have high internal reliability if all questions are of equal difficulty

Question 3

How do you assess internal reliability?

Answer 3

The split-half method
This involves comparing a person’s scores from the first and second half of the test
If the test is high in internal reliability the scores should be similar

Question 4

How can internal reliability be improved?

Answer 4

By conducting pilot studies to see if the questions are equally hard or easy- questions can be rephrased if some are harder than others

Question 5

What is external reliability?

Answer 5

Whether results are consistently found when the study is repeated
It’s important for studies to be replicable as external reliability can only be checked if the study is able to be carried out again

Question 6

How do you assess external reliability?

Answer 6

The test-retest method
This involves doing the study again to see if similar results are obtained
the results can be correlated and a positive correlation is indicative of high external reliability

Question 7

How do you improve external reliability?

Answer 7

Factors in the experimental setting e.g. EVs could affect participant performance and so control is the main factor that will improve external reliability
Ensuring all participants have the same standardised instructions and investigator effects are eliminated

Question 8

What is inter-observer/rater reliability?

Answer 8

The consistence between different researchers- if both researchers are collecting the same data
Most relevant to observational studies and content analysis as such studies often have more than one person recording data

Question 9

How do you assess inter-observer/rater reliability?

Answer 9

Researchers will observe the same situation and use the same method to record the data
Results are compared through the correlation method- whereby a positive correlation will be found if there is high reliability between the researchers

Question 10

How do you improve inter-observer/rater reliability?

Answer 10

Behavioural categories can be tightened/operationalised further so that they are not ambiguous and observers could receive training in observational techniques

Question 11

Explain how you would assess inter-rater reliability?

Answer 11

1. To assess inter-rater reliability a second research could work with the original researcher. They would agree on a set of categories and behaviours that fit them and them complete the observation using these categories
2. Each researcher would put a tally in the chart each time they observed that behaviour. Both researchers would then compare the results to ensure they have the same.
3. the researchers would then calculate the correlation between the two ratings
4. If the correlation coefficient is above 0.8 then we can assume there is inter-rater reliability