What is credibility?
Credibility is a criterion used to judge qualitative research and how it corresponds to internal validity.
How can researchers ensure credibility?
- used for qualitative research; not asked for an experimental study
- there are types of triangulation three types of triangulation that could be used to increase the credibility of the study
- data triangulation
- involves different data sources within the same method. If two different populations are used, then they could interview them at different points in time, in private vs. public settings
- to ensure that it was not the sample alone which led to the findings, but that the results were the same across all sources
- method triangulation
- utilising different data collection methods with the same sample in order to check the consistency of the findings
- to ensure that it was not the choice of the RM which influenced the results
- researcher triangulation
- uses different, other researchers to help carry out and analyse the data
- data triangulation
- Member checking can also be used, it is a technique in which the data, interpretations, and conclusions are shared with the participants. It allows participants to clarify what their intentions were, correct errors, and provide additional information if necessary. (used when things can be misinterpreted, to check that the data collected was concordant with what the participants meant)
- reflexivity must be ensured
- the researcher tries to constantly account for bias
What is generalisability?
Generalisability is a criterion used to judge quantitative research and how it corresponds to internal validity.
how can a researcher ensure generalisability?
Paper 3
When generalising from an experiment you should consider:
- Was there any sampling bias?
- Was the sample representative?
- Was the sample large enough?
- Does the study have a high level of ecological validity? (or is it artificial and overly controlled?)
The size of the sample should always be commented on, whilst also considering the likelihood that the study can be easily replicated to get the same findings (increasing the reliability of the data.
Representational Generalisation needs to be drawn- the sample needs to be representative of the group that they want to apply the research to
Inferential generalisation (transferability)- certain variables need to be identified in order to apply it to another population: (these can vary depending on the study)
- the location of the study
- cultural dimensions
- age of the participants
- economic status of the participants
What is bias
A researcher should always attempt to avoid bias when conducting research (in any form) as it can lead to incorrect data.
How can a researcher avoid bias in a study?
Single and double-blind controls
- in a single-blind control, the participants do not know whether they are in the experimental or the control group, this helps to prevent demand characteristics
- In a double-blind control, neither the participants nor the researchers who are analysing the data know which participants are in the experimental or control group
Randomisation should always be included, this helps ensure that any individual differences are spread evenly across groups
When selecting participants from a larger population, random sampling helps reduce the chance of bias in the sample
The procedures should be standardised; the participants should receive the same instructions in the same way reducing variability in how the study is administered
Standardised testing conditions: Consistency in the environment, materials and procedures to minimise bias.
If interviews are used:
- leading questions should be avoided and all questions asked should be neutral
- the questions should be reviewed ahead of time
- a series of pilot interviews should be carried out
- maybe use an unstructured interview
Personal reflexivity should be ensured
- this is where the researcher thinks about how their own personal values, beliefs and experiences may have influenced their ability to be objective when carrying out and interpreting the data in their study
Epistemological reflexivity should be ensured
- this is when a researcher reflects on how the way that the research was carried out may have influenced the findings e.g. the use of a focus group may lead to conformity effects.
All of the data collected should be Peer Reviewed
- peer review is the evaluation of a researchers work by others working in the same field
- all research should be approved by a committee of peers before being undertaken- and then the final paper should be reviewed before publication
- peers should be able to point out any bias which may compromise the way in which a study is done or how the findings are interpreted
Researcher Triangulation
- having more than one researcher carry out an interview or observation, or interpret data
- the goal of this is to have researchers work independently and, hopefully, come up with the same results.
- If all researchers agree upon a result, it can be said that there is inter-rater reliability
Selection Bias, this can lead to cultural bias, self-selection bias and ascertainment bias
- Cultural bias: e.g. WEIRD, western, educated, industrialised, rich and democratic cultures
- Self-selection bias: people that volunteer tend to be more highly motivated than the average person or have specific reasons why they want to be in this particular study
- Ascertainment bias: when a particular group is left out from research
