Examples of quantitative methods
Experiments (e.g. RCTs)
Controlled measurements (e.g. cohort study)
Surveys: questionnaires with closed questions
Databases
Examples of qualitative methods
Interview with open‐ended questions
Focus group discussions
Ethnography
THINGS TO CONSIDER WHEN READING A PAPER (OR
DOING/EVALUATING A RESEARCH PROJECT)
- The study did not address an important scientific issue
- The study was not original‐ that is, someone else has already done the same or a similar study
- The study did not actually test the authors’ hypothesis
- A different study design should have been used
- Practical difficulties (e.g. in recruiting participants) led
the authors to compromise on the original study protocol - The sample size was too small
- The study was uncontrolled or inadequately controlled
- The statistical analysis was incorrect or inappropriate
- The authors have drawn unjustified conclusions from
their data. - There is a significant conflict of interest (e.g. one of the authors, or a sponsor, might benefit financially from the publication of the paper and insufficient safeguards
were seen to be in place to guard against bias) - The paper is so badly written that it is incomprehensible.
RANDOMISED CONTROL TRIALS (RCTS)
Aripiprazole is a novel antipsychotic with a pharmacodynamic profile that distinguishes it from other antipsychotics.
Patients fulfilling study entry criteria were randomly assigned to receive aripiprazole, 30 mg/day, or placebo.
The primary efficacy measure was mean change from baseline in total score on the Young Mania Rating Scale.
RANDOMISED CONTROL TRIALS (RCTS) AND OTHER METHODOLOGIES
Cohort: following a group of patient using the aripiprazole and a group not using it, recording the outcomes of interest in each group over a period of time.
Case ‐control: comparing the exposure history of a group of bipolar patients who have suffered the outcome of interest, i.e. adverse events (the cases) with
a group who have not (the controls). Did they differ in terms of aripiprazole use?
Ecological: comparing population rates of aripiprazole and adverse events in different places or at different times, to see whether they vary together.
What can qualitative research do for randomised controlled trials?
RCTs are only designed to answer whether or not an
intervention is effective.
Qualitative research methods, on the other hand, do not suffer from such constraints. They are particularly
good at answering what and why type questions.
‘Will this intervention work in
a variety of health care settings?’ and ‘What might be the problems in trying to introduce this intervention throughout a health service?’
WHY MIGHT A RANDOMISED CONTROL
TRIAL NOT BE SUITABLE?
It might be unethical to randomize patients to an intervention for which there is evidence of harm.
The extent to which RCTs’ results are applicable outside the RCTs varies.
Factors include:
Where the RCT was performed (e.g., what works in one country may not work in another)
Characteristics of the patients (e.g., an RCT may include patients whose prognosis is better than average, or may exclude “women, children, the elderly, and those with common medical conditions”
Study procedures (e.g., in an RCT patients may receive intensive diagnostic procedures and follow‐up care difficult to achieve in the “real world”)
Outcome measures (e.g., RCTs may use surrogate measures rather than clinical outcomes).
Costs‐RCTs can be expensive.
Factors That Can Affect the External Validity of Randomised
Controlled Trials
Setting of the trial
- health care system
- country
- recruitment from primary, secondary, or tertiary care
- selection of participating centres
- selection of participating clinicians
Selection of patients
- methods of prerandomisation diagnosis and investigation
- eligibility of criteria
- exclusion criteria
- placebo run-in period
- enrichment strategies
- ratio of randomised patients to eligible nonrandomised patients in participating centres
- proportion of patients who declined randomisation
Characteristics of randomised patients
- baseline clinical characteristics
- racial group
- uniformity of underlying pathology
- stage in the natural history of their disease
- severity of disease
- comorbitiy
- absolute risks of a poor outcome in the control group
Differences between trial protocol and routine practice
- trial intervention
- timing of treatment
- appropriateness/relevance of control intervention
- adequacy of nontrial treatment - both intended and actual
- prohibition of certain nontrial treatments
- therapeutic or diagnostic advances since trial was performed
Outcome measures and follow up
- clinical relevance of surrogate outcomes
- clinical relevance, validity and reproducibility of complex scales
- effect of intervention on most relevant componenets of composite outcomes
- identification of who measured outcomes
- frequency of follow up
- adequacy of the length of follow up
Adverse effects of treatment
- completeness of reporting of relevant adverse effects
- rates of discontinuation of treatment
- selection of trial centres and/or clinicians on the basis of skills or experience
- exclusion of patients at risk of complications
- exclusion of patients who experienced adverse effects during a run in period
- intensity of trial safety procedures
