1
Q
Qualities of a research question
A
Specific: with respect to time/place/subjects/condition as appropriate
Answerable: such that the relevant data are available or able to be collected
Novel: in some sense so that the study either makes a contribution to knowledge or extends existing knowledge
Relevant: to current medicine
2
Q
Types of studies
A
- Interventional vs observational
- Time-course: prospective; retrospective; cross-sectional
- Source of data: new data; routine data; patient notes; existing data, e.g. secondary data analysis, meta-analysis
3
Q
Historical controls
A
- The comparison of the treatment group and the control group is not concurrent and may be problematic as other factors change over time, such as hospital staff and patient mix
- Interpretation is difficult – it is impossible to be sure that any differences observed between the new treatment group and the control group are solely due to the treatments received
4
Q
Approach for observational studies
A
- Collect as much data as possible on the subjects’ key characteristics.
- Use statistical analysis to adjust for these differences.
- Note that, even with statistical adjustment, there may still be differences between the groups that are unknown and so comparisons may still be biased. We probably won’t know.
- Interpretation of non-randomized trials is difficult and firm conclusions are hard to draw.
5
Q
Describe an RCT
A
A randomized controlled trial (RCT) is an intervention study in which subjects are randomly allocated to treatment options.
Randomized controlled trials (RCTs) are the accepted ‘gold standard’ of individual research studies. They provide sound evidence about treatment efficacy which is only bettered when several RCTs are pooled in a meta-analysis.
6
Q
Choice of the control group
A
When a best therapy currently exists this should be used as the control
It is unethical to randomise to placebo when a current treatment exists (Declaration of Helsinki, item 32)
When comparing an agent to placebo you are more likely to find a beneficial effect versus compared to best current therapy
7
Q
Declaration of Helsinki Item 32
A
‘The benefits, risks, burdens and effectiveness of a new intervention must be tested against those of the best current proven intervention, except in the following circumstances:
• The use of placebo, or no treatment, is acceptable in studies where no current proven intervention exists; or
• Where for compelling and scientifically sound methodological reasons the use of placebo is necessary to determine the efficacy or safety of an intervention and the patients who receive placebo or no treatment will not be subject to any risk of serious or irreversible harm’
8
Q
Declration of Helsinki
A
Developed in 1964 by World Medical Association (WMA)
Outlines principles for:
• Duties of those conducting research involving humans
- Importance of a research protocol
- Research involving disadvantaged or vulnerable persons
- Considering risks and benefits
- Importance of informed consent
- Maintaining confidentiality
- Informing participants of the research findings
9
Q
Why randomise participants?
A
Randomization ensures that the subjects’ characteristics do not affect which treatment they receive. The allocation to treatment is unbiased
In this way, the treatment groups are balanced by subject characteristics in the long run and differences between the groups in the trial outcome can be attributed as being caused by the treatments alone
This provides a fair test of efficacy for the treatments, which is not confounded by patient characteristics
Randomization makes blindness possible
10
Q
Minimization
A
Minimization is another method of allocating subjects to treatment groups while allowing for important prognostic factors.
The allocation takes place in a way that best maintains balance in these factors. At all stages of recruitment, the next patient is allocated to that treatment which minimizes the overall imbalance in prognostic factors
11
Q
Blocking
A
Blocking is used to ensure that the number of subjects in each group is very similar at any time during the trial.
The random allocation is determined in discrete groups or blocks so that within each block there are equal numbers of subjects allocated to each treatment.
12
Q
Consenting in research
A
Declaration of Helsinki item 24
Adequately informed of the aims, methods, sources of funding, any possible conflicts of interest, institutional affiliations of the researcher, the anticipated benefits and potential risks of the study and the discomfort it may entail
13
Q
Steps in informed consent
A
- This requires giving patients detailed description of the study aims, what participation is required, and any risks they may be exposed to
- Consent must be voluntary
- Consent is confirmed in writing and a cooling off period is provided to allow subjects to change their minds
- Consent must be obtained for all patients recruited to an RCT
- Giving or withholding consent must not affect patient treatment or access to services
- For questionnaire surveys, consent is often implicit if the subject returns the questionnaire where it is clear in the accompanying information that participation is voluntary
- Consent may not be required if the study involves anonymised analyses of patient data only
14
Q
Qualities of a placebo
A
- An inert treatment that is indistinguishable from the active treatment
- In drug trials it is often possible to use a placebo drug for the control which looks and tastes exactly like the active drug
- The use of a placebo makes it possible for both the subject and assessor to be blinded
15
Q
Qualities of a parallel group RCT
A
- This is a trial with a head-to-head comparison of two or more treatments
- Subjects are allocated at random to a single treatment or a single treatment programme for the duration of the trial
- Usually, the aim is to allocate equal numbers to each trial, although unequal allocation is possible
- The groups are independent of each other, UNLIKE a crossover trial
16
Q
Crossover trials
A
- This involves a single group study where each patient receives two or more treatments in turn
- Each patient therefore acts as their own control and comparisons of treatments are made within patients
- The two or more treatments are given to each patient in random order
- Crossover trials are useful for chronic conditions such as pain relief in long-term illness or the control of high blood pressure where the outcome can be assessed relatively quickly
- They may not be feasible for treatments for short-term illnesses or acute conditions that once treated are cured, for example antibiotics for infections
- It is important to avoid the carry-over effect of one treatment into the period in which the next treatment is allocated. This is usually achieved by having a gap or washout period between treatments to prevent there being any carry-over effects of the first treatment when the next treatment starts
- The simplest design is a two treatment comparison in which each patient receives each of the two treatments in random order with a washout period of non-treatment in between
- There are some particular statistical issues that may arise in crossover trials which are related to the washout period and carry-over effects, and how and whether to include patients who do not complete both periods.
17
Q
Advantages of parallel group designs
A
- The comparison of the treatments takes place concurrently
- Can be used for any condition, especially an acute condition which is cured or self-limiting such as an infection
- No problem of carry-over effects
18
Q
Disadvantages of parallel group designs
A
• The comparison is between patients and so usually needs a bigger sample size than the equivalent cross-over trial
19
Q
Advantages of crossover designs
A
- Treatments are compared within patients and so differences between patients are accounted for explicitly
- Usually need fewer subjects than the equivalent parallel group trials
- Can be used to test treatments for chronic conditions
20
Q
Disadvantages of crossover designs
A
- Cannot be used for many acute illnesses
- Carry-over effects need to be controlled
- Likely to take longer than the equivalent parallel designs
- Statistical analysis is more complicated if subjects do not complete all periods
21
Q
Advantages of Zelen’s single randomized design
A
- It avoids patient refusal at the outset due to the possibility of their being allocated to control
- It avoids later withdrawal in subjects who initially consent but then withdraw when they are allocated to the control group
- It allows a new and potentially desirable programme to be evaluated rigorously in a randomized trial
22
Q
Disadvantages of Zelen’s single randomized design
A
- Patients in the control group do not know they are in a trial, which has ethical implications
- The design leads to three groups and will lead to bias if subjects are not analysed in the group to which they were allocated irrespective off the treatment they chose or received
- Will only work if the data required are routinely collected, otherwise no data will be available for the control group
- It is less efficient statistically than a straightforward two-group design since, when subjects choose not to accept the allocated treatment, the true treatment effect is diluted
23
Q
Advantages of Zelen’s double randomized design
A
- It randomizes patients but allows them to choose which treatment they prefer
- It avoids the ethical problems of not seeking consent for patients allocated to control
- It thus allows a new and potentially desirable programme to be evaluated rigorously in a randomized trial
24
Q
Disadvantages of Zelen’s double randomized design
A
It almost inevitably leads to severe contamination of the groups since some patients will choose the opposite treatment to which they have been allocated
• It is less efficient statistically than a straightforward two-group design since, when subjects choose not to accept the allocated treatment, the true treatment effect is diluted
25
Equivalence trials
* Seek to test if a new treatment is similar in effectiveness to an existing treatment
* Appropriate if the new treatment has certain benefits such as fewer side effects, being easier to use, or being cheaper
* Trial is designed to be able to demonstrate that, within given acceptable limits, the two treatments are equally effective
* Equivalence is a pre-set maximum difference between treatments such that, if the observed difference is less than this, the two treatments are regarded as equivalent
* The limits of equivalence need to be set to be appropriate clinically
* The tighter the limits of equivalence are set, the larger the sample size that will be required
* If the condition under investigation is serious then tighter limits for equivalence are likely to be needed than if the condition is less serious
* The calculated sample size tends to be bigger for equivalence trials than superiority trials
26
Superiority trials
* Seek to establish that one treatment is better than another
* When the trial is designed the sample size is set so that there is high statistical power to detect a clinically meaningful difference between the two treatments
* For such a trial a statistically significant result is interpreted as showing that one treatment is more effective than the other
27
Importance in deciding superiority vs equivalence
A trial designed to test superiority is unlikely to be able to draw the firm conclusion that two treatments which are not significantly different can be regarded as equivalent
28
Intention to treat analysis
* Analyse subjects in the groups they were originally allocated to even if they don’t comply or change treatment
* This provides an unbiased comparison of the treatments i.e. balanced patient characteristics post randomisation remain intact
* Per protocol analysis may be useful but only in addition to ITT and not as the primary analysis
* Keep an record of all subjects to be able to account for their treatment and for any subjects who withdraw
29
Sample size for case-control studies
It is common to choose the sample size so that there is the same number of cases as controls. For a given total sample size this gives the greatest statistical power, i.e. the greatest possibility of detecting a true effect.
If the number of available cases is limited, then it is possible to increase the power by choosing more controls than cases However, the gain in power diminishes quickly so that it is rarely worth choosing more than 3 controls per case
30
Limitations of case-control design
* The choice of control group affects the comparisons between cases and controls
* Exposure to risk factor data is usually collected retrospectively and may be incomplete, inaccurate, or biased
* If the process that leads to the identification of cases is related to a possible risk factor, interpretation of results will be difficult (ascertainment bias) e.g. suppose the cases are young women with high blood pressure recruited from a contraception clinic. In this situation a possible risk factor, the oral contraceptive (OC) pill, is linked to the recruitment of cases and so OC use may be more common among cases than population controls for this reason alone.
31
Outcome limitations from case-control studies
* Time-course relationships need careful interpretation since changes in biological quantities may precede the disease or be a result of the disease itself. For example a raised serum troponin level is associated with myocardial infarction, but is only raised after the event. Therefore a case–control study may find that high troponin levels are associated with myocardial infarction but this cannot in fact be a risk factor
* Risk estimates for exposures cannot be estimated directly because the case and control groups are not representative samples of their respective target populations and so estimates of risks are biased. This has implications for the statistical analysis and the interpretation of results. Risks are usually estimated using odds and ratios of odds, and these only approximate to risks and ratios of risks when the disease under investigation is rare
* This limitation can be overcome with certain designs, for example where a case–control study is nested in a cohort study where all cases and controls are identified prospectively and a truly random sample of controls is available (Research design Cohort studies, p. [link]). In this situation, the relative risk can be calculated directly
32
Requirement for retrospective cohort studies
Cohorts can be retrospective but requires that full risk factor data are obtained on all individuals with and without the disease of interest using data that were recorded prospectively
33
Difficulties with cohort studies
* A large number subjects is needed to obtain enough individuals who get the disease or condition, particularly if it is uncommon
* The length of follow up may be substantial to get enough diseased individuals and so the cohort study is not feasible for rare diseases
* There is difficulty in maintaining contact with subjects, particularly if the follow-up is lengthy
* The resources required may be very high
34
Advantages of case-control
Useful for establishing associations in:
- Rare disease
- Acute outbreaks in which there is not time for a cohort study
- Diseases that take a very long time to develop
35
Nested case-control studies
In a cohort study it may be worthwhile to identify all individuals with a disease and then retrospectively select a sample of the non-diseased individuals for comparison. This design may be desirable if:
* The resource implications of collecting data on all non-diseased individuals is too high
* All information was available but unprocessed
* Biological samples were collected but not analysed
This study is known as a nested case–control study and provides an efficient way of investigating particular factors once the outcomes from the cohort have been established.
Bias in risk factor data
* In a nested case–control study such as this, the risk factor data should not be as biased as it may be in a conventional case–control study, since it was collected prospectively
* There is a potential problem if there is differential loss to follow-up as this would reduce the availability of true controls and bias the comparisons
36
Cross-sectional studies
Assess outcome at one-point in time
Use when:
• Surveys of prevalence, such as a survey to ascertain the prevalence of asthma
* Surveys of attitudes or views, such as: studies of patient satisfaction, patient/professional knowledge; studies of behaviour, such as alcohol use and sexual behaviour
* When inter-relationships between variables are of interest, for example a study to determine the characteristics of heavy drinkers, a cross-sectional study allows comparisons by sex, age, and so on
CANNOT assess:
-Temporal trends/causality e.g. did a disease cause HTN or did the HTN cause a disease
Cross-sectional studies that appear to be longitudinal
-Cross-sectional studies can be misinterpreted as if they were longitudinal studies. For example, a cross-sectional study in a sample of fetuses where the gestational age of the fetuses spans a range, say 22–28 weeks. Some researchers have used data such as these to estimate growth trends. This is dubious because each fetus is measured just once and so the trend is being estimated from different fetuses. Thus differences between fetuses are likely to contribute to some of the differences observed by gestational age.
37
Bradford-Hill Criteria
Allows causality to be more confidently inferred from observational studies:
* Strength of association
* Consistency in different studies, settings, etc.
* Specificity of association of risk factor with a particular disease
* Temporal relationship – exposure precedes disease
* Dose–response relationship
* Biological plausibility for causality
* Coherence – association is consistent with current knowledge
* Experimental evidence for causality
* Existence of analogous evidence between a similar exposure and disease
38
Clinical audit
‘Clinical audit is a quality improvement process that seeks to improve the patient care and outcomes through systematic review of care against explicit criteria and the implementation of change. Aspects of the structures, processes and outcomes of care are selected and systematically evaluated against explicit criteria. Where indicated, changes are implemented at an individual team, or service level and further monitoring is used to confirm improvement in healthcare delivery.’1
39
Secondary data
These are data collected and recorded for another research study, and which are available for use.
Advantages
* Relatively quick to obtain
* Usually already processed so that minimal checking and data cleaning is required
* Usually much lower cost than primary data collection
Disadvantages
* No control over data available
* Limited control over missing data and ability to fill gaps and resolve queries
* Data may not be in required or desirable format
* May be out of date
40
Advantages of composite outcomes
* They allow several outcomes to be combined in settings where different outcomes are of similar importance but reflect different clinical events, for example in a trial of treatment for gestational diabetes, the primary outcome was a composite measure of serious perinatal complications, defined as one or more of: fetal death, shoulder dystocia, bone fracture, and nerve palsy
* Main advantage of using a composite outcome is the gain in statistical power – where individual events are uncommon, a large sample will be required to demonstrate conclusive differences. Using a composite will increase the event rate and allows trials to recruit a lower sample size.
41
Disadvantages of composite outcomes
* It may be hard to determine the minimum clinical difference for the composite, this requires an estimate of the incidence of the composite itself and not just the incidence of the individual components as well as clinical judgement about what constitutes an important change in rate
* The interpretation of results may be difficult – it is important that the separate component effect sizes are each reported as well as the combined effect size, to allow clinical interpretation
* If the effect sizes (e.g. relative risks) vary among the components then overall interpretation of the findings is difficult, for example if a new treatment reduces subsequent adverse events but increases death rates
42
Surrogate marker
• A surrogate outcome should be closely related to the clinical outcome of interest such as a biomarker or process variable
43
Advantages and disadvantages in converting a continuous variable into categorical variable
e.g. cholesterol in mmol into normal vs high
Pros:
- Allows clinical applicability and aids clinical decision making
- Able to summarise data
Cons:
- leads to a loss of information which in turn has statistical consequences.
=loss of power
44
Estimating a mean with a specified precision
The following information is required:
* The standard deviation (SD) of the measure being estimated
* The desired width of the confidence interval (d)
* The confidence level
The standard deviation is needed because the sample size depends partly on the variability of the measure being estimated. The greater the variability of a measure, the greater the number of subjects needed in the sample to estimate it precisely.
The standard deviation can be estimated from previously published studies on the same topic, from contact with another worker in the field or from a small pilot study.
The desired width of the confidence interval, d, indicates the precision of the mean and is decided by the researcher.
The confidence level is usually set at 95%, giving a sample confidence interval that contains the true population mean with probability 95%. Other values such as 90% or 99% can be used, but are unusual in practice.
n=1.962×4SD2/d2
45
Estimating a proportion with a specified precision
The following information is required:
* The expected population proportion, p
* The desired width of the confidence interval (d)
* The confidence level
46
Type 1 error:
We conclude that there is a difference between the groups in the target populations when in fact there is not. This is actually the significance level of the test and so when we use 0.05 or 5% as the cut-off for statistical significance, then the probability of a type 1 error is 5%. This is often denoted by ‘α’.
47
Type 2 error:
We conclude that there is no difference between the groups in the target population when in fact a real difference of a given size does exist. The type 2 error is often denoted by ‘β’ and 1–β is the power of the study.
48
Calculating sample size for power in intervention studies comparing means
The following information is required:
* The standard deviation (SD) of the measure being compared
* The minimum difference (d) that is clinically important
* The significance level (α)
* The power of the test (1–β)
49
Calculating sample size for power in intervention studies comparing proportions
* The expected population proportion in group 1, P1
* The expected population proportion in group 2, P2
* The significance level (α)
* The power of the test (1–β)
The expected population proportion in group 1 and the expected population proportion in group 2 are the best estimates of what these values will be. The difference therefore reflects the anticipated change in the proportion which would be regarded as clinically important.
The significance level, α is the type 1 error and is usually set at 5%.
The power of the test, 1–β, is the probability of getting a significant result when the true difference between the proportions is d and is set at 80% or more, preferably 90%.
50
Assumptions of sample size formulae for means and proportions
* There is no attrition, i.e. the total number of patients successfully recruited and who complete the study is equal to the number required
* For comparative studies, there are equal numbers of subjects in each group
* Samples are simple random samples; any randomization is at the individual level. Sample size calculations are different for cluster samples or cluster randomization and the usual calculations will give too few subjects (see below)
* For comparative studies, a simple comparison of two groups only will be made. Multiple regression or logistic regression (Research design Chapter 12, p. [link]) is not planned
* The samples are large enough to use large sample methods for the analysis
51
When are sample size calculations not required?
- Qualitative (descriptive) studies
| - Pilot studies
52
Use of log scale in ratios
Odds ratio and risk ratios are usually based on log-transformed data which makes the data SYMMETRICAL!
Common feature is that ratio statistics all have an absolute lowest value of 0, which can extend up to infinity
Without log-transformed data, the number scale is not symmetric
e.g. OR of 0.5 is half risk
OR of 2 is double the risk
In the middle means equal risk but the average of 0.5 and 2 is not 1!
However, log to the base e of the above:
OR 0.5 --> -0.69
OR 2 --> 0.69
Average to 0 i.e. previous value of 1
53
Log-based meta analysis
Graphical displays for meta-analyses performed on ratio scales usually use a log scale.
This has the effect of making the confidence intervals appear symmetric, for the same reasons as log-based odds ratios.
54
Using odds ratios for effect measure
For interventions that increase the chances of events, the odds ratio will be larger than the risk ratio, so the misinterpretation will tend to overestimate the intervention effect, especially when events are common (with, say, risks of events more than 20%).
For interventions that reduce the chances of events, the odds ratio will be smaller than the risk ratio, so that, again, misinterpretation overestimates the effect of the intervention. This error in interpretation is unfortunately quite common in published reports of individual studies and systematic reviews.
55
Problems with using risk ratios for intervention effects
The risk difference is naturally constrained (like the risk ratio), which may create difficulties when applying results to other patient groups and settings. For example, if a study or meta-analysis estimates a risk difference of –0.1 (or –10%), then for a group with an initial risk of, say, 7% the outcome will have an impossible estimated negative probability of –3%. Similar scenarios for increases in risk occur at the other end of the scale. Such problems can arise only when the results are applied to populations with different risks from those observed in the studies.
56
Standardized mean difference
Used when outcome was measured in different ways
Necessary to standardize the results of the studies to a uniform scale before they can be combined. The SMD expresses the size of the intervention effect in each study relative to the between-participant variability in outcome measurements observed in that study.
57
Time to event: censored
Patients who contribute to some period of time but did not results in an event are said to be censored
58
Time to event analysis
For each participant, two factors are important:
1. Time that event did not happen
2. Whether the end-point was due to an event occurring or just end of observation
It is not appropriate to analyse time-to-event data using methods for continuous outcomes (e.g. using mean times-to-event), as the relevant times are only known for the subset of participants who have had the event.
59
Hazard ratio
Hazard ratio describes how many times more (or less) likely a participant is to suffer the event at a particular point in time if they receive the experimental rather than the comparator intervention.
When comparing interventions in a study or meta-analysis, a simplifying assumption is often made that the hazard ratio is constant across the follow-up period, even though hazards themselves may vary continuously. This is known as the proportional hazards assumption.
60
Cluster-randomised controlled trials
In cluster-randomized trials, groups of individuals rather than individuals are randomized to different interventions. We say the ‘unit of allocation’ is the cluster, or the group. The groups may be, for example, schools, villages, medical practices or families. Cluster-randomized trials may be done for one of several reasons. It may be to evaluate the group effect of an intervention, for example herd-immunity of a vaccine. It may be to avoid ‘contamination’ across interventions when trial participants are managed within the same setting, for example in a trial evaluating training of clinicians in a clinic. A cluster-randomized design may be used simply for convenience.
One of the main consequences of a cluster design is that participants within any one cluster often tend to respond in a similar manner, and thus their data can no longer be assumed to be independent. It is important that the analysis of a cluster-randomized trial takes this issue into account. Unfortunately, many studies have in the past been incorrectly analysed as though the unit of allocation had been the individual participants (Eldridge et al 2008). This is often referred to as a ‘unit-of-analysis error’ (Whiting-O’Keefe et al 1984) because the unit of analysis is different from the unit of allocation. If the clustering is ignored and cluster-randomized trials are analysed as if individuals had been randomized, resulting confidence intervals will be artificially narrow and P values will be artificially small. This can result in false-positive conclusions that the intervention had an effect. In the context of a meta-analysis, studies in which clustering has been ignored will receive more weight than is appropriate.
Medical statistics
flashcards
Decks in class (6)
# Cards
