1
Q
What is the difference between research, audit and service evaluation?
A
Research - Aim is to derive new knowledge
Audit - Continuous service improvement by assessing structure/process/outcomes and against previously agreed standards.
Service evaluation - To judge current care by providing a systematic assessment of its aims, objectives, activities, outputs, outcomes, cost effectiveness.
2
Q
What 6 things should a research protocol contain?
A
- Background
- Aims and objectives
- Study design and methodology
- Ethical considerations
- Project Plan
- References
3
Q
What are the 5 types of research designs?
A
One-off data collection point
- Descriptive (but can be hypothesis testing)
- Prevalence
- Usually use a questionnaire
- Most common
Cohort or longitudinal studies
- Forward in time
- Naturally divides exposed / non-exposed groups
Case control studies
- Compare group with disease to control group
- Look back in time
- Good to look at rare events (e.g. leukemia)
Case Studies
- Circumstances, complexity, dynamics of a single case or small number of cases
Experimental design
4
Q
What are the 5 difficulties of experimental design?
A
- Difficult to design
- Difficult to exclude confounders/isolate the effect of variables
- Can’t always randomise
- Matching
- Costly
5
Q
What are the 4 different types of Quantitative studies?
A
- Questionnaires
- Highly structured Interviews
- Analysis of pre-existing datasets (e.g. PMRs, general practice databases)
- Recording of activities (e.g. interventions)
6
Q
What are the 3 different types of Qualitative studies?
A
Interviews (unstructured; semi-structured)
Focus Groups
Observation (participant and non-participant)
7
Q
What is hypothesis testing?
A
A prediction about the relationship between the dependent and independent variables
A hypothesis comes from a theory about the nature of the relationship
Collect data to see if your hypothesis is true or false
8
Q
What is confounding?
A
Factors (other than those under study and therefore not controlled for) which distort the results
Makes the dependent and independent variables appear connected when they are not
9
Q
What is bias?
A
a systematic error or flaw in the research design or data collection process that leads to results that don’t accurately represent the population being studied
10
Q
What are the 5 types of bias?
A
- Sampling
- Interviewer
- Non-response
- Reporting
- Observer
11
Q
Choosing a method
A
The method you choose depends upon your assumptions about the world
Quantitative/Laboratory-based methods reflect a positivist view of the world
Qualitative/social science methods reflect an interpretivist/naturalistic view of the world
12
Q
What is positivism?
A
A philosophy behind how we gather knowledge
Positivism says knowledge is what we can see and observe
Positivism concerns the application of scientific method
Positivists believed in empiricism: the use of observation and measurement to discover new knowledge
An external reality ‘out there’: can be examined and described objectively
Research is repeatable and generalisable
13
Q
What is interpretivism?
A
Attempts to understand phenomena by the meanings people assign to them
Looks at the context in which a behaviour occurs
Focuses on the complexity of human behaviour
Research conducted in natural settings
14
Q
What is triangulation?
A
Using both quantitative and qualitative methods together
15
Q
What are the 4 advantages of mixed methods?
A
- Can offset weaknesses of each design
- Holistic view of the issue being researched
- Helps to focus on context
- Helps to explain findings
16
Q
What are the 4 disadvantages of mixed methods?
A
- Design can be complex
- Takes more time
- How do you link data from these two traditions
- How do you resolve discrepancies in findings?
17
Q
What are the 6 key principles of research ethics?
A
- Research should aim to maximise benefit for individuals and society and minimise risk and harm
- Rights and dignity of individuals and groups should be respected
- Wherever possible, participation should be voluntary and appropriately informed
- Research should be conducted with integrity and transparency
- Lines of responsibility and accountability should be clearly defined
- Independence of research should be maintained and where conflicts of interest cannot be avoided they should be explicitly stated.
18
Q
What was the declaration of Helsinki (1964)?
A
A set of ethical principles regarding human experimentation
Patient has autonomy to make informed decisions regarding participation
The investigator’s primary duty is to the patient
While there is always a need for research, the participant’s welfare must always take precedence over the interests of science and society
19
Q
What 3 aspects of research must be considered for study participants?
A
Informed consent
Public understanding of research
Incentives and motivation
20
Q
What 3 aspects of research must be considered for study design?
A
Funding
Recruitment
Data management
21
Q
What 2 aspects of research must be considered for recruitment?
A
Vulnerability
Data protection
22
Q
What 3 aspects of research must be considered for safety/integrity?
A
Confidentiality
Safeguarding
Competence
23
Q
What was the Vipeholm Caries Study?
A
- Vipeholm Hospital was an institute for the “mentally deficient” with patients having conditions such as “idiocy” and “imbecility”.
- Aimed to determine the relationship between sugar intake and dental caries
- Some groups fed sweets between meals, other groups received extra carbs at mealtimes.
- Study performed under direction of the Swedish Medical Board and funded by the government and sugar industry.
- Study conducted without the consent of patients or their families.
- All groups showed a slow but definite increase in bodyweight
- The number of dental interventions increased over the course of the study with the biggest increases in those groups consuming additional carbohydrates between meals.
- These increases stopped when the diet reverted to the pre-study diet
24
Q
Andrew Wakefield and MMR
A
- The authors hypothesised that the administration of the MMR vaccine could precipitate chronic IBD that would lead to autism.
- The number of children receiving measles immunization went from a 95% to 70%
- Significant funds were used to test the hypothesis
- 2001, the scientific and medical community had concluded that there was no evidence to support the hypothesis
- 2004, Sunday Times reported a significant conflict of interest: Dr Wakefield, had received money as part of legal action against the company that manufactured the MMR vaccine
25
15 points of the UK Policy Framework for Health and Social Care Research
1: Safety over interest of science
2: Competent and qualified to host
3: Scientific and Ethical Conduct
4: Patient, Service User and Public Involvement
5: Integrity, Quality and Transparency
6: Protocol
7: Legality
8: Benefits and Risks
9: Approval
10: Information publicaly available
11: Accessible Findings
12: Choice
13: Insurance and Indemnity
14: Respect for Privacy
15: Compliance
26
What are the 12 stages in managing and planning a research project?
- Develop the research question
- Identify aims and objectives
- Develop a protocol: consider method, staff, time, resources, sample size
- Seek funding
- Obtain ethics approval
- Design and pilot data collection methods
- Collect data
- Process data e.g. clean, cross check, account for missing data; enter/transcribe and code
- Analysis of results
- Test hypotheses / develop theory / reflect against what is already known and context
- Write report or publication
- Feed back to participants and other stakeholders
27
How to develop a focused research question?
Literature review either scoping or systematic, find a gap in the research
What methodology has been used before?
Available time and resources?
Sample size – might need a statistician?
How wide will the interest be in results? Will the results be of benefit?
Ask stakeholders: patients, public, health professionals, commissioners etc
28
Public and patient involvement (PPI)
Research that is done ‘with’ the public, not 'about' or 'for' them.
It means that patients or other people with relevant experience contribute to how research is designed, conducted and disseminated.
29
What are the 5 ways that patient/public involvement improves research studies?
- More relevant to participants
- Designed in a way which is acceptable to participants
- Have participant information which is understandable to participants
- Provide a better experience of research
- Have better communication of results to participants at the end of the study.
30
What are aims in research?
Broad goals
Not usually measurable
Usually long-term
31
What are objectives in research?
The steps to achieve the aim
SMART goals
32
What is a research protocol?
- The plan for conducting the project
- Convinces the reader of the need for project
- Translates the aims and objectives into detailed plans that are justified by reference to the literature
- Ensures everything is considered before starting data collection
- Stops researchers manipulating methods to get the results they want
33
What are feasibility studies?
- Research done before a main study to see feasability
- They are used to estimate important parameters needed to design the main study.
- A less expensive feasibility study can avoid an expensive full study that turns out to be not feasible
34
What are pilot studies?
- Miniature version of the main study to test whether all the planned steps work together
- An external pilot is where the data from the pilot is set aside once completed and not used in the findings
- An internal pilot is where the data is used to contribute towards the findings
- Can be as simple as a pilot interview or involve recruiting a much larger number of participants
35
What are qualitative methods good for?
- Understanding ‘Lay’ or Clinical Behaviour
- Patients’ perceptions of quality or appropriateness
- Organisational culture / managing change
- Evaluating complex policy initiatives
- ‘Taken for Granted’ Phenomena
36
What is observational research?
- Data collected as events occur at study site
- Observe & record details of events:
- Observation of pharmacists and counter assistants providing OTC medicines advice: Do they use WWHAM questions? How do they handle situations where customer does not engage?
- Systematic detailed observation of talk and behaviour in its natural environment
- Explores more than what people say
37
What are the types of observational research?
- Participant: Immersion of researcher in a social setting, researcher lives/participates as member of group under study
- Non-participant observation: researcher not participating, discrete: no interference with ‘normal activity’, first hand: not reliant on individual report of behaviour
Hawthorne effect: validity affected by observer presence (conscious or inadvertent)
38
What are the 3 types of data collected in observational studies?
- Structured: observe to a data collection form (sometimes called a schedule).
- Semi-structured: prompts to collect data on pre-specified concepts, with space of open-ended notes
- Unstructured: record behaviour of participants & develop narrative accounts
39
What are the 3 limitations of observational studies?
- Behavioural changes if people know they are being observed (observer or Hawthorne effect)
Observer bias
- you may see what you want to see
- you become tired/bored
- Assimilating into the group (adopting the norms of the group)
40
What are the 2 types of interviews?
- Structured (quantitative questionnaire), researcher-led, standardised, minimal variation, keeping to question order, closed questions/pre-coded/short answers
- Semi-structured or unstructured, data about attitudes, behaviour, views, participant-led, open questions first, then focused if needed/lengthy answers
41
How to ask good questions for a qualitative interview?
Mainly open questions
Closed questions only to clarify details
Neutral, non-leading questions
Short and clear questions
Use the participant’s terminology
42
What are the 5 types of probing question?
Explanatory (Why?)
Clarification (What do you mean?)
Impact (What effect?)
Scoping (Anything else?)
Challenging (Contradictions)
43
What is the impact of phone interviews?
- Little evidence that face-to-face interviews are better
- May be shorter – may need more focused questions
- Practical – cheaper and quicker
- May help overcome reticence
- Anonymous
- Lack of non-verbal cues – cannot see distress/anxiety
44
How do you conduct a good interview?
- Have clear objectives
- Comprehensive & familiar topic guide
- Good questioning
- Pace the interview – cover everything
- Avoid interrupting
- Active listening
- Comfortable
- Suitable location
45
What are focus groups?
Enables people to explore and clarify their views in ways that would be harder in a one-to-one interview
It stimulates wide-ranging discussion, generates comprehensive list of issues
46
What are the typical size of focus groups?
Typically between 6 to 8 participants
Over-invite – some people may not turn up
Possibly try a smaller group if participants may have a lot to say on the topic OR if controversial/complex topic
47
Who runs focus groups?
- Conducted by a facilitator responsible for running of session & steering discussion, achieving participation from all group members
reminding participants to respect each other, value contributions
- Second researcher (co-facilitator), does not take part in discussion, takes notes & assists with smooth running and admin of session, monitors who contributing
48
How you prepare for focus groups?
- Venue, catering, resources, recruitment
- Information to participants
- About the study
- About ethics/consent
- How to find the venue
- Reimbursement/inducements
- What happens after being involved
49
Conducting a focus group
- Introduce research team (facilitator & co-facilitator)
- Thanks for coming & taking part
- Goals of research
- Reasons for recording
- Format & timing (1 to 2 hrs), refreshments available
- Ground Rules: respect, one person speak at a time etc
- Confidentiality & anonymity (consent)
- Hand out demographic info forms
- Participants to introduce themselves: name tags
50
Focus group topic guide
- Start recorder & launch into the topic guide
- Encourage participants to clarify their view
- Explore why people think what they do
- Explore differences between members
- Draw to an end
- Thanks for taking part
- What will happen to the recording & findings
51
Level of facilitator involvement in a focus group
- Not too intrusive, not too structured, allow respondents to steer but keep focus
- Respond to points raised that are particularly relevant to research question, especially if not picked up on by other participants
Intervene if group struggling, if talk heats up, remind people to talk one at a time, do not allow people to dominate, help quieter ones
52
Ethical considerations in focus groups
- People may get upset or angry (more critical than in interviews?)
- People may want to stop and/or leave
- Cannot guarantee confidentiality
- People may talk afterwards even if you tell them not to
53
Strengths of focus groups
- Can highlight cultural values or group norms
- Does not discriminate against literacy levels
- Participation from those reluctant to be interviewed 1:1
- Can encourage contributions from those who think they have nothing to say
54
Weaknesses of focus groups
- Reticent speakers may not offer views
- Reluctance to express opposing or sensitive views, fear of judgement
- Dominant speakers can narrow and direct content
- Less opportunity to explore individual views
- Risk of poor turn out
- More complicated & expensive to organise
- Difficult to analyse
55
Sampling in qualitative research
- A form of non-random sampling – participants are deliberately selected
- In quantitative studies we want our subjects to be representative of the wider population
- In qualitative studies we want to map the range and diversity of the population with respect to answering the research question
56
Qualitative sample sizes
- Usually small – typically:
< 50 interviews
6-12 focus groups
Often, sample until you reach saturation (no new information or themes are observed in new interviews/focus groups)
57
Self-administered questionnaire
- A collection of questions administered and answered by respondents without the aid of an interviewer
- Postal questionnaire
- Researcher handing out questionnaires to a class and collecting completed questionnaires at the end of the lecture
- E-mail/web surveys
58
Structured interview
A research interview in which all respondents are asked exactly the same questions in the same order with the aid of a formal schedule.
59
Ethical considerations of questionnaires
- Anonymity
An individual is not identifiable from the data, can this be guaranteed
- Confidentiality
Data treated as confidential, no information published will identify someone without their permission
- Informed Consent (consent to participate can be implied)
60
3 pros of open questions
- Elicit a wide variety of responses
- Good for exploring
- Does not superimpose answers / expectations
61
3 cons of open questions
- Difficult to summarise/analyse (qualitative)
- Response must be reported accurately
- Unpopular if used a lot in a postal questionnaire
62
4 pros of closed questions
- Easier to analyse
- Good if questionnaire is long
- Better if motivation is low
- Quick and easy to answer
63
3 cons of closed questions
- Creates false options/bias if all options are not included
- Good design vital
- Loss of spontaneity
64
What are Likert scales?
Attitudinal
Mid point ‘neither agree nor disagree’ allows respondents to be non committal
Norm is 5 point scale
65
Questionnaire layout
Aim of the study – clear instructions
Confidential / anonymous
Sections – helps with filter questions
Number all questions
Demographic information at the end
Make sure code boxes are big enough for completion
Avoid small type face
Limited number of type faces
Don’t split questions across a page
Boxes or lines for open questions
66
Piloting
How long will it take to complete?
Instructions clear?
Questions ambiguous?
Questions objectionable?
Layout clear and easy to follow?
Any topics omitted?
67
What is a p-value?
Probability
p=0.5 = even chance
Tests whether your observations are consistent with your hypotheses
68
What does p<0.05 mean?
It is the probability of being wrong when asserting that a true difference exists
If we find that X is different from Y, a p< 0.05 indicates that there is still a 1 in 20 chance that there is no difference between X and Y
69
Non-significant Findings
What if p>0.05? Can we say there is no difference between X and Y? It could be:
1) There really was no difference
2) There was a difference but the sample was too small to detect it.
70
Why is p<0.05 ‘significant’?
Convention
Arbitrary
Ties in nicely with the 95% of data within a bell-shaped curve
71
Sample Size Calculation
Can be complex depending on the variation that needs to be detected and whether groups are of equal size
72
Central Limit Theorem
The central limit theorem says that the sampling distribution of the mean will always be normally distributed, as long as the sample size is large enough
73
Sampling
In research one is often unable to measure the attributes of a whole population on grounds of practicality or cost, therefore a sample of the whole population is chosen for an experiment or survey.
Avoid bias
Achieve maximum precision
74
Sampling Frame
A list of all members of a population. Used as the basis from which to select a sample
75
Representative Sample
Reflects the population.
If a sample is representative, we can make generalisations about it to the population.
With a random sample, estimates produced by the sample will be close to the true population figure.
Object is to avoid bias in the sample - where the sample is different on certain parameters from the population from which it is drawn.
76
Sampling Error
The chance difference between the sample and the population
If members of a sample are drawn randomly:
Chance differences will be small
Sample will closely resemble the population.
77
Two types of sampling methods
Probability sampling methods
each member of a population has an equal chance of being selected
Non-probability sampling methods
each member of the population does not have an equal chance of being selected.
78
4 Probability Sampling Methods
Simple random sampling
Systematic sampling
Use of sampling interval e.g. every 10th subject
Stratified random sampling
Structure population into known sub-sets (eg. male and female) and random sample from each group
Cluster sampling
Divide population into clusters (e.g. 1st 100, 2nd 100 etc.) and sub-sample from each cluster
79
Non-probability Sampling
Incidental sampling, e.g. Pharmacy customers.
Quota sampling, e.g. Opinion polls
Quotas set on gender, age, socio-economic group
Purposeful sampling
Snowball sampling
Final choice of sampling method will depend on cost, required accuracy, the nature of the research and what is possible.
80
Summarising data
Central Tendency measures try to pinpoint the average or typical response to a question:
- mode
- median
- mean
Measures of Dispersion refer to how closely the data cluster around the measure of central tendency:
- the variation ratio
- the inter-quartile range
- the standard deviation
81
Levels of Measurement
Categorical (Nominal)
Ordinal (Ranked)
Interval / ratio
82
Categorical (Nominal) Data
Based on being a member (or not) of a category
All measurements within group are equivalent
Use of mode to describe central tendency
83
Ordinal Data
Based on ranks or order
Median describes average
Quartiles describe measure of spread
The measure of dispersion is the inter-quartile range
84
Interval Data
- Where the properties of an ordinal scale are shown and the differences between measurements are meaningful and quantifiable
- The measure of central tendency is the mean.
- The mean is the sum of all observations divided by the number of observations
- Measure of dispersion is the standard deviation.
- The Standard Deviation is the square root of the average of the squared deviations of each number from the mean of all the numbers.
85
Ratio Data
Has zero origin, e.g.
height
exam percentages
For most purposes interval and ratio scales of measurement can be regarded as equivalent.
86
3 types of Statistics
Descriptive Statistics
Measures of Association
Inferential Statistics
87
Descriptive Statistics
Descriptive Statistics - are concerned with the presentation, organization and summarization of data
88
Inferential Statistics
Allow us to generalise from our sample of data to a larger group of subjects:
Opinion poll based on a sample of 2000 predicts Labour 35%, Con 45%, LD 10%, others 10%
Used to test and examine relationships between data parameters
89
Standard Error of the Mean
- While SD reflects how close individual scores cluster around their mean, SE shows how close mean scores from repeated samples will be to the true population mean
90
Why use Standard Error?
Useful in calculating confidence intervals
Useful in inferring characteristics for the whole population
SD tells us that 95% of results are within two SDs of the mean - SE tells us that in 95% of cases the true population mean is within approx 2 SEs of the sample mean
91
Confidence Intervals
95% CI = Mean ± 1.96 * SEM
Three elements:
Mean
Standard Error of the Mean
Z-score (1.96)
If you want a 95% confidence interval z=1.96
The areas + 1.96 standard deviations of the mean on a normal curve cover 95% of the cases
Z scores will be greater for small samples <30
92
Calculating a Confidence Interval
If, in a study of 400 untreated hypertensives, we found that mean systolic BP was 162mmHg, SD 16mmHg, the true population mean for all untreated hypertensives can be found.
If we use a confidence level = 0.05, z=1.96:
then the true population mean is:
i.e. we can be 95% sure that the true population value is within these limits
93
What is PICO?
Patient/population
Intervention
Comparison
Outcome
MPharm3 SL32114
flashcards
Decks in class (4)
# Cards
