1C Evaluation

Question 1

What are the different types of need?

Answer 1

Normative - deemed by clinician

Felt - a need to feel better

Expressed - demand for health

Comparative - one area compared to another

Question 2

What is a health needs assessment?

Answer 2

A systematic assessment of the health needs of a given population

Question 3

What is a joint strategic needs assessment?

Answer 3

• Statutory requirement for service commissioners and local authorities to produce these
• Analysis of the health and wellbeing and social care needs of the local population

Question 4

What are the different types of health needs assessment?

Answer 4

Epidemiological: This approach considers the epidemiology of the condition, current service provision, and the effectiveness and cost-effectiveness of interventions and services.

Comparative: This approach compares service provision between different populations. Large variations in service use may be influenced by a number of factors, and not just differing needs.

Corporate: This approach is based on eliciting the views of stakeholders - which may include professionals, patients and service-users, the public and politicians - on what services are needed. Elements of the corporate approach (i.e. community engagement and user involvement) are important in informing local policy.

Participatory: Community involved at every stage, including action plan development. Highlights voices of marginalised groups. May produce qualitative findings which can shed light on quantitative data.

Question 5

What data is typically required as part of a ‘minimum dataset’ for health service use?

Answer 5

• Patient characteristics
• Clinical info: diagnosis, procedures, prescriptions
• Pathway/service use: referred by, referred to, referral date, duration of stay, appointment/admission data
• Provider details: clinical team + lead clinician
• Specific/bespoke data: PROMs, friends and family test, research data

Question 6

What factors to consider in study design for health services research?

Answer 6

• Aim of the study
• Available resources
• New or existing service (for an existing service, unlikely to be able to identify a comparison group)
• Existing evidence base (if studies already exist, may just need to do a lit review)

Question 7

What are the limitations of using RCTs in health services research?

Answer 7

• Resources
• Time delay until effect of intervention seen
• Different delivery across sites may preclude multi-site studies
• Hard to randomise for organisational level changes e.g. restructuring a service

Question 8

What is Donabedian’s framework for assessing quality of care?

What are the advantages and disadvantages of the different domains?

Answer 8

Structure (inputs)
- A: Easy to obtain data
- D: Structural data may not be comparable across providers/systems

Processes (activities)
- A: Easy to obtain data, some processes e.g. immunisation may be related to outcomes
D: Processes do not necessarily predict outcomes

Outputs (products of activities)

Outcomes (changes in health status)
- A: Reflect the aim of health services
- D: May be affected by case mix rather than performance, may be long time delay until outcomes seen, difficult/expensive to measure, difficult to attribute outcome to the intervention

Question 9

What are Maxwell’s dimensions of quality?

Answer 9

Access
- Tangible barriers e.g. geography
- Intangible barriers e.g. language

Relevance
- Relevance to patients’ needs

Equity
- The fair distribution of healthcare amongst individuals or groups

Efficiency
- Relates the cost of healthcare to the outputs or benefits obtained

Effectiveness
- The benefits of healthcare measured by improvements in health

Acceptability/Humanity
- The social, psychological and ethical acceptability regarding the way people are treated in relation to healthcare

Question 10

What are examples of measures of health status, health-related quality of life and health care?

Answer 10

Health status
- Signs, symptoms
- Co-morbidities

Health-related quality of life
- Individual’s own perceptions
- No clear definition but as a minimum usually includes: physical health, mental health, social functioning, role functioning and general health perceptions
- Can be either generic (SF-36/EQ-5D) or disease specific (e.g. fibroids)
- NB differs from assessments of functional ability e.g. ADLs

Health care
- Donabedian & Maxwell

Question 11

What are population health outcome indicators?

Answer 11

These reflect the population level effect of healthcare activities and public health policies.

England’s Public Health Outcomes Framework has two keys aims:
- Improve and protect the nation’s health
- Reduce difference in life expectancy and healthy life expectancy between communities

Within this there are four domains
- Wider determinants: measure improvement against wider determinants e.g. school readiness
- Health improvement: supporting healthy choices and lifestyles e.g smoking prevalence at age 15
- Health protection: protection from major incidents e.g. MMR uptake
- Healthcare and premature mortality: reduce preventable health problems and premature mortality e.g. under 75 cancer mortality

https://fingertips.phe.org.uk/profile/public-health-outcomes-framework

Question 12

What are the characteristics of good population health indicators?

Answer 12

• Valid
• Collected consistently
• Available for relevant geographical levels
• Suitable for making comparisons e.g. can be linked to data on age/gender/ethnicity

Question 13

What are different measures of deprivation?

Answer 13

National Statistics Socio-economic Classification (NS-SEC)
- 8 categories based on employment with many subcategories

Index of Multiple Deprivation
- Includes the dimensions- employment, income, health and disability, education skills and training, barriers to housing and services, crime and disorder, living environment, plus overall score
- Issued at LSOA level

Jarman score
- Measure of GP workload
- Sometimes used as a proxy for deprivation
- Includes factors such as % people over 65 living alone, % people aged <5, unemployed > 16

Question 14

What are the features of good evaluation?

Answer 14

• Plan evaluation from before intervention is implemented
• Follow a theoretical model e.g. Donabedian
• Clear purpose
• Clear goals e.g. is it formative (feeding into improving the intervention) or summative (at the end of an intervention)
• Clear objectives and outcomes measures before starting e.g. SMART
• Have a control/comparison
• Robust processes for collecting data (so data is valid) (collection should not be burdensome for teams, processes should be agreed before start of intervention)
• Sufficient resources to conduct evaluation
• Flexible to adapt to changes in services
• Requires stakeholder engagement at all stages
• Follow ethical and legal principles

Question 15

What is horizontal vs vertical equity?

Answer 15

Horizontal = equal treatment for equal need

Vertical - unequal treatment for unequal need

Question 16

What are the different types of horizontal equity?

Answer 16

• Equal spending
• Equal access
• Equal usage
• Equal health

Question 17

Differences between evaluation and quality assessment/assurance?

Answer 17

Often used interchangeably in public health. In other setting quality assurance can mean ensuring a service meets specific standard.

Quality assurance is a proactive, ongoing process focused on ensuring standards are met throughout a project. Evaluation is a (sometimes) and used to judge the value, or effectiveness of an initiative.

Question 18

What is the clinical audit cycle?

Answer 18

1. Set standards
   - Clear criteria for what is being measured
   - Clear scope for which patients/how long/which aspects of care
2. Measure current practice
3. Compare to standards
   - Share finding with relevant teams, directors and boards
4. Reflect, plan and implement change
   - Consider change management approaches
5. Re-audit

Question 19

What is the process of a Delphi method?

Answer 19

1. Survey group of experts
2. Share findings highlighting areas of disagreement
3. Opportunity for experts to change their views in light of step 2
4. Repeat steps 2-3 until consensus is reached

Question 20

Advantages and disadvantages of Delphi methods?

Answer 20

Adv:
- Anonymity helps to ensure contributions are valued equally and less potential for influences of personality
- Time-efficient as the experts don’t all have to meet
- Encourages open critique and admission of errors

Disadvantages:
- Written survey format may not work for everyone
- Open to manipulation from administrators

Question 21

What are some ways of assessing acceptability/appropriateness/adequacy of services?

Answer 21

• Friends and family test
• CQC patient surveys
• PROMs - some mandatory e.g. knee replacement, hip replacement, groin hernia repair, varicose vein repair (Oxford Hip Score, Oxford Knee Score, groin hernia repair, Aberdeen Varicose Vein Questionnaire, EQ5D can be used generically)

Question 22

What is the process for undertaking a health impact assessment?

Answer 22

SSARM

2. 1. Screening - does the policy have health relevance?
3. Scoping
   - Questions HIA seeks to answer
   - Reporting arrangement
   - Determinants to be considered at individual/environmental/institutional level or all three
4. Appraisal
   - Either rapid or in depth
   - Based on info from: secondary data, interviews, field observation, peer reviewed and grey literature
5. Reporting
   - Write up conclusion
   - Provide recommendations to reduce harms and maximise benefits
6. Monitor actual impacts and implementation of recommendations

Question 23

Potential challenges in undertaking a HIA?

Answer 23

• Lack of time
• Lack of resources
• Lack of evidence: quant often more influential than qual; conflicting perspectives
• Actually implementing recommendations - requires decision-makers to be engaged in HIA