Chapter 14: Evaluating the Program

Question 1

What are some criteria to consider in evaluating tactics?

Answer 1

• time and resources
• ability to reach and influence target audience to achieve desired objectives
• compliance with ethical and legal guidelines
• ROI
• multiple methods, multiple touch points to reinforce consistent messages

Question 2

Why is it important to understand the difference between outputs and outcomes when referring to evaluating a program?

Answer 2

Outputs are the effort and process, while outcomes are the impact and effects. Outcomes cannot be assumed simply because information is disseminated, therefore the number of news releases (outputs) or the number of attendees at events are measurements of success.

Question 3

What does SMART stand for and why is it important?

Answer 3

S - specific
M - measurable
A - achievable
R - realistic / relevant
T - timebound

Broad and imprecise program objectives are obstacles to proper evaluation, therefore, objectives should be SMART.

Question 4

Research should be conducted for planning, managing and evaluating programs. What is “formulative research” and how does it relate to the evaluation process?

Answer 4

Formulative research is research done before and during implementation to informing planning and program adjustment. It creates a benchmark for the evaluation process.

Question 5

What is “summative research?”

Answer 5

Summative research is research conducted after the program to assess progress and to document program impact. aka evaluation research or measurement.

Question 6

What are the three basic questions posed in evaluation? Remember, it should be used to learn what happened and why, not to prove or justified decisions already made.

Answer 6

1. Program design - what’s the extent and distribution of the target population; is there a coherent rationale; what are the costs
2. Monitoring and accountability of implementation - is it reaching targeted publics; are efforts conducted as specified
3. Assessment of impact and efficiency - was it effective; are there unintended effects; what are the costs to deliver services; was it an efficient use of resources

Question 7

There is no single right way to gather data for evaluations. What are the three things to consider when determining the best way to gather evidence?

Answer 7

1. the question and purposes motivating the evaluation
2. the outcome criteria specified in the objectives
3. the cost of different research approaches

Question 8

What are the three stages or levels of evaluation? (Note: no evaluation is complete without addressing criteria at all levels.)

Answer 8

1. Preparation /inputs - assesses quality and adequacy of the info used to develop the program strategy and tactics
2. Implementation evaluation - monitors efforts and progress as the program unfolds (placement, distribution, # of people who attended)
3. Impact evaluation - documents the consequences of the program and provides feedback (social/cultural change, behavior change, attitude change, opinion change, knowledge gain)