PSYC*2360 Chapter 13: Program Evaluation

Question 1

What does a program evaluation involve?

Answer 1

Involves using the scientific method to assess whether an organized activity is achieving its intended objectives

Question 2

What are grants?

Answer 2

Monetary gifts to help organizations meet a specific goal or objective

Question 3

T or F: Program evaluations are helpful, but not often essential to the grant writing process.

Answer 3

False. They are often essential.

Question 4

What are three types of program evaluations?

Answer 4

• A needs evaluation
• A process evaluation
• An outcomes evaluation

Question 5

Which type of program evaluation involves an assessment of which features of a program are most valuable and who they benefit?

Answer 5

A needs evaluation

Question 6

What are the two main goals of a needs evaluation?

Answer 6

• Identify features to continue or discontinue
• Determine potential components to add

Question 7

Which type of program evaluation is particularly helpful for program planning?

Answer 7

A needs evaluation

Question 8

Which type of program evaluation involves an assessment of a general program operation, including whom the program serves and how it delivers services to that population?

Answer 8

A process evaluation

Question 9

What is the main goal of a process evaluation?

Answer 9

Determine ways to improve program implementation and delivery

Question 10

Which type of program evaluation seeks to ensure a match between program goals and those serviced?

Answer 10

A process evaluation

Question 11

Which type of program evaluation involves an assessment of whether a program effectively produces outcomes that are consistent with stated objectives or goals?

Answer 11

An outcome evaluation

Question 12

What are the two main goals of an outcome evaluation?

Answer 12

• Identify unmet goals and outcomes that can be improved
• Establish evidence of program effectiveness

Question 13

What are four potential benefits of program evaluations?

Answer 13

• Forces members to clarify the program’s purpose and intended outcomes
• Provides the organization with evidence of the program’s effectiveness and that the program is using resources efficiently
• Allows the organization to evaluate a new approach or resource
• Encourages organizations to identify areas for improvement

Question 14

What is the main goal of a program evaluation?

Answer 14

To provide actionable information that can have immediate implications to benefit a specific program

Question 15

T or F: In a program evaluation, there is often less control over extraneous variables that may bias findings.

Answer 15

True

Question 16

What is one way to avoid potential bias during a program evaluation?

Answer 16

Use an outside consultant to conduct the evaluation

Question 17

T or F: A program’s outcomes are often immediately obvious.

Answer 17

False

Question 18

T or F: Program evaluations seek to help improve a program, therefore no additional attention must be paid to potential consequences.

Answer 18

False

Question 19

T or F: If a program evaluation is too intrusive, researchers might undermine client’s trust or limit the program’s effectiveness.

Answer 19

True

Question 20

What are the three phases of a program evaluation?

Answer 20

• The planning phase
• The execution phase
• Communication of results

Question 21

What are the four steps involved in the planning phase of a program evaluation?

Answer 21

• Identify key stakeholders
• Describe the program
• Clarify the evaluation’s goals
• Create an evaluation plan

Question 22

What are the two steps involved in the execution phase of a program evaluation?

Answer 22

• Gather data
• Analyze data

Question 23

What are the three steps involved in the communication of results phase of a program evaluation?

Answer 23

• Form conclusions
• Make recommendations
• Report findings

Question 24

Which method of data collection involves assembling several individuals to discuss a specific topic?

Answer 24

Focus groups

Question 25

Ideally, how many participants should be included in a focus group?

Answer 25

6-12

Question 26

Ideally, should participants in focus groups know each other beforehand?

Answer 26

No

Question 27

Is the fact that in a focus group, participants have the opportunity to react to each other's contributions most often considered an advantage or a disadvantage?

Answer 27

An advantage because it provides richer information

Question 28

T or F: Focus groups provide a high volume of insightful content using participants' own words in a quick and efficient manner.

Answer 28

True

Question 29

In focus groups, are facilitators typically allowed to ask follow-up questions to clarify meaning?

Answer 29

Yes

Question 30

What is the most notable disadvantage of focus groups?

Answer 30

The inability to generalize to other populations

Question 31

What is the role of a moderator in a focus group?

Answer 31

To ask the questions and facilitate discussion

Question 32

What are the four characteristics of an ideal moderator?

Answer 32

- Has a thorough understanding of the focus group's topic - Has a strong foundation in research methods - Understands group dynamics (the behaviours and psychological processes of social groups) - Remains as neutral and bias-free as possible during discussions

Question 33

In a program evaluation, information from focus groups can be used to develop what additional method of data collection/measurement?

Answer 33

Surveys

Question 34

T or F: Although focus groups provide quantitative information about the program's effectiveness, surveys enable qualitative information to be gathered.

Answer 34

False. Focus groups provide QUALITATIVE data and surveys provide QUANTITATIVE data.

Question 35

What is a single-item indicator?

Answer 35

When only one item or question is being used to measure a variable

Question 36

What are two concerns when using single-item indicators?

Answer 36

- Poor reliability - The variable is only being measured in a general sense

Question 37

What is a single sample t-test used to evaluate?

Answer 37

Whether a sample mean statistically differs from a specific value

Question 38

T or F: Even though program evaluations aren't intended to expand generalizable knowledge, IRB approval is still always required.

Answer 38

False. IRB approval is often not required.

Question 39

Rather than having participants sign an informed consent form, it can be noted at the beginning of a survey that returning it implies consent to be part of a study. This is known as what?

Answer 39

Implied consent

Question 40

T or F: A program evaluation report should present findings in a clear and succinct way, avoid jargon, and be tailored to the audience.

Answer 40

True

Question 41

What are word clouds?

Answer 41

Visual representations of the frequency that certain words are used in qualitative assessments

Question 42

In a word cloud, what do larger words indicate?

Answer 42

Higher frequency of use

Question 43

What can be used to visually represent a study's findings that quickly conveys a lot of information in an engaging and easy-to-understand format?

Answer 43

Infographics

Question 44

The "conclusions and program recommendations" section of a program evaluation report is the equivalent to which section of an APA style report?

Answer 44

The "general discussion" section

Question 45

What are the three things that a "conclusion and program recommendations" section of a program evaluation report focuses on?

Answer 45

- How the outcome findings match with the program goals - What the findings indicate about the program's strengths and weaknesses - What steps the program should take for improvement

Question 46

When discussing findings that support a program, stakeholders often exhibit hindsight bias, therefore it is important to remind them of what?

Answer 46

The benefits of having data to support assumption

Question 47

When discussing findings of a program evaluation that disagree with prevailing opinions, what it is important to remind stakeholders of?

Answer 47

The natural flaws in thinking and the importance of data to support claims

Question 48

What type of research involves a systematic and planned sequence of related studies, where subsequent studies build directly from a previous study's findings?

Answer 48

Programmatic research