Research & Program Evaluation

Question 1

A researcher concludes an intervention is effective when p = .20. Which conclusion is MOST accurate?

• A. The null hypothesis should be rejected
• B. The results are not statistically significant
• C. A Type I error has occurred

Answer 1

B. The results are not statistically significant

A p-value of .20 indicates that the results are not statistically significant, as it is above the common threshold of .05.

Question 2

A study examines the effect of group therapy on reducing social anxiety. The dependent variable is:

• A. Participation in group therapy
• B. Level of social anxiety
• C. Number of group sessions

Answer 2

B. Level of social anxiety

The dependent variable is the outcome that is measured in a study.

Question 3

A measure that correlates strongly with other established tools measuring the same concept demonstrates:

• A. Convergent validity
• B. Divergent validity
• C. Content validity

Answer 3

A. Convergent validity

Convergent validity is demonstrated when a measure correlates well with other measures of the same construct.

Question 4

A researcher designs a study to test whether a new intervention reduces test anxiety. Which null hypothesis (H₀) is most appropriate?

• A. The intervention reduces test anxiety
• B. The intervention has no effect on test anxiety
• C. The intervention increases test anxiety

Answer 4

B. The intervention has no effect on test anxiety

The null hypothesis typically states that there is no effect or difference.

Question 5

A counselor notices that when the SAT is administered to students in urban schools, their scores tend to underestimate college GPA compared to suburban students. This suggests a problem with:

• A. Test-retest reliability
• B. Criterion-related validity
• C. Standard error of measurement

Answer 5

B. Criterion-related validity

Criterion-related validity refers to how well one measure predicts an outcome based on another measure.

Question 6

A counselor concludes a new intervention reduces depression when, in truth, it does not. This represents:

• A. Type I error
• B. Type II error
• C. Internal validity threat

Answer 6

A. Type I error

A Type I error occurs when a true null hypothesis is incorrectly rejected.

Question 7

When administering the Strong Interest Inventory, a counselor explains that the test compares the client’s interests to those of satisfied workers. This reflects which type of validity?

• A. Construct
• B. Criterion-related
• C. Face

Answer 7

B. Criterion-related

Criterion-related validity involves comparing test results to an external criterion.

Question 8

Which reliability measure would best assess consistency among different raters?

• A. Test-retest reliability
• B. Inter-rater reliability
• C. Split-half reliability

Answer 8

B. Inter-rater reliability

Inter-rater reliability measures the level of agreement between different raters.

Question 9

A standardized test that accurately measures future performance in graduate school is demonstrating:

• A. Predictive validity
• B. Concurrent validity
• C. Content validity

Answer 9

A. Predictive validity

Predictive validity refers to the extent to which a test predicts future performance.

Question 10

A program evaluation study tracks dropout rates, attendance, and client satisfaction while the program is ongoing. This is an example of:

• A. Formative evaluation
• B. Summative evaluation
• C. Experimental validity

Answer 10

A. Formative evaluation

Formative evaluation focuses on monitoring and improving a program during its implementation.

Question 11

A research article reports results as “p < .01.” What does this indicate?

• A. The results are highly likely due to chance
• B. The probability of error is less than 1%
• C. The findings cannot be replicated

Answer 11

B. The probability of error is less than 1%

A p-value less than .01 indicates a very low probability that the results are due to chance.

Question 12

In program evaluation, formative evaluation is BEST described as:

• A. An assessment during program development to improve effectiveness
• B. An assessment at the end of a program to judge overall success
• C. A statistical comparison between two interventions

Answer 12

A. An assessment during program development to improve effectiveness

Formative evaluation focuses on improving a program while it is being developed.

Question 13

A researcher asks participants to rank their satisfaction with a counseling program from “1 = least satisfied” to “5 = most satisfied.” This is an example of:

• A. Nominal data
• B. Ordinal data
• C. Ratio data

Answer 13

B. Ordinal data

Ordinal data involves order or ranking but not equal intervals between ranks.

Question 14

A counselor conducts a longitudinal study tracking substance use across 15 years. Which threat to validity is most concerning?

• A. Testing effect
• B. Attrition
• C. Instrumentation

Answer 14

B. Attrition

Attrition refers to participants dropping out of a study over time, which can threaten the study’s validity.

Question 15

A counselor administers a test that appears to measure self-confidence simply because it looks like it should. This reflects:

• A. Face validity
• B. Content validity
• C. Criterion validity

Answer 15

A. Face validity

Face validity refers to the extent to which a test appears to measure what it is supposed to measure.

Question 16

A researcher is testing the effectiveness of a new therapy technique. The null hypothesis (H₀) would state:

• A. The therapy has no effect on outcomes
• B. The therapy improves outcomes
• C. The therapy decreases outcomes

Answer 16

A. The therapy has no effect on outcomes

The null hypothesis typically states that there is no effect or difference.

Question 17

A test shows consistent results when administered multiple times, but it fails to measure what it claims. This instrument demonstrates:

• A. High reliability, low validity
• B. Low reliability, high validity
• C. Both high reliability and validity

Answer 17

A. High reliability, low validity

High reliability indicates consistency, while low validity indicates the test does not measure what it claims.

Question 18

A counselor concludes a new substance use program has no effect, when in fact it reduces relapse rates. This is an example of:

• A. Type I error
• B. Type II error
• C. Attrition

Answer 18

B. Type II error

A Type II error occurs when a false null hypothesis is not rejected.

Question 19

A counselor develops a new depression scale and gives it to a sample of 500 clients. He divides the test into two halves and compares results. This method assesses:

• A. Internal consistency
• B. Predictive validity
• C. Test-retest reliability

Answer 19

A. Internal consistency

Internal consistency measures how well the items on a test measure the same construct.

Question 20

In a normal curve, approximately what percentage of scores fall within one standard deviation of the mean?

• A. 34%
• B. 68%
• C. 95%

Answer 20

B. 68%

In a normal distribution, about 68% of data falls within one standard deviation of the mean.

Question 21

A researcher reports a correlation of r = 0.00 between two variables. What does this indicate?

• A. No linear relationship between the variables
• B. A weak but positive relationship
• C. A negative curvilinear relationship

Answer 21

A. No linear relationship between the variables

A correlation of 0 indicates no linear relationship.

Question 22

A researcher designs an experiment but does not randomly assign participants. Which type of design is this?

• A. Quasi-experimental
• B. True experimental
• C. Correlational

Answer 22

A. Quasi-experimental

Quasi-experimental designs lack random assignment, which can affect internal validity.

Question 23

A test designed to predict future academic performance is best evaluated by:

• A. Content validity
• B. Predictive validity
• C. Concurrent validity

Answer 23

B. Predictive validity

Predictive validity assesses how well a test predicts future outcomes.

Question 24

A client’s test score has a SEM of 4. If she scored 96, what is the 68% confidence interval?

• A. 92–100
• B. 88–104
• C. 94–98

Answer 24

A. 92–100

The 68% confidence interval is calculated as the score plus and minus one SEM.

Question 25

A counselor administers a standardized test to a group of clients. He notes that one client consistently scores much lower than expected due to language barriers. **This is an example of:** * A. Construct validity * B. Cultural bias * C. Internal consistency

Answer 25

B. Cultural bias ## Footnote Cultural bias occurs when a test unfairly disadvantages individuals from certain cultural backgrounds.

Question 26

In a correlational study, a researcher finds **r = -0.85** between self-esteem and social anxiety. How should this be interpreted? * A. High self-esteem is strongly associated with low social anxiety * B. The relationship is weak and insignificant * C. Low self-esteem causes social anxiety

Answer 26

A. High self-esteem is strongly associated with low social anxiety ## Footnote A strong negative correlation indicates that as one variable increases, the other decreases.

Question 27

A study reports **p < .05.** Which is the most accurate interpretation? * A. There is less than a 5% chance the results are due to random variation * B. The intervention is guaranteed to be effective * C. Type II error has been avoided

Answer 27

A. There is less than a 5% chance the results are due to random variation ## Footnote A p-value less than .05 indicates statistical significance, suggesting the results are unlikely due to chance.

Question 28

A counselor finds that clients tend to score higher on a personality test the second time they take it due to familiarity. **This illustrates:** * A. Maturation effect * B. Testing effect * C. History effect

Answer 28

B. Testing effect ## Footnote The testing effect occurs when familiarity with a test improves performance on subsequent administrations.

Question 29

A counselor is evaluating the effectiveness of a new stress-reduction program. She assigns students to the intervention group alphabetically by last name. **What is the primary limitation of this design?** * A. Selection bias * B. Instrumentation error * C. History effect

Answer 29

A. Selection bias ## Footnote Assigning participants alphabetically can introduce selection bias, affecting the study's validity.

Question 30

In a normal curve, approximately 95% of scores fall within how many **standard deviations of the mean?** * A. 1 * B. 2 * C. 3

Answer 30

B. 2 ## Footnote In a normal distribution, about 95% of data falls within two standard deviations of the mean.

Question 31

A SMART goal is written as follows: “Client will identify three previous situations in which they successfully managed anxiety and describe these in session over the next three weeks.” **Which of the following best reflects this treatment approach?** * A. Focusing on past trauma to create insight * B. Exploring maladaptive cognitive schemas * C. Highlighting client strengths and past exceptions * D. Engaging in somatic processing of emotion

Answer 31

C. Highlighting client strengths and past exceptions ## Footnote SFBT focuses on exceptions, strengths, and what is already working.

Question 32

A counselor divides test items into two halves and compares the results to measure consistency. **This is assessing:** * A. Split-half reliability * B. Inter-rater reliability * C. Test-retest reliability

Answer 32

A. Split-half reliability ## Footnote Split-half reliability measures internal consistency by comparing two halves of a test.

Question 33

Which is the main purpose of a **summative evaluation?** * A. To monitor progress and make adjustments during implementation * B. To assess long-term impact and outcomes after program completion * C. To ensure randomization in experimental studies

Answer 33

B. To assess long-term impact and outcomes after program completion ## Footnote Summative evaluation assesses the overall success of a program after its completion.

Question 34

A counselor is categorizing clients by religious affiliation (Christian, Muslim, Jewish, Hindu, etc.). This represents which **scale of measurement?** * A. Nominal * B. Ordinal * C. Interval

Answer 34

A. Nominal ## Footnote Nominal scales categorize data without any order or ranking.

Question 35

A school counselor develops a survey where scores can range from 0 to 100. **The scores represent:** * A. Ordinal data * B. Interval data * C. Ratio data

Answer 35

B. Interval data ## Footnote Interval data have equal intervals between values but no true zero.

Question 36

A counselor develops a scale to measure motivation to change in addiction recovery. Scores range from 1–10, and differences between points are equal, but there is no true zero. **This scale is:** * A. Interval * B. Ratio * C. Ordinal

Answer 36

A. Interval ## Footnote Interval scales have equal intervals between values but lack a true zero point.

Question 37

A counselor reviewing test results notes that a client’s confidence interval for an IQ score ranges from 95–105. **This suggests:** * A. The client’s true score likely falls within this range * B. The client’s exact IQ is 100 * C. The test lacks construct validity

Answer 37

A. The client’s true score likely falls within this range ## Footnote A confidence interval provides a range in which the true score is likely to fall.

Question 38

A researcher notices that as self-efficacy scores rise, academic achievement also rises. **This relationship is:** * A. Positive correlation * B. Negative correlation * C. No correlation

Answer 38

A. Positive correlation ## Footnote A positive correlation indicates that as one variable increases, the other also increases.

Question 39

A counselor calculates a client’s IQ score of 130. Which **percentile rank** does this MOST closely approximate? * A. 84th * B. 98th * C. 50th

Answer 39

B. 98th ## Footnote An IQ score of 130 typically falls within the 98th percentile, indicating superior performance.

Question 40

A researcher randomly assigns participants to treatment and control groups. **This design strengthens which aspect of the study?** * A. External validity * B. Internal validity * C. Construct validity

Answer 40

B. Internal validity ## Footnote Random assignment helps ensure that differences between groups are due to the treatment, enhancing internal validity.

Question 41

A test consistently yields the same scores across time, but does not measure what it intends to measure. **This test has:** * A. High reliability, low validity * B. Low reliability, high validity * C. High reliability and high validity

Answer 41

A. High reliability, low validity ## Footnote Reliability refers to consistency, while validity refers to accuracy.