Chapter 5: Validity

Question 1

1. Define and be able to apply the broad definition of validity.

Answer 1

Does your test measure what it reports to measure?

Question 2

2. What are the three main types of validity evidence?

Answer 2

Construct, criterion, and content

Question 3

3. What prerequisites exist for validity?

Answer 3

Reliability, variability, correlation

Question 4

4. Define Face Validity. How does it differ from other aspects of validity?

Answer 4

• Does the test have the appearance of measuring a specific construct.
• Not true validity and you could want high or low face validity based on what you’re testing

Question 5

5. Define Content Validity: How is it measured?

Answer 5

• Does the test represent a comprehensive look at the construct?
• Measure through CVRs and factor analysis

Question 6

6. How do construct underrepresentation and construct-irrelevant variance relate to content validity?

Answer 6

• underrepresentation: only getting at part of the construct
• construct-irrelevant variance: other factors relate to non-item pieces that effect correct answers (length of test, where items are in the test)

Question 7

7. What is the content validity ratio and how is it calculated?

Answer 7

• Number of items rated as essential, creates content validity
• (Essential questions - N/2)/N/2

Question 8

8. What is a criterion? What is Criterion-related Validity?

Answer 8

• criterion: the standard a test is compared against
• criterion-related validity: How well does your test relate to some outside criterion?

Question 9

9. Name and define the three subtypes of criterion related validity. Be able to give examples.

Answer 9

• Predictive: accuracy in using a test score to predict a criterion obtained later (SAT predicting college GPA)
• Concurrent: how related a test score is to the criterion as measured at the same time (job performance)
• Postdictive: accuracy of a test score in predicting a previously acquired criterion (ASPD scale in predicting criminal behavior)

Question 10

10. What is the validity coefficient? What is the meaning of a squared validity coefficient?

Answer 10

• validity coefficient: The correlation between a tests and a criterion
• validity coefficient squared: Percent of variation in criterion we can know because of knowing the test score/ coefficient of determination/R-squared

Question 11

11. What is incremental validity?

Answer 11

How much does your measure add to the literature (new info on the client, construct, price. etc.)

Question 12

12. What is a construct? What is Construct-related Validity?

Answer 12

an idea/phenomenon a test is measuring

Construct-related validity: how well does your measure for that construct

Question 13

13. What are the two types of evidence in construct validity?

Answer 13

Convergent and discriminant validity

Question 14

14. How are the two types of evidence for construct-validity defined and how are they different from one another?

Answer 14

• Convergent: measures of the same construct converge or narrow in on the same thing, shows that a test measures the same thing as other tests
• Discriminant: low correlations with measure of unrelated constructs or evidence for what the test does not measure

Question 15

15. What is the relationship between reliability and validity?

Answer 15

A test must be reliable to be valid, but reliability does not equate to validity

Question 16

16. Which two types of validity are logistical and not statistical? Why?

Answer 16

Content validity and face validity

Neither depend on statistical formulas

Question 17

17. Which type of validity has been referred to as “the mother of all validities”, or “the big daddy” and why?

Answer 17

Construct validity because it borrows from the actual definition of validity

Question 18

18. Be able to identify and/or give examples of the different types of validity.

Answer 18

• Content: A license exam
• Criterion: SAT, ACT, Class Exams
• Construct: does the test measure what it should