L5. Validity

Question 1

validity

Answer 1

• all validity is model off the concept of construct validity
• the degree to which a test measures what its supposed to measure
• only the interpretation and uses of a test scores can be valid
• is a continuum of strong to weak
• qualitative and quantitative analysis of validity

Question 2

contruct validity

Answer 2

• Construct validity refers to the degree to which test scores can be interpreted as reflecting a particular psychological construct.
• we have to use indirect measures

Question 3

Content validity
-> domain sampling
-> standard measure of estimate

face validity

Answer 3

1: Content validity
- A test may be suggested to be associated with good content validity when the items cover the entire breadth of the construct and do not exceed its boundaries
- tough for higher-order abstract concepts eg personality needs precise definitions to create boundaries
- domain sampling: an assortment of items that represent each domain of the construct are included in the test
- by experts
- psychometrically integral

the standard measure of estimate
- degree of correspondence between short-form and a long-form

2: Face validity
- Face validity represents the degree to which the items associated with a measure appear to be related to the construct of interest.
- susceptible to response bias
- by participants
- not psychometrically integral

Question 4

factorial validity
factor analysis

Answer 4

1: The number of dimensions measured by a test.
2: Whether the items of a test are related to the dimensions of interest.
3: Whether the dimensions of interest are related to each other.

• can be tested using various quantitative techniques.
• use factor analysis to evaluate the factorial validity of the scores derived from a test.
• There are two types of factor analysis:
  1. Unrestricted factor analysis (we only cover)
  2. Restricted factor analysis.

Question 5

response process

Answer 5

• should be a close match between the psychological processes that the respondents actually use when completing a measure and the process that they should use.
• response bias eg cheating or soc des

Question 6

convergent validity
consensual convergent

Answer 6

• It is the degree to which test scores are correlated with tests of similar constructs.
• Emotional intelligence should correlate positively with cognitive intelligence.
• when you get the corr between self report and rater report scores

Question 7

discriminant validity

Answer 7

• the degree to which test scores are uncorrelated with tests of unrelated constructs.
• In the emotional intelligence case, there should be very little in the way of a correlation between EI scores and a personality dimension such as morningness/ eveningness.
• Constructs should not correlate with everything under the sun.
• If they do, then the construct boundaries are overly expansive.
• correlation should be low 0.2
• should never be higher than 80 or as high

Question 8

concurrent validity

Answer 8

• when the scores from one measure correlate in a theoretically meaningful way with the scores of another measure considered to be the “gold standard”.
• Concurrent validity does not have to based on measures administered precisely at the same time.
• But the time period should be very close
• not impressive, minimum expectation

Question 9

predictive validity

Answer 9

• Refers to the degree to which test scores are correlated with relevant variables that are measured at a future point in time
• Predictive validity evidence is very impressive
• But it is also relatively rare, because of the time and resources required to keep track of people over time.
• eg grades w future earnings

Question 10

consequential validity

Answer 10

-it refers to the social/personal consequences associated with using a particular test.
- For example, if two tests were equally predictive of a criterion of interest, but one of the tests tended to yield scores that were biased against women, then we would consider the non-biased test to be associated with greater consequential validity.

Question 11

Criterion validity
(criterion groups validity)

Answer 11

• The observation of an association between a psychometric measure and a relevant outcome variable, such as different groups
• For example, different levels of achievement (undergraduate student, PhD student, and Professor) on taking the psych3302 exam
• concurrent and predictive validity together kind of
• oldest form of validity
• form of construct validity

Question 12

Induction-Construct Development Interplay

Answer 12

• measure is developed solely from an inductive perspective.
• eg measure of personality by including all of the “person-descriptive” adjectives in the dictionary. (e.g., gregarious, moody, unpredictable).
• People rate the degree to which all of the adjectives describe them.
• Then the researcher would factor analyse all of the responses to help uncover the common dimensions.
• This is how the Big Five personality model was discovered.

Question 13

Measurement as Theory

Answer 13

• measurement is considered a fundamental theory development end in its own right.
• where the primary objective of validation research is to offer a theoretical explanation of the processes that lead up to the measurement outcome.”
• This implies that validity research that involves correlating scores with other tests is not useful.
• In this approach, constructs and well articulated theories play a primary role in measurement and in psychology more generally.

Question 14

Validity vs reliability

Answer 14

• reliability may not be a trait that we are actually interested in
• Validity, by contrast, is directly related to the nature of the trait supposedly being assessed by the measure.
• Reliability is a property of test scores
• Validity is a property of test score interpretations
• Validity is closely tied to psychological theory (reliability is not)
• Reliability is a necessary but not sufficient condition for validity.
• In the absence of reliability, there can be no possibility of validity.