Validity
Truth (Accuracy of measurements, assumptions, …)
Reliability
Consistency (Accuracy of measurements, assumptions, …)
You can have with no, but you cannot have without .
reliability, validity, validity, reliability
Reliability Practices
- Psychological Tests (may already be available)
- Test-retest
- Alternate form/Parallel Form
- Internal Consistency (Inter-item)
a. split half (odd/even)
b. item/total - Inter-rater/Inter-judge/Inter-observer
Test-Retest
- static constructs (not changing)
- possible history effects (practice effects)
may improve
Alternate Form/Parallel Form
At the same time:
* make two tests (with the same mean and standard deviation)
* Hard to make “Equivalent” tests
Internal Consistency
each item can be considered a mini test, so…(use correlation matrix for measuring each item = average inter-item correlation
Split-half
- tests you compare are shorter the test you will use
- odd/even split (or any way to split them in half.)
- Cronbach’s alpha - all possible split test reliabilities.
nothing less than .70 RULE OF THUMB
Cronbach’s alpha
All possible split test reliabilities
Inter-judge/Observer/rater
*Controls consistency
provides a measure of the degree to which two judges concur in their respective sorting of N items into K mutually exclusive categories.
Cohen’s Kappa
External Validity
ability to generalize
Internal Validity
ability to determine cause/effect
Construct Validity
How well can you measure the construct you are interested in
* true score vs. measurement error (real value and fuzziness - errors)
* Observed score
Types of Constructs
- Face validity
- Divergent Validity (Discriminant)
- Convergent Validity
- Content Validity
- Predictive Validity (criterion validity)
- Concurrent Validity (criterion validity)
- Logical Argument
Face Validity
Does it seem to measure the construct?
Ex: Smith & Sons Welding Job Application
- Do you have any pets? (wrong)
- How many years have you been welding? (correct)
Divergent (Discriminant) Validity
Does it not correlate highly with a similar but different construct?
(tell apart one construct from another)
Ex: IQ Test ——–>Level of education
Convergent Validity
Does it correlate with another measure of the same construct?
Ex: Stanford-Binet IQ Test—> WAIS IQ Test (comparison to another test)
Content Validity
Do we sample from the universe of elements that compose the construct?
Ex: American History Test
(Not asking all the questions on only one subject)
Predictive Validity
Does it predict a future construct?
Ex: SAT Scores ——–> College Success
Concurrent Validity
Does it measure a current construct?
Ex: SAT Scores ——–> High School GPA
Logical Argument
Not empirical based
Ex: Does purchasing behavior translate into customer preference?
Coke sales are People like
higher than Pepsi —–> Coke better
External Validity
ability to generalize
Internal Validity
ability to determine cause and effect
construct validity
how well you can measure the construct you are interested in
