What are norms in psychological testing?
Test performance data from a group used as a reference to interpret individual scores.
What is a normative sample?
the reference group to which the performance of test takers are compared.
What is norm-referenced testing?
A method of evaluation and a way of deriving meaning from test scores by evaluating an individual test taker’s score and comparing it to scores of a group of test takers.
->The meaning of an individual test score is understood relative to other scores on the same test.
Why are relative scores important in psychology?
->Many traits (e.g., personality, psychopathology) have no absolute metric — they’re meaningful only in comparison to others.
->Contexts change and populations shift and grow so we need to compare scores to others
What does “eccentricity” mean in testing?
Distance from the population mean — neither good nor bad, just a measure of difference.
What is standardization in testing?
The process of administering a test to a representative sample of test takers for the purpose of establishing norms (that we compare others to)
What are the three meanings of “standardized”?
- Tests
->can be standardized by establishing population norms (e.g., collecting a large, representative sample to represent a population, so we can compare individual scores against this standard) - Scores
->can be standardized by altering their numeric value to fix the mean and standard deviation (e.g., converting raw scores into z-scores or T-scores); can be measured against a population standard or any sample (e.g., in a research study) - Administrations
->can be standardized by using the same instructions, similar test environments, interview schedules, etc. (e.g., using a script to introduce a test, ensuring a quiet space)
What is sampling in test development?
Test developers select a population, for which the test is intended.
->Test takers should have at least one common, observable characteristic. (could be gender, eye color, people in car accidents etc.)
What is stratified sampling?
Sampling that includes different subgroups, or strata, from the population: ensures representativeness
What is stratified-random sampling?
Every member of the population has an equal opportunity of being included in a sample.
What is a purposive sample?
Selecting a sample that is believed to be representative of the population.
What is an incidental/convenience sample?
A sample that is convenient or available for use; it may not be representative of the population.
EX. the people in your clinic waiting room, respondents on a website
Generalization of findings from convenience samples must be made with caution.
Having obtained a sample, test developers:
- Administer the test with a standard set of instructions.
- Recommend a setting for administering the test.
- Collect and analyze data.
- Summarize data using descriptive statistics including measures of central tendency and variability.
- Provide a detailed description of the standardization sample.
What is a percentile?
The percentage of people whose score on a test or measure falls below a particular raw score.
Is there a 100th percentile?
No one can score better than themselves so no.
Why are percentiles popular?
Easy to calculate and understand.
What is a limitation of percentiles?
- real differences between raw scores may be minimized near the ends of the distribution and exaggerated in the middle of the distribution
- This is because density increases near the center
What is a fixed reference group scoring system?
The distribution of scores obtained on the test from one group of test takers is used as the basis for the calculation of test scores for future administrations of the test.
- The SAT and GRE tests employ this method
What is criterion-referenced testing?
Comparing test performance to a set standard, not to others. (absolute level of performance)
EX’s.
- Passing grades in a university exam
- Choosing the correct diagnosis in a simulated patient exam
- Avoiding collisions and rule violations in a driving test
What is gradation?
Determining differences between individuals’ scores relative to each other.
->Talking about how split people into low-medium- high risk categories, same as norm referencing (comparing people to each other) but also interested in absolute risk
→Measure allows us to determine accurately peoples risk on a risk scale but not their actual score
What is calibration in assessment?
Using individual scores to predict an outcome
→Kinda like predicting the future/probability of an outcome
EX. Predicting a person’s risk of reoffending or developing a psychotic episode.
Why is calibration important?
“High risk” can mean very different probabilities depending on the population base rate.
How is calibration different from gradation?
Gradation compares individuals to each other.
Calibration estimates an individual’s absolute probability of an outcome.
Example of gradation vs calibration?
Gradation ranks Mr. X as higher risk than Ms. Y; calibration predicts Mr. X has a 76% chance of reoffending.
