EPPP Test Construction Wrong

Question 1

Positive hit rate

Answer 1

percent of positives on the predictor who were actually successful on the criterion

true positives / total number positives

Question 2

To ascertain if the test you have developed is valid as a screening test for determining whether a person has an anxiety or affective disorder, you would be most interested in evaluating the test’s

Answer 2

Concurrent validity

Question 3

To increase false negatives:

Answer 3

raise the predictor cutoff and lower the criterion cutoff

Question 4

Standard error of estimate

Answer 4

estimate the range within which an examinee’s true criterion score is likely to fall given their predicted score on the criterion.

Question 5

what is the mean and SD of a t-score

Answer 5

mean of 50, SD of 10

Question 6

item difficulty index

Answer 6

ranges from 0 (no one answers the item correctly) to 1.0 (everyone in a sample answers the item correctly)

.x = x% answered it right

Question 7

correction for attenuation formula

Answer 7

used to estimate the predictor’s validity coefficient if the predictor and/or criterion were perfectly reliable

Question 8

kappa statistic

Answer 8

estimates inter-rater reliability removing the effects of chance agreement

Question 9

distribution of percentile ranks

Answer 9

EVENLY distributed

distribution “curve” looks like a rectangle

Question 10

Assuming no constraints in terms of time, money, or other resources, what is the best way to demonstrate that a test has adequate reliability?

Answer 10

equivalent (alternate) forms

Question 11

Cronbach’s alpha

Answer 11

used to determine internal consistency reliability

Calculates average reliability with all possible test splits

underestimates reliability, but less so than regular split-half

Question 12

Criterion contamination

Answer 12

when a rater’s knowledge of a person’s predictor performance biases how he/she rates the person on the criterion

Question 13

KR-20

Answer 13

used to determine internal consistency reliability when items are scored dichotomously

Question 14

To maximize the inter-rater reliability of a behavioral observation scale, you should make sure that coding categories:

Answer 14

are mutually exclusive and well-operationally-defined

Question 15

standard error of estimate

Answer 15

standard deviation of the criterion score times the square root of one minus the validity coefficient squared

Question 16

what factors affect the size of the reliability coefficient?

Answer 16

test length (longer test, larger coeff), range of scores (larger range, larger coeff), guessing (higher probability of guessing correct, smaller coeff)

Question 17

After reviewing the data collected on a new selection test during the course of a criterion-related validity study on new hires, a psychologist decides to lower the selection test cutoff score. Apparently the psychologist is hoping to do which of the following?

Answer 17

Decrease the number of false negatives

Question 18

When a test user uses a correction for guessing formula that involves subtracting points from each examinee’s scores, the resulting distribution of scores will have a ____________________ than the original (non-corrected) distribution.

Answer 18

lower mean and larger standard deviation