16 - Objective Personality Testing

Question 1

Objective Testing

Answer 1

• Stimulus is presented to a respondent who makes a constrained (close-ended) response
• Cheap, fast, reliable
• Subject is asked to guess what examiner is thinking

Ex: MMPI

Question 2

Approaches to Developing Scales

Answer 2

1. EXTERNAL APPROACH: empirical/data-driven, not on theory, based on external criterion
   
   • Selects items in relation to criterion, regardless of content
   • Pros: no knowledge needed, likely generalizable, poor face validity, some practical utility bc it’s related to criterion
   • Cons: low content validity, criterion can change, shrinkage may occur (less predictive in second group), time consuming
2. DEDUCTIVE APPROACH: theory-driven, dependent on construct (deduce items from the construct)
   
   • Pros: fast, easy, short, content valid, face valid
   • Cons: if theory is wrong, scale is wrong; may have poor discriminant validity; face validity may cause fake good/bad
3. INDUCTIVE APPROACH: data-driven based on internal criterion; scales are made from pre-existing internal associations between items
   
   • Theoretical knowledge is used to interpret findings
   • Groups items into subscales
   • FACTOR ANALYSIS
   • Pros: simple/homogenous constructs, data reduction for ease of analysis, purer method bc it relies on natural structure, data “speak for themselves:
   • Cons: may be difficulty to interpret if not simple, we make a lot of decisions (indeterminacy), FA is not straightforward, and SPSS is not pure

BEST APPROACH (HYBRID): write a large set of items based on theory (deductive approach), then pick items to keep based on criterion-related validity (external approach), and group items into subscales based on internal structure (INDUCTIVE APPROACH).

Question 3

Factor Analysis

Answer 3

-Aims to identify the optimal latent structure (parsimonious) for a group of variables

• REFLECTIVE LATENT VARIABLES
• What is not common between variables is disregarded as errors
• Best to have at least 4 ordered categories to perform FA
• Does not always work
• When it does work, downfall is in cross-validation with another sample (this never works)

1. Confirmatory FA: examines how well a hypothesized model fits in comparison to others
2. Exploratory FA: no a-priori hypothesis about structure, just identifies a model that balances accuracy and parsimony

Question 4

Decisions to make in Factor Analysis

Answer 4

1. Method of factor extraction: FA or PCA?
2. If FA, what kind? Confirmatory or exploratory?
3. How many factors to retain
4. Whether and how to rotate factors
5. Model selection and interpretation (linear composite)

Question 5

Principal Component Analysis

Answer 5

• Reduces data matrix to a small number of components that explain as much variance as possible
• FORMATIVE MODEL
• EIGENVECTORS = principal components; can be as many as there are variables in the model
• Does not assume reflective latent construct
• Always works

Question 6

Retaining Factors after FA, PCA

Answer 6

1. Kaiser-Guttman Criterion: factors with eigenvalues greater than 1
2. Scree plot: “elbow” minus 1

Question 7

Rotating Factors

Answer 7

-Goal: make them more interpretable and simple

• ORTHOGONAL: if uncorrelated
• OBLIQUE: if correlated

Question 8

Flawed Nature of Self-Assessments

Answer 8

1. Response bias: faking good/bad, social desirability
2. Ambiguity of items
3. Lack of insight (and omission errors)

Improvements:

1. Use clear items that are behaviorally specific
2. Provide frequent, timely feedback
3. Self-testing (after a post-study delay)
4. Review past performance
5. Use peer assessment
6. Target motivational basis of over-confidence
7. Benchmark performance against others
8. Introduce desirable difficulties to instruction (spread/slow learning to improve rates of retaining)
9. Add in safety factors and buffer time

Question 9

Optimizing vs Satisficing

Answer 9

Optimizing: giving their best possible answer (unbiased)

Steps:

1. Interpret
2. Retrieve
3. Judge
4. Respond

Satisficing: when pt is not motivated, so just gives a satisfactory response

 - Weak: go through all 4 steps with less diligence
 - Strong: skip steps 2 and 3; superficial 

Reasons: task difficulty, low respondent ability or motivation, when “no opinion” responses are an option