How would you construct a self-report test?
Generate a lot of items
Give to two groups:
1. Non-patient sample
2. People diagnosed
See what items differentiate those two groups
Methods of Self-Report Test Construction #1
- Rational - items are written to capture understanding of what a trait is
- Characteristics
–> Tend to be face valid
–> Susceptible to response biases (easily faked)
–> May not be internally consistent or valid
Methods of Self-Report Test Construction #2
- Factorial - items are selected on the basis of factor analysis
- Characteristics
–> Highly internally consistent
–> Tend to be face valid
–> Somewhat susceptible to response bias
Methods of Self-Report Test Construction #3
- Empirical - items are selected on their ability to empirically distinguish one group from another
- Characteristics
–> Often have low internal consistency
–> Often items are not face valid
–> May be less susceptible to response biases
Clinical Judgement
Dawes, Faust, and Meehl (1989) “Clinical Versus Actuarial Judgment”
- Using statistical methods to derive algorithms (i.e., MMPI) for decision-making is superior to clinical decision-making
- “Some studies use artificial tasks that failed to tap their expertise”
- Humans do not have superior observational skills but better to enter them into decision-making algorithm
APA Report, June 1998 “Benefits and Costs of Psychological Assessment in Healthcare Delivery”
Nature of some errors in clinical judgement:
1. May only elicit information that confirms hypotheses and ignore questions/information that would disprove it
- May compare patient to prototype rather than systematically evaluating on specific criteria
- May be overconfident rather than appropriately tentative
- Hindsight bias (wrongly assume we could have predicted the result AFTER being told result)
- May not consider relative frequency of the event they are predicting (rare events are harder to predict than common ones)
APA Report (continued) #1
The assessment process provides some checks on these potential errors:
- Staying true to data forces one to consider many possibilities to accommodate conflicting data
–> Must integrate seemingly conflicting data
- Consider the nature of various types of data (symptoms, objective, projective, interpersonally gathered vs. solitary, etc.)
- Consider reliability and validity
- Consider the peculiarities of measures
- Consider the motivational and environmental circumstances of testing
- Test results must be reconciled with history
- Understanding must be integrated with assessor’s understanding of complex conditions being assessed
APA Report (continued) #2
- Corrective strategies one can use
- Be sure you can systematically identify characteristics of condition you want to diagnose
- Test indicators and their absence should be directly linked to these characteristics
- Systematically revise your impressions by considering data that temper your hypothesis
- Predictions of rare events should be made sparingly
- Use empirically validated, statistically derived predictions when available
- Anticipate making mistakes
APA Report (continued) #3
- Feedback from the client can be a powerful corrective mechanism
APA Report (cont.)
Clinical judgment can sometimes be as good as statistical decision rules, but it never exceeds them.
BUT there are problems:
1. Decision rules do not generalize well to different settings
- There are no decision rules for the vast majority of decisions that must be made (infinite complexity prohibits relying on rules)
Summary of Clinical Decision Making
- Clinical intuition is very fallible, but we tend to ignore this fact
- Actuarial algorithms are better than clinical judgment
Diversity Considerations #1
Multiculturalism is a very important social issue, but in clinical work, I prefer the term diversity (or idiographic) over multicultural
- Diversity is broader
- It focuses on the individual (idiographic)
(which is more appropriate in clinical work) - It better guards against stereotyping
Diversity Considerations #2
Central to any assessment is understanding and considering the unique environment in which the individual lives
Intersectionality adds enormous complexity
Social constructs and parlance certainly influence but don’t necessarily translate well to psychological conceptualizations
Example: Socially, privilege generally refers to the advantages enjoyed by majority social groups (whites, heterosexuals, cis-gendered), which no doubt have effects that influence everyone psychologically
But psychologically, privilege may be understood, for example, as the advantages of having parents who made possible a cohesive sense of self, and secure attachment status, because they lead to resilience, successful relationships, productive careers, and satisfying lives. This psychological level of our lives captures a universal concern - and perhaps a unifying force. And is certainly the focus of clinical work.
Diversity Considerations #3
Nomothetic tests/methods can be extremely useful when used wisely
BUT they can be misused. Thoughtless application of tests can:
- Unfairly discriminate
- Misdiagnose those from cultural groups not captured by the normative group as well as the ideographically different
- Unfairly deny opportunities
- Etc., etc., etc.
One of the most compelling reasons FOR testing:
To rise above our own biases/limitations
Testing as a bias mitigator (and as Finn says, an “empathy magnifier”)
Diversity Considerations #4
Temper results with good judgment that as much as possible rises above our own social context/experiences in the world
AND qualify your conclusions approximately
Diversity Considerations #5
Bottom line: We all have a multitude of unique life influences, and we make sense of them in a myriad of unique ways
–> Must consider the individual’s life influences AND what effect they have had on that particular individual
Psychological assessment, especially Collaborative/Therapeutic Assessment is a powerful agent of bias reduction and of understanding the person in front of us
Measurement Invariance across 16 Languages
Measurement invariance = Same thing in multiple places in the construct
See slide
Self Report: Unidimensional vs. Multidimensional
Unidimensional Measures
- Widely used for quick assessment of a specific issue
Multidimensional Measures
- Personality tests
- Batteries: Contain multiple scales
- Often include validity scales
Problems with Self Report Data
Impulsivity
Maturity
Self report is based in one’s theory of self
- Not responsive to behavior change that can be tracked
Types of Psychometric Data
- Observational data
- Life data
- Self report data
- Performance-base data
- Informant’s data
Self Report vs Other Report
S data vs. O data
Helpful when parents rate one partner rates child one way and the other partner rates a child another way
Report Measures from Collateral Sources
Parent/Teacher Reports
> Multidimensional
- Child Behavior Checklist (CBCL)
- Behavior Assessment Scale for Children (BASC)
> Unidimensional
- Conners-3
- Back Depression Inventory
Clinician Report
> Shedler Weston Assessment Procedure (SWAP)
History of the MMPI
Italicized
Hopeful that it would be a more efficient way to arrive at routine diagnosis
