How do we evaluate psychological tests
- What is the theoretical construct the test is measuring?
* Do test items correspond to the theoretical description or construct? - Standardization
* Is the population to be tested similar to the standardization group?
* Was the standardization sample size adequate?
* Have sub-group norms been established? - Reliability
* Reliability refers to the degree of stability, consistency, and predictability
* Concerned with:
◦ How was reliability estimated?
◦ What is the stability of the trait being measured?
◦ How does test format impact reliability? - Validity
* Does the test measure what it is intended to measure?
* Concerned with:
o Does it provide useful information for the clinician? - Administration & Scoring
a. Starting points / reverse sequence
b. Queries, prompts, repeating instructions, waiting time, sequence of subtests
c. Ceiling effects
d. Testing the limits
e. Supplemental tests
f. Subtest substitution
g. Short forms
h. Scaled scores - Interpretation
a. What guides are in the manual for interpretation
b. What are the signs of an objective / standard administration
c. What are the limits on reporting responsibly – and are they clearly articulated
d. What additional texts or resources are available regarding interpretation of scores
e. How do subscales related – and do they bring new information when considered in combination?
f. What recommendations can be formulated for treatment – are there resource materials available?
What to consider in evaluating psychological tests for THEORETICAL CONSTRUCT
What is the theoretical construct the test is measuring?
* Do test items correspond to the theoretical description or construct?
What to consider in evaluating psychological tests for STANDARDIZATION
- Is the population to be tested similar to the standardization group?
- Was the standardization sample size adequate?
- Have sub-group norms been established?
What to consider in evaluating psychological tests for RELIABILITY
Reliability refers to the degree of stability, consistency, and predictability
- Concerned with:
◦ How was reliability estimated?
◦ What is the stability of the trait being measured?
◦ How does test format impact reliability?
Examine:
* two administrations of the test on a single person and correlate scores
- alternate test forms - - degree of correlation btw test scores on one form compared to an alternate form - hard to come up with one good version of test nvm two
- internal consistency – through the split half reliability - rrelate scores on one half of the test to scores on the other half of the test - hard to to that when test is graduated - assessment fatigue - last ones may be affected
What to consider in evaluating psychological tests for VALIDITY
- Does the test measure what it is intended to measure?
- Concerned with:
o Does it provide useful information for the clinician?
Examine
* Content validity: measures all aspects of what it is intended to measure?
- Criterion validity
o Comparing scores with performance on an outside measurement
»>Concurrent validity: measurements taken at the same time as the test
»»Predictive validity: measurements are separated in time - Discriminant validity - A measure should not be correlated with things that you are not supposed to be measuring or things that are not related to the construct
What to consider in evaluating psychological tests for ADMINISTRATION and SCORING
a. Starting points / reverse sequence
—-Where to start based on age and ability, how to work backwards
b. Queries, prompts, repeating instructions, waiting time, sequence of subtests
—-How much time to wait for an answer
What order should the subtest be administered in
c. Ceiling effects
—-test seemed very easy for the child
Are there a certain percentage of people you would expect to experience this effect
d. Testing the limits
e. Supplemental tests
Look at strengths or weakness
f. Subtest substitution
Measuring cognitive ability but you notice verbal ability is not very strong
g. Short forms
As a clinician - have to plan out assessment schedule and it can be very time consuming - can help to shorten that time
h. Scaled scores
What limits to interpretation
What to consider in evaluating psychological tests for INTERPRETATION
—What guides are in the manual for interpretation
—-What are the signs of an objective / standard administration
—What are the limits on reporting responsibly – and are they clearly articulated
——————Recommendations about wording and interpretation
–What additional texts or resources are available regarding interpretation of scores
—How do subscales related – and do they bring new information when considered in combination?
———-Vs when they are considered in isolation
—What recommendations can be formulated for treatment – are there resource materials available?
Evaluating Psychological Tests: Reviews
- Look at test manual
- Reference books
* Mental Measurements Yearbook (Geisinger et al., 2007; www.un1.edu/buros/)
* Tests in print (Murphy et al., 2006)
* Tests: A comprehensive reference for assessment in psychology, education, and business (Maddox, 2003)
* Measures for clinical practice and research: A sourcebook (Fischer & Corcoran, 2007)
* Assessments that work (Hunsley & Mash, 2008)
* Neuropsychological assessment (Lezak, 2004)
* A compendium of neuropsychological tests (Sherman & Spreen, 2006) - Journals
Sattler Rapport Quote
To work successfully with children, you’ll need to have tact, ingenuity, patience, understanding, warmth, and respect. A competent examiner is flexible, vigilant, and self-aware and genuinely enjoys working with children.”
Establishing Rapport - Characteristics of Examiner
Flexibility
* Ability to adjust testing techniques
Vigilance
* Being present, paying attention to the behaviours of the child
* Administering tests should not become entirely automatic
* Is the child fatigued? Are they giving their best effort? Are they feeling discouraged?
— Younger children more likely to be fatigue
Self-awareness
* Awareness of your own thoughts, feelings, biases, past experiences, tendencies to behave in
conversations, body language, eye contact, skill level
Why is self-awareness so important?
Clinician’s overall impressions, expectations, or beliefs can lead to expectancy effects / halo
effects
This impacts:
* Assessment behaviours (e.g. more smiling, nodding, acting friendlier, providing more praise)
* Relationship with the child / parents
* Objective scoring
* Interpretation and recommendations
The extent to which an examiner likes a child can impact how the examiner scores ambiguous
responses. It is important to safeguard the objectivity of the test administration
It is YOUR responsibility to recognize the possible factors that might influence your rapport and
how the relationship can impact assessment results.
Feldman & Sullivan (1971)
* When assessors enhanced rapport with children, IQ scores on the WISC increased by 13 points
compared to administrations with more neutral interactions
* When assessors showed mildly disapproving feedback, IQ scores decreased
Ways to build a good relationship:
- Have empathy, genuineness, warmth, and respect
- Building rapport is a continuous process – it builds across the assessment
- Initially on meeting the child: use their first name
- Give brief account of purpose of examination
- Important to relay information at the child’s level of comprehension
- Be as prosocial, engaging, positive as possible.. Effusive, relaxed
o You’re are excited about meeting him/her and happy to spend time
Establishing Rapport
barriers that may occur
- Listen empathetically – not only to what is said but also to how it is said
»>Observant of voice and speech (volume, rate of speaking, ease of speech, relevance, organization)
»»Observant of non-verbal behaviour (motor, posture, facial expressions, eye contact) - Make adjustments and be fluid in the tone, pacing, materials, and content of questions
- Collaborative effort
Thought blocking
* Stoppage of speech
* Inhibition of recall
* Reduction in communication or ideation
» Acknowledge child’s reactions with a warm and expressive tone
» Provide reassurance and support
Helping the child feel at ease
Allow the child time to settle in to the space
- Use of non-threatening tests (e.g. the Goodenough test)
»Tests with immediate success
»>An opportunity to provide positive comments - Encourage the child to take a chance or offer an answer even when uncertain
- If child looks frustrated – short breaks should be taken or testing should be discontinued
- Graduated tests – let the child know that test items are expected to become harder
- Praise for efforts rather than correct answers
Am I right?
- For some children getting the answer right becomes extremely important
- Identify early on that there are some rules that you must follow – that you can’t share whether they got the answer right or wrong & they have to try their hardest
- Some children will want to see what you are recording
Behaviours that diminish rapport
Talking about previous children that have been tested
Being flippant or sarcastic
Disagreeing or arguing with child
Showing shock at something the child says
Completing the child’s sentences for them
Listening superficially to the child’s statements
Minimizing the child’s feelings
Being judgmental
Verbal and non-verbal expression of anxiety
Verbal expressions of anxiety
* Correcting oneself
* Making slips of the tongue
* Repetitions
* Stuttering
* Intruding
* Incoherent sounds
Non-verbal expressions of anxiety
* Sweating
* Trembling
* Fidgeting
* Restlessness
* Hand clenching
* Twitching
* Scowling
* Forced smiling
what to do to handle anxiety
Acknowledge the child’s anxiety – talk about it
- Emphasize collaborative effort needed – join together
- Watch out for signs of heightened anxiety: break / change /back-off
- Complement child for opening up
Clinical interviews: Domains
Demographic information
Presenting problems
History of problem - how long has it been happening
Psychological history
Medical history - might have to look at physical health as well
Family history - whats your family structure like, familys medical history
Previous treatments sought
Might have been an intake process, some might have already been answered
Clinical Interviews
Usually used in conjunction with other assessments, make better suggestions, referral question may not end up being the most important issue - realize its more complicated than identified by initial source
Referral is a ticket into a clinic, more important questions might need to be addressed
Open-ended questions
Broad focus
* “tell me about what
brings you here today”
close ended questions
Highly focused
* Yes / no answers
* “do you like school?”
- Concrete / direct answers
◦ “how old are you?”
Prompts & Probes
Some useful prompts when asking about
symptoms:
● How often does it happen?
● When does it occur?
● What happens when you feel that
way?
● What is it like?
● When was the last time you ____ ?
● When you ____ how does it affect
your school work?
Elaboration
E.g., “Tell me more about that
-
1 - Assessment12
-
Clinical Interviews30
-
Intelligence Testing45
-
Development35
-
Report Writing22
-
ASD32
-
ADHD27
-
Specific Learning Disorders29
-
Visual-motor Integration16
-
Neuropsychological Assessment20
-
ADI8
-
ADOS24
-
CBCL9
-
Hopkins Verbal Learning Test-Revised HVLT-R8
-
DOMINIC9
-
DANVA8
-
ADHD LAB20
