Difference between survey and psychological test?
- Survey = used to collect important info from individuals
-> more flexible
-> focus on groupoutcomes
-> reported at question level, percentage of respondents - Psychological test = representatively sample behaviors thought to measure an attribute or predict an outcome (measure well-defined construct)
-> standardization
-> focus on individual outcomes
-> often reported at test level as overall score
Steps to constructing a survey?
- Preparing for survey
- Pre-testing survey
- Administering survey
- Coding, entering, analysing data
What is a psychological construct?
= well-defined reference points fo theory
- tool to help us understand human behavior
- construct of multiple behaviors/characteristics
- summarized in operational definition
Different types of scoring
- Cumulative model of scoring = most common method for determining individual scores
- Categorical model = place test takers in particular group or class (f.ex. pattern of certain symptoms)
- Ipsative model of scoring = presented with statements, right or wrong
Difference between objective & subjective tests?
- Objective tests = often multiple choice (f.ex. preemployment test)
-> stem, the question part
-> distractors: incorrect responses
-> forced choice (personality tests f.ex.) - Subjective tests = essay questions (lengthy, general question)
-> judgement error, bias possible
-> intrascorer reliability needed
-> interview questions (follow-up possible) or projective responses (ambiguous to unstructured response)
Item difficulty
= percentage of test takers who respond “correctly”
-> dividing persons responding correctly by total number
-> differentiating & comparing to others
-> item difficulty of .5 yield = most variation/ half answered correctly
Item discrimination
= compares performance of those who obtained very high test scores (upper group) to those who obtained low on each item
-> appropriate differentiation
-> discrimination index ranges from -1.0 to 1.0
-> the more positive the index, the better quality of item
-> knowing answet to only 1 question will tell me a lot about how high/low construct is
Item-Total correlation
= measure of the strength and direction of relation between way test takers respond to one item and way they respond to all items as a whole
-> acceptable items: correlation of at least .2 (lower should be dropped)
Inter-Item Correlation
= displays correlation of each item with every other item
-> appropriate & consistent results?
-> how similar or different?
Item-Criterion Correlation
= correlation of item responses with a criterion measure
-> criterion = construct that should be related to measure, similar tests, and similar consequences of testing
-> if high -> confidence that item by itself represents construct
Item-Response Theory
= a measure of relationship between individual’s performances on one test item & test taker’s level of performance on overall measure of constructs
-> if i know answer to 1 question, how much do i know about overall construct?
Item Bias
When one item is easier for one group than for another group
Qualitative analysis
= non-statistical means of evaluating qualitative data (analysis of text usually)
-> when? when test takers give verbal/written feedback about questions
-> why? examine in more open-ended way
-> how?
a. individual level (survey)
b. group level (expert panel, giving feedback about items)
3 Interview collection methods
- Structured: rigid set of questions, interviewer cannot deviate from questions or ask follow-up (standardized)
- Unstructured: informal, free-flowing, interviewer can go any direction
- Semi-structured: more open, can ask follow-up
-> probing: follow up & seek more detail
-> prompt: follow up and give more info to help them answer
What are the standards for Educational and Psychological Testing?
- guidelines for professional practice
- american, but used internationally
- focuses on best practices for testing, less on ethics
- psychological associations tend to oversee
Canadian Code of Ethics for psychologists - 4 principles
= steps & principles to assist research & psychologists for ethical decisions
1. respect for dignity of persons and people (non-discrimination & justice)
2. Responsible caring (competent, maximizing benefit & minimizing harm)
3. Integrity in relationships (accurate & honest)
4. Responsibility to society (minimize risk of misuse)
10 steps for ethical decision-making
- Identification of individuals & groups potentially affected by decision
- Identification of ethically relevant issues & practises (legal, political context)
- Consideration of what may bias/influence decision
- Alternative courses of action
- Analysis of risks/benefits of individual/groups
- Choose course of action
- Action, and committing to assuming responsibility
- Evaluation of results of course of action
- Assumption of responsibility for consequences of action
- Appropriate action, as warranted & feasible, to prvent future occurences of dilemma
Certification
= a professional credential individuals earn by demonstrating that they have predetermined qualifications (basic)
Licensure
= mandatory credential they must obtain to practice within profession (has to be renewed, staying up to date)
What are the test publisher’s responsibilities?
- demonstrate highest level of professionalism & ethnics when selling and marketing tests
- must market tests accurately (can’t oversell psychometric quality)
- ensure test security (content does not become public)
- ensure that accompanied by user manuals
What are the test users responsibilities
= participates in purchasing, administering, interpreting, using results
- must be qualified to use
- sale & distribution (often need to purchase)
- qualification: background, training, certifications must be met
What are the test takers responsibility?
- must understand consequences of testing/not testing
- must ask questions if unclear
- must protect text security
- rights:
1. right to privacy
2. right to informed consent
3. right to know & understand results
4. right to protect from stigma
TCPS 2: ethnical conduct for research involving humans - CONSENT
- consent = free, informed, ongoing
- informed = full disclosure of all info necessary to decide if participating
- coercion: extreme undue influence, threat/harm for failure to participate
- incentives: anything offered fo rparticipating, must be proportionate!
-> this is overseen by Research Ethics Board
Research Ethics Board
- any research institution: to review ethical acceptability of research w/humans
- governed by TCPS 2 (establising policies for REB and procedures that need to be followed)
- at least 5 members
1. two members with relevant expertise (f.ex. doctor)
2. one member knowledgeable in ethics (academic background)
3. one member knowledgeable in relevant law
4. one community member with no affiliation w/institution
