Judgment Phase
- Do your participants understand what you are asking?
- Is it relevant to them?
- Is it clear?
Response Translation Phase
*Can you turn their answer into a useful piece of data?
*How finely do you want to measure a construct?
*How finely can you measure a construct?
You can do these by number of choices (7 is the magic number) and use of anchors
K.I.S.S.
Keep it simple stupid…simpler the better
The simpler the question, the less likely it is to be misinterpreted
Simple Language
- No jargon
- No technical terms
- No unexplained acronyms
- Be direct
If they don’t understand the question, they will answer it anyway (result in BAD DATA)
Double Negatives
- How do you like the “not” questions on test?
- Do you never not use your seatbelt?
AVOID NEGETIONS
Double-barreled questions
- Are you in favor of legalizing pot and heroine?
- Would you vote for a candidate that increased taxes and military spending
BREAK THEM UP (2 DIFFERENT ?’s)
Relevancy
Do you prefer mornings or evenings to do your homework?
PROBLEM: NOT EVERYONE DOES HOMEWORK
Other Wording Issues
- Hypothetical ?’s * Hidden assumptions
- Leading ?’s * Social Desirability
- Value Judgements
- Context effects
- Order in questions
- Sensitive Issues: sex, politics, religion, or drug use)
a. ease into them b. ask them carefully
c. guarantee anonymity
Open-ended questions
- Any answer is possible
- Unlimited
Closed-ended questions (Force Choice items)
*Limited set of answers provided to select
Examples: Yes/No or True/False
Provide enough response choices so all takers can have a relevant choice
Weight values vary per choice —emphasize what determines the bigger or smaller score with what you are measuring
Rating Scales Types
- Multiple alternative/choice sets
- Likert Scales
- Dichotomous Scales
- Visual analog scales
- Rank order scales
- Semantic differential scales
Multiple Choice Question
Consists of three or more exhaustive mutually exclusive categories
Likert Scale
Four to seven choices to select from based on a listed statement
Example … Strongly agree, slightly agree, undecided, slightly disagree, strongly disagree.
Anchors
label the beginning and end of numbered rating scale
Example …
Strongly Agree, 1, 2, 3, 4, 5, Strongly Disagree
Dichotomous Questions
Generally YES/NO or TRUE/FALSE ?’s
Visual Analog Scales
a.) 100mm line with a low end (No Pain) and a high end (Worst Possible Pain) that participant marks the spot on the line where their pain level is. This is later measured with a ruler to determine the weight of the participant’s responses.
b.) emotional faces used in the doctor’s office for determining the patient’s pain level.
Rank Order Scaling
Items that need to be ranked by criteria (Including instructions and numbers to be used with explanations)
Example:
Consider when buying a new car (1=most important, 2=next most important, through 6=least important.)
*Fuel Consumption *Max Speed
*Quick Acceleration *safety cage
*Servicing cost *Status/prestige
Semantic Differential Scale
Seven point rating scale that has two bi-polar adjectives at the end
Example: I think tattoos are:
7 - Attractive, 6, 5, 4, 3, 2, 1 - Very Unattractive
Variation = Variability
*AVOID floor/ceiling effects
*AVOID restriction of range artifacts
Measurement tools that always give the same value is not very useful
Ceiling Effects
Too easy (perfect performance)
No variance between participants to measure
Floor Effect
Too hard (practical limits)
No variance between participants to measure
Multiple items (each construct)
Should reinforce the measurements that you are after and dilute the rest (noise)
Response Set
Switch up the wording on questions:
* Distratctors
* Scales
VALIDITY ITEMS
Survey Research Methods
- Telephone
- Group Administration
- Interview
- Internet
