How do carefully prepared questions improve the construct validity of a poll or survey?
- Open-ended questions - participants may answer however they like
- Forced-choice questions - limited choice options, such as yes or no
- Likert scale questions - Ones where the response goes from strongly disagree to strongly agree
- Semantic differential questions - rating a target object using a numeric scale anchored with adjectives
How do researchers make observations with good construct validity?
- Well-trained coders (observers) and clear codebooks (clear rating instructions on behaviour) help ensure that observations are reliable and not influenced by observer expectations
- Some observational studies are susceptible to reactivity (a change in behaviour when study participants know they are being watched), masked designs (where observers are unaware of the purpose of the study and conditions to which participants have been assigned) and unobtrusive studies (make the observer less noticable) make it more likely that observers will not make biased rating and participants wont change their behaviour after being observed.
- Public settings can still be used to conduct ethical research
What can be seen as bad survey questions or bad answers (in regards to construct validity)?
- Sometimes a question is worded in a way that is leading, making people agree more or less than they actually would
- Double-barreled (asking 2 questions in one) and negatively worded (with negative phrasing, such as “does it seem impossible the holocaust never happened?”) questions are difficult to answer with validity
- People sometimes answer survey questions with a fence-siting response or in a way that makes them look good. They may also be unable to report their memories/behaviours accurately
- Question orders where a premise brought by an earlier question leads to a different answer in another
How good are surveys in data collection?
They are efficient and accurate ways to assess people’s subjective feelings and opinions, although they may be less appropriate for assessing people’s actual behavior, motivations or memories.
What are observational studies?
They are methods which record people’s true behaviour rather than what people say about their behaviour
What are response sets?
A type of shortcut people can take when answering survey questions - not thinking much of questions but just answering the same. It includes:
1. Acquiescence - yes saying
2. Fence sitting - answering neutrally all the time < can be mitigated with even answers and forced choice questions
How does one deal with socially desired responding?
By making sure the participants know that the data collected is anonymous, getting people around the person to rate them. They can also use computerised measures to evaluate people’s implicit bias.
What is observer effect and observer bias?
Effect - observers change the behavior of those they are observing
Bias - when observer’s expectations influence their interpretation of the participants behaviours
