Survey Research
- Describe thoughts, opinions, feelings
- Measure naturally occurring variables
- Allow predictions based on correlations
- Questionnaires
- Predetermined set of questions
- Sample represents a population
- Examine survey procedures and analyses for sources of bias
Sampling in Survey Research
- Use sample to represent the larger population
- “Representative”: similar to
- Requires careful selection of a sample
- Goal: Generalize survey findings from representative sample to the population
Biased Samples
- Characteristics of the sample differ systematically from those of the population.
- Sample over-represents or under-represents segment(s) of a population
Selection Bias
Researcher’s procedures for selecting sample cause bias
Response-rate bias
Individuals selected for the sample do not complete the survey
Populations
Set of all cases of interest
Sampling frame
We need to develop a specific list of members of the population in order to select a subset of that population
Sample
The subset of the population actually drawn from the sampling frame
*Representative of the population to the extent that it exhibits the same distribution of characteristics as the population
Representativeness
The ability to generalize from a sample to the population
Probability sampling
By selecting students randomly from the registrar’s list of students, each person (element) on the list has an equal chance of being included in the sample
- Far superior to nonprobability sampling in ensuring that selected samples represent the population
- All members of population have a specified chance of being selected for the survey
Nonprobability sampling
Does not guarantee that every element in the population has an equal chance of being included in the sample
*Sample likely not representative of population
Simple random sample
Random selection, random-digit dialing
- Basic technique of probability sampling
- Every element has an equal chance of being included in the sample
1. Define your population of interest
2. Determine your sampling frame
3. When a convenient list of manageable size is available, a random sample can be taken to achieve a representative sample.
Stratified random sample
The population is divided into subpopulations called strata (stratum) and random samples are drawn from each of these strate
- Two general ways to determine how many elements should be drawn from each stratum
- Draw equal-sized samples from each stratum
- Draw elements for the sample on proportional basis
- Only the stratified sample on a proportional basis would be representative.
Convenience sampling
Individuals are available and willing to respond to the survey
- Example: magazine surveys, call-in radio surveys
- Will result in a biased sample unless you have strong evidence confirming the representativeness of the sample
Four methods for obtaining survey data
- Mail surveys
- Personal interviews
- Telephone interviews
- Internet surveys
Ways to increase response rate
- Questionnaire has a “personal touch”
- Use name, not “Dear student”
- Responding requires minimal effort
- Topic of survey is interesting to respondents
- Respondents identify with organization or sponsor of survey
“Research design”
- A plan for conducting a research project
* Choose method best suited for answering a particular question
Three types of survey research designs
- Cross-sectional design
- Successive independent samples design
- Longitudinal design
Mail Surveys
Distribute self-administered questionnaires that respondents fill out on their own
*Advantage: they usually can be completed relatively quickly
-Avoid the problems due to interviewer bias
-Best for dealing with highly personal or embarrassing topics, especially anonymity of respondents is preserved.
Disadvantages: The cost of copying and mailing
-Self-explanatory
-The researcher has little control over the order in which the respondent answers the questions
-The order of questions may affect how respondents answer certain questions
-Low response rate-> response rate bias
Response rate bias
threatens the representativeness of a sample
Personal Interviews
- Allows greater flexibility in asking questions
- Ensure that all respondents complete the questions in the same order
- Higher response rate
- Allow researchers to gain more control over how the survey is administered
- Disadvantages: Costly
- Interviewer bias
Interviewer Bias
- Occurs when the interviewer records only selected portions of the respondent’ answers or tries to adjust the wording of a question to “fit” the respondent.
- Protection against interviewer bias: employ highly motivated, well-paid interviewers who are trained to follow question wording exactly, to record responses accurately, and to use follow-up questions judiciously.
- Interviewers should also be given a detailed list of instructions about how difficult or confusing situations are to be handled.
- Interviewers should be closely supervised by the director of the survey project.
Telephone Interviewers
- Are used frequently for brief surveys
- Random-digit dialing techniques
- Provides better access to dangerous neighborhoods, locked buildings, and respondents available only during evening hours.
- Can be completed more quickly
- Interviewers can be better supervised
- Limit to how long respondents are willing to stay on the phone
- Respond differently when talking to a “faceless voice”
Internet Surveys
- Participants complete a questionnaire online and click on a “submit” button to have their responses recorded.
- Permit manipulation of variables and the random assignment of participants to experimental conditions
- Advantages: efficiency, cost, access groups
- Disadvantages: Potential for sample biases
- No way to generate a random sample of Internet users
- Response rate bias and selection rate bias
