Chapter 2: developing the survey

Question 1

What are the 4 phases of the answering process

Answer 1

(1) Comprehension - attend to question and identify meaning
(2) Cognitive processing - retrieval of relevant information
(3) Generating an opinion - make the judgement
(4) Formatting the response - clear answer in mind needs to be formatted to available options

Question 2

What are the 4 typical response effects and explain them?

Answer 2

(1) Acquiesence - the tendency to agree
(2) Social desirability - the tendency to present oneself in a favorable light
(3) Primary effects - Most easy to just select the first answer option
(4) Satisficing - The first answer that seems ‘ OK’ is selected instead of evaluating every option

Question 3

What are the 3 standards of Groves on survey questions?

Answer 3

1. Content standards - do we ask the right things?
2. Cognitive standards - are questions understandable and are respondents willing/able to answer
3. Usability standards - can respondents complete the questionnaire easily and as they were intended to?

Question 4

What are the 4 different types of survey questions?

Answer 4

1. Duration questions
2. Attitudinal questions
3. Behavioral questions
4. Knowledge questions

Question 5

Name 4 different types of duration questions and explain them

Answer 5

1. Time-of-occurence (on what date did you go to..)
2. Duration questions (how long did your visit took)
3. Elapsed time questions (how Long since you last went to..)
4. Temporal frequency (how many times you went to … in …)

Question 6

Name 4 pitfalls when asking questions

Answer 6

1. Asking for attitudes is implying your respondents have one
2. Double barreled questions
3. Leading / suggestive questions
4. Sensitive questions (can’t be asked or won’t receive honest answer)

Question 7

What is an open-ended question. What is advantage and disadvantage, when is it appropriate to use and what should you keep in mind?

Answer 7

1. Question without response options
2. Advantage: receive ‘true’. answers instead of a formatted one
3. Disadvantage: Requires a lot of effort (item-nonresponse, abandoned surveys and unit non-response)
4. If answer form is obvious or if you want elaborations OR difficult to provide good set of alternatives such as occupation
5. Respondents often use round values, might not exactly reflect what the true answer is

Question 8

What can you tell about close-ended ordered questions (5)

Answer 8

1. Most useful when one has a well-defined concept of asked questions
2. Important: use verbal labels for all scalar points
3. 7-point scale: label only endpoints and use numerical values for all answer options
4. People are automatically drawed to middle response options, avoiding selection of extreme
5. People draw meaning from non-verbal cues such as number color and order

Question 9

What is more important, verbal cues or non-verbal cues?

Answer 9

1. Verbal cues overrule non-verbal cues

Question 10

What is a close-ended unordered question?

Answer 10

1. A list of items one has to select from

Question 11

What are the 3 different types of survey questions?

Answer 11

1. Open-ened
2. Close-ended ordered
3. Close-ended unordered

Question 12

What are important ordered response options effects (4)

Answer 12

1. Positivity/leniency bias - reluctancy to select negative end of a scale
2. Scale label effects - negative numbers are interpreted as more extreme than low positive numbers
3. Response contractions bias - avoid answering extreme response categories
4. Reference point effects - number may convey specific meaning that may encourage/discourage respondents from selecting a specific response option

Question 13

What is the recommended logic behind ordering of questions

Answer 13

1. Most salient and interesting to least interesting/salient
2. Start with general questions before you ask specific questions
3. Objectionable questions are placed near the end of the survey
4. Demographics are always at the end of the survey

Question 14

What is the ideal survey length in time?

Answer 14

5 - 10 minutes

Question 15

What are the 4 different types of pretesting the questionnaire and what do they do?

Answer 15

1. Expert review - assess validity of survey
2. Focus groups - pretest survey in a group discussion
3. Cognitive interview - understand the answering process in relation to survey
4. Pilot study - small sample of intended population. Good to find out how the instruments and field procedures work under realistic conditions

Question 16

What is bottom-up and top-down design of a survey and explain the difference?

Answer 16

1. Bottom-up - write down all the questions you cam come up with and then find the logic behind them and order them. Questions that are not needed or non-measurable are deleted
2. Defining all constructs from theory. Search for existing validated instruments and adjust if necessary

Difference is if you start with literature and validated instrument yes or no.