What are the characteristics of quality questionnaires?
- Appropriate - answers research question and objectives
- Simple and comprehensible - simple language, understood by all people i n target population
- Unambiguous and clear - questions mean the same thing to researcher and all respondents
- Unbiased - not more likely to elicit one kind of response than another
- Have reliability (precision) and validity (accuracy)
- Accommodate all important and relevant responses - omni-competent (tries to anticipate all posible answers and include “don’t know” option to avoid blank responses
- Reasonable length
- Appropriately coded for data capture, analysis and interpretation
- Validated
- Piloted - performance assessed, design faults ironed out
- Ethical - informed consent, principles and no trick questions
What are the pros and cons of interviewer administered questionnaires?
Widely used in health research, where researcher asks the questions in a private space
Pros:
- can encourage higher completion rate and better completion of questions since interviewer can re explain certain terms and can allay fears
- participants do not need to be literate
Cons:
- time consuming for researchers
- more expensive (hiring and training interviewers)
- bias introduced if interviewer does not use a standardised method for administration, impact of interpersonal factors wrt sensitive questions
What are the pros and cons of self administered questionnaires?
Questionnaire completed by participant in a private space using paper based or electronic questionnaire
Pros:
- good strategy for completion of sensitive infor
- less time consuming for researchers
- can use an audio recorded questionnaire so participant does not need to be literate
Cons:
- more preparation time - questionnaire must be clear and well laid out
- lack of skip patterns/explanations can impact completion of the questionnaire
- recording of translations can be expensive
- poor completion (not all questions answered if long)
What are the pros and cons of close ended questions?
Responses are restricted to predetermined answers, recommended in quantitative research
Pros:
- can focus on specific areas
- shortens the interview
- less time spent coding
Cons:
- responses could be limited depending on researcher’s knowledge
- information can be missed
What are the pros and cons of open ended questions?
Respondent answers in whatever way they choose, recommended in qualitative research
Pros:
- good for exploratory work
- can be used when there is an extensive list of possible options
Cons:
- generates a lot of data that is time consuming to read, categorise and code for quantitative analysis
- respondents can be confused as to hot to answer
What does reliability mean in the context of a questionnaire?
Precision
The questionnaire delivers consistent results on the same participant or group
What does validity mean in the context of a questionnaire?
Accuracy
The questionnaire measures what it is intends to measure (e.g. a survey designed to measure depression but which actually measures anxiety - NOT VALID)
How to maximise reliability in a questionnaire?
Accuracy of the instrument
Standardise data collection procedures
Taryn interviewers
Supervise data collection
Test-retest reliability: re-administer to same subjects after interval
How to maximise validity in a questionnaire?
Bias is minimised (recall/social desirability/ non participation)
Using a a validated tool, check results against an independent source
Self report vs biomarkers or lab rest result
