Week 7 - Selection Bias, sampling methods and information bias

Question 1

What is random error?

Answer 1

Random error is error introduced solely by chance and is
inherent in the sampling process

Question 2

What is systematic error?

Answer 2

Also called bias
Systematic error is introduced via manmade actions relating to the conduct of a study

Question 3

What is the sample vs. true population?

Answer 3

We do not measure the true population measure (mean,
%, etc) but an estimate of that based on representative
sampling

Question 4

How can we decrease the random error in epidemiological studies?

Answer 4

• Chance/random bias decreases with increase in the
  sample size
• Goes down to zero if the total population is included

Question 5

What is a confidence interval of sample estimates?

Answer 5

• A confidence interval indicates the level of uncertainty
  around the estimated measure
• Most studiesreport the 95% confidence interval (95%CI)
• 95%CI indicates a range within which we can be 95%
  certain/confident that the true population measure lies
  there; the larger the sample size the narrower is the
  95%CI

Question 6

How can we lower systematic error?

Answer 6

• Systematic bias are not influenced by sample size

Question 7

What is selection bias?

Answer 7

• Selection bias is systematic error resulting from the fact
  that the participants included in the study are not
  representative of the population from where they were
  selected (source population)
• Selection bias leads to a biased sample, which almost
  always, will give rise to biased estimates
• The sampling method of choice plays a major role in the
  representativeness of the sample

Question 8

What is a representative sample?

Answer 8

Question 9

What is a non-representative sample?

Answer 9

Question 10

What are the three sampling methods?

Answer 10

1. Probability (random) sampling: sample selected by
   probabilistic methods; involves random selection,
   allowing you to make strong statistical inferences about
   the whole group
2. Systematic sampling: sample selected according to some
   simple, systematic rule
3. Non-probability sampling: sample selected by easily
   employed (convenient); involves non-random selection
   based on convenience or other criteria, allowing you to
   easily collect data.

Question 11

Sampling methods summary

Answer 11

Question 12

What is simple random sampling?

Answer 12

• Often referred to simply as ‘random sampling’
• The most straight-forward of all random sampling methods
• All individuals in the sampling frame have the same
  probability of being selected independently of all others
• It is mainly used in quantitative research.
• Given a large sample size, random sampling ensures the
  chosen individuals are representative of the source
  population
  – Demography (e.g. age, sex, ethnicity)
  – Other important factors (e.g., clinical history, current disease status,
  lifestyle factors, etc.)

Question 13

What are the advantages and disadvantages of Simple Random Sampling?

Answer 13

Advantages
* Ensures a representative
sample from the source
population
– Provided that the sample size is
large enough
* Less costly and less time
consuming from other more
sophisticated sampling
methods
* Ideal for quantitative studies
& test of hypothesis
Disadvantages
* If the sampling frame is too
large and/or the population
is geographically diverse it
may be impractical to
perform
* If a large sample is required,
simple random sampling
may be time consuming and
costly

Question 14

What is Stratified Random Sampling?

Answer 14

• Same principles as simple random sampling but
  within strata (subgroups) of the population
  – in terms of key demographic characteristics
• The size of the random sample should be proportional
  to the specific stratum size in the population

Question 15

An example stratified random sampling.

Answer 15

• The company has 800 female employees and
  200 male employees.
• You need a sample of 100
• You sort the population into two strata based
  on gender.
• You want to ensure that the sample reflects
  the gender balance of the company so you use
  random sampling on each group, selecting 80
  women and 20 men, which gives you a
  representative sample of 100 people.

Question 16

What is the procedure Stratified Random Sampling?

Answer 16

Question 17

What are the advantages and disadvantages of Stratified Random Sampling?

Answer 17

Advantages
* It allows you draw more
precise conclusions by
ensuring that every
subgroup is properly
represented in the sample.
* Enables the comparison of
population sub-groups
Disadvantages
* More time-consuming than
simple random sampling
* Higher complexity might
give rise to errors (e.g.
stratification not conducted
properly)

Question 18

What is cluster sampling?

Answer 18

• Based on the hierarchical structure of natural clusters
  (groups) of individuals within the population
  – Natural clusters may be hospitals, schools, streets, city
  districts, etc.
• Involves taking a random sample of these natural clusters,
  and then selecting all individuals in the selected clusters
• The sampling frame is a list of all clusters.
• If it is practically possible, you might include every
  individual from each sampled cluster. If the clusters
  themselves are large, you can also sample individuals from
  within each cluster using one of the techniques above

Question 19

What are cluster sampling?

Answer 19

Question 20

What are the advantages and disadvantages of cluster sampling?

Answer 20

Advantages
* Good for dealing with large
and dispersed populations
* Less costly and less time
consuming
Disadvantages
* Substantial differences between
clusters can cause errors
* It’s difficult to guarantee that the
sampled clusters are really
representative of the whole
population
* Representativeness may be
compromised if
– Too few clusters are selected and/or
– Clusters are too specific and/or
– Clusters contain too few individuals

Question 21

What is multi-stage sampling?

Answer 21

• Utilizes the hierarchical structure of natural clusters (groups)
  of individuals within the population
  – Similarly to cluster sampling
• After randomly selecting clusters, there is a random
  selection of individuals within the cluster
• May involve several random sampling stages:
  – Stage 1: Random selection of large clusters e.g. schools
  – Stage 2: Random selection of smaller clusters within large clusters
  e.g. class
  – Stage 3: Random selection of individuals within smaller clusters

Question 22

What are the advantages and disadvantages of Multi-stage Sampling?

Answer 22

Advantages
* Multi-stage sampling may
improve sample
representativeness (compared to
simple random sampling)
– Especially if the population is
geographically diverse and/or the
sample is too small
* Less costly and less time
consuming (depending on the
number of stages however)
Disadvantages
* The representativeness of the
sample may be compromised if
– Too few clusters are selected
and/or
– Clusters are too specific and/or
– Clusters contain too few
individuals

Question 23

What is Systematic Sampling?

Answer 23

• Sample selected according to some simple, systematic rule,
  but not randomly
• Sample may end up being equivalent to a simple random
  sample, provided there was no biasing pattern in the system
  of selection

Question 24

What is the Systematic Sampling procedure?

Answer 24

Question 25

What are the advantages and disadvantages of Systematic Sampling?

Answer 25

*Advantages* * An acceptable, more convenient, alternative approach if for some reason random sampling is not possible * Faster and possibly also cheaper *Disadvantages* * The representativeness of the sample may be compromised if the system of choice selects individuals in a non-random fashion

Question 26

What is Proportional Quota Sampling?

Answer 26

* Same principle as stratified random sampling – The sample is selected on a weighted manner based on predefined strata (distinct population subgroups) * Strata instead of being filled by random sampling, they are filled by non-random sampling (systematic or other) – For example, if a total sample size of 1000 is required and the population consists of 40% women and 60% men, then (non-random) sampling will continue until these percentages are obtained and the overall sample quota met

Question 27

What is the Proportional Quota Sampling procedure?

Answer 27

*Advantages* * An acceptable, more convenient, alternative approach if for some reason stratified random sampling is not possible * Compared to simple systematic sampling, could ensure the original population structure as it uses predefined population strata *Disadvantages* * The representativeness of the sample may be compromised as individuals are selected in a nonrandom fashion

Question 28

What is convenience sampling?

Answer 28

* Convenience sampling is the most frequent example of non-probability sampling * Individuals are selected in a non-random fashion, solely based on convenience (i.e. they are easy to access)

Question 29

What is the Convenience Sampling procedure?

Answer 29

Question 30

What are the advantages and disadvantages of Convenience Sampling?

Answer 30

*Advantages* * Cheap, fast and convenient *Disadvantages* * The representativeness of the sample will definitely be compromised as individuals are selected in a nonrandom fashion

Question 31

How do you know which sampling method to choose?

Answer 31

* Depends on: – The aim of the study – The nature of the source population – The sample size – Other practical issues (i.e. financial resources, time availability, etc.) * When no financial and time constrains exist: – Always strongly advised to use probability (random) sampling techniques in order to minimize selection bias – Stratified random sampling is the ideal method if the sample is small * When non-random sampling techniques have been used: – The representativeness of the sample is always questionable – Assume that selection bias is operating at some extent

Question 32

How does sampling method affect descriptive research?

Answer 32

In descriptive research (i.e. investigating the prevalence of a disease in a population): – Extremely important to have a perfectly representative sample, as selection bias will greatly influence the findings

Question 33

How does sampling method affect analytic research?

Answer 33

In analytic research (i.e. investigating exposure-outcome associations): – Minor deviations from a perfectly representative sample may be acceptable * Minor selection bias may not affect the findings at a large extent

Question 34

Which sampling method is not prefered?

Answer 34

Convenience Sampling

Question 35

What are the 2 types Systematic Error (bias)?

Answer 35

1. Selection bias: Systematic error arising from mistakes conducted during the selection of the study sample. 2. Information bias: Systematic error arising from mistakes conducted during the measurement of key study variables (exposure and outcome).

Question 36

What is information bias?

Answer 36

* Information bias arises from wrong / inaccurate assessment of either the exposure or the outcome variables * Such mistakes may arise from the researchers’ part (unintentionally) or from the participants’ part (unintentionally or intentionally) * There is also instrument bias (fault of the instrument) which falls under researcher’s part

Question 37

What is assessor bias?

Answer 37

* Wrong/inaccurate diagnosis due to a clinical error * May occur when researchers are not “blinded” to exposure or outcome status of participants * Wrong/inaccurate measurements due to a faulty instrument/machine * Wrong/inaccurate measurements due to poor training of assessor * Mistakes during recording of the data and transferring data from paper form into electronic form

Question 38

How can information bias arise from participant action / misinterpreting?

Answer 38

* Wrong/inaccurate answers from participants due to misinterpretation of a question * Wrong/inaccurate answers from participants due to a sensitive issue relating to the question * Wrong/inaccurate answers from participants due to poor recall (recall bias) * Wrong/inaccurate answers from participants intentionally * Overall, information bias arising from participant actions is called response bias

Question 39

What are the 6 types of information bias?

Answer 39

1. Recall Bias 2. Interviewer Bias 3. Observer bias 4. Hawthorne effect 5. Surveillance bias 6. Misclassification bias

Question 40

What is recall bias?

Answer 40

Those participant with a particular outcome or exposure may remember events more clearly or amplify their recollections – very common in case-control studies- the primary difference arises more from under-reporting of exposures in the control group rather than over reporting in the case group

Question 41

What is interviewer bias?

Answer 41

A researcher’s knowledge may influence the structure of questions and the manner of presentation, which may influence responders – any study design (especially if they are not blinded to exposures)

Question 42

What is observer bias?

Answer 42

Researchers may have preconceived expectations of what they should find in an examination (especially if they are not blinded to exposures or medical history)

Question 43

What is the Hawthorne effect?

Answer 43

Participants act differently if they know they are being watched.

Question 44

What is Surveillance bias?

Answer 44

The group with the known exposure or outcome may be followed more closely or longer than the comparison group (researcher’s bias).

Question 45

What is Misclassification bias?

Answer 45

Errors are made in classifying either disease or exposure status (instrument).

Question 46

What are the two types of errors?

Answer 46

1. Systematic error: a. Information error b. Selection error 2. Random error

Question 47

How can you minimize bias?

Answer 47

* Be purposeful in the study design to minimize the chance for bias; Example: use more than one control group * Define, a priori, who is a case or what constitutes exposure so that there is no overlap; Define categories within groups clearly (age groups, aggregates of person years) * Set up strict guidelines for data collection – Train observers or interviewers to obtain data in the same fashion – It is preferable to use more than one observer or interviewer, but not so many that they cannot be trained in an identical manner – Optimize questionnaire

Question 48

How does information bias affect study results?

Answer 48

1. Fundamental principle of research: If you want to investigate any association between two factors, first make sure you measure these two factors accurately! 2. Information bias can be introduced in the assessment of both the main exposure and the main outcome, thus the association between them will definitely be distorted 3. Information bias arising from participant actions is much more common compared to information bias arising from researcher actions 4. Information bias affects mainly studies that rely on self-reports (i.e. questionnaire-based data collection) – In outcome assessment (measurement), in studies where self-reported disease status is used, there is usually double-checking (confirmation) with the personal GP of the participant or through medical records – Similarly, while assessing exposures (diet, physical activity, smoking, educational attainment, etc.), the most valid and reliable instruments have to be used 5. If a study relies solely on self-reports, then it should be assumed that information bias (measurement error) is operating to some extent 6. The presence of information bias always compromises the validity of the study results and in such a case, findings have to be interpreted with great caution

Question 49

What should all assessment tools have?

Answer 49

1. Validity 2. Reliability

Question 50

What is validity?

Answer 50

The extent to which an assessment tool (e.g. questionnaire, instrument, etc.) measures accurately what it is intended to measure

Question 51

What is criterion validity?

Answer 51

Criterion validity is the most common type of validity used in medical research. In such a case, the results from the assessment tool of interest are compared with those of an established (known as gold standard) assessment tool

Question 52

What is reliability?

Answer 52

The overall consistency of a measure, as regards producing the same results when administered under the same conditions in the same group of people. Also known as reproducibility or repeatability

Question 53

What are the two main types of reliability?

Answer 53

1. Inter-observer reliability: The degree of agreement between the results when two or more researchers (observers) administer the assessment tool on the same people under the same conditions 2. Intra-observer reliability: Describes the agreement between results when the assessment tool is used by the same researcher (observer) on two or more occasions (under the same conditions and in the same test population)

Question 54

What is internally valid?

Answer 54

If a determination is made that the findings of a study were not due to any one of these three sources of error, then the study is considered internally valid. In other words, the conclusions reached are likely to be correct for the circumstances of that particular study.

Question 55

What is external validity?

Answer 55

This does not necessarily mean that the findings can be generalized to other circumstances (external validity)

Question 56

NB!

Answer 56

DO NOT COMPROMISE INTERNAL VALIDITY IN THE GOAL OF GENERALISATION