Section: internal & external study validity

Question 1

What is internal validity?

Answer 1

Observed relationship between variables in study is causal & not due to confounding variables, biases, or other factors.

Question 2

What is external validity?

Answer 2

• Can our results be generalized to the general population or to other similar populations?
• Example: If we observe an exposure-outcome association in our study sample, can we expect to see the same association in the general population or other similar populations?

Question 3

A study found that among adolescents in Nicosia, screen time is positively associated with BMI. researchers wish to collect similar data in adolescents residing in other otwns in cyprus to test whether this association is found in other similar populations. What are the researchers testing?

A. The role of bias in their results.
B. Whether the internal validity of their original study is high
C. Whether the external validity of their original study is high
D. None of the above
E. Whether their results are affected by confounding

Answer 3

c - external validity

Question 4

How do we judge internal validity?

Answer 4

determine whether our results are influenced by any of the following 3 factors:
1. Chance (random/sampling error)
2. Bias (systematic error)
3. Confounding

Question 5

How do we determine whether the validity of our results is affected by chance?

Answer 5

• Check findings for statistical significance (based on p-value and/or the 95% Confidence Intervals).
• If exposure-outcome association is statistically significant, then we can conclude with relative safety
  that our finding is not by chance (i.e. there is indeed such an association in the source population).

Question 6

For example, if looking at Association between obesity and risk of hepatic cancer
* Risk Ratio: 1.70 (95% CI: 1.55 - 1.85). is the result affected by chance?

Answer 6

Statistically significant - Very unlikely that this is a chance finding.

Question 7

If Association between obesity and LDL-cholesterol levels
* Mean difference: 2.2 (95% CI: -1.1 - 5.5). Is valdiity of results affected by chance?

Answer 7

Not statistically significant - We cannot exclude the possibility of this
finding being due to chance.

Question 8

How can we minimize the influence of chance on the validity of our findings?

Answer 8

• By taking a large representative sample!
• large sample will reduce standard error and increase study power, thus
  giving more precise estimates.

Question 9

What are the types of bias?

Answer 9

• selection
  info

Question 10

What is selectiom bias?

Answer 10

Errors in the process of sampling which result in selecting a non-
representative sample and consequently any derived estimate is likely biased.

Question 11

What is info bias?

Answer 11

Errors in the process of data collection, which result in inaccurate
assessment of the exposure and/or outcome variables.

Question 12

How do we determine whether the validity of our results is
affected by bias?

Answer 12

investigators need to make their own reflection and self-criticism regarding
the appropriateness of the sample chosen and the accuracy of the measures taken.

Question 13

What samples will always suffer from bias?

Answer 13

Samples collected using convenience sampling and/or measurements resulting from
participant self-reports should always be expected to be suffering from bias!

Question 14

How do we minimize the influence of bias?

Answer 14

• By choosing sample representative of source pop (minimizes selection bias).
• By choosing assessment tools which have high accuracy (minimizes
  information bias). The aim should be that assessment tools are both valid & reliable.
• By making thorough investigation of accuracy of the data collected. This should be done in the process of data collection but also during data analysis (data cleaning).

Question 15

In a study investigating the association between age and weight, the use of
a non-calibrated weighing scale would lead to which type of bias?
A. Information bias only
B. Selection bias only
C. Both information bias and selection bias
D. None of the above

Answer 15

A - ifno bias only

Question 16

what is a confounding factor?

Answer 16

A factor explains all or part of the association between an
exposure and an outcome.

Question 17

How can confouding compromise valdity of a study?

Answer 17

by distorting the
estimate of a potential association, which can even lead to masking a true
association or making a false association ‘appear’, when no true association exists.

Question 18

How do we determine whether the validity of our results is
affected by confounding?

Answer 18

• Under non-experimental conditions (i.e. in observational studies) it should be expected that confounding is affecting results to some degree.
• If an unexpected (based on previous evidence) result is obtained or something that does not make sense biologically, then we should be very suspicious that confounding effects are at play.

Question 19

How can we minimize the influence of confounding
in our findings?

Answer 19

• Make list of potential confounders for a given exposure-outcome association.
• Adjust for potential confounders during data analysis. This will give confounder-adjusted estimates, which are free (as much as possible) of confounding effects.
• Even after adjustment, always be cautious of residual confounding

Question 20

Which of the following methods is appropriate for minimizing the effects of confounding in a research study?
A. Adjusting for potential confounders in the statistical analysis
B. Selecting a larger representative study sample
C. Choosing the study sample using random sampling
D. Using calibrated equipment for all measurements made
E. None of the above

Answer 20

A

Question 21

What is external validity?

Answer 21

interested in whether our
findings can be generalized to:
1. the general population
2. to other similar populations.

Question 22

How do we determine whether our results have a low
external validity?

Answer 22

• This is matter of correct judgment of the researcher, based on whether study’s sample is representative or not of the general pop or similar pops in other countries.
• Going through literature & investigating available evidence on
  association of interest is always helpful for evaluating external validity
• Never assume that our findings
  are externally valid without careful thought and investigation first.

Question 23

How do we improve the external validity of our results?

Answer 23

• Avoid highly selected samples if we want to generalize our results to general pop. In such a case, choosing a random sample is the
  ideal approach.
• If intentionally chosen a pop with specific characteristics (i.e. male smokers, University students, etc.), then we should make sure that our
  sample accurately represents the specific population sub-group.

1. In such a case, generalizations to general pop should be avoided.
2. Generalizations to other similar pop could be done but always under certain assumptions.