week 5 - optimizing design

Question 1

Lessons from Clinical Trials

Answer 1

• Standard of experimental designs = clinical trial → an experimental study in which 2+ treatments = assigned to human participants.
• Design of clinical trials = been refined b/c of costs of making a mistake w/ human = so high.
• Experiments on nonhuman subjects = “lab experiments”/“field experiments,” depend on location.
• Experimental studies in all areas of bio = been greatly infoed by procedures used in clinical trials.

Question 2

Lessons from Clinical Trials - Design Components

Answer 2

• A good experiment = designed with 2 objectives:
  
  • Reduce bias in esting & testing treatment effects
  • Reduce the effects of SE
• Most sig. elements in the design of the clinical trial addressed these 2 objectives.
• Reduce bias by:
  
  • A simultaneous control group
  • Randomization
  • Blinding
• Reduce the effects SE
  
  • Replication
  • Balance
  • Blocking
    Goal of experimental design = eliminate bias & reduce SE when esting & testing the effects.

Question 3

Lessons from Clinical Trials - Experimental Units (EUs)

Answer 3

• An indiv subject/group of indivs that = been assigned an experimental treatment independently of other indivs/groups.

Question 4

Reducing Bias - Simultaneous Control Groups

Answer 4

• Set of EUs that = treated the same way as all of the EUs but don’t receive the treatment of interest.
• Uncontrolled experiment → a group of subjects = treated & measured to see how they = responded.
• Lacking a control group for comparison = not determine whether the treatment of interest = cause of any of the obsed changes.
• Participants selected may tend to “bounce back” toward their avg condition regardless of any effect of the treatment
• Stress & other impacts assoced w/ administering treatment might themselves produce a response separate from the effect of the treatment of interest.
• Human participants often improves after treatment merely b/c of expectation that treatment = effect.
• Solution = include a control group of subjects measured for comparison.
• The treatment & control subjects should be tested simultaneously/in rando order to ensure that any temporal changes in experimental conditions don’t affect the outcome.

Question 5

Reducing Bias - Randomisation

Answer 5

• Researcher should randomize their assignment to EUs/subjects in the sample.
• Treatments = assigned to units at rando
• Chance rather than un/conscious decision determines which units = treatment & control
• A completely randomized design = experimental design in which treatments = assigned to all units by randomization.
• Breaks the assoc. betw possible confounding variables & explan variable → allows relationship betw explan & resp variables.
• Doesn’t eliminate variation contributed by confounding variables, only corr w/ treatment.
• Ensures that variation from confounding variables = spread more evenly betw the diff treatment groups & creates no bias.
• Any remaining influence of confounding variables occurs by chance alone → stats methods = account for.
• Use random-number generator/tables to assign indivs randomly to treatments.

Question 6

Reducing Bias - Blinding

Answer 6

• Concealing info from participants & researchers about which of them receive which treatments
• Prevents participants & researchers from changing behavior, un/consciously based on knowledge of which treatment they = given.
• Single-blind experiment → participants don’t know which treatment they = getting
  
  • Prevents participants from responding differly according to their knowledge of their treatment.
• Double-blind experiment → researchers & participants = unaware of treatment groups
  
  • Prevents researchers who interact w/ the participants from behaving differly toward them b/c of treatments.
• Researchers = pet hypotheses, & might treat experimental subjects in diff ways depending on hopes for outcome.
• Many resp variables = difficult to measure & require some subjective interpretation → makes results prone to bias in favor of the researchers’ wishes & expectations.
• Naturally more interested in the treated subjects than control subjects,
  
  • Increased attention = result in improved response.

Question 7

Reducing SE - Replication

Answer 7

• Repetition of every treatment on multi experimental units = essential.
• W/o replication → unknown whether response diffs = b/c of treatments/chance caused by other factors.
• Large sample sizes = smaller SEs & higher prob of getting the correct answer from a hypothesis test.
• Large samples = more info = better estimates & more powerful tests.
• Application of every treatment to multi, independent EUs
• Erroneously treating the single organism as the independent replicate when the chamber/field plot = EU = lead to calcs of SEs & p-values that = too small.
• More replication = always better.
• Increased precision yields narrower CIs & more powerful tests of the diff betw means.
• Increasing sample size also = costs in terms of time, money, & lives.

Question 8

Reducing SE - Balance

Answer 8

• All treatments = the same sample size.
• Better balance → SE = much smaller.
• Need precise ests of the means of both groups.
• Allocates the sampling effort in optimal way.
• Precision of an est of a diff betw groups always increases w/ larger sample sizes

Question 9

Reducing SE - Blocking

Answer 9

• Accounts for extraneous variation by dividing EUs into clusters/groups AKA blocks/strata
• Each of whose units share the same location/common features.
• Treatments = assigned randomly to EUs resulting in their interspersion.
• Repeats the same completely randomized experiment multiple times, 1x each block
• Diffs betw treatments = eval only within blocks.
• Much of the variation arising from diffs betw blocks = accounted for & won’t reduce the power of the study.
• Paired designs
  
  • 2 treatments = applied to each plot/other type of block.
  • Diff betw the 2 responses made on each block = measurement of the treatment effect.
• Units within blocks = relatively homogeneous, apart from treatment effects,
• Units in diff blocks vary b/c of envir/other diffs.
• Limitation = Effects of 1 treatment contaminate the effects of other in same block.

Question 10

Reducing SE - Extreme Treatments

Answer 10

• Treatment effects = easiest to detect when large.
• Small diffs betw treatments = difficult to detect & require larger samples,
• Larger diffs = more likely to stand out against rando variability in treatments.
• Larger dose = increase the prob of detecting a response.
• Effects of a treatment do not always scale linearly w/ the magnitude of a treatment.
• The effects of a large dose may differ from those of a smaller, realistic dose.

Question 11

Experiments w/ 1/+ Factors

Answer 11

• A factor = single treatment variable whose effects = of interest to the researcher.
• Many experiments investigate 1+ factor, b/c answering 2 qs from a single experiment = more efficient
• Experiments w/ multi factors = that the factors might interact.
• Factors might = synergistic/inhibitory effects not seen when each factor = tested alone.

Question 12

Experiments w/ 1/+ Factors - Factorial Design

Answer 12

• Most common experimental design used to investigate 1+ treatment variable, or factor, @ same time.
• Every combo of treatments from 2/+ treatment variables = investigated.
• Evals possible interactions betw variables.
• Interaction betw 2 explan variables = effect of 1 variable on the resp depends on 2nd variable.

Question 13

What if you can’t do the experiments?

Answer 13

• Obs studies = be v important, b/c detect patterns & help generate hypotheses.
• Best obs studies combos all of the features of experimental design used to minimize bias & SEs
• Randomization = out of the question b/c researcher does not assign treatments to subjects.
• Subjects come as they are.

Question 14

What if you can’t do the experiments? - Match & Adjust

Answer 14

• Used to limit the effects of confounding variables on a diff betw treatments
• Matching → With every indiv in the target group w/ a disease/ other health condition = paired w/ a healthy individual who = same
  
  • Used when designing case-control studies.
  • Reduces bias by limiting the contribution of suspected confounding variables to diffs betw treatments.
  • Doesn’t eliminate bias.
  • Reduces SE by grouping EUs into sim pairs, analogous to blocking in experimental studies.
• Adjustment → stats methods e.g. ANCOVA = used to correct for diffs betw treatment & control groups in suspected confounding variables.
  
  • Doesn’t effectively limit the role of confounding variables unless treatment & control overlap broadly in the freq dist of measurements for all confounding variables.

Question 15

Choosing Sample Size

Answer 15

• # of EUs to include = be chosen so as to achieve the desired width of CI for the diff betw treatment means.
• # of EUs to include = be chosen so that the probability of rejecting a false H0 (power) = high for a specified magnitude of the diff betw treatment means.
• Compensate for possible data loss when planning sample sizes