Chapter 3.2: Big Data - Multiple testing

Question 1

R:

How do you generally 101 variables for a sample size of 100 of standard normal data entries that are independent?

Answer 1

Question 2

How to I analyse this data in R?

How would I plot the linear regression model?

Answer 2

• Not also V6 look significant based on the F-value

Question 3

What is voodoo correlation and why is it a caution of multiple testing?

Answer 3

• If you set the significance level at 5% then out of 100 indepedent uniform samples you would expect 5 of them will fall below 0.05 –> this is simply by chance not necessarily because of any significant correlation
• Have to change the significance level or define significance in another way

Question 4

Give the null hypothesis: the defendent is guilty, what are Type I and Type II errors in this case?

Whis error is the worse and the one we try to control?

How do theser relate to False Positives and Negatives?

Answer 4

Question 5

Building upon from type I and II errors, what is the notion of the Family-wise error rate?

Answer 5

• Want to make as few False rejections as possible
• Probability you are NOT going to reject is (1-α-tilda), and hence m indepedent tests this is raised the the power of m –> Probability you are NOT going to reject all of them

*

Question 6

What is the can we do to reduce the FWER?

Answer 6

Question 7

What is the proof of this?

Answer 7

1. union of all H_0,i that are true –> using the union becuase if any of these happen you get at least one rejection of a true null.
2. Union bound argument –> Fundamental Tool countable union rules of a probability measures
3. Convert into P-values –> reject the null of the p-values is < α/m
4. When the null-hypothesis is true, the associated P-value will follw a uniform 0-1 –> hence the probability P_i is less than or equal to α/m is simply α/m.
5. Summing over all true nulls totals to m₀

Question 8

What is the False discovery rate (FDR)? What is the False discovery proporition (FDP)?

Answer 8

• FDP = number of FP over all rejections –> random number

Normally get more rejection for FDR than for the family-wise error rates especially when p is big

Question 9

Approach approach do we employ to control the FDR?

Answer 9

• Ramps/sorts all the p-values in order from smallest to largest
• What is the last time the sorted p-values <= α * (i/m)
• Reject all p-values smaller than the cutoff value