Google-Specifig A/B Testing

Question 1

In Google-scale experiments, why is SRM so common?

Answer 1

Because experiments run on heterogeneous systems: many surfaces, platforms, client versions, and logging pipelines.

Question 2

Why does Google emphasize CUPED?

Answer 2

Because variance is massive at global scale, so variance reduction significantly shortens experiments.

Question 3

Why must SRM checks be run BEFORE analyzing experiment metrics?

Answer 3

To avoid interpreting differences caused by bad assignment or logging — not by treatment.

Question 4

Google asks: “SRM in events but not assignments — why?”

Answer 4

Downstream logging or joining issue; assignment is correct but some events dropped.