A/B testing: online evaluation
An eval set measures offline. Measuring with real users has its own arithmetic, and the most common mistake is looking at the interim result.
4 steps
225 XP
Open without an account
Start the lesson →
Sources
- Kohavi, R., Tang, D. & Xu, Y. 2020 · Trustworthy Online Controlled Experiments · Cambridge University Press
- Gelman, A. & Carlin, J. 2014 · Beyond Power Calculations: Assessing Type S and Type M Errors · Perspectives on Psychological Science, 9(6)
- Johari, R. et al. 2017 · Peeking at A/B Tests: Why It Matters and What to Do About It · KDD 2017
ML Academy · an interactive machine learning course that runs in your browser ·
All lessons