Why it exploded now: data, compute, algorithms
We answer this question with an experiment rather than a slide. On one problem we pull the three levers separately and together, and measure the difference.
4 steps
175 XP
A free account is needed
Start the lesson →
Sources
- Hernandez, D. & Brown, T. 2020 · Measuring the Algorithmic Efficiency of Neural Networks · OpenAI
- Sutton, R. 2019 · The Bitter Lesson · incompleteideas.net
- Kaplan, J. et al. 2020 · Scaling Laws for Neural Language Models · arXiv:2001.08361
ML Academy · an interactive machine learning course that runs in your browser ·
All lessons