The computation graph: how the derivative flows
Backpropagation is not magic, it is applying the chain rule backwards over a graph. We compute the derivatives on this page three separate ways and compare them.
4 steps
175 XP
A free account is needed
Start the lesson →
Sources
- Griewank, A. & Walther, A. 2008 · Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, 2. baskı · SIAM
- Baydin, A. G. et al. 2018 · Automatic Differentiation in Machine Learning: A Survey · JMLR 18(153)
- Chen, T. et al. 2016 · Training Deep Nets with Sublinear Memory Cost · arXiv:1604.06174
ML Academy · an interactive machine learning course that runs in your browser ·
All lessons