Information theory: entropy, cross entropy, KL
In the next lesson you will use cross entropy as a loss function. First measure what that thing actually is.
4 steps
205 XP
A free account is needed
Start the lesson →
Sources
- Shannon, C. E. 1948 · A Mathematical Theory of Communication · Bell System Technical Journal, 27
- Cover, T. M. & Thomas, J. A. 2006 · Elements of Information Theory · 2nd ed., Wiley
- MacKay, D. J. C. 2003 · Information Theory, Inference, and Learning Algorithms · Cambridge University Press
ML Academy · an interactive machine learning course that runs in your browser ·
All lessons