ML Academy · Track 2 · Deep Learning

Activation functions

The thing that makes a network deep. And the reason for the biggest obstacle in front of deep learning until the 2010s.

2 steps 75 XP A free account is needed

Sources

Glorot, X. & Bengio, Y. 2010 · Understanding the Difficulty of Training Deep Feedforward Neural Networks · AISTATS 2010
Nair, V. & Hinton, G. 2010 · Rectified Linear Units Improve Restricted Boltzmann Machines · ICML 2010
He, K. et al. 2015 · Delving Deep into Rectifiers (PReLU / He başlatma) · ICCV 2015
Hendrycks, D. & Gimpel, K. 2016 · Gaussian Error Linear Units (GELU) · arXiv:1606.08415

ML Academy · an interactive machine learning course that runs in your browser · All lessons