Self-supervision: making the label out of the data
If there is no label, invent one. In this setup the raw counts carry provably zero information about the topic, so all of the gain comes from the pretext task.
4 steps
175 XP
A free account is needed
Start the lesson →
Sources
- Devlin, J. et al. 2019 · BERT: Pre-training of Deep Bidirectional Transformers · NAACL 2019
- Chen, T. et al. 2020 · A Simple Framework for Contrastive Learning of Visual Representations · ICML 2020
- Balestriero, R. et al. 2023 · A Cookbook of Self-Supervised Learning · arXiv:2304.12210
ML Academy · an interactive machine learning course that runs in your browser ·
All lessons