Domain-specific models: generalist or specialist
The answer to that question is not a preference but a measurable threshold. And what sets the threshold is how similar the domains are to each other.
4 steps
225 XP
A free account is needed
Start the lesson →
Sources
- Gururangan, S. et al. 2020 · Don't Stop Pretraining: Adapt Language Models to Domains and Tasks · ACL 2020
- Lee, J. et al. 2020 · BioBERT: a pre-trained biomedical language representation model · Bioinformatics 36(4)
- Bommasani, R. et al. 2021 · On the Opportunities and Risks of Foundation Models · arXiv:2108.07258
- Rosenstein, M. T. et al. 2005 · To Transfer or Not To Transfer · NeurIPS 2005 Workshop on Transfer Learning
ML Academy · an interactive machine learning course that runs in your browser ·
All lessons