LoRA: adapting a model without training all of it
The transformer lesson said "this is why LoRA and quantisation exist". The reason gets measured here.
4 steps
210 XP
A free account is needed
Start the lesson →
Sources
- Hu, E. J. et al. 2021 · LoRA: Low-Rank Adaptation of Large Language Models · arXiv:2106.09685 · ICLR 2022
- Aghajanyan, A., Zettlemoyer, L. & Gupta, S. 2021 · Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning · ACL 2021
- Dettmers, T. et al. 2023 · QLoRA: Efficient Finetuning of Quantized LLMs · NeurIPS 2023
ML Academy · an interactive machine learning course that runs in your browser ·
All lessons