Starter guide
LoRA and QLoRA
A language model learns by changing millions or billions of numbers called weights. Updating every weight uses a lot of GPU memory because training must keep the model and its updates in memory at the same time.
LoRA leaves the model's existing weights unchanged. It adds a small set of trainable weights to selected parts of the model. These additions form a LoRA adapter. During fine-tuning, the adapter learns the new task while the original model stays fixed.
How is QLoRA different?
QLoRA uses the same adapter approach and stores the fixed model in 4-bit form during training. This uses much less GPU memory than storing the model with 16-bit numbers. The adapter is trained with higher precision so it can still learn useful changes.
After training, you usually save the adapter instead of another full copy of the model. To use it later, load the same original model and attach the adapter. You can keep different adapters for different tasks without saving the full model each time.