Definition
Full fine-tuning continues training a pretrained model with all, or effectively all, of its existing parameters trainable. Updates can change weights throughout the model, rather than being limited to a small adapter or selected layers.
“Full” describes how much of the model can change. It does not specify the training objective. A full fine-tuning run might use supervised examples or preference data.
Simple example
A support team has labeled tickets. Each ticket has a target JSON record with a category and an escalation flag. The team fine-tunes a language model on these pairs with every existing parameter trainable. It saves the updated weights as a separate model version.
If the team used LoRA instead, the base weights would stay frozen and training would produce a much smaller adapter.
Why it matters
Full fine-tuning computes gradients for nearly every weight. Optimizers such as Adam also keep state for those parameters. Along with model weights and activations, this can require substantially more training memory than parameter-efficient fine-tuning (PEFT). Sharding and offloading can reduce the memory needed on each accelerator. Saving each adapted version as a full checkpoint also takes more storage than saving only adapter weights.
These costs matter when a team needs several versions or has limited accelerator memory.
One important nuance
More trainable parameters do not guarantee better results. Training on a narrow ticket dataset could hurt the model’s performance on other requests. Compare full fine-tuning and PEFT on held-out tickets, then test capabilities the model must retain. The choice depends on those results and the cost of training and maintaining each version.