Definition
Parameter-Efficient Fine-Tuning, commonly shortened to PEFT, is a family of methods that adapts a pretrained model while training far fewer parameters than full-parameter fine-tuning. A method may add small trainable components, update selected existing parameters, or learn compact prompt-like representations while most base-model weights remain frozen.
PEFT describes how adaptation is restricted to a relatively small set of trainable parameters. The training objective can still be supervised learning, preference optimization, or another adaptation objective.
Simple example
A team wants separate model adaptations for support triage and incident summarization. Instead of training and storing two complete copies of the base model, it trains a small adapter for each task while keeping the base weights unchanged.
At deployment, the serving system loads the compatible base model and selected adapter. The team versions and evaluates each adapter with the base-model version used during training.
Why it matters
Updating fewer parameters can reduce training memory, optimizer state, storage, and the cost of maintaining multiple task-specific adaptations. It can make experimentation possible on hardware that cannot support full fine-tuning.
The smaller trainable footprint does not remove the need for dataset governance, evaluation, model provenance, or deployment compatibility checks.
One important nuance
PEFT is a category, not a single technique. LoRA is one PEFT method, while other methods use different trainable components or parameter-selection strategies. Fewer trainable parameters also does not guarantee equal quality, faster inference, or lower end-to-end operational cost. Quality depends on the task, data, model, and training setup. Inference speed and cost depend on how the adaptation is served.