Definition
An adapter is a small trainable module added to a pretrained model. During adaptation, the original model weights usually stay frozen while training changes the adapter’s weights. A model can then use the same base weights with different adapters for different tasks.
In a common transformer design, an adapter sits inside a layer and processes its hidden states through a narrow bottleneck before passing the result back to the main computation. This adds a small number of trainable parameters compared with updating the whole model. The exact placement and structure vary by method.
Simple example
Suppose a team uses one language model for two tasks: classifying support tickets and summarizing incidents. It trains one adapter on reviewed ticket labels and a second on incident summaries. The base model stays the same. For each task, the application loads the matching adapter.
The team can store the two adapters separately instead of saving two complete fine-tuned models. Each adapter still needs its own evaluation on the task it was trained for.
Why it matters
Training only a small module usually requires optimizer state and parameter gradients for far fewer weights than full fine-tuning. Separate adapters also make it easier to keep several task adaptations without duplicating all base weights.
An adapter must fit the target model’s insertion points and tensor shapes. A different checkpoint can have the same structure but different base weights, which may change the adapter’s behavior. Evaluate that combination before using it. The serving system also needs to load the right adapter for each task. A bottleneck adapter adds computation during inference, so fewer trainable parameters do not necessarily make responses faster.
One important nuance
“Adapter” can refer to more than one kind of added component. A bottleneck adapter transforms hidden states inside a layer. A LoRA adapter instead learns a low-rank update for selected weight matrices, which can be merged into those weights for deployment. Both are parameter-efficient fine-tuning (PEFT) methods. PEFT also includes methods that do not add an adapter module.