AI Concepts category
Fine-Tuning & Model Adaptation
Direct Preference Optimization
Direct Preference Optimization trains a model from preferred and rejected responses using a direct preference objective.
Fine-Tuning
Fine-tuning continues training a pretrained model on additional data to adapt its behavior for a downstream need.
LoRA
LoRA adapts a model by learning low-rank weight updates while keeping the original weights frozen.
Parameter-Efficient Fine-Tuning
Parameter-efficient fine-tuning adapts a model by training a small subset of parameters or additional parameters instead of updating the full model.
QLoRA
QLoRA trains LoRA adapters with a frozen, four-bit quantized base model to reduce fine-tuning memory use.
Reinforcement Learning from Human Feedback
Reinforcement Learning from Human Feedback uses human preference information to define a reward signal and optimize model behavior.
Supervised Fine-Tuning
Supervised fine-tuning adapts a pretrained model using input-output examples with target responses.