AI engineering reference
This reference library separates the vocabulary of AI systems into small, connected ideas. Browse by topic, then follow explicit relationships when one concept depends on understanding another.
This reference is an evolving collection. I add new concepts over time and update existing explanations as terminology and engineering practice evolve. If a concept becomes outdated or its meaning changes materially, I prefer to mark that clearly instead of silently removing the URL.
Core concepts behind modern foundation models and how they work.
Attention combines information from available representations using input-dependent weights.
An autoregressive language model generates a sequence by predicting each next token from the tokens that precede it.
A foundation model is pretrained broadly enough to serve as a reusable base for multiple downstream applications or adaptations.
A mixture-of-experts model combines expert subnetworks through a routing mechanism, often using only a subset for each input or token.
Tokenization converts input into discrete token identifiers that a model can process.
A transformer is a neural-network architecture built from attention, feed-forward layers, and repeated processing blocks.
Concepts that shape model inputs and how outputs are generated.
Chain-of-thought is intermediate reasoning text generated as part of producing an answer, often elicited through prompting.
A context window is the bounded amount of input and generated tokens a model can process in one request.
In-context learning is adapting model behavior through instructions or examples in the current request without updating model weights.
Structured outputs constrain model responses to a machine-readable structure, often specified with JSON Schema.
System prompts shape model behavior through instructions or context supplied apart from user input.
Temperature adjusts the sharpness of the token probability distribution used during sampling.
Top-k sampling restricts token selection to the fixed number of highest-probability candidates at each generation step.
Top-p sampling restricts token selection to the smallest set of highest-probability tokens whose cumulative probability reaches a threshold.
Concepts for finding external information and supplying useful context to models.
Chunking divides source content into smaller units that can be indexed, retrieved, and placed in model context.
An embedding is a numerical representation that places an input in a learned vector space.
Query rewriting reformulates a request so a retrieval system can find more relevant content.
Reranking reorders retrieved candidates using a second, usually more expensive relevance step.
Retrieval top-k is the number of highest-ranked results requested or retained from a retrieval stage.
RAG retrieves relevant information at runtime and supplies it as context before a model generates an answer.
A vector database stores vectors and metadata and provides indexed similarity search over them.
Vector search finds items whose stored vectors are close to a query vector.
Concepts for systems where models choose actions and interact with external capabilities.
Agent memory is information an agent system retains or retrieves for use across model invocations.
Agent planning is the decomposition of a goal into intermediate steps or actions before or during execution.
Agent reflection uses a model to explicitly evaluate earlier outputs, actions, observations, or outcomes before an agent continues or revises its approach.
An AI agent is a system in which a model helps choose actions within an orchestrated execution loop.
Tool calling lets a model request a named operation with structured arguments for an application or runtime to evaluate and execute.
An AI system can choose the tool that fits the next step, or continue without one.
Concepts for measuring AI-system behavior and output quality.
AI-as-a-Judge uses a model to evaluate outputs against defined criteria, references, or competing responses.
Comparative evaluation judges outputs or systems directly against one another rather than assessing each independently.
An evaluation harness runs cases against an AI system in a repeatable way and records outputs, measurements, and run configuration.
Factual consistency assesses whether an output's claims are compatible with the reference information being evaluated.
Faithfulness measures whether an answer's claims are supported by the evidence supplied to the model.
Reference-based evaluation compares a system output with known answers, evidence, or other designated reference data.
A scoring rubric defines criteria and score levels that guide evaluation of an output.
Concepts for adapting pretrained models to tasks, domains, or preferences.
Direct Preference Optimization trains a model from preferred and rejected responses using a direct preference objective.
Fine-tuning continues training a pretrained model on additional data to adapt its behavior for a downstream need.
LoRA adapts a model by learning low-rank weight updates while keeping the original weights frozen.
Parameter-efficient fine-tuning adapts a model by training a small subset of parameters or additional parameters instead of updating the full model.
QLoRA trains LoRA adapters with a frozen, four-bit quantized base model to reduce fine-tuning memory use.
Reinforcement Learning from Human Feedback uses human preference information to define a reward signal and optimize model behavior.
Supervised fine-tuning adapts a pretrained model using input-output examples with target responses.
Concepts for running models efficiently and understanding serving performance.
Continuous batching dynamically schedules active inference sequences together as requests arrive, progress, and finish.
A KV cache stores attention key and value representations from earlier tokens for reuse during autoregressive generation.
Prompt caching reuses computation for repeated prompt prefixes when a model-serving system supports it.
Quantization limits model values to a smaller set, often reducing inference memory use.
Speculative decoding can accelerate generation by proposing tokens cheaply and verifying them with the target model.
Time per Output Token measures the ongoing latency of producing output tokens after generation has begun.
Time to First Token describes the delay to a request's first generated token, with timing boundaries that vary by measurement.
Concepts for adversarial inputs, unsafe behavior, and controls around AI systems.
Guardrails are controls around an AI system that constrain, validate, block, or mediate inputs, outputs, and actions.
Indirect prompt injection reaches a model through untrusted external content rather than the user's direct instruction.
Jailbreaking attempts to bypass restrictions on a model's outputs or behavior through crafted prompts or interactions.
Prompt injection is an attempt to make untrusted input influence model behavior beyond that input's authority.
Reverse prompt engineering attempts to reconstruct an unknown prompt from observable model outputs.