AI Concepts category
Safety & Security
Direct Prompt Injection
Direct prompt injection places adversarial instructions in user-controlled prompts to override higher-priority instructions or intended application behavior.
Guardrails
Guardrails are controls around an AI system that constrain, validate, block, or mediate inputs, outputs, and actions.
Indirect Prompt Injection
Indirect prompt injection reaches a model through untrusted external content rather than the user's direct instruction.
Jailbreaking
Jailbreaking attempts to bypass restrictions on a model's outputs or behavior through crafted prompts or interactions.
Model-Level Defense
Model-level defenses are learned behaviors that help a model resist certain unsafe requests or conflicting instructions.
Prompt Injection
Prompt injection is an attempt to make untrusted input influence model behavior beyond that input's authority.
Prompt-Level Defense
Prompt-level defenses use instructions and input structure to reduce the influence of untrusted content on model behavior.
Reverse Prompt Engineering
Reverse prompt engineering attempts to reconstruct an unknown prompt from observable model outputs.
System-Level Defense
System-level defenses are controls outside the model that limit access or check proposed actions.