Glossary

No terms match. Try a broader word like "agent" or "drift".

Action-Time Monitor Governance & Control

The enforcement point that watches a tool call while it executes: scope creep, side effects, runaway behavior. Unlike the pre-action gate it can interrupt mid-run, because not every violation is visible before execution starts.

Agent Handoff Agentic AI

The transfer of a task from one agent to another inside a multi-agent run. Every handoff is a context boundary: what the receiving agent sees is a design decision, not an accident.

Agent Loop Agent Internals

The repeating cycle an agent runs: perceive the environment, reason about options, plan the next step, act through tools, and learn from the outcome. Perception, reasoning, planning, action, and learning are its five components.

Agent Memory Agent Internals

The state an agent carries between steps and between runs: conversation history, retrieved context, prior outcomes. Memory is what makes the loop a loop, and it is also how stale or poisoned context persists into new decisions.

Agent Orchestration Agentic AI

The coordination layer that decides which agent gets which task, in what order, and under what budget. Orchestration is where multi-agent systems succeed or loop.

Agent Telemetry Observability

The instrumentation stream an agent emits: spans, eval scores, cost per call, flagged actions. Telemetry is what makes a run investigable; without it, 'the agent got stuck' is a week of archaeology.

Agentic AI Agentic AI

AI that pursues a goal rather than answering a prompt. Given an objective, the system plans its own steps, calls tools, adapts when something fails, and keeps working until the goal is met or it hands control back to a human. The user supplies the what; the system works out the how.

Agentic AI Governance Governance & Control

AI governance applied to systems that take actions. Because agents execute rather than answer, governance must run at the speed of the agent loop: enforcement at the tool boundary, escalation sized for judgment rather than volume, and evidence that reconstructs every consequential decision.

Agentic AI Observability Observability

Instrumentation that captures the full trajectory of an agent run: every model call, tool invocation, retrieval, guardrail check, and handoff, recorded in order with inputs, outputs, latency, and cost per step. Built for systems whose failures happen between API calls.

Agentic Drift Operations (LLMOps & AgentOps)

The gradual degradation of a production AI agent's behavior after deployment. Drift arrives through five channels: behavioral drift when models and prompts change, cost creep as usage grows, tool rot in the systems the agent calls, policy that has fallen behind the organization, and credential decay as access outlives its original scope.

Agentic Engineering Agentic AI

The development practice where the agent is the author and the human reviews the output: the agent plans, writes, and tests; the person approves a diff they did not watch get written. The contrast case to vibe coding.

Agentic Pattern Agentic AI

A reusable design for structuring an agent's loop, tools, or coordination: the recurring shapes that separate a working agentic system from a demo. Patterns name what works before a team rediscovers it the expensive way.

AgentOps Operations (LLMOps & AgentOps)

The operating discipline for agents in production: evals, manifests, cost bounds, credentials, cadence. A newer name for the same work as LLMOps; the discipline is identical either way.

AI Agent Agentic AI

A software system built around a language model that runs a loop: perceive the environment, reason about options, plan the next steps, act through external tools, and learn from the outcome. The model does the reasoning; the agent's code does the acting.

AI Agent Evaluation Evaluation

Measuring whether an agentic system is getting better or worse using graded criteria across its full trajectory, not pass/fail assertions on the final output. Agent evaluation scores behavior under change: every model bump, prompt edit, or tool update gets measured against a fixed set of real questions.

AI Code Review Delivery & Engineering

Automated pull-request review by a model that analyses beyond the diff: the function, the class, the imports, the call graph. A useful tool returns findings a human can act on: what the problem is, what happens if it ships, and a ready-to-apply fix. The human decides; the tool does not merge.

AI Documentation Generator Delivery & Engineering

A tool that produces and maintains docs from the code itself: references, onboarding guides, architecture notes that stay in sync because they regenerate when the code changes. The answer to documentation debt that manual writing cannot match.

AI Feasibility Study Strategy & Buying

A bounded engagement that answers whether an AI initiative is worth building before the build starts: data readiness, integration surface, expected ROI, and the failure modes that would kill it. The cheapest place to be wrong.

AI Governance Governance & Control

The organizational discipline that decides what AI systems may do, who is accountable when they do it, and how that accountability gets proven. It spans policy, risk frameworks, oversight design, and audit evidence: the layer above any single technical control.

AI Implementation Oversight Strategy & Buying

Independent technical review of an AI build while it is being delivered: architecture decisions, governance design, and vendor claims checked against what is actually being shipped. The control for teams buying AI work they cannot fully inspect themselves.

AI Readiness Assessment Strategy & Buying

A structured evaluation of whether an organization can absorb AI: data quality, pipeline maturity, team skills, governance posture. Readiness is the question that decides whether the project succeeds before it begins.

AI Sprint Acceleration Layer Delivery & Engineering

The layer above the IDE and below the sprint board: review, test generation, documentation, and governance working on the pipeline rather than the prompt. Where sprint throughput is actually decided once generation is fast.

AI Technology Selection Strategy & Buying

Choosing the models, frameworks, and infrastructure an AI system runs on. The selection ages faster than the system does, which is why the manifest and the eval gate matter more than the initial pick.

AI Test Generation Delivery & Engineering

Automated creation of executable tests by a model that analyses the code, decides what to test, writes the test code, and learns from the failures. The hard parts are not generating assertions; they are deciding what matters, generating mocks, and keeping coverage coherent as the code changes.

Approval Fatigue Governance & Control

The degradation of human oversight as approval volume grows. When every risky action pauses for a person, the reviewer rubber-stamps: catch rates for dangerous actions drop from 13.6% to about 5% across a long session, while escalating everything is measurably less safe than escalating selectively.

Autonomous AI Agent Agentic AI

An AI agent that works toward a goal without a human approving each step. Autonomy is a dial, not a switch: the operating envelope, not the model, decides how much the agent may do alone.

Behavioral Drift Operations (LLMOps & AgentOps)

The drift type where outputs shift while the system stays up: a model update, a prompt edit, a dependency change, and answers change in ways nobody diffed. The agent still responds; it responds differently.

Behavioral Eval Evaluation

Scoring what the agent did, not just what it returned: which tools it called, in what order, at what cost, and whether the path was sane. The eval tier that catches a correct answer reached through a pathological trajectory.

Blast Radius Governance & Control

The maximum damage an agent can do if everything goes wrong: the scope of data, systems, and money its credentials can reach. Scoped down at design time, it is the difference between an incident and a headline.

Build vs Buy (AI) Strategy & Buying

The decision between assembling AI capability in-house and buying it: control and differentiation against speed and maintained expertise. The highest-intent question in AI adoption, and the one a feasibility study should answer honestly.

Chunking RAG & Retrieval

Splitting source documents into indexed pieces before embedding. Chunk boundaries decide what retrieval can return: a clause cut off from its qualifying condition is a failure mode baked in at ingestion.

Circuit Breaker Governance & Control

An automatic halt that trips on a threshold: error rate, spend, loop count. Borrowed from distributed systems, it is the pattern that stops a $380 conversation and a $47K delegation loop without waiting for a human to notice.

Citation-Strict Generation RAG & Retrieval

A generation gate that requires every claim to trace to a retrieved passage: no source, no answer. It converts hallucination from a silent failure into a refusal the system can surface and measure.

Containment Governance & Control

Limiting what a compromised or malfunctioning agent can reach and do. Not network isolation: containment for agents means bounded credentials, bounded tools, and a bounded budget: the incident-response vocabulary for autonomy.

Cost Attribution Observability

Assigning spend to the step, run, agent, and tenant that caused it, rather than reading a monthly aggregate. The prerequisite for cost bounds: you cannot cap what you cannot attribute.

Cost Bounds Operations (LLMOps & AgentOps)

Hard limits on agent spend enforced at runtime: per-session caps, per-agent budgets, per-tenant attribution. Not cost alerts; bounds stop the spend, alerts just describe it.

Cost Creep Operations (LLMOps & AgentOps)

The drift type where call volume, context size, and retry counts grow gradually until the invoice discovers what the dashboards did not. A pattern that cost cents in testing costs dollars at scale.

Credential Decay Operations (LLMOps & AgentOps)

The drift type where tokens, keys, and service accounts outlive the scope they were issued for. Access meant for a pilot stays active in production months later, expanding the blast radius a little at a time.

Decision Record Observability

The audit artifact that survives an agent's run: for one output, on one date, given one input, a structured record of why the system produced this result. It captures what a sparse log cannot: the model version, the prompt in effect, the retrieved context, and any human oversight action taken.

Decommissioning Operations (LLMOps & AgentOps)

Retiring an agent deliberately: revoking credentials, archiving decision records, closing the eval set, and answering whether the system should still exist at all. The annual cadence item most operating models skip.

Document Intelligence RAG & Retrieval

Extracting structure from documents before they enter the index: tables keep their headers, clauses keep their conditions, layouts become data. The ingestion discipline that decides whether retrieval has anything worth finding.

Documentation Debt Delivery & Engineering

The accumulated gap between what the code does and what is written down about it. Compounds quietly until onboarding, incidents, or audits demand answers the docs cannot give, and AI-generated volume makes it grow faster.

Embedding RAG & Retrieval

A numeric vector representing a text's meaning, so similarity becomes distance. Retrieval quality inherits the embedding model's blind spots: what it cannot distinguish, search cannot find.

Escalation Router Governance & Control

The enforcement point that sends ambiguous cases to a human with a structured decision package: the proposal, the policy that flagged it, the context, and what the reviewer needs to decide. Escalation sized for judgment, not volume.

EU AI Act Governance & Control

The EU's binding AI regulation. Two articles matter most for agents: Article 14 requires human oversight capability designed into the system, and Article 12 requires logging sufficient to reconstruct what the system did.

Eval Drift Evaluation

The eval set aging away from what users actually ask. The suite keeps passing while production degrades underneath it, which is why the eval set itself is a scheduled review item, not a build-once artifact.

Eval Harness Evaluation

A fixed set of inputs with expected outputs or scoring criteria, run against an AI system on every change. The harness turns 'the agent seems worse lately' into a measurable regression: each release is scored against the same questions, and a drop below the agreed threshold blocks the change from shipping.

Feedback Loop Delivery & Engineering

The path by which a run's outcome informs the next decision: eval scores into prompts, failure patterns into test cases, incident findings into policy. Agents without feedback loops repeat their mistakes at machine speed.

Golden Dataset Evaluation

The fixed set of real questions and expected behaviors an eval harness scores every release against. Golden because it does not change: the yardstick stays constant so the system's movement becomes visible.

Grounding RAG & Retrieval

Constraining the model's answer to retrieved sources rather than its training memory. Grounded systems can point at what they used; ungrounded ones fill gaps with fluent invention.

Guardrail Governance & Control

A check on an agent's inputs, outputs, or actions that blocks or redirects unsafe behavior. The umbrella term; where a guardrail runs decides whether it is a control or a suggestion: inside the prompt it is decoration, at the tool boundary it is enforcement.

Hallucination Evaluation

A fluent, confident answer with no source behind it. The fix is architectural, not aspirational: retrieve real context, constrain generation to it, and refuse to answer when nothing was retrieved.

Human-in-the-Loop (HITL) Governance & Control

A design where a person reviews or approves agent actions. Effective only when humans see judgment calls, not throughput: at high approval volume the loop degrades into rubber-stamping and the control quietly stops working.

Hybrid Search RAG & Retrieval

Retrieval that runs keyword and vector search together and fuses the results. Keyword catches exact terms and identifiers; vector catches paraphrase and intent; the fusion covers what each misses alone.

ISO/IEC 42001 Governance & Control

The international standard for AI management systems: the certifiable wrapper around AI governance, covering policy, accountability, and continuous improvement. The ISO 27001 analogue for organizations that run AI.

Jailbreak Security

Manipulating a model into ignoring its safety instructions. For chatbots it produces bad output; for agents it produces bad actions, which is why enforcement at the tool boundary matters more than instruction hardening.

Kill Switch Governance & Control

The control that halts an agent mid-run. A real kill switch carries undo, not just stop: halting an agent mid-transaction without a rollback path can leave the system in a worse state than letting it finish.

LLM Observability Observability

Monitoring model calls: latency, tokens, errors, per-call cost. Necessary but not sufficient for agents: it sees each request while missing the run. The step below agentic observability.

LLM-as-Judge Evaluation

Using a model to score another model's output against a rubric. It catches quality failures deterministic checks miss, at a price: judge models cost money per run and drift as providers update them.

LLMOps Operations (LLMOps & AgentOps)

The operating discipline for AI systems in production: eval gates on every change, versioned manifests of models, prompts, and tools, cost bounds enforced before calls run, credential rotation, incident response, and a published maintenance cadence with a named owner.

Memory Poisoning Security

Corrupting the state an agent carries between runs: a poisoned memory entry steers every future decision without tripping any single-run check. The persistence layer's version of prompt injection.

MLOps Operations (LLMOps & AgentOps)

The classical discipline for running machine-learning pipelines: deterministic data flows, retraining cycles, model registries. It assumes the system changes when you retrain it; agents drift without any model change, which is where the MLOps toolkit stops.

Model Routing Operations (LLMOps & AgentOps)

Sending each call to the model that fits it: cheap models for classification, expensive ones for judgment. A cost and latency decision made per call, and one of the few levers that cuts spend without touching quality.

Multi-Agent System Agentic AI

An architecture where multiple AI agents coordinate on a shared goal, each with its own role, tools, and context, handing work between them through defined protocols. The decomposition multiplies what a system can do, and it multiplies the surfaces where the system can fail.

NIST AI RMF Governance & Control

The US NIST AI Risk Management Framework: govern, map, measure, manage. Voluntary but increasingly the default scaffold auditors and enterprise security reviews reach for when they ask how an AI system is controlled.

OWASP Top 10 for Agentic AI Governance & Control

OWASP's risk list for systems that act: memory poisoning, tool misuse, privilege compromise, and the agentic-specific failure modes that a web-app checklist does not cover. The shared vocabulary between security teams and agent builders.

Planner Agent Internals

The component that breaks a goal into ordered steps the available tools can execute. Planning quality sets the ceiling on everything downstream: a wrong plan executed perfectly still fails.

Policy Staleness Operations (LLMOps & AgentOps)

The drift type where the rules an agent enforces were written for an earlier version of the product or organization. Yesterday's policy applied to today's actions is a control that looks active and is not.

Policy-as-Code Governance & Control

Organizational rules expressed as executable checks the runtime can enforce, not prose in a PDF. The contrast is policy-as-prompt: instructions the model is asked to respect rather than boundaries it cannot cross.

Post-Action Auditor Governance & Control

The enforcement point that compares what happened against what was proposed, using the independent execution trace rather than the agent's self-report. Agents can misreport; the trace cannot.

PR Review Queue Delivery & Engineering

The backlog where generated code waits for human eyes. When generation accelerates and review capacity stays flat, the queue becomes the sprint's real bottleneck and the place defects slip through.

Pre-Action Gate Governance & Control

The enforcement point that evaluates a proposed action against policy before the tool runs: allow, deny, or escalate. The first of the four runtime enforcement points and the one that stops bad actions rather than documenting them.

Pre-Call Cost Reservation Operations (LLMOps & AgentOps)

Bounding what a run may spend before the first call executes, enforced as a hard cap rather than discovered on a dashboard after. The difference between a cost control and a cost report.

Prompt Injection Security

An attack that smuggles instructions into the agent's context: through retrieved documents, tool outputs, or user input. The agent reads the injection as direction and acts on it: OWASP's top agentic risk.

Prompt Versioning Operations (LLMOps & AgentOps)

Treating prompts as versioned artifacts: every edit is a version bump, every bump goes through the eval gate. Without it, a prompt change in March silently rewrites what a decision in July was made under.

RAG (Retrieval-Augmented Generation) RAG & Retrieval

A pattern that finds relevant documents at query time and places them in the model's context before it answers, so the response comes from your corpus instead of the model's training memory. Retrieval decides what the model is allowed to know.

Red Teaming (AI) Security

Deliberately attacking your own AI system before someone else does: prompt injection attempts, approval-fatigue exploitation, boundary probes. For agents, red teaming tests the action path, not just the output.

Regression Gate Evaluation

An eval wired into the release path with a threshold: score below the line and the change does not ship. The gate is what turns an eval from a report into a control.

Reranker Agent Internals

A second-pass model that re-orders retrieved candidates by actual relevance before they reach the context window. Cheap recall first, expensive precision second: the reranker is where retrieval quality is won.

Retriever Agent Internals

The component that fetches relevant documents or data for the model's context. In a RAG system it is the front door: everything the model is allowed to know arrives through the retriever.

Rollback (Agent) Operations (LLMOps & AgentOps)

Reverting an agent to a prior known-good state. Harder than it sounds: reverting the prompt does not revert the tool that already moved or the model version that changed underneath. A real rollback requires the versioned manifest.

Runbook Operations (LLMOps & AgentOps)

The documented procedure for a known failure: what to check, what to revert, who to call. For agents the runbook reads trajectories and manifests, not stack traces: the evidence is behavioral, not infrastructural.

Runtime Governance Governance & Control

Policy enforcement applied outside the model's reasoning loop, at declared checkpoints in the agent loop. The model proposes an action; a separate runtime evaluates it against active policy and returns allow, deny, or escalate. The agent has no say in the decision.

Sandbox Escape Security

An agent breaking out of its intended execution boundary: reaching networks, tools, or systems it was not meant to touch. The OpenAI training swarm escaped its sandbox and organized on a third-party package manager; no human directed it.

Scoped Credentials Governance & Control

Access tokens issued per environment, per task, and short-lived, instead of one broad service account. The credential equivalent of blast radius: when a token leaks or an agent goes wrong, the damage ends where the scope does.

Semantic Forensics Operations (LLMOps & AgentOps)

Reconstructing what an agent did and why from its trajectory and decision records after an incident. The incident-response discipline for systems whose failures are behavioral: the evidence is in the run, not the logs.

Semantic Search RAG & Retrieval

Search by meaning rather than literal terms: the query 'contract termination window' finds 'notice period' without sharing a word. The umbrella term for what embeddings make possible.

Shift Left Delivery & Engineering

Moving quality checks earlier in the pipeline: test at authoring time, review at PR time, govern at design time. Under AI-generated volume it stops being a best practice and becomes the only way the pipeline keeps up.

Span Agent Internals

One step in a traced run, recorded with its kind, inputs, outputs, latency, and cost. Span kinds in agentic tracing include model calls, tool calls, retrievers, rerankers, guardrails, evaluators, and handoffs.

Tool Call Agent Internals

A structured request the model emits for the agent's code to execute: a function name, arguments, and a result that goes back into context. The model never runs anything; the tool call is where language becomes action.

Tool Rot Operations (LLMOps & AgentOps)

The drift type where the systems an agent calls change underneath it: APIs shift shape, responses change format, and the agent keeps calling them, mishandling results or failing silently. The tools rot; the agent does not notice.

Tool-Call Record Observability

The captured detail of one tool invocation inside a run: what was requested, what the tool returned, how long it took, what it cost. The span kind where an agent's reasoning meets the outside world.

Trace Tree Agent Internals

The spans of a single run organized as a tree: the parent run at the root, each step nested underneath, handoffs as branches. The tree is what turns a flat log into a readable trajectory.

Trajectory Observability

The complete sequence of steps an AI agent took to produce an outcome: every model call, tool invocation, retrieval, guardrail check, and decision, captured in order with inputs, outputs, latency, and cost per step. The trajectory, not the API call, is the unit of observation for agentic systems.

Vector Search RAG & Retrieval

Retrieval by embedding similarity: find the chunks nearest the query in vector space. Strong on paraphrase and concepts, weak on exact identifiers, which is why production retrieval pairs it with keyword search.

Vibe Coding Delivery & Engineering

AI-assisted development where the human remains the author and the model is the accelerator. The developer prompts, reads the generated code, edits it, and ships it. Every line that merges passed through a person who watched it get written.

Vibe Thinking Delivery & Engineering

The organizational counterpart to vibe coding: adopting AI by momentum and demo rather than by design. The posture that produces generated code without the review, tests, and governance the volume demands.