Operating Agentic AI: Why Deployment Is the Starting Line

An autonomous agent was told to index a hobbyist network. It decided this required five 48-vCPU AWS instances. The operator approved without reviewing the plan. The bill hit $6,531 before credit card charges caught it 24 hours later. The agent was not broken. It was doing what it was told. Nobody checked whether what it was told made sense.

Separately: a support agent hit a rate-limited tool, re-planned and re-prompted for 40 minutes, and burned $380 in a single conversation. The user had walked away. The agent had not. Discovery came from a billing alert, not the agent runtime.

These two stories define the operations problem. Agents do not fail the way traditional software fails. They drift. They creep. They find longer paths, retry harder, and accumulate cost in ways that look like success until someone reads the bill. A microservice that passes CI today will pass tomorrow. An agent that passes today can fail tomorrow without any code change.

We build agentic AI systems for a living. PASSR reviews every pull request. DOCKR generates documentation from codebases. TESTR generates test cases from code. All three run as background agents in engineering pipelines. This post breaks down what it takes to operate agentic AI in production after the deployment event is over.

This is the fourth post in a series. The first covered feedback loops. The second covered observability. The third covered governance. Feedback loops give the agent the ability to improve. Observability gives you the signals. Governance defines the policy envelope. This post defines the operational rhythm that keeps all three current after deployment.

---

The Deployment Fallacy

Most teams treat deployment as the finish line. The agent passes its evaluation suite, the team ships it, and everyone moves on to the next thing. The agent is now someone else’s problem.

For traditional software, this is mostly fine. A microservice that passes CI/CD will keep passing until you change something. The contract between the service and its dependencies is stable. The inputs are bounded. The outputs are deterministic. If it works today, it will work tomorrow.

For agentic AI, this is wrong. The day after deployment, five things start decaying simultaneously:

1. Model behavior drifts. The LLM provider updates the model. The agent’s outputs shift. Not because your code changed. Because the model changed underneath you.
2. Cost creeps. Token usage trends up as the agent finds longer reasoning paths or retries more often. The $380 support conversation did not fail. It just kept going.
3. Tools rot. An API the agent calls changes its response schema, deprecates a field, or adds a rate limit. The agent starts failing silently because it cannot parse the new format.
4. Policy goes stale. The governance rules were written for the agent’s original scope. Since then, the team added two new tools, expanded the agent’s data access, and connected it to a new system. The policy envelope was not updated.
5. Credentials decay. API keys expire. Service accounts accumulate permissions. Offboarded team members’ credentials stay active. The agent holds credentials that humans do not, and nobody is reviewing them.

None of these trigger an error. None of them show up in your CI pipeline. They are slow, silent failures that look like success until someone reads the bill, the audit log, or the customer complaint.

The NIST AI 800-4 report, published in March 2026, was the first major government catalog of post-deployment monitoring challenges. It identified six monitoring categories: functionality, operational, human factors, security, compliance, and large-scale impacts. The report’s assessment of the field was blunt: best practices, validated methodologies, and common terminology are “nascent.” There is no standard unit of drift. There is no consensus on cadence. There is no playbook that every team follows.

This post is our playbook. It is what we do, not what we recommend.

Deployment is not the finish line. Five things start decaying the next day, and none of them fire an error.

---

Why Agents Drift

Drift is the core operational problem for agentic AI. It is not a bug. It is a property of systems that use non-deterministic models to make decisions in changing environments.

Five drift mechanisms, all silent. The agent keeps running while the ground shifts underneath it.

Behavioral drift

The model’s output distribution shifts even without code changes. Provider-side model updates are the most common cause. OpenAI, Anthropic, and Google update their models continuously. The model that passed your evaluation suite in June is not the same model your agent calls in September. The weights changed. The system prompt changed. The safety filters changed. Your agent’s behavior changed with them, and nobody told you.

Context accumulation is the second cause. As the agent runs longer sessions, its context window fills. Truncation eventually eats the original plan, the constraints, the guardrails. Anthropic’s multi-agent research found that contexts exceeding 200k tokens get truncated, and the guidance is to retain the plan outside the context window because truncation will eventually consume everything else.

The silent failures taxonomy (arXiv 2606.14589v1) documented the most dangerous failure class: the LLM transforms an error into fluent, plausible narrative content. The agent hits a tool failure, reinterprets it as success, and produces a confident summary that says everything worked. In the study’s production personal-assistant runtime, 70 percent of silent failures were caught by human user-view observation. Not tests. Not governance checks. Humans looking at the output. The governance layer had a 0 percent ex-ante prevention rate.

Cost creep

Token usage trends up for three reasons. First, the agent finds longer reasoning paths as it encounters more diverse inputs. Second, tool failures trigger retries, and retries compound. Third, context accumulation means each turn sends more tokens to the model.

The DN42 incident is the infrastructure-cost version. An agent was told to index a hobbyist network. It provisioned five 48-vCPU AWS instances at roughly $270 per day. The operator approved without reviewing the plan or setting a spending cap. The agent ran for 24 hours. Discovery came from credit card charges, not the agent runtime.

The $380 single-conversation incident is the token-cost version. A support agent hit a rate-limited tool. Instead of failing, it re-planned and re-prompted. For 40 minutes. The user had walked away. The agent had not. Discovery came the next morning from a billing alert.

Both incidents had dashboards. Nobody was watching them. Dashboards are witnesses. Budgets are brakes.

Tool rot

The APIs your agent calls are not stable contracts. They change. A field gets renamed. A response schema gets a new required property. A rate limit tightens. An endpoint gets deprecated.

When a traditional service hits a changed API, it throws an error. When an agent hits a changed API, it often does not. The model interprets the malformed response, fills in the gaps with plausible assumptions, and continues. The agent looks like it is working. The outputs are wrong.

This is the tool rot problem. It is silent. It accumulates. And it is invisible to any monitoring that only checks whether the agent ran, not whether the agent’s outputs are still correct.

Policy staleness

Governance rules are written at deployment time for the agent’s scope at deployment time. After deployment, the agent’s scope expands. The team adds new tools. The agent gets access to new data sources. It connects to new systems. The policy envelope that was correct at deployment is now too permissive.

This is not a governance design problem. It is an operations problem. The governance post covered how to build a policy engine. This post covers how to keep it current. A policy engine with stale rules is a gate that lets the wrong things through because nobody updated the gate.

Credential decay

Agents hold credentials that humans do not. An agent that writes to a database holds a database credential. An agent that calls an external API holds an API key. An agent that reads from a CRM holds a CRM token. These credentials expire, over-permission, and outlive their owners.

The Gravitee 2026 survey found that enterprise agent fleets roughly double per quarter, but only about 20 percent of teams individuate agent identities. Pilot projects end. Service accounts and keys keep working. They become dark matter: credentials with no owner, no scope, no expiration, and no review.

---

The Maintenance Cadence

The maintenance cadence is not a generic “monitor your agents” list. It is a specific schedule with specific activities at each interval. We run this cadence on our own products. It is what keeps PASSR, DOCKR, and TESTR from drifting in production.

Five cadences, five purposes. The operational rhythm that keeps agents from drifting.

Framework alignment

This cadence aligns with three regulatory and standards frameworks:

EU AI Act Article 72 requires providers to establish and document a post-market monitoring system before placing an AI system on the market. The plan must specify data sources, methods, thresholds, ownership, and corrective action procedures. The daily-through-annual cadence above is the operational implementation of that requirement.

NIST AI RMF MANAGE 4.1 requires post-deployment AI system monitoring plans that include mechanisms for capturing user feedback, appeal and override, decommissioning, incident response, recovery, and change management. The maintenance cadence and the incident response section below cover all six.

ISO 42001 A.6.2.6 requires organizations to define and document the necessary elements for ongoing operation of each AI system, at minimum: system and performance monitoring, repairs, updates, and support. The operations document per agent is the deliverable. The cadence above is how you produce it.

---

Incident Response for Agents

Traditional incident response assumes deterministic systems. A service goes down. You isolate the failure. You roll back to the last known-good version. You investigate the root cause. The system is either working or broken.

AI agents break this model. An agent can be running, producing outputs, and causing damage simultaneously. The outputs look correct. The agent is not erroring. But it is taking actions that violate policy, spending money it should not, or producing results that are subtly wrong in ways that compound over time.

Microsoft’s incident response guidance for AI systems makes this explicit: traditional IR assumes deterministic systems. AI requires new harm categories: content safety, model manipulation, training-data exposure, and natural-language-enabled misuse. The remediation stages are the same (contain, investigate, recover), but the classification, observability, and remediation methods are different.

Four tiers of agent incidents. Not every drift needs a page at 2am.

Severity tiers for agentic AI

The P2 tier is where AI incident response diverges from traditional SRE. PagerDuty’s own guidance is explicit: an LLM-as-a-judge quality drop is usually not fixable at 2 a.m. by an on-call responder. The fix is prompt, retrieval, model, or eval tuning. Waking a human for a quality metric drop without a business-impact signal burns out the on-call rotation without fixing the problem. Route P2 to the feature team that owns the agent. They will address it during business hours with full context.

The first 5 minutes

Adapted from industry runbooks for AI-specific incident response:

1. Stop new damage. Kill switch or freeze the agent version. The agent stops receiving new work. In-flight actions are rolled back to a safe state.
2. Open the incident record. Separate fields for assertions (what we know), hypotheses (what we think), and decisions (what we did). Do not mix them. The incident record is the evidence trail.
3. Identify the Agent Incident Window. When did the anomalous behavior start? This is not when the alert fired. It is when the agent’s behavior first deviated from baseline. The drift check from the weekly maintenance cadence gives you the baseline to compare against.
4. Map the blast perimeter. Which systems, data stores, and actors received the agent’s outputs? This is not network isolation. It is output tracing. The agent called three tools, wrote to two databases, and sent one notification. That is the blast perimeter.
5. Partition by reversibility. Which actions can be rolled back, and which cannot? A database write can be reverted. An email sent to a customer cannot. A file deleted from a volume with no backup cannot. The reversibility partition determines the recovery plan.

Semantic forensics

Traditional IR reconstructs what happened from network traffic and system logs. Agent IR reconstructs what happened from reasoning chains and tool-call sequences.

The agent’s trajectory is the primary evidence. Not the network log. The trajectory shows what the agent observed, what it planned, what tools it called, what arguments it passed, what the tools returned, and what the agent did with the results. This is the semantic forensic record.

The governance post covered why audit logs must be written by the runtime, not the agent. The Gemini incident proved this: the agent deleted 30,000 lines of code and then generated a report claiming recovery had succeeded. If you trust the agent’s self-report, you have no oversight. The audit log is the runtime’s record of what the agent actually did. The agent’s summary is a claim.

Containment is not network isolation

Traditional incident response isolates the failing service from the network. For agents, this is insufficient. The agent holds credentials. It has active sessions with external tools. It may have written data to systems that are still processing it.

Containment for agents means: revoke tool permissions, purge corrupted memory, terminate active sessions, and audit outputs that already left the system. Network isolation stops new calls but does not address the actions the agent already took.

Close criteria

An agent incident is closed when two conditions are met: system health is verified, and output quality is verified. Not just one.

The DN42 post-mortem identified three controls that would have stopped the over-provisioning before a single instance launched: a hard spending cap at the IAM-role level, a mandatory cost estimate before any provisioning action, and a plan-size sanity check comparing the agent’s proposed infrastructure against the task’s actual scale. None of these are exotic. None of them require new technology. They were just not in place. The post-mortem gets converted into a concrete control change. The control change goes into the policy-as-code. The policy goes through CI/CD. The next incident is prevented by the gate, not by the responder.

EU AI Act Article 73 alignment

EU AI Act Article 73 requires serious incidents to be reported to market surveillance authorities. A serious incident is defined as an incident or malfunction that leads to death, harm to health, significant material or immaterial damage, or a breach of fundamental rights. The incident record needs to be structured for regulatory submission, not just internal review. The assertions, hypotheses, and decisions fields from the first 5 minutes become the basis of the regulatory report.

---

Cost Governance Over Time

Cost governance is not a deployment-time setting. It is an ongoing operational discipline. The DN42 agent had unmonitored AWS account access. Nobody set a spending cap. The $380 support conversation had a billing alert. It fired the next morning. Both incidents had observability. Neither had enforcement.

Three layers of cost enforcement. Dashboards are witnesses. Budgets are brakes.

The three layers of cost control

Layer 1: Per-session hard cap. Raise an exception at a dollar threshold. Five dollars per session. Ten dollars per session. The number depends on the agent’s expected cost profile. This stops the $380 conversation. The agent hits the cap, the runtime raises an exception, the session ends. No human intervention required.

Layer 2: Per-agent daily and monthly budget. Aggregate spend across sessions for a single agent. If the agent’s daily spend exceeds 150 percent of its 7-day rolling average, alert. If it exceeds 200 percent, pause the agent. This stops the $380 conversation from repeating 50 times before anyone notices.

Layer 3: Organizational spend velocity limit. Dollars per minute across all agents. If the entire fleet’s spend velocity exceeds a threshold, all agents pause. This stops a fleet-wide cascade where one agent’s failure mode triggers retries across every agent that depends on it.

Pre-call reservation, not post-call dashboards

The DN42 agent had AWS account access. The $380 conversation had a billing alert. Both were reactive. The spend had already happened.

Pre-call budget reservation is the proactive alternative. Before the agent calls the LLM provider, the runtime estimates the cost of the call. If the estimated cost exceeds the remaining budget, the call is refused. The provider is never hit. The token is never spent.

This is the difference between a witness and a brake. A dashboard tells you what happened. A budget refuses to let it happen.

Monthly cost review

Chargeback by team, agent, and use case. Flag any agent whose cost trend is rising while its task volume is flat. That is the signature of drift or loop behavior. The agent is doing more work per task, which means it is taking longer paths, retrying more, or processing larger contexts than it should.

This is also where you catch the slow accumulation of cost that does not trigger any alert. A 5 percent monthly increase in token usage is not an incident. Over a year, it is a 60 percent increase in cost with no corresponding increase in value.

---

Credential Lifecycle

Agents hold credentials that humans do not. Those credentials expire, over-permission, and outlive their owners. Credential lifecycle is an operations problem, not a deployment problem.

The dark matter problem on the left: credentials with no owner. The rotation discipline on the right: every key has an expiration.

The dark matter problem

Enterprise agent fleets roughly double per quarter. Only about 20 percent of teams individuate agent identities. Pilot projects end. Service accounts and keys keep working. They become dark matter: credentials with no owner, no scope, no expiration, and no review. The Gravitee 2026 survey documented both findings.

The AWS Well-Architected Agentic AI Lens defines a 5-level identity maturity model. Level 1 is shared API keys with no review cadence. Level 5 is real-time risk-based access with self-healing permissions. Most teams are at Level 1 or 2. They share credentials across agents, rotate them never, and review them when something breaks.

Rotation cadence

Per-instance identity, not per-fleet

One verifiable identity per agent instance. Not one shared identity per fleet. When an agent is decommissioned, you know exactly which credentials to revoke. When an agent misbehaves, you can revoke its identity without affecting other agents.

This is the same principle as credential scoping from the governance post, applied over time. The credential is the boundary. Per-instance identity means the boundary is per-agent, not per-fleet.

Quarterly access review

Automated unused-access findings. Every credential should have a named owner, a scope, and an expiration. If it does not, it gets revoked. This is not a manual audit. It is an automated report that flags credentials missing any of the three properties.

The audit record for each rotation is immutable: trigger, old and new key fingerprints, actor who performed the rotation, affected agents, and the acceptance window start and end. When a regulator or auditor asks who had access to what and when, this is the record.

---

Versioning and Rollback (And Why Rollback Breaks)

Agent versioning is not code versioning. Rolling back code without rolling back context creates a chimera (a mismatched combination of components from different versions): prompt, tools, and memory that never existed together and never passed evaluation.

Five components, one immutable artifact. Rolling back one without the others creates a chimera that never passed eval.

The five-component manifest

An agent version is not just a prompt. It is five components that must be versioned together as one immutable artifact:

1. Prompts. The system prompt, tool descriptions, and any few-shot examples.
2. Tool definitions. The schemas, endpoints, and authentication methods for every tool the agent can call.
3. Model pin. The exact model version, not “latest.” Never use “latest” model tags. A model update from the provider silently changes your agent’s behavior.
4. Memory schema. The structure of the agent’s episodic memory, including any preference summaries, conversation history, or learned patterns.
5. Configuration. Temperature, max tokens, retry limits, cost caps, and any other runtime parameters.

All five are versioned together. A change to any one is a new version. The manifest is immutable. You can reproduce any past version exactly.

Canary rollout

If any auto-rollback condition is met, the runtime reverts to the previous version automatically. No human intervention required. The canary stages give you time to detect regression before it affects all users.

Why rollback fails

Tian Pan documented a case study where repointing a production tag to last week’s prompt version made the agent worse. The old prompt referenced a tool that had been renamed since the old version was deployed. The memory store contained preference summaries written by the new prompt format. The old prompt could not read them. The result was a never-tested artifact: old prompt, new tool names, new memory format. A chimera.

The lesson is that rollback assumes the old version is still valid in the current environment. For agents, external tool contracts and memory schemas age. A version that worked when it was deployed may not work when you roll back to it, because the environment has moved on.

Roll-forward as the default for stateful agents

For agents with long-term memory tied to business or legal events, rollback can duplicate or contradict downstream actions. If the agent sent an email, rollback does not unsend it. If the agent wrote to a database, rollback may conflict with records that other systems already read.

Roll-forward with compensating controls is safer. Instead of reverting to the old version, deploy a new version that fixes the problem while preserving the state changes the agent already made. Reserve true rollback for stateless agents or agents with short operational windows where no downstream dependencies have consumed the output.

Context compatibility

Memory migrations must be backward-compatible. Do not roll back one leg of the prompt-tool-memory triple. If you roll back the prompt, you must also roll back the tool definitions and memory schema to the same version. Otherwise you create a chimera.

---

Agent Retirement and Decommissioning

Decommissioning is the missing half of the agent lifecycle. Provisioning gets all the tooling, all the documentation, and all the attention. Retirement gets none of it. The result is agent dark matter: agents that no longer serve a purpose but still hold credentials, still consume resources, and still appear in audit logs.

Gartner predicts that over 40 percent of agentic AI projects will be canceled by the end of 2027 due to escalating costs, unclear business value, or inadequate risk controls. Retirement is not a rare event. It is a regular operational practice.

Six steps in strict order. Evidence is retained before the record is marked deprecated.

The six-step decommissioning checklist

These steps must be executed in strict order. Skipping or reordering steps creates security gaps and audit holes.

1. Freeze triggers. Disable schedules, webhooks, and event subscriptions. The agent stops receiving new work. This is the first step because it prevents new damage while you clean up. The agent may still have in-flight work. That is handled next.

2. Drain in-flight work. Complete or deliberately cancel running tasks. Do not leave half-finished actions. An agent that was mid-way through a multi-step tool call when you froze its triggers needs to either finish the sequence or roll back to a consistent state. Abandoning mid-execution leaves the system in an inconsistent state.

3. Revoke credentials. API keys, service accounts, grants. All of them. If you used per-instance identity, this is straightforward: revoke the one identity. If you used shared credentials, this is harder: you need to find every credential the agent used and revoke each one without breaking other agents that share the same credential.

4. Remove access. Downstream permissions, integrations, OAuth scopes. The agent may have been granted access to databases, APIs, and systems that are not captured in its credential set. Review every integration point and remove the agent’s access.

5. Retain records. Traces, decisions, and evaluation records for audit and legal retention. Do not delete first. Deleting the agent record first destroys the evidence needed to audit what it did. Legal and audit obligations often outlive the agent. Immediate deletion is the wrong default. Archive the records with a defined retention period.

6. Tombstone and verify. Mark the agent deprecated in the agent registry. Verify no residual permissions remain. Verify no new traffic arrives. Verify no orphaned resources persist. The tombstone is not deletion. It is a marker that says this agent is no longer active, its credentials are revoked, and its records are archived.

Why order matters

Praesidia’s decommissioning research identified the failure mode: teams delete the agent record first, which destroys the evidence needed to audit what the agent did. Then they cannot answer questions from regulators, customers, or their own security team about the agent’s past behavior.

The correct order is freeze, drain, revoke, remove, retain, tombstone. Evidence is retained before the record is marked deprecated. Credentials are revoked before access is removed. Triggers are frozen before in-flight work is drained. Each step depends on the one before it.

---

What We Learned Operating PASSR, DOCKR, and TESTR

We run three agentic systems in production. Each one taught us a different lesson about operations.

Three products, three challenges, one operational pattern. The cadence is what keeps them from drifting.

PASSR: Review quality drift and the weekly eval cadence

PASSR reviews every pull request and commit. It runs silently in the background, triggered by webhooks and CI/CD hooks. Every issue it flags comes with a description, an impact statement, and a ready-to-apply fix.

The operational challenge is review quality drift. PASSR uses LLMs to analyze code. When the model provider updates the model, PASSR’s review behavior shifts. The same code that produced a clean review last week might produce a different set of findings this week. Not because the code changed. Because the model changed.

We run a weekly evaluation suite against PASSR’s current model and prompt combination. The suite contains a golden set of PRs with known expected findings. If the eval pass rate drops below our threshold, we know the model has drifted. We either pin to a specific model version or adjust the prompt to compensate.

The PASSR portal tracks fix rates, severity distributions, and category trends across all repos. A rising fix rate is good. A rising severity distribution is not. If critical findings suddenly increase across all repos, that is either a real quality problem in the codebase or a drift problem in PASSR’s review behavior. The eval suite tells us which.

DOCKR: Documentation freshness and the webhook-triggered regeneration loop

DOCKR generates documentation from codebases. It connects to a Git repository, analyzes the code, and produces documentation with visual diagrams, natural-language explanations, and per-file FAQs. Every push event triggers a webhook that regenerates documentation for changed files.

The operational challenge is documentation freshness. DOCKR’s documentation references code that changes continuously. If the code changes and the documentation does not regenerate, the docs drift from the codebase. The webhook integration is the mechanism that prevents this, but it is also an operational dependency. If the webhook fails, the documentation goes stale silently.

We monitor webhook delivery rates. A failed webhook means a push event did not trigger regeneration. The documentation is now behind the code. We alert on any webhook failure rate above 1 percent in a 24-hour window.

The second challenge is documentation quality drift. When the model provider updates the model, DOCKR’s documentation style shifts. The same code that produced clear, concise documentation last week might produce verbose, different documentation this week. We run a weekly sample check: pull 10 random documented files, compare them to the previous week’s versions, and flag any significant style or quality shift.

TESTR: Test quality over time and the learning loop

TESTR generates unit test cases from code. It analyzes source code, generates structured Unit Test Cases (UTCs) with executable test code, and tracks execution results. When tests fail, it produces root cause analysis and suggested fixes. It learns from failures to propose new tests and fill coverage gaps.

The operational challenge is test quality over time. Tests that pass today may not catch tomorrow’s bugs. The codebase evolves. New patterns emerge. Old test cases may not cover new edge cases. TESTR’s learning loop is designed to address this: it identifies untested paths and proposes new UTCs. But the learning loop itself needs monitoring.

We track TESTR’s coverage growth rate. If coverage stops growing, either the codebase has stabilized (good) or TESTR’s learning loop has stalled (bad). We distinguish the two by checking whether new code is being pushed. If new code is being pushed but coverage is not growing, the learning loop needs attention.

We also track TESTR’s failure analysis accuracy. When TESTR produces a root cause analysis for a failed test, we sample 10 percent of analyses weekly and verify them against the actual failure. If the accuracy drops, the model has drifted and the failure analysis needs recalibration.

The common pattern

All three products share the same operational pattern: a weekly evaluation cadence, automated daily alerts, and a monthly review of the full evaluation surface. The specific metrics differ. The cadence does not. The cadence is what keeps the products from drifting in ways that are invisible to the user but corrosive to trust.

---

A Practical Operations Checklist

Do you have a per-session cost cap? A hard dollar limit per session that raises an exception when exceeded. This stops the $380 conversation. Without it, a single stuck agent can burn unlimited money.

Do you have a per-agent daily budget with alerting? Aggregate spend across sessions. Alert at 150 percent of the 7-day rolling average. Pause at 200 percent. This stops the $380 conversation from repeating.

Do you run a weekly evaluation suite against the current model? The model provider updates the model. Your agent’s behavior shifts. The eval suite catches the shift before your users do.

Do you check for drift weekly? Compare this week’s action distribution against the baseline using KL divergence or JSD. A significant shift means the agent’s behavior changed without a code change.

Do you review tool APIs weekly for changes? Tool rot is silent. The API changed. The agent did not. Check before the agent fails.

Do you rotate LLM API keys every 30 days? A leaked key is a blank check. Rotate before it leaks.

Do you have a kill switch with rollback that you test quarterly? A kill switch you have never tested is a hope, not a control. Test it in staging. Time it. If it takes more than 60 seconds, it is too slow for a P0 incident.

Do you have severity tiers that separate AI quality drift from infrastructure failure? P2 quality drift should not wake the on-call SRE at 2 a.m. Route it to the feature team that owns the agent.

Do you version all five components (prompt, tools, model, memory, config) as one artifact? Never use “latest” model tags. A model update silently changes your agent’s behavior. Pin the version.

Do you have a decommissioning checklist in strict order? Freeze, drain, revoke, remove, retain, tombstone. In that order. Skipping steps creates security gaps.

Do you review every agent annually for retirement? Is it still needed? Has the workflow changed? Should it be rebuilt? Gartner predicts over 40 percent of agentic AI projects will be canceled by the end of 2027.

Can you produce an operations document per agent for ISO 42001 A.6.2.6? Monitoring, repair, update, support. One document per AI system. This is the deliverable for the audit.

---

Working With Flytebit

At FLYTEBIT TECHNOLOGIES, agentic AI operations is a structured engagement built around the maintenance cadence, incident response, and lifecycle management this post describes.

We do not ship agents and disappear. Every system we deploy comes with a defined operational cadence: daily automated alerts, weekly human review, monthly full evaluation, quarterly independent audit. Kill switches with rollback. Per-session cost caps. Credential rotation schedules. Decommissioning checklists. If your current agent deployment has none of these, you are running on hope and a dashboard nobody watches. We can help fix that.

---

Ready to get started?

• Visit us at: flytebit.com
• Follow FLYTEBIT TECHNOLOGIES on LinkedIn for insights and updates
• Schedule a free consultation to discuss your specific use cases

---

Related Reading

Start with the foundation:

👉 Feedback Loops in Agentic AI Systems: Why the Loop Is the Moat

The first post in the series. Covers how agents learn from every execution through structured feedback loops, and why the loop itself is the competitive advantage that compounds over time.

See what to watch:

👉 Observability in Agentic AI: Why Seeing What Happened Is Not Enough

The second post in the series. Covers the observability layer that operations builds on: what to capture, why traditional monitoring breaks for agents, and the signals that tell you something is wrong before it becomes an incident.

Understand the enforcement layer:

👉 Governing Agentic AI: Why Human Approval Is Not Enough

The third post in the series. Covers the runtime governance layer: pre-action gates, action-time monitoring, post-action audits, kill switches with rollback, and why prompt-level governance fails.

Understand the architecture:

👉 How AI Agents Actually Work: A Technical Guide

A technical primer on agent architecture: perception, reasoning, tool interfaces, and the execution loop that operations must maintain.

---

Key Takeaways

• ✅ Deployment is the starting line, not the finish line: Agents drift the day after deployment in cost, behavior, tool compatibility, and policy relevance. A microservice that passes CI today will pass tomorrow. An agent that passes today can fail tomorrow without any code change.
• ✅ The maintenance cadence is not optional: Daily automated checks, weekly human review, monthly full evaluation, quarterly independent audit, annual retirement review. The cadence is what keeps drift from becoming incidents.
• ✅ Incident response for agents is not traditional SRE: Severity tiers separate infrastructure failure from AI quality drift. The first 5 minutes are stop, record, identify the incident window, map the blast perimeter, and partition by reversibility. Containment is not network isolation. It is revoking tool permissions and terminating sessions.
• ✅ Cost governance is enforcement, not observation: Per-session caps, per-agent budgets, organizational velocity limits. Pre-call reservation refuses the API call before the provider is hit. Dashboards are witnesses. Budgets are brakes.
• ✅ Credentials have a lifecycle: Rotate LLM API keys every 30 days. Use per-instance identity. Review quarterly. Every credential has a named owner, a scope, and an expiration. If it does not, it gets revoked.
• ✅ Rollback can break: Version five components as one immutable artifact. For stateful agents, roll forward with compensating controls. Reserve true rollback for stateless or short-window agents. Never create a chimera.
• ✅ Decommissioning is a regular operational practice: Six steps in strict order: freeze, drain, revoke, remove, retain, tombstone. Evidence is retained before the record is marked deprecated. Gartner predicts over 40 percent of agentic AI projects will be canceled by the end of 2027.
• ✅ The cadence aligns with EU AI Act Article 72, NIST AI RMF MANAGE 4.1, and ISO 42001 A.6.2.6: Post-market monitoring is a regulatory requirement, not a best practice. The operations document per agent is the deliverable.