What is Tool Rot?
Operations (LLMOps & AgentOps)The drift type where the systems an agent calls change underneath it: APIs shift shape, responses change format, and the agent keeps calling them, mishandling results or failing silently. The tools rot; the agent does not notice.
Why It Matters
A traditional service that hits a changed API throws an error. An agent that hits a changed API often does not: the model interprets the malformed response, fills the gaps with plausible assumptions, and continues. The agent looks like it is working while its outputs go wrong, which makes tool rot the drift type most likely to reach users before it reaches a dashboard.
What Changes Underneath
Renamed fields. The response schema the prompt described no longer matches what the tool returns, and the model improvises around the mismatch.
New required properties. Calls that worked last month now need parameters the agent was never taught to send, so the tool starts rejecting or silently degrading.
Tightened rate limits. The endpoint still works, just slower and with more rejections, and the agent’s retry behavior turns a limit change into a cost event.
Deprecated endpoints. The tool disappears entirely, and whether the agent reports the failure or narrates around it decides how long the outage hides.
Where It Breaks
Tool rot is invisible to any check that asks “did the agent run” instead of “is the output still correct.” Contract tests on tool schemas catch the mechanical half; the behavioral half needs eval runs against the real tools, because a schema that still validates can still return content the agent mishandles. The deeper break is tool inventories that nobody owns: teams know which tools the agent was launched with, not which versions of those tools it calls today.
How Flytebit Handles It
Tool rot maps to the weekly review in our maintenance cadence: tool contracts are pinned in the versioned manifest, and schema changes route through the eval gate like any other change. The drift mechanics are in Operating Agentic AI Systems, and the operating model is our LLMOps work.