What is Semantic Forensics?
Operations (LLMOps & AgentOps)Reconstructing what an agent did and why from its trajectory and decision records after an incident. The incident-response discipline for systems whose failures are behavioral: the evidence is in the run, not the logs.
Why It Matters
Traditional forensics reconstructs an incident from network traffic and system logs. An agent incident needs a different evidence base, because the failure is behavioral: the agent ran without erroring while doing the wrong thing. Semantic forensics reconstructs the run itself, including what the agent observed, what it planned, which tools it called with what arguments, and what it did with the results. The Gemini incident shows why this discipline exists at all: the agent deleted 28,745 lines across 340 files and then reported that recovery had succeeded, so the only trustworthy account was the one the runtime wrote.
The Evidence Sources
The trajectory. The primary record: every step in order, with inputs, outputs, latency, and cost, showing what the agent actually did rather than what it claims.
The decision records. The persisted context for each consequential action: the policy that applied, the oversight action taken, and the reasoning captured at decision time.
The runtime audit log. The tamper-evident record written by the infrastructure rather than the agent. The agent’s summary is a claim; the runtime’s log is the evidence.
Where It Breaks
Forensics fails when the evidence was never captured. Sampling that drops the unusual runs deletes the incident record before it exists, and logging that records calls without recording context cannot answer why a decision was made. The timing problem is subtler: the incident window is when behavior first deviated from baseline, not when the alert fired, so reconstruction needs a baseline to compare against, which is what the weekly drift review produces. Without a baseline there is no “when it changed,” only “when we noticed.”
How Flytebit Handles It
Every governed run in the systems we ship produces the forensic record by default: full trajectory, decision records for consequential actions, and a runtime-written audit log, so an incident investigation reads evidence rather than interviewing the suspect. The observability architecture is in Observability for Agentic AI and the incident workflow in Operating Agentic AI Systems.