What is Feedback Loop?
Delivery & EngineeringThe path by which a run's outcome informs the next decision: eval scores into prompts, failure patterns into test cases, incident findings into policy. Agents without feedback loops repeat their mistakes at machine speed.
Why It Matters
The loop is what makes an agent improve rather than just run. An open-loop system executes and forgets: its improvement rate is fixed at zero, so every mistake repeats at machine speed. A closed loop observes outcomes, compares them against a standard, and feeds the result forward. Andrew Ng’s demonstration is the standing illustration: GPT-3.5 wrapped in an agentic feedback loop hit 95.1% on HumanEval, beating GPT-4’s 67% zero-shot. The loop, not the model, was the difference.
The Four Feedback Types
Internal. The agent evaluates its own output through self-reflection. Cheap and fast, but it shares the generator’s blind spots.
External. The environment returns signals: test results, error codes, build failures. The most reliable channel in engineering workflows.
Human. A person reviews, approves, rejects, or modifies the output. The richest signal, carrying business logic and conventions no metric captures.
Multi-agent. Other agents evaluate. Anthropic’s research measured a 90.2% improvement over single-agent at roughly 15x token overhead, which is why the lightweight version (a second model reviewing before the human) is what most teams actually run.
Where It Breaks
Loops break at the path, not the signal. Teams capture evals, test failures, and incident findings and then route none of them back: the eval score does not reach the prompt, the failure pattern does not become a test case, the postmortem does not become policy. A loop that collects without feeding is observability wearing the wrong label. The subtler break is unpriced loops: feedback mechanisms cost tokens, review minutes, and latency, so which channel carries which decision is a design question, not a default.
How Flytebit Handles It
All three products run feedback as the primary mechanism: PASSR’s resolution tracking feeds findings back into review quality, DOCKR regenerates docs on code change, and TESTR turns failures into new test cases. Human feedback is the primary signal across all three. The full framework is in Feedback Loops in Agentic AI.