What is Governed AI?

Why It Matters

Most organisations have an AI governance policy by now. Far fewer run a system that enforces one. The gap between those two things is where incidents come from, and it has grown wide enough that the words have started to mean different things depending on who is using them.

Governance is the discipline. It covers the decisions about what AI may do, who answers when it does something else, and how any of that gets proven. Governed is a property of a running system. The model proposes an action, and something outside the model decides whether it happens. That second part is the whole of it.

The distinction carries commercial weight as well as legal weight. Under the EU AI Act, high-risk systems carry obligations that describe a running system rather than a documented intention to build one. An auditor asks what the system did, and what stopped it doing anything else.

The Four Properties

A defined authority boundary. Someone decided what this system may do, and wrote it down. The boundary covers actions rather than topics. “It can answer questions about claims” is not a boundary. “It can approve a claim under £2,000 and nothing else” is.

Policy enforced outside the model. The check sits between the model and the effect. The model proposes; a separate component compares the proposal against the boundary and returns a verdict. An instruction inside the prompt is a preference, and preferences bend under pressure.

A record of each decision. Not a log of API calls. What gets stored is the verdict, the reason behind it, the policy version it was reached under, and the input it was reached on. Traces tell you what happened. Why it was allowed to happen is a separate record, and it is the one an auditor asks for.

Revocability. Authority can be withdrawn, and withdrawing it changes what the system is able to do. And a kill switch that halts execution without reverting what already ran is a pause rather than a control.

Where It Breaks

Policy as prompt. The most common failure, and the hardest to spot, because the system behaves correctly in testing. A model told to stay inside limits usually does. Then the input turns out to be unusual.

Governance on paper. A framework document, a risk register, a signed-off policy, and nothing in the runtime path. The paperwork survives an internal review and fails the first incident.

Logs without verdicts. Teams collect traces, dashboards and token counts, then find during an incident that nothing recorded the decision to act. Observability and governance are often bought together and are not the same purchase.

Approval without authority. A reviewer who cannot say no, or whose refusal gets overridden. Sign-off that always signs off is a formality with a timestamp attached.

A boundary that ages. The policy was right when it was written and the system has moved since. Models get upgraded, tools get replaced, and the boundary quietly stops matching the thing it bounds.

How Flytebit Handles It

We treat governance as a runtime property rather than a documentation exercise. Actions pass through control layers that sit outside the model. Each decision is recorded against the policy version that produced it. We write the authority boundary in terms of what the system may do, never what it should avoid.

Nothing gets built until that boundary is agreed, which is why the engagement opens with a feasibility study. Runtime enforcement is the subject of AI governance and risk, and the operational detail sits in Operating Agentic AI Systems.