What is Grounding?
RAG & RetrievalConstraining the model's answer to retrieved sources rather than its training memory. Grounded systems can point at what they used; ungrounded ones fill gaps with fluent invention.
Why It Matters
A language model always produces an answer. Trained to be fluent and helpful, it fills gaps in its knowledge with confident invention, and the invention is indistinguishable from fact. Grounding is the architectural answer to that behavior: the model may only say what the retrieved sources support.
The difference is accountability. An ungrounded answer asks you to trust the model. A grounded answer shows you what it used.
How It Works
The retriever fetches passages relevant to the query and places them in the model’s context with instructions to answer from them. The model’s job narrows from “produce knowledge” to “compose an answer from this evidence”, a much smaller surface for invention.
Grounding has degrees. A loose instruction to “use the context” is a suggestion the model can drift from. Citation-strict generation is the enforced version: every claim must trace to a passage, and no source means no answer.
Where It Breaks
Grounding fails softly. Retrieval returns near-misses and the model grounds the answer in the wrong source: technically cited, factually wrong. Or the model partially grounds: starts from the passages, then lets training memory fill the edges, and the result mixes evidence and invention without marking the seam.
The hardest case is an honest gap: retrieval finds nothing relevant, and the model has to choose between saying so and improvising. Without an explicit refusal path, it improvises.
How Flytebit Handles It
Our RAG systems treat grounding as a gate, not a guideline: citation-strict generation requires a retrieved source behind every claim, and an empty or near-miss retrieval produces a refusal the system can measure instead of an invention it cannot. Retrieval failures and grounding failures are scored separately in evals, because they are different bugs.