What is Hallucination?

Why It Matters

The danger of a hallucination is not that the model invents things. It is that invented text is indistinguishable from grounded text: same tone, same structure, same confidence. The reader cannot tell which claims came from a document and which came from nowhere.

In a chatbot a hallucination is a wrong answer. In a RAG system it is a wrong citation filed into a report somebody trusts. In an agent it is a wrong action taken on a fact that never existed.

Why It Happens

The model’s job is to continue plausibly, not to know. When retrieval returns nothing useful, or returns a near-miss, the model does what it was trained to do and fills the gap from memory. Prompt instructions soften this tendency without removing it, because the instruction travels through the same suggestible channel as everything else the model reads.

Where It Breaks

Partial grounding is the subtle failure: the model starts from retrieved text, then lets training memory fill the edges, and the seam between evidence and invention is invisible in the output. Near-miss retrieval is the other one. The model grounds faithfully in the wrong document and produces an answer that is cited and wrong.

Hallucinations are also silent failures. The silent-failures taxonomy found 70 percent of silent failures in a production agent runtime were caught by human observation. Not tests, not governance checks. People reading output.

How Flytebit Handles It

The fix is architectural: retrieve real context, constrain generation to it, refuse when nothing adequate was retrieved. Our RAG builds default to citation-strict generation, so an unsupported claim does not ship and an uncovered question comes back as a logged refusal. Hallucination gets scored as its own metric in the eval harness, because it fails differently than retrieval does and counting them together hides both.