What is Retriever?

Why It Matters

A model can only reason over what it can see, and in a RAG system everything it sees arrives through the retriever. The retriever is the front door. Whatever it misses does not exist for the model, and whatever it returns shapes the answer whether the passage is right or not.

This is why retrieval quality is the variable the whole pipeline hangs on. Better generation cannot compensate for documents that were never found.

How It Works

The retriever takes the query (or the agent’s intermediate need) and returns candidate passages from the index. Production retrieval is usually hybrid search: keyword matching for exact terms and identifiers, vector search for paraphrase and intent, fused into one candidate set.

The candidates then pass to a reranker that re-orders them by actual relevance before the top results enter the context window. Retrieval is recall; reranking is precision.

Where It Breaks

TrustNLP’s taxonomy of 33 RAG failure modes puts most of them on the retrieval side, not the generation side. The index goes stale while the documents move on. Chunking cut a clause away from the condition that qualifies it. The retriever returns near-misses: passages plausible enough to ground the answer in the wrong place, which is worse than returning nothing.

The diagnostic mistake is debugging retrieval failures with a generation mindset: rewriting the prompt when the problem is what the prompt never received.

How Flytebit Handles It

Our RAG development work treats ingestion and retrieval as half the system: hybrid search as the default, index freshness wired to document change, and retrieval quality measured against eval questions built from real user queries. The evaluation method is covered in Evaluating Agentic AI.