What is RAG (Retrieval-Augmented Generation)?
RAG & RetrievalA pattern that finds relevant documents at query time and places them in the model's context before it answers, so the response comes from your corpus instead of the model's training memory. Retrieval decides what the model is allowed to know.
Why It Matters
A language model does not know your data. It knows what was public at training time, and it fills gaps with fluent, confident invention. RAG is the standard fix: retrieve the relevant passages, put them in front of the model, and require the answer to come from them. It is how enterprise AI answers questions about contracts, runbooks, and internal docs without a retraining cycle.
The Two Pipelines
RAG is two pipelines, not one.
Ingestion. Splits source documents into chunks, attaches metadata, embeds them, and indexes them for search.
Retrieval. Takes the user’s question, finds candidate passages (usually hybrid search over keyword and vector, then a reranker), and hands the top results to the model with instructions to answer from them and cite them.
Ingestion is half the system. Chunk boundaries decide whether a table keeps its header. Index freshness decides whether last week’s policy update is findable at all. Teams that treat ingestion as a one-time import end up debugging retrieval with a generation mindset, and the bugs are all upstream.
Where It Breaks
TrustNLP’s taxonomy counts 33 distinct RAG failure modes, and most are retrieval failures rather than generation failures. Chunking separates a clause from the condition that qualifies it, and the index goes stale while the documents move on. Retrieval returns near-misses that read plausibly enough to ground the answer in the wrong place.
Tutorial RAG (embed, vector search, generate) dies on the real corpus because none of these cases appear in demo data. The fix is unglamorous: eval questions built from real user queries, reranking tuned on the actual corpus, an index that re-ingests when documents change, and citation-strict generation that refuses to answer without a source.
How Flytebit Handles It
Our RAG development work treats ingestion as half the system and gates generation on citation: an answer that cannot point at a retrieved passage does not reach the user. How we evaluate retrieval quality before launch is covered in Evaluating Agentic AI.