tutorials

Retrieval Patterns: From Naive RAG to Agentic Retrieval

14 min read

The demo went well. You showed the chatbot answering questions over the company wiki, everyone nodded, and the project got its budget.

Then it shipped. A support engineer asked about the refund policy that had been updated the previous Friday, and the system answered from the old one. With a citation. Pointing at a document that no longer existed.

Every team that ships a RAG system has a version of this story. The retrieval worked fine in the demo because the demo corpus is small, clean, frozen, and forgiving. The corpus you ship is none of those things, and the gap between the two is what this series is about.

A chat panel answering a refund-policy question with a citation chip pointing at a deleted document, while a newer version sits unreached behind it The answer looks right. The evidence is a ghost.

Every fix has a name

How a production RAG failure gets described decides what happens next. “The answers feel off” turns into a long thread and a follow-up call. “The corpus has three versions of that policy and the index keeps surfacing the stale one” gets fixed. Describe the failure precisely enough and the fix usually has a name too: add a reranker.

That is what patterns are for. Since the RAG paper landed in 2020, researchers have named the ways naive RAG fails and the patterns that fix each one. Once you know the names, you can route a bad answer instead of debating it: which stage failed, and which pattern fixes that stage.

This post is the map. Nineteen patterns, grouped by the problem each one solves, and what each one costs to run. The companion posts in this series go deep on one group at a time.

Two pipelines, and the cracks

An ingestion job splits documents into chunks, attaches metadata, embeds them, and indexes them for search. At query time, the retriever embeds the question, finds the closest chunks, and hands them to the model with instructions to answer from them. Retrieval decides what the model is allowed to know. The glossary carries the full mechanics.

The two pipelines of naive RAG: ingestion building the index, retrieval answering from it, with the cracks marked Two pipelines, and the cracks the demo never shows.

It works on the demo corpus because nothing in the demo pushes back. Small document set and clean text, plus questions that happen to match how the docs were chunked. The failures below are what happen when the corpus gets bigger, messier, full of contradictions, and more stubborn about being searched.

The cracks are specific:

  • Chunks split a policy mid-clause, so no single chunk carries the whole rule.
  • Three versions of the same document sit side by side in the index, and a deleted page keeps answering questions because nothing told the index it was gone.
  • The user’s phrasing and the document’s phrasing share little vocabulary.
  • Exact strings, like part numbers and error codes, are invisible to an embedding trained for meaning.
  • Nowhere in either pipeline does a step ask whether what came back was any good.

Each of those has a fix, and the fixes cluster into four families.

Four families of failure

Named RAG patterns are fixes. Each one aims at a different stage of the baseline: before retrieval runs, inside retrieval itself, beyond what the index can reach, or the check on what comes back. Aim at the wrong stage and the fix solves a problem you do not have.

Four families of RAG failure radiating from the baseline: weak retrieval, bad questions, wrong sources, and missing judgement Four places the baseline cracks.

The retrieval itself is weak. The index returns near-misses or misses exact terms. You see it in the traces: the right chunk sits at position twenty-three while the model answers from the near-misses ranked above it. That is a retrieval problem, and it has retrieval fixes.

The question is the problem. The corpus holds the answer, but the user’s phrasing, ambiguity, or implied follow-ups are what retrieval keeps tripping on. The giveaway is that a colleague with the same index finds the document in seconds, because they reshaped the question the system took literally. The fix has to happen before retrieval runs.

The answer does not live in embedded text. It sits in a database row, behind an API, inside a relationship between entities, in an image, or in data that is still moving. It surfaces as a fluent answer assembled from whatever prose ranked highest, while the real answer stayed in a row, a response body, a graph edge, or an image the index never touched. Fixing it means reaching a different kind of source.

The system cannot tell when it is wrong. Single-pass retrieval has no way to notice a bad result, let alone retry. The tell is that bad answers look identical to good ones in the logs until a user reports them. That is a judgement problem, and fixing it is what turns a pipeline into something closer to an agent.

The nineteen patterns

Every crack has a named fix. All nineteen in this series, indexed by the family each one belongs to:

Pattern Family The failure it fixes
Hybrid retrieval Better retrieval Exact terms the vector index misses: part numbers, error codes, names
Two-stage reranking Better retrieval Candidates that are close but not the right answer
Semantic caching Better retrieval Full pipeline cost on questions already answered
HyDE Better questions Questions and documents sitting far apart in embedding space
Branched RAG Better questions Ambiguous questions with several valid readings
Speculative RAG Better questions Predictable follow-ups waiting on a slow pipeline
SEAKR Better questions Retrieval the model did not need to pay for
Structured retrieval (SQL RAG) Different sources Answers that live in database rows, not documents
API-augmented RAG Different sources Data fresher than the last ingestion run
GraphRAG Different sources Answers spanning relationships across documents
Multimodal RAG Different sources Answers locked in images, charts, or video
StreamingRAG Different sources An index blind to what changed an hour ago
Self-RAG Systems that think Weak retrievals reaching the prompt unchecked
Corrective RAG Systems that think Bad results with no retry path
Adaptive RAG Systems that think One retrieval strategy forced onto every question
Agentic RAG Systems that think Questions needing decomposition across multiple lookups
IRCoT Systems that think Reasoning steps that each need their own retrieval
Memory-augmented retrieval Systems that think Sessions that forget yesterday's context
Modular RAG Systems that think Pipelines that cannot swap or test stages on their own

Each family gets its own post. Here is what each one contains.

Group 1: Better retrieval

The first upgrades most production RAG systems make, because they fix the stage teams hit first.

  • Hybrid retrieval. Vector search is good at paraphrase and terrible at exact strings: part numbers, error codes, statute citations, product names. Hybrid retrieval runs keyword search alongside the vector index and fuses the two rankings, so the system catches “error 0x8004” and “the login keeps failing” with the same pipeline. Costs an extra index and a fusion step; worth it on almost any real corpus.

  • Two-stage reranking. Vector retrieval is fast and approximate, so it returns candidates, not answers. A reranker takes the top fifty and re-scores each against the actual question, which is how near-misses stop reaching the prompt. The standard move is a cheap dual-encoder for recall and a cross-encoder for precision. Adds latency per query, and it is the single biggest upgrade most systems make.

  • Semantic caching. If a new question is close enough to a question the system already answered, serve the cached answer instead of paying for the full pipeline again. The win is cost and latency on high-volume systems with repetitive queries. The trap is tuning: set the similarity threshold too loose and yesterday’s answer leaks into today’s question, too tight and the cache never hits.

These three fix what comes back. None of them touch the question itself, which is where the next group starts.

Group 2: Better questions

When the corpus is fine and the question is the problem, the fixes happen before retrieval.

  • HyDE. Instead of embedding the question, the system generates a hypothetical answer first, embeds that, and searches for real documents that resemble the imagined one. “What are the refund terms” and the refund-terms document are far apart in embedding space; a made-up paragraph of refund terms is right next to it. One extra model call per query, and it falls apart when the hypothetical drifts away from the question.

  • Branched RAG. “I want to change my subscription” could mean upgrade, downgrade, cancel, or a billing problem. Branched retrieval explores each interpretation in parallel and either returns the best or shows each. Honest trade: it multiplies cost per query, so it earns its keep where ambiguity is common and wrong guesses are expensive.

  • Speculative RAG. Predicts the follow-up questions a user is likely to ask and prefetches their answers while the current one is still processing. When the prediction lands, the next answer feels instant. When it misses, you paid for retrievals nobody used. Makes sense in conversational systems where follow-ups are predictable.

  • SEAKR. Selective retrieval: the model answers from what it already knows when it is confident, and reaches for the corpus when it is not. Great for cost on mixed query loads, dangerous the moment your model’s confidence scores stop matching reality. An overconfident model skips the retrieval it needed.

A better question cannot rescue a bad source. When the answer was never in the index at all, you need the next group.

Group 3: Different sources

Vector indexes answer “what text is similar to this question.” These patterns answer questions where similar text was never going to work.

  • Structured retrieval (SQL RAG). “What was revenue last quarter” has an exact answer in a database, and similarity search over embedded documents is the wrong way to get it. Text-to-SQL turns the question into a query, runs it, and hands the model a real number. Deterministic, auditable, exact, and bounded by whatever the schema can express.

  • API-augmented RAG. The freshest data is not in your index at all; it sits behind endpoints. The system calls live APIs at query time for stock prices, inventory, order status, anything that changes faster than you can re-ingest. Latency and reliability now depend on someone else’s uptime.

  • GraphRAG. “Which suppliers feed the factories that make the parts in this product recall” is a traversal, not a search. Graph retrieval follows relationships between entities across multiple hops, which is how you answer connected questions a chunk index cannot even see. The cost is real: someone has to model the graph and keep it current, on infrastructure a vector store never asked for.

  • Multimodal RAG. Some answers live in images, charts, scanned forms, or video. A unified embedding space lets a text question retrieve a figure, a slide, a table, or a clip. Worth it when the corpus is mixed media; pure overhead when it is not.

  • StreamingRAG. Most indexes are batch artifacts: the document changed an hour ago and the index has not noticed. Streaming retrieval maintains an evolving, timestamped graph of what is happening now, for systems where stale means wrong. It is also the failure that opened this post: the refund-policy citation pointed at a document the index never should have kept.

Sources fixed, one problem remains. But even a perfect retrieval can produce a wrong answer, and nothing so far can tell.

Group 4: Systems that think

The largest group, and the one this series ends on, because it is where retrieval systems cross into agent territory.

The first three patterns are self-checkers.

  • Self-RAG rewrites weak queries and scores what comes back before it generates.
  • Corrective RAG goes further: drop the weak sources and search again, falling back to the open web when the corpus comes up short.
  • Adaptive RAG learns from past queries and feedback which retrieval strategy works for which kind of question.

Each one is the system grading its own work before the user ever sees it. The next three are the logical conclusion of that instinct.

  • Agentic RAG hands the whole loop to an agent: decompose the question, pick the tool, evaluate what came back, retry with a different source until the evidence holds.

  • IRCoT interleaves reasoning and retrieval so each step decides what the next lookup should be.

  • Memory-augmented retrieval carries context across sessions, because a system that cannot remember yesterday treats every question like a stranger’s.

Modular RAG closes the group. Calling it a pattern is generous: production systems assemble these pieces per use case rather than picking one. Query understanding, retrieval, ranking, generation, evaluation, each swappable and testable on its own.

There is a reason this group sits at the end of the map. A system that decides whether its own retrieval is good enough, picks sources, retries on failure, and knows when to stop has already become an agent with a retrieval tool. Our agentic AI work is what the far end of this road looks like, and the question of which side of that line your system belongs on gets its own guide later in this series.

When retrieval is the wrong answer

One more boundary before the map closes. Sometimes the right move is no retrieval at all.

Fine-tuning bakes knowledge into the weights. Right call when the knowledge is stable and the volume is high enough to amortize training; wrong call for facts that change every month, because every update means another training run.

Prompt engineering and in-context learning shape how the model answers without touching knowledge at all, which is enough when the gap is format and style rather than facts.

Context-aware generation skips retrieval by pre-loading what the model needs: user profile, session state, and the handful of records this request touches.

The choice between these and retrieval has costs on both sides and deserves the same rigor as the patterns above. The RAG vs fine-tuning decision gets its own guide later in this series. The short version: RAG changes what the model can see; fine-tuning changes the model itself. Confusing the two is how teams end up retraining a model to fix a stale index.

Who this series is for

Builders who have a retrieval system in production, or who are speccing one and want to skip the stage where it fails in front of users. If you know what an embedding is and why it matters, you have the prerequisites. If not, the glossary entries cover the vocabulary and this post will still be here.

The companion posts take one group each: the retrieval upgrades, the query patterns, the source patterns, and the systems that check their own work. Each is written to stand alone, so start wherever your production system is hurting right now.


Working With Flytebit

At FLYTEBIT TECHNOLOGIES, RAG development is a structured engagement rather than a workshop and a tool licence.

We build retrieval systems that have to survive a real corpus: ingestion treated as half the system, generation gated on citation, and the pattern chosen per use case rather than per demo. If your current system is failing one of the families above and you want it diagnosed, that is the work.

We run these patterns in our own products. PASSR reviews every pull request against context retrieved from the repository: hybrid retrieval, because an identifier the vector index misses is a bug the reviewer never sees. DOCKR generates documentation from the codebase’s structure as much as its text, which is the group-3 problem at codebase scale. TESTR builds test cases from retrieved source behavior, where a stale index means tests written against yesterday’s code. The StreamingRAG failure, in production.


Ready to get started?


Where retrieval ends and memory begins:

👉 Designing the Memory Layer for Production Agents

Memory-augmented retrieval in production depth: append-only archives, curated briefings, and why our memory design avoids vector search.

Measuring whether any of this works:

👉 Evaluating Agentic AI: Why Pass/Fail Tests Don’t Work for Agents

The evaluation discipline behind every pattern above: golden datasets, pass^k scoring, and systems that grade themselves.

Retrieval inside a bigger integration:

👉 Generative AI Enterprise Integration Best Practices

Where RAG sits inside a full enterprise GenAI deployment, and the integration mistakes that kill it before the patterns matter.


Key takeaways

  • ✅ Find the stage that failed: a bad RAG answer is a routing problem once you can name the fix
  • ✅ Patterns are vocabulary: "add a reranker" is a plan; "make it better" is a meeting
  • ✅ Four families of failure: weak retrieval, bad questions, wrong sources, missing judgement
  • ✅ Start with group 1: hybrid retrieval plus a reranker covers most production systems
  • ✅ Retrieval is not always the fix: fine-tune for behavior, retrieve for knowledge
  • ✅ The map ends at agents: a system that judges its own retrieval is already partway there

Frequently Asked Questions

What is the difference between RAG and agentic RAG?

Classic RAG is a fixed pipeline: retrieve once, generate once. Agentic RAG puts the retrieval inside an agent loop, where the system decides when to retrieve, which source to use, whether the result was good enough, and what to try when it was not. The difference is whether retrieval is a step in a fixed pipeline or a decision the system makes per query. The trade is adaptability for latency and cost, plus a debugging surface that sprawls across every step the agent takes.

How many types of RAG are there?

Depends who is counting. The research literature names far more than nineteen, but many are conference variants that never reach production. This series covers nineteen patterns that a team deploying a real system is likely to encounter or build, grouped into four families by the failure each one fixes.

Which RAG pattern should I start with?

Hybrid retrieval plus a reranker. It fixes the two most common baseline failures, missed exact matches and weak relevance ranking, without restructuring the pipeline or multiplying cost. Most production systems never need to go further than group 1; reach for the later groups when a specific failure demands it, not before.

Is RAG still relevant now that context windows keep growing?

Yes. Cost is one reason: stuffing a whole corpus into context is far more expensive per query than retrieving the relevant slice. The other is accuracy: retrieval with citations gives you an answer that can point at a source, which a giant context window cannot. Long context changed where the threshold sits, not whether retrieval is needed.

When should we fine-tune instead of building better RAG?

When the problem is behavior rather than knowledge. Fine-tuning fixes format, style, task skill, and domain vocabulary; retrieval fixes missing or changing facts. If your answers are wrong because the model did not know something, that is retrieval. If they are wrong because the model answers in the wrong shape, that is fine-tuning.

#RAG#RetrievalAugmentedGeneration#AIArchitecture#TutorialSeries#AIEngineering#LLM
Jayaveer Bhupalam

Written by

FLYTEBIT Technologies

Founder · Chief Technology Officer · AI & Digital Transformation Leader

Ready to Transform Your Business with AI?

Let's discuss how Agentic AI and intelligent automation can help you achieve your goals.