RAG is not a product

Retrieval augmented generation gets sold as a category. It is not one. It is a technique for a specific failure: the model does not know something that exists in writing somewhere you control.

That distinction matters because it tells you when to reach for it and, more usefully, when not to.

What retrieval actually fixes

If the honest answer to "why did the model get this wrong" is "it had never seen that document", retrieval helps. If the answer is anything else, it will not.

Where it gets used and should not be

• The knowledge exists but is contradictory across sources. Retrieval will surface the contradiction, not resolve it.
• The task needs multi step reasoning over the whole corpus. Retrieval returns fragments, and fragments are the wrong input.
• The documents are stale. Retrieval makes stale answers more confident, not less.
• The real problem is that nobody agrees what the correct answer is. That is a process problem wearing a technical costume.

A cheaper test

Before building anything, take twenty real questions and hand a colleague the documents you would have retrieved. If they can answer confidently from those fragments alone, retrieval will probably work. If they need to read three more things first, you have a different problem and a retrieval system will bury it.

This test costs an afternoon. We run it on every engagement that arrives asking for RAG, and it changes the scope more often than not.