RAG Is Not a Product Strategy
Retrieval can be powerful, but users do not wake up wanting RAG. They need trusted knowledge at the moment a decision has to be made.
Reusable artifact
RAG Strategy Canvas
- 01Decision: what decision does the user need to make?
- 02Source: which source of truth must the system trust?
- 03Evidence: what citation, excerpt, timestamp, or provenance must be visible?
- 04Recovery: what happens when the answer is missing, stale, conflicting, or unsafe?
- 05Eval: what 20 real questions prove the workflow is reliable enough to ship?
Implement fast
- Collect 20 real user questions, including five ugly questions where the answer is missing or ambiguous.
- Score answer quality, citation quality, freshness, and recovery behavior.
- Prototype around the decision and evidence, not the retrieval architecture.
- Do not scale the corpus until the small eval set behaves well.
RAG is an implementation pattern, not the point.
A support agent does not need RAG. They need the right answer from the right source at the right moment. An employee does not need RAG. They need to stop searching across twelve internal pages that disagree with each other. A banker does not need RAG. They need a current, compliant, source-grounded response they can trust enough to act on.
Retrieval-augmented generation can be the right tactic. It can also become a very expensive way to avoid making product decisions. If the team has not defined the workflow, the decision, the source of truth, the evidence requirement, and the recovery behavior, retrieval will not save the product strategy.
The news is moving toward verifiable retrieval.
Google's expansion of Gemini API File Search toward multimodal support, metadata filtering, and page-level citations is a useful signal. The market is moving away from vague knowledge-base magic and toward verifiable source use across messy enterprise materials.
That matters because enterprise knowledge does not live neatly in one place. It lives in PDFs, screenshots, docs, spreadsheets, tickets, policies, decks, and conversations. A useful AI product has to help users inspect the evidence, not merely receive an answer.
Start with the decision.
The fastest way to improve a RAG initiative is to stop saying RAG for an hour. Ask: who is deciding, what do they need to know, what source should they trust, what evidence do they need to see, and what should happen when the system is not confident enough?
Those answers will tell you whether you need file search, metadata filters, a curated corpus, a policy engine, a manual review path, a freshness check, or no AI at all. Architecture should serve the strategy. It should not replace it.
Use ugly questions early.
Most retrieval demos fail because the eval questions are too clean. Real users ask incomplete, mixed, contradictory, policy-sensitive, stale, and sometimes unanswerable questions. That is where the product earns trust.
A 20-question eval with five ugly questions can save weeks of false confidence. If the system can answer with evidence when the source exists, admit uncertainty when it does not, and explain what to do next, you are closer to a product. If it only produces polished paragraphs, you are still in demo land.
Social carousel
The shareable artifact
