What is retrieval-augmented generation (RAG)?

Retrieval-augmented generation, or RAG, is a way to help an AI answer questions using information you select. The system searches your documents first, then gives relevant passages to the AI model to use in its reply. It is useful when the answer depends on your policies, products or procedures rather than general knowledge.

Short answer

• RAG combines search with AI-generated wording.
• It can use current business documents without retraining the underlying AI model for every update.
• It can show source links so a person can check an answer.
• It does not guarantee accuracy. Poor search results or conflicting documents can still produce a wrong answer.

What RAG actually does

A general AI model can write a plausible response to a question, but it may not know your latest delivery policy or internal process. In a RAG system, the application searches a chosen collection of material and includes relevant passages with the question. The model then writes a response using that material.

This is different from training a model on your documents. Your source documents remain separate and can be updated. The system must still pick up the updated version before it can use it. It is also different from ordinary search: search offers documents or snippets, while RAG attempts to explain the answer in plain language. Sometimes ordinary search is the better choice, especially when a reader needs the full document.

A RAG application can power an internal help tool, a customer-facing answer engine or an assistant for support staff. The method is the same, but the access rules and tolerance for mistakes differ.

How it works

1. Prepare the material. Choose approved documents, remove outdated copies and decide who may see each one.
2. Make it searchable. The application breaks long material into smaller passages and creates a way to find relevant passages later. This is often called an index, like the index of a book.
3. Retrieve passages for a question. When someone asks a question, the application searches the index and selects material that appears relevant.
4. Generate an answer. The question and selected passages go to the AI model, which writes a response. A well-designed answer points back to its sources.
5. Check the result. The application or a person checks whether the retrieved material really supports the answer. When it does not, the system should say it cannot answer or hand the question to a person.

Notice that finding the right passage comes before writing. If the search misses a crucial exception, a fluent response can still be wrong. Improving the model alone will not fix a missing policy or a poor search result.

A small business example

A supplier has product manuals, warranty terms and installation guides. Staff answer the same questions about compatibility and care, but finding the right passage takes time. A RAG tool could search the approved guides and draft a response that links to the relevant manual section.

If a customer asks whether a particular installation is covered under warranty, the tool may find a general warranty paragraph but miss an exception. That question should go to a staff member who can check the product, purchase and installation details. The system should not turn a general paragraph into a promise about a specific claim.

This is an illustrative example, not a client result. It shows why source links and a route to a person matter even when the documents are good.

What RAG can and cannot solve

RAG is useful when information is spread across many documents, changes often enough that general AI knowledge is unsuitable, and people ask questions in varied wording. It can reduce time spent locating material and help staff produce more consistent draft answers.

It cannot repair unclear policies, decide which of two conflicting documents is authoritative, or check facts that were never recorded. It also cannot automatically enforce privacy. If a confidential document is included in a collection available to everyone, the system may reveal it. Access to the retrieved passages must match what the questioner is allowed to see.

A small, stable FAQ may not need RAG at all. A well-organised help page or ordinary site search could be simpler and easier to maintain.

How to test it

Begin with one audience and a limited set of approved documents. Ask real questions people have asked before. Include questions with a clear answer, questions with an exception, and questions the documents cannot answer. For each response, check three things: Did the system find the right source? Does the wording match that source? Did it admit when the source was insufficient?

Track failures by cause. A missing document needs a content fix. A wrong passage needs a search fix. An answer that goes beyond a correct passage needs a response rule or more review. Keep these causes separate so the team does not waste time tuning the wrong part.

After a pilot, name the person responsible for keeping sources current. Retest when policies or documents change, and periodically check that private material stays within the intended audience.

What to watch out for

A citation is a useful clue, not a guarantee. The model may cite a related passage that does not actually support its claim. It may also combine passages that belong to different products, regions or dates. People should check consequential answers against the original source.

Retrieval adds work and running cost. The system must store a searchable copy of the material, update it and search it for each question. A narrow use case helps you judge whether that extra complexity earns its place.

How we can help

We can help prepare a suitable document collection, build a small pilot and test answers against real questions. Our AI Solutions service covers this kind of work.

If your team spends time searching its own documents, tell us what people ask and where the answers currently live. We can help decide whether search, a clearer FAQ or RAG is the right fit.

Facts checked on 29 October 2026. Sources: Microsoft's explanation of RAG and its limitations (opens in a new window) and Amazon's explanation of retrieval and cited answers (opens in a new window). The supplier example is illustrative, not a client result.