RAG in AI: definition, how it works and business use cases
RAG (retrieval-augmented generation) lets a language model answer from your documents and cite its sources. Here is how it works, what it is used for in business, and what you need to plan to put it into production.
- RAG connects a language model to your document base: it first retrieves the relevant passages, then writes an answer grounded in them.
- It reduces the risk of hallucination and improves traceability without eliminating it: evaluation remains essential.
- The quality of a RAG system depends mostly on the data: cleaning, chunking, metadata and access rights.
- For knowledge that changes often, RAG is generally preferable to fine-tuning.
- In production, document-level access rights, continuous evaluation and cost tracking matter as much as the choice of model.
RAG has become the reference architecture for building an assistant that answers from a company’s documents. The principle is simple; implementing it is much less so: quality depends on the data, on access rights and on how answers are evaluated. This article explains what RAG is, how it works, where it is useful and what you need to plan to run it in production. It is part of our approach to generative AI for business.
What is RAG in AI?
Definition: retrieval-augmented generation
RAG stands for retrieval-augmented generation. In its guide published in November 2024, the DGE (the French Directorate General for Enterprise) defines it as a technique that improves the answers of generative AI models by feeding them knowledge drawn from the company’s own databases. In other words, the model “consults” your documents before answering.
A RAG system has two modules:
- a retrieval module, which searches the corpus for the documents or extracts most relevant to the question asked;
- a generation module, a large language model (LLM) that writes the answer from the question and the retrieved extracts.
Why an LLM on its own does not know your documents
An LLM is trained on a general-purpose corpus, frozen at a given date. It knows nothing of your procedures, your contracts, your product sheets or anything that has changed since it was trained. Pasting entire documents into every question is expensive, limited by the size of the context window, and does not solve the question of access rights. RAG gives the model, for each question, only the passages it needs.
RAG and hallucinations
A hallucination is an answer that is plausible but wrong. By grounding the answer in identified extracts, RAG reduces this risk and improves traceability: the answer can cite its sources, which the user can check. It does not eliminate the risk. If the search returns the wrong passage, or if the model extrapolates beyond the extracts, errors remain possible. That is why evaluation is an integral part of any RAG project.
How RAG works, step by step
1. Ingestion and chunking
Documents are collected from their sources (file shares, SharePoint, Confluence, ticketing tools, databases), converted to text, including through optical character recognition for scanned PDFs, then cleaned: duplicates, outdated versions, repeated headers and footers. The DGE guide also recommends removing documents that contradict each other.
They are then split into segments (chunks). According to the DGE, the chunking method and size are a crucial parameter that calls for dialogue between technical and business experts. Chunks that are too small lose their context; chunks that are too large drown the useful information. Each chunk is enriched with metadata: source document, section, date, owning department, confidentiality level.
2. Embeddings and the vector database
Each chunk is converted into a vector (an embedding), a sequence of numbers that represents its meaning. The vectors are indexed in a vector database. When a question arrives, it is vectorised in turn, and the database finds the closest chunks.
Purely semantic search has its limits: it handles exact references (contract numbers, item codes) poorly. The DGE distinguishes three families of search: lexical, semantic and hybrid, the last combining the other two. In practice, hybrid search is the right starting point for an enterprise corpus.
3. Retrieval, reranking and generation with citations
The top-ranked chunks are often reordered by a reranking model, more accurate but slower, which keeps only the most relevant ones. They are then inserted into the prompt along with the question and instructions: answer only from the extracts provided, cite the sources, say “I don’t know” if the information is missing. The answer is returned with references to the documents used.
Examples of RAG in business
The DGE guide lists several recurring use cases. Here are the most common, with the associated document base.
HR assistant and internal procedures
Employees query company agreements, the applicable collective agreement, procedures and the internal FAQ in natural language. The HR team is relieved of repetitive questions, and the answer points to the reference text.
Customer support and knowledge base
The agent receives a draft reply built from the knowledge base, product sheets and resolved tickets. They validate or correct it before sending: the human stays in the loop.
Legal, contracts and tenders
Finding the liability clauses in all of a supplier’s contracts, or checking that a tender response is consistent with past commitments. Note: the DGE points out that RAG is less suited to queries that require going through an entire corpus, such as a comparative summary of many documents. These cases call for complementary architectures.
Technical documentation and industrial maintenance
Technicians query equipment manuals, bills of materials and the history of interventions. This is a case where the quality of the source documents (scans, diagrams, tables) weighs heavily: the DGE notes that RAG generally performs better on text than on tables.
RAG vs fine-tuning vs agents
Three approaches are often confused. They meet different needs and can be combined.
| Approach | What it does | Suitable when | Limits |
|---|---|---|---|
| RAG | Gives the model extracts from your documents for each question | Knowledge changes often, source traceability matters | Depends on the quality of the search and the data |
| Fine-tuning | Adjusts a model’s parameters on your examples | You need a very specific style, format or vocabulary | Knowledge frozen at training time, retraining cost, sources cannot be cited |
| AI agent | Chains steps and calls tools (APIs, search, calculation) | The task requires acting, not just answering | Requires stricter safeguards and oversight |
A few variants worth knowing:
- Agentic RAG: an agent breaks a complex question down into sub-questions, queries one or more bases, then assesses whether the information retrieved is sufficient. Amazon Bedrock and Azure AI Search now offer this kind of retrieval.
- GraphRAG: a knowledge graph captures the links between the company’s concepts. The DGE guide cites it as a pre-processing option for domains with complex vocabulary.
- MCP (Model Context Protocol): a protocol that lets an agent call tools and data sources in a standardised way. A knowledge base can thus be exposed to an agent as a tool.
Putting RAG into production
Access rights and document isolation
This is the most frequently underestimated point. An assistant must never return an extract that a user would not be allowed to open in the source tool. Two principles:
- filter at retrieval time based on the user’s identity, not only at display time;
- inherit permissions from the sources rather than copying them by hand.
Managed platforms are making progress here. The Amazon Bedrock documentation states that managed knowledge bases offer document-level permission filtering, using access control lists, at retrieval time (except for the web crawler connector). Azure AI Search describes document-level security filtering and inheritance of Microsoft Entra ID permissions for some sources. In both cases, check the exact behaviour for each of your sources.
Evaluating quality
A RAG system without measurement is a black box. The DGE guide cites three complementary methods:
- user feedback, quantitative (thumbs up or down) and qualitative;
- quantitative metrics: hallucination rate, relevance of the retrieved documents, response time;
- LLM as a judge: a model evaluates the generated answers by comparing them with reference answers.
In practice, build a reference question set with the business teams, measure retrieval quality (are the right passages found?) separately from generation quality (is the answer faithful to the passages?), and rerun the set whenever the model, chunking or prompt changes. These practices belong to LLMOps, covered on our page on putting AI into production.
Costs and hosting
There are three main options:
- Amazon Bedrock Knowledge Bases: with a managed knowledge base, AWS handles ingestion, indexing, storage and retrieval, with connectors (Amazon S3, SharePoint, Confluence, Google Drive, OneDrive, web crawler), citations in answers and reranking. With a customer-managed knowledge base, you choose the vector store (for example Amazon OpenSearch Serverless or Amazon Aurora) and keep control of the pipeline.
- Azure AI Search: hybrid search, semantic ranking, built-in chunking and vectorisation, document-level security filtering. The service also underpins Foundry IQ, the knowledge layer for agents in Microsoft Foundry.
- Open-source components deployed on your own infrastructure (vector database, embedding model, generation model): more control, more operational work.
The cost items are always the same: ingestion and embeddings, storage and indexes, search queries, reranking and, above all, generation tokens. Pricing changes often: estimate on your real volume using the official pricing pages. As a public order of magnitude, the DGE guide cites, based on interviews with practitioners, around 100,000 euros for an installation by an integrator, and points out that costs depend heavily on usage volume. These figures date from 2024.
Sovereignty and GDPR
If your document base contains personal data, the CNIL (the French data protection authority) states that the organisation connecting the system to its own base is the controller for that processing. For sensitive data, it generally considers on-premises deployment safer and, for the cloud, stresses the need for a data processing agreement and safeguards for transfers outside the European Union. The DGE mentions SecNumCloud-qualified cloud services (a French security qualification) as an alternative for high protection needs. In practice: choose a European region, check that the provider does not use your data to train its models, pseudonymise what can be pseudonymised and involve your data protection officer (DPO). Our page on cloud security and governance covers these foundations.
Common mistakes
- Indexing the entire network share without sorting: duplicates, outdated versions and contradictory documents degrade the answers.
- A single chunking strategy for every document type: a contract, an FAQ and a technical manual are not split the same way.
- Ignoring access rights in the prototype, then discovering that everything has to be reworked before production.
- No reference question set: there is no way of knowing whether a change improves or degrades the system.
- No citations: users cannot check, and trust collapses at the first mistake.
- No corpus owner: without someone responsible for keeping documents up to date, the assistant ages with them.
- No cost tracking by use case.
Checklist for scoping a RAG project
- Does the use case rely on a large, living internal document base?
- Is source traceability required?
- Do the source documents have an owner responsible for keeping them up to date?
- Are the access rights of the sources known and retrievable?
- Have personal or sensitive data been identified and the DPO involved?
- Can a reference question set be written with the business teams?
- Is the hosting model (managed, cloud, on-premises) compatible with the sensitivity of the data?
Conclusion: start from a well-controlled corpus
RAG is often a company’s first custom generative AI project, because it makes use of an asset that already exists: its documents. Its success depends less on the model chosen than on the care taken with data, access rights and evaluation. If you are considering an AI assistant over your internal documents, Feekr can help you scope the use case, choose the architecture on AWS or Azure and put it into production with your teams. To place this project within a broader approach, read our method for adopting AI in your business, or explore our generative AI for business offering.
Frequently asked questions
What is RAG in AI?
RAG stands for retrieval-augmented generation. The system searches your documents for the extracts relevant to a question, then passes them to a language model that writes the answer based on them. The model thus “consults” your data at the moment it answers, without being retrained.
Can you give an example of RAG?
An internal HR assistant answers the question “how many days of leave do I get for a wedding?” by retrieving the relevant clause of the company agreement, then phrasing the answer with a link to the document. If the agreement changes, you simply update the indexed document.
Does RAG eliminate hallucinations?
No. The guide published by the DGE (the French Directorate General for Enterprise) states that it reduces the risk of hallucinations compared with a model used on its own and improves traceability. A search that returns the wrong passages, or a model that extrapolates, is still possible: answers must be evaluated regularly and sources displayed so they can be checked.
What are a vector database and an embedding?
An embedding is a numerical representation of a text that places texts with similar meaning close to one another. A vector database stores these representations and quickly finds the ones closest to a question. It is often combined with keyword search, which is known as hybrid search.
How much does an Amazon Bedrock knowledge base cost?
The cost depends on the mode chosen (managed knowledge base or one with your own vector store), the volume of documents, the embedding model, the number of queries and the generation model. Prices change: refer to the official Amazon Bedrock pricing page and estimate on your real volume before committing.