What is retrieval augmented generation?
Retrieval augmented generation connects a language model to information available outside its original training data. When a question arrives, the application retrieves relevant material and includes it in the model's context. The model then generates an answer informed by those retrieved passages.
AWS describes RAG as grounding a model's response in external knowledge without retraining the model for each information update. That makes it relevant to help centers, product documentation, and other maintained information sources. The application's retrieval, permissions, and evaluation choices still determine whether the result is useful.
This guide uses an original fictional project-management app, TaskHarbor. Its help assistant should answer questions about TaskHarbor's export and retention rules. The downloadable miniature corpus contains only synthetic public documentation and one restricted internal note. It demonstrates design decisions rather than an actual deployed AI service.
Follow the ingestion and answer paths
During ingestion, select approved documents, attach ownership and access metadata, divide long material into useful passages, and build a searchable representation. A vector index is common, while keyword search or a hybrid approach may also be appropriate. Preserve source IDs, titles, versions, and passage locations.
At question time, establish the requesting user's identity and permitted source set. Retrieve candidate passages, check their relevance, and give the model the supported context with instructions about the answer format. Return source references that a person can inspect. If the evidence does not answer the question, the response should say what is missing.
Updating a source is a separate operational task. When a policy changes, replace or retire the indexed passages and keep enough version information to investigate older answers. Deleting a source from a shared folder while leaving an old copy searchable defeats that maintenance process.
Inspect the original TaskHarbor corpus
The JSON corpus contains four short documents. Public document TH-101 says that workspace owners can request a CSV export and that the prepared link expires after 24 hours. TH-102 says deleted projects enter a seven-day recovery window. TH-103 says invoice questions should go to a billing contact.
TH-INT-1 is an internal planning note about a possible audit export. It is marked internal and explicitly not a published commitment. A public help assistant must exclude it before selecting context. An internal label alone is metadata; the application must enforce an authorization rule that uses it.
For the question 'How long can I use my CSV export link?', TH-101 supports an answer of 24 hours. For 'Can I export a complete audit log today?', the public corpus provides no answer. The expected behavior is an explicit lack of supporting information and an appropriate route to ask the team, rather than using the restricted roadmap note.
| Question | Permitted supporting evidence | Expected behavior |
|---|---|---|
| Who can request a workspace CSV export? | TH-101, public export policy | Answer workspace owners and cite TH-101 |
| When does the prepared export link expire? | TH-101, public export policy | Answer 24 hours after preparation and cite TH-101 |
| How long can a deleted project be recovered? | TH-102, public project recovery | Answer seven days and cite TH-102 |
| Is a complete audit export available today? | No public source establishes availability | State the evidence gap; do not reveal or rely on TH-INT-1 |
Choose RAG, a direct prompt, or fine-tuning
If a task uses a short, stable passage that comfortably fits into the model's context, supplying that passage directly may be enough. Retrieval adds a way to select among a larger or changing source set. It also adds indexing, access control, maintenance, and evaluation work.
Fine-tuning changes model behavior through training examples. It can be useful for a recurring output style or task behavior, but it is not a substitute for checking current customer-specific facts. A product may combine fine-tuning with retrieval when each addresses a different need.
Start with the simplest approach that meets the actual requirements. A deterministic lookup may serve an exact policy question better than generated prose. Compare results on representative questions before adding architectural complexity.
| Need | Possible starting approach | Question to verify |
|---|---|---|
| One short, approved document | Supply the document directly in context | Does the whole source fit and remain maintainable? |
| Many changing documents with source references | Retrieve permitted evidence before generation | Can the system find the right current passages? |
| Consistent behavior or output structure across examples | Prompt design, then consider fine-tuning if needed | Does evaluation show a behavioral gap that training addresses? |
| Exact account state or a transaction | Authorized application lookup or workflow | Is the returned fact authoritative and is the action permitted? |
Evaluate retrieval and the answer separately
First check whether retrieval found the passages needed for a known answer and excluded material the requester cannot access. An elegant answer cannot recover a fact that never reached the model. Keep expected source IDs beside the test questions so you can inspect retrieval failures.
Then evaluate whether the answer follows the retrieved evidence. Check the specific claim beside each citation, missing conditions, conflicting versions, and unsupported additions. A citation can point to a real document while failing to support the sentence it accompanies.
Include answerable, unanswerable, ambiguous, stale-source, and permission-boundary cases. Treat instructions inside source material as content to interpret rather than authority to change system behavior. The Dify retrieval guide shows how to inspect these questions in one product-specific workflow.
Plan for freshness, cost, and failures
Name an owner for the corpus and a process for additions, corrections, and removals. Record when an index was last updated and how a failed update is detected. A help assistant needs a way to avoid presenting an old policy as current when the maintenance path stops working.
Measure cost and latency across ingestion, retrieval, and answer generation using the actual configuration. Keep useful context focused, but do not remove critical conditions merely to shorten a prompt. The AI SaaS cost guide helps define budgets and operational controls.
Give the user a useful failure path: show the uncertainty, provide the supported source when one exists, and explain how to reach a human or authoritative workflow. Keep logs sufficient for diagnosis while excluding unnecessary source contents and private user material.
Download the corpus and evaluation worksheet
The JSON file includes the four synthetic documents, access labels, public-user test questions, expected sources, and answer constraints. The worksheet gives you a separate record for retrieval quality, answer support, permissions, freshness, and the next correction. These downloads do not call a model or retrieve live information. Sources checked October 27, 2026.