RAG for business knowledge: build assistants that can show their sources

What retrieval-augmented generation does

A typical RAG system receives a question, searches an indexed collection for relevant passages, adds selected passages to the model's context and generates an answer. A useful implementation preserves references to the retrieved material so the answer can be checked. Retrieval improves access to current knowledge, but it does not guarantee that every answer will be correct.

Prepare content before embedding it

Start with a source inventory. Identify document owners, publication dates, access rules, formats and update frequency. Remove obsolete copies and decide how versions will be represented. Extract text while preserving headings, tables, identifiers and source locations. Poorly extracted content creates poor retrieval even when the model is capable.

Choose chunk boundaries that preserve meaning

Large documents are often split into smaller passages called chunks. Fixed character limits are simple, but they can cut a procedure in half or separate a table from its explanation. Prefer structure-aware splitting that respects headings, paragraphs, list items and document sections. Store metadata such as source ID, section, version, owner and access label with each chunk.

Use retrieval that matches the question

Vector similarity can help find semantically related passages, while keyword search can be better for product codes, policy names or exact phrases. Hybrid retrieval combines the strengths of both. A reranking stage can then reorder candidate passages based on relevance to the full question. Evaluate these choices on real queries rather than selecting them only by benchmark claims.

Enforce permissions before content reaches the model

Security belongs in the retrieval layer. Apply the user's authorization constraints when searching and fetching passages; do not rely on a prompt telling the model to ignore documents the user should not see. Re-check permissions when source content is updated or a user changes role. Treat retrieved text as data, not as trusted instructions.

Make citations useful

A citation should point to a source that a person can open and inspect. Include document title, version or date, and a page or section reference where available. Do not show citations for passages that were not actually retrieved. If the available evidence does not answer the question, the assistant should say so and route the user to an owner.

Evaluate retrieval and generation separately

Test whether the correct passages are retrieved before measuring the final answer. Useful retrieval measures include recall at a chosen cutoff and whether the source actually contains the answer. Generation checks can measure factual support, citation correctness, completeness and appropriate abstention. Include stale documents, conflicting versions, ambiguous questions and access-control edge cases.

Keep the index fresh

Define how content changes move into the index. A controlled pipeline can detect additions, updates and deletions, reprocess affected sections and preserve traceability. Monitor indexing failures and stale-source age. For time-sensitive content, show the source's last-updated date and clarify when a human owner must confirm the current rule.

Deliver a trustworthy knowledge assistant

A strong RAG product combines reliable document processing, permission-aware retrieval, concise answers, inspectable sources and honest fallbacks. The model is one component in a larger information system. Its quality depends on the quality, governance and maintenance of the knowledge behind it.