Make answers traceable
to the source.

1. Define the knowledge boundary

Start with a collection whose owner can confirm what should be included. SharePoint libraries, Confluence spaces and controlled file repositories are possible sources, but they are not automatically suitable in their entirety. Separate approved guidance from drafts, archived records and informal discussion. A broad document collection can increase confusion rather than improve answers.

Describe the questions the assistant is intended to address. An operational policy assistant and a technical product assistant need different sources and review arrangements. Record the authoritative source when documents disagree. If nobody can determine which instruction takes precedence, resolve that issue before indexing it. A conversation interface cannot create authority that the original information lacks.

2. Understand retrieval before generation

Retrieval-augmented generation, commonly called RAG, retrieves relevant material and supplies it to a model when answering a question. It differs from training a model on the organisation’s documents. This separation can make source updates easier to manage, but it does not guarantee factual answers. The retrieval stage may miss relevant material, and the model may misinterpret what it receives.

Divide documents into passages that retain enough context to be meaningful. Keep headings, document titles and source identifiers alongside the text. Keyword search is useful for exact product codes and names. Vector search compares numerical representations of meaning and can help with differently worded questions. A combined approach may be appropriate; choose it by evaluating your actual question types.

3. Apply access rules before retrieving text

An employee must not receive information through the assistant that they could not access in the source system. Enforce permissions while selecting passages, not merely by asking the model to avoid restricted information. Identity, group membership and document permissions must remain application responsibilities. Ensure cached answers cannot cross user or permission boundaries.

Check how a source connector represents access-control lists and how changes propagate into the index. Deleting a document or removing access should trigger corresponding updates. A source link that refuses to open is not an adequate protection if the answer already revealed its contents. Treat a retrieval index as another copy of sensitive information and secure it accordingly.

4. Make uncertainty and evidence visible

Ask the assistant to answer from retrieved material and provide links to the passages used. Citations must resolve to genuine sources and support the specific claim beside them. A plausible document title is not evidence. Give the interface a clear way to say that relevant information was not found or that approved sources disagree.

Keep instructions in retrieved documents separate from application instructions. A document can contain a prompt injection: text that attempts to redirect the model or reveal protected information. Restrict available tools and avoid giving a knowledge assistant unnecessary write access. Test unrelated questions, ambiguous wording, outdated documents and requests to ignore its boundaries, as well as ordinary questions.

5. Assign responsibility for changes

A knowledge assistant service can be scoped to source assessment, indexing, permission checks, an answer interface and an evaluation set. Agree which repositories and user groups are included. Specify how additions, deletions and permission changes are detected. Include a route for reporting an incorrect answer that preserves enough source context for someone to investigate it.

Review answer quality separately from retrieval quality. If the right passage was not retrieved, changing the prompt may not help. If the passage was retrieved but misunderstood, examine answer instructions and model behaviour. Maintain content owners who can correct the original material. Removing the assistant’s answer alone leaves the same error available to future users.

Consult the official documentation for RAG with Azure AI Search and Confluence when examining source and search options. For permissions, rely on the documentation for the exact connector and deployment you are considering.