Retrieval-Augmented Generation (RAG) combines search with a language model. At request time, the application retrieves relevant source material, places selected passages in the model context, and generates a response that can cite the underlying content. This gives an application access to private or changing information, but retrieval does not guarantee that an answer is complete, correct, or authorized. Microsoft Learn: RAG and indexes in Microsoft Foundry

Why grounding matters

A model's training data may not contain an organization's current policies, products, or internal procedures. RAG can supply relevant source material for those questions. The application should still handle missing, conflicting, stale, or low-confidence evidence explicitly rather than prompting the model to answer regardless.

A basic RAG request has three stages:

  1. Retrieve: Search an index or data store for relevant content.
  2. Augment: Add selected content and the user question to a model request.
  3. Generate: Produce a response, ideally with citations that point back to source material.

The citation should be checked by the application against the retrieved results. A generated citation string is not proof that a source supports the answer. Microsoft Learn: RAG and indexes in Microsoft Foundry

A production RAG architecture

The diagram shows a common query and ingestion path: source content is prepared and indexed, Azure AI Search retrieves passages, the application builds grounding context, and a model generates a response. Access control, safety checks, and evaluation apply across the flow.

Azure RAG architecture showing source content flowing through preparation and indexing, Azure AI Search retrieval, grounding context, and model generated answers with citations.
RAG uses retrieved enterprise passages to ground a model response and its citations.Open full-size imageDownload editable draw.io file

A practical implementation includes:

  1. Data ingestion: Connect to approved documents, manuals, policies, tickets, and structured records.
  2. Document processing: Extract text and structure, preserve source identifiers, and split content into useful passages.
  3. Embedding and indexing: Generate compatible embeddings where vector search is required, and store text, metadata, and vectors in the index.
  4. Retrieval: Run keyword, vector, or hybrid queries with relevant filters.
  5. Prompt construction: Add the user question, selected evidence, application instructions, and citation requirements.
  6. Generation: Ask the model to answer from the supplied evidence and to state when it is insufficient.
  7. Evaluation: Measure retrieval quality, groundedness, citation correctness, safety, latency, and cost.

Azure AI Search supports hybrid queries that run full-text and vector search together and merge results with Reciprocal Rank Fusion. Microsoft Learn: Hybrid search

Retrieval quality drives answer quality

If retrieval misses relevant passages or returns stale or unrelated material, generation starts from weak evidence. Evaluate retrieval independently from the final answer. Use representative queries and label relevant documents, then track recall, precision, ranking quality, and citation correctness. Tune chunk boundaries, metadata, query construction, and reranking against that evaluation set rather than relying on a few successful demos.

Microsoft's Azure Architecture Center recommends experimenting with index configuration and search strategies to identify a suitable retrieval setup for a workload. Microsoft Learn: Information retrieval for RAG

Enterprise trust and access control

RAG does not inherit source-system permissions automatically. Carry the required permission metadata into the index, use a trusted identity or access policy to filter retrieval results, and test both permitted and denied users. Azure AI Search provides multiple document-level access approaches; choose one for the source and identity model in use, and confirm its configuration and limitations. Microsoft Learn: Document-level access control in Azure AI Search

Retrieved content is untrusted evidence. Keep it separate from system instructions, restrict tools available to the model, and enforce authorization again at each action endpoint. Protect prompts, retrieved passages, outputs, and traces as potentially sensitive data. Provide citations that help people verify answers, and provide a human route for high-impact or uncertain cases.

Conclusion

RAG on Azure is an application architecture built from ingestion, indexing, retrieval, model calls, access control, and evaluation. Azure AI Search can provide the retrieval layer, but the application must preserve permissions and test whether the retrieved evidence actually supports the response.

References

  1. Microsoft Learn. Retrieval augmented generation and indexes in Microsoft Foundry
  2. Microsoft Learn. Hybrid search using vectors and full-text search in Azure AI Search
  3. Microsoft Learn. Develop a RAG solution on Azure: Information retrieval phase
  4. Microsoft Learn. Document-level access control in Azure AI Search