Retrieval-Augmented Generation (RAG) gives a language model access to selected external knowledge at request time. In an enterprise system, that makes the RAG application part of the data-access path. A design must control which sources enter the index, which passages a user can retrieve, how retrieved text is handled, and what actions the model can request.
What RAG does
A RAG application retrieves relevant documents or passages, adds them to a model request, and generates a response that can cite its sources. The original RAG research combined a pretrained generator with a dense vector index for knowledge-intensive tasks. In production, retrieval can improve access to current or private information, but it does not guarantee factual answers, correct citations, or authorized access by itself. Lewis et al. (2020)
Why enterprise RAG needs stronger security
A typical pipeline includes source connectors, extraction, chunking, embedding, indexing, retrieval, prompt construction, generation, and sometimes tool execution. Each stage has separate trust and failure conditions. A weak source-ingestion process can admit poisoned material; a retrieval filter can use stale permissions; and an unsafe tool can turn a misleading answer into an unauthorized action.
OWASP identifies risks including prompt injection and vector or embedding weaknesses. RAG does not remove prompt injection because the model still processes retrieved content that may contain attacker-controlled instructions. Treat retrieved material as untrusted input and enforce access in the retrieval path. OWASP LLM08:2025, OWASP LLM01:2025
Core security risks
Unauthorized data access
An index can contain documents from different tenants, departments, or sensitivity levels. If identity and document permissions are not represented and enforced, retrieval may return material the caller is not allowed to see. Filter results using a trusted identity or access policy as part of retrieval, verify the chosen search filter semantics, and ensure unauthorized passages never enter the model context. Azure AI Search provides several document-level access approaches, but each requires deliberate configuration and testing. Microsoft Learn: Document-level access control
Knowledge poisoning
A malicious or low-quality document can influence answers if it enters an index and ranks for relevant queries. Use approved connectors, validate source identity and file type, scan content, retain provenance, and provide a process to quarantine or remove a source. Retrieval ranking alone is not a source-trust control.
Indirect prompt injection
Retrieved text may include instructions aimed at changing the model's behavior. Keep application instructions separate from retrieved evidence, limit the tools available to the model, and test documents containing direct and hidden instructions. Prompt wording and output filtering can reduce risk, but neither should be treated as a complete authorization boundary. OWASP LLM01:2025
Embedding and privacy risks
Embeddings are derived from source content and should not be assumed to be anonymous or harmless. Apply the same data classification, access, retention, and deletion requirements to indexes and embeddings that apply to their source material. Redact or exclude sensitive fields when the use case does not require them. Risk varies by model, data, and exposure; avoid claiming that all embeddings can or cannot be reversed.
Excessive agency
If a RAG assistant can update records, send messages, or trigger transactions, retrieved instructions or incorrect answers may lead to unsafe actions. Expose narrow tools, validate every input in application code, enforce authorization in the downstream service, and require approval where the impact warrants it. OWASP tracks excessive agency as a distinct risk for LLM applications. OWASP Top 10 for LLM Applications 2025
A secure enterprise RAG architecture
The diagram shows the knowledge path: approved sources are validated and classified, content is stored in a search index, and permission-aware retrieval selects authorized passages for a grounded answer. Identity, tenant isolation, and audit controls apply across the path. Indexing occurs before a user query; authorization is evaluated during retrieval, not as a step that happens before the index exists.
| Layer | Purpose | Example controls |
|---|---|---|
| Identity | Establish who is asking | Microsoft Entra authentication, trusted claims, tenant context |
| Ingestion | Admit approved content | Source allow-list, file validation, malware scanning, provenance |
| Index | Store searchable content | Sensitivity metadata, tenant partitioning where required, retention and deletion |
| Retrieval | Select relevant authorized passages | Identity-aware filters, source constraints, denied-access tests |
| Generation | Produce a useful response | Separate instructions from evidence, citations, abstention behavior |
| Tools | Perform permitted actions | Narrow APIs, server-side authorization, approval thresholds |
| Operations | Investigate and improve | Access-controlled logs, privacy-aware retention, incident response |
Enforce permission-aware retrieval
Carry the source document's permission metadata into the indexed representation. At query time, use trusted identity or authorization data to restrict candidate documents. If the search product's filtering behavior is configurable, select and test the mode required by the security policy. Fail closed when identity or permission data is missing. Test users who should be allowed and denied, including group changes and stale index entries.
Govern ingestion and lifecycle
Record source identifiers, owners, classification, ingestion time, and deletion state. Validate connectors and extracted content, preserve links to the source, and define how changes and removals propagate to the index. Separate indexes or security boundaries when shared metadata filters do not provide an adequate isolation model.
Treat retrieved content as evidence
Pass only the required passages to the model. Clearly distinguish trusted application instructions, user input, and retrieved content. Ask for source references, but verify citations in application code against the documents actually retrieved. Test malicious documents and misleading content. The model should not decide access rights or treat instructions found in a document as policy.
Limit and authorize tools
Prefer read-only tools when they satisfy the workflow. Give each tool a narrow contract, validate its arguments, apply rate and transaction limits, and enforce access again in the target service. Require human approval for high-impact or irreversible actions. Record the decision and result without retaining more sensitive content than necessary.
Log retrieval decisions safely
Operational records can include user identifiers, queries, source IDs, permissions applied, model versions, tool calls, and outcomes. Decide which fields are necessary, protect them with role-based access and retention controls, and test that logging does not become a second route to sensitive data. NIST's Generative AI Profile recommends lifecycle risk management across design, development, use, and evaluation. NIST AI 600-1
Governance and compliance
Use an information-security management system to support ownership, risk assessment, access management, incident handling, and continual improvement. ISO/IEC 27001 is a general information-security management standard, not an AI or RAG-specific control set. Pair it with AI-specific risk assessment, data governance, and technical testing. ISO/IEC 27001
How to test the system
Test the full request path, not just answer quality.
| Test area | Example test |
|---|---|
| Access control | Can a user retrieve a document outside their authorized groups? |
| Tenant isolation | Can one tenant retrieve another tenant's content? |
| Permission updates | Does revoked access stop retrieval after source or group changes? |
| Poisoning | Can a malicious or unapproved source influence an answer? |
| Prompt injection | Can instructions in retrieved text change tool use or disclose data? |
| Privacy | Are sensitive fields exposed in embeddings, traces, or logs unnecessarily? |
| Tool safety | Can a proposed action bypass downstream authorization or approval? |
| Reliability | Do timeouts, empty retrieval, and service failures fail safely? |
Measure retrieval precision and recall, citation correctness, unauthorized retrieval, prompt-injection success, tool-policy violations, latency, and cost. Define thresholds for the workflow and review false positives as well as misses.
Implementation roadmap
- Inventory knowledge sources, owners, sensitivity, and deletion requirements.
- Define user, tenant, agent, and tool identities and their access boundaries.
- Build validated ingestion with provenance and permission metadata.
- Implement retrieval filters and verify their semantics with positive and negative access tests.
- Keep retrieved content separate from instructions and limit the model's tools.
- Evaluate routine cases, denied access, stale permissions, poisoning, and injection.
- Add privacy-aware monitoring, incident ownership, and a review process for changes.
Conclusion
Secure enterprise RAG is a data-access and action system, not just a vector database connected to a model. Treat permissions, source quality, retrieved content, tool execution, and telemetry as explicit trust boundaries. The architecture is only as secure as the controls enforced at each boundary.
References
- Lewis, P., et al. (2020). Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. NeurIPS 2020. Paper
- OWASP Gen AI Security Project. (2025). LLM08:2025 Vector and Embedding Weaknesses
- OWASP Gen AI Security Project. (2025). LLM01:2025 Prompt Injection
- Microsoft Learn. Document-level access control in Azure AI Search
- OWASP Gen AI Security Project. (2025). Top 10 Risks for LLM Applications
- National Institute of Standards and Technology. (2024). Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1
- International Organization for Standardization. ISO/IEC 27001:2022