Skip to content
← The blog
Blog postrag12 min read

Most Enterprise RAGs Have No Access Control: How to Fix Yours

Sunder K

Sunder K

AI architect & transformation strategist · Apr 23, 2026

Cover illustration for Most Enterprise RAGs Have No Access Control: How to Fix Yours

Your RAG system is a data leak waiting to happen

According to vendor research published in March 2026, most companies that have built a Retrieval-Augmented Generation system — a setup that lets an AI chatbot search a company's own documents before answering a question, commonly called RAG — have done so without checking whether the person asking is actually allowed to see the documents it retrieves. In plain terms: the system has no role-based access control, meaning it doesn't sort users by job title, department, or clearance level before handing them information.

Here's why that matters. When you feed your company's documents into a RAG system, they typically first pass through a Large Language Model (LLM) — the AI model, like the ones behind popular chatbots, that reads text and generates answers — and get converted into a "vector database," a specialized store that lets the software find documents by meaning rather than exact keywords. That conversion step often strips away the original permissions. A file that used to be locked to the legal team can end up sitting in the same searchable pile as the employee handbook. The practical result: a junior analyst types a question and gets an answer built, in part, from confidential merger documents or private HR files they were never supposed to open.

The core issue is that the LLM itself has no idea what it's allowed to show anyone. It only knows what's placed in front of it. If the retrieval step hands it a restricted document, it will happily use it to write an answer. That turns a system meant to make employees more productive into a quiet channel for leaking sensitive data and breaking compliance rules. The fix isn't to teach the AI model some notion of secrecy — models don't work that way. The fix is to check permissions before any document reaches the model at all, at the retrieval step, so the AI never even sees information a given user isn't cleared to view.

Diagram 1 illustrating Most Enterprise RAGs Have No Access Control: How to Fix Yours
Open original ↗
Diagram 1 illustrating Most Enterprise RAGs Have No Access Control: How to Fix Yours

The Anatomy of a Governed Retrieval Pipeline

Building a secure RAG system isn't about finding one magic database — it's about designing a data pipeline where each component is chosen for a specific task. A production-grade, governed RAG application isn't a single bucket and an index; it's a coordinated set of stores that manage data through its entire lifecycle. According to a design guide from Scality, a typical deployment relies on seven distinct stores, each with its own access pattern and latency budget.

  1. Source Archive: The permanent, original copy of every ingested document. This is your ground truth, often write-heavy during initial backfills and then read-rarely.

  2. Parsed Chunk Store: The pre-processed text segments from your source documents. These are the actual units of information that will be embedded. This store is read-heavy whenever you need to re-index your corpus with a new embedding model.

  3. Embedding Store: A durable, versioned copy of every vector embedding generated from your chunks. This allows you to rebuild indexes without the high cost of re-calculating embeddings for your entire corpus.

  4. Vector Index: The hot, performance-optimized data structure that serves similarity search queries at inference time. This is what most people think of as the "vector database," but it's really just the queryable tip of the iceberg.

  5. Keyword Index: A lexical index (often using an algorithm like BM25) that handles exact-match searches for things like product SKUs, error codes, or specific legal terms that semantic search might miss.

  6. Metadata Layer: This is the heart of RAG data governance. It stores the provenance, classification, version, and—most importantly—access control information attached to every single chunk.

  7. Retention Archive: A compliance-focused copy of source content and historical embeddings, held for a specific regulatory window to satisfy legal and auditing requirements.

Treating these distinct workloads as one is a common design failure. Governance lives in the metadata layer, but it is enforced at the vector and keyword indexes. Understanding this separation is the first step toward building a system that is both powerful and secure.

Diagram 2 illustrating Most Enterprise RAGs Have No Access Control: How to Fix Yours
Open original ↗
Diagram 2 illustrating Most Enterprise RAGs Have No Access Control: How to Fix Yours

The Core Mechanism: Pre-Retrieval Filtering

The central principle of RAG security is to filter out unauthorized data before it is retrieved, not after. A system that fetches 40 documents and then discards 30 because the user lacks permissions has still leaked information; those 30 documents were processed and accessed, creating an unnecessary risk and a potential audit trail of improper access. As a report from Scadea notes, this post-retrieval filtering is itself a data leak.

True security applies the access control filter as part of the retrieval call itself.

How It Works

The mechanism connects a user's identity to the metadata of the data chunks. As described by enterprise AI architects, the process works by tagging each chunk with identifiers that map to your organization's access control policies.

  1. Ingestion: When a document is ingested and broken into chunks, each chunk is stored alongside metadata that defines who can access it. This metadata isn't free-form text; it's structured data that reflects your company's directory, such as role:underwriter, department:legal, region:EU, or sensitivity:phi.

  2. Authentication: When a user submits a query, the system first identifies them and fetches their permissions from a central identity provider like Active Directory or Okta.

  3. Query Execution: The user's query is transformed into two parts: a vector for semantic similarity search and a security filter derived from their permissions. These two components are executed simultaneously against the vector database. The database must support this kind of hybrid search, querying both the vector index for semantic relevance and the metadata index for permissions in a single operation.

The result is a list of chunks that are both semantically relevant to the user's query and compliant with their access rights. Only this pre-filtered, permissible context is passed to the LLM.

A Worked Example: ACLs in Action

Imagine a global financial institution with a single, unified RAG system for all employees. A document containing a confidential legal brief for an upcoming acquisition in the European Union is ingested.

Now, two different users submit the same query: "What is our growth strategy for next quarter?"

User 1: A Junior Sales Associate in the US

User 2: A Senior Legal Counsel in the EU

This pre-filtering ensures the principle of "Need-to-Know" is enforced automatically, preventing the kind of catastrophic data leak that happens when security is an afterthought.

Code Example: A Converged Query

This process requires a database capable of combining vector search with structured metadata filtering in one atomic operation. Drawing on the concept of a converged database architecture, which combines different data models in a single query path, the request might look something like this in a hypothetical SQL-like language:

SELECT
  chunk_text,
  source_document_id
FROM
  document_chunks
WHERE
  -- Apply the user's access control permissions
  json_value(metadata, '$.department') IN ('legal', 'executive')
  AND json_value(metadata, '$.region') = 'EU'
  AND CAST(json_value(metadata, '$.sensitivity_level') AS INT) <= 4
ORDER BY
  -- Find the most semantically relevant chunks
  VECTOR_DISTANCE(embedding, :query_vector, COSINE)
LIMIT 10;

In this example:

The database optimizer is responsible for executing this hybrid query efficiently, ensuring that permissions are checked before the final results are returned.

Diagram 3 illustrating Most Enterprise RAGs Have No Access Control: How to Fix Yours
Open original ↗
Diagram 3 illustrating Most Enterprise RAGs Have No Access Control: How to Fix Yours

Implementation Strategies and Trade-Offs

While pre-retrieval filtering is the goal, there are two primary architectural patterns for achieving it. The right choice depends on your organization's specific needs for isolation, cost, and complexity.

Strategy 1: Per-User Filtering in a Shared Index

This is the most common and flexible approach, detailed in the examples above. All data lives in a single, large vector index (or set of indexes), and permissions are managed entirely through metadata filtering at query time.

This pattern is ideal for internal-facing enterprise RAG systems where employees have multifaceted roles.

Strategy 2: Per-Tenant or Per-Role Indexes

In this model, you physically segregate data into different indexes or "namespaces" based on who can access it. This is a common pattern in multi-tenant SaaS applications where customer data must be strictly isolated.

Several database vendors offer features that facilitate this pattern. The Scadea report notes that Pinecone uses namespace-scoped API keys, Weaviate supports tenant-aware classes with dedicated shards, and Qdrant allows for tenant-level sharding.

This pattern is the default choice for B2B SaaS products where each customer is a tenant, or in high-security environments where different data classifications (e.g., "Public," "Confidential," "Secret") are stored in completely separate indexes.

Beyond Access: Lifecycle and Audit

Effective governance doesn't stop at access control. It covers the entire lifecycle of the data, from deletion to retention, and includes the ability to audit what happened.

The Right to Be Forgotten: Handling Deletions

Under regulations like GDPR, a user has the right to request the deletion of their personal data. In a RAG context, this is more complicated than just deleting a source file. A single document may have spawned dozens of chunks, each with a corresponding embedding in your vector index.

A compliant deletion process must be able to trace a source document to all its downstream artifacts and remove them completely:

  1. Delete the document from the source archive.

  2. Delete its corresponding text segments from the parsed chunk store.

  3. Delete their embeddings from the embedding store and the live vector index.

This requires robust provenance tracking in your metadata layer. Each chunk and embedding must be traceable back to its parent document. Failure to do so means you are holding onto data you are legally required to delete.

Retention Policies

Just as some data must be deleted, other data must be kept. Many regulated industries require that communications and records be held for a specific period (e.g., 7 years). The retention archive serves this purpose. It's a long-term, low-cost storage tier for source documents and their historical embeddings to meet these compliance mandates. This data is typically "write-once, read-never" unless an audit or legal discovery event occurs.

Auditing for Accountability

If your AI gives a user a wrong or inappropriate answer, you must be able to explain why. A robust audit log is not optional in a production system. According to security analysts, your RAG system's audit log should capture:

This trail allows you to reconstruct the entire event. You can see exactly what the user asked, what information the system used to build the answer, and what the final output was. This is essential for debugging, regulatory compliance, and building trust in your AI systems.

Conclusion: Governance Is a Strategic Imperative

If most enterprise RAG systems really do ship without access control, as the March 2026 research suggests, then the gap between "we built an AI assistant" and "we built an AI assistant that respects who's allowed to see what" is where most of the risk in this technology currently lives. That gap doesn't close itself, and it doesn't shrink as a company connects more documents to the system — it grows, because every new source folder is another chance for a permission to get lost in translation.

The fix described here isn't exotic: tag every chunk with the same permissions the source document already had, check those permissions against the user's identity at query time, and only then let the model see the result. Add a way to trace and delete everything spawned from a document when someone exercises a right to be forgotten, a place to hold records for as long as regulations require, and a log detailed enough to answer "why did the AI say that?" months after the fact. None of this is optional polish — skipping it doesn't just risk an over-permissive chatbot, it risks a specific, namable incident: a confidential deal memo surfacing in a sales rep's answer, a customer's data appearing in another customer's session, a regulator asking for records that were never properly retained.

Teams evaluating or rebuilding a RAG system should treat the presence of pre-retrieval filtering, provenance tracking, and audit logging as baseline requirements to check for, not advanced features to add later. Whether that means adopting per-user metadata filtering in a shared index or splitting into per-tenant indexes depends on the shape of the organization, but the decision to check permissions before retrieval rather than after is not optional in either case.

References

1 reads
Share on LinkedInShare on XCopy link

Discussion (0)

Loading discussion…