Rethinking access control for RAG with Amazon Quick and Amazon Bedrock

來源原標題

Rethinking access control for RAG with Amazon Quick and Amazon Bedrock

作者

Amit Choudhary

原始標籤

Advanced (300) · Amazon Bedrock · Amazon Quick Suite · Announcements

原始連結

https://aws.amazon.com/blogs/machine-learning/rethinking-access-control-for-rag-with-amazon-quick-and-amazon-bedrock/

摘要原文

Enterprise organizations are adopting Retrieval Augmented Generation (RAG) to unlock insights from company knowledge sources like Microsoft SharePoint, Google Drive, and Atlassian Confluence. However, these knowledge sources contain sensitive information governed by complex permission structures. Making sure that AI-generated answers respect those permissions is one of the hardest challenges in enterprise AI. In this post, we explore how Amazon Quick and Amazon Bedrock Knowledge Bases solve this challenge through real-time access control list (ACL) enforcement, verifying permissions directly with authoritative sources at query time. Consider this scenario: A SharePoint site owner creates a knowledge base for their organization. Team members across multiple departments use an AI assistant to get answers from this knowledge base. The critical requirement is that each team member must only receive AI-generated insights from documents they’re authorized to access. This is a universal enterprise challenge. Organizations want to democratize access to AI-powered insights without compromising their existing security posture. A single unauthorized document surfaced in an AI response could expose confidential strategy documents, unreleased financial data, or sensitive HR information. A common approach to RAG access control uses a replicate-and-filter approach to enforce document-level permissions. Here’s how it typically works: While this approach seems reasonable on the surface, it has three fundamental weaknesses. In this model, the AI system takes sole responsibility for enforcement without being the authoritative source of permissions. This requires data connectors to accurately replicate complex, source-specific ACL logic across various data sources. Each data source has its own unique permission models. Mapping inheritance hierarchies, group memberships, conditional access policies, and deny rules across dozens of connectors is an error-prone undertaking. In general, data connectors support pull-based syncs that run on demand or on a customer-defined schedule. The ACLs in these AI solutions are a snapshot in time from when the last sync ran. Some solutions use event-based updates, but this doesn’t work universally. For example, a data source like Confluence doesn’t emit an event when group membership changes. Between syncs, a user who had their access revoked might still receive AI answers from documents they should no longer see. Data sources regularly change or introduce new mechanisms to control access to content. A new permission feature in SharePoint or a change to the Google Drive sharing model could create gaps in the ACL mapping logic. This can expose content until the connector is updated. To address these challenges, we implemented real-time ACL checks as an additional layer of security on top of existing pre-retrieval ACL filtering for Amazon Quick and Amazon Bedrock Knowledge Bases. This makes sure the system enforces the most current access controls by checking permissions directly with the authoritative source at query time. This avoids relying on potentially stale or incorrectly mapped ACL data. The following diagram illustrates our hybrid approach that delivers both semantic search performance and real-time security capabilities. Figure 1: Real-time ACL enforcement architecture for Amazon Quick and Amazon Bedrock Knowledge Bases, combining pre-retrieval filtering (Stage 1) with real-time verification against authoritative sources (Stage 2) When a user submits a query to an Amazon Quick agent that uses a Google Drive knowledge base, the system enforces access controls in two stages: Amazon Quick performs a semantic search against the vector index to find the most relevant document passages. The system applies access control lists that are already stored in the index. This produces a preliminary set of candidate documents. This stage is necessary because real-time API calls for every document in the index would be too costly at scale. Amazon Quick verifies the candidate documents in real time by calling the Google Drive APIs. It uses the service account credential that the administrator provided to generate user-specific access tokens through impersonation. Google Drive maintains the source of truth for access control lists associated with each document. Documents the user is not authorized to access are excluded from the retrieved result set. Only the verified and authorized document passages are passed to the large language model (LLM) as context. The model uses this knowledge to generate a response. This two-stage approach balances performance with security. It uses cached ACLs for efficiency while facilitating correctness through real-time checks. In addition to ACL enforcement, Amazon Bedrock provides responsible AI controls. These include Amazon Bedrock Guardrails for content filtering, grounding checks to reduce hallucinations, and configurable safety policies to help organizations deploy generative AI applications responsibly. This approach delivers three key benefits: “When we set out to evaluate AI solutions for our organization, our security and compliance teams were clear about their top priority: ensuring that colleagues would only ever see information they’re authorized to access. It’s a fundamental requirement, but one that many platforms struggle to address in a meaningful way. Amazon Quick’s approach to real-time access control answered that question definitively and demonstrated a level of rigor that stood out throughout our evaluation. It gave our internal review board the confidence to move forward and set a strong foundation for how we think about AI governance going forward.” — Jamahl Wiggins, Sr. Specialist – M365 Innovation, Mondelēz International Mondelēz International has deployed Amazon Quick for their over 35,000 employees across four regions. In this post, we talked about how Amazon Quick and Amazon Bedrock Knowledge Bases implement real-time ACL enforcement to solve a critical security challenge for enterprises. The dual-layer ACL architecture verifies permissions directly with authoritative sources at query time. This makes sure AI-generated answers include only content a user is authorized to access. To get started, visit Amazon Quick and Amazon Bedrock Knowledge Bases .