# Architecting Multi-Tenant AI Agents: Evaluating the Pool Model with Amazon Bedrock AgentCore > AWS details a shared-infrastructure approach to tenant isolation, but questions remain around granular cost attribution and service-level enforcement. **Published:** June 23, 2026 **Author:** PSEEDR Editorial **Category:** enterprise **Content tier:** free **Accessible for free:** true **Editorial format:** analysis **News quality eligible:** true **Source count:** 1 **Word count:** 1048 **Tags:** Multi-Tenancy, Amazon Bedrock, AI Agents, SaaS Architecture, Cloud Security **Canonical URL:** https://pseedr.com/enterprise/architecting-multi-tenant-ai-agents-evaluating-the-pool-model-with-amazon-bedroc --- As enterprises transition from single-user AI pilots to multi-tenant SaaS platforms, establishing robust isolation frameworks at the agent layer is critical to preventing data leaks and managing unpredictable API costs. As enterprises transition from single-user AI pilots to multi-tenant SaaS platforms, establishing robust isolation frameworks at the agent layer is critical to preventing data leaks and managing unpredictable API costs. A recent [technical breakdown from the AWS Machine Learning Blog](https://aws.amazon.com/blogs/machine-learning/shared-infrastructure-isolated-tenants-pool-model-multi-tenancy-with-amazon-bedrock-agentcore) details how Amazon Bedrock AgentCore can enforce a three-level hierarchy-Tier, Tenant, and User-to isolate tenants on shared infrastructure. By examining this architecture, engineering teams can better evaluate the trade-offs of the pool model versus the silo model for enterprise LLM applications. ## The Multi-Tenant Imperative in Generative AI Generative AI applications are notoriously difficult to scale in a multi-tenant environment. Unlike traditional web applications where tenant isolation can be achieved via row-level security in a relational database, AI agents interact with unstructured data, vector stores, and foundation models that do not inherently understand tenant boundaries. The AWS Machine Learning Blog outlines a pool model approach using Amazon Bedrock AgentCore, which allows multiple tenants to share underlying compute and model infrastructure while maintaining logical isolation. This contrasts with the silo model, where each tenant receives dedicated infrastructure. While the silo model offers maximum security and eliminates the noisy neighbor problem, it is often cost-prohibitive and operationally complex for SaaS providers to maintain at scale. When multiple tenants share a vector database in a pool model, row-level security must be translated into metadata filtering during the similarity search phase. If the metadata filter fails or is bypassed via a malicious prompt, the LLM might synthesize an answer using proprietary data from a competitor. This makes the orchestration layer-where Bedrock AgentCore sits-the most critical security boundary in the application, representing the necessary evolution for B2B AI platforms provided that logical isolation can be programmatically guaranteed. ## Enforcing the Tier-Tenant-User Hierarchy The core of the AWS proposed architecture relies on a strict three-level hierarchy: Tier, Tenant, and User. By implementing this structure, SaaS providers can differentiate service levels while ensuring that data remains partitioned between distinct organizations and individual operators. The AWS demonstration utilizes a healthcare AI agent serving multiple clinics and hospitals to illustrate this concept. In this scenario, a premium hospital network might have access to more advanced foundation models or larger context windows compared to a smaller, standard-tier clinic. A healthcare SaaS platform faces strict compliance requirements, meaning that a user operating under a specific tenant must have their requests routed with absolute precision. Bedrock AgentCore is positioned as the orchestration layer that natively enforces these boundaries. In a multi-tenant agent, conversation history cannot bleed across sessions. The orchestration layer must inject tenant context into every prompt dynamically, ensuring the LLM's stateless nature is properly managed with isolated state stores. This implementation requires rigorous identity and access management integration, ensuring that every API call to the foundation model or the retrieval-augmented generation (RAG) backend is tagged and validated against the tenant's specific context. ## Implications for SaaS AI Economics and Security The transition to a pool model carries significant implications for the unit economics of AI applications. In a shared infrastructure setup, maximizing resource utilization is the primary driver of profitability. By pooling tenants, SaaS providers can smooth out the bursty nature of LLM inference requests, reducing idle compute time. Furthermore, the ability to enforce service tiers allows for more sophisticated pricing models. Providers can throttle or prioritize requests based on the tenant's tier, ensuring that premium customers receive lower latency during peak loads. However, the pool model introduces the noisy neighbor problem in LLM inference. If one tenant runs a massive batch of complex agentic tasks, it could consume the shared rate limits or compute capacity, degrading performance for others. Implementing the pool model requires sophisticated rate limiting and quota management at the tenant level. From a security standpoint, the pool model introduces the risk of cross-tenant data leakage, particularly through prompt injection attacks. If an attacker successfully manipulates an agent into retrieving context from another tenant's partition, the resulting data breach could be catastrophic. The Bedrock AgentCore architecture attempts to mitigate this by enforcing isolation at the agentic orchestration layer, theoretically preventing the LLM from ever accessing out-of-scope data. ## Architectural Limitations and Open Questions Despite the structural advantages of the pool model, the AWS technical brief leaves several critical architectural questions unanswered. First, the exact mechanism distinguishing AgentCore from standard Amazon Bedrock Agents remains ambiguous. It is unclear whether AgentCore introduces novel isolation primitives or simply provides a reference architecture for chaining existing AWS services. Furthermore, the documentation lacks specifics on which AWS-managed services are strictly required to enforce the hierarchy, leaving engineers to guess the optimal combination of Amazon Cognito, AWS IAM, and custom middleware. Second, while the post emphasizes granular cost tracking, the specific mechanics of token attribution at the tenant level are not fully detailed. LLM costs involve embedding generation, vector search compute, and intermediate reasoning tokens generated during asynchronous agent steps. If an agent fails a tool call and retries multiple times, tracking this compute usage per tenant ID is notoriously complex. Without a native, low-latency mechanism to aggregate these costs, SaaS providers may be forced to build custom telemetry pipelines, negating some of the managed service benefits. ## Synthesis The evolution of AI agents from isolated, single-user tools to enterprise-grade, multi-tenant platforms requires a fundamental shift in architectural thinking. The pool model offers a viable path forward for SaaS providers looking to balance cost efficiency with logical isolation. However, the theoretical elegance of a three-level hierarchy must be matched by rigorous engineering practices, particularly concerning token-level cost attribution and RAG data partitioning. As the generative AI ecosystem matures, the burden of proof will remain on cloud providers to deliver native, transparent mechanisms that guarantee tenant boundaries without requiring extensive custom middleware. ### Key Takeaways * The pool model allows multiple tenants to share AI infrastructure, significantly reducing operational costs compared to siloed, dedicated environments. * Amazon Bedrock AgentCore enforces a three-level hierarchy (Tier, Tenant, User) to maintain logical isolation and differentiate service levels. * Implementing shared infrastructure requires strict metadata filtering in RAG pipelines to prevent cross-tenant data leakage via prompt injection. * Granular cost attribution remains a complex challenge, as tracking token usage across asynchronous agent steps requires sophisticated telemetry. --- ## Sources - https://aws.amazon.com/blogs/machine-learning/shared-infrastructure-isolated-tenants-pool-model-multi-tenancy-with-amazon-bedrock-agentcore