# Architectural Access Control: How Modular Pretraining Shifts AI Safety Beyond Behavioral Guardrails > Gradient Routed Auxiliary Modules (GRAM) offer a mechanism to isolate sensitive knowledge within a single model weights-file, moving safety from post-training refusals to structural toggles. **Published:** July 09, 2026 **Author:** PSEEDR Editorial **Category:** risk **Content tier:** free **Accessible for free:** true **Editorial format:** analysis **News quality eligible:** true **Source count:** 1 **Word count:** 1090 **Tags:** AI Safety, Modular Pretraining, Access Control, Model Architecture, Enterprise AI **Canonical URL:** https://pseedr.com/risk/architectural-access-control-how-modular-pretraining-shifts-ai-safety-beyond-beh --- Recent collaborative research from AE Studio and the Anthropic Alignment science team introduces Gradient Routed Auxiliary Modules (GRAM), a method for isolating sensitive knowledge into switchable components during pretraining. As detailed on [lessw-blog](https://www.lesswrong.com/posts/43vKjWuH4goLwrFHA/modular-pretraining-enables-access-control), this approach signals a critical shift in AI safety-moving away from fragile, jailbreak-prone behavioral post-processing toward architectural-level access control that could streamline enterprise deployment of frontier models. ## The Fragility of Post-Training Behavioral Safety Current frontier AI models possess vast amounts of knowledge, some of which-such as instructions for synthesizing biological agents, executing advanced persistent threats, or exploiting critical software vulnerabilities-poses severe security risks. Historically, AI developers have managed these risks through behavioral interventions applied after the initial pretraining phase. These interventions typically include Reinforcement Learning from Human Feedback (RLHF) to induce refusal behaviors, alongside external classifier models designed to intercept and block malicious prompts before they reach the model. However, as noted in the original research published on [lessw-blog](https://www.lesswrong.com/posts/43vKjWuH4goLwrFHA/modular-pretraining-enables-access-control), these behavioral layers are fundamentally superficial. They do not erase the dangerous knowledge from the model's weights; they merely train the model to hide it or refuse to access it. Consequently, these defenses are highly susceptible to jailbreaks-adversarial prompting techniques that bypass the refusal training and extract the underlying hazardous information by navigating the model's latent space from an unexpected angle. Furthermore, the alternative of tiered deployment-where different users get access to entirely different models based on their trust level-forces a coarse, inefficient trade-off. Users either receive full access to a model's capabilities, or they are relegated to a universally weaker model, requiring providers to maintain, fine-tune, and serve multiple distinct artifacts. ## Architectural Isolation via Gradient Routed Auxiliary Modules To address the structural flaws of behavioral safety, researchers have developed Gradient Routed Auxiliary Modules (GRAM). Rather than attempting to suppress knowledge after the fact, GRAM intervenes during the foundational pretraining phase. The architecture isolates specific categories of sensitive or dangerous data, routing the learning of that data into distinct, switchable auxiliary modules within the broader language model. By compartmentalizing knowledge at the architectural level, GRAM allows operators to toggle specific capabilities on or off. In experimental validations spanning models from 50 million to 5 billion parameters, the researchers demonstrated that a single GRAM-equipped model can effectively approximate the behavior of multiple distinct models, each trained with different subsets of dangerous data filtered out. When the auxiliary modules are deactivated, the model behaves as if it never encountered the restricted data during pretraining. This neutralizes the threat of jailbreaks because the knowledge is structurally inaccessible-the weights required to generate the dangerous output are simply not active in the computational graph. When activated for vetted users, the model regains its full utility and domain-specific expertise. ## Implications for Enterprise and Multi-Tenant Deployment For enterprise AI deployment, the shift from behavioral post-processing to modular pretraining represents a significant operational advantage. Currently, serving different trust tiers-such as public users, enterprise clients, and government or defense contractors-often requires training, fine-tuning, and hosting separate model weights. This fragmentation multiplies compute costs, complicates version control, and increases the latency of deploying updates across the ecosystem. GRAM introduces the possibility of granular, policy-secured access control within a single weights-file. An AI provider could deploy a single foundational model infrastructure. Through cryptographic or policy-based access keys, the system could dynamically activate or deactivate specific knowledge modules on a per-session or per-user basis. A cybersecurity researcher might authenticate to activate the "software vulnerability" module for defensive analysis, while a standard enterprise user interacting with the exact same base model would have that module structurally disabled. From an infrastructure perspective, this consolidation is highly efficient. Instead of managing separate KV caches and memory allocations for entirely different models, providers can serve a unified base model while only swapping the relatively small auxiliary modules into memory when authorized. This reduces the infrastructure burden while providing a mathematically stronger guarantee of safety than RLHF-based refusals, as the restricted knowledge cannot be extracted through adversarial prompting if the module containing it is offline. ## Current Limitations and Unresolved Frictions Despite the promising experimental results, several critical technical and operational questions remain unanswered. The research is preliminary and, as explicitly stated by the authors, has not yet been integrated into Anthropic's production models. First, the exact mechanics of the routing system-specifically how gradients are efficiently routed to the correct auxiliary modules during the massive scale of pretraining-require further elucidation. Pretraining is highly sensitive to computational overhead, and the research currently lacks detailed benchmarks on the performance cost or training latency introduced by GRAM compared to standard dense or Mixture-of-Experts (MoE) pretraining architectures. If the routing mechanism significantly slows down the training loop, it may face adoption friction among frontier labs optimizing for compute efficiency. Second, the definition and categorization of "dangerous data" remain a complex data-engineering challenge. The experiments validate the isolation mechanism, but in practice, identifying and cleanly separating hazardous training data from benign, highly correlated data is notoriously difficult. For example, separating malicious hacking techniques from standard system administration logs requires highly sophisticated data pipelines. If the routing mechanism misclassifies data during pretraining, benign capabilities could be inadvertently locked behind restricted modules, degrading the base model's utility for standard users. The specific types of dangerous data categories used in the 50M to 5B parameter experiments were not fully detailed, leaving the practical boundaries of this isolation untested at the frontier scale. ## The Trajectory of Structural AI Safety The development of Gradient Routed Auxiliary Modules highlights a necessary maturation in AI safety research. As models scale in capability, relying on behavioral guardrails to mask embedded knowledge is proving increasingly untenable against sophisticated adversaries. By pushing access control down to the architectural level during pretraining, developers can create models that are inherently modular in their capabilities. While the computational overhead and data-sorting challenges of GRAM must be resolved before production deployment, this approach provides a viable blueprint for the future of frontier AI-where safety is not a fragile behavioral wrapper, but a fundamental, configurable structural property of the model itself. ### Key Takeaways * GRAM isolates dangerous or sensitive knowledge into specific, switchable modules during the pretraining phase. * The architecture allows a single model to approximate multiple distinct models, scaling successfully in experiments from 50M to 5B parameters. * Modular pretraining reduces reliance on fragile behavioral safety layers like RLHF and classifiers, which are vulnerable to jailbreaks. * Enterprise deployment could be streamlined by hosting a single base model and toggling access to sensitive capabilities via policy or cryptographic keys. * Unresolved challenges include the computational overhead of gradient routing during pretraining and the difficulty of cleanly categorizing dangerous data. --- ## Sources - https://www.lesswrong.com/posts/43vKjWuH4goLwrFHA/modular-pretraining-enables-access-control