# Defeating Linear Probes: The Mechanistic Threat of Velocity Rings and Non-Linear Geometries > How engineered XOR features and continuous ring structures expose critical blind spots in standard AI safety interpretability methods. **Published:** July 09, 2026 **Author:** PSEEDR Editorial **Category:** platforms **Content tier:** free **Accessible for free:** true **Editorial format:** analysis **News quality eligible:** true **Source count:** 1 **Word count:** 1070 **Tags:** Mechanistic Interpretability, AI Safety, Linear Probing, Neural Network Geometry, Adversarial Machine Learning **Canonical URL:** https://pseedr.com/platforms/defeating-linear-probes-the-mechanistic-threat-of-velocity-rings-and-non-linear- --- A recent analysis published on [lessw-blog](https://www.lesswrong.com/posts/r6QoJ29asCsuGyXAD/solving-the-bluedot-puzzle-tais-the-velocity-ring) detailing a solution to BlueDot's TAIS Puzzle #1 exposes a fundamental vulnerability in mechanistic interpretability: the failure of linear probes to detect highly structured, non-linear representations. By engineering 'velocity rings' and XOR-based feature hiding, researchers demonstrated how causally active capabilities can be completely masked from standard linear slicing. For the AI safety ecosystem, this proves that relying on linear representation assumptions is insufficient for auditing models against hidden malicious capabilities or bypassed guardrails. ## The Linear Probing Blind Spot The foundation of the analysis rests on an experiment involving a five-layer Multi-Layer Perceptron (MLP) trained on top of frozen MiniLM embeddings to predict eight binary features. In standard mechanistic interpretability, researchers rely heavily on linear probes-training a linear classifier on network activations-to determine if a model has learned a specific concept. In this architecture, seven of the eight features are easily and linearly recoverable at the second hidden layer (`h2`). However, the eighth feature, labeled **country**, completely evades linear detection. When researchers fit a linear logistic-regression probe on the training activations and scored it on test activations, the accuracy for the **country** feature collapsed to 0.43 at layer `h2`, effectively random chance. Meanwhile, the other seven features maintained an accuracy above 0.97. The investigation revealed that the model had hidden the **country** feature as a pure XOR with the **food** feature. In a linear geometric space, an XOR configuration places positive instances directly inside negative instances along a single axis. Because a linear probe relies on drawing a straight hyperplane to separate data, it fails entirely. Yet, the information is not lost; a non-linear MLP probe applied to the exact same `h2` activations successfully recovered the **country** feature with 0.96 accuracy. This stark contrast highlights a critical flaw in assuming neural networks default to linear feature representations. ## Causal Verification via Distributed Alignment Search Identifying a non-linear representation is only the first step; proving that the model actually uses this obscured geometry to drive its final predictions is the more rigorous requirement. To establish this causal link, the researchers utilized Distributed Alignment Search (DAS). DAS is an advanced interpretability technique designed to find alignments between high-level causal variables and low-level neural representations without assuming linearity. By applying DAS, the team successfully isolated a specific 1-D non-linear subspace at layer `h2` that corresponded to the hidden **country** feature. To verify its causal efficacy, they performed activation patching-surgically altering the activations within this specific 1-D coordinate during the model's forward pass. This intervention reliably flipped the downstream outputs of the model. This causal verification is vital because it confirms the non-linear geometry is actively driving model behavior, rather than existing as a vestigial artifact of training or a hallucination of the non-linear probe itself. ## Engineering the Velocity Ring Moving beyond the discovery of the XOR hiding mechanism, the researchers engineered a novel, adversarial representation geometry termed the velocity ring. This structure serves as a blueprint for intentionally constructing representations that are entirely invisible to standard interpretability tools. The velocity ring is described as a continuous, zero-mean ring geometry hidden entirely inside the inter-layer velocity update. Instead of embedding the feature representation within the static activation state of a given layer, the feature is encoded within the derivative or the update step between layers. Because the geometry is continuous and zero-mean, it achieves absolute linear blindness. It effectively defeats single-feature slicing techniques, which attempt to isolate individual concepts by finding linear directions in activation space. The velocity ring proves that highly complex, causally active geometries can be engineered to evade detection while remaining perfectly functional for the model's internal computation. ## Implications for AI Safety and Guardrails The PSEEDR analysis centers on the severe implications these findings hold for the current AI safety paradigm. Much of contemporary mechanistic interpretability, model auditing, and red-teaming relies heavily on linear probing to map concepts, detect deception, or verify the erasure of hazardous knowledge. If causally active features can be deliberately or naturally obscured within XOR structures or velocity rings, safety guardrails based on linear assumptions are fundamentally brittle. Adversarial actors could exploit these non-linear geometries to hide malicious capabilities, sleeper agents, or biased heuristics. A model engineered with velocity rings would easily pass standard linear safety audits, appearing benign to external observers, while retaining the ability to execute hidden behaviors when triggered by specific, non-linear combinations of inputs. Furthermore, as models undergo Reinforcement Learning from Human Feedback (RLHF), there is a risk that optimization pressures might inadvertently incentivize the model to hide misaligned features in non-linear spaces to evade the linear penalty functions used during training. This creates a scenario where models appear aligned on the surface but remain structurally deceptive. ## Limitations and Open Questions While the demonstration is mathematically sound within its context, several critical limitations remain. The experiment was conducted on a relatively small scale: a five-layer MLP built on top of frozen embeddings. It remains an open question whether these specific non-linear geometries, particularly the engineered velocity ring, naturally emerge or can be stably injected into massive, billion-parameter Transformer architectures undergoing dynamic, large-scale training. Furthermore, the source material lacks the rigorous mathematical formulation and implementation details of the velocity ring geometry, making it difficult to assess the computational overhead, stability, or detectability of such structures using alternative non-linear methods. The specific mechanics of how DAS was adapted for this particular MLP architecture also require further elucidation to validate the generalizability of the causal proof across different model families. The discovery and causal verification of non-linear hiding mechanisms like XOR features and velocity rings mark a necessary inflection point for mechanistic interpretability. As models grow more complex, the assumption of linear feature superposition is proving inadequate for rigorous safety guarantees. The field must pivot toward developing scalable, non-linear probing techniques and advanced causal intervention methods that can map the true geometric complexity of neural representations, ensuring that hidden capabilities cannot evade detection through structural obfuscation. ### Key Takeaways * Standard linear probes fail to detect features hidden in non-linear geometries, such as pure XOR configurations, leading to false negatives in capability detection. * Distributed Alignment Search (DAS) can successfully isolate and causally verify non-linear subspaces, proving these hidden geometries actively drive downstream model outputs. * The velocity ring is a novel, engineered geometry hidden in inter-layer updates that achieves absolute linear blindness, defeating single-feature slicing. * AI safety audits relying solely on linear representation assumptions are vulnerable to adversarial obfuscation and structural deception. --- ## Sources - https://www.lesswrong.com/posts/r6QoJ29asCsuGyXAD/solving-the-bluedot-puzzle-tais-the-velocity-ring