# The Geometry of Safety: Single Direction vs. Low-Rank Refusal > Coverage of lessw-blog **Published:** March 02, 2026 **Author:** PSEEDR Editorial **Category:** risk **Content tier:** free **Accessible for free:** true **Word count:** 485 **Tags:** Mechanistic Interpretability, AI Safety, LLM Refusal, Qwen, Residual Stream **Canonical URL:** https://pseedr.com/risk/the-geometry-of-safety-single-direction-vs-low-rank-refusal --- A technical exploration into how small LLMs encode refusal behaviors and the implications for bypassing safety guardrails through linear arithmetic. In a recent technical analysis, **lessw-blog** investigates the mechanistic underpinnings of how small Large Language Models (LLMs) encode refusal behaviors. The post, titled _Single Direction vs Low-Rank Refusal in Small LLMs_, questions whether the safety mechanisms that prevent models from answering harmful queries rely on simple single linear directions or more complex low-rank subspaces within the model's residual stream. **The Context** As the deployment of open-weights models accelerates, the robustness of safety fine-tuning (RLHF) has become a critical area of study. Mechanistic interpretability aims to open the "black box" of neural networks to understand how specific behaviors are represented mathematically. Previous research has suggested that high-level concepts, including the refusal to generate harmful content, might be represented linearly. If refusal is merely a specific vector direction, it implies that safety guardrails could be easily bypassed-or "ablated"-by simply subtracting that direction from the model's internal activations without retraining the model. **The Gist** The author focuses on replicating and extending these findings using **Qwen-1.8B-Chat**. The core experiment involves extracting "refusal vectors" (RVs) and testing whether subtracting them allows the model to comply with harmful requests. The findings indicate that for many instances, refusal is indeed encoded as a single linear direction. By intervening in the residual stream, the author was able to remove the refusal behavior, resulting in the model complying with restricted prompts. Crucially, the post highlights that this ablation causes surprisingly little degradation in the model's general performance. This suggests that the "refusal" mechanism is largely orthogonal to the model's core capabilities. However, the analysis introduces necessary nuance: not all refusal representations are perfectly captured by a single direction. The author discusses the hypothesis of **low-rank subspaces**\-where refusal might exist across multiple dimensions rather than a single vector-and notes that current extraction methods using semantically unrelated safe prompts may introduce noise. **Methodology and Evaluation** The experimental pipeline is rigorous, utilizing layer-wise similarity checks and interventions. To validate the results, the author employed the **DeepSeek API** to evaluate the coherence of the generated responses and the **lm-eval-harness** to measure overall model performance post-intervention. This dual approach ensures that the removal of safety guards does not simply result in gibberish. This research is significant for developers working on AI alignment. It suggests that current safety fine-tuning techniques may be structurally brittle, relying on easily identifying linear features that can be mathematically removed. For a detailed look at the vector extraction process and the comparative analysis of single-direction versus low-rank representations, we recommend reading the full report. [Read the full post on LessWrong](https://www.lesswrong.com/posts/LMkvjDTLKFrgdzJdG/single-direction-vs-low-rank-refusal-in-small-llms-1) ### Key Takeaways * Refusal behaviors in Qwen-1.8B-Chat can often be removed by subtracting a single linear direction from the residual stream. * The intervention effectively bypasses safety guardrails while causing minimal degradation to general model performance. * The author contrasts the 'single direction' hypothesis with the possibility of refusal residing in a low-rank subspace. * Methodological challenges persist, specifically regarding noise introduced when extracting vectors using unrelated safe prompts. * Evaluations using DeepSeek API and lm-eval-harness confirm that the modified models remain coherent and capable. [Read the original post at lessw-blog](https://www.lesswrong.com/posts/LMkvjDTLKFrgdzJdG/single-direction-vs-low-rank-refusal-in-small-llms-1) --- ## Sources - https://www.lesswrong.com/posts/LMkvjDTLKFrgdzJdG/single-direction-vs-low-rank-refusal-in-small-llms-1