# Curated Digest: Latent Reasoning Sprint #4 - PCA Analysis on CoDI > Coverage of lessw-blog **Published:** April 18, 2026 **Author:** PSEEDR Editorial **Category:** platforms **Tags:** Mechanistic Interpretability, Large Language Models, PCA, Activation Steering, Chain of Thought **Canonical URL:** https://pseedr.com/platforms/curated-digest-latent-reasoning-sprint-4-pca-analysis-on-codi --- lessw-blog shares a technical deep dive into mechanistic interpretability, applying PCA and activation steering to the CoDI Llama 3.2 1B model to understand how it processes chain-of-thought reasoning. In a recent post, lessw-blog discusses the technical findings from their fourth Latent Reasoning Sprint, presenting a detailed Principal Component Analysis (PCA) on the CoDI Llama 3.2 1B model. This publication serves as a practical exploration into the mechanics of how language models manage internal reasoning steps before producing a final output. The broader landscape of artificial intelligence is currently heavily invested in making models reason more effectively. Techniques like chain of thought prompting have proven highly successful in improving performance on complex logic and math tasks. However, the internal mechanisms that govern these reasoning chains remain largely opaque. Mechanistic interpretability is the subfield dedicated to opening this black box, seeking to understand the exact neural pathways and activation patterns that correspond to specific concepts or behaviors. Tools such as the Logit Lens, which projects intermediate network layers into the vocabulary space to see what the model is computing at a given layer, and Activation Steering, which actively perturbs these hidden states to alter model behavior, are essential instruments in this research. Understanding these dynamics is critical because it establishes a foundation for more reliable, controllable, and transparent AI systems, moving beyond mere empirical observation into structural comprehension. lessw-blog explores these dynamics by applying advanced interpretability techniques to the CoDI Llama 3.2 1B checkpoint. The core of the investigation centers on identifying how the model internally represents the conclusion of its reasoning process, specifically marked by the <|eocot|> (end of chain of thought) token. By performing PCA on the hidden state activations, the author discovered a distinct pattern: the first principal component (PCA 1) strongly correlates with the emergence of the <|eocot|> token. This suggests that the model dedicates a primary dimension of its internal representation space to tracking whether its reasoning process is complete. Additionally, the post details rigorous experiments with activation steering. The author notes a significant divergence in the efficacy of different steering methods; while manipulating the Key-Value (KV) cache successfully steered the model's behavior, attempting to steer via hidden states did not produce the desired effects. This nuance highlights the complexity of intervening in model cognition and suggests that reasoning states might be more deeply embedded in the attention mechanism's memory rather than transient hidden states. The author also includes constructive critiques of the CoDI model itself toward the end of the analysis, providing a well-rounded perspective on the architecture's limitations. This sprint report is a detailed, technical contribution to the ongoing effort to decode large language models. By isolating the specific activation patterns associated with the termination of a thought process, the author provides a crucial puzzle piece for researchers building more interpretable AI. For engineers and researchers focused on mechanistic interpretability, activation steering, or the internal mechanics of reasoning models, this analysis offers both methodological inspiration and concrete experimental results. **[Read the full post](https://www.lesswrong.com/posts/zcdpKZyMj2jENtRuL/latent-reasoning-sprint-4-pca-analysis-on-codi-1)** ### Key Takeaways * PCA 1 from the model's hidden state activations shows a strong correlation with the <|eocot|> (end of chain of thought) token. * Activation steering experiments revealed that KV cache steering was effective, whereas hidden state steering was not. * The experimental setup leveraged the CoDI Llama 3.2 1B checkpoint alongside a tuned logit lens implementation. * The analysis contributes to the broader goal of mechanistic interpretability, aiming to make AI reasoning more transparent and controllable. [Read the original post at lessw-blog](https://www.lesswrong.com/posts/zcdpKZyMj2jENtRuL/latent-reasoning-sprint-4-pca-analysis-on-codi-1) --- ## Sources - https://www.lesswrong.com/posts/zcdpKZyMj2jENtRuL/latent-reasoning-sprint-4-pca-analysis-on-codi-1