Tracking the Velocity of Opaque AI Reasoning
Coverage of lessw-blog
In a recent post, lessw-blog discusses the evolving landscape of AI mathematical capabilities, specifically focusing on the measurement of "no-Chain-of-Thought" (no-CoT) time horizons.
In a recent post, lessw-blog discusses the evolving landscape of AI mathematical capabilities, specifically focusing on the measurement of "no-Chain-of-Thought" (no-CoT) time horizons. The analysis uses the Opus 4.5 model to estimate how much "thinking" an AI can perform in a single forward pass-a metric that serves as a potential proxy for opaque reasoning capabilities.
The Context: Why Opaque Reasoning Matters
For years, the safety and interpretability community has relied on "Chain-of-Thought" (CoT) prompting to understand model behavior. When a model outputs its step-by-step reasoning, human overseers can verify the logic and check for alignment. However, as models scale, they become increasingly capable of solving complex problems instantly, without generating intermediate reasoning steps. This phenomenon, often termed "opaque reasoning," presents a significant risk: if a model can perform complex planning or scheming within its internal weights and activations (a single forward pass) rather than in visible text, detecting deceptive behavior becomes exponentially more difficult.
The Gist: Measuring the Time Horizon
The post attempts to quantify this capability by establishing a "time horizon." This metric compares the AI's instantaneous performance against human effort. Specifically, it asks: How long does it take a competent human (an AIME participant) to solve a math problem that the AI can solve with 50% reliability in a single forward pass?
Using Opus 4.5, the analysis suggests a current no-CoT time horizon of approximately 3.5 minutes. While this absolute number might seem modest, the derivative is where the signal lies. The data indicates that this time horizon is doubling roughly every nine months. This exponential growth suggests that future iterations of foundation models may soon possess the ability to perform reasoning tasks in a split second that would take a skilled human hours to compute, all without leaving a paper trail of their thought process.
Additionally, the post highlights a notable prompt engineering nuance: simply repeating the problem statement multiple times in the prompt significantly boosts the model's no-CoT performance. This suggests that "computation" in LLMs can be somewhat forced by extending the context window, allowing the model more layers of processing to attend to the problem before generating the first token of the answer.
Conclusion
This analysis offers a concrete method for tracking a nebulous risk factor. As the gap between human effort and AI instant-inference widens, the window for effective oversight may narrow. We recommend reading the full post to understand the methodology and the implications for future model evaluations.
Key Takeaways
- Opus 4.5 demonstrates a no-CoT time horizon of 3.5 minutes, meaning it can instantly solve problems that take skilled humans that long to compute.
- The time horizon for AI opaque reasoning appears to be doubling every 9 months, indicating a rapid acceleration in internal processing capabilities.
- Opaque reasoning (solving without showing work) is a key risk factor for AI safety, as it may allow for undetectable scheming or misalignment.
- Repeating a problem statement multiple times in a prompt was found to substantially improve the model's ability to solve math problems without Chain-of-Thought.