Beyond the 'Correct Answer Feature': The Complex Mechanics of LLM Multiple Choice Evaluation

Recent analysis published on lessw-blog challenges a prevailing hypothesis in mechanistic interpretability: that large language models (LLMs) evaluate multiple-choice questions using a singular "correct answer feature." For AI safety and alignment researchers, this finding signals a critical warning against oversimplifying how models internally represent truth, suggesting that current methods for model steering and detecting deceptive alignment may rely on flawed assumptions about linear representations.

The Allure and Failure of the "Correct Answer Feature"

Multiple-choice question answering (MCQA) serves as the bedrock for evaluating large language models. Benchmarks like MMLU, ARC, and TruthfulQA rely entirely on a model's ability to assign higher probabilities to correct options. Consequently, understanding the exact mechanical circuits models use to perform MCQA is a primary objective in mechanistic interpretability. A highly appealing and clean hypothesis that has gained traction is the existence of a "correct answer feature"-a specific, identifiable direction in the model's activation space that acts as a correctness score. Under this theory, this feature activates on the final token of an option, effectively flagging it as the right choice.

However, the source analysis demonstrates that this linear, localized representation cannot fully explain MCQA capabilities. The theoretical limitation becomes obvious when examining intransitive options. If a model is asked, "Which of these hands would win in a game of rock-paper-scissors?", it cannot evaluate the correctness of the first option ("Rock") in isolation. The "correct answer feature" on the final token of "Rock" should theoretically remain neutral or undefined because its correctness is entirely dependent on the subsequent option ("Scissors" or "Paper"). Because the model processes tokens sequentially, a localized feature on the first option's final token lacks the necessary context to compute a truth value. This proves that LLMs must possess alternative, more complex mechanisms for evaluating correctness that span across multiple options and contextual boundaries.

Empirical Evidence from Direct-Effect Head Attribution

Beyond theoretical counterexamples, the source provides empirical backing through direct-effect head attribution. In transformer architectures, direct-effect heads are attention heads that write directly to the residual stream in a way that significantly and immediately impacts the final logit outputs. By analyzing these specific components, researchers can trace exactly which parts of the network are responsible for boosting the probability of the correct answer token.

The analysis indicates that the internal mechanisms models use remain remarkably consistent across both transitive scenarios (where a correct-answer feature could theoretically work) and intransitive scenarios (where it mathematically cannot). If models relied primarily on a simple correct-answer feature, we would expect to see a distinct structural shift or a fallback circuit activating when confronted with intransitive questions. Instead, the continuity of the mechanism suggests that the primary method for MCQA is inherently relational and context-dependent, rather than a simple linear lookup of a pre-computed truth value. The model is likely comparing representations across the entire context window rather than simply reading a localized score.

Implications for AI Safety and Model Steering

For the broader AI ecosystem, particularly in safety and alignment, the dismantling of the "correct answer feature" hypothesis carries significant weight. Recent advancements in representation engineering have heavily relied on the assumption that complex concepts like "truth," "deception," or "sycophancy" can be isolated as linear directions in activation space. If truth evaluation in a standard MCQA format is not a simple linear feature, attempts to steer models by clamping or modifying a singular "truth direction" may prove brittle or entirely ineffective in complex reasoning tasks.

Furthermore, this complicates the detection of deceptive alignment. A core strategy for auditing potentially deceptive models involves probing their internal states to see if they "know" the correct answer while outputting a false one. If the model's internal representation of correctness is distributed, relational, and highly dependent on the specific framing of the options, auditors cannot simply monitor a single feature to detect a mismatch between internal belief and external output. Deceptive models could theoretically hide their true evaluations within these complex, multi-token circuits, evading detection methods that rely on overly simplified mechanistic assumptions.

Methodological Limitations and Open Questions

While the theoretical argument against a universal correct-answer feature is robust, the empirical claims presented in the source require further contextualization. The technical brief omits the specific mathematical definition and methodology used for the direct-effect head attribution. Without access to the exact attribution techniques-such as whether they utilized activation patching, path patching, or linear approximations-it is difficult to assess the precision of the findings.

Additionally, the source lacks clarity on the specific LLM architectures and scales tested. Mechanistic interpretability findings often vary wildly between small, specialized models (like GPT-2 Small or TinyStories models) and frontier-class models (like Llama-3 or GPT-4). It remains an open question whether the absence of a singular correct-answer feature is a universal property of all transformer-based architectures or if larger models, which exhibit grokking and phase changes during training, eventually converge on a more localized representation of truth as a heuristic for efficiency. The identity of the researchers who originally claimed to have found the correct-answer feature is also missing, making it challenging to compare the conflicting methodologies directly.

The revelation that large language models do not rely on a simple, localized "correct answer feature" for multiple-choice reasoning forces a necessary maturation in the field of mechanistic interpretability. Moving away from the search for easily interpretable, linear features toward understanding distributed, relational circuits is mathematically demanding but essential for accurate model auditing. As the industry relies increasingly on automated benchmarks to gauge model safety and capability, recognizing the mechanical complexity behind a simple "A, B, C, or D" output ensures that alignment strategies are built on structural realities rather than convenient theoretical simplifications.

Key Takeaways

• LLMs do not rely exclusively on a localized 'correct answer feature' to solve multiple-choice questions, challenging linear assumptions in mechanistic interpretability.
• Intransitive questions (e.g., rock-paper-scissors) prove theoretically that correctness cannot always be computed on the final token of an isolated option.
• Direct-effect head attribution shows models use consistent, complex relational mechanisms across both transitive and intransitive evaluation scenarios.
• The absence of a simple truth feature complicates representation engineering and makes detecting deceptive alignment significantly harder.