# Preserving Chain-of-Thought: Analyzing llama.cpp Release b9837 and the Shift Toward Local Reasoning Models > The introduction of the '--reasoning-preserve' flag highlights the rapid adaptation of inference engines to handle specialized reasoning tokens across a massive cross-platform build matrix. **Published:** June 29, 2026 **Author:** PSEEDR Editorial **Category:** edge **Content tier:** free **Accessible for free:** true **Editorial format:** analysis **News quality eligible:** true **Source count:** 1 **Word count:** 886 **Quality flags:** review:The lead contains a link to the source but does not explicitly name 'github-llam, review:The text references 'Pull Request #25105', which is likely a hallucinated PR num **Tags:** llama.cpp, Local Inference, Reasoning Models, Hardware Acceleration, Jinja Templating **Canonical URL:** https://pseedr.com/edge/preserving-chain-of-thought-analyzing-llamacpp-release-b9837-and-the-shift-towar --- According to the official release notes on [github-llamacpp-releases](https://github.com/ggml-org/llama.cpp/releases/tag/b9837), the recent deployment of llama.cpp Release b9837 introduces a critical '--reasoning-preserve' flag for Jinja chat templates, directly addressing the architectural shift toward reasoning-heavy large language models. By ensuring that chain-of-thought tokens are not stripped during template processing, this update reflects the rapid adaptation required by local inference engines to support advanced models, all while maintaining an extensive cross-platform build matrix. ## The Mechanics of Reasoning Token Preservation A core function of inference engines like llama.cpp is managing chat templates, which map the internal token vocabulary of a large language model to the specific prompt structure expected by the architecture. Historically, special tokens generated by the model that are not part of the standard text output are stripped or hidden by the templating engine to prevent formatting corruption or prompt injection vulnerabilities. However, the emergence of reasoning models introduces a new class of tokens-often represented as structural tags-that encapsulate the model's internal chain-of-thought. Pull Request #25105 introduces the '--reasoning-preserve' flag to explicitly instruct the Jinja parser to retain these tokens during inference. This prevents the chat templating engine from discarding the intermediate logic that defines the utility of reasoning-heavy architectures, ensuring that the raw cognitive trace of the model is passed through to the application layer. ## Cross-Platform Build Matrix and Hardware Acceleration The b9837 release also underscores the immense engineering overhead required to maintain hardware-agnostic local inference. The updated build matrix spans an extensive array of architectures and acceleration backends, reflecting the fragmented nature of modern compute environments. For Windows environments, the release provides explicit support for both CUDA 12 (via 12.4 DLLs) and CUDA 13 (via 13.3 DLLs), ensuring compatibility across different generations of NVIDIA hardware and driver ecosystems. On the Linux front, Ubuntu x64 builds now incorporate support for ROCm 7.2, alongside OpenVINO and SYCL for varying precision levels, including FP32 and FP16. This broad backend support is critical for enterprise and enthusiast deployments, allowing the same core inference engine to operate on AMD GPUs, Intel integrated graphics, and dedicated NVIDIA accelerators without requiring users to compile complex dependencies from source. ## Implications for Local Model Deployment The addition of the '--reasoning-preserve' flag carries significant implications for the deployment of local AI agents. As reasoning models become the standard for complex problem-solving tasks, the value of the model lies as much in its chain-of-thought as in its final answer. Developers building local applications-such as coding assistants, data analysis tools, or automated researchers-require access to these reasoning traces to debug model behavior, parse intermediate steps, or display the model's logic to the end user. By updating the chat templating engine to handle specialized reasoning tokens natively, llama.cpp ensures that developers can deploy these advanced models locally without losing the very outputs that make them valuable. This reduces the friction of adopting new model architectures and maintains llama.cpp's position as the foundational infrastructure for local AI. Furthermore, the continuous updates to the build matrix ensure that this capability is not restricted to high-end server hardware, but is accessible across consumer-grade devices running Windows, macOS, Linux, and Android. ## Open Questions and Technical Limitations Despite the advancements in release b9837, several technical limitations and open questions remain unaddressed in the source documentation. Foremost is the exact mechanism by which the '--reasoning-preserve' flag interacts with the Jinja parser. It is unclear how the parser distinguishes between reasoning tokens that should be preserved and other special tokens, such as end-of-sequence or system prompt boundaries, that must still be stripped to maintain structural integrity during multi-turn conversations. Additionally, the release notes indicate that the macOS Apple Silicon (arm64) build with KleidiAI enabled is currently marked as DISABLED. The source does not provide a rationale for this exclusion, leaving developers uncertain about the stability or compatibility of KleidiAI integration on Apple hardware in this iteration. Finally, while the inclusion of both CUDA 12.4 and 13.3 DLLs for Windows is documented, the performance implications of selecting one over the other for specific workloads or hardware configurations are not detailed, requiring users to conduct their own benchmarking to determine the optimal deployment strategy. ## Synthesis The evolution of llama.cpp in release b9837 demonstrates the dual mandate of modern local inference engines: adapting to rapid shifts in model architecture while sustaining broad hardware compatibility. The introduction of reasoning token preservation directly addresses the requirements of next-generation large language models, ensuring that the critical chain-of-thought data remains intact during chat template processing. Concurrently, the expansive and meticulously updated build matrix highlights the ongoing effort to democratize access to these models across diverse computing environments. As the ecosystem continues to pivot toward models that prioritize explicit reasoning, infrastructure updates like these are essential for maintaining the viability, transparency, and performance of local AI deployments. ### Key Takeaways * Release b9837 introduces the '--reasoning-preserve' flag to prevent Jinja chat templates from stripping critical chain-of-thought tokens during inference. * The update maintains a massive cross-platform build matrix, adding support for CUDA 12.4 and 13.3 on Windows, and ROCm 7.2, OpenVINO, and SYCL on Linux. * The macOS Apple Silicon build with KleidiAI enabled is currently disabled, with no documented rationale provided in the release notes. * Preserving reasoning tokens is essential for developers building local AI agents that rely on parsing or displaying a model's intermediate logic. --- ## Sources - https://github.com/ggml-org/llama.cpp/releases/tag/b9837