# Llama.cpp Release b9982 Restores Per-Request Reasoning Budget Controls for Thinking Models > Fixing parameter parsing in the OpenAI-compatible API enables fine-grained latency and compute optimization for local agentic workflows. **Published:** July 13, 2026 **Author:** PSEEDR Editorial **Category:** stack **Content tier:** free **Accessible for free:** true **Editorial format:** analysis **News quality eligible:** true **Source count:** 1 **Word count:** 896 **Tags:** llama.cpp, Local Inference, Reasoning Models, API Compatibility, Compute Optimization **Canonical URL:** https://pseedr.com/stack/llamacpp-release-b9982-restores-per-request-reasoning-budget-controls-for-thinki --- In a critical update for local inference deployments, [llama.cpp release b9982](https://github.com/ggml-org/llama.cpp/releases/tag/b9982) addresses a parameter parsing bug that previously ignored per-request reasoning budget overrides in chat completions. By restoring dynamic control over reasoning parameters, this release ensures parity with upstream API standards and allows developers to programmatically throttle or suppress the reasoning overhead of advanced models on a call-by-call basis. ## The Architectural Shift Toward Reasoning Models The landscape of large language models has recently bifurcated, with a significant portion of the ecosystem moving toward reasoning-heavy architectures. Models inspired by the DeepSeek-R1 paradigm or the emerging Qwen3 series utilize an internal "thinking" phase, generating hidden or visible reasoning tokens before producing a final response. While this drastically improves performance on complex logical, mathematical, and coding tasks, it introduces substantial latency and compute overhead. Local inference servers like llama.cpp are rapidly evolving to support these models, but doing so requires strict adherence to new API parameters that dictate exactly how much compute a model is allowed to spend on thinking. ## Diagnosing the Parameter Parsing Bug The core issue resolved in release b9982 centered on the `oaicompat_chat_params_parse` function, which handles incoming requests formatted for the OpenAI-compatible API endpoint. Prior to this fix, the reasoning-budget block read only the server-level default (typically set via `opt.reasoning_budget` to -1) and the Anthropic-style alias `thinking_budget_tokens`. It entirely missed the canonical `reasoning_budget_tokens` field supplied in the actual request body. Because this key was written into `llama_params` before the server's generic body-copy loop executed, the copy loop detected that the key was already present and silently skipped the caller-supplied value. Consequently, any per-request override-such as passing a budget of 0 to suppress thinking entirely for a simple query-was discarded, forcing the server to fall back to its global defaults. A parallel bug existed for `reasoning_budget_message`, which dictates the message injected before the end tag when a model exhausts its allocated reasoning budget. ## Implementation and Testing Evidence The patch, co-authored by Xuan Son Nguyen from Hugging Face, restructures the parsing logic to prioritize the request body. The server now reads `reasoning_budget_tokens` and `reasoning_budget_message` directly from the incoming payload first, falling back to server defaults only if the keys are absent. Furthermore, the update cleans up the codebase by collapsing adjacent overrides into a single `json_value()` call, eliminating a redundant `body.contains()` guard since the JSON parser natively handles missing or null keys. To prevent future regressions, the maintainers introduced specific unit tests within `test-chat.cpp`. These tests exercise the `oaicompat_chat_params_parse` pathway using a Qwen3 template. Notably, the llama.cpp autoparser explicitly detects Qwen3 as a "thinking-capable" model. The tests assert that when a request specifies `reasoning_budget_tokens == 0`, the resulting `llama_params` correctly reflects that exact value, confirming that the sampling layer receives the intended constraints. ## Implications for Agentic Workflows and Compute Optimization The ability to dynamically control reasoning budgets is a fundamental requirement for cost and latency optimization in agentic systems. In a multi-agent architecture, not every prompt requires deep reasoning. A routing agent or a simple classification node requires immediate, low-latency outputs, whereas a complex code-generation agent benefits from a maximum reasoning budget. Without this fix, developers relying on llama.cpp as a local backend were forced into a binary state: either enable reasoning globally and suffer massive latency penalties on trivial tasks, or disable it globally and cripple the model's capabilities on complex tasks. By ensuring that per-request parameters are correctly passed to the sampling layer, llama.cpp restores fine-grained control. Developers can now programmatically toggle reasoning overhead on a call-by-call basis, aligning local deployment capabilities with managed services like OpenAI's o-series APIs. ## Limitations and Open Questions While release b9982 successfully patches the immediate parsing failure, several open questions remain regarding the broader ecosystem integration of reasoning models. The exact performance impact or latency overhead of parsing these parameters dynamically per request is not quantified in the release notes, though it is likely negligible compared to the inference time saved by suppressing unnecessary reasoning tokens. Furthermore, the fragmentation of API standards remains a point of friction. The commit notes highlight the handling of Anthropic's `thinking_budget_tokens` alongside OpenAI's `reasoning_budget_tokens`. As different client libraries and upstream providers adopt varying nomenclatures for reasoning constraints, local servers will need to maintain complex translation layers to ensure seamless compatibility. Finally, the specific release timeline and versioning of Qwen3 models that trigger the autoparser's thinking-capable detection remain somewhat opaque, requiring developers to closely monitor template configurations to ensure their specific model variants are correctly identified by the server. Ultimately, this release underscores the rapid maturation required of local inference engines. As models transition from simple text generators to complex reasoning engines, the infrastructure supporting them must provide precise, granular control over compute resources to remain viable for enterprise and agentic applications. ### Key Takeaways * Llama.cpp release b9982 fixes a bug where the server silently ignored per-request 'reasoning\_budget\_tokens' and 'reasoning\_budget\_message' overrides. * The fix ensures that caller-supplied values are read from the request body before falling back to server-level defaults, allowing developers to suppress thinking by setting the budget to 0. * Unit tests were added using a Qwen3 template, which the system's autoparser natively detects as a thinking-capable model. * This update is critical for agentic workflows, enabling developers to optimize latency and compute by toggling reasoning overhead on a call-by-call basis. * The patch highlights ongoing API fragmentation, as servers must handle multiple standards like OpenAI's reasoning parameters and Anthropic's thinking parameters. --- ## Sources - https://github.com/ggml-org/llama.cpp/releases/tag/b9982