# llama.cpp b9760 Refactors Server Schema for Native Base64 Video Ingestion > The transition to multimodal edge deployment accelerates as the inference engine eliminates external video preprocessing requirements. **Published:** June 22, 2026 **Author:** PSEEDR Editorial **Category:** edge **Content tier:** free **Accessible for free:** true **Editorial format:** analysis **News quality eligible:** true **Source count:** 1 **Word count:** 971 **Tags:** llama.cpp, Multimodal AI, Edge Computing, Vision-Language Models, Inference Server **Canonical URL:** https://pseedr.com/edge/llamacpp-b9760-refactors-server-schema-for-native-base64-video-ingestion --- The recent release of llama.cpp b9760 introduces a critical refactor to the project's server input file schema, enabling native support for raw base64 video inputs. By wiring up input\_video functionality directly into the server architecture, the project signals a definitive shift from a text-centric inference engine to a highly versatile, multimodal edge backend capable of supporting complex vision-language models (VLMs). The recent [release of llama.cpp b9760](https://github.com/ggml-org/llama.cpp/releases/tag/b9760) introduces a critical refactor to the project's server input file schema, enabling native support for raw base64 video inputs. By wiring up `input_video` functionality directly into the server architecture, the project signals a definitive shift from a text-centric inference engine to a highly versatile, multimodal edge backend capable of supporting complex vision-language models (VLMs). This analysis examines the technical mechanics of the schema refactor, the implications for edge deployment pipelines, and the unresolved performance questions surrounding native video ingestion. ## Refactoring the Server Schema for Multimodal Ingestion Historically, deploying multimodal models via llama.cpp required developers to manage complex preprocessing pipelines. To process video, client applications had to extract individual frames using external libraries, encode those frames as discrete images, and transmit them as an array of image payloads to the server. Pull Request #24299, integrated into release b9760, fundamentally alters this architecture by generalizing the input file schema. The release notes explicitly highlight the wiring of `input_video` to accept raw base64 encoded video data. By standardizing video ingestion at the server level, llama.cpp absorbs the preprocessing burden. Client applications can now transmit a single base64-encoded video stream directly to the inference endpoint. This structural change aligns the llama.cpp server with modern RESTful API standards for multimodal LLMs, positioning it as a drop-in replacement for cloud-based VLM APIs that already support direct video uploads. ## Broadening Cross-Platform Edge Viability The b9760 release also underscores the project's aggressive expansion across diverse hardware accelerators, which is crucial for processing compute-intensive video workloads at the edge. The release targets an extensive matrix of platforms, ensuring that the new multimodal capabilities are not restricted to high-end data center GPUs. Notable updates include the integration of CUDA 12.4 and CUDA 13.3 DLLs for Windows x64 environments, ensuring compatibility with the latest NVIDIA driver ecosystems. Furthermore, the release maintains robust support for Apple Silicon (arm64), Android (arm64), and various Linux backends including Vulkan, ROCm 7.2, OpenVINO, and SYCL. For enterprise and industrial deployments, the inclusion of openEuler support-specifically targeting the 910b architecture with ACL Graph-demonstrates a commitment to specialized edge hardware often utilized in telecommunications and smart city infrastructure. This broad hardware support ensures that video-capable VLMs can be deployed in highly constrained environments, from mobile devices to embedded industrial systems. ## Implications for Edge Video Analytics Pipelines The ability to ingest base64 video directly into the llama.cpp server carries significant implications for the architecture of edge video analytics. In traditional setups, the requirement for external video-to-frame middleware introduces latency, increases the application's memory footprint, and complicates the deployment container. By eliminating this middleware, developers can drastically simplify their software stack. This simplification is particularly valuable for real-time applications such as autonomous robotics, drone surveillance, and local security systems. In these scenarios, devices can capture video, encode it to base64, and pipe it directly to a local llama.cpp instance running a vision-language model. The reduction in architectural complexity lowers the barrier to entry for building local, privacy-preserving video analytics tools. It shifts the paradigm from relying on complex, multi-container deployments to utilizing a single, highly optimized inference binary that handles both the media ingestion and the neural network execution. ## Limitations and Open Performance Questions Despite the architectural benefits, the b9760 release leaves several critical technical questions unanswered, particularly regarding performance overhead and model compatibility. The release notes do not document the exact JSON payload structure required for the new generalized input schema, leaving developers to infer the API contract from the source code. Furthermore, the specific multimodal video-LLMs supported by this new schema remain unspecified in the brief. While the infrastructure now exists to accept video, the mapping of that video data to the specific tensor inputs required by various VLM architectures-which often differ significantly in how they project video frames into the LLM context space-is a complex challenge. The most pressing concern involves the performance impact and memory overhead of decoding raw base64 video streams directly within the server. Base64 encoding inherently inflates file sizes by approximately 33 percent. Transmitting large video files as base64 strings over local networks or inter-process communication channels can introduce latency. More importantly, decoding video requires substantial memory and compute. It is currently unclear whether llama.cpp leverages hardware-accelerated video decoders or relies on CPU-bound decoding. If the latter, processing high-resolution video inputs could severely bottleneck the inference engine, negating the latency benefits of a simplified pipeline. ## Synthesis The b9760 release represents a pivotal maturation point for llama.cpp. By refactoring the server schema to natively support base64 video ingestion, the project is actively dismantling the barriers between text-based inference and complex multimodal edge computing. While questions remain regarding memory overhead, hardware-accelerated decoding, and specific model compatibility, the architectural direction is clear. The project is positioning itself not just as an LLM runner, but as a comprehensive, localized inference server capable of handling the next generation of vision-language applications directly at the edge. ### Key Takeaways * llama.cpp release b9760 refactors the server input schema to natively accept raw base64 video inputs via the input\_video parameter. * The update eliminates the need for external video-to-frame preprocessing middleware, simplifying edge deployment pipelines for vision-language models. * The release expands hardware support, including CUDA 12.4/13.3 DLLs for Windows and openEuler ACL Graph support for enterprise edge environments. * Open questions remain regarding the exact JSON payload structure, specific VLM compatibility, and the memory overhead of decoding base64 video directly within the server. --- ## Sources - https://github.com/ggml-org/llama.cpp/releases/tag/b9760