# Apertus 1.5 Puts Images and Audio in the Same 262K Context Budget as Text > Images, audio and text share Apertus 1.5's context window. Budget the processed request and test what added media contributes. **Published:** October 15, 2026 **Author:** PSEEDR Editorial **Category:** platforms **Content tier:** free **Accessible for free:** true **Editorial format:** analysis **News quality eligible:** false **Source count:** 3 **Word count:** 1291 **Read time:** 6 min **Tags:** Apertus, Multimodal AI, Context Windows, Tokenization **Canonical URL:** https://pseedr.com/platforms/apertus-15-puts-images-and-audio-in-the-same-262k-context-budget-as-text --- A document assistant receives twenty scanned pages and a half-hour recording. Both fit the product description of a multimodal model. Whether they fit a useful request depends on what the model turns them into. In Apertus 1.5, a 512-by-512 image contributes 1,024 image codes and audio contributes 40 codes per second. At those rates, twenty pages scanned at 512 by 512 and thirty minutes of audio contribute 92,480 codes before the text instructions, conversation history, media markers or generated answer. That is a calculation from the published encoding rates, not a measured application workload. It shows why counting attachments or minutes is an incomplete way to budget a request. The same arithmetic gives an hour of audio 144,000 codes, more than half of the entire window. The [Apertus team's October 15 technical post](https://huggingface.co/blog/swiss-ai/apertus-v15-release-notes) describes a model whose modalities share one sequence. The post explains the training and design behind weights that had already been released on July 24; it announces no new weight release. It points to the project's technical report, an arXiv preprint dated October 11. The report sections and model card cited below are the versions PSEEDR read on November 1, 2026. For an engineer building a document or meeting assistant, the useful question is how to spend that sequence on evidence the answer actually needs. ## One sequence, several kinds of token Apertus converts pictures and sound into discrete codes before the language model processes them. Its [technical report, sections 2.1.1 and 2.1.2](https://arxiv.org/html/2610.15542), describes a vision tokenizer that produces one code for each 16-by-16 block of pixels. A 512-by-512 image therefore produces a 32-by-32 grid of codes. The tokenizer accepts arbitrary shapes, with the grid set by height and width each divided by 16, so a 1,024-by-1,024 scan produces 4,096 codes. In the sequence, a start marker and the image dimensions open each image, end-of-row markers separate the rows, and an end marker closes it. Those markers sit outside the 1,024-code figure. Audio takes a different route into the same sequence. The WavTokenizer codec consumes mono audio at 24 kHz and emits one code for each 600 samples. That produces the 40-codes-per-second rate, or 2,400 codes per minute. Audio at other sample rates is resampled to 24 kHz first, and every input is normalized to a fixed peak level before encoding, so the code count follows the recording's duration rather than its original sample rate. These numbers belong to this one processing path. Two limits matter for a meeting assistant. The team's post labels spoken-language understanding as experimental. The [model card](https://huggingface.co/swiss-ai/Apertus-v1.5-8B) states that native video is not supported; an application has to extract frames and pass them as images, where each frame costs what an image costs. The language model reads these codes alongside text through an extended input vocabulary. It generates text only. A common sequence makes it possible to place a question beside the relevant page or recording, but the representations do not become interchangeable. A thousand image codes carry a different kind of information from a thousand text tokens, and the model's answer quality on each is a separate question from how many positions each occupies. A related separation, between training throughput and application cost, changes how to read the team's statement that training runs at text-only speed. The report explains that training images and audio are tokenized once, offline, so no vision or audio encoder runs inside the language model's forward pass during training. That is a throughput benefit at the training stage, where the tokenized datasets already exist and are reused across runs. When an application receives a new recording that has not been tokenized, it must run the audio tokenizer and then have the model read the resulting positions. The statement therefore does not mean that encoding and answering questions about an hour of sound costs an application the same as reading a short transcript of it. ## Budget the processed request For the hypothetical assistant, 20 × 1,024 gives 20,480 image codes. Thirty minutes gives 30 × 60 × 40, or 72,000 audio codes. The total is 92,480, but it is not the final prompt length. The engineer still has to account for the media markers, text and conversation formatting produced by the processor, then leave room for the output. The reliable place to check is therefore after preprocessing. The model card's example exposes the tokenized input IDs, whose length can be checked before generating. A product limit such as twenty attachments can be useful for users, but it needs a corresponding limit on that measured sequence. Otherwise, resizing a scan, retaining another turn of conversation or changing the media mix alters what the model receives while the attachment count stays the same. Output space deserves its own allowance. The model card warns that in thinking mode, reasoning can run several times longer than the visible answer, and a truncated generation may end before the answer begins. A generation limit that looks generous for a short summary can therefore be consumed before that summary starts. This is one more reason not to treat the whole advertised window as space for source material. What remains is the choice of which material to send. If a question concerns one table, sending every page spends sequence space on material that adds no evidence. Selecting the relevant pages reduces the input, provided the selection step does not discard the footnote or adjoining table needed to interpret it. If a meeting question concerns how a decision developed, a narrow audio clip may omit the earlier qualification that changes its meaning. Selection saves space only when it preserves the relationship the question asks the model to explain. ## Fitting is only the first test A request can fit the window and still be answered poorly. Apertus's own training history makes that distinction concrete. In section 5.7 of the technical report, the team evaluates long-context ability on the RULER and HELMET benchmarks at every stage of its pipeline. The context-extension stage carries the model out to 256k tokens. Short-context supervised fine-tuning then collapses retrieval on long inputs, and a dedicated long-context fine-tuning stage restores it. The supported position limit stayed the same across those stages; the checkpoint's ability to use distant information did not. That result does not establish how the hypothetical assistant will perform on scanned reports and recordings. It establishes why the acceptance test should measure the intended task as the input grows. For a document comparison, keep the required facts fixed while adding irrelevant pages and moving the relevant ones. For a meeting question, test whether the answer preserves an earlier exception when the discussion continues. These are proposed application tests, not tests PSEEDR or the Apertus team has reported running on this example. The test also needs a comparison against a shorter, selected input. If both produce the right answer, latency and memory measurements can decide between them. If the shorter version loses an essential connection, the extra context has a demonstrated purpose. If the longer version loses the answer among irrelevant material, raising the request limit will not fix a selection problem. ## Spend context on the relationship that matters Apertus's encoding scheme gives engineers a way to estimate media input before they deploy. Its practical value is that it makes the budget visible: image area, audio duration, surrounding text and generated output all need room in one request. The deployment decision then rests on two measurements. Count the sequence the processor actually produces, then check whether the model answers the intended question with that sequence. Keep additional pages or audio when they preserve evidence or a necessary connection, and remove them when a shorter input answers just as well. The 262,144-token specification defines capacity. The application test determines how much of that capacity is useful. --- ## Sources - https://huggingface.co/blog/swiss-ai/apertus-v15-release-notes - https://arxiv.org/html/2610.15542 - https://huggingface.co/swiss-ai/Apertus-v1.5-8B