In its recently announced ICML 2026 research portfolio, Together AI outlines a systematic approach to verticalizing the AI training and inference stack. By optimizing everything from agentic scheduling down to multi-GPU kernels, the research demonstrates that targeted software-level interventions can yield hardware-equivalent performance gains without requiring physical infrastructure upgrades.
\nIn its recently announced ICML 2026 research portfolio, Together AI outlines a systematic approach to verticalizing the AI training and inference stack. By optimizing everything from agentic scheduling down to multi-GPU kernels, the research demonstrates that targeted software-level interventions can yield hardware-equivalent performance gains without requiring physical infrastructure upgrades.
As enterprise AI shifts from single-turn chatbot interactions to multi-step agentic workflows, traditional inference engines are exposing severe architectural bottlenecks. Because standard schedulers treat each step of a multi-turn agent workflow as an isolated request, latency can inflate by up to 7.14x under heavy load. Together AI addresses this at the systems layer with ThunderAgent, a program-aware inference system that treats the entire workflow as a first-class object. By allowing the scheduler to reason about the workflow end-to-end, ThunderAgent achieves 1.5x to 3.6x higher serving throughput and up to 4.2x disk savings for agent workloads.
A similar systems-level intervention is applied to Mixture-of-Experts (MoE) models through Opportunistic Expert Activation (OEA). While MoE architectures are designed for sparse, efficient execution, standard batching mechanisms destroy this sparsity. At a batch size of 16, a model designed to route tokens to just eight experts may end up loading over 80 experts into memory. OEA introduces batch-aware routing that recovers this sparsity at inference time. By allowing tokens to share experts that the batch has already loaded, OEA reduces MoE decode latency by up to 39% without requiring architectural changes or model retraining.
The portfolio highlights a distinct pivot away from relying solely on larger base models, focusing instead on test-time compute and novel alignment strategies. TTT-Discover exemplifies this shift by applying reinforcement learning at test time. Rather than prompting a frozen model thousands of times and selecting the best output, TTT-Discover allows the model to learn continuously during the generation process. Using an open-source 120B parameter model, this method set a tighter bound on a 60-year-old Erdős mathematics problem and outperformed prior submissions on the GPUMode kernel leaderboard, matching or exceeding the capabilities of closed frontier models for a fraction of the compute cost.
Furthermore, Together AI is addressing the bottleneck of ground-truth verification in reasoning tasks. Traditional reinforcement learning for reasoning requires a verifier to score correctness, limiting its application to domains like math and code where objective answers exist. RARO (Relativistic Adversarial Reasoning Optimization) bypasses this by training a single model to act as both a generator and a relativistic critic. This adversarial setup achieved a 25% win rate against expert responses on tasks lacking a verifier, compared to just 5.9% for standard supervised fine-tuning. Complementing this, V1-PairRL unifies generation and self-verification, improving Pass@1 accuracy by up to 10% using a near-linear Swiss-tournament verifier.
At the infrastructure layer, managing memory for long-context windows remains a primary constraint. Adding more GPUs does not inherently solve the activation memory limits inside the attention layer. Untied Ulysses tackles this through headwise chunking, processing a few attention heads at a time and reusing buffers across stages. This technique reduces peak attention memory by up to 87.5% on a 32B Transformer, enabling a 5M-token context window on a single 8xH100 node. As a drop-in replacement for DeepSpeed-Ulysses, it allows organizations to train on massive contexts without expanding their physical cluster size.
Algorithmic efficiency is further improved through Aurora, an adaptive speculative decoding framework. Static speculators often degrade in performance as live traffic patterns shift away from their offline training data. Aurora reframes speculator learning as an asynchronous, online reinforcement learning problem. By using accepted and rejected tokens as a live reward signal, the speculator updates continuously in production, delivering a 1.5x day-0 speedup on models like Qwen3-Coder-Next 80B, with an additional 1.25x speedup as it adapts to traffic.
Together AI's research signals a maturation in how the industry approaches AI deployment. The prevailing strategy of brute-force model scaling is hitting economic and hardware limits. By verticalizing its research, Together AI demonstrates that the most significant margin improvements now reside in the connective tissue of the stack: the schedulers, memory managers, and test-time algorithms.
This full-stack optimization approach commoditizes the base model while shifting the competitive moat to the inference engine. If open-source models paired with advanced test-time compute and memory-efficient parallelism can rival proprietary ecosystems, the premium charged by closed-model providers becomes harder to justify. Organizations can achieve frontier-level performance on existing hardware footprints, fundamentally altering the unit economics of enterprise AI deployment.
Despite the robust performance claims, several technical details remain obscured. The TTT-Discover paper relies heavily on an open model designated as gpt-oss-120b, yet the exact architecture, training lineage, and parameter distribution of this model are not detailed in the source material. Without this context, replicating the exact cost-to-performance ratio of the Erdős problem discovery is challenging.
Additionally, while Together AI notes that the Aurora adaptive speculator maps to its production ATLAS system, the specific implementation differences between the academic paper and the commercial deployment are undefined. Finally, the ParallelKernelBench introduces 87 multi-GPU kernel tasks, but the source truncates the specific workloads included, leaving the exact scope of the benchmark ambiguous.
The ICML 2026 portfolio from Together AI illustrates that the next frontier of AI performance is not exclusively tied to training larger models on larger clusters. By engineering solutions that span from agent-aware scheduling down to multi-GPU kernel generation, the company is proving that software-level ingenuity can extract hardware-equivalent gains. As the industry grapples with compute scarcity and scaling costs, this verticalized approach offers a sustainable blueprint for maximizing the utility of open-source models in production environments.
\n\n