together-blog has announced a new architecture for building highly responsive real-time voice agents, leveraging co-located STT, LLM, and TTS infrastructure to achieve end-to-end latency under 700 milliseconds.
\nIn a recent post, together-blog discusses the launch of their new infrastructure designed specifically for building real-time voice agents. By integrating co-located Speech-to-Text (STT), Large Language Models (LLMs), and Text-to-Speech (TTS) systems, Together AI is targeting one of the most persistent bottlenecks in conversational AI: latency. This announcement signals a major shift in how developers can architect voice-first applications, moving away from fragmented API calls toward unified, high-speed environments.
The demand for voice-native AI applications is surging across various sectors, including automated customer service, interactive gaming, real-time translation, and accessibility tools. However, creating a fluid, human-like voice interaction requires chaining multiple complex models together in rapid succession. Traditionally, a developer must send user audio to an external STT provider, route the transcribed text to an LLM for processing, and then forward the generated text response to a separate TTS service to produce the final audio output. Each of these steps introduces compounded network delays, often referred to as the network hop problem. When end-to-end latency exceeds one second, conversations begin to feel unnatural and robotic, leading to user frustration, awkward pauses, and frequent interruptions. Addressing this fundamental infrastructure challenge is critical for the next generation of voice interfaces to achieve true commercial viability.
To solve this compounding delay, together-blog's post explores the mechanics and benefits of a co-located infrastructure approach. By hosting the STT, LLM, and TTS components within the same physical network environment, Together AI drastically reduces the transit time between these distinct processing steps. The data no longer needs to traverse the public internet multiple times to generate a single response. Furthermore, the platform introduces native, optimized support for specialized, high-performance providers like Deepgram for highly accurate speech recognition and Cartesia for expressive, rapid speech synthesis. This tight integration allows developers to build sophisticated voice applications that achieve an impressive end-to-end latency of under 700 milliseconds. Crossing this specific sub-second threshold is widely considered the benchmark required for natural, conversational pacing, allowing AI agents to respond as quickly as a human counterpart.
For developers, product managers, and engineers working on conversational AI, this architectural shift represents a meaningful step forward in reducing the friction of voice interactions. The emphasis on tight integration with industry-leading audio models suggests a robust, enterprise-ready ecosystem for low-latency applications. By abstracting the complexity of managing multiple disparate APIs and focusing on co-location, Together AI is lowering the barrier to entry for building highly responsive voice agents. We highly recommend reviewing the original publication to understand the implementation details and explore how this infrastructure can be applied to your own AI projects.
Read the full post on together-blog
\n\n