# The "Street Name" Gap: Why High-Performing Speech Models Stumble on Real-World Data

> Coverage of together-blog

**Published:** February 23, 2026
**Author:** PSEEDR Editorial
**Category:** platforms
**Content tier:** free
**Accessible for free:** true



**Word count:** 385


**Tags:** Speech Recognition, ASR, Machine Learning, AI Reliability, Whisper, Deepgram, Data Science

**Canonical URL:** https://pseedr.com/platforms/the-street-name-gap-why-high-performing-speech-models-stumble-on-real-world-data

---

In a recent technical post, the team at Together AI identifies a significant reliability issue in state-of-the-art speech models, specifically regarding the transcription of geolocation data.

The narrative surrounding modern Automatic Speech Recognition (ASR) often centers on the achievement of "near-human" performance. Flagship models such as OpenAI's Whisper and Deepgram's offerings have consistently excelled at standard academic benchmarks, leading to widespread adoption across industries. However, synthetic benchmarks and curated datasets often fail to capture the chaotic nature of real-world deployment, particularly when it comes to proper nouns and unique identifiers.

This context is crucial because the utility of ASR in vertical applications-such as ride-sharing, logistics, and emergency response-hinges on the accurate capture of specific entities rather than general conversational flow. In these scenarios, context clues are often minimal. When a user dictates a sentence, the model relies on probability to predict the next word; however, proper nouns like street names often defy standard probability distributions. A model that transcribes 99% of a sentence correctly but misses the specific address renders the entire transaction a failure.

Together AI's research exposes precisely this vulnerability. Their analysis indicates that while top-tier models perform exceptionally well on general speech, they suffer from a staggering 39% failure rate when processing street names. This finding highlights a critical "long-tail" problem: models struggle to generalize when faced with phonetically complex or uncommon proper nouns that lack surrounding semantic scaffolding. The post argues that current evaluation metrics mask these deficiencies, giving developers a false sense of security regarding model robustness.

The publication goes beyond merely identifying the discrepancy; it outlines the methodology used to uncover these specific failure modes and discusses a proposed solution to bridge the performance gap. For machine learning engineers and product leaders, this serves as a vital reminder that low Word Error Rates (WER) on general tests do not guarantee reliability in domain-specific contexts where precision is paramount.

We recommend this analysis to any team currently deploying speech-to-text solutions in production, particularly those dealing with high-cardinality data like names, addresses, or product SKUs.

[Read the full post at Together AI](https://www.together.ai/blog/how-speech-models-fail)

### Key Takeaways

*   State-of-the-art models like Whisper and Deepgram show a 39% failure rate on street names despite high benchmark scores.
*   Standard evaluation metrics (like WER on LibriSpeech) often mask poor performance on proper nouns and edge cases.
*   The failure to transcribe addresses accurately poses significant risks for logistics, navigation, and emergency services applications.
*   Together AI has proposed specific methodologies to identify and fix these robustness gaps.

[Read the original post at together-blog](https://www.together.ai/blog/how-speech-models-fail)

---

## Sources

- https://www.together.ai/blog/how-speech-models-fail