Sierra released the τ-Entity benchmark on October 22, 2026, to test voice agents on the most basic job of all: hearing a name, phone number, address or email correctly, writing it down right, and fixing it when it may be wrong. The suite holds 200 tasks across ten entity types, each call captures a single field, the caller is cooperative and spells things out when asked, and most calls take under two minutes. Even in this deliberately simplified setting, today's voice agents fail often.
The benchmark grew out of a bottleneck Sierra kept seeing in τ-Voice: a conversation can flow beautifully, but one wrong digit fails authentication and the actual service never starts. Before a voice model can do work, it has to pass the dictation test, and now that Mistral's Voxtral 3 folds listening, speaking and task work into one model, this basic reliability deserves measuring on its own.
Even the leader's score wilts when conditions change
τ-Entity scores with Pass³: a task only counts when the agent succeeds under regular, noise-heavy and speech-heavy conditions alike. On the leaderboard pooled across six languages, GPT Live 1 leads, capturing the value correctly in 60.3% of regular calls but passing all three conditions on only 34.0% of tasks; Gemini 3.8 Live and Grok Voice Think Fast 2.0 reach 43.8% and 44.2% in regular conditions and fall to 19.5% and 24.0% across all three. English alone looks better, with GPT Live 1 at 74.0% regular and 47.5% across all three, but Korean Pass³ drops to 3.5 to 8.5% and Mandarin to 12.5 to 24.5% across agents, so the gap opens as soon as the language changes.
Slowing it down to verify helps, at half a minute a call
Sierra's companion τ-Elicitation study also measured verification itself: whether agents read back or ask for spelling when unsure. They recognize only some warning signs, verifying harder and unfamiliar values more carefully, while only Grok verified more under noise and none worked harder on the caller voice it handled worst. Handing them a prescribed sequence, ask, spell or read back, confirm and correct, then submit, lifts all-conditions success from 14 to 41% up to 37 to 54%, a gain of 14 to 31 percentage points, at a cost of 21 to 28 extra seconds per call. And verification is not correction: of initially wrong captures that agents did verify, only 24 to 37% ended up fixed.
For customer-service deployment the arithmetic is plain: on a three-field call, even with individual fields still 75% accurate, only 53% of calls fully succeed, while validating each field and allowing retries raised multi-field success from 64% to 82%. What a voice agent should be examined on before launch is not how human it sounds, but whether it knows to stop when unsure, and how far it falls in noise and other languages; that is exactly the homework benchmarks like τ-Entity are starting to do for the industry.
