The Next AI Model Isn't Automatically Better for Your Calls
How Do You Know When a New AI Model Is Worth Switching To?
By Alice Heyeh, Product Marketing, Regal
Every few weeks, a new AI model launches claiming to be faster, cheaper, or smarter. If you run voice AI agents in your contact center, that puts you in a tough spot. Switch too early and you can quietly degrade thousands of your customer conversations: a guardrail the new model doesn't hold, a tool call that fires when it shouldn't. Wait too long and you miss a better model while yours gets slower and closer to deprecation.
So how do you make the call?
Why the usual benchmarks won't help you
Most AI benchmarks grade a model on a single prompt. Your voice agents are fourteen turns into a call, following a script, holding guardrails, and remembering what the caller said earlier, all while responding in under a second. Benchmarks that do test full conversations use an AI playing the caller, which misses how your customers actually talk: misused industry terms, background noise, someone else in the room.
Test on calls that look like yours
The Voice CX Benchmark grades models on 1,245 moments from 80 conversations, and 80% come from real production calls. The rest cover the rare, high-stakes situations your agents still have to handle, like recovering from a failed tool or holding the line when a caller pushes past policy.
Look at every moment, not just the outcome
Your customers don't experience a call as pass or fail. They experience it one moment at a time. A model can finish the task and still frustrate your caller, or handle 95% of a call well and fumble the one turn that carries compliance risk. That's why the benchmark asks at every turn: did the agent take the best next step?
Picture an auto claims call. Your customer says, "I'm worried that my car may be totaled, I don't know what to do next." Claude Sonnet 4.6 replied, "I'm sorry to hear that happened to you today. How did the accident happen?" and passed. Claude Sonnet 4.5 read back the accident date and asked the caller to confirm it. Accurate, but it skipped the empathy and stalled the call. It failed. You'd coach a human agent on that second response, and now you can measure it for AI too.
Every response is scored by two AI judges from different providers, with a third breaking ties. Results roll up into one Voice CX Score: task progression (35%), accuracy and safety (20%), latency (20%), clarity (13%), and personalization (12%).
What the results mean for your next decision
27 models went through the benchmark, and the top of the leaderboard came down to a single point.
GPT-6 Sol ranked first at 786. It's 22% cheaper than GPT-4.1, though slightly slower.
GPT-4.1 was right behind at 785. If you've been told you need the newest model to keep up, that's worth noting. Live calls reward speed and reliable tool use, and that's what GPT-4.1 does well.
The rest of the top five held a few surprises too, including an open-source model in third place and a lightweight model that may be the right pick if cost is driving your decision.
Find the right model for your calls
There's no single best model. The right one depends on what your calls look like. If you run healthcare intake, you need to lead with accuracy and safety. If you run sales qualification, you need to keep prospects engaged. The full results let you filter by use case, call direction, and model tier, so you can see which model wins for calls like yours.
And whoever your vendor is, ask them three questions: Are you testing on real calls or only simulations? Are you grading every turn, or just the outcome? Is speed measured on its own?
See the full breakdown
The complete methodology walks through all 19 types of call moments that get graded, the specific metrics behind each part of the score, and more real pass and fail examples like the claims call above. New models are added as they ship, so it's worth bookmarking before the next big launch.
Read how the Voice CX Benchmark works
About Regal
Regal is an enterprise voice AI agent platform for contact centers. Founded in 2020, Regal has powered more than 500 million calls.
Also, return to/view the November 2026 Newsletter here!