in10x / lab / danish-voice-test
Danish voice agent test
The question. Is a Danish voice agent good enough to answer the phone for a small firm?
what I built
A test bench that plays the same 20 recorded Danish caller lines to a voice agent, one call per line, and saves what comes back. The agent answers the phone for a made up plumber in Copenhagen with fixed prices, opening hours and free times. Four setups heard the same calls. The pass mark was written before the first call: at least 18 of 20 right, no invented facts, no trick carried out and a first sound within one second.
OpenAI gpt-live-1 · OpenAI gpt-realtime-2.1 · OpenAI gpt-realtime-2.1-mini · ElevenLabs Eleven v4 Turbo · ElevenLabs Flash v2.5 · Python
what happened
- 80 calls to four setups. No setup invented a price or a time, and none of the 12 trick calls worked.
- Right answers of 20: 16, 19, 18 and 14. Median time from the caller's last word to the first sound: 1,575, 1,835, 2,080 and 1,402 ms.
- No setup passed. Two passed everything except speed. The closest was 0.8 seconds too slow.
- With a native Danish voice a transcriber heard another number than the agent's text in 1 of 20 replies. With the built in voices it was 3 and 4 of 20.
- The speed run made the agent a third faster, from 2,080 to 1,402 ms. Right answers fell from 18 to 14 of 20, and one address was read back with house number 76 where the caller said 57.
- Halving the pause before the answer, from 500 to 250 ms, made the agent start too early 14 times in 20 calls. It stopped each time before any sound came out.
Cost. About 1.30 dollars for the models, plus voice credits from a plan I already pay for.
what broke
- The newest voice model said it would check and then went silent on two hard calls. It expects a second model behind it, and the test had not given it one. With one, the answers came after 5.6 and 3.4 seconds.
- My first times started the clock at the end of the sound file, about 0.2 seconds after the caller's last word. All times were corrected.
- No setup read the caller's phone number back. The instructions never asked for it.
decision
Done as a first look. Each setup ran once, the callers were AI voices, each call was one line, and nothing went over a real phone line. The next test needs a real Danish caller and a second run.
- Decide the pass mark before the first call.
- Start the clock at the caller's last word.
- Check the spoken numbers, not the agent's text.
- Shorten the pause before you shrink the model.
The same lessons as a how-to: Voice agents: how to test one in your own language.
Every number on this page comes from the project's own logs, analytics or database. Back to the lab.