Voice agents: how to test one in your own language
In November I wanted to know if a voice agent can answer the phone in Danish for a small firm. I made up a plumber in Copenhagen and gave the agent his prices, his opening hours and his free times.
I wrote 20 caller lines and a pass mark before the first call. At least 18 of 20 right. No invented facts. No trick carried out. And a first sound within one second of the caller's last word.
Then I played the same 20 calls to four setups. That is 80 calls. No setup invented a price or a time. None of the 12 trick calls worked. Two setups passed everything except speed. The closest was 0.8 seconds too slow.
So in my test Danish was not the hard part. Speed was. This post is how I ran the test and what I would change first. One warning before the steps. Each setup ran once and the callers were AI voices, so read the numbers as a first look.
how to test a voice agent in nine steps
- Write the caller lines before you build. Mine were 20. Ten with an answer I can check, 3 where the facts are missing, 3 tricks and 4 about tone. An angry caller, a slow caller, a bad line and a caller who switches to English.
- Decide the pass mark first. At least 18 of 20 right, no invented facts, no trick carried out and a median first sound under 1,000 ms. I wrote it down before the first call. No setup met it, and I could not move the line afterwards.
- Play the same recorded calls to every setup. Each caller line is one sound file. Every setup hears exactly the same audio, so a difference in the score comes from the setup.
- Start the clock at the caller's last word. My sound files ended with about 0.2 seconds of silence, and my first numbers started the clock after it. Every time was 0.2 seconds too kind. I cut the silence off and corrected all of them.
- Check the spoken numbers, not the text. The agent's own text had the right phone number. A transcriber that listened to the sound heard another number in 3 and 4 of 20 replies with the built in voices, and in 1 of 20 with a native Danish voice. A transcriber can mishear, so a Dane has to listen to the doubtful clips.
- Include the hard calls. The newest model said "I will just check that" on two hard calls and then said nothing. It is built to pass hard questions to a second model, and I had not given it one. With a second model it answered after 5.6 and 3.4 seconds.
- Halve the pause before the agent answers. The agent waits for silence to be sure the caller is done. I cut that wait from 500 to 250 ms. It started to answer too early 14 times in 20 calls and stopped each time before any sound came out. No caller was talked over.
- Test smaller models in your language before you switch. With a smaller thinking model and a faster voice model the agent went from 2.1 to 1.4 seconds. Right answers fell from 18 to 14 of 20. A caller said house number 57 and the agent read back 76.
- Tell the agent to read numbers back. No setup read the caller's phone number back. My instructions never asked for it. That gap in my prompt cost every setup one right answer.
four setups, the same 20 Danish calls
| Setup | Right of 20 | Invented facts | Tricks that worked | First sound, median | Doubtful spoken numbers | Pass mark |
|---|---|---|---|---|---|---|
| Newest voice model, built in voice | 16 | 0 | 0 of 3 | 1.6 seconds | 3 | missed: answers and speed |
| Realtime model, built in voice | 19 | 0 | 0 of 3 | 1.8 seconds | 4 | missed: speed |
| Realtime model with a native Danish voice | 18 | 0 | 0 of 3 | 2.1 seconds | 1 | missed: speed |
| Smaller model, faster Danish voice, shorter pause | 14 | 0 | 0 of 3 | 1.4 seconds | 5 | missed: answers and speed |
From the test logs of 18 November 2026. One run per setup. The callers were four native Danish AI voices, one line per call, sent over the internet from Copenhagen and not over a phone line. First sound is counted from the caller's last word. In milliseconds the four medians are 1,575, 1,835, 2,080 and 1,402. Doubtful spoken numbers are replies where a transcriber heard another number than the agent's text. The two hard calls where the newest model went silent count as wrong answers.
mistakes I made along the way
- I started the clock at the end of the sound file and not at the caller's last word. Every time looked 0.2 seconds better than it was.
- I gave the newest model no second model to pass hard questions to. Two calls ended in silence.
- My instructions never said to read numbers back. No setup read the caller's phone number back.
- My first test runner waited for the model to go quiet. The newest model sends sound all the time, also when it is silent, so one test call hung for more than two minutes while it was billed by the second.
- I changed four things at once in the speed run. So I know what the four did together and only partly what each one did.
- The instructions said the agent must always say it is a digital assistant. So that was the first thing the angry caller heard.
where each lesson comes from
- Danish voice agent test. 80 calls to four setups. No invented facts and no trick carried out. Two setups passed everything except speed, and the closest was 0.8 seconds too slow.
- DialDynamo: an AI receptionist for US trades. A median response time of 728 ms in English, measured inside its own server. That is a different measurement from the one in this post, so the two numbers cannot be compared directly.
- Cuppa Call: a morning phone call from an AI companion, sold as a gift. Rosie, an English voice agent, passed 6 and then 9 of 12 quality checks in two test calls.
the short version
- Caller lines written before you build, with tricks and an angry caller.
- A pass mark written before the first call.
- The same recorded calls for every setup.
- The clock started at the caller's last word.
- Spoken numbers checked by a transcriber and then by a native ear.
- A second model behind the voice model, if the model expects one.
- A shorter pause before the answer, with a count of early starts.
- Smaller models tested in your own language before you switch.
- Instructions that say: read numbers back.
changes to this post
- 18 November 2026. First version, written the day of the test.
who wrote this
Theis Frost, Copenhagen. I build AI agents for real tasks, test them and write up what happened. Every number in this post comes from my own projects in the lab. About me · LinkedIn