How Real-Time Voice Translation Works
Ten years ago, "real-time translation" meant, at best, a human interpreter in your headphones. Today software can do the same thing — and it doesn't sound like a robot, it sounds like your own voice in another language. Let's break down what's actually happening under the hood.
What a voice translation system is made of
There are three sequential stages. First, automatic speech recognition (ASR) turns audio into text in the source language. Then machine translation (MT) translates that text into the target language. Finally, speech synthesis (TTS) voices the translated text — either in a standard voice or, as with Loquora, in a voice cloned from a sample of the speaker. Each stage used to run separately, with seconds of lag between them; modern systems run this as a stream, in chunks of a few words at a time, instead of waiting for a full sentence to finish.
Why latency is the real problem
A conversation stops feeling live once the pause between a line and its translation stretches past a couple of seconds — people start talking over each other or awkwardly waiting. So the real engineering challenge in voice translation isn't the translation itself (text translation has been solved reasonably well for years) — it's running recognition, translation, and voicing in parallel and as a stream, instead of one after another. Loquora averages about one second, which is enough to keep a conversation feeling natural.
The role of AI in preserving tone and voice
A separate challenge is not losing the person behind the translation. A voice model trained on a short recording (about a minute of speech) can carry the original's timbre, pace, and intonation over into synthesized speech in another language. This isn't pre-recorded phrases — it's real-time speech generation designed to sound as close as possible to how that same person would sound speaking the target language.
Which language pairs are harder than others
Not all languages are equally easy for machine translation. Tonal languages (Chinese, Vietnamese), languages with very different word order (Japanese, Korean), or languages with rich gender and case grammar need more context for an accurate translation — the system has to "wait" for more words before the translation becomes unambiguous, which can add a bit of latency on those pairs.
Where this technology is already being used
Real-time voice translation covers three main scenarios today: work calls without an interpreter, video meetings with overseas teams and partners, and personal conversations with people who speak a different language. Each one gets its own deep dive elsewhere in this blog.