Voice Assistant for Messenger

Names are the hard part, not the speech

General transcription quality was never the bottleneck. Personal names are the part of speech recognition with the least context to lean on: they are not in the language model's ordinary vocabulary, they are frequently foreign to the language being spoken, and a user's address book is a small closed set that the recogniser knows nothing about. A spoken name therefore has to be matched against that particular user's contacts, tolerating spelling variants across languages, rather than transcribed correctly in the abstract.

Built on GPT, with the matching written separately

The assistant runs on OpenAI GPT for understanding what the user asked for, and on a separate algorithm written specifically for resolving spoken contact names, including variant spellings drawn from different languages. Splitting the two is the design decision worth naming: the general model handles intent, and the part where a mistake is expensive and unambiguous is handled by code that can be tested against a known answer.

The two obvious builds, and why neither one

Both alternatives are cheaper to reach for, and both were the wrong shape for this problem.

Better speech-to-text

The intuition is that a stronger recogniser fixes it. It does not, because the problem is not acoustic. A recogniser produces the most probable string given the language it is working in, and a personal name is precisely the token where that prior is worthless - often foreign to the language being spoken, often absent from the vocabulary entirely. Improving general transcription improves everything except the word that mattered.

Hand the address book to the model

Passing the contact list as context looks like the modern answer and carries three costs. It sends a user's entire social graph to a third-party API on every request, which is a privacy decision disguised as an implementation detail. It grows the prompt with the address book, on a path that has to answer in real time. And it makes the expensive failure untestable - you can score a matching algorithm against a list of known answers, but you cannot do the same with a model that will phrase its uncertainty differently each time.

The latency budget is different for voice

A person waiting for a voice assistant is not the same person waiting for a chat reply. In text, a pause reads as thinking; in speech, the same pause reads as the thing being broken, and the user repeats themselves and makes it worse. That constraint is why the processing architecture was designed for it rather than assembled and then optimised - recognition, resolution and response generation all sit inside one budget that a person is measuring against conversation, and the matching step cannot be the place it goes.

Every knowledge base has names it stores differently from how people say them

Resolving a spoken name against one user's address book is fuzzy entity retrieval, and the shape of it repeats everywhere. Every knowledge base has entities whose names people say differently from how the system stores them: suppliers with three legal spellings, a part number a technician reads out with the dashes in the wrong places, a client the sales team has always called something other than what the contract says. The lesson transfers with the nouns swapped. Search that matches on meaning does not solve identity, and identity is usually where the costly mistakes are.