Speech recognition
Speech recognition (STT) turns the caller's audio into text for the bot. This chapter lists every STT provider the gateway supports, the variables each one reads, and the settings that shape what the recogniser hears and what is passed on to the bot. Pick a provider, set its required key, and restart the gateway. Ready-made combinations are in Configuration recipes.
Variables for interruptions, noise floor and voice-activity detection are covered in Conversation behaviour.
Choosing
Set STT_PROVIDER to one of the values below.
STT_PROVIDER=sarvam| Provider (STT_PROVIDER) | Mode | Best for |
|---|---|---|
| sarvam | Streaming over a WebSocket | Indian languages; the default path for them |
| sarvam-r | Batch, with local voice detection | Indian languages without a persistent socket |
| Streaming | Enterprise and compliance needs, Google Cloud estates | |
| vexyl-stt | Streaming | Self-hosted recognition for Indian languages |
| deepgram | Batch, with local voice detection | Speed, English-first deployments |
| groq | Batch, with local voice detection | Accuracy on international callers |
| gemini | Batch, with local voice detection | Cost-effective, high volume |
| openai | Batch, with local voice detection | Quality transcription |
| auto | Chosen by the call's language | Letting the gateway decide per language |
Streaming providers decide on their own side when the caller has finished, and return text while the caller is still speaking, so replies start sooner. Batch providers wait: the gateway buffers audio until its own voice detection hears silence, then sends the whole utterance in one request. Expect batch providers to add a noticeable delay per turn compared with streaming ones.
With auto, the gateway picks the provider from the call's language: Indian languages (English India, Hindi, Malayalam, Tamil, Telugu, Kannada, Marathi, Gujarati, Bengali, Punjabi, Odia) go to Sarvam; other languages, and any language the gateway does not recognise, go to Groq. It does not check which keys are configured, so with auto set both SARVAM_API_KEY and GROQ_API_KEY; the gateway refuses to start under auto without SARVAM_API_KEY.
A single call can override the global choice by sending stt_provider in its session metadata, or an X-STT-Provider header on a SIP call (Per-call settings reference). An unrecognised value is ignored with a warning and the configured default is used.
Sarvam (streaming)
The gateway holds one WebSocket open to Sarvam for the life of the call.
| Variable | Default | Set it when |
|---|---|---|
SARVAM_API_KEY (required) |
unset | Always (also mandatory under auto) |
SARVAM_STT_NATIVE_8K |
true | Leave on: telephone audio is sent as it arrives, with no upsampling. Set false only to restore the older upsampled path |
SARVAM_STT_WS_MODEL |
empty, so Sarvam's current default | You want a fixed model. saaras:v3 pins it so a change on Sarvam's side cannot alter your results |
SARVAM_STT_WS_MODE |
transcribe | Only with saaras:v3. Choices: transcribe, verbatim, codemix, translit |
SARVAM_STT_VAD_SIGNALS |
true | Leave on. Sarvam decides when a turn ends, so its speech start and end markers are the only record of when the caller stopped. Without them the per-turn latency report shows 0 for the recognition step |
SARVAM_STT_FIRST_TURN_MIN_SPEECH_FRAMES |
2 | A very short first reply, such as a single "yes", is otherwise swallowed because Sarvam applies a stricter gate to the first utterance of a call. 0 restores Sarvam's own stricter gate |
STT_INGRESS_MAX_CHUNKS |
2 | How many 20 ms frames are pushed to Sarvam at once. Raise it only to trade delay for fewer sends |
STT_INGRESS_FLUSH_MS |
40 | A backstop timer that pushes any trailing frames. Rarely needs changing |
Do not enable Sarvam's high VAD sensitivity option in any custom integration: it chops real turns into fragments, and the gateway never sets it.
Sarvam supports Indian languages including English (India), Hindi, Malayalam, Tamil, Telugu, Kannada, Bengali, Marathi, Gujarati, Odia and Punjabi.
STT_PROVIDER=sarvam
SARVAM_API_KEY=<sarvam-key>
SARVAM_STT_WS_MODEL=saaras:v3Sarvam batch (sarvam-r)
Sends each utterance to Sarvam over HTTP after the gateway's own voice detection ends it.
| Variable | Default | Set it when |
|---|---|---|
SARVAM_API_KEY (required) |
unset | Always, for this provider |
SARVAM_STT_MODEL |
saarika:v2.5 | You want a different batch model. This is separate from SARVAM_STT_WS_MODEL, which only affects the streaming provider |
Google (streaming)
| Variable | Default | Set it when |
|---|---|---|
GOOGLE_APPLICATION_CREDENTIALS (required) |
unset | Always: path to the service-account key file |
GOOGLE_STT_MODEL |
phone_call | You want another model: latest_long or chirp_2 |
GOOGLE_STT_USE_ENHANCED |
true | Set false to turn off Google's telephony-enhanced models, which are billed at a higher rate |
GOOGLE_STT_REGION |
unset | Data residency needs a regional endpoint such as us or eu |
GOOGLE_STT_INTERIM_TIMEOUT |
1500 | Milliseconds of silence after which the last interim result counts as final. Lowering it cuts the wait but can truncate turns |
GOOGLE_STT_DEDUPE_MS |
2500 | Window in milliseconds in which a repeat of the same transcript is rejected |
The interim timeout sets how soon Google results can arrive, so it is the main reason Google feels slower than Sarvam on the same call.
Vexyl STT (self-hosted, streaming)
Connects to a recognition server you run yourself, so there is no per-minute API cost.
| Variable | Default | Set it when |
|---|---|---|
VEXYL_STT_URL (required) |
unset (the provider counts as configured only when you set it; ws://localhost:8091 is the fallback value) |
The server is not on the same machine as the gateway |
VEXYL_STT_API_KEY |
unset | Your server requires a key; it is sent as an X-API-Key header |
STT_PROVIDER=vexyl-stt
VEXYL_STT_URL=ws://<stt-host>:8091Deepgram (batch)
| Variable | Default | Set it when |
|---|---|---|
DEEPGRAM_API_KEY (required) |
unset | Always, for this provider |
DEEPGRAM_STT_MODEL |
nova-2 | You want another model, for example nova-3 |
DEEPGRAM_STT_PUNCTUATE |
on | Set false to receive text without punctuation |
DEEPGRAM_STT_SMART_FORMAT |
on | Set false to stop numbers, dates and currency being formatted |
Deepgram runs in batch mode here because its own endpointing behaved poorly on noisy phone lines.
Groq (batch)
| Variable | Default | Set it when |
|---|---|---|
GROQ_API_KEY (required) |
unset | Always, for this provider |
GROQ_MODEL |
whisper-large-v3-turbo | You prefer whisper-large-v3: slower, with the best accuracy |
Gemini (batch)
| Variable | Default | Set it when |
|---|---|---|
GEMINI_API_KEY (required) |
unset | Always, for this provider |
GEMINI_STT_MODEL |
gemini-2.5-flash-lite | You want a different Gemini model |
GEMINI_STT_PROMPT |
unset | You want to guide transcription, for example to list the technical terms callers use |
OpenAI (batch)
| Variable | Default | Set it when |
|---|---|---|
OPENAI_API_KEY (required) |
unset | Always, for this provider |
OPENAI_STT_MODEL |
whisper-1 | You want gpt-4o-transcribe or gpt-4o-mini-transcribe |
Short replies and quiet handsets
Some handsets send nothing at all while the caller is silent. The recogniser then never hears the pause that closes a turn, and a short reply stays unfinished until the caller says more. On the SIP path the gateway can fill those gaps with silence while it is listening, so turns close normally. The fill is skipped while the bot is speaking or the model is working.
| Variable | Default | Set it when |
|---|---|---|
STT_INBOUND_CONTINUOUS |
true | Set false to feed the recogniser only the packets that actually arrive |
STT_INBOUND_GAP_MS |
60 | Milliseconds without a real frame before filling starts. The default rides out normal jitter |
STT_INBOUND_TRANSPORTS |
sip | Which call paths get the fill: sip, audiosocket or all. AudioSocket already delivers continuous audio, so opt it in only if a log shows gaps |
Each transcript line in the log ends with a summary like inbound N frames / Tms = X% of continuous. Near 100% means audio arrived continuously. Far below 100% means a handset that suppresses silence, so keep the fill on. Above 100% is harmless: it means the trunk uses 10 ms packets.
Held transcripts
Words the caller says while the bot is thinking or talking are not thrown away. They are held and replayed when the turn ends. Short acknowledgements said while the bot was still thinking are stale and dropped; the bot's own echo is dropped too. The log shows "transcript held during llm" and "replaying held transcript".
| Variable | Default | Set it when |
|---|---|---|
STT_HOLD_DURING_LLM |
true | Set false to drop such speech silently, as older versions did |
STT_HOLD_MAX |
3 | Pieces kept per turn; older ones fall off first |
What never reaches the bot
Between recognition and the bot, a sanity gate filters out text that is not the caller. Hold music and a carrier's own announcements arrive as "speech", and recognisers sometimes loop on one word. Settings apply to every provider.
| Variable | Default | What it does |
|---|---|---|
STT_SANITY_GATE |
true | Master switch. False passes everything through |
STT_GATE_DROP_FILLER |
false | True drops turns made only of fillers such as "ആ". Off by default because a bare "ആ" is a valid yes |
STT_GATE_CARRIER_MARKERS |
built-in list | Comma-separated phrases that identify a hold or carrier announcement; replaces the built-in list |
STT_GATE_CARRIER_MIN_HITS |
2 | Markers that must appear before a turn is treated as carrier audio |
STT_GATE_MIN_RUN |
3 | Length of a repeated-word run before it is collapsed. Keep it at 3 or more so "twenty twenty six" survives |
STT_GATE_LOOP_MIN_TOKENS |
5 | Words needed before a single repeated word counts as a recogniser loop |
STT_GATE_DOMINANCE_RATIO |
0.6 | Share of one word, after collapsing, that marks a turn as junk |
STT_GATE_DOMINANCE_MIN_TOKENS |
8 | Words needed before the dominance check applies |
STT_GATE_MAX_CYCLE_LENGTH |
4 | Longest repeating block of words the gate detects |
If callers report that the bot ignores something they said, see Troubleshooting.