Voice

This chapter covers the two ways a bot talks by voice. In the first, a client streams audio to LiteBot's /voice WebSocket and LiteBot runs the call as a Gemini Live session, executing the bot's tools itself. In the second, the VEXYL Voice Gateway handles the phone call and uses LiteBot as its brain, either turn by turn over the chat API or as a realtime /voice client. Read it when you connect a phone gateway, a mobile app or a browser to a bot that speaks.

Two ways to run a voice bot

The pipeline gives you any LLM provider, any gateway voice, and the hooks. Realtime gives the most natural turn-taking and barge-in, at Gemini Live prices.

Enable voice on a bot

Set the voice block on the bot (dashboard: the bot's Voice tab). Every field is in Bot configuration reference; the ones you touch most:

Thinking is off for voice by default (budget 0), because it adds first-word delay and some native-audio models speak their reasoning. VOICE_THINKING_BUDGET changes it for every voice bot: -1 for the model's own default, or a token count. The bot field voice.thinkingBudget is read by the voice handler but the bot schema rejects it, so use the variable.

/voice uses the Gemini API key (GEMINI_API_KEY, or the bot tenant's own Gemini key), not Vertex AI. At startup LiteBot logs an error if a bot has voice.enabled: true and GEMINI_API_KEY is empty.

Connect to /voice

ws://127.0.0.1:3345/voice?token=<token>
ws://127.0.0.1:3345/voice?api_key=lbk_...

LiteBot checks the credential before it accepts the upgrade. Browsers cannot set headers on a WebSocket, so both go on the query string.

If both are present, token is used. A missing or bad credential gets HTTP 401 Unauthorized and the socket never opens; an error while checking the key gets 500; any path other than /chat or /voice gets 404. Failed API key checks count toward the same per-IP lockout as the X-API-Key header. The credential ends up in proxy access logs, so use wss:// and treat logs accordingly.

Session flow

Every frame is one JSON object; audio travels base64-encoded inside it. A connection carries one voice session.

server  connection_established
client  init_session
server  session_ready            -> client may send audio_chunk / text_input
...     transcripts, audio, tool frames, turn_complete per turn
server  session_end              -> on hangup, silence or time limit

Client to server

init_session fields:

{
  "type": "init_session",
  "botId": "clinic-reception",
  "userId": "550e8400-e29b-41d4-a716-446655440000",
  "mode": "audio",
  "variables": { "callerName": "Priya" },
  "context": { "callerNumber": "+919876543210" },
  "llm_context": { "preferredLanguage": "en-IN" }
}

Keys and values in llm_context are cleaned of newlines and control characters before they reach the prompt, so a value cannot inject new instructions. update_context with llm_context adds the facts to the live conversation as a silent turn; the reply is context_updated with the stored keys and injected: true when the model received them.

Server to client

Keep the last usage_summary you receive: a caller hangup can close the socket before the final one. Strip cost before showing the frame to end users.

Audio format

LiteBot does not transcode. A telephony client decodes µ-law or A-law, resamples 8 kHz to the input rate, and resamples 24 kHz back to 8 kHz for the phone leg. Chunks of 20 to 250 ms work. Inbound audio is capped at VOICE_AUDIO_BYTES_PER_SEC (256 KB per second; a chunk over it is dropped with an error) and VOICE_AUDIO_TOTAL_BYTES_CAP (100 MB per session; over it the session ends). One frame may be up to 4 MB.

Tools, hangup and transfer

The bot's tools are offered to Gemini Live with the same schemas as in chat. When the model calls one, LiteBot runs it through the tool registry and returns the result. A voice tool call times out after 30 seconds; the model then gets { "error": "timeout" }. A parameter named access_token is hidden from the model and filled from the last token any tool returned in that session, because models mistype long tokens.

LiteBot reads two intents from a tool's result:

On hangup, stop the microphone, let the goodbye audio play, then close the socket. LiteBot does not close it for you on this path.

Limits and session end

A session ends in one of these ways:

Treat a plain close as the end of the session. Every path saves pending transcripts, marks the conversation ended, writes the usage to analytics and closes the Gemini session. Conversations from /voice are stored with metadata.transport: "voice" and appear with chat sessions in the dashboard (see Conversations and analytics).

Errors

GET /api/voice/stats (login required) shows active voice sessions.

With the VEXYL Voice Gateway

The gateway answers the phone call and can use LiteBot in either mode. The gateway side is in the VEXYL Voice Gateway manual, chapters AI brain and Realtime mode.

Pipeline. The gateway turns speech into text and calls POST /api/chat for each turn, with "stream": true by default (its LITEBOT_STREAMING=false turns that off) so it can speak the first sentence early. It sends the bot from its call settings (botId, else its LITEBOT_BOT_ID), the caller id as userId, and caller details in context. LiteBot needs nothing special. The gateway:

• speaks response, and ends the call after speaking when shouldHangup is true (or when a tool named in its LITEBOT_HANGUP_TOOLS, default hangupCall, succeeded);
• stays silent on shouldWaitForMore and shouldGiveTime, and skips its wait sound on skipProcessingSound;
• calls DELETE /api/chat/session/:id?reason=remote_hangup when the caller hangs up, or reason=transfer on a transfer, which runs the bot's onSessionEnd hooks;
• relies on STREAM_HOLD_TOOL_TURNS (default on) so it never speaks text the bot wrote next to a tool call.

LiteBot Live. With the call's realtime provider set to litebot, the gateway opens /voice itself, using its LITEBOT_LIVE_URL and an lbk_ key or token. Its init_session carries its call UUID as userId, callerName, callerNumber and language as variables, the other call settings plus callerId, channel and callDirection as context, and the call's llm_context. Give the gateway its own API key, and make sure the per-key session limit (50 by default) and MAX_VOICE_SESSIONS_CONCURRENT (10 by default) cover the gateway's realtime call capacity.

Dashboard test. Set VEXYL_GATEWAY_WS_URL (a wss:// or ws:// address of a gateway whose LiteBot setting points at this LiteBot), and optionally VEXYL_GATEWAY_API_KEY and VEXYL_GATEWAY_TTS_PROVIDER, then restart. The chat screen then offers Voice: Gateway, which runs the full phone pipeline from the browser. GET /api/voice-gateway/config (login required) returns enabled, serverUrl, apiKey and ttsProvider. The key is visible to any logged-in user; the gateway's origin allowlist is the real control.

Known gaps

• /voice sessions do not run onSessionEnd hooks. Lead capture that must run at the end of a realtime call has to happen in a tool during the call.
• Setting VOICE_SILENCE_TIMEOUT_MS, MAX_VOICE_SESSION_DURATION_MS or VOICE_WS_PING_INTERVAL_MS to 0 does not disable them; the default applies. Use a large value instead.
• voice.thinkingBudget is not accepted in bot config; use VOICE_THINKING_BUDGET.

Where to go next

• Every voice.* field: Bot configuration reference
• Text chat, streaming and ending sessions: Chat API
• Outbound calling campaigns through the gateway: Auto-dialer
• API keys and their limits: Users, roles and tenants