Realtime mode

Realtime mode hands the whole conversation to a single speech-to-speech model instead of running the speech pipeline of recognition, AI brain and synthesis. This chapter explains how to choose between the two modes, how to set up the two realtime providers (Gemini Live and LiteBot Live), how hang-up and transfer behave, and the separate Gateway Mode for OpenAI-compatible realtime services.

Pipeline or realtime

One gateway can serve both modes at the same time. The mode is decided per call.

In realtime calls no speech recognition or text-to-speech provider variable applies.

How the mode is chosen

The first matching rule wins, from most specific to least.

Metadata is posted per call, see HTTP API. The bot configuration is only consulted when the metadata does not name a provider.

REALTIME_PROVIDER=

Leave it empty for pipeline by default. Set it to gemini-live or litebot to make realtime the default for every call.

Gemini Live

Gemini Live runs the bot inside the gateway. The bot's prompt, tools, model and voice come from files you control.

REALTIME_PROVIDER=gemini-live
GEMINI_LIVE_API_KEY=<your-key>

The price variables override the built-in rates used for call cost; see Webhooks and billing. The bundled default bot uses the model gemini-3.1-flash-live-preview; other Gemini Live models can be set per bot.

Bots, prompts and tools

Each bot is a JSON file that points at a prompt and lists the tools it may use. Your own files go under config/ next to the gateway. Bundled bots, prompts and tools ship inside the product and are loaded first.

BOT_REGISTRY_EXTERNAL_DIR=./config
BOT_REGISTRY_HOT_RELOAD=true
DEFAULT_BOT_ID=default

With hot reload on, edits under config/ are picked up without a restart. A bot file looks like this.

{
  "id": "support",
  "promptFile": "support.txt",
  "realtime": {
    "provider": "gemini-live",
    "model": "gemini-3.1-flash-live-preview",
    "voice": "Aoede",
    "language": "en-US"
  },
  "tools": ["transferCall", "hangup"]
}

A tool module exports a name, a description, a parameters schema in JSON Schema form, and an asynchronous function that receives the parameters and the call context and returns the result. The bundled tools a bot can list are hangup and transferCall; add your own modules under config/tools/*.js and list their names.

Choosing a bot per call

Post a botId in the session metadata for the call. The id is case-sensitive; an unknown id falls back to the bot named by DEFAULT_BOT_ID.

curl -X POST http://127.0.0.1:8081/session/<call-uuid>/metadata \
  -H "Content-Type: application/json" \
  -d '{"botId": "support"}'

LiteBot Live

LiteBot Live makes the gateway a thin audio bridge to LiteBot's own voice endpoint. LiteBot hosts the model, the bot, the prompt, the tools, the voice, the language and the transcripts. The local bots, prompts and tools folders are ignored in this mode.

REALTIME_PROVIDER=litebot
LITEBOT_LIVE_URL=wss://<litebot-host>:<port>
LITEBOT_LIVE_API_KEY=<server-key>
LITEBOT_LIVE_BOT_ID=<bot-id>

Leave LITEBOT_LIVE_CONTINUOUS_STREAM on. Some handsets stop sending audio during silence, and without a steady stream the model never decides the caller's turn has ended.

The gateway passes session details to LiteBot as variables so the bot can use them: caller name, caller number, channel and call direction, plus anything else you posted in the session metadata.

Hang-up and transfer in realtime mode

Both providers handle the end of a call the same way. When the bot decides to hang up, the gateway lets the goodbye finish playing and then closes the call. The wait is bounded by the drain ceiling (GEMINI_LIVE_DRAIN_MS or LITEBOT_LIVE_DRAIN_MS), and it ends early if the queued audio stops shrinking for the stall time. A hard safety net closes the call shortly after the ceiling in case anything is stuck.

Where the call is sent is configured as for pipeline calls: TRANSFER_SIP_URI in SIP trunk for SIP-bridge calls, TRANSFER_ENDPOINT in Asterisk and AudioSocket for AudioSocket calls, and the caller-facing behaviour in Conversation behaviour.

Gateway Mode

Gateway Mode is a separate alternative to REALTIME_PROVIDER. It bridges an AudioSocket call straight to any WebSocket service that speaks the OpenAI Realtime protocol. There are no bots, tools or transfers in this mode; the remote service does everything.