AI brain

The AI brain is the language model side of a call: it receives what the caller said and returns the reply the gateway speaks. The gateway supports two brains, LiteBot and your own HTTP endpoint. This chapter covers how to choose between them, every variable each one reads, and what callers hear when the brain is slow or fails. Speech in and out is covered in Speech recognition and Text to speech; ready-made combinations are in Configuration recipes.

Choosing

Set LLM_PROVIDER to one of the two values below. Always set it explicitly; leaving it unset is not a supported configuration.

LLM_PROVIDER=litebot

Realtime LiteBot Live is a different mode that replaces the whole speech pipeline; see Realtime mode.

LiteBot

Point the gateway at your LiteBot server and choose a bot.

LLM_PROVIDER=litebot
LITEBOT_API_URL=http://<litebot-host>:3345
LITEBOT_BOT_ID=customer-service

Choosing a bot per call

Post a botId in the session metadata for a call and that bot answers instead of LITEBOT_BOT_ID. The id is case-sensitive; an unknown id falls back to the default bot. The metadata request is described in HTTP API.

What the gateway sends

Each turn the gateway sends LiteBot the following.

Streaming

With LITEBOT_STREAMING on, the first sentence is spoken while the rest is still being written. Bots that call tools deliver their final answer as one block even with streaming on; that is LiteBot's behaviour, not a gateway setting.

Hang-up, transfer and cost

• A tool named in LITEBOT_HANGUP_TOOLS ends the call after the goodbye is spoken. Add your own tool names, comma-separated.
• A tool named in LITEBOT_TRANSFER_TOOLS hands the caller to a human. Transfer set-up and what happens when it fails are in Conversation behaviour.
• Per-turn cost can miss turns that finish after the call ends. With LITEBOT_RECONCILE_COST on, the gateway reads LiteBot's own total at hang-up and only ever raises the recorded cost to match. Where costs go is covered in Webhooks and billing.
• With LITEBOT_SESSION_END on, the bot's end-of-session hooks (a CRM log, for example) run at hang-up instead of at LiteBot's idle sweep. The call is fire-and-forget and never delays hang-up. An older LiteBot without that route returns 404, which is logged once and is harmless.

When LiteBot relays an error

If LiteBot's own model fails (usually an invalid or expired key on the bot), LiteBot may return the failure as the reply text, starting with Error:. With LITEBOT_ERROR_TEXT_GUARD on, the gateway does not speak it. The caller hears the final-timeout message from When the AI is slow or fails, and the log records that LiteBot returned an upstream error as the reply. Fix the cause on the LiteBot side. LITEBOT_ERROR_TEXT_PATTERN replaces the default match, a reply beginning with error: in any case.

Custom HTTP endpoint

The gateway POSTs each turn to your URL and speaks the text in the JSON you return.

LLM_PROVIDER=custom
CUSTOM_LLM_URL=https://your-server.example.com/chat
CUSTOM_LLM_AUTH_TYPE=bearer
CUSTOM_LLM_AUTH_TOKEN=your-secret-token

Request

Your endpoint receives a POST with this JSON body.

{
  "message": "Hello, I need help with my order",
  "sessionId": "custom-session-1706234567890-abc123xyz",
  "history": [
    { "role": "user", "content": "Hi" },
    { "role": "assistant", "content": "Hello! How can I help you today?" }
  ],
  "context": {
    "userId": "550e8400-e29b-41d4-a716-446655440000",
    "callerName": "John Doe",
    "callerPhone": "+9198xxxxxxxx",
    "language": "en-IN",
    "timestamp": "2024-01-25T10:30:00.000Z",
    "custom_field_1": "value1",
    "custom_field_2": "value2"
  }
}

Response

Return JSON. The recommended shape:

{
  "response": "I'd be happy to help you with your order. Can you please provide your order number?",
  "sessionId": "your-session-id-123",
  "shouldEscalate": false,
  "shouldHangup": false
}

The gateway also finds the reply text under any of these field names.

{ "response": "..." }
{ "text": "..." }
{ "message": "..." }
{ "output": "..." }
{ "result": "..." }
{ "answer": "..." }
{ "reply": "..." }
{ "content": "..." }

Nested formats also work:
{ "data": { "response": "..." } }
{ "data": { "text": "..." } }
{ "data": { "message": "..." } }

If your JSON uses another shape, set CUSTOM_LLM_RESPONSE_FIELD.

Flags

All four flags are optional booleans. They are also accepted nested under metadata or data.

shouldWaitForMore and shouldGiveTime only take effect when ENABLE_INCOMPLETE_TURN_DETECTION=true.

To transfer, return:

{
  "response": "I'll connect you with a customer service representative right away.",
  "shouldEscalate": true
}

To end the call, return:

{
  "response": "Thank you for calling! Have a great day. Goodbye!",
  "shouldHangup": true
}

A transfer that fails or is not configured plays a message and ends the call; see Conversation behaviour.

When the AI is slow or fails

Two messages cover a struggling brain. They play only when the brain is slow or fails.

Only the _ML variants exist, and they apply only to Malayalam (ml-IN) calls. Every other language uses the base message.

• Slow-reply default: "I'm experiencing some technical difficulties. Please hold on while I try to process your request."
• Hard-failure default: "I'm sorry, I'm having trouble processing your request right now. Please try again in a moment."

The hard-failure message is spoken on a request error, a final timeout, or an error text relayed by LiteBot.

Processing sound

A short filler sound can play while the brain thinks, so the line is not silent.