Conversation behaviour

This chapter describes how a call feels to the person on the line and which settings shape it: who speaks first, whether the caller can interrupt, how the gateway copes with echo and noise, what happens when the AI is slow, and how a call ends or moves to a person. Every behaviour below follows the same pattern: what the caller experiences, what the gateway does, the variables, and the log line that shows it at work.

The settings go in .env; restart the gateway after changing them. Recognition settings live in Speech recognition, voices in Text to speech, and the brain in AI brain.

Greeting

Inbound calls (pipeline mode). The bot speaks first. When the call is answered the gateway sends the brain a fixed opening prompt ("Hello , welcome to our service. How can I help you today?") together with the session metadata, and plays the brain's reply as the greeting. The wording therefore comes from your bot's prompt (LiteBot) or from your endpoint (custom). If the brain fails to answer, the gateway speaks a built-in fallback ("Hello! Welcome to our service. How can I help you?", with the caller's name when it is known). Metadata such as caller_name and language_code, posted before the call connects, shapes the greeting; see Asterisk over AudioSocket.

Inbound calls (realtime mode). Gemini Live and LiteBot Live greet through their own kick-off; see Realtime mode.

Outbound calls. The called party answers first with "hello", so you choose how the bot opens. Set the default with OUTBOUND_GREETING_MODE, or per call with greeting_mode in the session metadata (greetingMode in the /outbound/initiate body).

The _ML_IN suffix is the language code in upper case with the hyphen replaced by an underscore; use _HI_IN, _EN_IN and so on for other languages. All message files are WAV, 8 kHz, 16-bit PCM, mono. Outbound dialling itself is covered in HTTP API.

Interrupting the bot

A caller who talks over the bot expects it to stop. With barge-in on, playback stops as soon as the caller speaks. Because a speech detector cannot tell a person from hold music, the gateway then holds the part that was not yet played: if real words arrive, the held audio is discarded; if nothing qualifying arrives in time, the bot resumes where it stopped.

ENABLE_BARGE_IN=true
BARGE_IN_USE_VAD=true
BARGE_IN_RESUME=true

Log lines: BARGE-IN: hold armed, BARGE-IN: false interruption (followed by the resume), and BARGE-IN: confirmed by N word(s), discarding held audio.

Not hearing itself

Some handsets leak the bot's own voice back into the line. Without protection the bot would cut itself off, and a one-word echo transcript would count as the caller interrupting. The echo guard measures how loud the echo is while the bot speaks and only accepts an interruption that is clearly louder. After the bot stops, a transcript that repeats the bot's own words is dropped.

Log line: BARGE-IN: echo transcript ignored. If callers report that the bot talks over itself or stops mid-sentence for no reason, look for this line first.

Noisy lines

Street noise and a loud room can make a fixed volume threshold fire constantly. The gateway therefore learns each call's background level. For the first moments of a call (default 1 s) it only listens (no interruption can fire); afterwards it tracks the quiet level over a rolling window and sets the barge-in threshold a fixed margin above it, kept between a minimum and a maximum.

For recognisers that work on whole utterances (the batch providers in Speech recognition), a second gate throws away segments that are both short and quiet before they are sent for recognition, so clicks and coughs cost no recognition request.

Sarvam streaming recognition ends turns on the Sarvam side, so these two gates matter most with batch providers. The remaining detector settings are advanced tuning and are listed in the environment reference.

Speech while the bot is busy

If the caller says something while the AI is still thinking, or while the bot is speaking and they did not interrupt, the words are not lost. The gateway keeps them, then answers once the current turn has finished; a bare "okay" said while the bot was still thinking is stale and is dropped. The variables and log lines are in Held transcripts.

Hold music and carrier announcements

When the caller's phone plays hold music or a network announcement, the recogniser can mistake it for speech. A sanity gate between recognition and the AI drops carrier announcements, hallucinated loops and repeated-word runs before they reach the bot. See What never reaches the bot for the gate settings.

Thinking sounds and waiting messages

While the AI works the caller hears a short filler sound instead of silence, and it is cut as soon as the reply audio is ready. If the AI takes unusually long, a spoken waiting message plays; if it fails outright, a final apology is spoken. The gateway ships several filler files in sounds/, chosen with PROCESSING_SOUND_TYPE (default chime); PROCESSING_SOUND_DIR points to your own set and PROCESSING_SOUND_FILE to a single file. PROCESSING_SOUND_HARD_CUT (default true) stops the filler the moment speech is ready, and TTS_TAIL_DELAY_MS (default 40) is the pause after playback before listening resumes. The waiting messages are LLM_TIMEOUT_MESSAGE and LLM_FINAL_TIMEOUT_MESSAGE, with per-language variants, switched by TIMEOUT_MESSAGE_ENABLED. All of these are in When the AI is slow or fails.

Incomplete sentences

Callers pause mid-sentence ("I would like to book... um"). With incomplete-turn detection on, the brain can answer with a flag saying the sentence is unfinished (shouldWaitForMore) or that the caller is thinking (shouldGiveTime). The gateway then stays silent and waits for the next transcript, which is joined to the first. Only brains that return these flags use it: your custom endpoint, and LiteBot where the bot is set up for it. See Flags.

Transfer to a person

The bot hands the call to a human when the brain sets its escalate flag, or when a tool you listed in LITEBOT_TRANSFER_TOOLS runs. How the call actually moves depends on how it arrived:

• SIP-bridge calls are transferred with a SIP REFER to TRANSFER_SIP_URI; see SIP trunk.
• AudioSocket calls call your TRANSFER_ENDPOINT, an HTTP webhook that moves the Asterisk channel. It needs the channel metadata; see Transfers.
• Realtime calls are covered in Realtime mode.

TRANSFER_ENABLED=true
TRANSFER_SIP_URI=sip:7000@pbx.example.com
TRANSFER_MESSAGE_TEXT_EN=Please hold while I transfer you to a human agent.

The language suffix is the first segment of the language code in upper case (EN, ML, HI); the same suffix works on the TRANSFER_MESSAGE_TEXT_ and TRANSFER_ERROR_TEXT_ variables. After an error message the call ends. Log line for a SIP transfer: SIP REFER transfer initiated.

Ending the call

When the brain asks to hang up (shouldHangup, or a LiteBot tool named in LITEBOT_HANGUP_TOOLS, default hangupCall), the gateway first finishes speaking the reply, so the caller hears the goodbye in full, and only then ends the call. If the brain gave no closing words, it plays the goodbye message: the GOODBYE_MESSAGE WAV (a per-language GOODBYE_MESSAGE_ML overrides it) or, when no file exists, the spoken text GOODBYE_MESSAGE_TEXT_EN or its language variant.

When the caller hangs up first, the session ends immediately and the end-of-call webhook fires; see Webhooks and billing.