LetsDial ExternalMedia WebSocket API

The ExternalMedia WebSocket API provides real-time call audio, audio playback, DTMF events, and playback controls over a persistent WebSocket connection.

All protocol messages are JSON. Audio is transported as base64-encoded data inside JSON messages.

Supported codecs

A stream uses one codec for its entire lifetime.

If no codec is selected, ulaw is used.

The exact packet time and optimal frame size are reported in the start event. Clients should use those negotiated values instead of assuming the typical values above.

Opus is not supported.

Connection lifecycle

After connecting:

1. LetsDial initializes the media stream.
2. A start event is sent when the stream is ready.
3. Media and call-control messages can flow in either direction.
4. A stop event is sent when the stream closes normally.

Do not send playback audio until the start event has been received.

Every event sent by LetsDial contains:

{
  "event": "eventName",
  "sequenceNumber": 0,
  "streamId": 123
}

Events sent by LetsDial

start

Indicates that the stream is ready.

{
  "event": "start",
  "sequenceNumber": 0,
  "streamId": 123,
  "callId": 456,
  "organizationId": 12,
  "type": "HANDOVER",
  "direction": "BOTH",
  "dialledNumber": "27111234567",
  "clid": "27119876543",
  "codec": "slin16",
  "packetTime": 20,
  "optimalFrameSize": 640
}

The decoded audio format of both incoming media events and outgoing playbackStream actions is determined by codec.

media

Contains audio received from the call.

{
  "event": "media",
  "sequenceNumber": 1,
  "streamId": 123,
  "media": {
    "payload": "<base64 raw audio>"
  }
}

media.payload is raw audio without a WAV header.

dtmf

Indicates that a complete DTMF digit was received.

{
  "event": "dtmf",
  "sequenceNumber": 2,
  "streamId": 123,
  "dtmf": {
    "digit": "5"
  }
}

Supported digits are:

0–9, A–D, *, #

DTMF events are exposed on HANDOVER streams.

playbackComplete

Indicates that audio submitted using playback has completed.

{
  "event": "playbackComplete",
  "sequenceNumber": 3,
  "streamId": 123
}

This event applies to complete-file playback requests. Individual playbackStream messages do not receive completion events.

A clear action or a newer playback action can cancel pending playback without generating playbackComplete.

playbackStreamBackpressure

Indicates whether the client should pause or resume sending live playback audio.

{
  "event": "playbackStreamBackpressure",
  "sequenceNumber": 4,
  "streamId": 123,
  "state": "paused",
  "queuedMilliseconds": 420
}

State behavior:

An overflow occurs when the locally queued audio exceeds approximately 10 seconds.

The byte limit is codec-aware and calculated from packetTime and optimalFrameSize.

playbackStatus

Returned in response to getPlaybackStatus.

{
  "event": "playbackStatus",
  "sequenceNumber": 5,
  "streamId": 123,
  "queueLength": 3200,
  "xonLevel": 1600,
  "xoffLevel": 6400,
  "isQueueFull": false,
  "isBuffering": true,
  "isPaused": false
}

The response is asynchronous. Clients should not assume that it immediately follows the request if other events are also being processed.

playbackMarkProcessed

Indicates that a playback mark has reached its position in the playback queue.

{
  "event": "playbackMarkProcessed",
  "sequenceNumber": 6,
  "streamId": 123,
  "markId": "sentence-42"
}

markId matches the identifier supplied with markPlayback.

Marks are ordered relative to playback audio. This allows a client to determine when a particular point in a complete file or streaming playback has been reached.

stop

Indicates that the stream is closing.

{
  "event": "stop",
  "sequenceNumber": 7,
  "streamId": 123
}

Clients should stop sending messages and release resources associated with the stream.

A WebSocket closure should also be treated as the end of the stream, even if stop was not received.

Actions sent to LetsDial

Messages sent to LetsDial must be JSON objects containing an action field.

Malformed JSON, unknown actions, and invalid action fields are ignored.

continue

End the media handover and allow the call to continue through its configured flow.

{
  "action": "continue"
}

hangup

End the associated call.

{
  "action": "hangup"
}

stopListening

Stop the current listening stream without ending the associated call.

{
  "action": "stopListening"
}

This action is primarily intended for SPY streams.

mohStart

Start music on hold.

{
  "action": "mohStart",
  "class": "default"
}

If class is omitted or invalid, the default class is used.

mohStop

Stop music on hold.

{
  "action": "mohStop"
}

addCallContext

Attach customer-defined context to the current call-flow interaction.

{
  "action": "addCallContext",
  "context": {
    "result": "verified",
    "reference": "abc-123",
    "score": 0.95
  }
}

context may contain any valid JSON value. An object is recommended when submitting multiple fields.

playback

Play a complete WAV file into the call.

{
  "action": "playback",
  "media": {
    "payload": "<base64 WAV file>"
  }
}

Requirements and behavior:

• media.payload must contain a complete base64-encoded WAV file.
• ulaw output is converted to 8 kHz G.711 μ-law.
• alaw output is converted to 8 kHz G.711 A-law.
• slin16 output is converted to 16 kHz signed 16-bit little-endian PCM.
• PCM, μ-law, A-law, and IMA ADPCM WAV inputs are supported by the conversion path.
• Starting playback clears previous playback and queued streaming audio.
• playbackComplete is emitted once the submitted playback completes.
• Playback respects pause/resume flow control.
• A newer playback, playbackStream, or clear action can cancel it.

This action is available only for HANDOVER streams.

playbackStream

Send real-time audio for playback into the call.

{
  "action": "playbackStream",
  "media": {
    "payload": "<base64 raw audio>"
  }
}

Unlike playback, this payload is not transcoded. It must already be encoded using the codec reported by the start event.

Do not include a WAV header.

For reliable streaming:

• Send small, regular audio chunks.
• Use packetTime and optimalFrameSize to determine the preferred chunk size.
• Stop sending after receiving backpressure state paused.
• Resume only after receiving resumed.
• Restart or abandon the current stream after receiving overflow.

Small chunks may be accumulated for up to approximately 250 milliseconds before playback begins. The corresponding byte threshold is calculated from the negotiated format.

Examples for a 20 ms packet time:

This action is available only for HANDOVER streams.

clear

Cancel playback and discard queued playback audio.

{
  "action": "clear"
}

This clears:

• Complete-file playback
• Streaming playback
• Queued audio
• Pending playback marks
• Pending playback completion

No acknowledgement event is sent.

This action is available only for HANDOVER streams.

pausePlayback

Pause outgoing playback.

{
  "action": "pausePlayback"
}

This pauses playback audio but does not pause incoming media events or the call itself.

No direct acknowledgement is sent. Use getPlaybackStatus to confirm the current state.

This action is available only for HANDOVER streams.

resumePlayback

Resume playback after pausePlayback.

{
  "action": "resumePlayback"
}

Queued playback continues from its paused position.

No direct acknowledgement is sent. Use getPlaybackStatus to confirm the current state.

This action is available only for HANDOVER streams.

getPlaybackStatus

Request the current playback queue and pause state.

{
  "action": "getPlaybackStatus"
}

LetsDial responds asynchronously with a playbackStatus event.

markPlayback

Insert a named marker into the playback timeline.

{
  "action": "markPlayback",
  "markId": "sentence-42"
}

Behavior:

• markId must be a non-empty string.
• Keep identifiers short; marks exceeding the control-message size limit are ignored.
• During playback, the mark is placed after audio uploaded before the mark request.
• During playbackStream, the mark maintains its order relative to queued audio chunks.
• If no playback is active, the mark is submitted immediately.
• LetsDial sends playbackMarkProcessed when the mark reaches the playback position.

This action is available only for HANDOVER streams.

Stream modes

Playback-related actions sent to a SPY stream are ignored, except getPlaybackStatus, which may still request the current status.

Recommended client behavior

• Wait for start before sending playback.
• Read codec, packetTime, and optimalFrameSize from every new stream.
• Do not assume all streams use the same codec.
• Decode incoming media according to the reported codec.
• Encode playbackStream using exactly the reported codec.
• Process events in sequenceNumber order.
• Keep streaming chunks small and regular.
• Immediately observe playbackStreamBackpressure events.
• Use marks when application behavior depends on precise playback progress.
• Use getPlaybackStatus after pause/resume when confirmation is required.
• Treat WebSocket closure as final even if stop was not received.
• Implement connection timeout and retry handling at the application level.
• Validate base64 audio before sending it.
• Avoid logging raw audio or sensitive call context unless required by your retention policy.