Stream Call Audio in Real-Time

The Sinch Voice API v2 can stream live PCM audio from a phone call bidirectionally to a WebSocket server. This unlocks feeding raw audio into AI models, speech-to-text engines, sentiment analysis pipelines, or custom voice bots, and it lets you send synthesized audio back to the caller over the same socket.

When a call leg is routed to a STREAM destination, Sinch opens a WebSocket connection to your endpoint and exchanges audio as binary frames. To get a working echo loop you only need three things running, in this order:

1. A WebSocket echo server (reads binary PCM, writes it straight back).
2. ngrok exposing that server as a public wss:// URL.
3. An API call whose SVAML dials a STREAM leg pointed at that URL, bridged with a phone leg.

The rest of this document is reference: the PCM/streamOptions contract, the inbound (ICE webhook) path, the on-the-wire message protocol, and the other language servers.

When to use STREAM vs. Voice Relay. Use STREAM when you want the raw audio: you run your own STT/TTS, or you forward the PCM to a third-party agent. If you'd rather have Sinch do the speech-to-text and text-to-speech for you and just exchange text turns, use VOICE_RELAY instead. See Connect an AI Chatbot (Voice Relay). If you want a fully external agent SDK (e.g. ElevenLabs Conversational AI) on the far end of the stream, see ElevenLabs Bridge. For a deeper walkthrough of the server side itself, see WebSocket Server.

All scripts for this tutorial live in the sinch/sinch-voice-tutorials repository. Clone it and move into this tutorial's folder:

git clone https://github.com/sinch/sinch-voice-tutorials.git
cd sinch-voice-tutorials/4.2-stream-audio

The layout you'll be working with:

sinch-voice-tutorials/
├── .env                       # shared credentials (repository / tutorials root)
├── 4.2-stream-audio/
│   ├── description.md          # this document
│   └── scripts/
│       ├── ws-server.py
│       ├── ws-server.node.js
│       ├── ws-server.php
│       ├── WsServer.java
│       ├── ice-callback.py
│       ├── ice-callback.node.js
│       ├── trigger-call.sh
│       └── trigger-call.js
└── ...                         # sibling tutorials (4.1, 4.3, 4.4)

Every command below assumes your working directory is sinch-voice-tutorials/4.2-stream-audio, so the scripts/... paths resolve. The scripts read the shared .env two levels up at the repository root (../../.env).

All scripts read the shared .env at the repository root (sinch-voice-tutorials/.env). Auth is HTTP Basic (KEY_ID:KEY_SECRET). Create the file at the repo root (copy .env.example if the repo ships one) and fill in:

PROJECT_ID=...            # Sinch project
KEY_ID=...                # API key id  (Basic auth username)
KEY_SECRET=...            # API key secret (Basic auth password)
SINCH_NUMBER=+1...        # your Sinch virtual number (E.164)
DESTINATION_NUMBER=+1...  # the phone that will ring (E.164)
WS_ENDPOINT=wss://...     # public WebSocket URL from ngrok (see below); must be wss:// or ws://
PORT=...                  # port for the ICE webhook server (inbound path only; optional)

WS_ENDPOINT must be a WebSocket URL (wss:// recommended, ws:// allowed) and reachable from the public internet. It is not an https:// URL. Convert your ngrok forwarding URL (see step 2 below).

ice-callback.{node.js,py} is only needed for the inbound path: when a caller dials your Sinch number and you want Sinch to route that call into your WebSocket server. It is an HTTP webhook server that handles the call.incoming event and replies with SVAML that bridges the inbound PSTN leg to a STREAM leg.

For the first-success path below you do NOT need ice-callback. The outbound trigger (trigger-call.{sh,js}) embeds the SVAML inline and calls the API directly. Skip ahead to "Inbound calls" only when you want callers to reach your stream.

• A Sinch account with API credentials and a Sinch virtual number.
• The sinch/sinch-voice-tutorials repository cloned (see "Get the code" above).
• ngrok (or any public tunnel) for development.
• One server runtime: Node.js 18+ (ws), Python 3.8+ (websockets), PHP 8+ (Ratchet), or Java 11+ (Tyrus).

This is the shortest loop that proves the stream works end to end. The Python server below echoes every binary PCM frame straight back, so when the call connects you should hear yourself.

pip install websockets
python scripts/ws-server.py

It listens on port 8765 (override with WS_PORT). The Python server in this tutorial already echoes audio back (await websocket.send(message)), so it doubles as an echo demo. The Node, PHP, and Java servers in scripts/ log/record audio but do not echo by default. See "Sending audio back" for the one-line change.

ngrok http 8765

Copy the Forwarding URL and convert the scheme to a WebSocket URL:

• https://abc123.ngrok-free.app becomes wss://abc123.ngrok-free.app

Set it in .env (or export it):

export WS_ENDPOINT=wss://abc123.ngrok-free.app

bash scripts/trigger-call.sh

This dials DESTINATION_NUMBER from SINCH_NUMBER; when the phone answers it bridges the phone leg with a STREAM leg pointed at WS_ENDPOINT. Answer the phone and speak, and you should hear your own voice echoed back, while the server console prints frame/byte counts.

Order matters. The server and ngrok must be running before you trigger the call. If WS_ENDPOINT points at a dead tunnel, the STREAM leg fails to connect and onHangup tears the call down, so you'll hear nothing.

(There is also scripts/trigger-call.js, the same payload in browser/Node fetch form. Note: calling the Sinch API directly from a browser hits CORS, so proxy it through a backend in production.)

• Server console: New WebSocket connection ..., then ConnectRequest — callId: ... appId: ..., then Sent ConnectResponse: answer, then periodic Audio frames: 100 | Bytes: ....
• Your phone: you hear your own voice echoed back (Python server) or silence (the other servers, which only record).
• API call: trigger-call.sh prints Call created successfully (HTTP 201).
• On disk: a call-<timestamp>.pcm file grows. Play it back to confirm encoding (see "PCM format" below).

The outbound trigger sends this shape. Two bridgeCall commands (same bridgeName) connect the phone leg and the stream leg so audio flows both ways. This mirrors the spec's own dialToStream / echo examples.

{
  "commands": [
    {
      "command": "dial",
      "callName": "phone-leg",
      "from": { "type": "PHONE", "phone": { "number": "+1SINCH_NUMBER" } },
      "to":   { "type": "PHONE", "phone": { "number": "+1DESTINATION" } },
      "dialTimeoutDurationSeconds": 30,
      "maxCallDurationSeconds": 1800,
      "events": {
        "onAnswer": [
          { "command": "bridgeCall", "bridgeName": "stream-bridge" },
          {
            "command": "dial",
            "callName": "stream-leg",
            "to": {
              "type": "STREAM",
              "stream": {
                "endpoint": "wss://abc123.ngrok-free.app",
                "streamOptions": {
                  "version":    1,
                  "codec":      "PCM",
                  "sampleRate": 8000
                },
                "callHeaders": [
                  { "key": "X-Tutorial", "value": "sinch-ws-agent" }
                ]
              }
            },
            "dialTimeoutDurationSeconds": 10,
            "events": {
              "onAnswer": [{ "command": "bridgeCall", "bridgeName": "stream-bridge" }],
              "onHangup": [{ "command": "hangup", "callName": "phone-leg" }]
            }
          }
        ],
        "onHangup": [{ "command": "hangup", "callName": "stream-leg" }]
      }
    }
  ]
}

Use callHeaders (max 16 pairs, key/value up to 255 chars each) to pass identifying metadata your server can read on connect.

Heads up: not in the OpenAPI spec. The spec defines the SVAML shape (endpoint, streamOptions, callHeaders) but does not document the messages exchanged over the WebSocket itself (ConnectRequest, the answer reply, heartbeat/StreamControl). The protocol below reflects the Streams product behavior the example servers implement; treat the exact JSON field names as unverified against the spec and confirm them against the current Sinch Streams documentation before relying on them in production.

The example servers all follow the same lifecycle: on the text connect message they record callId/applicationId/headers, reply {"command":"answer"}, then treat every binary frame as PCM audio and (optionally) write PCM back.

• Codec: PCM only (the spec fixes codec to the constant PCM). If you need Opus or G.711-µ, transcode to PCM on your own gateway before the WebSocket hop.
• Sample rate: one of 8000 | 16000 | 24000 | 44100 | 48000 | 96000 Hz; default 8000. For PSTN calls audio is typically sampled at 8 kHz, so a higher rate does not improve perceived quality on a PSTN-only path; it just increases bandwidth and processing load. Higher rates help on non-PSTN (SIP/streaming) paths or when a downstream STT model prefers wideband.
• Sample encoding / framing: The spec says "uncompressed raw audio" but does not state the bit depth, endianness, or channel count of each frame. The example servers treat frames as opaque bytes (write to file, echo back), which works regardless. Before decoding the PCM yourself, confirm the format. It is almost certainly 16-bit signed little-endian, mono, but this is not stated in the OpenAPI spec. To verify, play a recorded call-*.pcm with the assumed parameters, e.g.:
  ffplay -f s16le -ar 8000 -ac 1 call-<timestamp>.pcm

  Adjust -ar to match the sampleRate you negotiated. If it sounds correct, your assumptions hold.

Audio you send back must match the negotiated codec and sampleRate, in the same frame encoding.

Write raw PCM binary frames to the same connection. The Python server already does this (echo); to turn the other servers into echo servers, send the received binary frame straight back:

• Node.js (scripts/ws-server.node.js, in the binary branch): ws.send(data);
• PHP (scripts/ws-server.php, in the binary branch): $from->send($msg);
• Java (scripts/WsServer.java, in onBinaryMessage): session.getBasicRemote().sendBinary(ByteBuffer.wrap(bytes));

In a real bot you'd replace the echo with: STT, then LLM, then TTS, then send the synthesized PCM back. Each server marks the spot with a --- HERE: plug in your AI / STT processing --- comment.

To route calls that come into your Sinch number to the stream, run the ICE webhook server instead of the outbound trigger.

# Node.js
npm install express
node scripts/ice-callback.node.js     # listens on PORT (default 3001)

# Python
pip install flask python-dotenv
python scripts/ice-callback.py        # listens on PORT (default 8081)

Note: the two ice-callback scripts have different built-in default ports (Node 3001, Python 8081). If PORT is set in .env, it overrides both. Pick one runtime and expose its port with a second ngrok tunnel; set your Sinch service webhook URL to https://<that-ngrok>.ngrok-free.app/webhook.

On call.incoming, the server answers the inbound leg, bridges it, dials a STREAM leg to WS_ENDPOINT, and bridges that into the same stream-bridge. When either side hangs up, the other is torn down.

Couldn't verify: inbound event shape. The scripts read the caller number from body.call.from.phone.number. The spec's prose webhook example (call.answered) shows a flat shape ("from": "+1234567890"), and the call.incoming payload is not formally schematized in the OpenAPI file. Log the raw request body once and confirm the actual nesting for your account before depending on call.from.phone.number.

All files live under 4.2-stream-audio/scripts/ in the repository.

• STREAM is one of to's three discriminator values (PHONE, STREAM, VOICE_RELAY).
• stream.endpoint is required; it must be a valid ws:// or wss:// URL reachable from the public internet.
• stream.streamOptions.codec is the constant PCM.
• stream.streamOptions.sampleRate is one of {8000, 16000, 24000, 44100, 48000, 96000}, default 8000.
• stream.streamOptions.version defaults to 1.
• stream.callHeaders[] allows up to 16 {key, value} pairs, each field up to 255 chars.
• The WebSocket wire protocol (ConnectRequest, answer, heartbeat) and the PCM sample encoding (bit depth/endianness/channels) are not in the OpenAPI spec; verify them against the Sinch Streams product docs.