{"templateId":"markdown","sharedDataIds":{"sidebar":"sidebar-docs/voice-2.0/sidebars.yaml"},"props":{"metadata":{"markdoc":{"tagList":[]},"product":"Voice v2 (preview)","type":"markdown"},"seo":{"title":"WebSocket Server","siteUrl":"https://developers.sinch.com","llmstxt":{"title":"Sinch Developer Documentation","description":"LLMs.txt containing a map of all the documentation files for Sinch.","sections":[{"title":"Numbers API","description":"The Numbers API enables you to search for, view, and activate numbers. It's considered a precursor to other APIs in the Sinch product family. The numbers API can be used in tandem with any of our APIs that perform messaging or calling.","includeFiles":["docs/numbers/**/*.md","docs/numbers/**/*.yaml"],"excludeFiles":["docs/numbers/index.md"]},{"title":"Conversation API","description":"Send and receive messages globally on many popular channels with ease and confidence when using Sinch's Conversation API. Conversation API is the preferred API for sending mobile messages on SMS and other social channels with Sinch. It is a simple API with unified error messages, consistent request payloads, and common webhook payloads that are channel-agnostic.","includeFiles":["docs/conversation/**/*.md","docs/conversation/**/*.yaml"],"excludeFiles":["docs/conversation/index.md"]},{"title":"Voice API","description":"The Voice API works as a big telephony switch. The Voice API handles incoming phone calls (also known as incoming call “legs”), sets up outgoing phone calls (or outgoing call “legs”), and bridges the two. The incoming call leg may come in over a data connection (from a smartphone or web application using the Sinch SDKs) or through a local phone number (from the PSTN network). Similarly, the outgoing call leg can be over data (to another smartphone or web application using the Sinch SDKs) or the PSTN network.","includeFiles":["docs/voice/**/*.md","docs/voice/**/*.yaml"],"excludeFiles":["docs/voice/index.md"]},{"title":"Voice API v2","description":"The Voice API works as a big telephony switch. The Voice API handles incoming phone calls (also known as incoming call “legs”), sets up outgoing phone calls (or outgoing call “legs”), and bridges the two. The incoming call leg may come in over a data connection (from a smartphone or web application using the Sinch SDKs) or through a local phone number (from the PSTN network). Similarly, the outgoing call leg can be over data (to another smartphone or web application using the Sinch SDKs) or the PSTN network.","includeFiles":["docs/voice-2.0/**/*.md","docs/voice-2.0/**/*.yaml"],"excludeFiles":["docs/voice-2.0/index.md"]},{"title":"Verification API","description":"The Verification API is a platform for phone number verification. It consists of the API and different software development kits (the Sinch SDKs) that you integrate with your smartphone or web application and cloud based back-end services. Together they enable SMS, Flashcall, Phone Call and Data verification in your application.","includeFiles":["docs/verification/**/*.md","docs/verification/**/*.yaml"],"excludeFiles":["docs/verification/index.md"]},{"title":"Provisioning API","description":"Provisioning API allows you to programmatically set up your senders, accounts and templates on your favorite messaging platforms on the Conversation API. For now, you can create your first WhatsApp channel through Meta's Embedded sign up, you can configure your first SMS App and configure your webhooks. As development continues, we will be adding the most commonly used channels.","includeFiles":["docs/provisioning-api/**/*.md","docs/provisioning-api/**/*.json"],"excludeFiles":["docs/provisioning-api/index.md"]},{"title":"Elastic SIP Trunking API","description":"With Elastic SIP Trunking you can create and manage your SIP trunks and phone numbers programmatically.","includeFiles":["docs/est/**/*.md","docs/est/**/*.yaml"],"excludeFiles":["docs/est/index.md"]},{"title":"Fax API","description":"Send and receive HIPAA compliant faxes on our modern fax platform using our developer-friendly API.","includeFiles":["docs/fax/**/*.md","docs/fax/**/*.yaml"],"excludeFiles":["docs/fax/index.md"]},{"title":"In-app Voice and Video SDK","description":"The In-app Voice and Video SDK enables you to add voice and video calling capabilities directly into your mobile or web application using the Sinch SDKs.","includeFiles":["docs/in-app-calling/**/*.md"],"excludeFiles":["docs/in-app-calling/index.md"]},{"title":"Number Lookup API","description":"The Number Lookup API is designed to provide in-depth information about phone numbers, helping enterprises enhance their communication strategies and prevent fraud. By identifying the type of phone line (for example, mobile, landline, VoIP) and the associated carrier, the API allows businesses to optimize routing, reduce unnecessary costs, and improve customer engagement.","includeFiles":["docs/number-lookup-api-v2/**/*.md","docs/number-lookup-api-v2/**/*.yaml"],"excludeFiles":["docs/number-lookup-api-v2/index.md"]},{"title":"Functions","description":"Serverless compute for voice and messaging. Deploy your code and Sinch routes live calls and messages to it — no infrastructure to run.","includeFiles":["docs/functions/functions/**/*.md","docs/functions/concepts/**/*.md","docs/functions/reference/**/*.md"],"excludeFiles":["docs/functions/functions/index.md"]},{"title":"CLI","description":"One command line for every Sinch API and the full Functions lifecycle — scaffold, run locally, deploy, and manage.","includeFiles":["docs/functions/cli/**/*.md"],"excludeFiles":["docs/functions/cli/index.md"]}]}},"dynamicMarkdocComponents":[],"compilationErrors":[],"ast":{"$$mdtype":"Tag","name":"article","attributes":{},"children":[{"$$mdtype":"Tag","name":"Heading","attributes":{"level":1,"id":"websocket-server","__idx":0},"children":["WebSocket Server"]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"overview","__idx":1},"children":["Overview"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["A WebSocket server is the runtime piece on ",{"$$mdtype":"Tag","name":"em","attributes":{},"children":["your"]}," side that the Sinch Voice API talks to whenever a call leg is routed to a ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["STREAM"]}," (raw PCM audio) or ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["VOICE_RELAY"]}," (text-in/text-out) destination. The other tutorials in this section focus on the ",{"$$mdtype":"Tag","name":"em","attributes":{},"children":["Sinch"]}," side: the SVAML payload that points a call at your server. This tutorial is the server-side reference: what messages you must handle, in what order, how to reply with audio or text, and how to ship the thing to production."]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["The fastest way to understand it is to run an echo server and connect a tiny WebSocket client to it locally. No phone call, no ngrok, and no Sinch account required. You see the exact frames a real call would send, echoed straight back. Start there (below), then read the protocol reference and the production checklist."]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["There are two distinct WebSocket protocols, picked by ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["to.type"]}," in your SVAML:"]},{"$$mdtype":"Tag","name":"ul","attributes":{},"children":[{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["STREAM"]}]},": raw PCM audio frames interleaved with JSON control messages. Pairs with ",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"/docs/voice-2.0/tutorials/stream-audio"},"children":["Stream Call Audio in Real-Time"]},"."]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["VOICE_RELAY"]}]},": text-only protocol; Sinch does STT and TTS for you. Pairs with ",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"/docs/voice-2.0/tutorials/voice-relay"},"children":["Connect an AI Chatbot (Voice Relay)"]},"."]}]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["This tutorial ships minimal echo servers for both, in Node.js and Python."]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"get-the-code","__idx":2},"children":["Get the code"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["All sample code lives in the ",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://github.com/sinch/sinch-voice-tutorials"},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["sinch/sinch-voice-tutorials"]}]}," repository. Clone it and change into this tutorial's folder:"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"bash","header":{"controls":{"copy":{}}},"source":"git clone https://github.com/sinch/sinch-voice-tutorials.git\ncd sinch-voice-tutorials/4.4-websocket-server\n","lang":"bash"},"children":[]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["The scripts for this tutorial are under ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["scripts/"]},". Every relative link in this page (",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["../4.1-voice-relay/"]},", ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["../4.2-stream-audio/"]},", and so on) resolves against the same repo when you browse it on GitHub or on your own clone, so you can follow the cross-references without leaving the tree."]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"setup","__idx":3},"children":["Setup"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["You need ",{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["one"]}," of these runtimes:"]},{"$$mdtype":"Tag","name":"div","attributes":{"className":"md-table-wrapper"},"children":[{"$$mdtype":"Tag","name":"table","attributes":{"className":"md"},"children":[{"$$mdtype":"Tag","name":"thead","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"Runtime"},"children":["Runtime"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"Install the WebSocket library"},"children":["Install the WebSocket library"]}]}]},{"$$mdtype":"Tag","name":"tbody","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Node.js 18+"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["npm install ws"]}," (the scripts use ES modules; add ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["\"type\": \"module\""]}," to ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["package.json"]},")"]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Python 3.8+"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["pip install websockets"]}]}]}]}]}]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["For the local first-success test below, that is all you need. To drive a server from a ",{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["real call"]}," you additionally need:"]},{"$$mdtype":"Tag","name":"ul","attributes":{},"children":[{"$$mdtype":"Tag","name":"li","attributes":{},"children":["A ",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://dashboard.sinch.com"},"children":["Sinch account"]}," with API credentials and a virtual number (see ",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"/docs/voice-2.0/tutorials/outbound-tts"},"children":["Outbound Call"]}," or ",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"/docs/voice-2.0/tutorials/inbound-pstn"},"children":["Inbound PSTN"]},")."]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":["A publicly reachable ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["wss://"]}," endpoint. ",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"https://ngrok.com"},"children":["ngrok"]}," gives you one for free: ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["ngrok http 8765"]},"."]}]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["The scripts default to ",{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["port 8765"]}," (override with the ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["PORT"]}," environment variable). Note: the shared ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":[".env"]}," at the repository root sets ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["PORT=8081"]}," for other tutorials, so either unset it for this one or pass ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["PORT=8765"]}," explicitly so the port matches the ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["ngrok http 8765"]}," commands below."]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"first-success-see-frames-echoed-no-call-required-start-here","__idx":4},"children":["First success: see frames echoed, no call required (start here)"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["You can validate the entire WebSocket contract on your laptop before touching Sinch. Run an echo server, point a local client at it, and watch the frames come back."]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":3,"id":"1-start-the-stream-echo-server","__idx":5},"children":["1. Start the STREAM echo server"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"bash","header":{"controls":{"copy":{}}},"source":"# Python\npip install websockets\npython scripts/ws-stream-server.py        # listens on ws://localhost:8765\n\n# ...or Node.js\nnpm install ws\nnode scripts/ws-stream-server.node.js\n","lang":"bash"},"children":[]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["You should see ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["STREAM echo server on :8765"]},"."]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":3,"id":"2-connect-a-local-client-and-send-a-fake-call","__idx":6},"children":["2. Connect a local client and send a fake call"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Save this as ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["test-stream.py"]}," and run it in a second terminal (",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["pip install websockets"]}," if you haven't):"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"python","header":{"controls":{"copy":{}}},"source":"import asyncio, json, websockets\n\nasync def main():\n    async with websockets.connect(\"ws://localhost:8765\") as ws:\n        # Pretend to be Sinch opening the call:\n        await ws.send(json.dumps({\"command\": \"connect\", \"callId\": \"test-1\", \"applicationId\": \"app-1\"}))\n        print(\"server replied:\", await ws.recv())     # expect {\"command\":\"answer\"}\n        # Pretend to be the caller's audio (4 bytes of fake PCM):\n        await ws.send(b\"\\x01\\x02\\x03\\x04\")\n        echo = await ws.recv()\n        print(\"audio echoed back:\", echo)              # expect b'\\x01\\x02\\x03\\x04'\n\nasyncio.run(main())\n","lang":"python"},"children":[]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"header":{"controls":{"copy":{}}},"source":"server replied: {\"command\": \"answer\"}\naudio echoed back: b'\\x01\\x02\\x03\\x04'\n"},"children":[]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["That is the complete STREAM handshake: you sent ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["connect"]},", the server answered, you sent a binary PCM frame, and it came straight back. On a real call those binary frames are the caller's voice and the echo is what the caller hears."]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":3,"id":"3-optional-do-the-same-for-voice_relay","__idx":7},"children":["3. (Optional) Do the same for VOICE_RELAY"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"bash","header":{"controls":{"copy":{}}},"source":"python scripts/ws-relay-server.py          # or: node scripts/ws-relay-server.node.js\n","lang":"bash"},"children":[]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"python","header":{"controls":{"copy":{}}},"source":"import asyncio, json, websockets\n\nasync def main():\n    async with websockets.connect(\"ws://localhost:8765\") as ws:\n        await ws.send(json.dumps({\"command\": \"connect\", \"callId\": \"test-1\"}))\n        print(await ws.recv())   # {\"command\":\"answer\"}\n        print(await ws.recv())   # greeting text\n        await ws.send(json.dumps({\"command\": \"text\", \"text\": \"hello world\"}))\n        print(await ws.recv())   # {\"command\":\"text\",\"text\":\"You said: hello world\",\"isLast\":true}\n\nasyncio.run(main())\n","lang":"python"},"children":[]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Once both pass, the plumbing is proven correct for these echo cases. Everything after this is wiring the same server up to a live call and replacing the echo with real STT/LLM/TTS."]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":3,"id":"4-go-live-with-a-real-call","__idx":8},"children":["4. Go live with a real call"]},{"$$mdtype":"Tag","name":"ol","attributes":{},"children":[{"$$mdtype":"Tag","name":"li","attributes":{},"children":["Expose the running server: ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["ngrok http 8765"]},", then copy the ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["wss://<id>.ngrok-free.app"]}," Forwarding URL."]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":["Put that URL in the ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["endpoint"]}," of a ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["STREAM"]}," or ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["VOICE_RELAY"]}," destination. See ",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"/docs/voice-2.0/tutorials/stream-audio"},"children":["Stream audio"]}," (STREAM, outbound trigger) or ",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"/docs/voice-2.0/tutorials/voice-relay"},"children":["Voice relay"]}," (VOICE_RELAY, inbound via dashboard)."]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":["Place the call. A STREAM server will echo your voice back to you; a VOICE_RELAY server will read back whatever you say."]}]},{"$$mdtype":"Tag","name":"blockquote","attributes":{},"children":[{"$$mdtype":"Tag","name":"p","attributes":{},"children":["The shared ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":[".env"]}," field ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["WS_ENDPOINT"]}," holds the ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["wss://"]}," URL used by the call-trigger scripts in Stream audio/Voice relay. Update it to your current ngrok URL each session (ngrok URLs change on restart unless you have a reserved domain)."]}]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"real-life-examples","__idx":9},"children":["Real-life examples"]},{"$$mdtype":"Tag","name":"ul","attributes":{},"children":[{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["Custom STT/TTS pipeline"]},": You run your own ASR (for example Whisper) and TTS (for example Coqui). Use ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["STREAM"]}," so Sinch never touches the audio's meaning."]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["Embedded LLM with off-the-shelf voice"]},": You do not want to host ASR/TTS. Use ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["VOICE_RELAY"]}," and keep your server in pure text."]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["Hybrid agent"]},": Most turns go through ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["VOICE_RELAY"]},"; rare \"play this exact recording\" turns hand off via a separate SVAML branch."]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["Auditing or wiretap"]},": Stream a copy of the audio to an auditing service alongside the live conversation (set up two ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["STREAM"]}," legs on one call)."]}]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"the-stream-protocol-pcm-in-pcm-out","__idx":10},"children":["The STREAM protocol: PCM in, PCM out"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["When Sinch dials ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["to: { type: \"STREAM\", stream: { endpoint: \"wss://...\" } }"]},", it opens a WebSocket to your ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["endpoint"]}," and exchanges these messages:"]},{"$$mdtype":"Tag","name":"div","attributes":{"className":"md-table-wrapper"},"children":[{"$$mdtype":"Tag","name":"table","attributes":{"className":"md"},"children":[{"$$mdtype":"Tag","name":"thead","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"Direction"},"children":["Direction"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"Format"},"children":["Format"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"Message"},"children":["Message"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"When"},"children":["When"]}]}]},{"$$mdtype":"Tag","name":"tbody","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Sinch to server"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["JSON (text)"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["ConnectRequest"]},": ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["{\"command\":\"connect\",\"callId\":\"...\",\"applicationId\":\"...\",\"headers\":{…}}"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Once, immediately after the WS handshake."]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Server to Sinch"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["JSON (text)"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["ConnectResponse"]},": ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["{\"command\":\"answer\"}"]}," (or ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["\"busy\""]}," / ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["\"reject\""]},")"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Server's reply that opens the audio path."]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Sinch and server"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Binary"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Raw PCM audio frames"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Continuous bidirectional flow until the WS closes."]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Server to Sinch"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["JSON (text)"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["StreamControl"]},": ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["{\"command\":\"clear\"}"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Cancel any audio Sinch had queued; used for barge-in."]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Server to Sinch"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["JSON (text)"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["StreamControl"]},": ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["{\"command\":\"heartbeat\"}"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Optional; keeps idle connections alive."]}]}]}]}]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["The audio is ",{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["linear PCM"]},", codec fixed to ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["PCM"]},", at the sample rate negotiated in ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["streamOptions.sampleRate"]}," (one of ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["8000 / 16000 / 24000 / 44100 / 48000 / 96000"]}," Hz; default ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["8000"]},"). Frame size is implementation-defined: write whatever PCM bytes you have, when you have them."]},{"$$mdtype":"Tag","name":"blockquote","attributes":{},"children":[{"$$mdtype":"Tag","name":"p","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["Verify:"]}," the field and command names above (",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["connect"]},", ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["answer"]},", ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["busy"]},", ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["reject"]},", ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["clear"]},", ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["heartbeat"]},", ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["headers"]},", ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["callId"]},", ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["applicationId"]},") come from the 4.2 tutorial and the working sample servers, not from the OpenAPI spec. The spec defines only the SVAML side (",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["endpoint"]},", ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["streamOptions.codec"]},", ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["streamOptions.sampleRate"]},", ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["streamOptions.version"]},", ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["callHeaders[]"]},"). The on-the-wire JSON protocol is documented in the Streams product docs. Confirm exact names there before relying on ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["busy"]}," / ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["reject"]}," / ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["clear"]},"."]}]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":3,"id":"minimal-stream-server-nodejs","__idx":11},"children":["Minimal STREAM server (Node.js)"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["scripts/ws-stream-server.node.js"]},", requires ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["npm install ws"]},":"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"js","header":{"controls":{"copy":{}}},"source":"import { WebSocketServer } from \"ws\";\n\nconst PORT = Number(process.env.PORT || 8765);\nconst wss = new WebSocketServer({ port: PORT });\nconsole.log(`STREAM echo server on :${PORT}`);\n\nwss.on(\"connection\", (ws, req) => {\n  console.log(`new connection from ${req.socket.remoteAddress}`);\n\n  // Keepalive so corporate proxies don't drop an idle connection.\n  const heartbeat = setInterval(() => {\n    if (ws.readyState === ws.OPEN) ws.send(JSON.stringify({ command: \"heartbeat\" }));\n  }, 30_000);\n\n  ws.on(\"message\", (data, isBinary) => {\n    if (!isBinary) {\n      let msg;\n      try { msg = JSON.parse(data.toString()); } catch { return; }  // ignore malformed JSON\n      console.log(\"text frame\", msg);\n      if (msg.command === \"connect\") ws.send(JSON.stringify({ command: \"answer\" }));\n      return;\n    }\n    // Binary frame == raw PCM from the caller. Echo it back.\n    ws.send(data, { binary: true });\n  });\n\n  ws.on(\"close\", () => { clearInterval(heartbeat); console.log(\"connection closed\"); });\n});\n","lang":"js"},"children":[]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":3,"id":"minimal-stream-server-python","__idx":12},"children":["Minimal STREAM server (Python)"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["scripts/ws-stream-server.py"]},", requires ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["pip install websockets"]},":"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"python","header":{"controls":{"copy":{}}},"source":"import asyncio, json, os\nimport websockets\n\nPORT = int(os.environ.get(\"PORT\", \"8765\"))\n\nasync def heartbeat(ws):\n    try:\n        while True:\n            await asyncio.sleep(30)\n            await ws.send(json.dumps({\"command\": \"heartbeat\"}))\n    except websockets.exceptions.ConnectionClosed:\n        return\n\nasync def handler(ws):\n    print(f\"new connection from {ws.remote_address}\")\n    hb = asyncio.create_task(heartbeat(ws))\n    try:\n        async for msg in ws:\n            if isinstance(msg, (bytes, bytearray)):\n                await ws.send(bytes(msg))   # echo PCM\n                continue\n            try:\n                payload = json.loads(msg)\n            except json.JSONDecodeError:\n                continue                    # ignore malformed JSON\n            print(\"text frame\", payload)\n            if payload.get(\"command\") == \"connect\":\n                await ws.send(json.dumps({\"command\": \"answer\"}))\n    finally:\n        hb.cancel()\n        print(\"connection closed\")\n\nasync def main():\n    async with websockets.serve(handler, \"0.0.0.0\", PORT):\n        print(f\"STREAM echo server on :{PORT}\")\n        await asyncio.Future()  # run forever\n\nif __name__ == \"__main__\":\n    asyncio.run(main())\n","lang":"python"},"children":[]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Both servers handle the connection lifecycle (heartbeat cancelled on close), ignore malformed JSON instead of crashing, and never busy-loop."]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"the-voice_relay-protocol-text-in-text-out","__idx":13},"children":["The VOICE_RELAY protocol: text in, text out"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["When ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["to.type"]}," is ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["VOICE_RELAY"]},", Sinch does STT and TTS on its end. Your server only ever sees and sends text:"]},{"$$mdtype":"Tag","name":"div","attributes":{"className":"md-table-wrapper"},"children":[{"$$mdtype":"Tag","name":"table","attributes":{"className":"md"},"children":[{"$$mdtype":"Tag","name":"thead","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"Direction"},"children":["Direction"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"Command"},"children":["Command"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"When"},"children":["When"]}]}]},{"$$mdtype":"Tag","name":"tbody","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Sinch to server"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["connect"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Session start; carries ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["callId"]},", metadata, and any ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["callHeaders"]}," from the SVAML."]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Server to Sinch"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["answer"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Accept the relay session."]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Server to Sinch"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["text"]}," (+ ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["isLast"]},", optional ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["isInterruptible"]},")"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["TTS text to read to the caller. Send ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["isLast: true"]}," on the final chunk of a turn."]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Sinch to server"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["text"]}," / ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["prompt"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Transcribed speech from the caller."]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Sinch to server"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["interrupt"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Caller spoke during TTS playback (barge-in)."]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Sinch to server"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["dtmf"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["DTMF digits pressed by the caller."]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Sinch to server"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["textPlaybackStart"]}," / ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["textPlaybackStop"]}," / ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["textPlaybackCancel"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["TTS playback lifecycle."]}]}]}]}]},{"$$mdtype":"Tag","name":"blockquote","attributes":{},"children":[{"$$mdtype":"Tag","name":"p","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["Verify:"]}," as with STREAM, these command names are not in the OpenAPI spec. The spec defines only the SVAML ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["voiceRelay"]}," object (",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["endpoint"]},", ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["ttsVoice"]},", ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["sttLanguage"]},", ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["enableInterruptions"]},", ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["callHeaders[]"]},"). The JSON message shapes come from the Voice Relay product docs (an AsyncAPI definition) and the working reference server. The full reference implementation in ",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"/docs/voice-2.0/tutorials/voice-relay"},"children":["Voice Relay"]}," (",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["../4.1-voice-relay/server.py"]},") handles every command above."]}]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":3,"id":"minimal-voice_relay-server-nodejs","__idx":14},"children":["Minimal VOICE_RELAY server (Node.js)"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["scripts/ws-relay-server.node.js"]},", requires ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["npm install ws"]},":"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"js","header":{"controls":{"copy":{}}},"source":"import { WebSocketServer } from \"ws\";\n\nconst PORT = Number(process.env.PORT || 8765);\nconst wss = new WebSocketServer({ port: PORT });\nconsole.log(`VOICE_RELAY echo server on :${PORT}`);\n\nwss.on(\"connection\", (ws) => {\n  ws.on(\"message\", (data) => {\n    let msg;\n    try { msg = JSON.parse(data.toString()); } catch { return; }\n    console.log(\"rx\", msg);\n\n    if (msg.command === \"connect\") {\n      ws.send(JSON.stringify({ command: \"answer\" }));                                   // accept\n      ws.send(JSON.stringify({ command: \"text\", text: \"Hi! Anything you say I will repeat back.\", isLast: true }));\n      return;\n    }\n    // Sinch delivers transcribed user speech as `text` (or `prompt`).\n    if ((msg.command === \"text\" || msg.command === \"prompt\") && msg.text) {\n      ws.send(JSON.stringify({ command: \"text\", text: `You said: ${msg.text}`, isLast: true }));\n    }\n  });\n\n  ws.on(\"close\", () => console.log(\"connection closed\"));\n});\n","lang":"js"},"children":[]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":3,"id":"minimal-voice_relay-server-python","__idx":15},"children":["Minimal VOICE_RELAY server (Python)"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["scripts/ws-relay-server.py"]},", requires ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["pip install websockets"]},":"]},{"$$mdtype":"Tag","name":"CodeBlock","attributes":{"data-language":"python","header":{"controls":{"copy":{}}},"source":"import asyncio, json, os\nimport websockets\n\nPORT = int(os.environ.get(\"PORT\", \"8765\"))\n\nasync def handler(ws):\n    print(\"new connection\")\n    try:\n        async for msg in ws:\n            if isinstance(msg, (bytes, bytearray)):\n                continue                    # Voice Relay is text-only; ignore stray binary\n            try:\n                payload = json.loads(msg)\n            except json.JSONDecodeError:\n                continue\n            print(\"rx\", payload)\n            cmd = payload.get(\"command\")\n            if cmd == \"connect\":\n                await ws.send(json.dumps({\"command\": \"answer\"}))\n                await ws.send(json.dumps({\"command\": \"text\",\n                                          \"text\": \"Hi! Anything you say I will repeat back.\",\n                                          \"isLast\": True}))\n                continue\n            if cmd in (\"text\", \"prompt\") and payload.get(\"text\"):\n                await ws.send(json.dumps({\"command\": \"text\",\n                                          \"text\": f\"You said: {payload['text']}\",\n                                          \"isLast\": True}))\n    finally:\n        print(\"connection closed\")\n\nasync def main():\n    async with websockets.serve(handler, \"0.0.0.0\", PORT):\n        print(f\"VOICE_RELAY echo server on :{PORT}\")\n        await asyncio.Future()\n\nif __name__ == \"__main__\":\n    asyncio.run(main())\n","lang":"python"},"children":[]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Both echo servers reply to ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["connect"]},", greet the caller, and read back each transcribed turn. That is enough to confirm the relay plumbing before you swap the echo for a real LLM call (see 4.1 for a LangChain-backed version supporting OpenAI, Claude, and Gemini)."]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"choosing-between-stream-and-voice_relay","__idx":16},"children":["Choosing between STREAM and VOICE_RELAY"]},{"$$mdtype":"Tag","name":"div","attributes":{"className":"md-table-wrapper"},"children":[{"$$mdtype":"Tag","name":"table","attributes":{"className":"md"},"children":[{"$$mdtype":"Tag","name":"thead","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"Question"},"children":["Question"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"STREAM"},"children":["STREAM"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"VOICE_RELAY"},"children":["VOICE_RELAY"]}]}]},{"$$mdtype":"Tag","name":"tbody","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Do you control STT?"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Yes, your problem"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["No, Sinch handles it"]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Do you control TTS?"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Yes, your problem"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["No, Sinch handles it"]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Audio format"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["PCM only (codec fixed)"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["N/A, text only"]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Bandwidth"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Audio: sampleRate × bytes/sec"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Tiny (just JSON)"]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Latency floor"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Sub-second possible"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Sinch STT/TTS + your model per turn"]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["LLM provider lock-in"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["None"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["None"]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Use when..."]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Custom voice agent, audio-to-audio model"]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Standard LLM + Sinch voices"]}]}]}]}]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Rule of thumb: reach for ",{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["VOICE_RELAY"]}," (4.1) unless you specifically need to own the audio path; reach for ",{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["STREAM"]}," (4.2) when you run your own ASR/TTS or an audio-to-audio model."]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"production-checklist","__idx":17},"children":["Production checklist"]},{"$$mdtype":"Tag","name":"div","attributes":{"className":"md-table-wrapper"},"children":[{"$$mdtype":"Tag","name":"table","attributes":{"className":"md"},"children":[{"$$mdtype":"Tag","name":"thead","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"Concern"},"children":["Concern"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"What to do"},"children":["What to do"]}]}]},{"$$mdtype":"Tag","name":"tbody","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["TLS"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Always run behind ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["wss://"]},". ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["ws://"]}," is local dev only. ngrok gives you ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["wss://"]}," for free during development."]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["Auth"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["The endpoint URL is on the open internet. Validate a token from the query string, a header on the upgrade request, or a value in ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["callHeaders"]}," / the ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["connect"]}," message. Drop the connection if it doesn't match."]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["Resource limits"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Each call = one WS connection = some memory + CPU. Set per-process connection caps and decide your horizontal-scale strategy up front."]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["Backpressure"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["When the downstream model can't keep up, drop frames. Never buffer indefinitely; buffers inflate perceived latency."]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["Heartbeat / keepalive"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Send ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["{\"command\":\"heartbeat\"}"]}," (STREAM) or rely on WS pings (VOICE_RELAY) every ~30 s so corporate proxies don't kill idle connections."]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["Observability"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Log ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["callId"]}," + ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["applicationId"]}," (STREAM) or ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["callId"]}," (VOICE_RELAY) on every message. They are your join key with ",{"$$mdtype":"Tag","name":"MarkdownLink","attributes":{"href":"/docs/voice-2.0/tutorials/track-call-status"},"children":["Track Call Status"]},"."]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["Reconnect strategy"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Sinch does not reconnect on its own. If your server drops mid-call, the call leg ends. Crash-only design plus supervised process restarts beat fragile reconnection logic."]}]}]}]}]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"what-the-openapi-spec-actually-defines","__idx":18},"children":["What the OpenAPI spec actually defines"]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["The spec covers only the ",{"$$mdtype":"Tag","name":"strong","attributes":{},"children":["SVAML side"]},": how you point a call at your server, not the bytes exchanged afterward."]},{"$$mdtype":"Tag","name":"ul","attributes":{},"children":[{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["STREAM"]}]}," (",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["stream"]}," schema): requires ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["stream.endpoint"]}," (",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["ws://"]}," or ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["wss://"]},", public). ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["streamOptions.codec"]}," is fixed to ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["PCM"]},". ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["streamOptions.sampleRate"]}," is in ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["{8000, 16000, 24000, 44100, 48000, 96000}"]},", default ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["8000"]},". ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["streamOptions.version"]}," default ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["1"]},". Optional ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["callHeaders[]"]}," (max 16, key/value up to 255 chars)."]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":[{"$$mdtype":"Tag","name":"strong","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["VOICE_RELAY"]}]}," (",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["voiceRelay"]}," schema): requires ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["voiceRelay.endpoint"]}," (URI), ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["ttsVoice"]},", and ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["sttLanguage"]}," (BCP-47, for example ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["en-US"]},"). Optional ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["enableInterruptions"]}," (default ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["true"]},") and ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["callHeaders[]"]}," (max 16)."]},{"$$mdtype":"Tag","name":"li","attributes":{},"children":["The on-the-wire JSON/binary protocols between Sinch and your WebSocket server are out of band of the OpenAPI spec; they live in the Streams / Voice Relay product docs. Every command name in this tutorial is marked \"verify\" above for that reason."]}]},{"$$mdtype":"Tag","name":"Heading","attributes":{"level":2,"id":"sample-code-in-this-tutorial","__idx":19},"children":["Sample code in this tutorial"]},{"$$mdtype":"Tag","name":"div","attributes":{"className":"md-table-wrapper"},"children":[{"$$mdtype":"Tag","name":"table","attributes":{"className":"md"},"children":[{"$$mdtype":"Tag","name":"thead","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"File"},"children":["File"]},{"$$mdtype":"Tag","name":"th","attributes":{"data-label":"What it does"},"children":["What it does"]}]}]},{"$$mdtype":"Tag","name":"tbody","attributes":{},"children":[{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["scripts/ws-stream-server.node.js"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Minimal echo ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["STREAM"]}," server (Node.js + ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["ws"]},")."]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["scripts/ws-stream-server.py"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":["Minimal echo ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["STREAM"]}," server (Python + ",{"$$mdtype":"Tag","name":"code","attributes":{},"children":["websockets"]},")."]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["scripts/ws-relay-server.node.js"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["VOICE_RELAY"]}," text-echo server (Node.js)."]}]},{"$$mdtype":"Tag","name":"tr","attributes":{},"children":[{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["scripts/ws-relay-server.py"]}]},{"$$mdtype":"Tag","name":"td","attributes":{},"children":[{"$$mdtype":"Tag","name":"code","attributes":{},"children":["VOICE_RELAY"]}," text-echo server (Python)."]}]}]}]}]},{"$$mdtype":"Tag","name":"p","attributes":{},"children":["Each echoes back what it receives so you can sanity-check the WebSocket plumbing, using the local client snippets above or a live call, before plugging in a real LLM / ASR / TTS."]}]},"headings":[{"value":"WebSocket Server","id":"websocket-server","depth":1},{"value":"Overview","id":"overview","depth":2},{"value":"Get the code","id":"get-the-code","depth":2},{"value":"Setup","id":"setup","depth":2},{"value":"First success: see frames echoed, no call required (start here)","id":"first-success-see-frames-echoed-no-call-required-start-here","depth":2},{"value":"1. Start the STREAM echo server","id":"1-start-the-stream-echo-server","depth":3},{"value":"2. Connect a local client and send a fake call","id":"2-connect-a-local-client-and-send-a-fake-call","depth":3},{"value":"3. (Optional) Do the same for VOICE_RELAY","id":"3-optional-do-the-same-for-voice_relay","depth":3},{"value":"4. Go live with a real call","id":"4-go-live-with-a-real-call","depth":3},{"value":"Real-life examples","id":"real-life-examples","depth":2},{"value":"The STREAM protocol: PCM in, PCM out","id":"the-stream-protocol-pcm-in-pcm-out","depth":2},{"value":"Minimal STREAM server (Node.js)","id":"minimal-stream-server-nodejs","depth":3},{"value":"Minimal STREAM server (Python)","id":"minimal-stream-server-python","depth":3},{"value":"The VOICE_RELAY protocol: text in, text out","id":"the-voice_relay-protocol-text-in-text-out","depth":2},{"value":"Minimal VOICE_RELAY server (Node.js)","id":"minimal-voice_relay-server-nodejs","depth":3},{"value":"Minimal VOICE_RELAY server (Python)","id":"minimal-voice_relay-server-python","depth":3},{"value":"Choosing between STREAM and VOICE_RELAY","id":"choosing-between-stream-and-voice_relay","depth":2},{"value":"Production checklist","id":"production-checklist","depth":2},{"value":"What the OpenAPI spec actually defines","id":"what-the-openapi-spec-actually-defines","depth":2},{"value":"Sample code in this tutorial","id":"sample-code-in-this-tutorial","depth":2}],"frontmatter":{"seo":{"title":"WebSocket Server"}},"lastModified":"2026-08-31T13:48:15.000Z","pagePropGetterError":{"message":"","name":""}},"slug":"/docs/voice-2.0/tutorials/websocket-agent","userData":{"isAuthenticated":false,"teams":["anonymous"]},"isPublic":true}