ElevenLabs SDK-compatible realtime STT (WebSocket)
Streams audio and receives ElevenLabs-compatible transcript events with `client.speechToText.realtime.connect(...)`. The request must include `Upgrade: websocket`. **Client → server** frames (JSON): `{ "message_type": "input_audio_chunk", "audio_base_64": "...", "sample_rate": 16000, "commit": true }` **Server → client** frames (JSON), `message_type` one of: `session_started`, `partial_transcript`, `committed_transcript`, `committed_transcript_with_timestamps`, `error`. **Commit strategy.** `commit_strategy=vad` finds utterance boundaries automatically with the model's native turn detection or VAD. Tune it with the `vad_*` / `min_*` params. `auto` is accepted as a legacy alias of `vad`. `commit_strategy=manual` instead lets the client close an utterance explicitly by sending an `input_audio_chunk` frame with `commit: true` (the SDK's `conn.commit()`) and is the compatibility surface's default. With providers that do not support manual commits, completed transcript text is returned together on commit. The ElevenLabs SDK does not support custom headers on this connection. Select a provider with a `<provider>/<model>` value in `model_id`. Use `provider_options` for provider-specific settings. You can also pass supported options directly as query parameters. If both forms set the same option, `provider_options[<name>]=<value>` takes precedence. Recognized enum and boolean option values are case-insensitive and are normalized to each provider's wire spelling; free-form values such as prompts, keyterms, and voice IDs retain their original case. The message-level protocol is documented at [the realtime WebSocket reference](/realtime-reference) (AsyncAPI spec: [/asyncapi.yaml](/asyncapi.yaml)).
/el/v1/speech-to-text/realtimeStreams audio and receives ElevenLabs-compatible transcript events with
client.speechToText.realtime.connect(...). The request must include
Upgrade: websocket.
Client → server frames (JSON):
{ "message_type": "input_audio_chunk", "audio_base_64": "...", "sample_rate": 16000, "commit": true }
Server → client frames (JSON), message_type one of:
session_started, partial_transcript, committed_transcript,
committed_transcript_with_timestamps, error.
Commit strategy. commit_strategy=vad finds utterance boundaries
automatically with the model's native turn detection or VAD. Tune it with the
vad_* / min_* params. auto is accepted as a legacy alias of vad.
commit_strategy=manual instead lets the client close an
utterance explicitly by sending an input_audio_chunk frame with
commit: true (the SDK's conn.commit()) and is the compatibility
surface's default. With providers that do not support
manual commits, completed transcript text is returned together on commit.
The ElevenLabs SDK does not support custom headers on this connection. Select a
provider with a <provider>/<model> value in model_id.
Use provider_options for provider-specific settings. You can also pass
supported options directly as query parameters. If both forms set the same
option, provider_options[<name>]=<value> takes precedence. Recognized enum
and boolean option values are case-insensitive and are normalized to each
provider's wire spelling; free-form values such as prompts, keyterms, and voice
IDs retain their original case.
The message-level protocol is documented at the realtime WebSocket reference (AsyncAPI spec: /asyncapi.yaml).
Tenant key (ElevenLabs SDK convention).
In: header
Query Parameters
Model id: bare name or {author}/{modelName} slug (deepgram/nova-3).
Comma-separated or repeated provider IDs in preferred order. Providers not listed remain eligible.
Comma-separated or repeated provider IDs. Only these providers may serve the request.
Comma-separated or repeated provider IDs that must not serve the request.
Set to false to prevent fallback to another provider. Automatic provider retries are not currently supported.
Input audio format, such as pcm_16000 or ulaw_8000.
Keyword/keyterm biasing. Repeat the param for multiple terms.
VAD tuning (auto commit): trailing silence, in seconds, that ends an utterance.
VAD tuning (auto commit): speech-probability threshold (0–1) for detecting speech.
VAD tuning (auto commit): minimum speech length, in ms, before an utterance starts.
VAD tuning (auto commit): minimum silence length, in ms, before an utterance ends.
When set, committed transcripts include word/char timestamps (committed_transcript_with_timestamps frames).
Request provider language detection metadata where supported.
Utterance segmentation. vad lets the model decide boundaries automatically — its native turn detection or VAD (tune via the vad_* / min_* params); auto is an accepted legacy alias of vad; manual (default) lets the client commit utterances explicitly with an input_audio_chunk frame carrying commit: true (conn.commit()).
"manual"Value in
- "vad"
- "manual"
- "auto"
When set, requests non-verbatim (cleaned/formatted) transcripts where the provider supports it.
Provider-specific options in bracket notation, such as provider_options[encoding]=linear16. Provider-native names are accepted only inside this namespace; unknown bare query parameters are rejected.
Header Parameters
Must be websocket to perform the protocol upgrade.
Response Body
application/json
application/json
application/json
application/json
import { readFile } from "node:fs/promises";import WebSocket from "ws";const audioChunk = await readFile("chunk.pcm");const ws = new WebSocket("wss://api.allmodels.io/el/v1/speech-to-text/realtime?model_id=deepgram/nova-3&audio_format=pcm_16000&sample_rate=16000", { headers: { Authorization: `Bearer ${process.env.ALLMODELS_API_KEY}` }});ws.on("open", () => { ws.send(JSON.stringify({ message_type: "input_audio_chunk", audio_base_64: audioChunk.toString("base64"), sample_rate: 16000, commit: true }));});ws.on("message", (data, isBinary) => { if (isBinary) process.stdout.write(data); else console.log(JSON.parse(data.toString()));});ws.on("error", console.error);import asyncioimport base64import jsonimport osfrom pathlib import Pathimport websocketsasync def main(): async with websockets.connect( "wss://api.allmodels.io/el/v1/speech-to-text/realtime?model_id=deepgram/nova-3&audio_format=pcm_16000&sample_rate=16000", additional_headers={"Authorization": f"Bearer {os.environ['ALLMODELS_API_KEY']}"}, ) as socket: audio = base64.b64encode(Path("chunk.pcm").read_bytes()).decode("ascii") await socket.send(json.dumps({"message_type": "input_audio_chunk", "audio_base_64": audio, "sample_rate": 16000, "commit": True})) async for message in socket: print(message if isinstance(message, str) else f"{len(message)} audio bytes")asyncio.run(main())curl --http1.1 -i "https://api.allmodels.io/el/v1/speech-to-text/realtime?model_id=deepgram/nova-3&audio_format=pcm_16000&sample_rate=16000" \ -H "Authorization: Bearer $ALLMODELS_API_KEY" \ -H "Connection: Upgrade" \ -H "Upgrade: websocket" \ -H "Sec-WebSocket-Version: 13" \ -H "Sec-WebSocket-Key: SGVsbG9BbGxNb2RlbHMhIQ=="import { AudioFormat, CommitStrategy, ElevenLabsClient, RealtimeEvents } from "@elevenlabs/elevenlabs-js";import { readFile } from "node:fs/promises";const client = new ElevenLabsClient({ apiKey: process.env.ALLMODELS_API_KEY, baseUrl: "https://api.allmodels.io/el"});const connection = await client.speechToText.realtime.connect({ modelId: "deepgram/nova-3", audioFormat: AudioFormat.PCM_16000, sampleRate: 16000, languageCode: "en", includeTimestamps: true, commitStrategy: CommitStrategy.MANUAL});connection.on(RealtimeEvents.PARTIAL_TRANSCRIPT, ({ text }) => console.log("partial", text));connection.on(RealtimeEvents.COMMITTED_TRANSCRIPT, ({ text }) => console.log("final", text));connection.send({ audioBase64: (await readFile("chunk.pcm")).toString("base64") });connection.commit();import asyncioimport osfrom pathlib import Pathfrom elevenlabs.realtime.scribe import AudioFormat, CommitStrategy, ScribeRealtime, websocket_connectasync def main(): client = ScribeRealtime( api_key=os.environ["ALLMODELS_API_KEY"], base_url="https://api.allmodels.io/el", ) url = client._build_websocket_url( model_id="deepgram/nova-3", audio_format=AudioFormat.PCM_16000.value, commit_strategy=CommitStrategy.MANUAL.value, language_code="en", ) socket = await websocket_connect( url, additional_headers={"Authorization": f"Bearer {os.environ['ALLMODELS_API_KEY']}"}, ) await socket.send(Path("chunk.pcm").read_bytes()) async for event in socket: print(event)asyncio.run(main()){ "detail": { "message": "unsupported model 'bogus' for provider 'soniox'", "status": "invalid_model", "provider": "soniox", "model": "bogus", "supported": [ "stt-rt-v5" ] }}{ "error": "missing_api_key"}{ "detail": { "message": "websocket upgrade required", "status": "websocket_upgrade_required" }}{ "detail": { "message": "authentication was not initialized", "status": "auth_not_initialized" }}ElevenLabs SDK-compatible STT (file upload) POST
Transcribes an uploaded audio file with `client.speechToText.convert(...)`. Supported formats are WAV, MP3, FLAC, MP4/M4A, OGG, and WebM. Unsupported or unreadable files return `422 indeterminate_audio_duration`. Use `provider_options` for provider-specific settings. You can also pass supported options directly as query parameters. If both forms set the same option, `provider_options[<name>]=<value>` takes precedence. Recognized enum and boolean option values are case-insensitive and are normalized to each provider's wire spelling; free-form values such as prompts, keyterms, and voice IDs retain their original case.
List voices (SDK get_all) GET
Lists voices with ElevenLabs `voices.get_all()`. Use `provider_only` to choose a provider. The returned `voice_id` values can be used directly with the `/el` TTS endpoints.
