Native streaming TTS (WebSocket)
Opens a WebSocket using the selected provider's text-to-speech protocol. The request must include `Upgrade: websocket`. After connecting, send provider-specific JSON messages and receive binary audio. For example, Grok accepts `{"type":"text","text":"..."}` plus optional `{"type":"flush"}` / `{"type":"close"}` frames. Use `provider_options` for provider-specific settings. You can also pass supported options directly as query parameters. If both forms set the same option, `provider_options[<name>]=<value>` takes precedence. Recognized enum and boolean option values are case-insensitive and are normalized to each provider's wire spelling; free-form values such as prompts, keyterms, and voice IDs retain their original case. Authentication failures close the connection with code `4401` or `4403`. The close reason contains `{ "error": "<code>" }`. The message-level protocol is documented at [the realtime WebSocket reference](/realtime-reference) (AsyncAPI spec: [/asyncapi.yaml](/asyncapi.yaml)).
/v1/ttsOpens a WebSocket using the selected provider's text-to-speech protocol.
The request must include Upgrade: websocket.
After connecting, send provider-specific JSON messages and receive binary
audio. For example, Grok accepts {"type":"text","text":"..."} plus optional
{"type":"flush"} / {"type":"close"} frames.
Use provider_options for provider-specific settings. You can also pass
supported options directly as query parameters. If both forms set the same
option, provider_options[<name>]=<value> takes precedence. Recognized enum
and boolean option values are case-insensitive and are normalized to each
provider's wire spelling; free-form values such as prompts, keyterms, and voice
IDs retain their original case.
Authentication failures close the connection with code 4401 or 4403.
The close reason contains { "error": "<code>" }.
The message-level protocol is documented at the realtime WebSocket reference (AsyncAPI spec: /asyncapi.yaml).
Authorization
BearerAuth Tenant key supplied as the Authorization Bearer token.
In: header
Query Parameters
Use a specific TTS provider.
Value in
- "deepgram"
- "elevenlabs"
- "soniox"
- "fish"
- "groq"
- "grok"
- "minimax"
- "cartesia"
- "together"
- "openai"
- "gemini"
- "fal"
- "cloudflare"
Provider voice id (e.g. eve for Grok, EXAVITQu4vr4xnSDxMaL for ElevenLabs).
Model id: bare name (eleven_flash_v2_5) or {author}/{modelName} slug (elevenlabs/eleven_flash_v2_5).
Comma-separated or repeated provider IDs in preferred order. Providers not listed remain eligible.
Comma-separated or repeated provider IDs. Only these providers may serve the request.
Comma-separated or repeated provider IDs that must not serve the request.
Set to false to prevent fallback to another provider. Automatic provider retries are not currently supported.
Output audio container.
Value in
- "mp3"
- "pcm"
Auto-flush buffered text. Any value except the literal false enables it.
"true"Provider-specific options in bracket notation, such as provider_options[encoding]=linear16. You can also pass supported options directly as query parameters. If both forms set the same option, the bracketed value takes precedence.
Header Parameters
Must be websocket to perform the protocol upgrade.
Response Body
application/json
application/json
application/json
application/json
application/json
application/json
import WebSocket from "ws";const ws = new WebSocket("wss://api.allmodels.io/v1/tts?provider=grok&model=grok/grok-tts&voice=eve&format=pcm", { headers: { Authorization: `Bearer ${process.env.ALLMODELS_API_KEY}` }});ws.on("open", () => { ws.send(JSON.stringify({ type: "text", text: "Hello from AllModels" }));});ws.on("message", (data, isBinary) => { if (isBinary) process.stdout.write(data); else console.log(JSON.parse(data.toString()));});ws.on("error", console.error);import asyncioimport jsonimport osimport websocketsasync def main(): async with websockets.connect( "wss://api.allmodels.io/v1/tts?provider=grok&model=grok/grok-tts&voice=eve&format=pcm", additional_headers={"Authorization": f"Bearer {os.environ['ALLMODELS_API_KEY']}"}, ) as socket: await socket.send(json.dumps({"type": "text", "text": "Hello from AllModels"})) async for message in socket: print(message if isinstance(message, str) else f"{len(message)} audio bytes")asyncio.run(main())curl --http1.1 -i "https://api.allmodels.io/v1/tts?provider=grok&model=grok/grok-tts&voice=eve&format=pcm" \ -H "Authorization: Bearer $ALLMODELS_API_KEY" \ -H "Connection: Upgrade" \ -H "Upgrade: websocket" \ -H "Sec-WebSocket-Version: 13" \ -H "Sec-WebSocket-Key: SGVsbG9BbGxNb2RlbHMhIQ=="{ "error": "invalid_tts_request"}{ "error": "missing_api_key"}{ "error": "provider_not_allowed", "tried": [ "deepgram", "soniox", "assemblyai" ]}{ "error": "instructions_not_supported", "model": "grok-tts", "tried": [ "grok" ]}{ "error": "websocket_upgrade_required"}{ "error": "auth_not_initialized"}Create a Realtime client secret POST
Creates credentials for an untrusted client, such as a browser or mobile app, to connect directly to AllModels without receiving your API key.
Native streaming STT (WebSocket) GET
Opens a WebSocket and emits normalized streaming speech-to-text events. The request must include `Upgrade: websocket`. Stream binary audio and use `stt.audio.commit`, `stt.audio.clear`, and `stt.session.close` controls. Server events share a sequenced `stt.*` envelope. Transcript updates remain separate from a unified flat `stt.signal` event whose state is `started`, `end_candidate`, `resumed`, or `end`. Optional `turn_scope` preserves utterance-versus-turn meaning; it is distinct from transcript-final `scope`. Only signals explicitly supplied by the selected provider are surfaced. Use `event_format=provider` for provider-native passthrough in both directions. Provider-mode server frames are forwarded without signal mapping, timestamp rewriting, deduplication, or router-generated JSON events. This mode requires a direct upstream realtime WebSocket; router-buffered Groq and Fal relays reject it because no provider event stream exists. `audio_format` is required and describes the unchanged raw bytes. No resampling, transcoding, or FFmpeg occurs in the streaming path. Portable flat parameters control commits, turn detection, VAD thresholds, partials, timestamps, language detection, keyterms/context, diarization, formatting, and filler words. Unknown bare parameters are rejected. Put provider-native names under `provider_options[<name>]`; native overrides win except that conflicting encoding/sample-rate/channel declarations return `conflicting_audio_format`. Portable input mapping: | Family | Default automatic behavior | Native translation / fallback | |---|---|---| | Deepgram Nova | acoustic endpointing | `endpointing`, `utterance_end_ms`, `vad_events` | | Deepgram Flux | semantic turns | `eot_threshold`, `eager_eot_threshold`, `eot_timeout_ms` | | ElevenLabs Scribe realtime | provider VAD | `commit_strategy=vad`, `vad_*`, `min_*` | | Soniox realtime | acoustic endpointing | `enable_endpoint_detection`, `max_endpoint_delay_ms` | | AssemblyAI streaming | provider turn detection | model-specific turn silence/confidence fields | | xAI Grok | acoustic; semantic when requested | `endpointing` or Smart Turn fields | | Cartesia ink-2 | semantic turns | `stream_mode=turns`; manual selects `manual` | | Cartesia ink-whisper | acoustic router VAD | no provider turn detector; router VAD segments the stream | | OpenAI transcription | server VAD | nested `turn_detection` session object | | OpenAI realtime Whisper | acoustic router VAD | provider detector disabled; router sends native commits | | Together | provider VAD | `threshold`, `min_silence_duration_ms`, `min_speech_duration_ms` | | Groq / Fal buffered relays | acoustic router VAD | router inspects declared little-endian PCM and commits without modifying bytes | Router VAD supplies `turn_detection=acoustic` only for OpenAI realtime Whisper, Cartesia `ink-whisper`, and the buffered Groq/Fal relays. Deepgram Flux has no router-VAD path — its client grammar carries no commit frame — so it serves semantic turn detection only and rejects `turn_detection=acoustic` with `unsupported_stt_option`. Each model family advertises only the portable options it can translate to a native parameter; an explicit option outside that set fails before audio with `unsupported_stt_option` rather than riding along as a silent no-op. Omitted options fall back to the family's own behavior — `interim_results`, for example, has no global default. Router VAD can create transcript commits but never manufactures provider-originated `stt.signal` events, and the signals a session may emit are fixed by the `capabilities` object echoed in `stt.session.started`. Authentication failures close the connection with code `4401` or `4403`. The message-level protocol is documented at [the realtime WebSocket reference](/realtime-reference) (AsyncAPI spec: [/asyncapi.yaml](/asyncapi.yaml)).
