Native streaming STT (WebSocket)

Opens a WebSocket and emits normalized streaming speech-to-text events. The request must include `Upgrade: websocket`. Stream binary audio and use `stt.audio.commit`, `stt.audio.clear`, and `stt.session.close` controls. Server events share a sequenced `stt.*` envelope. Transcript updates remain separate from a unified flat `stt.signal` event whose state is `started`, `end_candidate`, `resumed`, or `end`. Optional `turn_scope` preserves utterance-versus-turn meaning; it is distinct from transcript-final `scope`. Only signals explicitly supplied by the selected provider are surfaced. Use `event_format=provider` for provider-native passthrough in both directions. Provider-mode server frames are forwarded without signal mapping, timestamp rewriting, deduplication, or router-generated JSON events. This mode requires a direct upstream realtime WebSocket; router-buffered Groq and Fal relays reject it because no provider event stream exists. `audio_format` is required and describes the unchanged raw bytes. No resampling, transcoding, or FFmpeg occurs in the streaming path. Portable flat parameters control commits, turn detection, VAD thresholds, partials, timestamps, language detection, keyterms/context, diarization, formatting, and filler words. Unknown bare parameters are rejected. Put provider-native names under `provider_options[<name>]`; native overrides win except that conflicting encoding/sample-rate/channel declarations return `conflicting_audio_format`. Portable input mapping: | Family | Default automatic behavior | Native translation / fallback | |---|---|---| | Deepgram Nova | acoustic endpointing | `endpointing`, `utterance_end_ms`, `vad_events` | | Deepgram Flux | semantic turns | `eot_threshold`, `eager_eot_threshold`, `eot_timeout_ms` | | ElevenLabs Scribe realtime | provider VAD | `commit_strategy=vad`, `vad_*`, `min_*` | | Soniox realtime | acoustic endpointing | `enable_endpoint_detection`, `max_endpoint_delay_ms` | | AssemblyAI streaming | provider turn detection | model-specific turn silence/confidence fields | | xAI Grok | acoustic; semantic when requested | `endpointing` or Smart Turn fields | | Cartesia ink-2 | semantic turns | `stream_mode=turns`; manual selects `manual` | | Cartesia ink-whisper | acoustic router VAD | no provider turn detector; router VAD segments the stream | | OpenAI transcription | server VAD | nested `turn_detection` session object | | OpenAI realtime Whisper | acoustic router VAD | provider detector disabled; router sends native commits | | Together | provider VAD | `threshold`, `min_silence_duration_ms`, `min_speech_duration_ms` | | Groq / Fal buffered relays | acoustic router VAD | router inspects declared little-endian PCM and commits without modifying bytes | Router VAD supplies `turn_detection=acoustic` only for OpenAI realtime Whisper, Cartesia `ink-whisper`, and the buffered Groq/Fal relays. Deepgram Flux has no router-VAD path — its client grammar carries no commit frame — so it serves semantic turn detection only and rejects `turn_detection=acoustic` with `unsupported_stt_option`. Each model family advertises only the portable options it can translate to a native parameter; an explicit option outside that set fails before audio with `unsupported_stt_option` rather than riding along as a silent no-op. Omitted options fall back to the family's own behavior — `interim_results`, for example, has no global default. Router VAD can create transcript commits but never manufactures provider-originated `stt.signal` events, and the signals a session may emit are fixed by the `capabilities` object echoed in `stt.session.started`. Authentication failures close the connection with code `4401` or `4403`. The message-level protocol is documented at [the realtime WebSocket reference](/realtime-reference) (AsyncAPI spec: [/asyncapi.yaml](/asyncapi.yaml)).

import { readFile } from "node:fs/promises";import WebSocket from "ws";const audioChunk = await readFile("chunk.pcm");const ws = new WebSocket("wss://api.allmodels.io/v1/stt?provider=deepgram&model=deepgram/nova-3&audio_format=pcm_16000&event_format=normalized", {  headers: { Authorization: `Bearer ${process.env.ALLMODELS_API_KEY}` }});ws.on("open", () => {  ws.send(audioChunk);  ws.send(JSON.stringify({ type: "stt.audio.commit" }));});ws.on("message", (data, isBinary) => {  if (isBinary) process.stdout.write(data);  else console.log(JSON.parse(data.toString()));});ws.on("error", console.error);

import asyncioimport base64import jsonimport osfrom pathlib import Pathimport websocketsasync def main():    async with websockets.connect(        "wss://api.allmodels.io/v1/stt?provider=deepgram&model=deepgram/nova-3&audio_format=pcm_16000&event_format=normalized",        additional_headers={"Authorization": f"Bearer {os.environ['ALLMODELS_API_KEY']}"},    ) as socket:        await socket.send(Path("chunk.pcm").read_bytes())        await socket.send(json.dumps({"type": "stt.audio.commit"}))        async for message in socket:            print(message if isinstance(message, str) else f"{len(message)} audio bytes")asyncio.run(main())

curl --http1.1 -i "https://api.allmodels.io/v1/stt?provider=deepgram&model=deepgram/nova-3&audio_format=pcm_16000&event_format=normalized" \  -H "Authorization: Bearer $ALLMODELS_API_KEY" \  -H "Connection: Upgrade" \  -H "Upgrade: websocket" \  -H "Sec-WebSocket-Version: 13" \  -H "Sec-WebSocket-Key: SGVsbG9BbGxNb2RlbHMhIQ=="

{  "error": "invalid_model",  "provider": "soniox",  "model": "bogus",  "supported": [    "stt-rt-v5"  ]}

{  "error": "missing_api_key"}

{  "error": "provider_not_allowed",  "tried": [    "deepgram",    "soniox",    "assemblyai"  ]}

{  "error": "model_not_priced",  "provider": "deepgram",  "model": "flux-general-en"}

{  "error": "websocket_upgrade_required"}

{  "error": "auth_not_initialized"}