Audio

Realtime API

Low-latency speech and text conversations over a WebSocket connection.

WS /v1/realtime

The Realtime API keeps a WebSocket connection open so the client and the model can exchange text and audio events in both directions with low latency — ideal for voice assistants. RuToken relays the OpenAI Realtime protocol (realtime=v1 events) to the upstream provider and bills the token usage reported by each response.

text

wss://<your-host>/v1/realtime?model=gpt-4o-realtime-preview

Replace https:// in https://rutoken.ai with wss:// (or http:// with ws://).

Authentication

Server-side clients send the key in the handshake header Authorization: Bearer sk-xxxx.

Browsers cannot set headers on a WebSocket, so pass the key as a subprotocol instead:

text

Sec-WebSocket-Protocol: realtime, openai-insecure-api-key.sk-xxxx, openai-beta.realtime-v1

A key used in the browser is visible to the user. Only do this with a dedicated, limited key (quota and model restrictions), never with your main key.

Query parameters

Parameter	Type	Required	Description
`model`	string	Yes	Realtime model, e.g. `gpt-4o-realtime-preview`, `gpt-4o-mini-realtime-preview`.

Event flow

The client connects; the server sends session.created.
The client sends session.update to set instructions, voice, modalities and audio formats.
The client sends input — either conversation.item.create with text, or audio chunks with input_audio_buffer.append (Base64 PCM16, 24 kHz, mono) followed by input_audio_buffer.commit (not needed when server-side voice activity detection is on).
The client sends response.create (automatic with server VAD).
The server streams response.text.delta, response.audio.delta and response.audio_transcript.delta events, then response.done with usage.
Errors arrive as error events; the connection stays open.

Client event	Purpose
`session.update`	Change session settings.
`input_audio_buffer.append`	Append Base64 audio to the input buffer.
`input_audio_buffer.commit`	Commit the buffered audio as a user message.
`conversation.item.create`	Add a message (text, audio or function output).
`response.create`	Ask the model to respond.
`response.cancel`	Stop the current response.

Server event	Purpose
`session.created` / `session.updated`	Current session configuration.
`response.text.delta`	Text chunk.
`response.audio.delta`	Base64 audio chunk.
`response.audio_transcript.delta`	Transcript of the generated audio.
`response.done`	Response finished; contains `usage`.
`error`	Something went wrong.

Request example

A minimal text-only conversation:

Node.js

import WebSocket from "ws";

const ws = new WebSocket("wss://<your-host>/v1/realtime?model=gpt-4o-realtime-preview", {
  headers: {
    Authorization: "Bearer sk-xxxx",
    "OpenAI-Beta": "realtime=v1",
  },
});

ws.on("open", () => {
  ws.send(JSON.stringify({
    type: "session.update",
    session: { modalities: ["text"], instructions: "You are a helpful assistant." },
  }));
  ws.send(JSON.stringify({
    type: "conversation.item.create",
    item: { type: "message", role: "user", content: [{ type: "input_text", text: "Say hello in three languages." }] },
  }));
  ws.send(JSON.stringify({ type: "response.create" }));
});

ws.on("message", (raw) => {
  const event = JSON.parse(raw.toString());
  if (event.type === "response.text.delta") process.stdout.write(event.delta);
  if (event.type === "response.done") ws.close();
  if (event.type === "error") console.error(event.error);
});

Python

import asyncio
import json
import websockets  # pip install websockets

URL = "wss://<your-host>/v1/realtime?model=gpt-4o-realtime-preview"
HEADERS = {"Authorization": "Bearer sk-xxxx", "OpenAI-Beta": "realtime=v1"}

async def main():
    # websockets >= 14 uses additional_headers; older versions use extra_headers
    async with websockets.connect(URL, additional_headers=HEADERS) as ws:
        await ws.send(json.dumps({
            "type": "session.update",
            "session": {"modalities": ["text"], "instructions": "You are a helpful assistant."},
        }))
        await ws.send(json.dumps({
            "type": "conversation.item.create",
            "item": {"type": "message", "role": "user",
                     "content": [{"type": "input_text", "text": "Say hello in three languages."}]},
        }))
        await ws.send(json.dumps({"type": "response.create"}))

        async for raw in ws:
            event = json.loads(raw)
            if event["type"] == "response.text.delta":
                print(event["delta"], end="", flush=True)
            elif event["type"] == "response.done":
                break
            elif event["type"] == "error":
                print(event["error"])

asyncio.run(main())

Response

Every server message is a JSON event. The response.done event closes a turn and reports usage:

json

{
  "type": "response.done",
  "event_id": "event_AbC123",
  "response": {
    "id": "resp_XyZ789",
    "status": "completed",
    "output": [
      {
        "type": "message",
        "role": "assistant",
        "content": [{ "type": "text", "text": "Hello! Hola! Bonjour!" }]
      }
    ],
    "usage": {
      "total_tokens": 61,
      "input_tokens": 37,
      "output_tokens": 24,
      "input_token_details": { "text_tokens": 37, "audio_tokens": 0 },
      "output_token_details": { "text_tokens": 24, "audio_tokens": 0 }
    }
  }
}

Notes

Text and audio tokens are billed separately, at the prices shown in the Model Square.
Usage is charged after every response. If the account or key runs out of quota during a session, the session is ended.