Realtime API Reference#

Overview#

Riva Realtime Server provides a WebSocket-based API for real-time speech processing (ASR - Automatic Speech Recognition). This API allows you to stream audio data and receive real-time transcription results.

Reference#

The WebSocket server provides real-time communication capabilities for transcription services. To establish a connection, clients must connect to the WebSocket endpoint with the required query parameter.

Transcription Sessions Endpoint#

The transcription sessions endpoint allows you to create transcription sessions.

Base URL:

http://<address>:9000

Endpoint:

/v1/realtime/transcription_sessions

Method: POST

Mint API Key Authentication#

This endpoint issues the short-lived client_secret that authenticates the WebSocket handshake, so it is the boundary you protect with a long-lived secret. Enforcement is controlled server-side by the REALTIME_AUTH_MINT_API_KEY environment variable:

Key set on the server: every request must present the same key, using either an Authorization: Bearer <key> or an X-API-Key: <key> header. The server compares the value in constant time. Missing or incorrect credentials return 401 Unauthorized with a WWW-Authenticate: Bearer realm="riva-realtime-mint" response header.
Key unset on the server (default): the endpoint is unauthenticated.

curl -X POST http://localhost:9000/v1/realtime/transcription_sessions \
    -H "Authorization: Bearer $REALTIME_AUTH_MINT_API_KEY"

Warning

REALTIME_AUTH_MINT_API_KEY is a backend-only secret. Only your session-brokering service should hold it; never ship it to a browser or embed it in a client application. Browsers authenticate the WebSocket handshake with the short-lived client_secret.value that this endpoint returns, described in Ephemeral Token Authentication.

Response: Returns the initial default transcription session configuration.

{
  "id": "sess_<uuid4>",
  "object": "realtime.transcription_session",
  "modalities": ["text"],
  "input_audio_format": "pcm16",
  "input_audio_transcription":
  {
      "language": "en-US",
      "model": "conformer",
      "prompt": ""
  },
  "input_audio_params":
  {
      "sample_rate_hz": 16000,
      "num_channels": 1
  },
  "recognition_config":
  {
      "max_alternatives": 1,
      "enable_automatic_punctuation": false,
      "enable_word_time_offsets": false,
      "enable_profanity_filter": false,
      "enable_verbatim_transcripts": false
  },
  "speaker_diarization":
  {
      "enable_speaker_diarization": false,
      "max_speaker_count": 8
  },
  "word_boosting":
  {
      "enable_word_boosting": false,
      "word_boosting_list": []
  },
  "endpointing_config":
  {
      "start_history": 0,
      "start_threshold": 0,
      "stop_history": 0,
      "stop_threshold": 0,
      "stop_history_eou": 0,
      "stop_threshold_eou": 0
  },
  "client_secret": {
    "value": "<uuid4>",
    "expires_at": 1753872000
  }
}

The client_secret field is an object that carries the ephemeral token for the WebSocket handshake, or null when the server does not mint tokens. value is the single-use token to present on the handshake, and expires_at is the Unix timestamp at which it expires, 60 seconds after it is minted.

Parameters#

Parameter	Type	Required	Description	Default
id	string	No	Session identifier	auto-generated (“sess_”)
object	string	Yes	Object type identifier	“realtime.transcription_session”
modalities	array	Yes	List of supported modalities	[“text”]
input_audio_format	string	Yes	Audio format. Currently only “pcm16” supported	“pcm16”
input_audio_transcription.language	string	No	Transcription language	“en-US”
input_audio_transcription.model	string	No	ASR model to use	“conformer”
input_audio_transcription.prompt	string	No	Optional prompt for transcription	“”
input_audio_params.sample_rate_hz	integer	No	Audio sample rate in Hz	16000
input_audio_params.num_channels	integer	No	Number of audio channels	1
recognition_config.max_alternatives	integer	No	Maximum number of recognition alternatives	1
recognition_config.enable_automatic_punctuation	boolean	No	Enable automatic punctuation	false
recognition_config.enable_word_time_offsets	boolean	No	Enable word-level timing information	false
recognition_config.enable_profanity_filter	boolean	No	Enable profanity filtering	false
recognition_config.enable_verbatim_transcripts	boolean	No	Enable verbatim transcription	false
speaker_diarization.enable_speaker_diarization	boolean	No	Enable speaker diarization	false
speaker_diarization.max_speaker_count	integer	No	Maximum number of speakers to detect	8
word_boosting.enable_word_boosting	boolean	No	Enable word boosting	false
word_boosting.word_boosting_list	array	No	List of words to boost	[]
endpointing_config.start_history	integer	No	Start history for endpointing	0
endpointing_config.start_threshold	integer	No	Start threshold for endpointing	0
endpointing_config.stop_history	integer	No	Stop history for endpointing	0
endpointing_config.stop_threshold	integer	No	Stop threshold for endpointing	0
endpointing_config.stop_history_eou	integer	No	Stop history for end-of-utterance	0
endpointing_config.stop_threshold_eou	integer	No	Stop threshold for end-of-utterance	0
client_secret	object \| null	No	Ephemeral WebSocket token object (`value`, `expires_at`), or `null` when the server does not mint tokens	null
client_secret.value	string	No	Single-use token presented on the WebSocket handshake	—
client_secret.expires_at	integer	No	Unix timestamp when the token expires (60 seconds after mint)	—

WebSocket Connection Details#

Base URL:

ws://<address>:9000

Endpoint:

/v1/realtime

Required Query Parameter:

intent=transcription

Ephemeral Token Authentication#

The handshake is authenticated with the client_secret.value returned by the transcription sessions endpoint. The token travels on the Sec-WebSocket-Protocol header, where the client offers two subprotocol entries:

Sec-WebSocket-Protocol: realtime, realtime-token.<client_secret.value>

The server validates the realtime-token.<value> entry and echoes back only the plain realtime identifier, so the secret never appears in a response header. This is also what makes the flow work from a browser, which cannot set arbitrary headers on a WebSocket upgrade but can supply a subprotocol list:

new WebSocket(
  "wss://localhost:9000/v1/realtime?intent=transcription",
  ["realtime", `realtime-token.${clientSecret}`]
);

Whether a token is required is controlled server-side by the REALTIME_AUTH_REQUIRE_TOKEN environment variable:

REALTIME_AUTH_REQUIRE_TOKEN=true: every handshake must present a valid, unused, unexpired token. This is the recommended posture for any deployment reachable from an untrusted network.
Unset or false (default): handshakes without a token are accepted, preserving compatibility with client fleets that predate the token flow.

Important

A token that is presented is always validated, even when REALTIME_AUTH_REQUIRE_TOKEN is false. The flag controls whether an absent token is tolerated, not whether a supplied token is checked.

Tokens are single-use, expire 60 seconds after they are minted, and are bound to the intent they were minted for, so a token from the transcription endpoint cannot be used on a synthesize connection.

The following conditions close the connection with code 1008 (Policy Violation):

Condition	Cause
Missing token	`REALTIME_AUTH_REQUIRE_TOKEN=true` and no token was offered
Invalid token	The token is unknown, expired, or was already consumed
Intent mismatch	The token was minted for a different intent
Unsafe subprotocol offer	A token entry was offered without a non-token carrier entry such as `realtime`

Note

The last condition is rejected deliberately. With no non-token entry available to echo, the server would have to return the token itself in the Sec-WebSocket-Protocol response header, exposing it to browser developer tools, HAR captures, and any reverse proxy that records response headers.

URL query-string tokens (?token=<value>) are never accepted, regardless of the flag, because they routinely end up in reverse-proxy and load-balancer access logs, a category of exposure that request-header values do not share.

Example Connection URL#

Here is a complete example for connecting to the WebSocket server:

ws://localhost:9000/v1/realtime?intent=transcription

The client_secret.value is supplied in the Sec-WebSocket-Protocol handshake header, never in the URL.

Connection Requirements#

To establish a connection, the WebSocket client must include the intent query parameter in the URL, specifying a supported value. At present, the only valid intent is “transcription”. The server listens on port 9000 by default and uses the standard WebSocket protocol (ws://). If the intent parameter is missing or invalid, the server will close the connection and return WebSocket code 1008 (Policy Violation).

When the server runs with REALTIME_AUTH_REQUIRE_TOKEN=true, the client must additionally present a valid ephemeral client_secret.value on the Sec-WebSocket-Protocol header, as described in Ephemeral Token Authentication. Connections that omit the token, or that supply an invalid, expired, or already-used one, are closed with the same 1008 code.

Usage Notes#

Clients should support WebSocket connections and maintain the connection for the entire duration of the transcription session. It is important to implement proper error handling and reconnection logic within the client to ensure a robust and reliable experience.

Because tokens are single-use and expire 60 seconds after they are minted, clients should request a fresh client_secret immediately before opening each socket rather than caching one for reuse. Reconnect logic needs its own mint call. The server never writes the raw token value to its application logs, and because the token is carried on a request header rather than in the URL, it also stays out of reverse-proxy and load-balancer access logs that record URLs but not header values.

Health Check Endpoint#

The health check endpoint provides a way to verify the server’s operational status.

Endpoint:

/v1/health

Method: GET

Response:

{
  "status": "ok"
}

Status Codes:

200 OK: Server is healthy and ready to accept connections
503 Service Unavailable: Server is not ready to accept connections

Use Cases:

Pre-flight check before establishing WebSocket connections
Load balancer health monitoring
System status monitoring

Events#

WebSocket Events#

The realtime server uses a WebSocket-based event system for communication between clients and the server. Events are JSON messages that follow a specific format and are used to handle various operations like session management, audio processing, and transcription.

Each event has:

A unique event_id for tracking
A type field indicating the event type
Additional fields specific to the event type

Events are categorized into:

Client Events: Events sent from client to server
- Session management (create, update)
- Audio buffer operations (append, commit)
Server Events: Events sent from server to client
- Session responses (created, updated)
- Transcription results (delta, completed, failed)
- Error notifications
- Status updates

The server validates all incoming events and sends appropriate error messages for:

Invalid event formats
Unsupported features
Message size limits
Server errors

Client Events#

Events that can be sent from the client to the server.

List of Client Events#

Event Type	Description
`transcription_session.update`	Updates session configuration
`input_audio_buffer.append`	Sends audio data for processing
`input_audio_buffer.commit`	Commits the current audio buffer
`input_audio_buffer.clear`	Clears the audio bytes in the buffer
`input_audio_buffer.done`	Tells the server that the client is done sending audio data

transcription_session.update#

Send this event to update a transcription session.

{
    "event_id" : "event_<uuid4>",
    "session":
    {
        "modalities": ["text"],
        "input_audio_format": "pcm16",
        "input_audio_transcription":
        {
            "language": "en-US",
            "model": "conformer",
            "prompt": ""
        },
        "input_audio_params":
        {
            "sample_rate_hz": 16000,
            "num_channels": 1
        },
        "recognition_config":
        {
            "max_alternatives": 1,
            "enable_automatic_punctuation": false,
            "enable_word_time_offsets": false,
            "enable_profanity_filter": false,
            "enable_verbatim_transcripts": false
        },
        "speaker_diarization":
        {
            "enable_speaker_diarization": false,
            "max_speaker_count": 8
        },
        "word_boosting":
        {
            "enable_word_boosting": false,
            "word_boosting_list": []
        },
        "endpointing_config":
        {
            "start_history": 0,
            "start_threshold": 0,
            "stop_history": 0,
            "stop_threshold": 0,
            "stop_history_eou": 0,
            "stop_threshold_eou": 0
        }
    },
    "type": "transcription_session.update"
}

Parameters#

Parameter	Type	Required	Description	Default
event_id	string	No	Event identifier	auto-generated (“conv_uuid4”)
type	string	Yes	Event type	“transcription_session.update”
session.modalities	array	Yes	List of supported modalities	[“text”]
session.input_audio_format	string	Yes	Audio format. Currently only “pcm16” is supported	“pcm16”
session.input_audio_transcription.language	string	No	Transcription language	“en-US”
session.input_audio_transcription.model	string	No	ASR model to use	“conformer”
session.input_audio_transcription.prompt	string	No	Optional prompt for transcription	“”
session.input_audio_params.sample_rate_hz	integer	No	Audio sample rate in Hz	16000
session.input_audio_params.num_channels	integer	No	Number of audio channels	1
session.recognition_config.max_alternatives	integer	No	Maximum number of recognition alternatives	1
session.recognition_config.enable_automatic_punctuation	boolean	No	Enable automatic punctuation	false
session.recognition_config.enable_word_time_offsets	boolean	No	Enable word-level timing information	false
session.recognition_config.enable_profanity_filter	boolean	No	Enable profanity filtering	false
session.recognition_config.enable_verbatim_transcripts	boolean	No	Enable verbatim transcription	false
session.speaker_diarization.enable_speaker_diarization	boolean	No	Enable speaker diarization	false
session.speaker_diarization.max_speaker_count	integer	No	Maximum number of speakers to detect	8
session.word_boosting.enable_word_boosting	boolean	No	Enable word boosting	false
session.word_boosting.word_boosting_list	array	No	List of words to boost	[]
session.endpointing_config.start_history	integer	No	Start history for endpointing	0
session.endpointing_config.start_threshold	integer	No	Start threshold for endpointing	0
session.endpointing_config.stop_history	integer	No	Stop history for endpointing	0
session.endpointing_config.stop_threshold	integer	No	Stop threshold for endpointing	0
session.endpointing_config.stop_history_eou	integer	No	Stop history for end-of-utterance	0
session.endpointing_config.stop_threshold_eou	integer	No	Stop threshold for end-of-utterance	0

input_audio_buffer.append#

Sends audio data to the server for processing.

{
  "event_id": "event_0000",
  "type": "input_audio_buffer.append",
  "audio": "<Base64EncodedAudioData>"
}

Parameters#

Parameter	Type	Required	Description	Default
event_id	string	No	Optional event identifier	event_0000
type	string	Yes	Event type	“input_audio_buffer.append”
audio	string	Yes	Base64-encoded audio data. Maximum size: 15MB	-

input_audio_buffer.commit#

Commits the current audio buffer for processing.

{
  "event_id": "event_0000",
  "type": "input_audio_buffer.commit"
}

Parameters#

Parameter	Type	Required	Description	Default
event_id	string	No	Optional event identifier	event_0000
type	string	Yes	Event type	“input_audio_buffer.commit”

input_audio_buffer.done#

Tells the server that the client is done sending audio data and wants to stop the inference processing. This event triggers the server to process any remaining audio chunks in the buffer and then stop the inference task.

Note

This parameter is mandatory for audio file processing.

{
    "event_id": "event_0000",
    "type": "input_audio_buffer.done"
}

Parameters#

Parameter	Type	Required	Description	Default
event_id	string	No	Optional event identifier	event_0000
type	string	Yes	Event type	“input_audio_buffer.done”

input_audio_buffer.clear#

Clears the current audio buffer.

{
  "event_id": "event_0000",
  "type": "input_audio_buffer.clear"
}

Parameters#

Parameter	Type	Required	Description	Default
event_id	string	No	Optional event identifier	event_0000
type	string	Yes	Event type	“input_audio_buffer.clear”

Server Events#

These are events emitted from the server to the client.

List of Server Events#

Event Type	Description
`conversation.created`	Returned when a conversation is created
`transcription_session.updated`	Sent when session configuration is updated
`input_audio_buffer.committed`	Returned when an input audio buffer is committed
`input_audio_buffer.cleared`	Returned when the input audio buffer is cleared
`conversation.item.input_audio_transcription.delta`	Sent when new transcription results are available
`conversation.item.input_audio_transcription.completed`	Sent when new transcription results are available
`conversation.item.input_audio_transcription.failed`	Sent when transcription fails
`error`	Sent when an error occurs

conversation.created#

Returned when a conversation session is created.

{
    "event_id": "event_<uuid4>",
    "type": "conversation.created",
    "conversation": {
        "id": "conv_<uuid4>",
        "object": "realtime.conversation"
    }
}

Parameters#

Parameter	Type	Required	Description	Default
event_id	string	No	Optional event identifier	auto-generated (“event_uuid4”)
type	string	Yes	Event type	“conversation.created”
conversation.id	string	Yes	The unique ID of the conversation	auto-generated (“conv_uuid4”)
conversation.object	string	Yes	Must be ‘realtime.conversation’	“realtime.conversation”

transcription_session.updated#

Returned when a transcription session is updated.

{
    "event_id" : "event_<uuid4>",
    "session":
    {
        "modalities": ["text"],
        "input_audio_format": "pcm16",
        "input_audio_transcription":
        {
            "language": "en-US",
            "model": "conformer",
            "prompt": ""
        },
        "input_audio_params":
        {
            "sample_rate_hz": 16000,
            "num_channels": 1
        },
        "recognition_config":
        {
            "max_alternatives": 1,
            "enable_automatic_punctuation": false,
            "enable_word_time_offsets": false,
            "enable_profanity_filter": false,
            "enable_verbatim_transcripts": false
        },
        "speaker_diarization":
        {
            "enable_speaker_diarization": false,
            "max_speaker_count": 8
        },
        "word_boosting":
        {
            "enable_word_boosting": false,
            "word_boosting_list": []
        },
        "endpointing_config":
        {
            "start_history": 0,
            "start_threshold": 0,
            "stop_history": 0,
            "stop_threshold": 0,
            "stop_history_eou": 0,
            "stop_threshold_eou": 0
        }
    },
    "type": "transcription_session.updated"
}

Parameters#

Parameter	Type	Required	Description	Default
event_id	string	No	Event identifier	auto-generated (“event_”)
type	string	Yes	Event type	“transcription_session.updated”
session.modalities	array	Yes	List of supported modalities	[“text”]
session.input_audio_format	string	Yes	Audio format. Currently only “pcm16” supported	“pcm16”
session.input_audio_transcription.language	string	No	Transcription language	“en-US”
session.input_audio_transcription.model	string	No	ASR model to use	“conformer”
session.input_audio_transcription.prompt	string	No	Optional prompt for transcription	“”
session.input_audio_params.sample_rate_hz	integer	No	Audio sample rate in Hz	16000
session.input_audio_params.num_channels	integer	No	Number of audio channels	1
session.recognition_config.max_alternatives	integer	No	Maximum number of recognition alternatives	1
session.recognition_config.enable_automatic_punctuation	boolean	No	Enable automatic punctuation	false
session.recognition_config.enable_word_time_offsets	boolean	No	Enable word-level timing information	false
session.recognition_config.enable_profanity_filter	boolean	No	Enable profanity filtering	false
session.recognition_config.enable_verbatim_transcripts	boolean	No	Enable verbatim transcription	false
session.speaker_diarization.enable_speaker_diarization	boolean	No	Enable speaker diarization	false
session.speaker_diarization.max_speaker_count	integer	No	Maximum number of speakers to detect	8
session.word_boosting.enable_word_boosting	boolean	No	Enable word boosting	false
session.word_boosting.word_boosting_list	array	No	List of words to boost	[]
session.endpointing_config.start_history	integer	No	Start history for endpointing	0
session.endpointing_config.start_threshold	integer	No	Start threshold for endpointing	0
session.endpointing_config.stop_history	integer	No	Stop history for endpointing	0
session.endpointing_config.stop_threshold	integer	No	Stop threshold for endpointing	0
session.endpointing_config.stop_history_eou	integer	No	Stop history for end-of-utterance	0
session.endpointing_config.stop_threshold_eou	integer	No	Stop threshold for end-of-utterance	0

input_audio_buffer.committed#

Returned when an input audio buffer is committed. At the same time, the buffer is sent for inference.

{
    "event_id" : "event_0000",
    "type": "input_audio_buffer.committed",
    "previous_item_id": "msg_0000",
    "item_id": "msg_0001"
}

Parameters#

Parameter	Type	Required	Description	Default
event_id	string	No	Optional event identifier	event_0000
type	string	Yes	Event type	“input_audio_buffer.committed”
previous_item_id	string	No	ID of the preceding item	msg_0000
item_id	string	No	ID of the current item	msg_0001

input_audio_buffer.cleared#

Returned when the input audio buffer is cleared.

{
    "event_id": "event_0000",
    "type": "input_audio_buffer.cleared"
}

Parameters#

Parameter	Type	Required	Description	Default
event_id	string	No	Optional event identifier	event_0000
type	string	Yes	Event type	“input_audio_buffer.cleared”

conversation.item.input_audio_transcription.delta#

Returned transcription text, when response received from gRPC server

{
    "event_id": "event_0000",
    "type": "conversation.item.input_audio_transcription.delta",
    "item_id": "item_001",
    "content_index": 0,
    "delta": "Hello"
}

Parameters#

Parameter	Type	Required	Description	Default
event_id	string	No	Optional event identifier	event_0000
type	string	Yes	Event type	“conversation.item.input_audio_transcription.delta”
item_id	string	No	Optional item identifier	item_0000
content_index	integer	No	The index of the content part	0
delta	string	Yes	Transcription result in streaming mode	“”

conversation.item.input_audio_transcription.completed#

Returns transcription text when a response is received from the gRPC server.

{
    "event_id": "event_0000",
    "type": "conversation.item.input_audio_transcription.completed",
    "item_id": "msg_0000",
    "content_index": 0,
    "transcript": "Hello, how are you?",
    "words_info": {
        "words": [
            {
                "word": "Hello",
                "start_time": 0.0,
                "end_time": 1.0,
                "confidence": 0.95,
                "speaker_tag": 0
            }
        ]
    },
    "vad_states": {
        "vad_states": [
            {
                "timestamp": 0.0,
                "prob": 0.5
            }
        ]
    },
    "is_last_result" : false
}

Parameters#

Parameter	Type	Required	Description	Default
event_id	string	No	Optional event identifier	event_0000
type	string	Yes	Event type	“conversation.item.input_audio_transcription.completed”
item_id	string	Yes	The ID of the item	msg_0000
content_index	integer	Yes	The index of the content part	0
transcript	string	Yes	The complete transcribed text	-
words_info	object	No	Word-level information container	-
words_info.words	array	No	Array of word objects with timing and confidence	[]
words_info.words[].word	string	Yes	The transcribed word	-
words_info.words[].start_time	float	Yes	Start time of the word in seconds	-
words_info.words[].end_time	float	Yes	End time of the word in seconds	-
words_info.words[].confidence	float	Yes	Confidence score for the word (0.0-1.0)	-
words_info.words[].speaker_tag	integer	Yes	Speaker identifier for diarization	0
vad_states	object	No	Voice Activity Detection states container	-
vad_states.vad_states	array	No	Array of VAD state objects	[]
vad_states.vad_states[].timestamp	float	Yes	Timestamp in seconds	-
vad_states.vad_states[].prob	float	Yes	VAD probability (0.0-1.0)	-
is_last_result	boolean	No	Indicates if this is the final transcription result for the audio stream	false

conversation.item.input_audio_transcription.failed#

Returned when a transcription request fails.

{
    "event_id": "event_0000",
    "type": "conversation.item.input_audio_transcription.failed",
    "item_id": "msg_0000",
    "content_index": 0,
    "error": {
        "type": "transcription_error",
        "code": "audio_unintelligible",
        "message": "The audio could not be transcribed.",
        "param": null
    }
}

Parameters#

Parameter	Type	Required	Description
event_id	string	No	Optional event identifier
type	string	Yes	Must be ‘conversation.item.input_audio_transcription.failed’
item_id	string	Yes	The ID of the user message item
content_index	integer	Yes	The index of the content part
error.type	string	Yes	The type of error
error.code	string	Yes	Error code
error.message	string	Yes	A human-readable error message
error.param	string	No	Parameter related to the error, if any

error#

Returned when an error occurs.

{
    "event_id": "<auto_generated>",
    "type": "error",
    "error": {
        "type": "invalid_request_error",
        "code": "invalid_event",
        "message": "The 'type' field is missing.",
        "param": null
    }
}

Parameters#

Parameter	Type	Required	Description
event_id	string	No	Optional event identifier
type	string	Yes	Must be ‘error’
error.type	string	Yes	The type of error
error.code	string	Yes	Error code
error.message	string	Yes	A human-readable error message
error.param	string	No	Parameter related to the error, if any

Configuration#

Server Parameters#

Parameter	Default Value	Description
expiration_timeout_secs	3600	Session expiration timeout in seconds (1 hour)
inactivity_timeout_secs	60	Inactivity timeout in seconds
max_connections	100	Maximum number of concurrent connections
max_message_size	10485760	Maximum message size in bytes (10MB)
input_min_chunk_seconds	0.08	Minimum audio chunk size for processing

Authentication Environment Variables#

Variable	Type	Default	Description
`REALTIME_AUTH_MINT_API_KEY`	string	unset	Static key required on `POST /v1/realtime/transcription_sessions`. When unset, the endpoint is unauthenticated. See Mint API Key Authentication.
`REALTIME_AUTH_REQUIRE_TOKEN`	boolean	`false`	Require an ephemeral `client_secret.value` on every `/v1/realtime` handshake. See Ephemeral Token Authentication.

Boolean values accept 1, true, yes, or on, case-insensitively. Both variables take precedence over the server configuration file, so you can change the authentication posture at docker run time without rebuilding the image. The 60-second token lifetime is fixed and cannot be configured through the environment.

The two variables are independent, which gives four postures:

`REALTIME_AUTH_MINT_API_KEY`	`REALTIME_AUTH_REQUIRE_TOKEN`	Behavior	Suitable for
unset	`false`	Anyone can mint a session and connect anonymously	Local development on a trusted network
set	`false`	Minting requires the key; sockets still accept anonymous clients	Migrating an existing client fleet incrementally
unset	`true`	Sockets require a token, but anyone can mint one	Blocking anonymous sockets without a broker in place
set	`true`	Minting requires the key and sockets require a fresh token	Production and browser-facing deployments

Launching the NIM with Authentication Enabled#

Pass both variables with -e flags alongside the standard runtime parameters described in Runtime Parameters for Speech NIM Containers. Refer to the ASR support matrix to choose the values for CONTAINER_ID and NIM_TAGS_SELECTOR, and to Run Your First NVIDIA Speech NIM Microservice for Automatic Speech Recognition for a full deployment walkthrough.

The following example enables both tiers, the recommended posture for a deployment reachable from an untrusted network:

export CONTAINER_ID=parakeet-1-1b-ctc-en-us
export NIM_TAGS_SELECTOR="name=parakeet-1-1b-ctc-en-us,mode=all"

# Long-lived secret, held only by your session-brokering backend.
export REALTIME_AUTH_MINT_API_KEY=$(openssl rand -hex 32)

docker run -it --rm --name=$CONTAINER_ID \
    --runtime=nvidia \
    --gpus '"device=0"' \
    --shm-size=8GB \
    -e NGC_API_KEY \
    -e NIM_TAGS_SELECTOR \
    -e NIM_HTTP_API_PORT=9000 \
    -e NIM_GRPC_API_PORT=50051 \
    -e REALTIME_AUTH_REQUIRE_TOKEN=true \
    -e REALTIME_AUTH_MINT_API_KEY \
    -p 9000:9000 \
    -p 50051:50051 \
    nvcr.io/nim/nvidia/$CONTAINER_ID:latest

Note

Passing -e REALTIME_AUTH_MINT_API_KEY without a value makes Docker inherit it from your shell, which keeps the secret out of your shell history and the host process list. The value is still visible via docker inspect, so on orchestrated deployments prefer a Kubernetes secret or an equivalent secret store over a literal value in a manifest.

To verify the deployment end to end, mint a session with the backend key and then open the socket with the returned token inside the 60-second window:

export TOKEN=$(curl -sS -X POST http://localhost:9000/v1/realtime/transcription_sessions \
    -H "Authorization: Bearer $REALTIME_AUTH_MINT_API_KEY" \
    | python3 -c 'import json, sys; print(json.load(sys.stdin)["client_secret"]["value"])')

python3 - <<'PY'
import asyncio, os, websockets

async def main():
    token = os.environ["TOKEN"]
    async with websockets.connect(
        "ws://localhost:9000/v1/realtime?intent=transcription",
        subprotocols=["realtime", f"realtime-token.{token}"],
    ) as ws:
        print("connected, negotiated subprotocol:", ws.subprotocol)

asyncio.run(main())
PY

A successful run prints realtime as the negotiated subprotocol, confirming that the server accepted the token without echoing it back.

Error Handling#

The realtime server implements comprehensive error handling for various scenarios:

WebSocket Error Codes#

Code	Description	Action
1000	Normal closure	Connection closed normally
1008	Policy violation	Invalid intent, failed authentication, or unsupported operation
1011	Internal error	Server encountered an error
1013	Try again later	Server temporarily unavailable

Common Error Scenarios#

Invalid Intent: Connection closed with code 1008 if intent is missing or unsupported
Message Size Limits: Errors returned for messages exceeding 10MB limit
Audio Data Issues: Validation errors for malformed or unsupported audio formats
Session Timeout: Connections closed after inactivity timeout (60 seconds default)
Server Overload: Connection refused when max connections (100) is reached
Missing or Invalid Token: Connection closed with code 1008 when REALTIME_AUTH_REQUIRE_TOKEN=true and no token is offered, or when the supplied token is unknown, expired, already consumed, or minted for a different intent
Unauthorized Mint Request: 401 Unauthorized returned when REALTIME_AUTH_MINT_API_KEY is set on the server and the request presents no key or the wrong one

Authentication failures on the mint endpoint are HTTP rather than WebSocket errors, and return 401 Unauthorized with a WWW-Authenticate: Bearer realm="riva-realtime-mint" header.

Error Response Format#

All errors follow the standard error event format:

{
    "event_id": "event_0000",
    "type": "error",
    "error": {
        "type": "error_type",
        "code": "error_code",
        "message": "Human-readable error message",
        "param": "Additional parameter if applicable"
    }
}

Client Development Resources#

For building realtime WebSocket clients in Python, refer to the NVIDIA Riva Python Clients repository.

Quick Start#

git clone https://github.com/nvidia-riva/python-clients.git
pip install -r requirements.txt
python scripts/asr/realtime_asr_client.py --help

See the repository for complete examples and API documentation.