Is this page helpful?

Troubleshooting NVIDIA ASR NIM Microservice Issues#

This page covers troubleshooting issues specific to the NVIDIA ASR NIM microservices. For issues shared across all NVIDIA Speech NIM microservices, see Common Issues.

Idle Stream Sequence Error#

Symptom#

Server logs show an error requiring the START flag on the first request of the sequence.

Cause#

The error occurs when:

A streaming sequence is idle (no audio chunks received) longer than the configured max_sequence_idle_microseconds timeout.
The server automatically releases the idle sequence.
The client sends a new audio chunk for the already-released sequence.
The server rejects the request because it expects a START flag for what it now considers a new sequence, but the client does not provide it.

The stream timeout in the NIM HTTP layer is set to 60 seconds (stream_timeout=60000000 microseconds).

Solution#

Option 1: Implement Client-Side Mitigations (Recommended)#

A. Send Silence Buffers During Pauses#

If microphone is muted but stream should stay active:

def handle_mic_mute():
    while mic_is_muted and stream_is_active:
        send_silence_buffer(stream_id)
        time.sleep(30.0)  # Send every 30s

B. Stream Timeout Mechanism#

Implement a fail-safe to detect and handle idle streams:

STREAM_TIMEOUT = 30  # seconds

def monitor_stream_activity(stream_id, last_activity_time):
    idle_duration = time.time() - last_activity_time
    if idle_duration > STREAM_TIMEOUT:
        send_stream_end_signal(stream_id)
        return new_stream_id()

Option 2: Increase Idle Timeout (Server-Side)#

During Model Build#

Set the parameter during riva-build:

riva-build ... --asr_ensemble_backend.max_sequence_idle_microseconds=120000000

Configuration Edit in the Existing Model Repository#

Edit models/conformer-<LANG>-asr-streaming-*-asr-bls-ensemble/config.pbtxt:

parameters: {
  key: "max_sequence_idle_microseconds"
  value: {
    string_value: "120000000"  # 120 seconds (doubled from default 60s)
  }
}

Restart the RIVA server after saving the configuration changes.

Too Many Open Files Error#

Symptom#

The NIM container fails to start with an error such as: OSError: [Errno 24] Too many open files: '/tmp/tmp64in_90a'

Cause#

The NIM container starts with a too-low file-descriptor limit because the correct ulimit was not propagated from the host system to the container.

Solution#

Add --ulimit nofile=2048:2048 to the docker run command, where 2048:2048 represents the soft and hard limits for the number of open file descriptors.

docker run --ulimit nofile=2048:2048 [other options] <image>

If the error persists, increase the limit further (for example, 4096:4096). Ensure the host hard limit supports the value you set.

Audio File Too Large#

Symptom#

The HTTP transcription request returns 400 Bad Request with the message audio too long.

Cause#

The NIM enforces a maximum audio file size of 25 MB per request. Files exceeding this limit are rejected before processing.

Solution#

Ensure the audio file is under 25 MB. Check the file size before uploading:
```
ls -lh audio.wav
```

For large audio files, split them into smaller segments:

sox input.wav output_%03d.wav trim 0 60 : newp : restart

Use a compressed audio format (OPUS or FLAC) instead of uncompressed WAV to reduce file size while maintaining quality.

Model Not Found for Language#

Symptom#

The transcription request returns 404 Not Found with the message Model not found for language <language_code>.

Cause#

The specified language code does not match any deployed model. The NIM first attempts an exact match on the full language code (for example, en-US), then falls back to matching the base language code (for example, en). If neither matches a deployed model, the request fails.

Solution#

Verify the language code matches the deployed model. Confirm which models are loaded by checking the container startup logs for lines containing Found ASR model. Alternatively, use the --list-models option with the Riva Python client script for ASR to query the deployed models on the server:
```
python python-clients/scripts/asr/transcribe_file.py --list-models --server localhost:50051
```
Use the correct language code format. Common formats:

Language

Code

English (US)

en-US

Simplified Chinese

zh-CN

Spanish

es-ES or es-US
If the model or language parameter is omitted entirely, the request returns 400 Bad Request, need model or language. Provide at least one.

Language	Code
English (US)	`en-US`
Simplified Chinese	`zh-CN`
Spanish	`es-ES` or `es-US`