Is this page helpful?

Run Streaming Speech-to-Speech Translation#

Speech-to-speech (S2S) translation accepts audio in a source language and returns synthesized audio in a target language. The streaming S2S pipeline combines three Speech NIM microservices:

  1. The ASR NIM transcribes the source audio.

  2. The NMT NIM translates the transcript.

  3. The TTS NIM synthesizes the translated text.

For the integrated streaming API, the NMT NIM coordinates the pipeline. Configure the NMT container with NMT_REMOTE_ASR_SERVICE and NMT_REMOTE_TTS_SERVICE so that it can reach the ASR and TTS gRPC endpoints.

Prerequisites#

Create the Container Network#

Create a cache directory that the three services can share:

export LOCAL_NIM_CACHE=$HOME/.cache/nim
mkdir -p $LOCAL_NIM_CACHE
chmod 777 $LOCAL_NIM_CACHE

The containers can run as different UIDs but share this directory, so each one needs write access to it. Use a dedicated cache directory only; do not point LOCAL_NIM_CACHE to a directory that contains other files.

Save the following configuration as docker-compose.yaml:

services:
  nmt:
    image: nvcr.io/nim/nvidia/riva-translate-1_6b:latest
    container_name: riva-translate-nmt
    runtime: nvidia
    shm_size: 16gb
    environment:
      NGC_API_KEY: ${NGC_API_KEY}
      NIM_HTTP_API_PORT: 9000
      NIM_GRPC_API_PORT: 50051
      NMT_REMOTE_ASR_SERVICE: asr:50052
      NMT_REMOTE_TTS_SERVICE: tts:50053
    ports:
      - "9001:9000"
      - "50051:50051"
    volumes:
      - ${LOCAL_NIM_CACHE}:/opt/nim/.cache
    healthcheck:
      test: ["CMD-SHELL", "/bin/grpc_health_probe -addr=:50051"]
      interval: 10s
      retries: 60
      start_period: 30s
      timeout: 10s
    depends_on:
      asr:
        condition: service_healthy
      tts:
        condition: service_healthy
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              device_ids: ["0"]
              capabilities: [gpu]

  asr:
    image: nvcr.io/nim/nvidia/parakeet-1-1b-ctc-en-us:latest
    container_name: parakeet-ctc-en-us-asr
    runtime: nvidia
    shm_size: 16gb
    environment:
      NGC_API_KEY: ${NGC_API_KEY}
      NIM_HTTP_API_PORT: 9000
      NIM_GRPC_API_PORT: 50052
      NIM_TAGS_SELECTOR: name=parakeet-1-1b-ctc-en-us,mode=str,vad=default,diarizer=disabled
    ports:
      - "9002:9000"
      - "50052:50052"
    volumes:
      - ${LOCAL_NIM_CACHE}:/opt/nim/.cache
    healthcheck:
      test: ["CMD-SHELL", "/bin/grpc_health_probe -addr=:50052"]
      interval: 10s
      retries: 60
      start_period: 30s
      timeout: 10s
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              device_ids: ["0"]
              capabilities: [gpu]

  tts:
    image: nvcr.io/nim/nvidia/magpie-tts-multilingual:latest
    container_name: magpie-tts-multilingual
    runtime: nvidia
    shm_size: 16gb
    environment:
      NGC_API_KEY: ${NGC_API_KEY}
      NIM_HTTP_API_PORT: 9000
      NIM_GRPC_API_PORT: 50053
      NIM_TAGS_SELECTOR: name=magpie-tts-multilingual
    ports:
      - "9003:9000"
      - "50053:50053"
    volumes:
      - ${LOCAL_NIM_CACHE}:/opt/nim/.cache
    healthcheck:
      test: ["CMD-SHELL", "/bin/grpc_health_probe -addr=:50053"]
      interval: 10s
      retries: 60
      start_period: 30s
      timeout: 10s
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              device_ids: ["0"]
              capabilities: [gpu]

The services use distinct internal gRPC ports. The NMT service connects to the ASR service at asr:50052 and the TTS service at tts:50053 through Docker Compose service discovery.

Note

This example assigns all three services to GPU 0. Change device_ids to distribute the services across GPUs when the combined model profiles exceed the memory available on one GPU.

Start the Services#

From the directory that contains docker-compose.yaml, start the network:

docker compose up -d

Model download and initialization can take several minutes. Monitor the services until each one is healthy:

docker compose ps
docker compose logs -f

Press Ctrl+C to stop following the logs without stopping the services.

Run the S2S Notebook#

Download the S2S translation notebook and its English source audio and Spanish reference output. Keep all three files in the same directory, then start Jupyter from that directory.

The notebook connects to the NMT NIM at localhost:50051, streams the English audio, translates it to Latin American Spanish, and plays and saves the synthesized response.

Stop the Services#

Stop and remove the containers and network without deleting the cached models:

docker compose down

Next Steps#