Running NIM on WSL2

This section explains how to set up and run Riva TTS NIM on WSL2, whether the client machine is the same as or different from the WSL2 host. Note that not all Riva TTS NIM models are supported on WSL. Refer Supported Models section to see which models are supported on WSL.

Prerequisites

WSL2 is required for hosting any NIM. Refer to the official NVIDIA NIM on WSL2 documentation for setup instructions.

Make sure you complete the NGC Authentication steps.

Running NIM When the Client and WSL2 Host Are the Same

If the client and WSL2 host machine are the same, follow these steps:

1. Launch NIM on WSL2

Open WSL2 terminal and execute following commands to deploy any of the supported Riva TTS NIM model.

For more information refer Launching the NIM.

export CONTAINER_ID=magpie-tts-flow
export NIM_TAGS_SELECTOR=name=magpie-tts-flow

docker run -it --rm --name=$CONTAINER_ID \
   --runtime=nvidia \
   --gpus '"device=0"' \
   --shm-size=8GB \
   -e NGC_API_KEY \
   -e NIM_HTTP_API_PORT=9000 \
   -e NIM_GRPC_API_PORT=50051 \
   -p 9000:9000 \
   -p 50051:50051 \
   -e NIM_TAGS_SELECTOR \
   nvcr.io/nim/nvidia/$CONTAINER_ID:latest

2. Run Client Inference

When the NIM is running, open a new WSL2 terminal and follow the steps in Running Inference and follow the instructions for respective model.

Running NIM When the Client and WSL2 Host Are Different

If the client is on a different machine than the WSL2 host, you need to set up port forwarding, because WSL2 uses NAT.

The process involves forwarding traffic from your Windows host’s external IP and port to the WSL instance’s internal IP and port. All these steps require a Windows terminal with admin privilege.

1. Find the WSL2 Internal IP

In the WSL2 terminal, execute this command:

hostname -I

2. Forward Traffic from Windows Host to WSL2

Open Windows terminal with admin privilege and run below commands.

Forward HTTP port

netsh interface portproxy add v4tov4 listenport=9000 listenaddress=0.0.0.0 connectport=9000 connectaddress=<wsl2-ip>

Forward gRPC port

netsh interface portproxy add v4tov4 listenport=50051 listenaddress=0.0.0.0 connectport=50051 connectaddress=<wsl2-ip>

Note

WSL2 IP addresses change after every restart, so you need to update the forwarding rule with the correct IP every restart.

3. Verify Port Forwarding

netsh interface portproxy show v4tov4

You should see an output similar to this:

Listen on IPv4:            Connect to IPv4:
Address         Port       Address         Port
--------------- ---------- --------------- ----------
0.0.0.0         9000       <wsl2-ip>       9000
0.0.0.0         50051      <wsl2-ip>       50051

4. Allow Traffic Through Windows Firewall

To allow the client from the other machine proper access, add a firewall rule for Port 8001 in Windows Powershell with admin privilege:

New-NetFirewallRule -DisplayName "Allow gRPC on Port 9000" -Direction Inbound -Action Allow -Protocol TCP -LocalPort 9000
New-NetFirewallRule -DisplayName "Allow gRPC on Port 50051" -Direction Inbound -Action Allow -Protocol TCP -LocalPort 50051

5. Run the Client Inference

Follow the steps in Run Client Inference. Make sure to change the IP address in the --server arguments to the Windows host machine IP, which is running the WSL2 NIM.