Remote Locations

A Remote Location is a remote machine, like a virtual machine (VM) or server, that you connect to over a network. AI Workbench must be installed on the machine to use it as a Remote Location.

You can use a remote machine as a location for your AI Workbench projects. Currently, the only supported OS for remote locations is Ubuntu 22.04.

To set up your remote machine, follow these steps:

• Make sure you have SSH access configured for your user account on this system using key-based authentication.

• Ensure that your SSH key does not have a password.

• Properly install the AI Workbench server on the remote machine. The easiest way to do this is to run the CLI-based installers.

• Once you’ve completed these steps, you’ll be able to add your remote machine as a location as described below.

You can create locations in either the Desktop app or the CLI and see the change reflected in both interfaces.

Desktop

To create a new location in the Desktop app, there are two places to start:

1. Open a new window, which will start on the “Locations” screen. Click the “Add Remote Location” button in the top right corner.

2. Alternatively, you can start from an open AI Workbench window by clicking on the location widget in the top right corner. Once expanded, click on the green “Add Location” button.

When you click the “Add Location” button, a modal window will open. Fill out the form with the following information:

• Name: Enter a unique name for the location, without spaces or special characters. This name will also be used to activate the location in the CLI.

• Description: Provide an optional description to help identify the machine.

• Hostname or IP address: Enter the hostname or IP address of the remote machine.

• SSH port: The default SSH port is 22, but you can change this if you have a custom SSH server configuration.

• SSH username: Enter the username on the remote machine. This is the username you see when you manually SSH into the machine.

• SSH key file: Provide the absolute path to your SSH key file on your local system. Typically, this will be in the hidden .ssh directory in your home directory. You can use the file picker to select the file.

• Workbench Directory: The working directory for the AI Workbench service on the remote system. If you performed a default installation, this will auto-populate based on your SSH username and you shouldn’t have to change it.

Once you’ve filled out the form, click “Add Location”. This will attempt to SSH to the remote system and verify that the AI Workbench server is installed.

CLI

To create a new location in CLI use the nvwb create context command. You can specify all flags or simply run the command with no arguments to activate the interactive form, similar to the Desktop app.

You must provide the following information:

• Name: Enter a unique name for the location, without spaces or special characters. This name will also be used to activate the location in the CLI.

• Description: Provide an optional description to help identify the machine.

• Hostname or IP address: Enter the hostname or IP address of the remote machine.

• SSH port: The default SSH port is 22, but you can change this if you have a custom SSH server configuration.

• SSH username: Enter the username on the remote machine. This is the username you see when you manually SSH into the machine.

• SSH key file: Provide the absolute path to your SSH key file on your local system. Typically, this will be in the hidden .ssh directory in your home directory. You can use the file picker to select the file.

• Workbench Directory: The working directory for the AI Workbench service on the remote system. If you performed a default installation, this will auto-populate based on your SSH username and you shouldn’t have to change it.

The CLI will attempt to SSH to the remote system and verify that the AI Workbench server is installed before adding the location.

When using AI Workbench you always work in a Location. A Location represents a single machine where your Project files and containers are stored and run. You can move Projects between locations as needed.

Whether the local or a remote Location, when you “connect” or “activate” a Location, AI Workbench will:

1. Ensure the server version is compatible with your client.

2. Start the AI Workbench service if it is not already running

3. Start the container runtime if needed

4. Push your credentials into AI Workbench server memory

5. Push your git author configuration into the remote location if needed.

Desktop

To start working in a Location, you must open a new AI Workbench window and connect to the Location. There are several ways to do this using the Desktop app:

From a New Window

• Depending on your OS, click on the AI Workbench icon or right click and select “New Window” to open a new AI Workbench window.

• This will open to the the “Locations” screen.

• Select the Location you want to use from the list.

From an Existing Workbench Window

• Click on the Location widget in the top right corner of an existing AI Workbench window.

• Once expanded, click on the location you want to use.

When you are done, close the AI Workbench window to shut everything down.

CLI

• You can explicitly specify the location/context to use by using the -c or --context flag on any command.

• You can also work in a terminal session in a more interactive way. In this approach, each terminal session is for a single Location

• Run nvwb activate LOCATION_NAME to connect to the Location. This will update your prompt indicating the current location.

You can explicitly specify the location/context to use by using the -c or --context flag on any command. This allows you to specify the location you want to work with in each individual command and can be useful when scripting.

Typically though, you can work in a terminal session in a more interactive way. In this approach, each terminal session is for a single Location. To connect to a Location, run the command nvwb activate LOCATION_NAME. This will update your prompt indicating the current location.

You cannot delete the local location.

You can delete a remote location via the Desktop app or CLI as long as the location is not running. This will remove the connection information for that location. However, it’s important to note that deleting the remote location will not delete any files or projects stored in that location.

To securely connect to a remote location, AI Workbench utilizes SSH.

When connecting to a remote location, AI Workbench automatically opens two SSH tunnels that map ports from the remote system to your local system. One tunnel is for the service, and the other is for the reverse proxy. This allows you to securely access the remote AI Workbench service and reverse proxy from localhost. By default AI Workbench will assign ports starting at 10000 & 10001 for the local location’s proxy and service, respectively. This information is saved in the connections.json file.

SSH Keys

On your local system, you’ll need to have the SSH key file and it should be located in the ~/.ssh directory. You should set the permissions for the SSH key file to 400.

Note that if you’re using Windows, AI Workbench will symlink your .ssh directory from the Windows side into /home/workbench/.ssh. This means that the permissions may not appear as expected, but you can then use your normal SSH keys in the AI Workbench application. If you’re using the desktop app, you can simply pick the file from the Windows side, and AI Workbench will automatically translate the path.

SSH into a Location

• There are two easy ways to SSH directly into a remote location.

• If you are using the CLI and have already activated a location/context, you can use nvwb attach --host to SSH into the host.

• AI Workbench updates your ~/.ssh/config file to create an alias automatically when creating a Location. This means from your terminal you can simply do ssh LOCATION_NAME to SSH into the machine.

• Note, on Windows you must do this from inside the WSL distro NVIDIA-Workbench

AI Workbench offers two convenient ways to SSH directly into a remote location. If you’re using the CLI and have already activated a location/context, you can use the command nvwb attach --host to SSH into the host.

Alternatively, AI Workbench updates your ~/.ssh/config file to create an alias automatically when creating a Location. This means you can simply use ssh LOCATION_NAME to SSH into the machine from your terminal.

Note that on Windows, you must perform this action from inside the WSL distro NVIDIA-Workbench.

• Projects

• Integrations

Remote Location fails to connect

If your Remote Location has GPUs, sometimes a driver update or other system update requires a restart before the GPUs will work. This can prevent service start because nvidia-smi command fails during boot up. To verify, SSH into the machine and run nvidia-smi. If an error is reported, you should reboot your system.