Skip to navigation
Platform ManagementScheduling and Resource OptimizationResource OptimizationQuick Starts

Launching Workloads with GPU Compute Modes

View as Markdown

This quick start provides a step-by-step walkthrough for running two workspaces that share a single GPU using SM-sharing, and for verifying the compute mode after submission.

A GPU compute mode always accompanies a GPU memory fraction — the memory fraction determines how much GPU memory the workload receives, and the compute mode determines how the workloads sharing that GPU divide the GPU compute. For background on both modes, refer to GPU compute modes.

Prerequisites

Before you start, make sure:

  • You have created a project or have one created for you.
  • The project has an assigned quota of at least 1 GPU.
  • A GPU is available that is not currently running other workloads using GPU fractions.

Note

  • The compute mode is exclusive per GPU. The first workload placed on a GPU fixes the compute mode of that GPU, and every workload that joins afterwards must request the same mode. Starting this walkthrough on a GPU that is already running time-slicing workloads prevents the SM-sharing workspaces from being placed on it.
  • Flexible workload submission is enabled by default. If unavailable, contact your administrator to enable it under General settings → Workloads → Flexible workload submission.

Step 1: Logging In

Browse to the provided NVIDIA Run:ai user interface and log in with your credentials.

Step 2: Submitting the First Workspace

  1. Go to the Workload manager → Workloads

  2. Click +NEW WORKLOAD and select Workspace

  3. Select under which cluster to create the workload

  4. Select the project in which your workspace will run

  5. Select Start from scratch to launch a new workspace quickly

  6. Enter a name for the workspace (if the name already exists in the project, you will be requested to submit a different name)

  7. Click CONTINUE

    In the next step:

  8. Under Environment, click the load icon. A side pane appears, displaying a list of available environments. Select the ‘jupyter-lab’ environment for your workspace (Image URL: jupyter/scipy-notebook)

  9. Under Compute resources, click the load icon. A side pane appears, displaying a list of available compute resources. To add a new compute resource:

    • Click the + icon to create a new compute resource
    • Enter sm-sharing-fraction as the name for the compute resource. The name must be unique.
    • Set GPU devices per pod - 1
    • Enable GPU fractioning to set the GPU memory per device:
      • Select GB - Fraction of a GPU device’s memory
      • Set the memory Request - 20GB
    • Click CREATE COMPUTE RESOURCE
    • Select the newly created compute resource from the side pane

  10. Open the General section and locate Set annotation(s). Click +ANNOTATION and set the compute mode:

    • Name - nvidia.com/container.<container-name>.gpu-compute.mode
    • Value - "sm-sharing", including the quotation marks

  11. Click CREATE WORKSPACE

Step 3: Submitting the Second Workspace

Repeat Step 2 with a different workload name, using the same compute resource and the same sm-sharing compute mode. Both workspaces request 20 GB of GPU memory, so both can be placed on the same GPU.

Note

A workload requesting time-slicing, or a workload submitted without a compute mode annotation, is not placed on this GPU while the SM-sharing workspaces are running on it.

Step 4: Verifying the Compute Mode

  1. Go to the Workload manager → Workloads
  2. Select one of the workspaces you created and click SHOW DETAILS
  3. Open the YAML tab
  4. Confirm that nvidia.com/container.<container-name>.gpu-compute.mode is present with the value sm-sharing

Both workspaces now run concurrently on separate streaming multiprocessors of the same GPU. Had the workloads used time-slicing, they would have taken sequential turns on the GPU compute engines instead.

Next Steps

  • Choose between the two modes for your own workloads using GPU compute modes.
  • Manage and monitor your newly created workloads using the Workloads table.