Skip to navigation
Platform ManagementScheduling and Resource OptimizationResource OptimizationQuick Starts

Launching Workloads with GPU Memory Fractions

View as Markdown

This quick start provides a step-by-step walkthrough for running a Jupyter Notebook workspace using GPU memory fractions.

NVIDIA Run:ai’s GPU memory fractions provide an agile and easy-to-use method to share a GPU or multiple GPUs across workloads. With GPU memory fractions, you can divide the GPU memory into smaller chunks and allocate each chunk to a different workload, resulting in higher GPU utilization and more efficient resource allocation. How those workloads share the GPU compute is set separately — refer to GPU compute modes.

Prerequisites

Before you start, make sure:

  • You have created a project or have one created for you.
  • The project has an assigned quota of at least 0.5 GPU.

Note

Flexible workload submission is enabled by default. If unavailable, contact your administrator to enable it under General settings → Workloads → Flexible workload submission.

Step 1: Logging In

Browse to the provided NVIDIA Run:ai user interface and log in with your credentials.

Step 2: Submitting a Workspace

  1. Go to the Workload manager → Workloads

  2. Click +NEW WORKLOAD and select Workspace

  3. Select under which cluster to create the workload

  4. Select the project in which your workspace will run

  5. Select Start from scratch to launch a new workspace quickly

  6. Enter a name for the workspace (if the name already exists in the project, you will be requested to submit a different name)

  7. Click CONTINUE

    In the next step:

  8. Under Environment, click the load icon. A side pane appears, displaying a list of available environments. Select the ‘jupyter-lab’ environment for your workspace (Image URL: jupyter/scipy-notebook)

    • If ‘jupyter-lab’ is not displayed in the gallery, follow the below steps to create a one-time environment configuration:

      • Enter the jupyter-lab Image URL - jupyter/scipy-notebook

      • Tools - Set the connection for your tool

        • Click +TOOL
        • Select Jupyter tool from the list

      • Set the runtime settings for the environment. Click +COMMAND & ARGUMENTS and add the following:

        • Enter the command - start-notebook.sh
        • Enter the arguments - --NotebookApp.token=''

        Note: If path-based routing is enabled on the cluster, enter --NotebookApp.base_url=/${RUNAI_PROJECT}/${RUNAI_JOB_NAME} --NotebookApp.token=''.

  9. Under Compute resources, click the load icon. A side pane appears, displaying a list of available compute resources. Select the ‘small-fraction’ compute resource for your workspace.

    • If ‘small-fraction’ is not displayed in the gallery, follow the below steps to create a one-time compute resource configuration:
      • Set GPU devices per pod - 1
      • Enable GPU fractioning to set the GPU memory per device:
        • Select % (of device) - Fraction of a GPU device’s memory
        • Set the memory Request - 10 (the workload will allocate 10% of the GPU memory)
      • Optional: set the CPU compute per pod - 0.1 cores (default)
      • Optional: set the CPU memory per pod - 100 MB (default)

  10. Click CREATE WORKSPACE

Step 3: Connecting to the Jupyter Notebook

  1. Select the newly created workspace with the Jupyter application that you want to connect to
  2. Click CONNECT
  3. Select the Jupyter tool. The selected tool is opened in a new tab on your browser.

Next Steps

Manage and monitor your newly created workload using the Workloads table.