> This page is for Self-Hosted, version v3.0 (default). > For other versions, use one of these documentation indexes: > - v3.0 (default): https://docs.nvidia.com/run-ai/self-hosted/v3.0/llms.txt > - v2.26: https://docs.nvidia.com/run-ai/self-hosted/v2.26/llms.txt > - v2.25: https://docs.nvidia.com/run-ai/self-hosted/v2.25/llms.txt > - v2.24: https://docs.nvidia.com/run-ai/self-hosted/v2.24/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/run-ai/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/run-ai/_mcp/server. # Launching Workloads with Dynamic GPU Memory Fractions This quick start provides a step-by-step walkthrough for running a Jupyter Notebook with [dynamic GPU memory fractions](/run-ai/self-hosted/documentation/platform-management/runai-scheduler/resource-optimization/gpu-fractions/dynamic-memory-fractions). NVIDIA Run:ai’s dynamic GPU memory fractions optimize GPU utilization by enabling workloads to adjust their memory usage at runtime. A workload declares a memory Request that is guaranteed to it, and a higher memory Limit that it can grow into when the additional memory is not needed by others. How those workloads share the GPU compute is set separately — refer to [GPU compute modes](/run-ai/self-hosted/documentation/platform-management/runai-scheduler/resource-optimization/gpu-fractions/compute-modes). ## Prerequisites Before you start, make sure: * You have created a [project](/run-ai/self-hosted/documentation/platform-management/manage-ai-initiatives/organization/projects) or have one created for you. * The project has an assigned quota of at least 0.5 GPU. * [Dynamic GPU memory fractions](/run-ai/self-hosted/documentation/platform-management/runai-scheduler/resource-optimization/gpu-fractions/dynamic-memory-fractions) is enabled. **Note** * Flexible workload submission is enabled by default. If unavailable, contact your administrator to enable it under **General settings** → Workloads → Flexible workload submission. * Dynamic GPU memory fractions is disabled by default. To set a memory Limit through the UI, the CLI, or the APIs, it must be enabled by your administrator under **General settings** → **Resources** → **GPU resource optimization**. ## Step 1: Logging In Browse to the provided NVIDIA Run:ai user interface and log in with your credentials. Run the below --help command to obtain the login options and log in according to your setup: ```sh runai login --help ``` To use the API, you will need to obtain a token as shown in [API authentication](/run-ai/self-hosted/reference/rest-api/getting-started/how-to-authenticate-to-the-api). ## Step 2: Submitting the First Workspace 1. Go to the Workload manager → Workloads 2. Click **+NEW WORKLOAD** and select **Workspace** 3. Select under which **cluster** to create the workload 4. Select the **project** in which your workspace will run 5. Select **Start from scratch** to launch a new workspace quickly 6. Enter a **name** for the workspace (if the name already exists in the project, you will be requested to submit a different name) 7. Click **CONTINUE** In the next step: 8. Under **Environment**, click the **load** icon. A side pane appears, displaying a list of available environments. To add a new environment: * Click the **+** icon to create a new environment * Enter quick-start as the **name** for the environment. The name must be unique. * Enter the **Image URL** - `quay.io/jupyter/pytorch-notebook` * Tools - Set the connection for your tool: * Click **+TOOL** * Select **Jupyter** tool from the list * Set the runtime settings for the environment. Click **+COMMAND & ARGUMENTS** and add the following: * Enter the command - `start-notebook.sh` * Enter the arguments - `--NotebookApp.token=''` **Note:** If [path-based routing](/run-ai/self-hosted/documentation/installation-and-configuration/config/networking/external-access-to-containers#access-to-the-running-workloads-container) is enabled on the cluster, enter `--NotebookApp.base_url=/${RUNAI_PROJECT}/${RUNAI_JOB_NAME} --NotebookApp.token=''`. * Click **CREATE ENVIRONMENT** * Select the newly created environment from the side pane 9. Under **Compute resources**, click the **load** icon. A side pane appears, displaying a list of available compute resources. To add a new compute resource: * Click the **+** icon to create a new compute resource * Enter request-limit as the **name** for the compute resource. The name must be unique. * Set **GPU devices** per pod - 1 * Enable **GPU fractioning** to set the GPU memory per device: * Select **GB** **-** Fraction of a GPU device’s memory * Set the memory **Request** - 4GB (the workload will allocate 4GB of the GPU memory) * Set the memory **Limit** - 12GB * Optional: set the **CPU compute per pod** - 0.1 cores (default) * Optional: set the **CPU memory per pod** - 100 MB (default) * Select **More settings** and toggle **Increase shared memory size** * Click **CREATE COMPUTE RESOURCE** * Select the newly created compute resource from the side pane 10. Click **CREATE WORKSPACE** 1. Go to the Workload manager → Workloads 2. Click **+NEW WORKLOAD** and select **Workspace** 3. Select under which **cluster** to create the workload 4. Select the **project** in which your workspace will run 5. Select **Start from scratch** to launch a new workspace quickly 6. Enter a **name** for the workspace (if the name already exists in the project, you will be requested to submit a different name) 7. Click **CONTINUE** In the next step: 8. Create an environment for your workspace * Click **+NEW ENVIRONMENT** * Enter quick-start as the **name** for the environment. The name must be unique. * Enter the **Image URL** - `quay.io/jupyter/pytorch-notebook` * Tools - Set the connection for your tool * Click **+TOOL** * Select **Jupyter** tool from the list * Set the runtime settings for the environment. Click **+COMMAND & ARGUMENTS** and add the following: * Enter the command - `start-notebook.sh` * Enter the arguments - `--NotebookApp.token=''` **Note:** If [path-based routing](/run-ai/self-hosted/documentation/installation-and-configuration/config/networking/external-access-to-containers#access-to-the-running-workloads-container) is enabled on the cluster, enter `--NotebookApp.base_url=/${RUNAI_PROJECT}/${RUNAI_JOB_NAME} --NotebookApp.token=''`. * Click **CREATE ENVIRONMENT** The newly created environment will be selected automatically 9. Create a new “**request-limit**” compute resource for your workspace * Click **+NEW COMPUTE RESOURCE** * Enter request-limit as the **name** for the compute resource. The name must be unique. * Set **GPU devices** per pod - 1 * Enable **GPU fractioning** to set the GPU memory per device: * Select **GB** **-** Fraction of a GPU device’s memory * Set the memory **Request** - 4GB (the workload will allocate 4GB of the GPU memory) * Set the memory **Limit** - 12GB * Optional: set the **CPU compute per pod** - 0.1 cores (default) * Optional: set the **CPU memory per pod** - 100 MB (default) * Select **More settings** and toggle **Increase shared memory size** * Click **CREATE COMPUTE RESOURCE** The newly created compute resource will be selected automatically 10. Click **CREATE WORKSPACE** Copy the following command to your terminal. Make sure to update the below with the name of your project and workload. For more details, see [CLI reference](/run-ai/self-hosted/reference/cli/runai/workspace/runai-workspace-submit): ```sh runai project set "project-name" runai workspace submit "workload-name" \ --image quay.io/jupyter/pytorch-notebook \ --gpu-memory-request 4G --gpu-memory-limit 12G --large-shm \ --external-url container=8888 --name-prefix jupyter \ --command -- start-notebook.sh \ --NotebookApp.token= ``` Copy the following command to your terminal. Make sure to update the below parameters. For more details, see [Workspaces](https://docs.nvidia.com/run-ai/self-hosted/v3.0/reference/rest-api/endpoints/workloads/workspaces/create-workspace-1) API: ```bash curl -L 'https:///api/v1/workloads/workspaces' \ -H 'Content-Type: application/json' \ -H 'Authorization: Bearer ' \ -d '{ "name": "workload-name", "projectId": "", "clusterId": "", "spec": { "command" : "start-notebook.sh", "args" : "--NotebookApp.token=''", "image": "quay.io/jupyter/pytorch-notebook", "compute": { "gpuDevicesRequest": 1, "gpuMemoryRequest": "4G", "gpuMemoryLimit": "12G", "largeShmRequest": true }, "exposedUrls" : [ { "container" : 8888, "toolType": "jupyter-notebook", "toolName": "Jupyter" } ] } } ``` * `` - The link to the NVIDIA Run:ai user interface * `` - The API access token obtained in [Step 1](/run-ai/self-hosted/documentation/platform-management/runai-scheduler/resource-optimization/quick-starts/dynamic-gpu-fractions-quickstart#step-1-logging-in) * `` - The ID of the Project the workload is running on. You can get the Project ID via the [Get Projects](https://docs.nvidia.com/run-ai/self-hosted/v3.0/reference/rest-api/endpoints/organizations/projects/get-projects) API. * `` - The unique identifier of the Cluster. You can get the Cluster UUID via the [Get Clusters](https://docs.nvidia.com/run-ai/multi-tenant/v3.0/reference/rest-api/endpoints/organizations/clusters/get-clusters) API. * `toolType` will show the Jupyter icon when connecting to the Jupyter tool via the user interface. * `toolName` will show when connecting to the Jupyter tool via the user interface. **Note** The above API snippet runs with NVIDIA Run:ai clusters of 2.18 and above only. ## Step 3: Submitting the Second Workspace 1. Go to the Workload manager → Workloads 2. Click **+NEW WORKLOAD** and select **Workspace** 3. Select the **cluster** where the previous workspace was created 4. Select the **project** where the previous workspace was created 5. Select **Start from scratch** to launch a new workspace quickly 6. Enter a **name** for the workspace (if the name already exists in the project, you will be requested to submit a different name) 7. Click **CONTINUE** In the next step: 8. Under **Environment**, click the **load** icon. A side pane appears, displaying a list of available environments. Select the environment created in [Step 2](/run-ai/self-hosted/documentation/platform-management/runai-scheduler/resource-optimization/quick-starts/dynamic-gpu-fractions-quickstart#step-2-submitting-the-first-workspace). 9. Under **Compute resources**, click the **load** icon. A side pane appears, displaying a list of available compute resources. Select the compute resources created in [Step 2](/run-ai/self-hosted/documentation/platform-management/runai-scheduler/resource-optimization/quick-starts/dynamic-gpu-fractions-quickstart#step-2-submitting-the-first-workspace). 10. Click **CREATE WORKSPACE** 1. Go to the Workload manager → Workloads 2. Click **+NEW WORKLOAD** and select **Workspace** 3. Select the **cluster** where the previous workspace was created 4. Select the **project** where the previous workspace was created 5. Select **Start from scratch** to launch a new workspace quickly 6. Enter a **name** for the workspace (if the name already exists in the project, you will be requested to submit a different name) 7. Click **CONTINUE** In the next step: 8. Select the environment created in [Step 2](/run-ai/self-hosted/documentation/platform-management/runai-scheduler/resource-optimization/quick-starts/dynamic-gpu-fractions-quickstart#step-2-submitting-the-first-workspace) 9. Select the compute resource created in [Step 2](/run-ai/self-hosted/documentation/platform-management/runai-scheduler/resource-optimization/quick-starts/dynamic-gpu-fractions-quickstart#step-2-submitting-the-first-workspace) 10. Click **CREATE WORKSPACE** Copy the following command to your terminal. Make sure to update the below with the name of your project and workload. For more details, see [CLI reference](/run-ai/self-hosted/reference/cli/runai/workspace/runai-workspace-submit): ```sh runai project set "project-name" runai workspace submit "workload-name" \ --image quay.io/jupyter/pytorch-notebook --gpu-memory-request 4G \ --gpu-memory-limit 12G --large-shm --external-url container=8888 \ --name-prefix jupyter --command -- start-notebook.sh \ --NotebookApp.token= ``` Copy the following command to your terminal. Make sure to update the below parameters. For more details, see [Workspaces](https://docs.nvidia.com/run-ai/self-hosted/v3.0/reference/rest-api/endpoints/workloads/workspaces/create-workspace-1) API: ```bash curl -L 'https:///api/v1/workloads/workspaces' \ -H 'Content-Type: application/json' \ -H 'Authorization: Bearer ' \ -d '{ "name": "workload-name", "projectId": "", "clusterId": "", "spec": { "command" : "start-notebook.sh", "args" : "--NotebookApp.token=''", "image": "quay.io/jupyter/pytorch-notebook", "compute": { "gpuDevicesRequest": 1, "gpuMemoryRequest": "4G", "gpuMemoryLimit": "12G", "largeShmRequest": true }, "exposedUrls" : [ { "container" : 8888, "toolType": "jupyter-notebook", "toolName": "Jupyter" } ] } } ``` * `` - The link to the NVIDIA Run:ai user interface * `` - The API access token obtained in [Step 1](/run-ai/self-hosted/documentation/platform-management/runai-scheduler/resource-optimization/quick-starts/dynamic-gpu-fractions-quickstart#step-1-logging-in) * `` - The ID of the Project the workload is running on. You can get the Project ID via the [Get Projects](https://docs.nvidia.com/run-ai/self-hosted/v3.0/reference/rest-api/endpoints/organizations/projects/get-projects) API. * `` - The unique identifier of the Cluster. You can get the Cluster UUID via the [Get Clusters](https://docs.nvidia.com/run-ai/multi-tenant/v3.0/reference/rest-api/endpoints/organizations/clusters/get-clusters) API. * `toolType` will show the Jupyter icon when connecting to the Jupyter tool via the user interface. * `toolName` will show when connecting to the Jupyter tool via the user interface. **Note** The above API snippet runs with NVIDIA Run:ai clusters of 2.18 and above only. ## Step 4: Connecting to the Jupyter Notebook 1. Select the newly created workspace with the Jupyter application that you want to connect to 2. Click **CONNECT** 3. Select the Jupyter tool. The selected tool is opened in a new tab on your browser. 4. Open a terminal and use the `watch nvidia-smi` to get a constant reading of the memory consumed by the pod. The free memory shown always covers the rest of the 4 GB Request, and grows beyond that when the GPU has memory to spare, up to the 12 GB Limit. 5. Open the file `Untitled.ipynb` and move the frame so you can see both tabs 6. Execute both cells in `Untitled.ipynb`. This will consume about **3 GB of GPU memory** and be well below the 4GB of the **GPU Memory Request** value. 7. In the second cell, edit the value after `--image-size` from 100 to 200 and run the cell. This will increase the GPU memory utilization to about 11.5 GB which is above the Request value. 1. To connect to the Jupyter Notebook, browse directly to `https:////` 2. Open a terminal and use the `watch nvidia-smi` to get a constant reading of the memory consumed by the pod. The free memory shown always covers the rest of the 4 GB Request, and grows beyond that when the GPU has memory to spare, up to the 12 GB Limit. 3. Open the file `Untitled.ipynb` and move the frame so you can see both tabs 4. Execute both cells in `Untitled.ipynb`. This will consume about **3 GB of GPU memory** and be well below the 4GB of the **GPU Memory Request** value. 5. In the second cell, edit the value after `--image-size` from 100 to 200 and run the cell. This will increase the GPU memory utilization to about 11.5 GB which is above the Request value. 1. To connect to the Jupyter Notebook, browse directly to `https:////` 2. Open a terminal and use the `watch nvidia-smi` to get a constant reading of the memory consumed by the pod. The free memory shown always covers the rest of the 4 GB Request, and grows beyond that when the GPU has memory to spare, up to the 12 GB Limit. 3. Open the file `Untitled.ipynb` and move the frame so you can see both tabs 4. Execute both cells in `Untitled.ipynb`. This will consume about **3 GB of GPU memory** and be well below the 4GB of the **GPU Memory Request** value. 5. In the second cell, edit the value after `--image-size` from 100 to 200 and run the cell. This will increase the GPU memory utilization to about 11.5 GB which is above the Request value. ## Next Steps Manage and monitor your newly created workload using the [Workloads](/run-ai/self-hosted/documentation/workloads-in-nvidia-run-ai/workloads) table.