Resource Shape

A resource shape specifies the combination of resources (GPU, CPU, memory, storage, etc.) that is reserved for a workload to use.

For example, the shape 8xH100 GPUs, 128 CPUs, 1 TB Memory will reserve 8 H100 GPUs, 128 CPU cores, and 1 TB of memory.

DGX Cloud Lepton supports two types of resource shapes:

• Default GPU shapes provided by Lepton.
• Custom shapes defined by the node group administrator.

Default GPU Shapes

DGX Cloud Lepton provides a set of default GPU shapes, designed for general-purpose workloads uses and focus on GPU allocation.

For the default GPU shapes, the upper bound for other resource dimensions (vCPU, memory, etc.) is equal to total available resources divided by the number of GPUs. For example, on a node with 8 GPUs and 128 vCPUs, the default 1-GPU shape is limited to 16 vCPUs (128 vCPUs divided by 8 GPUs).

While this is the upper bound, the GPU shape initially requests a much smaller CPU/memory footprint to leave room for CPU-only/memory shapes to be colocated on the same node. If required, the GPU shape can then burst into its max allocation explained above. To reiterate, a 1-GPU shape on an 8-GPU machine can burst up to ⅛ th of the node's total CPU and memory.

If your workload is sensitive to these issues, we recommend defining custom GPU shapes with CPU/memory limits such that the total allocation between the CPU-only and GPU shapes does not exceed the total system resources.

Custom Shapes

Navigate to your node group details page, click on the Config dropdown menu, and select the Shapes option, you will be navigated to a new page where you can define custom shapes.

Click on the Add Shape button on the top right corner, you can see a popup modal where you can define the shape's name, description, GPU, CPU and memory.

• Name: The identifier for the shape, used to distinguish it when selecting or configuring resources.
• Number of CPU: The total number of CPU cores allocated to the shape.
• Memory: The amount of memory reserved for the shape, measured in MB.
• Storage: The amount of storage reserved for the shape, measured in GB.
• Number of GPU: The number of GPU units allocated to the shape. You can also set this option to "None" to create a CPU-only shape.
• GPU Type: The specific type of GPU assigned to the shape. Available types are determined by the node group.
• Description (Optional): An optional field to provide additional details or context about the shape.

After filling out the form, click on the Create button to create the custom shape.

Different from the default GPU shapes, Custom shapes will only reserve the specified resources and won't burst beyond the limits.