NIMDeploymentConfigParam

class nemo_microservices.types.NIMDeploymentConfigParam

Bases: TypedDict

gpu: Required[int]

The number of GPUs needed for a deployment.

image_name: Required[str]

The name of the Docker image.

image_tag: Required[str]

The tag of the Docker image.

additional_envs: Dict[str, str]

Additional environment variables to pass to the deployment.

disable_lora_support: bool

When true, prevents setting default values for NIM_PEFT_SOURCE, NIM_PEFT_REFRESH_INTERVAL and related service-level environment variables. This will load the NIM without support for LoRAs. This is required for some NIMs which do not support LoRAs. This feature is experimental and may change in future versions.

Type:

EXPERIMENTAL

namespace: str

The Kubernetes namespace of the deployment.