`nemo_microservices.types.v2.inference.k8s_nim_operator_config_param`#

Module Contents#

class nemo_microservices.types.v2.inference.k8s_nim_operator_config_param.K8sNIMOperatorConfigParam#

Bases: typing_extensions.TypedDict

node_selector: Dict[str, str]#

None

Kubernetes node selector for pod placement.

Example: {‘node-type’: ‘gpu-node’, ‘zone’: ‘us-west1-a’}

resources: Dict[str, object]#

None

Kubernetes resource requirements including requests and limits.

Example: {‘requests’: {‘cpu’: ‘2’, ‘memory’: ‘8Gi’}, ‘limits’: {‘memory’: ‘16Gi’}}

startup_probe_grace_seconds: int#

None

Grace period in seconds for NIM startup.

Determines how long Kubernetes will wait for the NIM to become ready before restarting it. Example: 600 (10 minutes). Must be a positive integer.

tolerations: Iterable[Dict[str, object]]#

None

Kubernetes tolerations for pod scheduling.

Example: [{‘key’: ‘nvidia.com/gpu’, ‘operator’: ‘Exists’, ‘effect’: ‘NoSchedule’}]