`nemo_microservices.types.v2.inference.k8s_nim_operator_config`#

Module Contents#

class nemo_microservices.types.v2.inference.k8s_nim_operator_config.K8sNIMOperatorConfig(/, **data: typing.Any)#

Bases: nemo_microservices._models.BaseModel

node_selector: Optional[Dict[str, str]]#

None

Kubernetes node selector for pod placement.

Example: {‘node-type’: ‘gpu-node’, ‘zone’: ‘us-west1-a’}

resources: Optional[Dict[str, object]]#

None

Kubernetes resource requirements including requests and limits.

Example: {‘requests’: {‘cpu’: ‘2’, ‘memory’: ‘8Gi’}, ‘limits’: {‘memory’: ‘16Gi’}}

startup_probe_grace_seconds: Optional[int]#

None

Grace period in seconds for NIM startup.

Determines how long Kubernetes will wait for the NIM to become ready before restarting it. Example: 600 (10 minutes). Must be a positive integer.

tolerations: Optional[List[Dict[str, object]]]#

None

Kubernetes tolerations for pod scheduling.

Example: [{‘key’: ‘nvidia.com/gpu’, ‘operator’: ‘Exists’, ‘effect’: ‘NoSchedule’}]