ModelArtifactDeParam

class nemo_microservices.types.ModelArtifactDeParam

Bases: TypedDict

files_url: Required[str]

The location where the artifact files are stored.

status: Required[Literal['created', 'upload_failed', 'upload_completed']]

The status of the model artifact.

backend_engine: Literal['nemo', 'trt_llm', 'vllm', 'faster_transformer', 'hugging_face']

Types of backend engine.

gpu_arch: str

precision: Literal['int8', 'bf16', 'fp16', 'fp32']

Types of model precision.

tensor_parallelism: int