Get Latest ModelDeploymentConfig Version

GET

https://host.com/apis/models/v2/workspaces/:workspace/deployment-configs/:name

GET

/apis/models/v2/workspaces/:workspace/deployment-configs/:name

$ curl https://host.com/apis/models/v2/workspaces/workspace/deployment-configs/name \
>      -H "Content-Type: application/json"

1 {
2   "name": "llama-3.1-8b",
3   "workspace": "research_team_alpha",
4   "created_at": "2024-01-15T09:30:00Z",
5   "updated_at": "2024-04-20T16:45:00Z",
6   "entity_version": 3,
7   "engine": "nim",
8   "model_spec": {
9     "model_type": "llm",
10     "model_namespace": "meta",
11     "model_name": "llama-3.1-8b",
12     "model_revision": "v3.1.0",
13     "chat_template": "{% raw %}{{ user_message }}{% endraw %}",
14     "tool_call_config": {
15       "tool_call_parser": "openai",
16       "tool_call_plugin": "research_team_alpha/custom_tool_plugin",
17       "auto_tool_choice": true
18     },
19     "lora_enabled": false
20   },
21   "executor_config": {
22     "gpu": 2,
23     "disk_size": "100Gi",
24     "image_name": "nemo/nimservice-llama",
25     "image_tag": "v3.1.0",
26     "health_check_path": "/v1/health/ready",
27     "additional_envs": {
28       "NIM_LOG_LEVEL": "info",
29       "MAX_BATCH_SIZE": "16"
30     },
31     "additional_args": [
32       "--enable-logging"
33     ],
34     "k8s_nim_operator_config": {
35       "resources": {
36         "requests": {
37           "cpu": "4",
38           "memory": "16Gi"
39         },
40         "limits": {
41           "cpu": "8",
42           "memory": "32Gi"
43         }
44       },
45       "tolerations": [
46         {
47           "key": "nvidia.com/gpu",
48           "operator": "Exists",
49           "effect": "NoSchedule"
50         }
51       ],
52       "node_selector": {
53         "node-type": "gpu-node",
54         "zone": "us-west1-a"
55       },
56       "startup_probe_grace_seconds": 600
57     },
58     "override_config": {}
59   },
60   "id": "d4f7a9b2-3c1e-4f8a-9b7d-2e5f6a7c8d9e",
61   "project": "urn:nemo:projects:language-models/v1",
62   "description": "Deployment config for LLaMA 3.1 8B model optimized for chat applications.",
63   "model_entity_id": "meta/llama-3.1-8b"
64 }

Get the latest version of a ModelDeploymentConfig.

Path parameters

workspacestringRequired

namestringRequired

Response

Return the latest version of a model deployment configuration

namestringformat: "^[\w\-.]+$"<=255 characters

Name of the entity. Name/workspace combo must be unique across all entities. Allowed characters: letters (a-z, A-Z), digits (0-9), underscores, hyphens, and dots.

workspacestringformat: "^[\w\-.]+$"<=255 characters

The workspace of the entity. Allowed characters: letters (a-z, A-Z), digits (0-9), underscores, hyphens, and dots.

created_atdatetime

The timestamp of model entity creation

updated_atdatetime

The timestamp of the last model entity update

entity_versioninteger

Version of this deployment config. Automatically managed.

engineenum

Inference engine selecting the compiler path (nim/vllm/generic)

model_specobject

What model to serve and how — independent of the executor it runs on

executor_configobject

Compute + container settings for the executor the deployment runs on

idstring

Unique identifier for the deployment config

projectstringformat: "^[\w\-./]+$"<=255 characters

The URN of the project associated with this entity.

descriptionstring<=1000 characters

Optional description of the deployment configuration

model_entity_idstring<=255 characters

Optional reference to the base model entity ID for this deployment

Errors

422

Unprocessable Entity Error