Skip to navigation
Reference

Response Fields for NIM Metadata API

View as Markdown

The response body has four top-level sections, and all four are optional. Real responses contain a subset of the fields described on this page. A complete example with every field populated is available in the API reference documentation.

The following table describes the top-level sections:

SectionTypeDescription
profilesarray of objectThe deployable configurations. This is the section you use most.
release_detailsobjectRelease-level facts, such as version, documentation, compliance, and supported architectures.
featuresobjectCapability flags, such as context window, tool calling, and backend support.
related_urlsarray of stringLinks to related resources for this NIM microservice.

Serialization Rules

The following rules hold across the entire JSON response body and are enforced by the service. Use these rules to interpret anything you get back.

  1. The service never emits null. The service serializes with non-null inclusion. If a field has no value, the key is absent entirely.
  2. Absent means unknown, not false. A missing feat_lora does not mean LoRA adapters are unsupported; it means nothing was recorded. Never treat an absent boolean as false, or an absent number as 0.
  3. Only populated fields appear. A response contains what was supplied at publish time and nothing else. Two profiles in the same response body can carry different sets of keys and values.
  4. One deliberate exception applies. An explicitly empty tested_gpu_devices array is preserved, so “tested on nothing” stays distinguishable from “unknown.”
  5. Tolerate keys you do not recognize. The set of fields grows as the schema evolves, so a response body can contain keys that postdate your integration. Parse permissively, ignore what you do not know, and never fail on an unrecognized key.

Profiles

The profiles array holds the deployable configurations. Only profile_id and tested_gpu_devices are guaranteed present, so treat every other field as optional.

The following table describes the profile fields:

FieldTypeDescription
profile_idstringUnique identifier for this profile. Use this value when selecting a profile at deployment time. Always present.
gpustringHuman-readable GPU this profile targets, such as H100 or A100. Absent on GPU-agnostic profiles.
gpu_devicestringPCI identifier of the target GPU, in the form pciDeviceId:pciVendorId, such as 2330:10de. More precise than gpu. Absent on GPU-agnostic profiles.
tested_gpu_devicesarray of stringPCI identifiers this profile has been validated on. Can be empty, as described in Core Concepts Overview.
precisionstringNumeric precision, such as fp8, bf16, or fp16.
inference_backendstringServing backend, such as tensorrt_llm, vllm, or sglang.
tensor_parallel_sizeintegerNumber of GPUs the model is sharded across tensor-wise.
pipeline_parallel_sizeintegerNumber of pipeline stages.
min_vram_per_device_gbnumberMinimum vRAM required per GPU, not a total across GPUs. This is the primary hardware-fit field. A profile with tensor_parallel_size set to 4 and min_vram_per_device_gb set to 80.0 requires four GPUs with 80 GB each, not 80 GB in aggregate.
disk_space_per_device_gbnumberDisk space required per device, in GB.
system_memory_per_device_gbnumberHost RAM required per device, in GB.
min_cpu_countintegerMinimum CPU cores.
min_driver_versionstringMinimum NVIDIA driver version.
max_gpu_countintegerMaximum GPUs this profile can use.
form_factorstringPhysical GPU form factor, such as SXM or PCIe.
workspace_hashstringDigest over the workspace contents of the profile. Use it to verify supply-chain integrity after download.
throughputintegerRecorded throughput figure. Units are not standardized, as described in Current Limitations.
feat_lorabooleanWhether this profile supports LoRA adapters.
buildable_profilebooleanWhether the engine for the profile can be built locally rather than downloaded prebuilt.
languagesarray of stringLanguage codes the profile supports, such as ["en-US", "ja-JP"]. Typically present only on speech and translation NIM microservices.

Release Details

The release_details object holds facts about the release of a container. All fields are optional.

The following table describes the release fields:

FieldTypeDescription
idstringRelease identifier.
container_versionstringVersion of the container this metadata describes.
model_cardstringModel card content, as Markdown.
minimum_gpu_supportedstringLowest GPU generation supported by the release as a whole.
compliance_certificationsarray of stringCertifications held, such as ["ISO", "SOC2"].
documentation_urlstringLink to the documentation for the NIM microservice.
architecturearray of stringCPU architectures supported, such as ["AMD 64", "ARM 64"].
is_gov_readybooleanWhether the release is approved for government deployment.
allowed_deployment_regionsarray of stringRegions the release can be deployed in, such as ["us-west-1"]. An absent value means no recorded restriction. Confirm license terms separately rather than inferring them from absence.

Features

The features object holds capability flags for the NIM microservice. All fields are optional and boolean unless noted otherwise.

The following table describes the feature flags:

FieldTypeDescription
context_windowintegerMaximum context length in tokens.
reasoning_supportbooleanSupports reasoning-style generation.
tool_callingbooleanSupports tool or function calling.
parallel_tool_callingbooleanSupports multiple simultaneous tool calls.
vgpu_supportbooleanRuns on virtualized GPUs.
trt_llm_buildable_supportbooleanTensorRT-LLM engines can be built locally.
prebuilt_enginesbooleanShips prebuilt inference engines.
vllm_supportbooleanSupports the vLLM backend.
sglang_supportbooleanSupports the SGLang backend.
trtllm_pytorch_supportbooleanSupports the TensorRT-LLM PyTorch backend.
suffix_supportbooleanSupports suffix or fill-in-the-middle completion.
boost_decodebooleanBoosted decoding optimization available.
steer_decodebooleanSteered decoding available.
rule_decodebooleanRule-constrained decoding available.
serve_scalebooleanServing-side scaling optimization available.
cache_scalebooleanCache scaling optimization available.

The related_urls field is an array of strings that holds links to related resources for this NIM microservice.