Skip to navigation
Guides

Notes for Automated Agents

View as Markdown

This page is written for programmatic consumers of the NIM Metadata API, such as tooling, schedulers, and agents that select profiles without a human in the loop. If you are exploring the API by hand, start with the Quickstart instead.

The response is stable, machine-readable JSON, but the schema grows over time. Following the rules on this page keeps an integration working across schema changes rather than breaking on the first new field.

Forward-Compatibility Rules

Apply the following six rules to every integration.

RuleDescription
1. Never assume a field is presentOnly profile_id and tested_gpu_devices are guaranteed present. Access everything else defensively, with an explicit branch for absence.
2. Never treat absence as a valueAbsent is not false, 0, or an empty string. If a decision depends on an absent field, surface the outcome as unknown rather than defaulting it.
3. Ignore unrecognized fieldsNew fields are added over time. Deserialize permissively and never fail on an unknown key.
4. Tolerate gpu and gpu_device becoming arraysA single profile_id can be valid on multiple compatible GPUs. These fields can change from a string to a list, so accept either shape.
5. Treat precision and inference_backend as open stringsThese fields are not enumerations. Carry through new values instead of failing on an unrecognized value.
6. Do not parse statusDescriptionBranch on the HTTP status code and requestStatus.statusCode. The description text is written for people and changes over time.

The following example accepts either GPU field shape. Complete the Quickstart first to define metadata:

def as_list(value):
    if value is None:
        return []
    return value if isinstance(value, list) else [value]
for profile in metadata.get("profiles", []):
    gpus = as_list(profile.get("gpu"))
    devices = as_list(profile.get("gpu_device"))
    print(profile.get("profile_id"), gpus, devices)

Freshness and Caching

The API offers no ETag header, no Last-Modified header, and no conditional-request support, so cache based on tag stability rather than cache validation. The following table describes how long to trust a cached document:

Tag TypeStabilityGuidance
Semantic tags, such as 1.2.0Stable in practice, but not guaranteed immutableCache for hours or days, not forever. The service refuses to create metadata over a tag that already has it, but the publishing pipeline can still amend or remove it.
Rolling tags, such as latestNot stableDo not cache beyond your tolerance for staleness, because these tags are republished.

Publication lags the container. A newly pushed container tag can exist before its metadata does, so a 404 status code immediately after a release can resolve shortly afterward. If you poll, use a bounded number of attempts with backoff rather than polling indefinitely.

Response Sizes

Plan for variability rather than a fixed size. A typical NIM microservice returns roughly 8 KB across eight profiles. NIM microservices with many profiles reach 20 KB to 50 KB, and the largest observed historical documents approach 300 KB across several hundred profiles.

There is no hard cap on profile count, so stream the response or size your buffers accordingly and do not assume a small fixed maximum.

Rate Limits

No endpoint-specific rate limit is published, and standard NGC gateway limits apply. Be a considerate client by caching immutable tags, avoiding tight polling loops, and backing off on a 429 status code or any 5xx status code.

Best Practices

Apply the following practices when you build against this API:

  • Branch on status codes, not text. Treat a 404 status code as a definitive answer about metadata availability only after bounded retries for newly published tags are exhausted.
  • Model unknowns explicitly. Return a three-valued result from fit checks so that an unrecorded requirement is distinguishable from a failed one.
  • Log the request identifier. Record requestStatus.requestId on every non-success response, because that is the value support needs to trace a specific request.
  • Do not retry service-side data problems. A well-formed request that returns a 400 or 500 status code because of stored data will not succeed on retry. Report it instead.
  • Design for partial coverage. Metadata is attached at publish time, so build the no-metadata path first rather than treating it as an edge case.

Next Steps

Explore the following resources to complete your integration: