nemo_gym.data.tasks.harbor.registry

Read an image’s configuration from its OCI registry.

Parses Docker-style references with Docker Hub defaults, selects the platform’s manifest from an index, follows the bearer token challenge, drops authorization across redirects, retries transient failures and returns the configuration keys the records keep, pinned to the digest of the manifest the reference named. Nothing here knows about tasks or Compose. The module keeps the record key list, the platform constants and ImageConfigError, which the rest of the package imports from here.

Module Contents

Classes

Name	Description
`ImageConfigError`	An image reference could not be resolved to a configuration for the platform the sandboxes run.
`ImageRef`	A parsed image reference: `[registry/]repository[:tag][@digest]`.
`PlatformMismatchError`	A single-platform reference resolved to an image of another platform than the one asked for; `resolved`
`RegistryClient`	Read manifests and configuration blobs over the OCI distribution API with the standard library.
`_RedirectHandler`	Follow a registry redirect without carrying the registry’s token to another host.

Functions

Name	Description
`credentials_for_image_prefixes`	`RegistryClient` credentials from a sandbox provider’s image-prefix auth map.
`image_prefix_key`	An `images.auth` prefix as `RegistryClient` keys credentials: `registry/repository-prefix`, the
`normalize_platform_parts`	`(os, architecture, variant)` as containerd spells them, so that two spellings of one platform compare
`parse_image_ref`	Split an image reference the way Docker does, defaulting to Docker Hub.
`platform_name`	`os/arch[/variant]` of a (normalised) platform tuple, the variant shown only when it is not the
`platform_pair`	The normalised `(os, architecture, variant)` of an `os/arch[/variant]` platform string such as

Data

DOCKER_HUB_REGISTRY

INDEX_MEDIA_TYPES

MANIFEST_ACCEPT

PLATFORM

RECORDED_CONFIG_KEYS

_AMD64_BASELINE

_ARCHITECTURES

_DEFAULT_VARIANTS

API

class nemo_gym.data.tasks.harbor.registry.ImageConfigError()

Bases: RuntimeError

An image reference could not be resolved to a configuration for the platform the sandboxes run.

class nemo_gym.data.tasks.harbor.registry.ImageRef(
    registry: str,
    repository: str,
    tag: str | None,
    digest: str | None
)

Dataclass

A parsed image reference: [registry/]repository[:tag][@digest].

digest

str | None

manifest_ref

str

pinned_repository

str

The repository a digest pin is written against: as the registry names it, host included off Docker Hub.

registry

str

repository

str

tag

str | None

class nemo_gym.data.tasks.harbor.registry.PlatformMismatchError(
    message: str,
    resolved: str,
    requested: str
)

Bases: ImageConfigError

A single-platform reference resolved to an image of another platform than the one asked for; resolved and requested name the two as os/arch[/variant].

class nemo_gym.data.tasks.harbor.registry.RegistryClient(
    credentials: dict[str, tuple[str, str]] | None = None,
    timeout_s: float = 60,
    retries: int = 3,
    platform: tuple[str, ...] = PLATFORM
)

Read manifests and configuration blobs over the OCI distribution API with the standard library.

credentials maps a registry host, or an image prefix as image_prefix_key spells it (registry/repository-prefix, what credentials_for_image_prefixes returns), to (username, password) for private repositories: a reference takes the longest matching entry’s, a bare host being that whole registry, as the sandbox provider selects an images.auth entry at pull; public ones use the anonymous token the registry hands out. platform is the (os, architecture) or (os, architecture, variant) an image reference is resolved for (linux/amd64 by default), kept normalised (normalize_platform_parts): the manifest an index names for it, matched as docker --platform matches one, and the only one a single-platform reference may be.

_opener

= urllib.request.build_opener(_RedirectHandler())

_tokens

dict[tuple[str, str], str] = {}

credentials

= credentials or {}

platform

= normalize_platform_parts(*platform)

nemo_gym.data.tasks.harbor.registry.RegistryClient._credentials_for(
    ref: nemo_gym.data.tasks.harbor.registry.ImageRef
) -> tuple[str, str] | None

The credentials of the longest entry that covers ref (registry/repository): a bare host key is the whole registry, a prefix key matches as the provider matches images.auth; None when no entry covers the reference, which is then read anonymously.

nemo_gym.data.tasks.harbor.registry.RegistryClient._get(
    ref: nemo_gym.data.tasks.harbor.registry.ImageRef,
    path: str,
    accept: str
) -> tuple[bytes, dict[str, str]]

nemo_gym.data.tasks.harbor.registry.RegistryClient._manifest(
    ref: nemo_gym.data.tasks.harbor.registry.ImageRef,
    manifest_ref: str
) -> tuple[dict[str, typing.Any], str]

nemo_gym.data.tasks.harbor.registry.RegistryClient._open(
    url: str,
    headers: dict[str, str]
) -> tuple[bytes, dict[str, str]]

One GET; connection-level failures (not HTTP errors) are retried with a short backoff.

nemo_gym.data.tasks.harbor.registry.RegistryClient._select_platform(
    ref: nemo_gym.data.tasks.harbor.registry.ImageRef,
    index: dict[str, typing.Any]
) -> str

The digest of the first manifest of index for platform, matched as containerd matches one: an arm64 entry with variant: v8 (the Docker Official Images’ shape) or without one both answer linux/arm64, and an amd64 entry with variant: v1 or none answers linux/amd64, while one at a higher level (v3) does not: it is another platform, selected only when asked for by variant.

nemo_gym.data.tasks.harbor.registry.RegistryClient._token(
    ref: nemo_gym.data.tasks.harbor.registry.ImageRef,
    challenge: str
) -> str

nemo_gym.data.tasks.harbor.registry.RegistryClient.image_config(
    reference: str
) -> dict[str, typing.Any]

One compose-images.json entry for reference.

class nemo_gym.data.tasks.harbor.registry._RedirectHandler()

Bases: HTTPRedirectHandler

Follow a registry redirect without carrying the registry’s token to another host.

Registries answer a blob request with a redirect to a storage host (Docker Hub, GHCR and ECR all do). The OCI distribution spec says a client must not send its Authorization there, and the default handler forwards every header, so the header is dropped whenever the host changes.

nemo_gym.data.tasks.harbor.registry._RedirectHandler.redirect_request(
    req,
    fp,
    code,
    msg,
    headers,
    newurl
)

nemo_gym.data.tasks.harbor.registry.credentials_for_image_prefixes(
    auth: typing.Any,
    where: str = 'images.auth'
) -> dict[str, tuple[str, str]]

RegistryClient credentials from a sandbox provider’s image-prefix auth map.

OpenSandbox’s images.auth maps an image prefix (registry.example.com:5005/, docker.io/org/) to {username, password}, longest matching prefix first at pull. Each prefix keeps its own entry, keyed as image_prefix_key spells it, so the client selects a reference’s credentials the way the provider does: two repositories of one registry with different credentials are each read with their own. auth that is not a mapping of such entries raises ValueError naming where.

nemo_gym.data.tasks.harbor.registry.image_prefix_key(
    prefix: str
) -> str

An images.auth prefix as RegistryClient keys credentials: registry/repository-prefix, the registry read the way Docker reads a reference (parse_image_ref: the first component is a host when it has a dot, a colon or is localhost, else the prefix names a Docker Hub repository, and docker.io is Hub’s registry), the rest kept as written. registry.example.com:5005/ is registry.example.com:5005/, docker.io/org/ is registry-1.docker.io/org/, org/ is registry-1.docker.io/org/ and a bare host is that host’s whole registry. Raises ImageConfigError for text that is not an image prefix.

nemo_gym.data.tasks.harbor.registry.normalize_platform_parts(
    os: typing.Any,
    architecture: typing.Any,
    variant: typing.Any = ''
) -> tuple[str, str, str]

(os, architecture, variant) as containerd spells them, so that two spellings of one platform compare equal: linux/arm64, linux/aarch64 and linux/arm64/v8 are all ("linux", "arm64", "v8"), and linux/amd64/v1 is ("linux", "amd64", ""); linux/amd64/v3 keeps its v3, another platform. A missing part is an empty string.

nemo_gym.data.tasks.harbor.registry.parse_image_ref(
    reference: str
) -> nemo_gym.data.tasks.harbor.registry.ImageRef

Split an image reference the way Docker does, defaulting to Docker Hub.

nemo_gym.data.tasks.harbor.registry.platform_name(
    platform: tuple[str, ...]
) -> str

os/arch[/variant] of a (normalised) platform tuple, the variant shown only when it is not the architecture’s default: linux/amd64, linux/arm64, linux/arm/v6.

nemo_gym.data.tasks.harbor.registry.platform_pair(
    platform: str | None
) -> tuple[str, str, str]

The normalised (os, architecture, variant) of an os/arch[/variant] platform string such as builds.platform (normalize_platform_parts: linux/arm64 and linux/arm64/v8 are one value); None (or an empty string) is linux/amd64, the default platform sandboxes run. An amd64 variant other than the baseline v1 is refused: the records keep an image’s os and architecture only (reusable_record, recorded_image_mismatch), so a configuration read for linux/amd64/v3 would serve a later baseline run.

nemo_gym.data.tasks.harbor.registry.DOCKER_HUB_REGISTRY = 'registry-1.docker.io'

nemo_gym.data.tasks.harbor.registry.INDEX_MEDIA_TYPES = {'application/vnd.oci.image.index.v1+json', 'application/vnd.docker.distribution...

nemo_gym.data.tasks.harbor.registry.MANIFEST_ACCEPT = ', '.join(['application/vnd.oci.image.index.v1+json', 'application/vnd.oci.image...

nemo_gym.data.tasks.harbor.registry.PLATFORM = ('linux', 'amd64')

nemo_gym.data.tasks.harbor.registry.RECORDED_CONFIG_KEYS = ('Cmd', 'Entrypoint', 'Env', 'ExposedPorts', 'Healthcheck', 'Shell', 'User', 'Wo...

nemo_gym.data.tasks.harbor.registry._AMD64_BASELINE = 'v1'

nemo_gym.data.tasks.harbor.registry._ARCHITECTURES = {'x86_64': 'amd64', 'x86-64': 'amd64', 'aarch64': 'arm64', 'armhf': 'arm', 'arme...

nemo_gym.data.tasks.harbor.registry._DEFAULT_VARIANTS = {'arm64': 'v8', 'arm': 'v7'}