Privilege boundaries
The operator writes Argo CD Applications that Argo CD applies with its own broad identity, and it pushes the charts they install. Treat it as a tier-0 component, like Argo CD itself: installed and upgraded by platform administrators, with its images verified and its actions audited. Its RBAC and admission policies keep each identity’s privileges explicit and stop accidental or casual misuse; they do not contain a fully compromised operator.
Identities
The chart creates no role for editing the ClusterRuntime. Grant it to the
runtime administrators yourself, and keep them apart from approvers, so nobody
both changes the runtime and approves the result.
Deployment validation is cluster-admin-equivalent with every AICR version:
enable it (validation.enabled) only where that is acceptable.
Admission policies
All are ValidatingAdmissionPolicies; the operator uses no webhooks and no mutating admission. Their parameters (registry prefix, namespace allowlist, AICR image) are rendered into the policies themselves, which only administrators can edit.
Constraints on the operator. These bind only the operator’s ServiceAccount and are deleted with it:
Protection of operator state. These bind everyone and are kept on uninstall:
The CRDs add their own rules: a rollout’s spec is immutable except its approval, and an approval or revocation is set once and never cleared.
The aicr AppProject is enforced by Argo CD independently: sources limited to
the registry’s charts/ and wrappers/, destinations limited to the component
namespaces on the in-cluster server.
The runtime registry
The registry holds what the operator mirrors and renders: upstream charts, wrapper charts, bundles and evidence. The operator treats it as storage, not as a trust boundary. Each plan pins every chart by its manifest digest, Argo CD pulls by that digest, and the policies above admit no other source, so an artifact lost or altered in the registry fails the pull instead of deploying different content.
The default in-cluster registry (registry.mode: inCluster):
- runs CNCF Distribution (
ghcr.io/nvidia/aicr-registry), built, scanned and attested with each release like the operator image, under the restricted Pod Security profile; - requires a login for every read and write. The chart generates it on
install, keeps it across upgrades and reinstalls, and places it only where it
is used: the operator, Job and validation namespaces and Argo CD’s
repo-creds; - admits connections only from the operator, the operator’s Jobs in the Job
and validation namespaces (pods carrying the labels the operator puts on
its Jobs) and Argo CD’s repo server, where the CNI enforces NetworkPolicy.
Where it does not, and for anyone allowed
pods/portforwardin the operator namespace (kubectl port-forwardbypasses the policy), the login is the only control; - serves plain HTTP. Traffic between pods is unencrypted unless your CNI or service mesh encrypts it; integrity still rests on the digests;
- is never pulled from by nodes: the operator copies no container images.
With an external registry, its access control is yours, and the operator’s
identity can write anywhere under registry.repository, so give each cluster
its own prefix.
Namespaces and Pod Security
The chart keeps every namespace it creates on uninstall, so removing the
chart never removes the GPU stack. The operator creates no namespaces, and
Argo CD’s CreateNamespace stays off: a plan that renders a release into any
other namespace is refused before it reaches Argo CD (NamespaceNotAllowed).
The operator, the in-cluster registry and AICR’s recipe, bundle and
upgrade-check Jobs run as non-root with a read-only root file system, all
capabilities dropped, RuntimeDefault seccomp and size-limited scratch space.
Credentials
- With the
awsprovider, the operator’s AWS identity mints ECR tokens. It refreshes Argo CD’srepo-credsSecret, and gives each AICR Job that pushes a per-run docker config Secret, owned by the Job and deleted with it, even when the operator is stopped. Scope the identity’s repository prefix to one cluster: it can push there. - With
static, you provide every Secret, and the operator only reads its own copy by name. - With the in-cluster registry, the chart generates the login and its Secrets; the operator only reads its own copy by name.
- Credentials never appear in plans, values, status, Events, logs or evidence; errors name the Secret, never its content. Approvers can read assessment reports but no Secret.
Network egress
The chart ships no egress NetworkPolicy; the in-cluster registry’s ingress policy is above. If you restrict egress, allow:
Tenants
Tenants should have no access to the operator’s kinds, the Argo CD
Applications, the operator, Job, discovery and validation namespaces, or GPU
Operator operands. Give them dedicated namespaces with the restricted Pod
Security profile and quotas. Namespace isolation is not GPU or fabric
isolation; what each profile isolates is part of its qualification
(support policy).