Using AICR as a Go library

AICR ships as both a CLI and a Go library. External projects that need to resolve validated recipes, generate bundles, or collect observed state can import AICR directly. This page is for those consumers.

Which package to import

Import the github.com/NVIDIA/aicr/pkg/client/v1 package. This is the compatibility-reviewed facade and the surface AICR intends to stabilize at v1.0.

1
import aicr "github.com/NVIDIA/aicr/pkg/client/v1"

The facade provides a single Client type with constructors for the supported recipe sources. Internally it delegates to the functional packages under pkg/*.

You may also import pkg/* subpackages directly, but their APIs are not covered by the same stability guarantees — see the public API surface for the details.

Installing

$
go get github.com/NVIDIA/aicr@latest

For reproducibility in downstream projects, pin a specific tag:

$
go get github.com/NVIDIA/aicr@v0.19.0

Quick start

1
package main
2
3
import (
4
	"context"
5
	"log"
6
	"time"
7
8
	aicr "github.com/NVIDIA/aicr/pkg/client/v1"
9
)
10
11
func main() {
12
	// FilesystemSource layers an external recipe directory over the
13
	// embedded recipe data. Use this in production today; OCISource
14
	// is reserved but not yet implemented (NewClient returns
15
	// ErrCodeUnavailable when given one — see the constructor's
16
	// godoc for the current state).
17
	client, err := aicr.NewClient(
18
		aicr.WithRecipeSource(
19
			aicr.FilesystemSource("/etc/aicr/recipes"),
20
		),
21
	)
22
	if err != nil {
23
		log.Fatal(err)
24
	}
25
	// Always Close when done — releases this Client's cached
26
	// metadata store and component registry from the recipe
27
	// package's per-DataProvider caches.
28
	defer client.Close()
29
30
	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
31
	defer cancel()
32
33
	result, err := client.ResolveRecipe(ctx, aicr.RecipeRequest{
34
		Service:     "eks", // K8s flavour, not cloud vendor — map aws→eks etc. on your side
35
		Region:      "us-east-1",
36
		Accelerator: "h100",
37
		Nodes:       8, // worker-node count, not GPU count
38
		OS:          "ubuntu", // REQUIRED to reach the OS-pinned kubeflow overlay; see "Recipe sources" below
39
		Intent:      "training",
40
		Platform:    "kubeflow",
41
		// Profile:  "gpuStack=operator-managed", // only when the composition declares one (embedded adopter: AKS; values azure-managed [default] / operator-managed)
42
	})
43
	if err != nil {
44
		log.Fatalf("resolve recipe: %v", err)
45
	}
46
47
	log.Printf("resolved recipe %s (%d components)", result.Name, len(result.Components))
48
}

Snapshotting and validation

Beyond recipe resolution, the facade exposes the rest of the Snapshot → Validate workflow. Both methods are stateless w.r.t. the Client’s recipe source; they are surfaced through the Client only to keep the facade uniform and leave room for future per-Client telemetry hooks.

1
// CollectSnapshot deploys a snapshotter Job to the target cluster and
2
// returns the resulting Snapshot. cfg is a facade-owned struct that
3
// mirrors pkg/snapshotter.AgentConfig field for field; the mirror is
4
// enforced by a test, so a field added upstream cannot silently stay at
5
// its zero value here.
6
//
7
// The returned Snapshot carries the parsed form plus Snapshot.Raw — the
8
// exact bytes the agent emitted. Persist Raw rather than re-serializing
9
// the parsed snapshot: a newer agent image can emit fields this module
10
// version does not model, and a typed round trip drops them silently.
11
//
12
// CollectSnapshot itself writes the snapshot nowhere unless AgentConfig.Output
13
// names a ConfigMap (cm://namespace/name), in which case the agent Job stages
14
// it there directly. To persist it anywhere else, hand Raw to
15
// snapshotter.DeliverSnapshot — a file, stdout, a ConfigMap, or a Go template
16
// render — which is what `aicr snapshot` does.
17
//
18
// On AKS, set AKSGPUPoolsPath to an `az aks nodepool list -o json` dump
19
// on the machine running this client: the pool projection is merged
20
// controller-side into the returned snapshot, and AKS profile-qualified
21
// resolution from that snapshot REQUIRES the resulting
22
// K8s.aks-gpu-pools.gpu-driver reading (a snapshot without it fails
23
// closed).
24
// Give the Job-backed snapshot its own deadline: contexts cap the
25
// configured timeouts from the parent side, so reusing the 30-second
26
// resolve ctx above would override the 5-minute AgentConfig.Timeout.
27
snapCtx, cancelSnap := context.WithTimeout(context.Background(), 10*time.Minute)
28
defer cancelSnap()
29
snap, err := client.CollectSnapshot(snapCtx, &aicr.AgentConfig{
30
	Kubeconfig:         "/path/to/target-kubeconfig",
31
	Namespace:          "aicr-snapshot",
32
	Image:              "ghcr.io/nvidia/aicr:v0.11.1",
33
	ServiceAccountName: "aicr-agent",
34
	Timeout:            5 * time.Minute,
35
	Cleanup:            true,
36
	AKSGPUPoolsPath:    "/path/to/aks-gpu-pools.json", // AKS only
37
})
38
if err != nil {
39
	log.Fatalf("collect snapshot: %v", err)
40
}
41
42
// NOTE: AgentConfig.AKSGPUPoolsPath and ResolveRecipeFromSnapshotWithProfile
43
// require the release containing the AKS gpuStack adoption (PR #1967) —
44
// newer than the module pin shown under Installation; update the pin to
45
// that release when reproducing this example.
46
// On AKS, resolve FROM the collected snapshot so the profile selection is
47
// verified against the recorded pool modes (ResolveRecipeFromSnapshot uses
48
// the declaration default, azure-managed, which requires pools reading
49
// Install; gpuStack=operator-managed as below requires a pool dump reading None —
50
// i.e. pools created with --gpu-driver none). A snapshot whose reading
51
// mismatches the selection — or that was collected without
52
// AKSGPUPoolsPath — fails closed.
53
resolveCtx, cancelResolve := context.WithTimeout(context.Background(), 30*time.Second)
54
defer cancelResolve()
55
aksResult, err := client.ResolveRecipeFromSnapshotWithProfile(resolveCtx,
56
	&aicr.Criteria{
57
		Service:     "aks",
58
		Accelerator: "h100",
59
		OS:          "ubuntu",
60
		Intent:      "training",
61
	}, snap, "gpuStack=operator-managed")
62
if err != nil {
63
	log.Fatalf("resolve from snapshot: %v", err)
64
}
65
66
// ValidateState runs the validation phases against the resolved recipe +
67
// observed snapshot. Pass the same kubeconfig you used for snapshot collection
68
// so that namespace, RBAC, ConfigMap, validator Job, and result operations all
69
// target that cluster. With no WithValidationPhases option it runs all three
70
// phases (Deployment, Conformance, Performance) in canonical order.
71
// Validation runs cluster Jobs per phase and can take well over an
72
// hour on the performance phase — bound it independently of the short
73
// resolve context (the SDK's own per-phase caps still apply inside).
74
valCtx, cancelVal := context.WithTimeout(context.Background(), 2*time.Hour)
75
defer cancelVal()
76
// WithValidationTimeout(0) removes the facade's default 75-minute
77
// operation cap (a per-check ordering guarantee, not a bound on a
78
// serial all-phase run) so valCtx above is the governing deadline.
79
results, err := client.ValidateState(valCtx, aksResult, snap,
80
	aicr.WithValidationKubeconfig("/path/to/target-kubeconfig"),
81
	aicr.WithValidationTimeout(0))
82
if err != nil {
83
	log.Fatalf("validate state: %v", err)
84
}
85
for _, r := range results {
86
	log.Printf("phase=%s status=%s duration=%s", r.Phase, r.Status, r.Duration)
87
}

When WithValidationKubeconfig is omitted or passed an empty string, ValidateState uses the shared default Kubernetes client and its standard discovery chain: KUBECONFIG, ~/.kube/config, then in-cluster configuration. When an explicit path is provided, the SDK reloads that kubeconfig and creates a fresh client for each validation run. The run reuses that client for all of its Kubernetes operations.

The recipe argument to ValidateState MUST be the *RecipeResult returned by the same Client’s ResolveRecipe (or LoadRecipe) call — the unexported internal recipe state is required for constraint evaluation.

To restrict the run to specific phases, pass WithValidationPhases in the order you want them executed:

1
results, err := client.ValidateState(ctx, result, snap,
2
	aicr.WithValidationPhases(aicr.PhaseDeployment, aicr.PhaseConformance))

Valid phase values are PhaseDeployment, PhaseConformance, and PhasePerformance (canonical execution order). An unrecognized phase is rejected with ErrCodeInvalidRequest before any cluster work, so a typo cannot silently degrade to an empty run.

Loading an existing recipe

When a recipe has already been resolved and persisted (for example a recipe file checked into a GitOps repo, or a cm:// ConfigMap URI), load it back through the same Client with LoadRecipe instead of re-resolving from criteria:

1
result, err := client.LoadRecipe(ctx, "/etc/aicr/recipe.yaml", "")
2
if err != nil {
3
	log.Fatalf("load recipe: %v", err)
4
}

LoadRecipe hydrates overlay inputs (kind: RecipeMetadata) against the Client’s own data provider and returns a Client-owned *RecipeResult ready for ValidateState / BundleComponents — it passes the same ownership check as a ResolveRecipe result. An already-hydrated RecipeResult file is returned with its provider bound to the Client. For a profile-bearing overlay, the effective declaration resolved from that provider must structurally match the file’s declaration after JSON normalization; otherwise loading fails rather than returning a recipe selected from a different profile contract. Note that bundle generation runs blocking preflight validations (for example CheckDriverOwnershipCoherence, which rejects a recipe whose snapshot recorded gpuDriverState: absent under a preinstalled-driver profile). For recipes carrying metadata.selectedProfile (the AKS family), the remedy is out-of-band: fix or recreate the GPU pools, recapture the snapshot, and regenerate — the driver-ownership paths are profile-owned, so --set overrides diverging from the selected value are rejected. Only legacy pre-profile artifacts are remedied through --set override flags, whose SDK surface is MakeBundle with BundleOptions.Config — BundleComponents takes no overrides, so a blocked legacy recipe must be bundled through MakeBundle (or regenerated) rather than retried on the same call. The kubeconfig argument (third parameter) is only needed when the recipe path (first argument) is a cm:// ConfigMap URI.

For unit tests that exercise the facade surface without a live cluster, pass aicr.WithValidationNoCluster(true): every check reports as “skipped - no-cluster mode” and no Kubernetes resources are created. Other facade options (WithValidationNamespace, WithValidationRunID, WithValidationCleanup, WithValidationImagePullSecrets, WithValidationTolerations, WithValidationNodeSelector, WithValidationKubeconfig) cover the production-controller knobs.

Recipe sources

AICR exposes one production recipe source today; pick it via aicr.WithRecipeSource:

EmbeddedSource resolves against the recipe data compiled into the AICR binary — no filesystem path required. Use it when you want AICR’s bundled recipe data and no local overrides. FilesystemSource layers an external directory over that same embedded data, so files in the directory override their embedded equivalents.

Client options

Beyond WithRecipeSource, NewClient accepts these functional options:

1
allowLists, err := aicr.ParseAllowListsFromEnv()
2
if err != nil {
3
	log.Fatal(err)
4
}
5
6
client, err := aicr.NewClient(
7
	aicr.WithRecipeSource(aicr.EmbeddedSource()),
8
	aicr.WithVersion("1.2.3"),
9
	aicr.WithAllowLists(allowLists),
10
)

• WithVersion(version string) stamps the given version string into resolved recipe metadata (accessible via result.Resolved().Metadata.Version). Typically the consuming binary’s build version.
• WithAllowLists(al *AllowLists) fences which criteria values the Client’s resolve path accepts. A resolve whose criteria fall outside the allowlist is rejected before the recipe is built. Pass nil (or omit the option) to allow all values.
• ParseAllowListsFromEnv() builds an AllowLists from the AICR_ALLOWED_ACCELERATORS, AICR_ALLOWED_SERVICES, AICR_ALLOWED_INTENTS, and AICR_ALLOWED_OS environment variables. It returns nil when none are set — WithAllowLists treats a nil AllowLists as allow-all, so the result is always safe to pass straight to WithAllowLists.

AllowLists is a facade-owned struct whose Accelerators, Services, Intents, and OSTypes fields are plain []string slices, so callers can construct one directly without depending on pkg/recipe’s enum identifiers. When you already hold a pkg/recipe.AllowLists, use aicr.WrapAllowLists to project it onto the facade shape.

Resolving from criteria

ResolveRecipe takes the stable RecipeRequest shape and returns the facade RecipeResult — a deliberately small struct exposing the Name, Version, Components, and optional SelectedProfile of the resolved recipe. Set RecipeRequest.Profile to the exact name=value selection when the resolved composition declares a profile. Empty applies the declaration’s required default; a nonempty selection against an unprofiled composition fails closed.

Components lists enabled (deployable) components only; disabled refs remain visible via Resolved().ComponentRefs. When you already hold an *aicr.Criteria value — for example, a REST handler that parsed criteria from an incoming HTTP request and wrapped them with aicr.WrapCriteria — use ResolveRecipeFromCriteria. Use ResolveRecipeFromCriteriaWithProfile for an explicit selection and ResolveRecipeFromSnapshotWithProfile for snapshot-filtered resolution. These return the same facade *RecipeResult; call result.Resolved() when you need the complete underlying *pkg/recipe.RecipeResult (constraints, deployment order, validation config, metadata):

1
rec, err := client.ResolveRecipeFromCriteria(ctx, aicr.WrapCriteria(criteria))
2
if err != nil {
3
	log.Fatalf("resolve recipe: %v", err)
4
}
5
6
// Facade surface — Name, Version, Components.
7
log.Printf("recipe %s components: %d", rec.Name, len(rec.Components))
8
if rec.SelectedProfile != nil {
9
	log.Printf("profile %s=%s", rec.SelectedProfile.Name, rec.SelectedProfile.Value)
10
}
11
12
// Full upstream shape, when needed.
13
resolved := rec.Resolved()
14
log.Printf("recipe constraints: %d", len(resolved.Constraints))

For a per-resolution Slurm accounting mode, use ResolveRecipeFromCriteriaWithOptions or ResolveRecipeFromSnapshotWithOptions with aicr.WithAccountingMode("customer-managed"). The original criteria and snapshot method signatures remain unchanged for source compatibility.

The returned *RecipeResult carries:

• Name, Version, TranslatedAt — stable identity
• Components — []ComponentRef (Name, Kind, Version, Source, Chart, Namespace)
• SelectedProfile — selected name/value and declaration-wide OwnedPaths; nil for legacy recipes
• Resolved() — the upstream *pkg/recipe.RecipeResult for callers that need constraints, deployment order, validation config, or metadata (e.g., evidence emission). Do not mutate; do not retain past the facade RecipeResult’s lifetime — marshal first if persistence is needed.

Criteria is a facade-owned struct whose enum-typed fields project to plain strings, decoupling the public surface from pkg/recipe’s enum identifiers. Construct one directly or wrap an upstream *pkg/recipe.Criteria via aicr.WrapCriteria. Allowlist enforcement (WithAllowLists) applies here just as it does on ResolveRecipe; a nil Client, nil context, or nil criteria each return ErrCodeInvalidRequest, and the same facade-level timeout bounds the resolve.

ListCatalog projects the effective inherited profile declaration on each entry as CatalogEntry.Profile. The summary contains its name, description, required default, and sorted value names; it is nil when the composition is unprofiled.

To extract a single value from a resolved recipe, use SelectFromRecipeWithContext with a dot-path selector. It hydrates the recipe’s component values and returns the value at the path; an empty selector returns the entire hydrated structure, and a nil *RecipeResult returns ErrCodeInvalidRequest. Hydration reads values files through the recipe’s DataProvider, so the context bounds real I/O — cancel it and the hydration aborts. This is the same call the aicr query CLI command and the REST query handler run:

1
v, err := aicr.SelectFromRecipeWithContext(ctx, rec, "components.gpu-operator.values.driver.version")
2
if err != nil {
3
	log.Fatalf("select: %v", err)
4
}
5
log.Printf("driver version: %v", v)

SelectFromRecipe is the context-less form, kept for source compatibility. It derives a defaults.FileReadTimeout-bounded context internally, so the reads stay bounded but the caller cannot cancel them. Prefer the context-aware form wherever a context.Context is available.

The outermost structured code distinguishes the two failure stages, so a caller can shape a response without reimplementing hydrate-then-select: ErrCodeNotFound means the selector path does not exist, and any other code (ErrCodeInternal, ErrCodeTimeout, …) means hydration failed. Match with errors.As on the outermost error rather than errors.Is — Is walks the wrap chain and would match an ErrCodeNotFound cause nested inside a hydration failure.

Delivering a collected snapshot

snapshotter.DeliverSnapshot(ctx, raw, snapshotter.SnapshotDelivery{...}) writes captured bytes to a destination independent of where the agent staged them:

1
err = snapshotter.DeliverSnapshot(snapCtx, snap.Raw, snapshotter.SnapshotDelivery{
2
	Output:     "snapshot.yaml",                 // file; "" or "-" for stdout; cm://ns/name for a ConfigMap
3
	Kubeconfig: "/path/to/target-kubeconfig",    // only used for a cm:// Output
4
})

A cm:// destination is written, not assumed — including when it differs from the AgentConfig.Output used at collection time. Set TemplatePath to render through a Go template instead of copying bytes; Output then names the rendered report.

WrapResolved turns a *pkg/recipe.RecipeResult — typically one taken from RecipeResult.Resolved() and then projected by the caller — back into a facade *RecipeResult that SelectFromRecipeWithContext accepts. The result is queryable only: it carries no owning Client, so MakeBundle, BundleComponents, and ValidateState reject it. Use Client.AdoptRecipe when you need a bundle-able result.

Errors

All errors returned by the facade are *pkg/errors.StructuredError values carrying an ErrorCode. Use errors.As to inspect:

1
import (
2
	stderrors "errors"
3
	aicr "github.com/NVIDIA/aicr/pkg/client/v1"
4
	aicrerrors "github.com/NVIDIA/aicr/pkg/errors"
5
)
6
7
_, err := client.ResolveRecipe(ctx, req)
8
var se *aicrerrors.StructuredError
9
if stderrors.As(err, &se) && se.Code == aicrerrors.ErrCodeInvalidRequest {
10
	// handle invalid input
11
}

Context handling

ResolveRecipe (and every other context-aware facade method) honours context cancellation. Each facade entry point unconditionally wraps the caller’s context with context.WithTimeout against its per-operation cap. The effective deadline is the smaller of the caller’s deadline and the facade cap, per context.WithTimeout semantics — a caller passing a tighter deadline keeps it; a caller passing context.Background() gets the facade cap.

Per-operation caps:

• ResolveRecipe / BundleComponents: defaults.RecipeOperationTimeout
• CollectSnapshot: caller-controlled via AgentConfig.Timeout (falling back to defaults.SnapshotOperationTimeout when unset), plus defaults.SnapshotOperationGrace. The grace exists because AgentConfig.Timeout budgets Job completion only — deployment and result retrieval sit outside it, so a bare cap would silently shrink the completion budget you asked for.
• ValidateState: defaults.ValidationOperationTimeout
• MakeBundle: opt-in via BundleOptions.Timeout. When unset (0) the caller’s context governs unchanged — large bundles, --vendor-charts, and attestation/signing can exceed any fixed cap. The REST /v1/bundle handler sets it to defaults.BundleHandlerTimeout; the CLI bundle command leaves it 0.

Passing a nil context.Context returns ErrCodeInvalidRequest. Use context.Background() (or a deadline-bounded child) for unbounded callers.

Compatibility

Today AICR is pre-1.0. Under Go module versioning, a v0 minor release may contain breaking API changes. The project mechanically detects and explicitly records incompatible changes to the facade, but consumers must pin a patch version in go.mod and audit upgrades.

Starting with v1.0, the facade’s exported API follows Semantic Versioning:

• Major bumps may rename, remove, or change the shape of exported types and function signatures.
• Minor bumps may add new exported types, fields, or methods.
• Patch bumps contain compatible bug fixes.

See also

• Public API surface — stability matrix per package
• Automation guide — CI integration patterns
• Recipe development — authoring recipes