Local Development (k3d)

View as Markdown

Run a full NVCF self-hosted stack on your laptop using k3d for development, testing, or demos. The canonical local-k3d tooling lives at tools/ncp-local-cluster/ in this repo.

This setup is for local development only. It uses fake GPUs, a single Cassandra replica, and ephemeral storage. Do not use this for production workloads.

Clone the public repository before using any local development flow:

$git clone https://github.com/nvidia/nvcf.git
$cd nvcf

Pick a flow

Four canonical flows are documented, one per page.

TopologyInstall pathPage
Single-clusterCLI (nvcf-cli self-hosted install)Single-cluster CLI
Single-clusterHelmfile (make install HELMFILE_ENV=...)Single-cluster Helmfile
Multi-clusterCLI (nvcf-cli self-hosted install)Multi-cluster CLI
Multi-clusterHelmfile (make install HELMFILE_ENV=...)Multi-cluster Helmfile

Topologies

Single-cluster brings up one k3d cluster named ncp-local. Control plane and compute plane share the cluster. The fastest path for function-lifecycle testing and basic install validation.

Multi-cluster brings up ncp-local-cp plus ncp-local-compute-N (N=1 by default). Control plane lives on the cp cluster; compute plane on the compute cluster. Required when you need to exercise cross-cluster registration, the OIDC/JWKS discovery flow, or the .test hostname routing the cp Gateway exposes to compute workers.

The two topologies are mutually exclusive: both claim host ports 8080/8443/4222. Destroy one before bringing up the other:

$# from single-cluster -> multi-cluster
$make -C tools/ncp-local-cluster destroy
$
$# from multi-cluster -> single-cluster
$make -C tools/ncp-local-cluster destroy-multicluster

Install paths

CLI drives the install through nvcf-cli self-hosted install --control-plane (writes a control-plane profile YAML), nvcf-cli init (mints the admin JWT against the live api-keys service), and nvcf-cli compute-plane register/install (writes compute-plane values and applies them). The CLI manages URL block selection (in-cluster vs cross-cluster reachable) based on the kube contexts you pass.

Helmfile drives the install through split Make targets: deploy/stacks/self-managed/Makefile for control plane (make template, make install) and deploy/stacks/nvcf-compute-plane/Makefile for compute plane (make register-cluster, make install). The operator authors topology-correct URLs into an environment file (different fixture per topology) instead of relying on a CLI-managed profile.

The two install paths intentionally diverge. See tests/bdd/AGENTS.md (the “CLI vs Helmfile install paths” section) for the rationale.

Prerequisites (common to all flows)

  • Docker (running)

  • k3d v5.x or later

  • kubectl

  • helm >= 3.12

  • helmfile >= 1.1.0, < 1.2.0 (Helmfile flows only)

  • helm-diff plugin (Helmfile flows only): helm plugin install https://github.com/databus23/helm-diff

  • An NGC API key with access to the NVCF chart and image registry.

  • nvcf-cli built from this repo:

    $go build -C src/clis/nvcf-cli -o ../../../nvcf-cli .

Each flow page restates exactly the prerequisites that flow needs.

Cleanup

The BDD suite ships destructive cleanup helpers reused for hand-driven local dev:

ScopeCommand
Stack-only (helm releases) on single-clustertests/bdd/scripts/destroy-stack.sh single
Stack-only on multi-clustertests/bdd/scripts/destroy-stack.sh multi
Whole single-cluster topologymake -C tools/ncp-local-cluster destroy
Whole multi-cluster topologymake -C tools/ncp-local-cluster destroy-multicluster
Every ncp-local* k3d cluster on the hostmake -C tools/ncp-local-cluster destroy-all-ncp-local