Quickstart: Local k3d Installation

Install a self-hosted NVCF stack on one local k3d cluster, register that cluster with the control plane, and confirm that the local deployment is healthy.

This quickstart uses a single k3d cluster named ncp-local, fake GPUs, and local route hostnames. It is for local development and validation only. For a remote deployment, or for separate control-plane and GPU clusters, use Helmfile Installation and Self-Managed Clusters.

Run the commands from the NVCF repository root unless a step says otherwise. The nvcf-cli self-hosted up command runs on your workstation. It does not run inside Kubernetes.

Prerequisites

Before you start, install and prepare:

• Docker running on your workstation
• k3d v5.x or later
• kubectl
• helm >= 3.14
• helmfile >= 1.0. Use helmfile >= 1.5.0 with Helm 4.
• helm-diff plugin
• nvcf-cli on your PATH. See Installation to build it from the repository or download it from NGC.
• An NGC API key with access to the NVCF chart and image registry
• The NGC organization and team slugs for that registry access

Use these install helpers if you do not already have the local tools.

$
docker version

$
brew install k3d kubectl helm helmfile
$
$
k3d version
$
kubectl version --client
$
helm version
$
helmfile version

$
if helm version --short | grep -q '^v4'; then
$
  helm plugin install https://github.com/databus23/helm-diff --verify=false
$
else
$
  helm plugin install https://github.com/databus23/helm-diff
$
fi
$
$
helm plugin list

$
./setup.sh
$
bazel build //src/clis/nvcf-cli:nvcf-cli
$
$
mkdir -p "${HOME}/.local/bin"
$
install -m 0755 \
>
  bazel-bin/src/clis/nvcf-cli/nvcf-cli_/nvcf-cli \
>
  "${HOME}/.local/bin/nvcf-cli"
$
$
export PATH="${HOME}/.local/bin:${PATH}"
$
nvcf-cli version

self-hosted up defaults to --env local and supports only the single local k3d layout. It requires a current k3d-* kube context.

Step 1: Create the local k3d cluster

Export the registry credentials used by the local cluster bootstrap:

$
export NGC_API_KEY="<ngc-api-key>"
$
export SAMPLE_NGC_ORG="<ngc-org>"
$
export SAMPLE_NGC_TEAM="<ngc-team>"

Create the single-cluster local topology:

$
make -C tools/ncp-local-cluster build-and-deploy-cluster
$
kubectl config use-context k3d-ncp-local
$
kubectl config current-context

Expected output:

k3d-ncp-local

Step 2: Prepare local routing

Add these entries to /etc/hosts on the workstation running the CLI if they do not already resolve:

127.0.0.1 api.localhost
127.0.0.1 api-keys.localhost
127.0.0.1 sis.localhost
127.0.0.1 invocation.localhost

Use the local CLI configuration that points commands at the local routes:

$
export NVCF_CLI_CONFIG=deploy/stacks/self-managed/nvcf-cli-local.yaml

Log in to the NGC container registry:

$
helm registry login nvcr.io -u '$oauthtoken' -p "${NGC_API_KEY}"

Step 3: Create the local stack secrets file

Create the local secrets file used by the self-managed stack:

$
cp deploy/stacks/self-managed/secrets/secrets.yaml.template \
>
  deploy/stacks/self-managed/secrets/local-secrets.yaml
$
$
BASE64_CRED="$(printf '%s' "\$oauthtoken:${NGC_API_KEY}" | base64 | tr -d '\n')"
$
sed -i.bak "s|REPLACE_WITH_BASE64_DOCKER_CREDENTIAL|${BASE64_CRED}|g" \
>
  deploy/stacks/self-managed/secrets/local-secrets.yaml
$
rm deploy/stacks/self-managed/secrets/local-secrets.yaml.bak

Step 4: Run the install

Check local tools and Kubernetes access:

$
nvcf-cli --config "${NVCF_CLI_CONFIG}" self-hosted check --pre

Run the local install:

$
nvcf-cli --config "${NVCF_CLI_CONFIG}" self-hosted up \
>
  --env=local \
>
  --cluster-name=ncp-local \
>
  --nca-id=nvcf-default \
>
  --region=us-west-1 \
>
  --icms-url=http://sis.localhost:8080 \
>
  --stack=deploy/stacks/self-managed \
>
  --refresh-token

Expected result: the final screen reports a successful install, a registered cluster, and a healthy backend.

Step 5: Verify the install

Run the full self-hosted health checks:

$
nvcf-cli --config "${NVCF_CLI_CONFIG}" self-hosted check --all \
>
  --cluster-name=ncp-local

Inspect the local Kubernetes resources:

$
kubectl get pods -A
$
kubectl get httproute -A
$
kubectl get nvcfbackends -A

Expected result: the CLI checks do not report failed checks, and the ncp-local NVCFBackend reports healthy.

Step 6: Deploy and invoke a sample function

Create a function from the load_tester_supreme sample image in the registry organization and team you exported earlier:

$
nvcf-cli --config "${NVCF_CLI_CONFIG}" function create \
>
  --name quickstart-load-tester-supreme \
>
  --image "nvcr.io/${SAMPLE_NGC_ORG}/${SAMPLE_NGC_TEAM}/load_tester_supreme:0.0.8" \
>
  --inference-url /echo \
>
  --inference-port 8000 \
>
  --health-uri /health \
>
  --health-port 8000 \
>
  --health-timeout PT30S

Deploy the function to the local fake GPU backend:

$
nvcf-cli --config "${NVCF_CLI_CONFIG}" function deploy create \
>
  --gpu H100 \
>
  --instance-type NCP.GPU.H100_8x \
>
  --backend ncp-local \
>
  --regions us-west-1 \
>
  --min-instances 1 \
>
  --max-instances 1 \
>
  --timeout 900

Generate an API key for invocation:

$
nvcf-cli --config "${NVCF_CLI_CONFIG}" api-key generate \
>
  --description quickstart-load-tester-supreme \
>
  --scopes invoke_function,list_functions,queue_details,list_functions_details

Invoke the sample function:

$
nvcf-cli --config "${NVCF_CLI_CONFIG}" function invoke \
>
  --request-body '{"message":"quickstart-echo","repeats":1}' \
>
  --timeout 120 \
>
  --poll-duration 5

Expected result: the invocation response contains quickstart-echo.

For other local fake GPU configurations, choose a --gpu and --instance-type that match the discovered node labels and GPU count.

Clean up

Remove the sample function deployment and function:

$
nvcf-cli --config "${NVCF_CLI_CONFIG}" function deploy remove
$
nvcf-cli --config "${NVCF_CLI_CONFIG}" function delete

Remove the compute-plane components:

$
nvcf-cli --config "${NVCF_CLI_CONFIG}" self-hosted uninstall \
>
  --compute-plane \
>
  --cluster-name ncp-local

Remove the control plane:

$
nvcf-cli --config "${NVCF_CLI_CONFIG}" self-hosted uninstall --control-plane

Destroy the local k3d cluster:

$
make -C tools/ncp-local-cluster destroy

Troubleshooting

If the quickstart fails, start with Troubleshooting.

Common local k3d issues:

• sis.localhost must resolve from the workstation running nvcf-cli.

• kubectl config current-context must print k3d-ncp-local.

• lookup api-keys.nvcf.nvidia.com: no such host means the CLI is using the hosted default endpoint. Run commands with --config "${NVCF_CLI_CONFIG}" from this quickstart.

• node inotify limits below NVCA minimums means the local k3d nodes need higher Linux fs.inotify limits. This does not change your macOS shell limits. From the repository root, apply tools/ncp-local-cluster/apps/node-tuning/node-tuning.yaml, wait for the node-tuning DaemonSet in the kube-system namespace, then rerun nvcf-cli --config "${NVCF_CLI_CONFIG}" self-hosted check --pre:

  $
  kubectl apply -f tools/ncp-local-cluster/apps/node-tuning/node-tuning.yaml
  $
  kubectl -n kube-system rollout status ds/node-tuning --timeout=5m

  For non-local clusters, see Node inotify limits.

See Also

• Local Development for local k3d variants and cleanup commands.
• Helmfile Installation for remote or manual control-plane installs.
• Self-Managed Clusters for registering GPU clusters outside the local quickstart.
• src/clis/nvcf-cli/examples/ in this repository for sample CLI input files.