Is this page helpful?

Kata Sandbox Workloads (Experimental)#

Warning

This feature is experimental and not fully supported. It is included here as a preview for testing environments and is not recommended for production use cases. There may be changes to functionality, implementation, and APIs in future releases. Kata Containers are the foundational technology for extending confidential computing to native Kubernetes deployments. This release adds support for the Kata sandbox. Support for Confidential Containers is planned for future releases of the NIM Operator.

The NIM Operator leverages the NVIDIA GPU Operator to run NIMs inside Kata Containers. This page outlines how to deploy a NIM workload inside a Kata Sandbox container.

Kata Containers are lightweight Virtual Machines (VMs) that act like containers, but still provide the workload isolation and security advantages of VMs. A Kata container runs in a virtual machine on the host. The virtual machine has a separate operating system and kernel. Hardware virtualization and a separate kernel provide improved workload isolation in comparison with traditional containers.

Running NIM inside Kata containers enables lightweight virtualized isolation for enhanced security.

Note

Confidential Containers and NIM Cache deployments have not been tested and are not supported in this release. This use case has been tested for NIM Service deployments using Kata sandbox containers only. Support for Confidential Containers is planned for a future release.

Procedure:

Install the NIM Operator#

After using the GPU Operator Kata Deploy guide to deploy Kata cCntainers and the GPU Operator to your cluster, install the the NIM Operator:

  1. Add the Helm repository for NVIDIA:

$ helm repo add nvidia https://helm.ngc.nvidia.com/nvidia \
    && helm repo update

  1. Create the Operator namespace:

    $ kubectl create namespace nim-operator

  2. Install the Operator:

    $ helm upgrade --install nim-operator nvidia/k8s-nim-operator -n nim-operator --version=3.1.1

For full install details, refer to the NIM Operator installation instructions.

Deploy NIM in a Kata Container#

  1. Create the nim-service namespace:

    $ kubectl create namespace nim-service

  2. Create an image pull secret in the nim-service namespace for downloading the NIM container image from NVIDIA NGC:

    $ kubectl create secret -n nim-service docker-registry ngc-secret \
    --docker-server=nvcr.io \
    --docker-username='$oauthtoken' \
    --docker-password=<ngc-api-key>

    The sample manifest also references an ngc-api-secret for model downloads. Refer to Image Pull Secrets for the full set of secrets required by NIM Services.

  3. Create a file, llm-kata-sandbox.yaml, based on the sample manifest.

    # NIMService example: Kata VM sandbox (runtimeClassName: kata-qemu-nvidia-gpu); does *not* enable encryption
---
apiVersion: apps.nvidia.com/v1alpha1
kind: NIMService
metadata:
  name: meta-llama-3-2-1b-instruct-kata-sandbox
  namespace: nim-service
spec:
  image:
    repository: nvcr.io/nim/meta/llama-3.2-1b-instruct
    tag: "1.12.0"
    pullPolicy: IfNotPresent
    pullSecrets:
      - ngc-secret
  authSecret: ngc-api-secret
  storage:
    emptyDir:
      sizeLimit: 10Gi
  replicas: 1
  resources:
    limits:
      nvidia.com/pgpu: "1"
      cpu: "8"
      memory: "16Gi"
  expose:
    service:
      type: ClusterIP
      port: 8000
  runtimeClassName: kata-qemu-nvidia-gpu
  userID: 0
  groupID: 0

    The following fields are required to deploy in a Kata container:

    • Set spec.runtimeClassName to kata-qemu-nvidia-gpu. This is the Kata runtime class.

    • Set spec.userID to 0 and spec.groupID to 0.

    • Use the spec.storage.emptyDir field to configure resource requirements for your NIM model.

  4. Apply the manifest:

    $ kubectl apply -f llm-kata-sandbox.yaml

Validate NIM Running in Kata Container#

Confirm the NIMService is running in a Kata container by checking the kernel version in the pod. This should differ from the kernel version running on the host where the pod is running.

  1. Retrieve the NIM pod name and save it as POD.

    $ POD=$(kubectl get pods -n nim-service -o name | grep meta-llama-3-2-1b-instruct-kata-sandbox | head -1 | cut -d/ -f2)

  2. Verify the pod is running inside a Kata environment:

    $ kubectl exec -it $POD -n nim-service -- uname -a

    Example output:

    Linux meta-llama-3-2-1b-instruct-kata-sandbox-5689f9bc67-ljnlf 6.18.12-nvidia-gpu #1 SMP Fri Feb 27 09:33:52 UTC 2026 x86_64 x86_64 x86_64 GNU/Linux

    Note that if you are using a non‑default Kata VM, the output will display the corresponding kernel.