Is this page helpful?

Confidential Computing for AI#

Enabling Zero-Trust AI Factory with Confidential Computing#

In regulated industries such as healthcare and financial services, sensitive data must be protected when in use. In many cases regulation requires the use of confidential computing to satisfy strict compliance, latency, and data residency constraints. Enterprises in regulated industries should not have to choose between self-managed open source models or risk data breach or non-compliance by sharing private data with a cloud-hosted model.

However, deploying proprietary frontier models on shared or private infrastructure (untrusted from the model owners point of view) introduces a complex, three-way “Trust Dilemma” among key stakeholders:

  • Model Owners must protect their algorithmic IP and model weights. They cannot trust that the host operating system, hypervisor, or root infrastructure administrator will not inspect, steal, or extract their proprietary models.

  • Data Owners (Tenants) must ensure their data remains private and secure. They cannot risk infrastructure providers accessing plaintext data during execution, nor can they trust model providers not to misuse or leak data during inference.

  • Infrastructure Providers must protect the underlying cluster, unable to fully trust that tenant workloads are free from malicious code or privilege escalation attempts.

The Vulnerability of Data in Use: Traditional security paradigms break down in these collaborative environments because data in use is not inherently encrypted. While traditional controls like Identity and Access Management (IAM) and network encryption effectively protect data at rest and in transit, data and models must eventually be decrypted in system memory for active processing. This decryption creates a critical exposure window where sensitive data can be exposed and compromised. Hardware-Rooted Zero Trust via Confidential Computing Confidential Computing mitigates this vulnerability by ensuring that data and models remain cryptographically protected throughout the entire lifecycle of execution. It shifts the security boundary away from traditional perimeter defenses to hardware-enforced Trusted Execution Environments (TEEs).

By isolating the execution environment at the hardware level, utilizing CPU TEEs paired with Confidential GPUs, Confidential Computing ensures that even infrastructure administrators with full system privileges cannot access the data or models running inside the enclave. Traditional security defines who is trusted; Confidential Computing minimizes who must be trusted.

Confidential Containers (CoCo) Architecture To maintain the agility, scalability, and orchestration benefits of modern cloud-native environments, this architecture leverages Confidential Containers (CoCo). Instead of sharing the host kernel, standard Kubernetes pods are packaged into lightweight virtual machines with Kata Containers. This allows platform teams to deploy secure microservices and AI pipelines without requiring application rewrites.

Container images and model artifacts are pulled and unpacked strictly inside this encrypted guest environment, ensuring the host infrastructure cannot inspect or tamper with application code.

Cryptographic Attestation and Secure Key Release Trust within this system is not assumed; it is mathematically enforced through a strict remote attestation workflow:

  1. Evidence Collection: The workload generates cryptographic hardware evidence proving the integrity of the TEE.

  2. Validation: An external Attestation Service evaluates this evidence against strict security policies and known-good measurements to ensure the software is untampered.

  3. Secure Key Release: Only after the environment successfully proves it is secure will a Key Broker Service (KBS) release the decryption keys directly into the protected memory enclave.

Because the model and data remain encrypted until they are safely inside the TEE, they are never exposed to the host infrastructure. This verifiable, hardware-backed architecture allows enterprises to confidently deploy AI inference workloads on their most sensitive data, meeting strict compliance mandates while fully protecting proprietary IP.

The Enterprise AI Factory Validated Design now delivers a secure, trusted execution environment (TEE) tailored for running high-performance AI runtimes. It supports frontier models through Kata confidential containers, ensuring data and code isolation from the host operating system, hypervisor, and privileged infrastructure operators. At the same time, it integrates essential attestation services to cryptographically verify enclave integrity, along with robust key management for encryption keys—all critical for production-grade confidential computing workflows. You can learn more about NVIDIA Confidential Computing here.