Attestation Troubleshooting Guide — Attestation SDK Python#

Note: This guide applies only to the Python SDK.

Full list of errors from CC_Admin tool#

The table below displays the various outputs that CC_admin tool can generate and what causes them. Please note that these outputs are generated only while using CC_admin tool and not with attestation SDK. Attestation SDK will only output a claims list as shown in the following section.

ID Error info Reason for failure Mitigations
1 Attestation report signature verification failed. The attestation report may be invalid due to corruption, tampering, or a software bug in the device that generated an incorrect report. The user should retrieve the attestation report once more and run verification. If the problem persists, they should report the issue to Nvidia.
2 No GPU runtime measurements found
3 Could not parse the GET_MEASUREMENT response message
4 There are no measurement blocks in the response message
5 Measurement block at index " XX" not following DMTF specification
6 No certificates found in certificate chain
7 The number of certificates fetched from the GPU is unexpected. The certificate chain must contain 5 certificates, otherwise, it will in this failure. The user should attempt to reinstall the current driver or install a different version, as specified in the NVIDIA H100 GPU Confidential Computing guide. If the issue persists, the user should report it to Nvidia.
8 GPU certificate chain revocation validation failed. The OCSP validation check for the GPU certificate chain has failed because one or more certificates in the chain have been revoked. It is recommended that the user stop the use of the current device or software and obtain a replacement
9 GPU certificate chain validation failed The signature validation checks for the GPU certificate chain are not successful. If the error message indicates that the “verifier_device_root.pem” file is missing, the user may try to reinstall the attestation SDK. If the issue persists, the user should try using a different driver version, as outlined in the NVIDIA H100 GPU Confidential Computing guide and report the issue to Nvidia if it continues.
10 Attestation report verification failed

Attestation report verification has failed due to one of the reasons below:

  • Nonce mismatch in GET_MEASUREMENTS

  • Driver version mismatch

  • VBIOS version mismatch.

  • Report retrieval has failed

  • GPU certificate chain retrieval has failed

  • Extracting individual certificate has failed

Nonce mismatch

The user should attempt to verify the attestation again, using either the same or a different driver version. If the problem persists, the user should report the issue to Nvidia.

Driver/VBIOS version mismatch

To confirm these errors, the user can utilize the nvidia-smi utility to extract the driver/VBIOS versions and cross-check them with the versions mentioned in the attestation report. The user can try reinstalling the driver or using different driver versions. If the issue persists, the user should contact Nvidia for further assistance.

Report/Certificate chain retrieval has failed

Ensure that the system has CC mode enabled and that the Driver is loaded in persistence mode. To verify that the driver has loaded successfully, the user can use the nvidia-smi conf-compute –f command.

Extracting certificates has failed

This issue may arise due to a bug in the driver or corruption during the retrieval of the certificate chain. It is recommended to try again with an updated driver version. If the issue persists, the user should report it to Nvidia for further assistance.

11 The nonce in the SPDM GET MEASUREMENT request message is not matching with the generated nonce.
12 The driver version in attestation report is not matching with the driver version fetched from the driver
13 The vbios version in attestation report is not matching with the vbios version fetched from the driver
14 Something went wrong while fetching the attestation report from the gpu
15 Something went wrong while fetching the certificate chains from the gpu.
16 Something went wrong while extracting the individual certificates from the certificate chain.
17 Unknown GPU architecture. The architecture of the detected GPU is not recognized. To ensure successful confidential computing and attestation, the user must verify that the GPU connected to the system is compatible and that the attestation SDK is updated to the latest version.
18 GPU architecture is not supported. The architecture of the detected GPU is not supported. To ensure successful confidential computing and attestation, the user must verify that the GPU connected to the system is compatible and that the attestation SDK is updated to the latest version.
19 No GPU found No GPU has been detected in the system. The user must verify that the GPU is detected on the PCI bus and that the driver is loaded in persistence mode.
20 The call to fetch attestation report timed out Failures due to time out in runtime APIs. To resolve these issues, the user should verify that the driver is operating in persistence mode and that the system can establish a connection with NVIDIA Remote Attestation services.
21 The call to fetch GPU Cert chain timed out
22 The {function_name} call timed out
23 Could not fetch the rim file : {rim_id} The retrieval of Driver or VBIOS RIM was unsuccessful due to the absence of files or problems with the network. To address these issues, the user should confirm that a connection can be established with NVIDIA Remote Attestation services. Additionally, when using the local verifier tool, the user should ensure that the correct RIM file path is specified as input.
24 Could not find the required VBIOS RIM file <path to VBIOS RIM file>
25 Unable to read <path to Driver RIM file>
26 No Meta element found in the RIM Failures due to improperly formed or incorrectly formatted driver or VBIOS RIM.

Steps to try:

  1. Reinstall the attestation SDK.

  2. Attempt to retrieve the RIMs again.

  3. Switch to a new driver version.

If the issue continues, the user should seek assistance from Nvidia and the machine owner.

27 No Signature found in the RIM
28 No KeyInfor found in the RIM
29 X509Data not found in the RIM
30 X509Certificates not found in the RIM.
31 Driver version not found in the RIM
32 There was a problem while extracting the X509 certificate from the RIM.
33 No golden measurements found in Driver/VBIOS RIM
34 Schema validation of Driver/VBIOS RIM failed.
35 SWID schema file not found
36 Multiple measurements are assigned to the same index in {self.rim_name} rim
37 RIM signature verification failed Failures when there are issues with the validation of the RIM certificate chain

If the error message indicates that the “verifier_device_root.pem” file is missing, the user may try to reinstall the attestation SDK.

An OCSP revocation status indicates that the driver or VBIOS is no longer usable, and the user must switch to an unrevoked version. In the event of a RIM verification failure, the user must ensure that the correct, supported versions of the driver and VBIOS are installed. If the issue persists, the user should report it to Nvidia.

38 Driver/VBIOS RIM cert chain verification failed
39 Driver/VBIOS RIM cert chain ocsp status verification failed
40 Driver/VBIOS RIM verification failed
41 The runtime measurements are not matching with the
golden measurements at the following indexes (starting from 0)
There is a mismatch between one or more measurements in the attestation report and the reference values from the RIMs. This could be a result of using devtools mode or unsupported versions of the driver or VBIOS. The user must ensure that the device is booted in production mode. If the issue persists in production mode with supported versions, the user should stop using the system and find a replacement that passes attestation.
42 The driver and vbios RIM have measurement at the same index XX Conflicting measurement indices between the VBIOS and Driver RIMs. A conflicting index is marked as active in both the driver and VBIOS RIM. The user should try using different versions of the driver or VBIOS and report the issue to Nvidia for further assistance
43 Invalid Nonce Size. The nonce should be 32 bytes in length represented as Hex String Failures due to invalid nonce size. The user must ensure that the length of Nonce passed to Attestation SDK is 32 bytes and retry attestation.
44 Length of Nonce is greater than max nonce size allowed

NVIDIA Remote Attestation Service – Error codes#

Below is a list of all the error codes returned by the Nvidia Remote Attestation Service (NRAS). In the event of an error, NRAS returns one of these error codes along with an empty claim.

CODE

ERROR_MESSAGE

DESCRIPTION

4001

EMPTY_REQUEST

Attestation request is empty.

4002

INVALID_REQUEST

Attestation request is invalid because Attestation report length is less than expected.

4003

INVALID_NONCE

Nonce in the attestation report is either null or of length 0

4004

INVALID_GPU_ARCH

GPU architecture in the attestation report is either null or of length 0

4005

INVALID_EVIDENCE

GPU Evidence is either null or of length 0

4006

INVALID_EVIDENCE_FORMAT

Attestation Evidence could not be parsed by NRAS.

4007

INVALID_CERTIFICATE_CHAIN

Certificate chain is invalid, and it does not belong to NVIDIA PKI.

4008

INVALID_GOLDEN_MEASUREMENT

RIM file data could not be parsed by NRAS.

4009

DRIVER_AND_RIM_MEASUREMENT_SAME_INDEX

Driver and VBIOS Golden Measurement has measurement at same index

4010

NONCE_NOT_MATCHING

Nonce from request is not matching with evidence nonce

4011

CERTIFICATE_EXPIRED

Certificate has expired

4012

GPU_ARCHITECTURE_NOT_SUPPORTED

GPU Architecture is not one of AMPERE or HOPPER

4013

INVALID_EVIDENCE_SIGNATURE

Attestation Report Signature is Invalid

4014

INVALID_ATTESTATION_CERTIFICATE_CHAIN

Attestation Certificate chain doesn’t belong to Nvidia PKI

4015

INVALID_RIM_CERTIFICATE_CHAIN

RIM Certificate chain doesn’t belong to Nvidia PKI

4016

FWID_NOT_MATCHING

FWID from the Attestation Report does not match the FWID in the Device Certificate.

4017

INVALID_SWITCH_ARCH

Invalid Switch Architecture

4018

SWITCH_ARCHITECTURE_NOT_SUPPORTED

Switch Architecture not supported

4019

INVALID_CLAIMS_VERSION

Invalid claims version

4020

NO_RIM_META_ELEMENT_FOUND

No Meta element found in RIM file

4021

PARSE_ERROR

Colloquial Version not found in the RIM

5000

INTERNAL_SERVER_ERROR

Internal Server Error

5001

ERROR_DURING_OCSP_QUERY

Error creating OCSP request or communicating with OCSP service.

5002

CERTIFICATE_STATUS_REVOKED

OCSP Service returned a “REVOKED” status for the certificate

5003

CERTIFICATE_STATUS_UNKNOWN

OCSP Service returned a “UNKNOWN” status for the certificate

5004

ERROR_VALIDATING_SIGNATURE

Error during validating evidence signature

5005

ATTESTATION_TOKEN_FAILURE

Fail to generate Attestation Token, please retry

5006

GPU_DRIVER_VERSION_NOT_AVAILABLE

GPU Driver Version not available in evidence

5007

GPU_VBIOS_VERSION_NOT_AVAILABLE

GPU VBIOS Version not available in evidence

5008

ERROR_DURING_RIM_DOWNLOAD

NRAS is not able to download RIM file from RIM Service.

5009

RIM_BUNDLE_NOT_FOUND

RIM file is not found in the RIM Service.

5010

ERROR_PARSING_RIM_CERTIFICATE

RIM Certificate parsing failed.

5011

INVALID_RIM_CERTIFICATE

RIM Certificate chain is invalid.

5012

RIM_NOT_SIGNED

RIM is not signed.

5013

INVALID_RIM_SIGNATURE

RIM Signature is invalid.

5014

FAIL_TO_VALIDATE_RIM_SIGNATURE

Parsing error when trying to validate RIM Signature.

5015

ERROR_ATTESTING_EVIDENCE

Error talking to enclave to Attest the evidence.

5016

NITRO_ATTESTATION_DOCUMENT_FETCH_ERROR

Fail to download Nitro Attestation Document

Reporting an issue to Nvidia#

If the remediations above do not help users fix the problems, they can report their issues at NVIDIA/nvtrust#issues.