Attestation Troubleshooting Guide — Attestation SDK Python#
Note: This guide applies only to the Python SDK.
Full list of errors from CC_Admin tool#
The table below displays the various outputs that CC_admin tool can generate and what causes them. Please note that these outputs are generated only while using CC_admin tool and not with attestation SDK. Attestation SDK will only output a claims list as shown in the following section.
ID | Error info | Reason for failure | Mitigations |
---|---|---|---|
1 | Attestation report signature verification failed. | The attestation report may be invalid due to corruption, tampering, or a software bug in the device that generated an incorrect report. | The user should retrieve the attestation report once more and run verification. If the problem persists, they should report the issue to Nvidia. |
2 | No GPU runtime measurements found | ||
3 | Could not parse the GET_MEASUREMENT response message | ||
4 | There are no measurement blocks in the response message | ||
5 | Measurement block at index " XX" not following DMTF specification | ||
6 | No certificates found in certificate chain | ||
7 | The number of certificates fetched from the GPU is unexpected. | The certificate chain must contain 5 certificates, otherwise, it will in this failure. | The user should attempt to reinstall the current driver or install a different version, as specified in the NVIDIA H100 GPU Confidential Computing guide. If the issue persists, the user should report it to Nvidia. |
8 | GPU certificate chain revocation validation failed. | The OCSP validation check for the GPU certificate chain has failed because one or more certificates in the chain have been revoked. | It is recommended that the user stop the use of the current device or software and obtain a replacement |
9 | GPU certificate chain validation failed | The signature validation checks for the GPU certificate chain are not successful. | If the error message indicates that the “verifier_device_root.pem” file is missing, the user may try to reinstall the attestation SDK. If the issue persists, the user should try using a different driver version, as outlined in the NVIDIA H100 GPU Confidential Computing guide and report the issue to Nvidia if it continues. |
10 | Attestation report verification failed | Attestation report verification has failed due to one of the reasons below:
|
Nonce mismatch The user should attempt to verify the attestation again, using either the same or a different driver version. If the problem persists, the user should report the issue to Nvidia. Driver/VBIOS version mismatch To confirm these errors, the user can utilize the nvidia-smi utility to extract the driver/VBIOS versions and cross-check them with the versions mentioned in the attestation report. The user can try reinstalling the driver or using different driver versions. If the issue persists, the user should contact Nvidia for further assistance. Report/Certificate chain retrieval has failed Ensure that the system has CC mode enabled and that the Driver is loaded in persistence mode. To verify that the driver has loaded successfully, the user can use the nvidia-smi conf-compute –f command. Extracting certificates has failed This issue may arise due to a bug in the driver or corruption during the retrieval of the certificate chain. It is recommended to try again with an updated driver version. If the issue persists, the user should report it to Nvidia for further assistance. |
11 | The nonce in the SPDM GET MEASUREMENT request message is not matching with the generated nonce. | ||
12 | The driver version in attestation report is not matching with the driver version fetched from the driver | ||
13 | The vbios version in attestation report is not matching with the vbios version fetched from the driver | ||
14 | Something went wrong while fetching the attestation report from the gpu | ||
15 | Something went wrong while fetching the certificate chains from the gpu. | ||
16 | Something went wrong while extracting the individual certificates from the certificate chain. | ||
17 | Unknown GPU architecture. | The architecture of the detected GPU is not recognized. | To ensure successful confidential computing and attestation, the user must verify that the GPU connected to the system is compatible and that the attestation SDK is updated to the latest version. |
18 | GPU architecture is not supported. | The architecture of the detected GPU is not supported. | To ensure successful confidential computing and attestation, the user must verify that the GPU connected to the system is compatible and that the attestation SDK is updated to the latest version. |
19 | No GPU found | No GPU has been detected in the system. | The user must verify that the GPU is detected on the PCI bus and that the driver is loaded in persistence mode. |
20 | The call to fetch attestation report timed out | Failures due to time out in runtime APIs. | To resolve these issues, the user should verify that the driver is operating in persistence mode and that the system can establish a connection with NVIDIA Remote Attestation services. |
21 | The call to fetch GPU Cert chain timed out | ||
22 | The {function_name} call timed out | ||
23 | Could not fetch the rim file : {rim_id} | The retrieval of Driver or VBIOS RIM was unsuccessful due to the absence of files or problems with the network. | To address these issues, the user should confirm that a connection can be established with NVIDIA Remote Attestation services. Additionally, when using the local verifier tool, the user should ensure that the correct RIM file path is specified as input. |
24 | Could not find the required VBIOS RIM file <path to VBIOS RIM file> | ||
25 | Unable to read <path to Driver RIM file> | ||
26 | No Meta element found in the RIM | Failures due to improperly formed or incorrectly formatted driver or VBIOS RIM. | Steps to try:
If the issue continues, the user should seek assistance from Nvidia and the machine owner. |
27 | No Signature found in the RIM | ||
28 | No KeyInfor found in the RIM | ||
29 | X509Data not found in the RIM | ||
30 | X509Certificates not found in the RIM. | ||
31 | Driver version not found in the RIM | ||
32 | There was a problem while extracting the X509 certificate from the RIM. | ||
33 | No golden measurements found in Driver/VBIOS RIM | ||
34 | Schema validation of Driver/VBIOS RIM failed. | ||
35 | SWID schema file not found | ||
36 | Multiple measurements are assigned to the same index in {self.rim_name} rim | ||
37 | RIM signature verification failed | Failures when there are issues with the validation of the RIM certificate chain | If the error message indicates that the “verifier_device_root.pem” file is missing, the user may try to reinstall the attestation SDK. An OCSP revocation status indicates that the driver or VBIOS is no longer usable, and the user must switch to an unrevoked version. In the event of a RIM verification failure, the user must ensure that the correct, supported versions of the driver and VBIOS are installed. If the issue persists, the user should report it to Nvidia. |
38 | Driver/VBIOS RIM cert chain verification failed | ||
39 | Driver/VBIOS RIM cert chain ocsp status verification failed | ||
40 | Driver/VBIOS RIM verification failed | ||
41 | The runtime measurements are not matching with the golden measurements at the following indexes (starting from 0) |
There is a mismatch between one or more measurements in the attestation report and the reference values from the RIMs. This could be a result of using devtools mode or unsupported versions of the driver or VBIOS. | The user must ensure that the device is booted in production mode. If the issue persists in production mode with supported versions, the user should stop using the system and find a replacement that passes attestation. |
42 | The driver and vbios RIM have measurement at the same index XX | Conflicting measurement indices between the VBIOS and Driver RIMs. A conflicting index is marked as active in both the driver and VBIOS RIM. | The user should try using different versions of the driver or VBIOS and report the issue to Nvidia for further assistance |
43 | Invalid Nonce Size. The nonce should be 32 bytes in length represented as Hex String | Failures due to invalid nonce size. | The user must ensure that the length of Nonce passed to Attestation SDK is 32 bytes and retry attestation. |
44 | Length of Nonce is greater than max nonce size allowed |
NVIDIA Remote Attestation Service – Error codes#
Below is a list of all the error codes returned by the Nvidia Remote Attestation Service (NRAS). In the event of an error, NRAS returns one of these error codes along with an empty claim.
CODE |
ERROR_MESSAGE |
DESCRIPTION |
---|---|---|
4001 |
EMPTY_REQUEST |
Attestation request is empty. |
4002 |
INVALID_REQUEST |
Attestation request is invalid because Attestation report length is less than expected. |
4003 |
INVALID_NONCE |
Nonce in the attestation report is either null or of length 0 |
4004 |
INVALID_GPU_ARCH |
GPU architecture in the attestation report is either null or of length 0 |
4005 |
INVALID_EVIDENCE |
GPU Evidence is either null or of length 0 |
4006 |
INVALID_EVIDENCE_FORMAT |
Attestation Evidence could not be parsed by NRAS. |
4007 |
INVALID_CERTIFICATE_CHAIN |
Certificate chain is invalid, and it does not belong to NVIDIA PKI. |
4008 |
INVALID_GOLDEN_MEASUREMENT |
RIM file data could not be parsed by NRAS. |
4009 |
DRIVER_AND_RIM_MEASUREMENT_SAME_INDEX |
Driver and VBIOS Golden Measurement has measurement at same index |
4010 |
NONCE_NOT_MATCHING |
Nonce from request is not matching with evidence nonce |
4011 |
CERTIFICATE_EXPIRED |
Certificate has expired |
4012 |
GPU_ARCHITECTURE_NOT_SUPPORTED |
GPU Architecture is not one of AMPERE or HOPPER |
4013 |
INVALID_EVIDENCE_SIGNATURE |
Attestation Report Signature is Invalid |
4014 |
INVALID_ATTESTATION_CERTIFICATE_CHAIN |
Attestation Certificate chain doesn’t belong to Nvidia PKI |
4015 |
INVALID_RIM_CERTIFICATE_CHAIN |
RIM Certificate chain doesn’t belong to Nvidia PKI |
4016 |
FWID_NOT_MATCHING |
FWID from the Attestation Report does not match the FWID in the Device Certificate. |
4017 |
INVALID_SWITCH_ARCH |
Invalid Switch Architecture |
4018 |
SWITCH_ARCHITECTURE_NOT_SUPPORTED |
Switch Architecture not supported |
4019 |
INVALID_CLAIMS_VERSION |
Invalid claims version |
4020 |
NO_RIM_META_ELEMENT_FOUND |
No Meta element found in RIM file |
4021 |
PARSE_ERROR |
Colloquial Version not found in the RIM |
5000 |
INTERNAL_SERVER_ERROR |
Internal Server Error |
5001 |
ERROR_DURING_OCSP_QUERY |
Error creating OCSP request or communicating with OCSP service. |
5002 |
CERTIFICATE_STATUS_REVOKED |
OCSP Service returned a “REVOKED” status for the certificate |
5003 |
CERTIFICATE_STATUS_UNKNOWN |
OCSP Service returned a “UNKNOWN” status for the certificate |
5004 |
ERROR_VALIDATING_SIGNATURE |
Error during validating evidence signature |
5005 |
ATTESTATION_TOKEN_FAILURE |
Fail to generate Attestation Token, please retry |
5006 |
GPU_DRIVER_VERSION_NOT_AVAILABLE |
GPU Driver Version not available in evidence |
5007 |
GPU_VBIOS_VERSION_NOT_AVAILABLE |
GPU VBIOS Version not available in evidence |
5008 |
ERROR_DURING_RIM_DOWNLOAD |
NRAS is not able to download RIM file from RIM Service. |
5009 |
RIM_BUNDLE_NOT_FOUND |
RIM file is not found in the RIM Service. |
5010 |
ERROR_PARSING_RIM_CERTIFICATE |
RIM Certificate parsing failed. |
5011 |
INVALID_RIM_CERTIFICATE |
RIM Certificate chain is invalid. |
5012 |
RIM_NOT_SIGNED |
RIM is not signed. |
5013 |
INVALID_RIM_SIGNATURE |
RIM Signature is invalid. |
5014 |
FAIL_TO_VALIDATE_RIM_SIGNATURE |
Parsing error when trying to validate RIM Signature. |
5015 |
ERROR_ATTESTING_EVIDENCE |
Error talking to enclave to Attest the evidence. |
5016 |
NITRO_ATTESTATION_DOCUMENT_FETCH_ERROR |
Fail to download Nitro Attestation Document |
Reporting an issue to Nvidia#
If the remediations above do not help users fix the problems, they can report their issues at NVIDIA/nvtrust#issues.