IMEX GSSAPI Kerberos support#

As mentioned in “Multi-Node Memory Sharing Model” on page 9, IMEX relies on the gRPC framework for inter-node communication, which supports the SSL/TLS security mechanism. With the CUDA 12.8 release, IMEX provides integration with GSSAPI using Kerberos for inter-node mutual authentication and encryption of memory message exchanges.

About GSSAPI Kerberos#

GSSAPI: Generic Security Services is an application program interface (API) that, as defined by RFC 2743, provides a generic authentication and secure messaging interface.

Kerberos is a widely deployed network authentication protocol that provides strong authentication for client/server applications by using secret-key cryptography (AES).

After a client and server has used Kerberos to prove their identity, they can also encrypt all their communication to assure privacy and data integrity.

IMEX supports GSSAPI with MIT Kerberos v5 as the underlying security mechanism.

Terminology#

Table 7‑1 provides a list of terms used in this guide.

Table 7‑1. Terminology

Term

Definition

Key Distribution Center (KDC)

A trusted server that issues Kerberos tickets to clients and servers to communicate securely.

Principal

A user, a program, or a machine. Can be clients or servers.

Realm

A logical network, like a domain, which defines a group of systems under the same KDC.

Ticket

A block of data that is presented as the user’s credentials when attempting to access a Kerberized service. A ticket contains information about the user’s identity and an encryption key.

Security Context

A “state of trust” between two applications that know each other and can permit data transfers.

Enabling GSSAPI Kerberos#

These are the minimum steps required to provision an IMEX domain to employ GSSAPI Kerberos for secure messaging. Installing, configuring and maintaining Kerberos and its intersection with other enterprise tools, such as Active Directory, is beyond the scope of this user guide and responsibility of the system administrator.

Cluster/Sysadmin Level#

To enable GSSAPI at the cluser and sysadmin level:

  1. Set up the KDC and add Service Principal Names (SPN) for all compute nodes.

  2. Supply Kerberos config (krb5.conf) with realm, domain, KDC details to all compute nodes.

  3. Prepare and supply Kerberos keytable (krb5.keytab) with respective SPN entries to all compute nodes.

Here is an example of the default keytab with only the host service principal, and the default IMEX config (blank):

$ sudo klist -kte
Keytab name: FILE:/etc/krb5.keytab
KVNO Timestamp           Principal
---- ------------------- ------------------------------------------------------
 1 01/01/2024 10:00:00 host/computenode-1@NVIDIA.COM (aes256-cts-hmac-sha1-96)
 
$ grep SERVICE_NAME /etc/nvidia-imex/config.cfg
IMEX_GSS_SERVICE_NAME=

Keytab with the nvidia-imex Service Principal and Corresponding IMEX Config#

Here is an example of the keytab with the nvidia-imex service principal and the corresponding IMEX config:

$ sudo klist -kte
Keytab name: FILE:/etc/krb5.keytab
KVNO Timestamp           Principal
---- ------------------- ------------------------------------------------------
 1 01/01/2024 10:00:00 nvidia-imex/computenode-1@NVIDIA.COM (aes256-cts-hmac-sha1-96)
 
$ grep SERVICE_NAME /etc/nvidia-imex/config.cfg
IMEX_GSS_SERVICE_NAME=nvidia-imex

Compute-Node Level#

To enable GSSAPI Kerberos at the compute-node level:

  1. Update the IMEX config file and enable the GSSAPI/Kerberos mechanism before starting nvidia-imex daemon.

  2. IMEX assumes that the default Client keytable location is /etc/krb5.keytab.

  3. IMEX gRPC server/client will create a channel, establish the Kerberos security_context, and start IMEX messaging.

Configuration#

When enabled, IMEX GSSAPI provides mutual authentication and message integrity services by default. Depending on the configured mode, the confidentiality service can be bypassed for unencrypted gRPC communication. Refer to the following sections for more information about providing the configs to enable IMEX authentication and/or encryption using GSSAPI:

  • “Enable/Disable Secure Connections” on page 21

  • “Control Authentication and Encryption Method” on page 22

The system administrator also needs to populate the following standard Kerberos environment variables for the keytab location.

export KRB5_KTNAME=/etc/krb5.keytab
export KRB5_CLIENT_KTNAME=/etc/krb5.keytab

Refer to the following sections for more information about IMEX-specific GSSAPI configuration parameters:

State Monitoring and Logging#

After the nvidia-imex daemon is launched with GSSAPI enabled, inter-node mutual authentication status can be monitored using the nvidia-imex-ctl tool. Relevant logs for GSSAPI security context establishment, message wrapping, and key rotation will be captured in the standard IMEX logging file.