Appendix – Security Features

NVIDIA UFM Enterprise User Manual v6.11.1

Standard SA has a concept of trust-based requests on the SA_Key that is part of each SA MAD. A trusted request is when the SA_Key value is not equal to zero but equals the SA configured value, while an untrusted request is when the SA_Key value equals zero in the request. If a request has a non-zero SA_Key value that is different from the configured SA key, it will be dropped and reported.

When SAETM is enabled, the SA limits the set of untrusted requests allowed. Untrusted requests that are not allowed according to SAETM will be silently dropped (for the set of untrusted requests allowed, see the following section below).

SAETM feature is disabled by default. To enable it, set the sa_enhanced_trust_model parameter to TRUE.

Additional SAETM Configuration Parameters

Parameter

Description

sa_etm_allow_untrusted_guidinfo_rec

Defines whether to allow GUIDInfoRecord as part of the SAETM set of untrusted requests allowed (see section below)

sa_etm_allow_guidinfo_rec_by_vf

Defines whether to drop GUIDInfoRecord from non-physical ports (see section below)

sa_etm_allow_untrusted_proxy_requests

Defines the behavior for proxy requests (see section below)

sa_etm_max_num_mcgs/

sa_etm_max_num_srvcs/

sa_etm_max_num_event_subs

Defines the registration limits in SAETM (see section below)

Set of Untrusted SA Requests Allowed

The following table lists the untrusted requests allowed when SAETM is enabled:

Request

Request Type

MCMemberRecord

Get/Set/Delete

PathRecord

Get

PathRecord

GetTable (only if both destination and source are specified (e,g. only point to point))

ServiceRecord

Get/Set/Delete

ClassPortInfo

Get

InformInfo

Set (for non-SM security traps)

GUIDInfoRecord

Set/Delete – this request can only be part of this set depending on the values of sa_etm_allow_untrusted_guidinfo_rec and sa_etm_allow_guidinfo_rec_by_vf – see elaboration below.

When sa_etm_allow_untrusted_guidinfo_rec is set to FALSE (and SAETM is enabled), the SA will drop GUIDInfoRecord Set/Delete untrusted requests.

When sa_etm_allow_guidinfo_rec_by_vf is set to FALSE (and SAETM is enabled), the SA will drop GUIDInfoRecord Set/Delete requests from non-physical ports.

If sa_etm_allow_untrusted_guidinfo_rec=FALSE, GUIDInfoRecord Set/Delete requests will become part of the SAETM set of untrusted requests allowed. Note that if sa_etm_allow_guidinfo_rec_by_vf=FALSE, the requests will only be allowed from physical ports.

Proxy SA Requests

SA modification request (SET/DELETE) is identified as a proxy operation when the port corresponding with the requester source address (SLID from LRH/SGID from GRH) is diffident than the port for which the request applies:

  • For MCMemberRecord, when the MCMemberRecord.PortGID field does not match the requester address

  • For ServiceRecord, when the ServiceRecord.ServiceGID field does not match requester address

  • For the GUIDInfoRecord, when the LID field in the RID of the record does not match the requester address

When sa_etm_allow_untrusted_proxy_requests is set to FALSE and SAETM is enabled, untrusted proxy requests will be dropped.

Registration Limits

When any of sa_etm_max_num_mcgs, sa_etm_max_num_srvcs or sa_etm_max_num_event_subs parameters is set to 0, the number of this parameter’s registrations can be unlimited. When the parameter’s value is different than 0, attempting to exceed the maximum number of registrations will result in the request being silently dropped. Consequently, the requester and request info will be logged, and an event will be generated for the Activity Manager.

The following parameters control the maximum number of registrations:

Parameter

Description

sa_etm_max_num_mcgs

Maximum number of multicast groups per port/vport that can be registered.

sa_etm_max_num_srvcs

Maximum number of service records per port/vport that can be registered.

sa_etm_max_num_event_subs

Maximum number of event subscriptions (InformInfo) per port/vport that can be registered.

SAETM Logging

When requesting an operation that is not part of the SAETM set of untrusted requests, it will be silently dropped and eventually written to the SM log.

The logging of the dropped MADs is repressed to not overload the OpenSM log. If the request that needs to be dropped was received from the same requester many times consecutively, OpenSM logs it only if the request number is part of the following sequence:

0, 1, 2, 5, 10, 20, 50, 100, 200... (similar to the trap log repression).

SA can validate requester addresses by comparing the SLID and SGID of the incoming request. SA determines the requester port by the SLID and SGID field of the request. SGID spoofing is when the SGID and SLID do not match.

When sa_check_sgid_spoofing parameter is enabled, SA checks for SGID spoofing in every request that includes GRH, unless the SLID belongs to a router port in that same request. In case the request SGID does not match its SLID, the request will be dropped. The default value of this parameter is TRUE.

M_Key Authentication Enablement

In order to enable M_Key authentication in the InfiniBand fabric, the following parameters must be set in opensm.conf:

Argument

Value

Description

m_key

64-bit integer
Default: 0

The value must be set to some random number.

m_key_protection

0-2
Default: 0

  • 0 – weakest level of protection
    SubnGet(*) shall succeeds for any key in the MADHeader:M_Key and SubnGetResp(PortInfo) shall return the contents of the PortInfo:M_Key component.
    SubnSet(*) and SubnTrapRepress(*) shall fail if MADHeader:M_Key does not match the PortInfo:M_Key component in the port.

  • 1
    SubnGet(*) shall succeed for any key in the MADHeader:M_Key and SubnGetResp(PortInfo) shall return the contents of the PortInfo:M_Key component set to zero if MADHeader:M_Key does not match the PortInfo:M_Key component in the port.
    SubnSet(*) and SubnTrapRepress(*) shall fail if MADHeader: M_Key does not match the PortInfo:M_Key component in the port.

  • 2
    SubnGet(*), SubnSet(*), and SubnTrapRepress(*) shall fail if MADHeader:M_Key does not match the PortInfo:M_Key component in the port.

m_key_lease_period

0-65535
Default: 0

The lease period used for the M_Key on this subnet in seconds.
Recommended value is 60 seconds.

m_key_lookup

TRUE/FALSE
Default: FALSE

Must be enabled when M_key is non-zero

M_Key Per Port

This feature increases protection on the fabric as a unique M_Key is generated and set for each HCA, router, or switch port.

OpenSM calculates an M_Key per port by performing a hash function on the port GUID of the device and the M_Key configured in opensm.conf.

To enable M_Key per port, set the parameter below in addition to the parameters listed in the previous section:

Copy
Copied!
            

m_key_per_port TRUE

Once enabled, OpenSM forces the values of the following parameters:

Copy
Copied!
            

m_key 0x1 (unless configured to non-zero value) m_key_protection to 2 (unless configured to 3) m_key_lookup to TRUE


Subnet Manager Protection

To protect UFM subnet manager from a hostile SM that may be enabled in the fabric, the SM_Key parameter must be set to some random value in addition to the M_Key protection described before:

Copy
Copied!
            

sm_key <random_64b_integer>

Once a hostile SM is detected and queried by UFM SM, UFM SM compares the SM_Key provided by the hostile SM to the SM_Key configured in UFM opensm.conf.

As UFM SM_Key is a random 64-bit number, there is a high probability that the SM_Key provided by hostile SM will not match the UFM SM_Key.

As a result UFM SM, ignores hostile SMs and reports them in opensm.log and the syslog.

Example from opensm.log:

Copy
Copied!
            

ERR 2F18: Got SM <direct_path_to_the_hostile_SM_node> with sm_key <hostile_SM_KEY> that doesn't match our local sm_key. Ignoring SMInfo.

Example from syslog:

Copy
Copied!
            

Found remote SM <direct_path_to_the_remote_SM> with non-matching sm_key


© Copyright 2023, NVIDIA. Last updated on Sep 5, 2023.