Access Layer

Low level operating system infrastructure (plumbing) used for accessing the interconnect fabric (VPI™, InfiniBand®, Ethernet, FCoE). It includes all basic transport services needed to support upper level network protocols,
middleware, and management agents.

AH (Address Handle)

An object which describes the path to the remote side used in UD QP

CA (Channel Adapter)

A device which terminates an InfiniBand link, and executes transport level functions

CI (Channel Interface)

Presentation of the channel to the Verbs Consumer as implemented through the combination of the network adapter, associated firmware, and device driver software

CM (Communication Manager)

An entity responsible to establish, maintain, and release communication for RC and UC QP service types

The Service ID Resolution Protocol enables users of UD service to locate QPs supporting their desired service.

There is a CM in every IB port of the end nodes.

Compare & Swap

Instructs the remote QP to read a 64-bit value, compare it with the compare data provided, and if equal, replace it with the swap data, provided in the QP.

CQ (Completion Queue)

A queue (FIFO) which contains CQEs

CQE (Completion Queue Entry)

An entry in the CQ that describes the information about the completed WR (status size etc.)

DMA (Direct Memory Access)

Allowing Hardware to move data blocks directly to and from the memory, bypassing the CPU

Fetch & Add

Instructs the remote QP to read a 64-bit value and replace it with the sum of the 64-bit value and the added data value, provided in the QP.

GUID (Globally Unique IDentifier)

A 64 bit number that uniquely identifies a device or component in a subnet

GID (Global IDentifier)

A 128-bit identifier used to identify a Port on a network adapter, a port on a Router, or a Multicast Group.
A GID is a valid 128-bit IPv6 address (per RFC 2373) with additional properties / restrictions defined within IBA to facilitate efficient discovery,
communication, and routing.

GRH (Global Routing Header)

A packet header used to deliver packets across a subnet boundary and also used to deliver Multicast messages
This Packet header is based on IPv6 protocol.

Network Adapter

A hardware device that allows for communication between computers in a network.


A computer platform executing an Operating System which may control one or more network adapters



Join operation

An IB port must explicitly join a multicast group by sending a request to the SA to receive multicast packets.


A number that is received upon registration of MR is used locally by the WR to identify the memory region and its associated permissions.

LID (Local IDentifier)

A 16 bit address assigned to end nodes by the subnet manager.
Each LID is unique within its subnet.

LLE (Low Latency

RDMA service over CEE (Converged Enhanced Ethernet) allowing IB transport over Ethernet.

NA (Network Adapter)

A device which terminates a link and executes transport level functions.

MGID (Multicast Group ID)

IB multicast groups, identified by MGIDs, are managed by the SM. The SM associates a MLID with each MGID and explicitly programs the IB switches in the fabric to ensure that the packets are received by all the ports that joined the multicast group.

MR (Memory Region)

A contiguous set of memory buffers which have already been registered with access permissions. These buffers need to be registered for the network adapter to make use of them. During registration an L_Key and R_Key are created and associated with the created memory region

MTU (Maximum
Transfer Unit)

The maximum size of a packet payload (not including headers) that can be sent /received from a port

MW (Memory Window)

An allocated resource that enables remote access after being bound to a specified area within an existing Memory Registration. Each Memory Window has an associated Window Handle, set of access privileges, and current R_Key.

Outstanding Work Request

WR which was posted to a work queue and its completion was not polled

pkey (Partition key)

The pkey identifies a partition that the port belongs to. A pkey is roughly analogous to a VLAN ID in ethernet networking. It is used to point to an entry within the port’s partition key (pkey) table. Each port is assigned at least one pkey by the subnet manager (SM).

PD (Protection Domain)

Object whose components can interact with only each other.
AHs interact with QPs, and MRs interact with WQs.

QP (Queue Pair)

The pair (send queue and receive queue) of independent WQs packed together in one object for the purpose of transferring data between nodes of a network.
Posts are used to initiate the sending or receiving of data.
There are three types of QP: UD Unreliable Datagram, Unreliable Connection, and Reliable Connection.

RC (Reliable Connection)

A QP Transport service type based on a connection oriented protocol.
A QP (Queue pair) is associated with another single QP. The messages are sent in a reliable way (in terms of the correctness and order of the information.)

RDMA (Remote Direct Memory Access)

Accessing memory in a remote side without involvement of the remote CPU

RDMA_CM (Remote Direct Memory Access Communication

API used to setup reliable, connected and unreliable datagram data transfers. It provides an RDMA transport neutral interface for establishing connections. The API is based on sockets, but adapted for queue pair (QP) based semantics: communication must be over a specific RDMA device, and data transfers are message based.


The side of the connection that will initiate a data transfer (by posting a send request)


The side of the connection that will respond to commands from the requestor which may include a request to write to the responder memory or read from the responder memory and finally a command requesting the responder to receive a message.


A number that is received upon registration of MR is used to enforce permissions on incoming RDMA operations

RNR (Receiver Not Ready)

The flow in an RC QP where there is a connection between the sides but a RR is not present in the Receive side

RQ (Receive Queue)

A Work Queue which holds RRs posted by the user

RR (Receive Request)

A WR which was posted to an RQ which describes where incoming data using a send opcode is going to be written. Also note that a RDMA Write with immediate will consume a RR.

RTR (Ready To Receive)

A QP state in which an RR can be posted and be processed

RTS (Ready To Send)

A QP state in which an SR can be posted and be processed

SA (Subnet

The interface for querying and manipulating subnet management data

SGE (Scatter /Gather

An entry to a pointer to a full or a part of a local registered memory block.
The element hold the start address of the block, size, and lkey (with its associated permissions).

S/G Array

An array of S/G elements which exists in a WR that according to the used opcode either collects data from multiple buffers and sends them as a single stream or takes a single stream and breaks it down to numerous buffers

SM (Subnet Manager)

An entity that configures and manages the subnet

Discovers the network topology

Assign LIDs

Determines the routing schemes and sets the routing tables

One master SM and possible several slaves (Standby mode)

Administers switch routing tables thereby establishing paths through the fabric

SQ (Send Queue)

A Work Queue which holds SRs posted by the user

SR (Send Request)

A WR which was posted to an SQ which describes how much data is going to be transferred, its direction, and the way (the opcode will specify the transfer)

SRQ (Shared Receive Queue)

A queue which holds WQEs for incoming messages from any RC/UC/UD QP which is associated with it.
More than one QPs can be associated with one SRQ.

TCA (Target Channel Adapter)

A Channel Adapter that is not required to support verbs, usually used in I/O devices

UC (Unreliable

A QP transport service type based on a connection oriented protocol,
where a QP (Queue pair) is associated with another single QP. The QPs do not execute a reliable Protocol and messages can be lost.

UD (Unreliable

A QP transport service type in which messages can be one packet length and every UD QP can send/receive messages from another UD QP in the subnet
Messages can be lost and the order is not guaranteed. UD QP is the only type which supports multicast messages. The message size of a UD packet is limited to the Path MTU


An abstract description of the functionality of a network adapter.
Using the verbs, any application can create / manage objects that are needed in order to use RDMA for data transfer.

VPI (Virtual Protocol Interface)

Allows the user to change the layer 2 protocol of the port.

WQ (Work Queue)

One of Send Queue or Receive Queue.

WQE (Work Queue

A WQE, pronounced “wookie”, is an element in a work queue.

WR (Work Request)

A request which was posted by a user to a work queue.

© Copyright 2023, NVIDIA. Last updated on May 23, 2023.