iSCSI Extensions for RDMA (iSER)
iSCSI Extensions for RDMA (iSER) extends the iSCSI protocol to RDMA. It permits data to be transferred directly into and out of SCSI buffers without intermediate data copies.
iSER uses the RDMA protocol suite to supply higher bandwidth for block storage transfers (zero time copy behavior). To that fact, it eliminates the TCP/IP processing overhead while preserving the compatibility with iSCSI protocol.
There are three target implementation of ISER:
Linux SCSI target framework (tgt)
Linux-IO target (LIO)
Generic SCSI target subsystem for Linux (SCST)
Each one of those targets can work in TCP or iSER transport modes.
iSER also supports RoCE without any additional configuration required. To bond the RoCE interfaces, set the fail_over_mac option in the bonding driver (see "Bonding IPoIB").
RDMA/RoCE is located below the iSER block on the network stack. In order to run iSER, the RDMA layer should be configured and validated (over Ethernet or InfiniBand). For troubleshooting RDMA, please refer to "HowTo Enable, Verify and Troubleshoot RDMA" on Mellanox Community.
The iSER initiator is controlled through the iSCSI interface available from the iscsi-initiator-utils package.
To discover and log into iSCSI targets, as well as access and manage the open-iscsi database use the iscasiadm utility, a command-line tool.
To enable iSER as a transport protocol use "I iser" as a parameter of the iscasiadm command.
Example for discovering and connecting targets over iSER:
iscsiadm -m discovery -o
new-o old -t st -I iser -p <ip:port> -l
Note that the target implementation (e.g. LIO, SCST, TGT) does not affect he initiator process and configuration.
Setting the iSER target is out of scope of this manual. For guidelines of how to do so, please refer to the relevant target documentation (e.g. stgt, targetcli).
Targets settings such as timeouts and retries are set the same as any other iSCSI targets.
If targets are set to auto connect on boot, and targets are unreachable, it may take a long time to continue the boot process if timeouts and max retries are set too high.
For various configuration, troubleshooting and debugging examples, refer to Storage Solutions on Mellanox Community.