This section of the user manual describes installation and operation of NVIDIA® Mellanox® RShim drivers.
The Rshim drivers will be installed only on Windows Server 201 2R2 and above Operating Systems.
Installing RShim Drivers
- Open the Device Manager when no drivers are installed to make sure a new PCIe device is available as below.
- Run the installer to install all 3 drivers (MlxRshimBus.sys, MlxRshimCom.sys and MlxRshimEth.sys).
- Make sure the Bus driver created 2 child devices after the installation (Com port and the Ethernet adapter).
At this time, PuTTY application or any other network utility can be used to communicate with DPU via Virtual Com Port or Virtual Ethernet Adapter (ssh). The Com Port can be used using the 9600 baud-rate and default settings.
RShim drivers can be connect via PCIe (the drivers we are providing) or via USB (external connection) but not both at the same time. So when the bus driver detects that an external USB is already attached, it will not create the child virtual devices for data access. Access via PCIe is available once the USB connection is removed.
For instructions on how to find interface index in registry <nn>, please refer to section Finding the Index Value of the Network Interface.
Update the MAC address manually using registry key if there are more than one BlueField DPU in the system.
RShim Bus Driver
This driver does all the read/write work to the hardware registers. User space application can send down IOCTL’s to restart the system on chip or to push a new BlueField boot stream image.
RShimCmd Tool
RShimCmd is a command line tool that enables the user to:
- Restart the DPU.
- Push a boot stream file (.bfb). A .bfb file is a generated BlueField boot stream file that contains Linux operating system image that runs on the DPU. BFB files can be downloaded from the NVIDIA DOCA SDK webpage.
Usage |
|
Example |
|
Detailed Usage |
|
The BFB image can be either CentOS or Ubuntu. Ubuntu credentials are: ubuntu/ubuntu and for Centos credentials are: root/centos, IP address of RShim Ethernet component (called tmfifo_net0) on the BlueField side is 192.168.100.2/30 by default. Please set IP address on the Windows side accordingly to be able to communicate via SSH.
EventLogs and Driver Logging
All driver logging is part of the Mellanox-WinOF2-Kernel trace session that comes with the network drivers installation. The default location to the trace is at %SystemRoot%\system32\LogFiles\Mlnx\Mellanox-WinOF2-System.etl.
The following are the Event logs RShim drivers generate:
RShim Bus Driver
| Event ID | Severity | Message |
|---|---|---|
2 | Informational | RShim Bus driver loaded successfully. |
3 | Informational | Device successfully stopped. |
4 | Error | The SmartNic adapter card seems to be stuck as the boot fifo data is not being drained. |
5 | Error | Driver startup failed due to failure in creation of the child device. |
6 | Error | Smartnic is in a bad state. Please restart smartnic and reload bus drivers. Please refer to user manual on how to restart smartnic. |
7 | Warning | Smartnic is in LiveFish mode. |
8 | Warning | Failed creating child virtual devices as a backend USB device is attached and accessing RShim FIFO. Please refer to user manual for more details. |
RShim Serial Driver
| Event ID | Severity | Message |
|---|---|---|
2 | Informational | RShim Serial driver loaded successfully. |
3 | Informational | device successfully stopped. |
RShim Ethernet Driver
| Event ID | Severity | Message |
|---|---|---|
2 | Error | MAC Address read from registry is not supported. Please set valid unicast address. |
3 | Informational | device is successfully stopped. |
4 | Warning | value read from registry is invalid. Therefore use the default value. |
5 | Error | SmartNic seems stuck as transmit packets are not being drained. |
6 | Informational | RShim Ethernet driver loaded successfully. |
DevX Interface
As DevX is not enabled by default to work in WinOF-2 driver, manual configuration is required as described below:
- Open Device manager and locate the Mellanox device.
- Right click and open the Properties.
- Go to the Details tab.
- Select the Driver key in the Property list.
- Save the value you received.
For example: “{4d36e972-e325-11ce-bfc1-08002be10318}\0003” - Open the registry editor (in console type regedit).
- Navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class
- Select the class as shown in the driver key you extracted in step 5.
For example: {4d36e972-e325-11ce-bfc1-08002be10318}. - Select the device number as in step 5.
For example: 0003. - Create a new key with name DevxEnabled of type DWORD and set the value ‘1’.
- Restart the driver. DevX Lib will be able to detect your device now.
- Verify DevX=True for the enabled adapter, run "cmd mlx5cmd -stat”
How to Integrate Windows DevX in Your Development Environment
- Find the MLNX_WinOF2_DevX_SDK_<version>.exe file in the WinOF-2 driver package located in the "DevX_SDK" folder.
Example: C:\Program Files\Mellanox\Mlnx_WinOF2\DevX_SDK - Install the SDK in your development system.
The SDK will expose the following new environment variables required for the library detection:
- variable name: DEVX_LIB_PATH will be the path to the DevX lib file
- variable name: DEVX_INC_PATH will be the path to the DevX header files - Make sure you update your session before start compiling.
- In case the library is installed, and the environment variable does not exist, you may export the environment variable manually to point to the SDK path.
