NVIDIA DOCA DPU CLI

This guide provides quick access to a useful set of CLI commands and utilities on the NVIDIA® BlueField® DPU environment.

This guide provides a concise guide on useful commands for DOCA deployment and configuration.

The tables in this guide provide two categories of commands:

  • General commands for Linux/networking environment

  • DOCA/DPU-specific commands

Warning

For more information about these commands, such as usage instructions, flag options, arguments and so on, use the -h option after the command or use the manual (e.g., man lspci).

Command

Description

ifconfig

Used to configure kernel-resident network interfaces. It is used at boot time to set up interfaces as necessary. After that, it is usually only needed when debugging or when system tuning is needed.

If no arguments are given, ifconfig displays the status of the currently active interfaces. If a single interface argument is given, it displays the status of the given interface only. If a single -a argument is given, it displays the status of all interfaces, even those that are down. Otherwise, it configures an interface.

ethtool <devname>

Used to query and control network device driver and hardware settings, particularly for wired Ethernet devices.

<devname> is the name of the network device on which ethtool should operate.

lspci

Displays information about PCIe buses in the system and devices connected to them. By default, it shows a brief list of devices.

tcpdump

Dump traffic on a network. Usage: tcpdump -i <interface> where <interface> is any port interface (physical/SF rep/VF port rep).

ovs-vsctl

Utility for querying and configuring ovs-vswitchd. The ovs-vsctl program supports the model of a bridge implemented by Open vSwitch in which a single bridge supports ports on multiple VLANs.

mount 10.0.0.10:/vol/myshare/ myshare/

Used for mounting a work directory on the DPU.

scp

Secure copy (remote file copy program). Useful for copying files from BlueField to the host and vice versa.

iperf

Used for server-client connection. Useful to check if the network connection achieves the speed of the network card on the DPU (line rate).

Command

Description

ibdev2netdev

Displays available mlnx interfaces

mst

Used to start MST service, to stop it, and for other operations with NVIDIA devices like reset and enabling remote access

cat /etc/mlnx-release

Displays the full BlueField image (bfb) version

cat /etc/os-release

Displays the details of the underlying OS installed on BlueField

ibv_devinfo

Displays the current InfiniBand connected devices and relevant information. Useful for checking current firmware version.

ipmitool power cycle

Power cycle

echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages

DPDK setup. Allocates hugepages for DPDK environment abstraction layer (EAL).

mlxdevm tool

The mlxdevm tool is found under /opt/mellanox/iproute2/sbin/. With this tool it is possible to create an SF and set its state to active, configure a HW address and set it to trusted, deploy the created SF and print info about it.

/opt/mellanox/iproute2/sbin/mlxdevm port add pci/<pci_address> flavour pcisf pfnum <correspondig_physical_function_number> sfnum <unique_sf_number>

Creates an SF in the flavor of the given PF with the given unique SF number. Example:

Copy
Copied!
            

/opt/mellanox/iproute2/sbin/mlxdevm port add pci/0000:03`:00.0 flavour pcisf pfnum 0 sfnum 4

/opt/mellanox/iproute2/sbin/mlxdevm port show

Displays information about the available SFs

/opt/mellanox/iproute2/sbin/mlxdevm port function set pci/0000:03:00.0/<sf_index> hw_addr <HW_address> trust on state active

Configures SF capabilities such as setting the HW address, making it "trusted", and setting its state to active. <sf_index> the SF. To obtain this index, you may run mlxdevm port show. Example:

Copy
Copied!
            

/opt/mellanox/iproute2/sbin/mlxdevm port function set pci/0000:03:00.0/229377 hw_addr 02:25:f2:8d:a2:4c trust on state active

$ echo mlx5_core.sf.<next_serial> > /sys/bus/auxiliary/drivers/mlx5_core.sf_cfg/unbind

$ echo mlx5_core.sf. <next_serial> > /sys/bus/auxiliary/drivers/mlx5_core.sf/bind

These two commands deploy the created SF. The first command unbinds the SF from the default driver, while the second command binds the SF to the actual driver. The deployment phase should be done after the capabilities of the SF are configured. The SF is identified by <next_serial> which can be obtained by running the command below.

ls /sys/bus/auxiliary/devices/mlx5_core.sf.*

Displays additional information about the created SFs and their "next serial numbers".

For example, if mlx5_core.sf.2 exists in the output of the command, then running cat /sys/bus/auxiliary/devices/mlx5_core.sf.2/sfnum would output the sfnum related to mlx5_core.sf.2.

/opt/mellanox/iproute2/sbin/mlxdevm port function set pci/<pci_address>/<sf_index> state inactive

/opt/mellanox/iproute2/sbin/mlxdevm port del pci/<pci_address>/<sf_index>

These two commands must be executed to delete a given SF. First, users must set the state of the SF to inactive, and only then should it be deleted.

/opt/mellanox/iproute2/sbin/mlxdevm port help

Displays additional information about operations that can be used on created SF ports

crictl pods

Displays currently active K8S pods, and their IDs (it might take up to 20-30 seconds for the pod to start)

crictl ps

Displays currently active containers and their IDs

crictl ps -a

Displays all containers, including containers that recently finished their execution

crictl logs <container-id>

Examines the logs of a given container

crictl exec -it <container-id> /bin/bash

Attaches a shell to a running container

journalctl -u kubelet

Examines the Kubelet logs. Useful when a pod/container fails to spawn.

crictl stopp <pod-id>

Stops a running K8S pod

crictl stop <container-id>

Stops a running container

crictl rmi <image-id>

Removes a container image from the local K8S registry

© Copyright 2023, NVIDIA. Last updated on Feb 9, 2024.