NVIDIA Docs Hub NVIDIA Networking BlueField DPUs / SuperNICs & DOCA NVIDIA BlueField Virtio-net v24.10 Configuration File

Download PDF

Configuration File

Configuration File Options

The controller service has an optional JSON format configuration file which allows users to customize several parameters. The configuration file should be defined on the DPU at /opt/mellanox/mlnx_virtnet/virtnet.conf. This file is read every time the controller starts.

Note

Controller systemd service should be restarted when there is configuration file change. Dynamic change of virtnet.conf is not supported.

Parameter

Default Value

Type

Description

ib_dev_p0

mlx5_0

String

RDMA device (e.g., mlx5_0) used to create SF on port 0. This port is the EMU manager when is_lag is 0.

ib_dev_p1

mlx5_1

String

RDMA device (e.g., mlx5_1) used to create SF on port 1

ib_dev_for_static_pf

mlx5_0

String

The RDMA device (e.g., mlx5_0) which the static virtio PF is created on

ib_dev_lag

Null

String

RDMA LAG device (e.g., mlx5_bond_0) used to create SF on LAG. Default value is mlx5_bond_0. This port is EMU manager when is_lag is 1. ib_dev_lag and ib_dev_p0/ib_dev_p1 cannot be configured simultaneously.

static_pf

N/A

List

The following sub-parameters can be used to configure the static PF:

Sub-parameter

Default Value

Type

Description

mac_base

Null

String

Base MAC address for static PFs. MACs are automatically assigned with the following pattern: mac_base→pf_0, mac_base + 1→pf_1, etc.

Note

Controller does not validate the MAC address (other than its length). The user must ensure the MAC is valid and unique.

features

Auto

Number

Virtio spec-defined feature bits for static PFs.

Note

If unsure, leave features out of the JSON file and a default value is automatically assigned. The default value is determined dynamically when controller starts. Refer to the "Feature Bits" page for more information.

is_lag

0

Number

Specifies whether LAG is used

Note

If LAG is used, make sure to use the correct IB dev for static PF

single_port

0

Number

Specifies whether the DPU is a single port device. It is mutually exclusive with is_lag.

recovery

1

Number

Specifies whether recovery is enabled. If unspecified, recovery is enabled by default. To disable it, set recovery to 0. Refer to the "Recovery" page for the items which are recovered and more information.

sf_pool_percent

0

Number

Determines the initial SF pool size as the percentage of PF_TOTAL_SF of mlxconfig. Valid range: [0, 100]. For instance, if the value is 5, an SF pool with 5% of PF_TOTAL_SF is created. 0 indicates that no SF pool is reserved beforehand (default).

Note

PF_TOTAL_SF is shared by all applications. The user must ensure that the percent request is guaranteed, or else the controller would not be able to reserve the requested SFs resulting in failure.

sf_pool_force_destroy

0

Number

Specifies whether to destroy the SF pool. When set to 1, the controller destroys the SF pool when stopped/restarted (and the SF pool is recreated if sf_pool_percent is not 0 when starting). Otherwise, it does not. Default value is 0.

packed_vq

0

Number

Specifies whether packed VQ mode is enabled. If unspecified, packed VQ is disabled by default. To enable, set packed_vq to 1. For VQ types, refer to the "Virt Queue Types" page.

Note

The virtio driver on the guest OS must be unloaded when restarting the controller if the packed_vq setting is enabled for the first time or is modified.

mrg_rxbuf

0

Number

When enabled, the mergeable buffers feature is negotiated with the host driver. This feature allows the guest driver to use multiple RX descriptor chains to receive a single receive packet, hence increase bandwidth.

Note

The virtio driver on the guest OS must be unloaded when restarting the controller if the mrg_rxbuf setting is enabled for the first time or is modified.

dpa_core_start

0

Number

Specifies the start DPA core for virtnet application. Valid only for NVIDIA® BlueField®-3 and up. Value must be greater than 0 and less than 11. Together with dpa_core_end, dpa_core_start defines how many DPA cores are used for the virtio-net data plane.

Note

This is advanced options when there are multiple DPA applications running at the same time. Regular user should keep this option as default.

dpa_core_end

10

Number

Specifies the end DPA core for virtnet application. Valid only for BlueField-3 and up. Value must be greater than dpa_core_start and less than 11.

vf

N/A

List

The following sub-parameters can be used to configure the VF:

Sub-parameter	Default Value	Type	Description
`mac_base`	Null	String	Base MAC address for VFs. MACs are automatically assigned with the following pattern: `mac_base`→`vf_0`, `mac_base + 1`→`vf_1`, etc. Note Controller does not validate the MAC address (other than its length). The user must ensure the MAC is valid and unique.
`features`	Auto	Number	Virtio spec-defined feature bits for VFs. Note If unsure, leave `features` out of the JSON file and a default value is automatically assigned. The default value is determined dynamically when controller starts. Refer to the "Feature Bits" page for more information.
`vfs_per_pf`	`0`	Number	The number of VFs to create on each PF . For example: if `vfs_per_pf` is 100, then `vf_0` on `pf_1` will use `mac_base + 100` as its MAC. Note `vfs_per_pf` ≤ `VIRTIO_NET_EMULATION_NUM_VF` in mlxconfig. Note User is responsible for ensuring, on each static PF, that the created VFs ≤ `vfs_per_pf.` Note This parameter is mandatory if `mac_base` is specified.
`max_queue_pairs`	Auto	Number	Number of queue pairs to use. If not specified, default queue pair number is inherited from the parent PF.
`max_queue_size`	Auto	Number	Virtqueue size (i.e., vq depth) to use. If not specified, default vq size is inherited from the parent PF.
`mtu`	`1500`	Number	Maximum transmission unit for the VF, can be ≤ 9216.

Configuration File Examples

Note

Validate the JSON format of the configuration file before restarting the controller, especially the syntax and symbols. Otherwise, the controller may fail to start.

Configuring LAG on Dual Port BlueField

Refer to the "Link Aggregation" page in NVIDIA BlueField BSP documentation for information on configuring BlueField in LAG mode.

Refer to the "Link Aggregation" page for information on configuring virtio-net in LAG mode.

Configuring Static PF on Dual Port BlueField

The following configures all static PFs to use mlx5_0 (port 0) as the data path device in a non-LAG configuration, and the default MAC and features for the PF:

Copy
Copied!

            
            {
  "ib_dev_p0": "mlx5_0",
  "ib_dev_p1": "mlx5_1",
  "ib_dev_for_static_pf": "mlx5_0",
  "is_lag": 0,
  "static_pf": {
    "mac_base": "08:11:22:33:44:55",
    "features": "0x230047082b"
  }
}

Configuring VF Specific Options

The following configures VFs with default parameters. With this configuration, each PF assigns the MAC based on mac_base up to 126 VFs. Each VF creates 4 queue pairs, with each queue having a depth of 256.

Note

If vfs_per_pf is less than the VIRTIO_NET_EMULATION_NUM_VF in mlxconfig, and more VFs are created, duplicated MACs would be assigned to different VFs.

Copy
Copied!

            
            {
  "vf": {
    "mac_base": "06:11:22:33:44:55",
    "features": "0x230047082b",
    "vfs_per_pf": 126,
    "max_queue_pairs": 4,
    "max_queue_size": 256
  }
}

On This Page

Configuration File

Configuration File Options

Configuration File Examples

Configuring LAG on Dual Port BlueField

Configuring Static PF on Dual Port BlueField

Configuring VF Specific Options