Counters

To query the packet counters, use stats command.

            

            
[dpu]# virtnet stats [-h] {[-p PF] [-v VF] | [-u VUID]} [-q QUEUE_ID]
        

The following command queries PF 0 and VQ 0 (i.e., RQ):

            

            
[dpu]# virtnet stats -p 0 -q 0
        

Output:

            

            
# virtnet stats -p 0 -q 0
{'pf': '0x0', 'queue_id': '0x0'}
{
  "device": {
    "pf_id": 0,
    "packet_counters": "Enabled",
    "queues-stats": [
      {
        "VQ Index": 0,
        "rx_64_or_less_octet_packets": 0,
        "rx_65_to_127_octet_packets": 259,
        "rx_128_to_255_octet_packets": 0,
        "rx_256_to_511_octet_packets": 0,
        "rx_512_to_1023_octet_packets": 0,
        "rx_1024_to_1522_octet_packets": 0,
        "rx_1523_to_2047_octet_packets": 0,
        "rx_2048_to_4095_octet_packets": 199,
        "rx_4096_to_8191_octet_packets": 0,
        "rx_8192_to_9022_octet_packets": 0,
        "received_desc": "4096",
        "completed_desc": "0",
        "bad_desc_errors": "0",
        "error_cqes": "0",
        "exceed_max_chain": "0",
        "invalid_buffer": "0",
        "batch_number": "64",
        "dma_q_used_number": "0",
        "handler_schd_number": "44",
        "aux_handler_schd_number": "43",
        "max_post_desc_number": "0",
        "total_bytes": "0",
        "err_handler_schd_num": "0",
        "rq_cq_max_count": "0",
        "rq_cq_period": "0",
        "rq_cq_period_mode": "1"
      }
    ]
  }
}
        

The output has two sections.

• The first section, wrapped by device, are device details along with the packet counter statics enable state.

  Entry	Type	Description
  device	String	Entries under this section is per device information
  pf_id	String	Physical function ID
  packet_counters	String	packet counters feature: enabled/disabled

• The second section, wrapped by queues-stats, are information for each receive VQ.

  Entry	Type	Description
  VQ Index	Number	The VQ index starts at 0 (the first RQ) and continues up to the last SQ
  rx_64_or_less_octet_packets	Number	The number of packets received with a size of 0 to 64 bytes. Relevant for BlueField-3 RQ when packet counter is enabled.
  rx_65_to_127_octet_packets	Number	The number of packets received with a size of 65 to 127 bytes. Relevant for BlueField-3 RQ when packet counter is enabled.
  rx_128_to_255_octet_packets	Number	The number of packets received with a size of 128 to 255 bytes. Relevant for BlueField-3 RQ when packet counter is enabled.
  rx_256_to_511_octet_packets	Number	The number of packets received with a size of 256 to 511 bytes. Relevant for BlueField-3 RQ when packet counter is enabled.
  rx_512_to_1023_octet_packets	Number	The number of packets received with a size of 512 to 1023 bytes. Relevant for BlueField-3 RQ when packet counter is enabled.
  rx_1024_to_1522_octet_packets	Number	The number of packets received with a size of 1024 to 1522 bytes. Relevant for BlueField-3 RQ when packet counter is enabled.
  rx_1523_to_2047_octet_packets	Number	The number of packets received with a size of 1523 to 2047 bytes. Relevant for BlueField-3 RQ when packet counter is enabled.
  rx_2048_to_4095_octet_packets	Number	The number of packets received with a size of 2048 to 4095 bytes. Relevant for BlueField-3 RQ when packet counter is enabled.
  rx_4096_to_8191_octet_packets	Number	The number of packets received with a size of 4096 to 8191 bytes. Relevant for BlueField-3 RQ when packet counter is enabled.
  rx_8192_to_9022_octet_packets	Number	The number of packets received with a size of 8192 to 9022 bytes. Relevant for BlueField-3 RQ when packet counter is enabled.
  received_desc	Number	Total number of received descriptors by the device on this VQ
  completed_desc	Number	Total number of completed descriptors by the device on this VQ
  bad_desc_errors	Number	Total number of bad descriptors received on this VQ
  error_cqes	Number	Total number of errors CQ entries on this VQ
  exceed_max_chain	Number	Total number of chained descriptors received that exceed the max allowed chain by the device
  invalid_buffer	Number	Total number of times device tried to read or write buffer that is not registered to the device
  batch_number	Number	The number of RX descriptors for the last received packet. Relevant for BlueField-3.
  dma_q_used_number	Number	The DMA q index used for this VQ. Relevant for BlueField-3.
  handler_schd_number	Number	Scheduler number for this VQ. Relevant for BlueField-3.
  aux_handler_schd_number	Number	Aux scheduler number for this VQ. Relevant for BlueField-3.
  max_post_desc_number	Number	Maximum number of posted descriptors on this VQ. Relevant for DPA.
  total_bytes	Number	Total number of bytes handled by this VQ. Relevant for BlueField-3.
  rq_cq_max_count	Number	Event generation moderation counter of the queue. Relevant for RQ.
  rq_cq_period	Number	Event generation moderation timer for the queue in 1 µ sec granularity. Relevant for RQ.
  rq_cq_period_mode	Number	Current period mode for RQ 0x0 – default_mode – use device best defaults 0x1 – upon_event – queue_period timer restarts upon event generation 0x2 – upon_cqe – queue_period timer restarts upon completion generation The second section wrapped by queues-stats IS information for each receive VQ.

VQ Statistics

To query Rx VQ statistics, use the corresponding VQ index. For example, If there are 3 queues configured then to query Rx, VQ uses queue 0, Tx VQ uses queue 1, and Ctrl VQ uses queue 2.

The following is the command to query PF 0, VF 0, and VQ 0 (i.e., Rx).

            

            
[dpu]# virtnet query -p 0 -v 0 -q 0
        

Output:

            

            
"enabled-queues-info": [
  {
    "index": "0",
    "size": "256",
    "msix_vector": "0x1",
    "enable": "1",
    "notify_offset": "0",
    "descriptor_address": "0xffffe000",
    "driver_address": "0xfffff000",
    "device_address": "0xfffff240",
    "received_desc": "256",
    "completed_desc": "19",
    "bad_desc_errors": "0",
    "error_cqes": "0",
    "exceed_max_chain": "0",
    "invalid_buffer": "0",
    "batch_number": "64",
    "dma_q_used_number": "0",
    "handler_schd_number": "4",
    "aux_handler_schd_number": "3",
    "max_post_desc_number": "0",
    "total_bytes": "6460",
    "rq_cq_max_count": "0",
    "rq_cq_period": "0",
    "rq_cq_period_mode": "1"
  }
        

The following are some of the important VQ counters:

When DPA is the data path provider, each RQ has its corresponding drop counter, which counts the number of packets dropped inside the DPA virtio RQs.

The drop counter only increments (initial value being 0), and its value gets reset to 0 when disabled.

RQ drop counter can be enabled and disabled as follows (using VF 0 on PF 0):

            

            
[dpu]# virtnet modify -p 0 -v 0 device -dc enable
[dpu]# virtnet modify -p 0 -v 0 device -dc disable
        

To query the drop counter value(s), run:

            

            
[dpu]# virtnet query -p 0 -v 0 | grep num_desc_drop_pkts
        

If there are more than one RQ for a device, the drop count is the sum of all RQ's value.

The packet counter feature helps the user query the byte-wise packet counters for each Rx queue.

By default, byte-wise packet counters are disabled as that negatively impacts performance. When the user is interested in the debug, enable the packet counter feature using the below command

Packet counter can be enabled and disabled as follows (using VF 0 on PF 0):

            

            
[dpu]# virtnet modify -p 0 -v 0 device -pkt_cnt enable
[dpu]# virtnet modify -p 0 -v 0 device -pkt_cnt disable
        

• When enabled, byte-wise packet counters are initialized to zero.

• When disabled, the previous values are retained for debugging purposes. The command will still return these old, disabled counter values.

The health statistics are for displaying real-time health information of a specific device.

Output example (using VF 0 on PF 0):

            

            
[dpu]# virtnet health -p 0 -v 0 show
{
  "pf_id": 0,
  "vf_id": 0,
  "type": "VF",
  "vuid": "MT2306XZ00BPVNETS0D0F2",
  "dev_status": {
    "value": "0xf",
    "    0": "ACK",
    "    1": "DRIVER",
    "    2": "DRIVER_OK",
    "    3": "FEATURES_OK"
  },
  "health_status": "Good",
  "health_recover_counter": 0,
  "dev_health_details": {
    "control_plane_errors": {
      "sf_rqt_update_err": 0,
      "sf_drop_create_err": 0,
      "sf_tir_create_err": 0,
      "steer_rx_domain_err": 0,
      "steer_rx_table_err": 0,
      "sf_flows_apply_err": 0,
      "aarfs_flow_init_err": 0,
      "vlan_flow_init_err": 0,
      "drop_cnt_config_err": 0
    },
    "data_plane_errors": {
      "sq_stall": 0,
      "dma_q_stall": 0,
      "spurious_db_invoke": 0,
      "aux_not_invoked": 0,
      "dma_q_errors": 0,
      "host_read_errors": 0
    }
  }
        

Where

• health_status represents the overall status of the device (Good or Fatal)

• dev_health_details has two sections, control_plane_errors and data_plane_errors, as explained in the following table:

  Counter Name	Description
  Control Plane Errors
  sf_rqt_update_err	Counter tallying receive queue table update failures
  sf_drop_create_err	Counter tallying drop RQ creation failures
  sf_tir_create_err	Counter tallying TIR create failures
  steer_rx_domain_err	Counter tallying RX steering rule creation failures
  steer_rx_table_err	Counter tallying RX table creation failures
  sf_flows_apply_err	Counter tallying packet flow rule creation failures
  aarfs_flow_init_err	Counter tallying packet flow initialization failures
  vlan_flow_init_err	Counter tallying VLAN flow rule initialization failures
  drop_cnt_config_err	Counter tallying drop counter configuration failures
  Data Plane Errors
  sq_stall	One or more network send queues stalled without getting completions. This leads traffic stalling for packets flowing over this VQ.
  dma_q_stall	QP which is paired to itself issues a read request from the DPA to the host to read either available index or descriptor table. This request does not result in a completion and hangs in a loop waiting for a response.
  spurious_db_invoke	Doorbell handler is repeatedly invoked but DPA finds no new data to be read and posted. This could be due to a faulty driver or issue on the DPA side.
  aux_not_invoked	To speed up descriptor processing, an auxiliary execution (EU) unit is used if available. The primary thread invokes this EU and waits for the expected thread to run on the auxiliary execution unit. If this EU is not invoked, the primary thread hangs.
  dma_q_errors	QP which is paired to itself issues a read request from the DPA to the host to read either an available index or the descriptor table. This request results in an error and the QP becomes unavailable. An internal mechanism detects this error QP and recycles it for use at later stage.