NVIDIA MLNX_OFED Documentation v4.9-7.1.0.0 LTS
Linux Kernel Upstream Release Notes v5.17

RX Page Cache Size Limit

The RX page cache size changes dynamically (extends/reduces) in response to the RX load. The RX page cache tables of different RQs are independent and the sizes of these tables change independently per RQ.

By default, the RX page cache size can extend up to 16 times the original size of the wq, at most. When the RX default limit is too high, the table may extend too much causing iommu allocation (iommu_alloc) problems.

To prevent this, the RX page cache size limit can be set to a lower value using sysfs.

The unit of this value is log of multiplies for the basic cache size. In other words, when the value is set to 4, the RX cache table can extend up to 16 times its original size and when the value is 2, the table can extend up to 4 times its original size.

Because the size of tables changes fast, changing the RX page cache size limit has a quick impact, e ven if the table size is larger than the new limit.

In cases of iommu_alloc problem, try and reducing the limit of the RX page cache size because, in some setups, the reduction of RX page cache size limit does not impair performance.

Originally, the log limit of the RX page cache size was always 4, therefore 4 is the default value set by the feature. The value has to be set to an integer between 0 and 4 included, where 0 means that the table will not be able to extend at all.

Below is an example of how to check the current log limit of the RX page cache size (example):

Copy
Copied!
            

cat /sys/class/net/<ifs-name>/rx_page_cache/log_mult_limit   screen: log rx page cache mult limit is 4


Below is an example of setting a new log limit to the RX page cache size where < val> is the log of multiplies for the basic cache size.

Copy
Copied!
            

echo < val> > /sys/class/net/eth3/rx_page_cache/log_mult_limit

© Copyright 2023, NVIDIA. Last updated on Oct 11, 2023.