Cumulus Linux 5.14 Release Notes

Download 5.14 Release Notes xls    Download all 5.14 release notes as .xls

5.14.0 Release Notes

Open Issues in 5.14.0

Issue IDDescriptionAffectsFixed
4562200
On the Spectrum-1 switch, switchd might crash during warm boot.5.14.0
4557007
Incorrect prefix boundaries are set for traffic class, switch priority, and port groups for the qos/interfaces/interface[interface-id] metrics, which sends redundant data.5.14.0
4554858
The default poll interval for on-change notifications is set to 10 seconds instead of 1 second for gNMI packet trimming metrics.5.14.0
4551249
The NVUE service might fail during switch upgrade. To work around this issue, stop the sysmonitor with the sudo systemctl stop sysmonitor command, then upgrade the switch with the nv action upgrade system packages to latest command.5.14.0
4549896
When you try to set a VXLAN with a bridge, you see the error sx_sdk: 22985 [BRIDGE] [ERROR ]: Port(0x200A000D)) already added to a bridge. You can safely ignore this error as it doesn’t impact switch functionality.5.14.0
4548514
When a connected route for an SVI interface with VRR configured is installed by FRR, the route might be installed with the next hop interface of the VRR device instead of the SVI. For example, instead of interface vlan10, the route might install against vlan10-v0. This prevents next-hop tracking and route installation into hardware.
This issue can occur in the following conditions:
  • When initially configuring VRF route leaking, the target VRF might not install a route into hardware when leaking directly connected routes for SVI interfaces with VRR enabled.
  • In an MLAG and VRR configuration, static routes fail to install when the route is resolved through a connected route and the interface of the connected route undergoes a link state change for any reason, such as link flaps, interface bring-up, clagd restart, switchd restart, and so on.

To work around this issue, restart the FRR service using the sudo systemctl restart frr.service command.
5.9.1-5.14.0
4548512
When a connected route for an SVI interface with VRR configured is installed by FRR, the route might be installed with the next hop interface of the VRR device instead of the SVI. For example, instead of interface vlan10, the route might install against vlan10-v0. This prevents next-hop tracking and route installation into hardware.
This issue can occur in the following conditions:
  • When initially configuring VRF route leaking, the target VRF might not install a route into hardware when leaking directly connected routes for SVI interfaces with VRR enabled.
  • In an MLAG and VRR configuration, static routes fail to install when the route is resolved through a connected route and the interface of the connected route undergoes a link state change for any reason, such as link flaps, interface bring-up, clagd restart, switchd restart, etc.

To work around this issue, restart the FRR service using the sudo systemctl restart frr.service command.
5.11.0-5.14.0
4547463
When you try to run nv action boot-next commands during optimized image upgrade, the commands time out. This does not necessarily mean an issue occurred; the system might still be executing the action.
To check that the action command completed before going to the next step to reboot the system:
  1. Find the Request ID for the REST API invocation corresponding to the nv action boot-next command by doing a grep for ActionKey.*boot-next in the /var/log/nvued.log file. For example, the value 3 in the Ran Job running ActionKey('@boot-next', ‘/system/image’, (), 3, ((‘partition’, ‘partition1’),)) line indicates the Request ID.
  2. Run the curl -u ‘:<password>’ -X GET https://127.0.0.1:8765/nvue_v1/action/ -k command from the shell to show the status of the action command. If the value of state is action_success, the action command completed successfully. If the value of state is running, the system is still processing. If the value of state is action_error, the system encountered an error.
5.14.0
4546056
When an ESI changes from remote to local, the switch does not uninstall remote neighbors from the ASIC. To work around this issue, flap the layer 3 VNI associated with the SVI to clean up stale neighbors from the ASIC.5.14.0
4535749
The uc-discards field in the nv show interface counters qos egress-queue-stats command output is actually the number of packets discarded per queue, but it is wrongly interpreted as bytes. To work around this issue, convert the data shown in bytes to packets by multiplying by 1024 if the data is in KB, 1024x1024 if the data is in MB, and 1024x1024x1024 if the data is in GB.5.11.0-5.14.0
4535748
The uc-discards field in the nv show interface counters qos egress-queue-stats command output is actually the number of packets discarded per queue, but it is wrongly interpreted as bytes. To work around this issue, convert the data shown in bytes to packets by multiplying by 1024 if the data is in KB, 1024x1024 if the data is in MB, and 1024x1024x1024 if the data is in GB.5.9.2-5.14.0
4535699
When you configure the RADIUS authentication order with local first and radius second, the RADIUS user is authenticated as a default user name.5.11.3-5.14.0
4535696
When you configure the RADIUS authentication order with local first and radius second, the RADIUS user is authenticated as a default user name.5.12.1-5.14.0
4531960
The GNMI Subscription to xpath interfaces/interface[name=swp61s0]/state/counters/out-pkts with a high sample interval results in an initial response of zero but in subsequent updates, the value is correct. You do not see this issue when the sample interval is 1 second.5.14.0
4509255
In PTP two-step, the hardware incorrectly modifies the SYNC message correction field.5.12.0-5.14.0
4508830
Cumulus Linux allows you to add bond ports of mismatched speeds (such as 10G and 25G) to the same LACP bond without error and the bond reports UP.5.11.2-5.14.0
4507163
In some cases after a package upgrade with ISSU (warm boot mode), you see continuous errors in the syslog similar to the following:
NOTICE  CORE_ASYNC: Error at pre send callback status [Resource is in use]
5.11.2-5.14.0
4501510
When you configure the RADIUS authentication order with local first and radius second, the RADIUS user is authenticated as a default user name.5.12.1-5.14.0
4499106
A newly provisioned dynamic BGP neighbour with an MD5 password in a VRF does not come up until you delete, then re-add the password.5.9.2-5.14.0
4495383
NVUE configuration yaml file translation converts unset commands to set commands because the translation logic expects only set commands.5.13.0-5.14.0
4495231
If the hardware clock date is later than 2038-01-19 03:14:07 UTC, the image might fail to install due to a grub-install failure on the EFI filesystem, which is a VFAT filesystem. As a result, you see the grub prompt immediately after reboot. To work around this issue, reboot from the grub prompt to go into ONIE. From ONIE, use the date command to set a date before 2038-01-19, then run the hwclock –systohc command to add it to the hardware clock. You can then use onie-nos-install to install the image.5.9.0-5.14.0
4475401
External input such as Ctrl+\ might trigger core dumps on the serial console from /bin/login. This behavior is caused by external sources, such as console servers or automation tools, and does not reflect a fault in the operating system. As a potential result, the serial console might become unresponsive.5.14.0
4475111
When you try to convert a layer 3 port that is part of ECMP to a bond member, you might see a failure in the switchd logs. This issue does not have any functional impact.5.11.2-5.14.0
4440766
When you try to delete a trusted ca key, Cumulus Linux shows an incorrect error message. To remove a trusted ca key, you must unset the key ID, not the key literal.5.13.0-5.14.0
4423248
If you unset an interface static IP address when the interface IP gateway is configured, the nv config apply command fails with an ifreload.service error. To work around this issue, unset both the static IP address and gateway together.5.9.0-5.14.0
4423244
When you enable, then disable adaptive routing, the BGP neighbors might go down because of an unresolved MAC address. To resolve this issue configure another attribute on the interface.5.9.0-5.14.0
4423175
When you configure an API port with a TCP port already in use, the nginx server fails to restart.5.13.0-5.14.0
4413589
A MAC-only EVPN type-2 route is wrongly advertised with the layer 3 VNI label in the BGP NLRI (Network Layer Reachability Information). Although this does not have any functional impact, it is not the desired RFC behavior.5.13.0-5.14.0
4398031
On the NVIDIA SN5400 switch, if the PTP slave port is swp66 and is operating at 1Gbps, the PTP mean path delay value might be reported as a negative value.5.11.0-5.14.0
4343690
The Swagger UI for the Cumulus OpenAPI model is not rendering correctly (https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-512/api/index.html#). The interface shows screaming emojis and Could not render.5.11.0-5.14.0
4337278
In certain cases, statically configured VXLAN entries age out after a peerlink flap.5.12.0-5.14.0
4329931
In previous releases, Cumulus Linux incorrectly allowed SyncE and PPS In features to be enabled at the same time. Upgrading systems with both features configured using NVUE to 5.12.0 or higher results in a failure to apply the startup configuration as part of the first boot of the upgraded version. To work around the issue, unset one of the features before you upgrade.5.12.0-5.14.0
4309876
When you configure an invalid switch port (swp), NVUE adds the invalid configuration instead of rejecting it. The invalid interface in the configuration does not have any functional impact.5.12.0-5.14.0
4308857
When you use tls_crlcheck in the /etc/nslcd.conf file, the optional nslcd service fails due to a missing library.5.9.2-5.14.0
4294156
After you clear the RoCE counters with the nv action clear interface qos roce counters command, the buffer-max-usage counters in the nv show interface qos roce counters command output do not show updated (cleared) values.5.12.0-5.14.0
4277042
On the NVIDIA SN5600 switch, you see low power alarms immediately after a reboot. The alarms disappear after showing up initially. Certain modules typically show low power alarms on initialization. No action is needed.5.12.0-5.14.0
4249096
Binary upgrade from Cumulus Linux 4.3.1 to 5.12.0 and later is not supported. To work around this issue, perform a binary upgrade from Cumulus Linux 4.3.1 to 5.9.0, then perform a binary upgrade from Cumulus Linux 5.9.0 to 5.12.0 or later.5.12.0-5.14.0
4236419
On the Spectrum-3 switch, the PTP offset for 25GbE fluctuates within a range of plus or minus 50 nanoseconds beyond the expected values.5.12.0-5.14.0
4214678
Changes to open telemetry configuration or export states restarts the telemetry service and resets all health metrics.5.12.0-5.14.0
4177067
When performing a package upgrade, nslcd installation might open an interactive dialog to configure nslcd.conf .
To avoid this interactive dialog, set the DEBIAN_FRONTEND environment variable to noninteractive. For example:
cumulus@switch:~$ sudo apt-get update
cumulus@switch:~$ sudo DEBIAN_FRONTEND=noninteractive apt-get -y -o “Dpkg::Options::=–force-confdef” -o “Dpkg::Options::=–force-confold” upgrade
5.11.0-5.14.0
4154839
When you run certain SDK commands (such as sudo sx_api_port_counter_dump_all.py or sx_api_fdb_dump) first as the cumulus user, then with sudo, you see the following error:
PermissionError: Errno 13] Permission denied: ‘/tmp/python_err_log.txt’

To resolve this issue, run the rm -rf /tmp/python_err_log.txt command.
5.11.0-5.14.0
4142857
The switch drops PTP packets received with extra ethernet padding and you see syslog [ptp4l.ERR] messages.5.10.0-5.14.0
4139511
When you generate the cl-support file on the Spectrum-4 switch, the following messages appear on the serial console. You can safely ignore this issue.
[ 1903.595131] mlxsw_minimal 2-0048: Could not acquire lock
[ 1903.601089] mlxsw_minimal 2-0048: Reg cmd access failed (reg_id=900a(mtmp),type=query)
[ 1903.609961] hwmon hwmon28: Failed to query module temperature thresholds
[ 1903.846724] mlxsw_minimal 2-0048: Could not acquire lock
[ 1903.852689] mlxsw_minimal 2-0048: Reg cmd access failed (reg_id=900a(mtmp),type=query)
[ 1903.861573] hwmon hwmon28: Failed to query module temperature
5.11.0-5.14.0
4134447
The journal logs might include the error message ERR kernel: [ 7.453789] usb usb2-port2: connect-debounce failed. You can safely ignore this log message.5.11.0-5.14.0
4129757
If you include a comma in the BGP community list, extended community list, or large community list regex expression of a routing policy, you see error messages and FRR reload fails. Make sure the regex expression does not contain a comma.
For example, instead of ^65550:([0-9]{1,2}|[1-9][1-9]):.$, specify ^65550:([0-9]|[0-9][0-9]):.$ and instead of ^65550:([0-4]{1,2}|[7-9][8-9]):.$, specify ^65550:([0-4]|[0-4][0-4]|[7-9][8-9]):.$.
5.11.0-5.14.0
4124376
The SN3700C-S, SN5400, and SN5600 secure boot switch running Cumulus Linux 5.11.0 or later boots with shim 15.8 that adds entries to the SBAT revocations to prevent the switch from booting shim 15.7 or earlier (included in Cumulus Linux 5.10 and earlier).
To downgrade a secure boot switch from Cumulus Linux 5.11.0 or later, or to recover a downgraded switch that does not boot, follow the steps in Downgrade a Secure Boot Switch.
5.11.0-5.14.0
4118970
When running PTP, the performance for 100Gx2 and 400Gx8 can have a high offset in up to 1.5% of the sampling.5.11.0-5.14.0
4105127
Any sFlow configuration changes that require an hsflowd restart are operational only after an initial delay of 60 seconds.5.11.0-5.14.0
4100629
NVUE show command outputs show LLDP neighbor changes only after the LLDP update frequency multiplied by the hold time.5.11.0-5.14.0
4082210
When you change the CPU resource limit with the nv set service control rsyslog resource-limit cpu command, the rsyslog agent does not start. To work around this issue, increase the CPU resource limit, then restart the service manually.5.11.0-5.14.0
4077921
You cannot use package upgrade to upgrade from Cumulus Linux 5.9.2 to Cumulus Linux 5.10.1 or later. You must install the Cumulus Linux image instead.5.10.1-5.14.0
4049213
When there are routes that point to a single next hop, an ECMP entry is created in the SDK. After the single next hop from the route is removed, the ECMP entry might stay in the SDK until the time the next hop is present in the kernel.5.9.2-5.14.0
4047798
Packet distribution based on ECMP hash is not working for VXLAN on top of rack switches where it is encapped. To work around this issue, enable lag hashing with gtp-teid to distribute the VXLAN encapped packets across egress interfaces.5.10.0-5.14.0
4030380
When you roll back interface configuration to the default setting with the nv unset interface command, NVUE removes the complete entry for the interface from the /etc/network/interface file, and puts the interface in admin down. As a result, you cannot configure FEC on the interface at the lower layers.5.10.0-5.14.0
4005422
When you upgrade Cumulus Linux 5.9.1 to Cumulus Linux 5.10 or later with package upgrade, the NTP service stops. To restart the NTP service, enable, then restart the service in the VRF in which it was running with the systemctl enable ntpsec@ and systemctl restart ntpsec@ commands.5.10.0-5.14.0
3985682
On Spectrum-4 switches, multicast flows containing packets smaller than 512 bytes might not reach full line rate. Cumulus Linux supports 512 byte and larger multicast packets.5.10.0-5.14.0
3966312
When connecting the SN5xxx switch to third party test equipment (such as IXIA) using copper cables at 100GbE, 200GbE, 400GbE, or 800GbE, links do not come up.5.10.0-5.14.0
3948068
On the SN3700 and SN3700c switch, the nv show platform environment voltage command output shows a failed state for the PSU-n-12V-RAIL-OUT sensors. This is a known hardware limitation that cannot be corrected by the PSU vendor.5.10.0-5.14.0
3915878
If you configure remote syslog export through a non-default VRF, you might see repeated error logs from the rsyslog process while the switch is booting:
rsyslogd: create UDP socket bound to device failed: No such device [v8.2302.0]rsyslogd: No UDP socket could successfully be initialized, some functionality may be disabled.  [v8.2302.0]

The logs occur because the rsyslog service starts before the networking service creates the configured VRF for syslog export. There is no functional impact with this issue.
5.9.0-5.14.0
3879717
Running snmpwalk on the switch with the management IP address does not work. To work around this issue, use the localhost option (snmpwalk -v 2c -c public28 localhost 1.3.6.1.2.1.14) or create a control plane ACL whitelist rule.5.10.0-5.14.0
3878394
When ZTP runs a script that contains wget, ZTP fails and you see a message similar to the following:
ZTP: ZTP DHCP: Unexpected error: ‘ascii’ codec can’t decode byte 0xe2 in position 181: ordinal not in range(128)ZTP: Script returned failure 

To work around this issue, use the -q option with wget.
5.9.0-5.14.0
3877516
When you connect two NVIDIA switches and configure 400G speed in force mode, links don’t come up.
To work around this issue, make sure auto-negotiation is always on when connecting NVIDIA to NVIDIA in PAM4.
5.9.0-5.14.0
3875373
When you run the nv config apply empty command, NVUE removes the cumulus user. If you are logged in as the cumulus user when you run the nv config apply empty command, the command fails.5.9.0-5.14.0
3861745
On UEFI hardware (where the /sys/firmware/efi directory exists), using the update-grub program might generate a /boot/grub/grub.cfg that is incorrect for booting ONIE if the ONIE option is selected on the console while booting. To work around this issue, run mount LABEL=“EFI System” /boot/efi before using update-grub.5.9.0-5.14.0
3855796
When configuring a Unicast Master Table for clients, the server addresses must be reachable and the route to the destination must exist. The unicast table can have one directly-connected port for a client. This restriction is only for directly connected ports and doesn’t apply to Unicast Servers on other devices or switches.5.9.0-5.14.0
3847439
In rare cases on the Spectrum 1 switch, where a dual connected host transmits all traffic flows to only one switch in a connected MLAG pair, and the host changes behavior to hash all flows to the other MLAG switch, there might be traffic loss if the MAC FDB entry on the original switch ages out.5.9.0-5.14.0
3819945
When you connect an NVIDIA SN4410, SN4700, or SN5600 switch to any Spectrum 1, Spectrum-2, or Spectrum-3 peer switch (with four lanes) using a 4x breakout configuration and the default lanes per port setting, links do not come up. To work around this issue, provide the lanes per port configuration shown below:
cumulus@switch:~$ nv set interface  link breakout 4x lanes-per-port 1
5.9.0-5.14.0
3818545
The terminal monitoring software SecureCRT has a known issue when running on both Windows and Mac systems where it gets stuck when monitoring the serial port of the switch as Cumulus Linux boots up. When this occurs, the serial port stops as shown below and SecureCRT is unable to receive any more serial data from the switch (it is able to transmit).
Mounting dev-hugepages.mount - Huge Pages File System..
Mounting dev-mqueue.mount
5.9.0-5.14.0
3774274
When you manually configure the /etc/cumulus/datapath/qos/qos_features.conf file without applying the QoS configuration with NVUE, running the nv config apply empty command later does not clean up the QoS configuration. If the QoS configuration includes breakout ports, the nv config apply empty command fails due to a switchd reload trigger failure. To work around this issue, clean up the configuration manually in the /etc/cumulus/datapath/qos/qos_features.conf, then run the nv config apply empty command.5.8.0-5.14.0
3773177
When you try to upgrade a switch from Cumulus Linux with package upgrade, you might see errors for expired GPG keys that prevent you from upgrading. To work around this issue, install the new keys with the following commands, then upgrade the switch.
cumulus@switch:~$ wget https://download.nvidia.com/cumulus/apt.cumulusnetworks.com/repo/pool/cumulus/c/cumulus-archive-keyring/cumulus-archive-keyring_4-cl5.6.0u5_all.deb
cumulus@switch:~$ sudo apt install ./cumulus-archive-keyring_4-cl5.6.0u5_all.deb
cumulus@switch:~$ sudo apt update
cumulus@switch:~$ sudo apt upgrade
4.0.0-4.4.5, 5.0.0-5.14.0
3771168
When you perform an ISSU upgrade on a Spectrum 1 switch, the switchd service might crash.5.8.0-5.14.0
3677533
Due to resource constraints on the Spectrum 1 switch, staticd performance drops and takes longer to read static routes compared to the time BGP takes to complete a graceful restart and advertise routes and EOR to its helpers. As a result, static routes are advertised after the EOR is sent to graceful restart helpers, which delete the stale static routes and relearn them after receiving the EOR from the restarting node. Temporary traffic loss might occur.5.7.0-5.14.0
3655681
When you disable, then enable STP auto-edge on a port, the port might not transition to the operational edge even though the port does not receive bpdus. To work around this issue, configure the port as an admin-edge port.5.7.0-5.14.0
3637444
Applying an inbound control plane ACL on the eth0 management interface does not take effect. To work around this issue, apply the ACL on the mgmt interface; for example, nv set interface mgmt acl inbound control-plane.5.7.0-5.14.0
3591918
The nv action disconnect system aaa user command does not clear all open sessions. To work around this issue, run the command as many times as the number of sessions.5.6.0-5.14.0
3587393
If you use the NVIDIA SN5600 (Spectrum-4) switch with Ixia test equipment, you might experience delayed link up due to intermittent link flaps.
To work around this issue when using copper cables:
  • Use Ixia IxOS version 9.37 with HF002156, or version 9.39 or later.
  • Use the DAC cable on ports 10 through 50.
  • Use NVIDIA cables with the recommended firmware revision.
  • Configure AN on the Cumulus Linux switch and AN/LT on Ixia.

To work around this issue when using fiber cables:
  • Configure 800g/Force.
  • Use the recommended NVIDIA optical adapters.
5.6.0-5.14.0
3556762
On rare occasions, SPT switchover might not happen cleanly in PIM resulting in some dropped packets. If you use PIM-SM to replicate EVPN BUM traffic, you might see a brief drop of multicast traffic before recovering due to normal PIM-SM traffic timeout.5.5.0-5.14.0
3540510
400Gx8 ports only support traffic line rate with packets that are larger than 172 bytes.5.6.0-5.14.0
3538321
In rare cases, an STP topology change on PTP over a VLAN can result in the switch losing the slave state and one of the ports might remain in an uncalibrated state. To work around this issue, toggle the port that is in the uncalibrated state up and down so that one of the ports is selected as the slave.5.6.0-5.14.0
3497622
When you remove PSUs, then plug them back in, you might experience traffic loss and some ports might be in a down state.5.6.0-5.14.0
3472163
On a switch with the Spectrum-4 ASIC, packets that are smaller than 256 bytes are not included in multicast flows. Multicast flows support packets that are 256 bytes or larger.5.6.0-5.14.0
3444490
Migration from ONYX to Cumulus Linux is supported and tested with ONYX version 3.10.4302 GA only.5.6.0-5.14.0
3442569
When trying to access the NVUE API, user accounts authenticated with a newly-configured external service, such as TACACS, RADIUS, or LDAP, receive a 401 forbidden error. To work around this issue, after enabling a new authentication service, make sure to restart both nginx.service and nvued.service to begin authenticating users against the new authentication service.5.5.0-5.14.0
3430430
When you configure PTP on 50G ports, the offset correction might be higher, which can affect the time synchronization of the node. To work around this issue, configure PTP on 100G ports, or on 10G or 1G ports with PTP shaper enabled.5.5.0-5.14.0
3424967
sudo for TACACS+ users with privilege level 15 does not work when reaching the TACACS+ server through the default VRF. To work around this issue, specify the interface name that the default VRF uses in the vrf= setting of the /etc/tacplus_servers file or run the NVUE nv set system aaa tacacs vrf command. If you don’t run either command, a TACACS+ user with privilege level 15 can run vrf task exec default sudo … to execute the sudo command.5.0.0-5.14.0
3420056
The ADVA 5401 SFP module with hardware revision 5.01 does not come up at layer 1 when you use 10G QSA adaptors. To work around this issue, use 25G QSA adaptors.4.4.0-4.4.5, 5.0.0-5.14.0
3362113
If you restore an NVUE startup.yaml file or run the nv config patch command after an upgrade that includes breakout ports with QoS configuration, the NVUE configuration fails to apply. Subsequent attempts to run nv config apply fail with a message similar to Invalid config [rev_id: 11] qos config is not supported on the following invalid interface: swp1s0. Supported on swp and bond interface types. To work around this issue, run nv unset on the configured QoS settings, then apply the breakout port configuration before you configure QoS. Alternatively, you can remove the QoS configuration from the yaml file and patch it separately after applying the breakout configuration.5.4.0-5.14.0
3347538
When connecting NVIDIA-to-NVIDIA in PAM4, you must enable auto-negotiation.5.4.0-5.14.0
3341214
If you use the NVUE REST API to configure a local user with a hashed password, the user cannot log in and the /etc/nvue.d/startup.yaml file shows the password as plain text.5.4.0-5.14.0
3329518
When using TACACS+, if the /etc/nsswitch.conf file specifies passwd: files tacplus (files is listed before tacplus), a user that is present in both the local /etc/passwd file and the TACACS+ server cannot log into the switch. NVIDIA recommends that when using TACACS+, you list tacplus before files in /etc/nsswitch.conf. When using NVUE, ensure that tacacs has priority over local.5.4.0-5.14.0
3327477
If you use su to change to a user specified through TACACS+, the user becomes the local tacacs0 thru tacacs15 user instead of the named user to run sudo commands. As a result, the named user password might not match the local tacacs0 thru tacacs15 user password.3.7.0-3.7.16, 4.0.0-4.4.5, 5.0.0-5.14.0
3326659
If you have a large number of MAC addresses, they do not age out at the MAC ageing timeout value configured on the switch. It might take up to 30 seconds more for the MAC addresses to age out and be deleted from the hardware. To work around this issue, wait for the ageing timeout value plus 30 seconds.5.4.0-5.14.0
3253218
Auto-negotiation isn’t supported on Spectrum-2 and Spectrum-3 switches using the 1G SFP-T module; FORCE 1G is configured instead.5.4.0-5.14.0
3241567
When you apply switch configuration for the first time on a freshly booted switch and you run the nv config apply command after setting the hostname with nv set system hostname, you might see the error message Failed to start Hostname Service. To work around this issue, run the nv config apply command a second time.5.3.0-5.14.0
3226506
The l1-show eth0 command does not show port information and is not supported in this release.5.3.0-5.14.0
3225117
Occasionally, packet loss might occur on 25G ports when the link is raised without FEC.5.4.0-5.14.0
3172504
When you connect the NVIDIA SN4600C switch to a Spectrum 1 or Spectrum-3 switch with a 40GbE passive copper cable (Part Number: MC2210126-005) on edge ports 1-4 and 61-64, there is an Effective BER of 1E-12 in PHY.5.2.0-5.14.0
3147782
You cannot use NVUE to configure an SNMP view to include a subtree beginning with a period. For example:
cumulus@switch:~$ nv set service snmp-server viewname cumulusOnly included .1.3.6.1.4.1.40310Error: GET /nvue_v1/service/snmp-server/viewname/cumulusOnly/included?pointers=%5B%22%2Fparameters%22%2C+%22%2Fpatch%2FrequestBody%2Fcontent%2Fapplication~1json%2Fschema%22%2C+%22%2Fpatch%2Fparameters%22%2C+%22%2Fpatch%2Fresponses%2F200%2Flinks%22%5D responded with 404 NOT FOUND
To work around this issue, reference the OID without the preceding period ( . ) in the command.
5.3.0-5.14.0
3145869
On a Spectrum-3 switch, the PTP offset in 10GbE changes between plus or minus 27. The average offset is around 7.5.2.0-5.14.0
3135952
PAM4 split cables (such as 2x100G, 4x100G, and 4x50G) do not work with a forced speed setting (when auto-negotiation is off) as the default speed enabled is for NRZ mode (such as 100G_4X). To work around this issue, set the appropriate lanes for forced speed (with auto-negotation off) with the ethtool -s swpX speed <port_speed> autoneg off lanes <no_of_lanes> command. For example:
cumulus@switch:~$ sudo ethtool -s swp1 speed 100000 autoneg off lanes 2
5.2.0-5.14.0
3115242
When you configure two VNIs in the same VLAN, ifupdown2 shows a vlan added to two or more VXLANS warning, which is only issued after the VNI is already added to the bridge. This leaves the new VNI in the PVID even if there is already an existing VNI configured in that PVID.5.1.0-5.14.0
3103821
On the NVIDIA SN4700 switch, inserting and removing the PSU might cause loss of frames.5.2.0-5.14.0
3084476
After you disable traffic shaping in the /etc/cumulus/datapath/qos/qos_features.conf file, the default QOS traffic shaping configuration does not restore. To work around this issue, restart switchd.4.4.3, 5.0.0-5.14.04.4.4-4.4.5
3084027
Under a high load, you might see ingress drop counters increase. The drops are classified as HwIfInDiscards in ethtool and shown as ingress_general in hardware.4.3.0-4.4.5, 5.0.0-5.14.0
3071652
On rare occasions, after you reboot or restart switchd on a Spectrum 1 switch, any 25G connections with Direct Attach Copper (DAC) cables that connect from the switch to a non-NVIDIA device might flap continuously. To work around this issue, bring the affected link administratively down for a few seconds on the non-NVIDIA device, then bring the link back up.4.4.4-4.4.5, 5.1.0-5.14.0
3061656
When the CPU load is high during a warm boot, bonds with a slow LACP rate fail to forward layer 2 traffic for up to 60 seconds (depending on the duration of the CPU load) and static bonds fail to forward layer 2 traffic for up to 5 seconds.5.1.0-5.14.0
2972540
With RADIUS enabled for user shell authentication, there might be a delay in local user authentication for non cumulus user accounts.5.0.0-5.14.0
2964279
When a VNI flaps, an incorrect list of layer 2 VNIs are associated with a layer 3 VNI. The NCLU net show evpn vni detail command output shows duplicate layer 2 VNIs under a layer 3 VNI.3.7.15, 4.4.2-4.4.5, 5.0.0-5.14.03.7.16
2951110
The net show time ntp servers command does not show any output with the management VRF.3.7.15-3.7.16, 4.1.1-4.4.5, 5.0.0-5.14.0
2932083
CVE-2021-45944 CVE-2021-45949: Multiple security issues were discovered in Ghostscript, the GPL PostScript/PDF interpreter, which could result in denial of service and potentially the execution of arbitrary code if malformed document filesare processed
Vulnerable: <= 9.27~dfsg-2+deb10u4Fixed: 9.27~dfsg-2+deb10u5
5.0.0-5.14.04.4.3-4.4.5
2904450
When you run the ethtool -m or the l1-show command, the 400G interface optical values do not show.4.4.0-4.4.5, 5.0.0-5.14.0
2891255
CVE-2021-39925: Buffer overflow in the Bluetooth SDP dissector in Wireshark 3.4.0 to 3.4.9 and 3.2.0 to 3.2.17 allows denial of service via packet injection or crafted capture file.
Vulnerable: <= 2.6.20-0+deb10u1
Fixed: 2.6.20-0+deb10u2
4.0.0-4.4.1, 5.0.0-5.14.04.4.2-4.4.5
2890681
CVE-2021-42771: relative path traversal in Babel, a set of tools for internationalising Python applications, could result in the execution of arbitrary code
Vulnerable: 2.6.0+dfsg.1-1Fixed: 2.6.0+dfsg.1-1+deb10u1
4.0.0-4.4.1, 5.0.0-5.14.04.4.2-4.4.5
2885305
Certain Murata PSU attributes show intermittently in the sensors command output. To work around this issue, upgrade to the latest PSU firmware on Murata.5.0.0-5.14.0
2867042
When connecting the NVIDIA SN4600 switch to another NVIDIA Spectrum switch, you must use auto-negotiation mode (not force mode); otherwise the switch might use the wrong Tx configuration.5.0.0-5.14.0
2823307
Cumuls Linux does not support a bond with more than 64 ports. Any configuration with more than 64 ports in a bond changes all ports to down when you apply the configuration.5.0.0-5.14.0
2736108
When you change the VRRP advertisement interval on the master, the master advertisement interval field in the show vrrp command output does not show the updated value.4.4.0-4.4.5, 5.0.0-5.14.0
2684925
The NVUE nv show vrf default router bgp peer command produces a 404 not found error.4.4.0-4.4.5, 5.0.0-5.14.0
2671652
In VXLAN routing environments, you might experience sub-optimal route convergence delays (longer than five seconds) when a prefix transitions to a new ECMP next-hop group. This condition might occur when a VTEP loses ECMP routes through all uplink peerings, then installs the routes through a different path, such as an MLAG peerlink.4.4.0-4.4.5, 5.5.0-5.14.0

Fixed Issues in 5.14.0

Issue IDDescriptionAffects
4555938
When one of the FRR processes such as bgpd or zebra goes down, watchfrr tries to restart those processes and also restarts the routing telemetry service, which blocks watchfrr causing cascading failures on other processes.5.12.0-5.13.1
4546389
The nv config show -o commands command displays numbered BGP neighbors in IP address range format; however, the nv set commands fail for BGP numbered neighbors configured in IP address ranges.5.13.0-5.13.1
4544433
When you enable bandwidth gauge on an interface, the asic-monitor leaks memory steadily over time. The memory leak rate depends on how many interfaces have the bandwidth gauge enabled and the snapshot rate.5.13.1
4540986
The sudo program included with Cumulus Linux is vulnerable to a local privilege escalation attack CVE-2025-32463.5.9.2
4540985
If you have a transceiver module with a vendor name containing invalid characters and OTEL is enabled, the switch generates an interface_stats core file.5.13.1
4540976
When an SVI interface is brought up with routes installed against it, during subsequent interface updates that are applied to the interface during bringup such as MTU changes, interface forwarding state, MAC learning updates, or multicast forwarding state changes, the routes installed for the interface might be programmed as blackhole routes. This can occur during a fresh boot of a switch. To work around this issue, toggle the interface state of the SVI.5.8.0-5.13.1
4539084
When you lower the speed for an interface, the nv show interface rates command output might show the link utilization percentage above 100 percent. This issue is corrected automatically after the load interval duration.5.12.0-5.13.1
4531952
When a connected route for an SVI interface with VRR configured is installed by FRR, the route might be installed with the next hop interface of the VRR device instead of the SVI. For example, instead of interface vlan10, the route might install against vlan10-v0. This prevents next-hop tracking and route installation into hardware.
This issue can occur in the following conditions:
  • When initially configuring VRF route leaking, the target VRF might not install a route into hardware when leaking directly connected routes for SVI interfaces with VRR enabled.
  • In an MLAG and VRR configuration, static routes fail to install when the route is resolved through a connected route and the interface of the connected route undergoes a link state change for any reason, such as link flaps, interface bring-up, clagd restart, switchd restart, etc.

To work around this issue, restart the FRR service using the sudo systemctl restart frr.service command.
5.11.1-5.13.1
4531104
In certain cases, on a switch with SDK 4.6.1062 or earlier, you might see FW fatal health events.5.6.0-5.13.1
4526096
The global VRRP priority level change is not applied to the VRRP interface. It is always set to 100.5.13.1
4524763
Cumulus Linux does not provide a specific error message if you try to configure an ACL rule with conflicting match protocol criteria. For example, if you configure a single rule with both match ip protocol tcp and match ip udp dest-port , you see the following error:
Unable to run ‘install_acls.sh’ script:

RAN: sudo -S bash /var/lib/nvue/config/install_acls.sh

To work around this issue, configure consistent protocol match statements in each ACL rule.
5.13.1
4522594
In a VXLAN environment, if the bridge MAC address changes for a VTEP, the switch drops VXLAN traffic entering this VTEP from a remote VTEP. The switch only drops layer 3 routed VXLAN traffic because the RMAC is not updated correctly in the SDK.5.11.1-5.13.1
4521203
The SATA driver on the NVIDIA SN2410 switch sometimes exhibits failures similar to the following:
2025-06-20T20:03:38.785966+09:00 S1-RF10B kernel: 
[49878794.456350] ata1.00: failed command: WRITE FPDMA QUEUED
2025-06-20T20:03:38.785968+09:00 S1-RF10B kernel: [49878794.461984] ata1.00: cmd
61/08:08:50:2b:c4/00:00:01:00:00/40 tag 1 ncq dma 4096 out
2025-06-20T20:03:38.785970+09:00 S1-RF10B kernel: [49878794.461984] res 40/00:0c:50:2b:c4/00:00:01:00:00/40 Emask 0×10 (ATA bus error)

To work around this issue, power cycle the switch.
5.5.1-5.13.1
4518510
You cannot erase all data from the switch SSD on the NVIDIA SN5610 switch.
4517549
If you use TACACS on the default VRF while using multiple switch ports (ECMP) to reach the TACACS server, Cumulus Linux only allows you to configure a single port, even though there are multiple ports to reach the server. When configuring the single port, you see issues such as long delays to login, authentication failures, sudo command failures after login, username changes to an unknown tacacsNN user on the shell prompt.5.13.0-5.13.1
4516848
The nv show system health command output might show status-led amber and rasdaemon inactive due to a failed rasdaemon start even though the actual system status LED is green.5.13.0-5.13.1
4509053
When the default route is available through a floating static route (admin distance 254) and through an eBGP learned route where the next hop is the eBGP peer, the route that comes up first (the floating static) becomes the best path, even though the BGP learned route is presented as the best path after the neighbor comes up. To work around this issue, set the nv set vrf router bgp peer-group nexthop-connected-check off command in NVUE or configure neighbor disable-connected-check in vtysh.5.13.0-5.13.1
4505138
Transceiver OIR (removal and insertion) or link flaps might cause EMAD timeouts, eventually causing switchd to crash. You see logs similar to EMAD_TRANSACTION] [ERROR ]: ACCESS_REG TIMEOUT.5.6.0-5.13.1
4502199
The TFTP client drops packets due to default firewall rules.5.13.0-5.13.1
4501642
When you connect the QLOUIE cable with FW 52.181.1003 to an NVIDIA SN5600 switch, you might see link flaps.5.11.0-5.11.1
4500170
When you use the API to compare a revision between a revision ID and an applied revision, the output adds the encrypted values in the diffs incorrectly, indicating that there are diffs when there are no diffs. The NVUE nv config diff command shows the correct output.5.13.1
4499025
You see a high volume of NAT and NFCT errors flooding switchd logs. This issue has no functional impact.5.11.2-5.13.1
4493540
The CBS field in the nv show system control-plane policer command output might not show the configured value.5.13.0-5.13.1
4486200
If you enable dynamic NAT and try to install two identical dynamic NAT rules, switchd might crash.5.11.2-5.13.1
4482933
The age and last-update counters in the nv show bridge domain mac-table command output are reversed. The last-update counter shows age data and the age counter shows last-update data. Cumulus Linux uses the age timer to determine when to remove an old MAC entry.5.13.0-5.13.1
4482790
There is an inconsistency in the structure of certain gNMI streaming interface metrics.5.13.0-5.13.1
4478125
When using optimized image (two partition) upgrade to upgrade the switch to Cumulus Linux 5.13, the /etc/sudoers.d/prometheus file might be missing. To work around this issue, create a file with 0440 permissions and add the following content:
cumulus@switch:~$ sudo nano /etc/sudoers.d/prometheus
# commands needed for Prometheus Node Exporter to get LLDP metrics
prometheus ALL = NOPASSWD: /usr/sbin/lldpcli -f json show chassis
prometheus ALL = NOPASSWD: /usr/sbin/lldpcli -f json show neighbor
5.13.0-5.13.1
4477903
Cumulus Linux drops SRv6 traffic to the switch due to a missing hardware ECMP entry even though the SRv6 ACL is programmed correctly. To work around this issue, restart the switchd service.5.13.0-5.13.1
4475117
After you perform an optimized, two partition upgrade followed by a rollback to the previous release, certain services, including the NVUE service, might fail to start. If you see this issue after rollback, run the following commands on the switch to ensure that file ownership is correct:
sudo chown -R nvue /var/lib/nvue
sudo chown -R ntpsec /var/lib/ntpsec
sudo chown -R Debian-snmp /var/lib/snmp

To restart the services, either reboot the switch, or run the following commands:
sudo systemctl restart nvued.service
sudo systemctl restart nvue-startup.servicesystemctl is-enabled snmpd && sudo systemctl restart snmpd.servicesystemctl is-enabled ntpsec && sudo systemctl restart ntpsec.service
5.11.1
4475104
On the NVIDIA SN5610 switch, bringing up around 256 links takes a long time (approximately 90 seconds).5.13.0-5.13.1
4475095
The BGP service might crash when you have overlapping routes between two VRFs and you are leaking these routes between these VRFs in an EVPN environment. Avoid leaking overlapping IP addresses or subnets between two VRFs. This is typically accomplished by filtering the overlapping IP address or subnet routes with a prefix list and route map.5.13.1
4474924
The nv config show -o commands command displays numbered BGP neighbors in IP address range format; however, the nv set commands fail for BGP numbered neighbors configured in IP address ranges.5.13.0-5.13.1
4471932
ERSPAN is not generated with the intended VLAN tag but is always sent out untagged. If the interface is a trunk port or a subinterface, the peer might discard these packets.5.13.0-5.13.1
4469498
When a host moves to a new VTEP during mobility or network failover events in an EVPN multihoming environment, the host might be unreachable due to ARP resolution failures. To work around this issue, restart the FRR service with the sudo systemctl restart frr.service command.5.4.0-5.13.1
4469479
If you use NVUE to unset an extremely large IP prefix list (around 50K), the command might time out when unconfiguring FRR. As a result, you see the message Failure during apply. Ignore? [y/N] and the FRR service stops.5.13.0-5.13.1
4469349
When you try to import an invalid server certificate file, Cumulus Linux does not import the certificate file but fails to show an error message.5.13.0-5.13.1
4467245
Transceiver channel power values in dBm reported through streaming telemetry protocols such as OTLP and GNMI are imprecise and are rounded to whole numbers. The NVUE nv show platform transceiver command or Linux ethtool -m command provides more precise, floating-point values.5.13.0-5.13.1
4466525
Radius priority values can range between 1 and 8; however Cumulus Linux allows you to configure an invalid priority.5.12.0-5.13.1
4461102
In certain cases, when a port is down and you apply adaptive routing with the link utilization threshold setting to the port as it goes up, you might see log errors while the port is not yet up.5.11.0-5.13.1
4460588
The NVIDIA SN5610 switch might experience FEC burstiness with multiple optics, which can impact link stability and performance.5.13.0-5.13.1
4458873
The SN5610 switch records a High FEC Bin Error at room temperature.5.13.0-5.13.1
4457165
When a spine switch reboots and comes back online, leaf switches might cause prolonged convergence times due to premature traffic forwarding. The leaf switch immediately begins ECMP load-balancing traffic to the recovered spine before the spine has fully programmed its routing tables, resulting in dropped traffic during route convergence on the spine.5.13.0-5.13.1
4449947
The nv config patch and nv config replace commands have no effect with filenames with relative paths, such as ./config.yaml, ../config.yaml}, and so on.5.13.0-5.13.1
4447797
When you remove an FRU, such as a PSU, from the switch, stale statistical data from the component is still reported over GNMI and OTLP.5.13.0-5.13.1
4447794
The gNMI value for the sensor alarm-status shows incorrectly. For a raised alarm, the alarm-status is True(1) but normally functioning sensors also show True(1).5.13.0-5.13.1
4447661
During link down events affecting a subset of local links, switchd might observe extra route deletions due to timing mismatches between Zebra and kernel processing of Next Hop Group (NHG) changes. Routes should transition directly from one NHG to another, but instead get temporarily deleted and reinstalled, causing unnecessary churn. Zebra logs might show Extended Error: Nexthop id does not exist messages during partial link down scenarios
5.13.0-5.13.1
4447419
Users with nv show privileges only can still execute the nv config apply and related commands.5.9.2-5.11.1
4443870
When running tens of thousands of nv set commands, the /var/lib/nvue directory might grow to several GBs in size, potentially using all the disk space. To work around this issue, run the following commands to reduce the disk space in the /var/lib/nvue directory:
cumulus@switch:~$ sudo su
cumulus@switch:~$ cd /var/lib/nvue/config
cumulus@switch:~$ git gc
5.13.0-5.13.1
4438933
When gNMI subscription and configuration changes to an interface occur simultaneously, the interface statistics process can crash and leave a core file.5.13.0-5.13.1
4438061
Trusted CA key names and literals do not have a size limit, which can result in buffer overflow risks.5.13.0-5.13.1
4433969
Cumulus Linux generates multiple cl-support files when there are multiple errors exporting OTEL metrics.5.13.0-5.13.1
4427224
The nv show interface command output shows the operational status as down for a link flap error disabled state instead of the real protodown reason. To work around this issue, run the nv show interface status command, which shows if any of the interfaces are protodown with the protodown reason.5.12.0-5.13.1
4427085
cl-route-check fails if there are INCOMPLETE entries in the Kernel neighbor table.5.13.0-5.13.1
4425975
On rare occasions, when bridge or L2VNI interfaces are coming up or transitioning state, type 2 EVPN routes might not be properly installed. To work around this issue, flap the VNI interface.5.9.3-5.11.1
4425299
After upgrading from Cumulus Linux 5.12 to 5.13 on the NVIDIA SN5400 switch bonus port, PTP does not converge. To work around this issue, disable, then enable the bonus port after upgrade.5.13.0-5.13.1
4425288
When gNMI streaming is enabled on the switch, Cumulus Linux generates a cl-support file the first time you perform an FRR operation that reloads or restarts the FRR service (such as remove or add a neighbor or add or modify static routes).5.13.0-5.13.1
4414935
Interface counters retrieved from the kernel are not reset when switchd restarts. These kernel counters are inconsistent with other telemetry counters that do reset when switchd restarts.5.13.0-5.13.1
4413450
When displaying BGP neighbor advertised routes in JSON format with very large routing tables, the process consumes excessive memory resulting in out of memory crashes.5.0.0-5.11.1, 5.13.0-5.13.1
4408549
UMF processes do not include log rotation and the logs can grow very large causing operational failures when you generate cl-support files and run the nv config apply command.5.13.0-5.13.1
4408387
BGP crashes during EVPN route install due to incorrect memory access.5.11.0-5.13.1
4408161
On the NVIDIA SN2201 switch, the fan tray LED status update fails and you see the following syslog errors:
systemd-udevd116276: mlxreg:fan1:green: Process ‘/usr/bin/hw-management-chassis-events.sh fantray-led-event mlxreg:fan1:green 255’ failed with exit code 1.

To work around this issue, restart the hw-management service with the sudo systemctl restart hw-management command.
5.7.0-5.11.1
4393369
If you generate a cl-support file after configuring a breakout port, the syslog file includes errors due to a statistic collection failure for stale ports. This issue has no functional impact.5.13.0-5.13.1
4389433
On switches with ONIE 5.3.0012 (might be displayed as 2023.11-5.3.0012-115200), reinstalling Cumulus Linux with onie-install when SecureBoot is enabled fails during CMS verification of a found image.
To determine the ONIE version on the switch:
sudo mount LABEL=“ONIE-BOOT” /mnt
/mnt/onie/tools/bin/onie-version
If the ONIE version is earlier than 5.3.0012 (might be displayed as 2023.11-5.3.0012-115200), either disable SecureBoot during reinstallation, or if SecureBoot is enabled and ONIE finds the image on one of the disks (for example, /dev/sda5), then fails CMS verification, log into the ONIE shell as root and run the following commands:
onie-stop
mkdir /$devname
mount $devname /$devname
onie-nos-install /$devname/onie-installer
5.12.0-5.13.1
4386779
When you start gNMI subscription from a remote client, interface rates reset and values start from 0.5.13.0-5.13.1
4372795
With high SSH scale you might see LTTNG high memory usage errors causing an out of memory condition.5.13.0-5.13.1
4360826
On rare occasions, when you run the NVUE nv config apply command, switchd crashes, then restarts after the crash and resumes its normal flow of operation.5.13.0-5.13.1
4341806
The BGP uptime differs between vtysh and NVUE command output.5.12.0-5.13.1
3879809
What Just Happened (WJH) now works on Spectrum-4 switches.5.9.0-5.13.1
3875687
After changing the VXLAN local tunnel IP address, the new IP address is not reflected in the Local Vtep Ip field of the nv show evpn vni command output. To work around this issue, either run the ifup command or reboot the switch.5.9.0-5.13.1