If you are using the current version of Cumulus Linux, the content on this page may not be up to date. The current version of the documentation is available here. If you are redirected to the main page of the user guide, then this page may have been renamed; please search for it there.

Cumulus Linux 3.6 Release Notes

These release notes support Cumulus Linux 3.6.0, 3.6.1, and 3.6.2 and describe currently available features and known issues.

What’s New in Cumulus Linux 3.6.2

Cumulus Linux 3.6.2 contains the following new features, platforms, and improvements:

What’s New in Cumulus Linux 3.6.2

Cumulus Linux 3.6.1 contains bug fixes and security fixes.

What’s New in Cumulus Linux 3.6.0

Cumulus Linux 3.6.0 contains a number of new platforms, features and improvements:

Licensing

Cumulus Linux is licensed on a per-instance basis. Each network system is fully operational, enabling any capability to be utilized on the switch with the exception of forwarding on switch panel ports. Only eth0 and console ports are activated on an un-licensed instance of Cumulus Linux. Enabling front panel ports requires a license.

You should have received a license key from Cumulus Networks or an authorized reseller. To install the license, read the Quick Start Guide.

Installing Version 3.6

If you are upgrading from version 3.0.0 or later, use apt-get to update the software.

Cumulus Networks recommends you use the -E option with sudo whenever you run any apt-get command. This option preserves your environment variables, such as HTTP proxies, before you install new packages or upgrade your distribution.

  1. Retrieve the new version packages: cumulus@switch:~$ sudo -E apt-get update

  2. If you are using any early access features from an older release, remove them with: cumulus@switch:~$ sudo -E apt-get remove EA_PACKAGENAME

  3. Upgrade the release: cumulus@switch:~$ sudo -E apt-get upgrade

  4. To include additional Cumulus Linux packages not present in your current version, run the command: cumulus@switch:~$ apt-get install nclu hostapd python-cumulus-restapi linuxptp

    If you already have the latest version of a package installed, you see messages similar to: nclu is already the newest version. You might also see additional packages being installed due to dependencies.

  5. Reboot the switch: cumulus@switch:~$ sudo reboot

If you see errors for expired GPG keys that prevent you from upgrading packages when upgrading to Cumulus Linux 3.6 from 3.5.1 or earlier, follow the steps in  Upgrading Expired GPG Keys .

In Cumulus Linux 3.6.0, the upgrade process has changed. During an upgrade to 3.6.0 from 3.5 or earlier, certain services might be stopped. These services are not restarted until after the switch reboots, which results in some functionality being lost during the upgrade process.

During the upgrade, you will see messages similar to the following:

/usr/sbin/policy-rc.d returned 101, not running 'stop switchd.service'
/usr/sbin/policy-rc.d returned 101, not running 'start switchd.service'

At the end of the upgrade, if a reboot is required, you see the following message:

*** Caution: Service restart prior to reboot could cause unpredictable behavior
*** System reboot required ***

Do not restart services manually until after rebooting, or services will fail.

For upgrades post 3.6.0, if no reboot is required after the upgrade completes, the upgrade will stop and restart all upgraded services and will log messages in the /var/log/syslog file similar to the ones shown below. (In the examples below, only the frr package was upgraded.)

Policy: Service frr.service action stop postponed
Policy: Service frr.service action start postponed
Policy: Restarting services: frr.service
Policy: Finished restarting services
Policy: Removed /usr/sbin/policy-rc.d
Policy: Upgrade is finished

For additional information about upgrading, see Upgrading Cumulus Linux.

New Install or Upgrading from Versions Older than 3.0.0

If you are upgrading from a version older than 3.0.0, or installing Cumulus Linux for the first time, download the Cumulus Linux 3.6.0 installer for Broadcom or Mellanox switches from the Cumulus Networks website, then use ONIE to perform a complete install, following the instructions in the Quick Start Guide.

This method is destructive; any configuration files on the switch are not saved; copy them to a different server before upgrading via ONIE.

After you install, run apt-get update, then apt-get upgrade on your switch to make sure you update Cumulus Linux to include any important or other package updates.

Updating a Deployment that Has MLAG Configured

If you are using MLAG to dual connect two switches in your environment, and those switches are still running Cumulus Linux 2.5 ESR or any other release earlier than 3.0.0, the switches will not be dual-connected after you upgrade the first switch. To ensure a smooth upgrade, follow these steps:

  1. Disable clagd in the /etc/network/interfaces file (set clagd-enable to no), then restart the switchd, networking, and FRR services.

    cumulus@switch:~$ sudo systemctl restart switchd.service
    cumulus@switch:~$ sudo systemctl restart networking.service
    cumulus@switch:~$ sudo systemctl restart frr.service
    
  2. If you are using BGP, notify the BGP neighbors that the switch is going down:

    cumulus@switch:~$ sudo vtysh -c "config t" -c "router bgp" -c "neighbor X.X.X.X shutdown"
    
  3. Stop the Quagga (if upgrading from a version earlier than 3.2.0) or FRR service (if upgrading from version 3.2.0 or later):

    cumulus@switch:~$ sudo systemctl stop [quagga|frr].service 
    
  4. Bring down all the front panel ports:

    cumulus@switch:~$ sudo ip link set swp<#> down
    
  5. Run cl-img-select -fr to boot the switch in the secondary role into ONIE, then reboot the switch.

  6. Install Cumulus Linux 3.6 onto the secondary switch using ONIE. At this time, all traffic is going to the switch in the primary role.

  7. After the install, copy the license file and all the configuration files you backed up, then restart the switchd, networking, and Quagga services. All traffic is still going to the primary switch.

    cumulus@switch:~$ sudo systemctl restart switchd.service
    cumulus@switch:~$ sudo systemctl restart networking.service
    cumulus@switch:~$ sudo systemctl restart quagga.service
    
  8. Run cl-img-select -fr to boot the switch in the primary role into ONIE, then reboot the switch. Now, all traffic is going to the switch in the secondary role that you just upgraded to version 3.6.

  9. Install Cumulus Linux 3.6 onto the primary switch using ONIE.

  10. After the install, copy the license file and all the configuration files you backed up.

  11. Follow the steps for upgrading from Quagga to FRRouting.

  12. Enable clagd again in the /etc/network/interfaces file (set clagd-enable to yes), then run ifreload -a.

    cumulus@switch:~$ sudo ifreload -a
    
  13. Bring up all the front panel ports:

    cumulus@switch:~$ sudo ip link set swp<#> up
    
  14. Now the two switches are dual-connected again and traffic flows to both switches.

Perl, Python and BDB Modules

Any Perl scripts that use the DB_File module or Python scripts that use the bsddb module won’t run under Cumulus Linux 3.6.

Issues Fixed in Cumulus Linux 3.6.2

The following is a list of issues fixed in Cumulus Linux 3.6.2 from earlier versions of Cumulus Linux.

# Control which users/groups are allowed to run "show" commands. users_with_show = root, cumulus, vagrant groups_with_show = netshow, netedit, tacacs

New Known Issues in Cumulus Linux 3.6.2

The following issues are new to Cumulus Linux and affect the current release.

HW Table Utilization

Utilization for HW resource TCAM is 42.9 Utilization for HW resource KVD Hash is 69.9 Utilization for HW resource KVD Linear is 49.9 Utilization for HW resource PGT is 0.0 Utilization for HW resource Flow Counter is 0.0 Utilization for HW resource ACL Regions is 1.0

Logical Free Entries Count

============================================================ | Resource| Free Entries| ============================================================ | UC MAC Table | 67181| | MC MAC Table | 67181| | FIB IPV4 UC Table | 132628| | FIB IPV6 UC Table | 95802| | FIB IPV4 MC Table | 2288| | ARP IPV4 Table | 32569| | ARP IPV6 Table | 12292| | Unicast Adjacency Table| 8197| | L2 MC VECTORS Table | 6999| | ACL Extended Actions Table | 8197| | ACL PBS Table| 8197| | eRIF List | 8197| | ILM Table| 67181| | VLAN Table| 1| | VPorts Table| 67181| | FID Table| 16362| | Policy Based MPLS ILM Table| 8197| | ACL Regions| 396| | ACL Rules 18B Key| 2254| | ACL Rules 32B Key| 1024| | ACL Rules 54B Key| 1022| | RIF Counter Basic| 3276| | RIF Counter Enhanced| 1092| | Flow Counter| 2048| | ACL GROUPS Table | 396|

Logical Table Utilization

================================================================================================ | Resource| HW Table|Logical Entries | HW Entries| Utilization(%)| ================================================================================================ | UC MAC Table | KVD Hash| 43| 43| 0.0| | FIB IPV4 UC Table | KVD Hash| 89| 65790| 26.5| | FIB IPV6 UC Table | KVD Hash| 51| 28926| 11.6| | FIB IPV4 MC Table | TCAM | 0| 192| 1.1| | ARP IPV4 Table | KVD Hash| 199| 32768| 13.2| | ARP IPV6 Table | KVD Hash| 4092| 32768| 179.6| | Unicast Adjacency Table| KVD Linear| 8187| 8187| 49.9| | VPorts Table| KVD Hash| 0| 22| 0.0| | FID Table| KVD Hash| 22| 22| 0.0| | ACL Regions| ACL Regions| 4| 4| 1.0| | ACL Rules 18B Key| TCAM | 2| 64| 0.3| | ACL Rules 54B Key| TCAM | 2| 5760| 35.1| | ACL GROUPS Table |ACL Group Table| 4| 400| 100.0| cumulus@mlx-2700-08:~$

+route-map apply_aspath_prepend permit 10

  • match ip address prefix-list default_route
  • set as-path prepend last-as 1 +end

You can see that the AS "1" is added to the as-path:

cumulus@switch:~$ net show bgp vrf internal ipv4 unicast ==================================

BGP table version is 16, local router ID is 172.18.5.12

Status codes: s suppressed, d damped, h history, * valid, > best, = multipath,

          i internal, r RIB-failure, S Stale, R Removed

Origin codes: i - IGP, e - EGP, ? - incomplete

Network Next Hop Metric LocPrf Weight Path

  • 0.0.0.0 172.16.3.4 0 65000 65021 0 i

  •               172.16.3.4                             0 65000 65021 0 i
    
  •               172.16.3.4                             0 65000 65020 0 i
    
  •               172.16.3.4                             0 65000 65020 0 i
    
  •               172.16.3.3                             0 65000 65019 i
    
  •               172.16.3.3                             0 65000 65019 i
    
  •               172.16.3.3                             0 65000 65018 i
    

*> 172.16.3.3 0 65000 65018 i

VRF internal: B>* 0.0.0.0/0 [20/0] via 172.16.3.4, vlan4000 onlink, 00:12:06

auto lo iface lo inet loopback address 10.26.10.11/32

auto swp9 iface swp9 bridge-access 100

auto swp10 iface swp10 bridge-access 100

auto bridge iface bridge bridge-ports swp9 swp10 vni-10 bridge-vids 100 bridge-vlan-aware yes bridge-mcquerier 1

auto vni-10 iface vni-10 vxlan-id 10 vxlan-local-tunnelip 10.0.0.11 bridge-access 100

auto bridge.100 vlan bridge.100 bridge-igmp-querier-src 123.1.1.1

auto vlan100 iface vlan100 address 10.26.100.2/24 vlan-id 100 vlan-raw-device bridge

Issues Fixed in Cumulus Linux 3.6.1

The following is a list of issues fixed in Cumulus Linux 3.6.1 from earlier versions of Cumulus Linux.

auto bridge iface bridge bridge-vlan-aware yes bridge ports vx-4001 bridge-vids 4001

auto vx-4001 iface vx-4001 vxlan-id 4001 <… usual vxlan config …> bridge-access 4001 # where vnid 4001 represents the L3 VNI

auto vlan4001 iface vlan4001 vlan-id 4001 vlan-raw-device bridge vrf vrf1

auto bridge iface bridge bridge-vlan-aware yes bridge ports swp3 vx-4001 vx-16000000 bridge-vids 4001 2001 # where vx-4001 is the existing VNI and vx-16000000 is a new temporary VNI # this is now bridging the port (swp3), the VNI (vx-4001), # and the new temporary VNI (vx-16000000) # the bridge VLAN IDs are now 4001 and 2001

auto vlan2001 iface vlan2001 vlan-id 2001 vrf vrf1 address 45.0.0.2/24 vlan-raw-device bridge # create a VLAN 2001 with the associated VRF and IP address

auto vx-16000000 iface vx-16000000 vxlan-id 16000000 bridge-access 2001 <… usual vxlan config …> # associate the temporary VNI (vx-16000000) with bridge 2001

auto vx-4001 iface vx-4001 vxlan-id 4001 <… usual vxlan config …> bridge-access 4001 # where vnid 4001 represents the L3 VNI

auto vlan4001 iface vlan4001 vlan-id 4001 vlan-raw-device bridge vrf vrf1

New Known Issues in Cumulus Linux 3.6.1

The following issues are new to Cumulus Linux and affect the current release.

# Control which users/groups are allowed to run "show" commands. users_with_show = root, cumulus, vagrant groups_with_show = netshow, netedit, tacacs

Issues Fixed in Cumulus Linux 3.6.0

The following is a list of issues fixed in Cumulus Linux 3.6.0 from earlier versions of Cumulus Linux.

auto bridge iface bridge bridge-vlan-aware yes bridge ports vx-4001 bridge-vids 4001

auto vx-4001 iface vx-4001 vxlan-id 4001 <… usual vxlan config …> bridge-access 4001 # where vnid 4001 represents the L3 VNI

auto vlan4001 iface vlan4001 vlan-id 4001 vlan-raw-device bridge vrf vrf1

auto bridge iface bridge bridge-vlan-aware yes bridge ports swp3 vx-4001 vx-16000000 bridge-vids 4001 2001 # where vx-4001 is the existing VNI and vx-16000000 is a new temporary VNI # this is now bridging the port (swp3), the VNI (vx-4001), # and the new temporary VNI (vx-16000000) # the bridge VLAN IDs are now 4001 and 2001

auto vlan2001 iface vlan2001 vlan-id 2001 vrf vrf1 address 45.0.0.2/24 vlan-raw-device bridge # create a VLAN 2001 with the associated VRF and IP address

auto vx-16000000 iface vx-16000000 vxlan-id 16000000 bridge-access 2001 <… usual vxlan config …> # associate the temporary VNI (vx-16000000) with bridge 2001

auto vx-4001 iface vx-4001 vxlan-id 4001 <… usual vxlan config …> bridge-access 4001 # where vnid 4001 represents the L3 VNI

auto vlan4001 iface vlan4001 vlan-id 4001 vlan-raw-device bridge vrf vrf1

Known Issues in Cumulus Linux 3.6.0

The following issues are new to Cumulus Linux and affect the current release.

#stats.vxlan.member = BRIEF