NVIDIA Onyx User Manual v3.10.2002
NVIDIA MLNX-GW User Manual for NVIDIA Skyway Appliance v8.2.2200

Link Aggregation Group (LAG)

Link Aggregation Group (LAG) protocol describes a network operation in which several same speed links are combined into a single logical entity with the accumulated bandwidth of the originating ports. LAG groups exchange Lag Aggregation Control Protocol (LACP) packets in order to align the functionality between both endpoints of the LAG. To equally send traffic on all LAG links, the switch uses a hash function which can use a set of attributes as key to the hash function.

As many as 32 physical ports can be aggregated on a single LAG.

  1. Create a port-channel entity.

    Copy
    Copied!
                

    switch (config) # interface port-channel 1 switch (config interface port-channel 1) #

  2. Change back to config mode.

    Copy
    Copied!
                

    switch (config interface port-channel 1) # exit switch (config) #

  3. Add a physical port to the LAG.

    Copy
    Copied!
                

    switch (config interface ethernet 1/4) # channel-group 1 mode on switch (config interface ethernet 1/4) #

    Warning

    If the physical port is operationally up, this port becomes an active member of the aggregation. Consequently, it becomes able to convey traffic.

  1. Create a port-channel entity.

    Copy
    Copied!
                

    switch (config) # interface port-channel 1 switch (config interface port-channel 1) #

  2. Change back to config mode.

    Copy
    Copied!
                

    switch (config interface port-channel 1) # exit switch (config) #

  3. Enable LACP in the switch.

    Copy
    Copied!
                

    switch (config) # lacp

  4. Add a physical port to the LAG.

    Copy
    Copied!
                

    switch (config interface ethernet 1/4) # channel-group 1 mode active

    Or:

    Copy
    Copied!
                

    switch (config interface ethernet 1/4) # channel-group 1 mode passive

For more information about this feature and its potential applications, please refer to the following community posts:

interface port-channel

interface port-channel <1-4096>[-<2-4096>]
no interface port-channel <1-4096>[-<2-4096>]

Creates a LAG and enters the LAG configuration mode. There is an option to create a range of LAG interfaces.
The no form of the command deletes the LAG, or range of LAGs.

Syntax Description

1-4096 / 2-4096

LAG number

Default

N/A

Configuration Mode

config

History

3.1.1400

3.2.1100

Added range support

Example

switch (config)# interface port-channel 1
switch (config interface port-channel 1) # exit
switch (config)# interface port-channel 1-10
switch (config interface port-channel 1-10) #

Related Commands

show interface port-channel

Notes

  • If a LAG is also an IPL, attempting to delete it without first deleting the IPL is rejected by the management

  • LAGs have forwarding mode in accordance with the global configuration

lacp

lacp
no lacp

Enables LACP in the switch.
The no form of the command disables LACP in the switch.

Syntax Description

N/A

Default

LACP is disabled

Configuration Mode

config

History

3.1.1400

Example

switch (config)# lacp

Related Commands

Notes

lacp system-priority

lacp system-priority <1-65535>
no lacp system-priority

Configures the LACP system priority.
The no form of the command sets the LACP system-priority to default.

Syntax Description

1-65535

LACP system-priority

Default

32768

Configuration Mode

config

History

3.1.1400

Example

switch (config)# lacp system-priority 1

Related Commands

show lacp interfaces port-channel

Notes

Each device that runs LACP has an LACP system priority value. A value between 1 and 65535 can be configured. LACP uses the system priority with the MAC address to form the system ID. When setting the priority, a higher number means a lower priority.

lacp (interface)

lacp {rate fast | port-priority <1-65535>}
no lacp {rate fast | port-priority}

Configures the LACP interface parameters.
The no form of the command sets the LACP interface configuration to default.

Syntax Description

rate fast

Sets LACP PDUs on the port to be in fast (1 second) or slow rate (30 seconds)

1-65535

LACP port-priority

Default

rate—slow (30 seconds)
port-priority—32768

Configuration Mode

config interfaces ethernet

History

3.1.1400

Example

switch (config interfaces ethernet 1/7)# lacp rate fast

Related Commands

Notes

Configuring LACP rate (fast or slow) will configure the peer port to send (fast or slow), it does not make any affect on the local port LACP rate.

port-channel load-balance ethernet

port-channel load-balance ethernet {<method> | [symmetric]}
no port-channel load-balance ethernet {<method> | [symmetric]}

Configures the port-channel load balancing distribution function method, with symmetric hashing enabled or not.
The no form of the command sets the distribution function method to default, or disabling symmetric hashing.

Syntax Description

method

destination-ip

Destination IP address

destination-mac

Destination MAC address

destination-port

Destination UDP/TCP port

flow-label

IPv6 flow-label field

l2-protocol

Ethertype field

l3-protocol

IP protocol field

ingress-port

Ingress port

source-destination-ip

Source and destination IP addresses

source-destination-mac

Source and destination MAC addresses

source-destination-port

Source and destination UDP/TCP ports

source-ip

Source IP address

source-mac

Source MAC address

source-port

Source UDP/TCP port

symmetric

Symmetric hashing; bidirectional flows follow same path

symmetric

Enables symmetric hashing

Default

source-destination-mac, source-destination-ip, source-destination-port, l3-protocol, l2-protocol, flow-label

Configuration Mode

config

History

3.1.1400

3.8.1000

Updated syntax

3.8.2100

Changed the method options. Modified default LAG HASH to support TCP/UDP ports.

Example

switch (config) # port-channel load-balance ethernet ?
destination-ip                 Destination IP address
destination-mac Destination MAC address
destination-port Destination UDP/TCP port
flow-label IPv6 flow-label field
l2-protocol Ethertype field
l3-protocol IP protocol field
ingress-port                   Ingress port
source-destination-ip Source and destination IP addresses
source-destination-mac Source and destination MAC addresses
source-destination-port Source and destination UDP/TCP ports
source-ip Source IP address
source-mac Source MAC address
source-port Source UDP/TCP port
symmetric Symmetric hashing; bidirectional flows follow same path

Related Commands

show interface port-channel load-balance

Notes

  • As of 3.8.2100, the default value of port-channel load-balance has been changed from "source-destination-mac" to "source-destination-mac, source-destination-ip, source-destination-port, l3-protocol, l2-protocol, flow-label". This occurs only upon fresh installations or after "reset factory". Upgrading users will retain the old load balancing value and show running-config will indicate this.

  • Several load balance methods can be configured (refer to the example)

  • "ingress-port" and "symmetric" cannot both be set at the same time. The command will be rejected under the following conditions:

    • 1) "ingress-port" and "symmetric" both appear in the same command.

    • 2) "ingress-port" is requested while "symmetric" is in force from a previous command. It needs to be cancelled first with "no port-channel load-balance ethernet symmetric".

    • 3)"symmetric" is requested BY ITSELF while "ingress-port" is in force from a previous command. If "symmetric" is part of a larger list that does not include "ingress-port", the meaning is to exclude "ingress-port" and the command will be accepted.

  • When symmetric is set without other methods: only symmetric hashing can be set while other methods remain unchanged

  • When symmetric is set together with other methods: symmetric hashing is set in parallel with other methods

  • When other methods are set without symmetric: other methods are set, while symmetric hashing remains unchanged

channel-group

channel-group <1-4096> [mode {on | active | passive}]
no channel-group

Assigns and configures a physical interface to a LAG.
The no form of the command removes a physical interface from the port-channel.

Syntax Description

1-4096

The port channel number

mode on

Static assignment the port to LAG. LACP will not be enabled on this port.

mode active/passive

Dynamic assignment of the port to LAG. LACP will be enabled in either passive or active mode.

Default

N/A

Configuration Mode

config interface ethernet

History

3.1.1400

3.4.0008

Added a note

3.6.3640

Added a note

3.6.4006

Added a note

Example

switch (config interface ethernet 1/7) # channel-group 1 mode active

Related Commands

show interfaces port-channel summary
show interfaces port-channel compatibility-parameters
show lacp interfaces ethernet

Notes

  • Setting the mode to active/passive is possible only in LACP is enabled

  • The first port in the LAG decide if the LAG will be static (“on”) or LACP (“active” , “pasive”)

  • All the ports in the LAG must have the same configuration, determines by the first port added to the LAG. The port with a different configuration will be rejected, for the list of dependencies refer to “show interfaces port-channel compatibility-parameters”.

  • A physical port may only be part of one channel-group

  • Added support to check if the forwarding mode of the interface is the same as the forwarding mode of LAG. Error output:
    % Channel-group and Ethernet port have different port forwarding mode configuration

  • Port cannot be added to port-channel when storm-control is configured on port. Error output:
    % Interface * has storm control configuration and can't be added to LAG

lacp-individual enable

lacp-individual enable [force]
no lacp-individual enable [force]

Configures the LAG to act with LACP-individual capabilities.
The no form of the command disables the LACP-individual capability.

Syntax Description

force

Toggles the interface after enabling LACP-individual

Default

N/A

Configuration Mode

config interface port-channel

History

3.4.1100

Example

switch (config interface port-channel 10) # lacp-individual enable force

Related Commands

Notes

If a switch is connected via LAG to a host without LACP capability, running this command on that LAG allows a member port (with the lowest numerical priority value), acting as an individual, to communicate with the host

ip address dhcp

ip address dhcp
no ip address dhcp

Enables DHCP on this LAG interface.
The no form of the command disables DHCP on this LAG interface.

Syntax Description

N/A

Default

Disabled

Configuration Mode

config interface port-channel set as router interface

History

3.4.2008

Example

switch (config interface port channel 10) # ip address dhcp

Related Commands

interface port-channel
show interface port-channel

Notes

show lacp counters

show lacp counters

Displays the LACP PDUs counters.

Syntax Description

N/A

Default

N/A

Configuration Mode

config interface port-channel set as router interface

History

3.1.1400

3.6.6000

Updated example

Example

switch (config) # show lacp counters
VRF Name: default
Port-channel 5:
-----------------------------------------------------------------------------------
LACPDUs Marker Marker Marker Rsp Marker Rsp LACPDUs LACPDUs Illegal Unknown
Port Sent Recv Sent Recv Sent Recv
-----------------------------------------------------------------------------------
1/12 0 0 0 0 0 0 0 0
1/11 0 0 0 0 0 0 0 0
1/10 0 0 0 0 0 0 0 0

Related Commands

interface port-channel
show interface port-channel

Notes

show lacp interfaces ethernet

show lacp interface ethernet <inf>

Displays the LACP interface configuration and status.

Syntax Description

inf

Interface number (e.g., “1/1”)

Default

N/A

Configuration Mode

Any command mode

History

3.1.1400

3.6.6102

Updated example

Example

switch (config) # show lacp interfaces ethernet 1/1 
Port: 1/1
Port State: Down
Channel Group: 1
Pseudo port-channel: Po1
LACP port-priority: 32768
LACP Rate: Slow
LACP Activity: Active
LACP Timeout: Short
Aggregation State: Aggregation, Defaulted,
-------------------------------------------------------------
LACP Port Admin Oper Port Port
Port State Priority Key Key Number State
-------------------------------------------------------------
1/1 Down 32768 13826 13826 0x1 0x0

Related Commands

Notes

show lacp interfaces neighbor

show lacp interfaces neighbor

Displays the LACP interface neighbor status.

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

3.1.1400

3.4.0000

Updated example

Example

switch (config) # show lacp interfaces neighbor
Flags:
A - Device is in Active mode
P - Device is in Passive mode

Channel group 1 neighbors

Port 1/4
----------
Partner System ID : 00:00:00:00:00:00
Flags : A
LACP Partner Port Priority : 0
LACP Partner Oper Key : 0
LACP Partner Port State : 0x0

Port State Flags Decode
------------------------
Activity : Active
Aggregation State : Aggregation, Sync, Collecting, Distributing

MLAG channel group 25 neighbors

Port 1/49
----------
Partner System ID : 00:02:c9:fa:c4:c0
Flags : A
LACP Partner Port Priority : 255
LACP Partner Oper Key : 33
LACP Partner Port State : 0xbc

Port State Flags Decode
------------------------
Activity : Active
Aggregation State : Aggregation, Sync, Collecting, Distributing,

MLAG channel group 28 neighbors

Port 1/51
----------
Partner System ID : f4:52:14:10:d8:f1
Flags : A
LACP Partner Port Priority : 255
LACP Partner Oper Key : 33
LACP Partner Port State : 0xbc

Port State Flags Decode
------------------------
Activity : Active
Aggregation State : Aggregation, Sync, Collecting, Distributing,

Related Commands

Notes

show lacp

show lacp

Displays the LACP global parameters.

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

3.4.0000

Example

switch (config) # show lacp
Port-channel Module Admin Status is enabled

Related Commands

Notes

show lacp interfaces system-identifier

show lacp interfaces {mlag-port-channel | port-channel} <instance> system-identifier

Displays the system identifier of LACP.

Syntax Description

instance

LAG or MLAG instance

Default

N/A

Configuration Mode

Any command mode

History

3.4.0000

Example

switch (config)# show lacp interfaces port-channel 2 system-identifier
Priority: 12345
MAC: 00:02:c9:ac:2a:60

Related Commands

Notes

show interfaces port-channel

show interfaces port-channel <port-channel>

Displays LAG configuration properties.

Syntax Description

port-channel

LAG interface whose properties to display

Default

N/A

Configuration Mode

Any command mode

History

3.3.4000

3.4.1100

Updated example

3.6.1002

Added “error packets” counter to Tx

3.6.5000

Updated example with telemetry

3.6.8008

Updated example

3.7.1000

Updated example

3.9.1000

Updated example

Example

switch (config) # show interfaces port-channel 10

Po10:
Admin state : Enabled
Operational state : Down
Description : N/A
Mac address : N/A
MTU : 1500 bytes (Maximum packet size 1522 bytes)
lacp-individual mode: Disabled
Flow-control : receive off send off
Actual speed : N/A
Width reduction mode: Not supported
Switchport mode : access
MAC learning mode : Enabled
Forwarding mode : inherited cut-through
FCS Ingress : Enabled CRC check
FCS Egress : Disabled CRC recalculate
FCS Timestamping : Enabled

Telemetry sampling: Disabled TCs: N/A
Telemetry threshold: Disabled TCs: N/A
Telemetry threshold level: N/A

Last clearing of "show interface" counters: Never
60 seconds ingress rate : 0 bits/sec, 0 bytes/sec, 0 packets/sec
60 seconds egress rate : 0 bits/sec, 0 bytes/sec, 0 packets/sec

Rx:
0 packets
0 unicast packets
0 multicast packets
0 broadcast packets
0 bytes
0 discard packets
0 error packets
0 fcs errors
0 undersize packets
0 oversize packets
0 pause packets
0 unknown control opcode
0 symbol errors
0 discard packets by storm control

Tx:
0 packets
0 unicast packets
0 multicast packets
0 broadcast packets
0 bytes
0 discard packets
0 error packets
0 hoq discard packets

Related Commands

Notes

show interfaces port-channel counters

show interfaces port-channel <port-channel> counters

Displays the extended counters for the interface.

Syntax Description

port-channel

LAG interface whose properties to display.

Default

N/A

Configuration Mode

Any command mode

History

3.6.1002

3.9.1000

Added ability to use a range of ports

Example

switch (config) # show interfaces port-channel 2-3 counters
Po2:
Rx:
0 packets
0 unicast packets
0 multicast packets
0 broadcast packets
0 bytes
0 packets of 64 bytes
0 packets of 65-127 bytes
0 packets of 128-255 bytes
0 packets of 256-511 bytes
0 packets of 512-1023 bytes
0 packets of 1024-1518 bytes
0 packets Jumbo
0 error packets
0 discard packets
0 fcs errors
0 undersize packets
0 oversize packets
0 pause packets
0 unknown control opcode
0 symbol errors

Tx
1000000 packets
0 unicast packets
1000000 multicast packets
0 broadcast packets
1505000000 bytes
1000000 error packets
0 discard packets
0 pause packets
0 ECN marked packets

Po3:

. . .

Related Commands

Notes

As of version 3.9.1000, the "port-channel" attribute is optional. If nothing is selected, information for all ports will be displayed

show interfaces port-channel compatibility-parameters

show interfaces port-channel compatibility-parameters

Displays LAG parameters.

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

3.3.4000

3.6.3640

Added “forwarding mode” as compatibility parameter to output

3.6.6000

Updated example

3.6.8008

Updated example

Example

switch (config) # show interfaces port-channel compatibility-parameters

Compatibility-parameters:
* Port-mode
* Speed
* MTU
* Forwarding mode
* Flow Control
* Access VLAN
* Allowed VLAN list
* Flowcontrol & PFC
* Channel-group mode
* QoS parameters
* MAC learning disable

Static configuration on the port should be removed:
* ACL port binding
* Static mrouter
* sflow
* OpenFlow
* port mirroring local analyzer port
* Static mac address

Related Commands

Notes

show interfaces port-channel load-balance

show interfaces port-channel load-balance

Displays the type of load-balancing in use for LAGs.

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

3.3.4000

Example

switch (config) # show interfaces port-channel load-balance

source-destination-mac

Related Commands

port-channel load-balance ethernet ?

Notes

show interfaces port-channel summary

show interfaces port-channel summary

Displays a summary for LAG interfaces.

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

3.1.1400

3.4.1100

Updated example

Example

switch (config) # show interfaces port-channel summary

Flags: D - Down, U - Up, P - Up in port-channel (members)
S - Suspend in port-channel (members), I - Individual

-----------------------------------------------------------------------
Group Port- Type Member Ports
Channel
-----------------------------------------------------------------------
1 Po2(U) LACP Eth1/58(D) Eth1/59(I) Eth1/60(S)
2 Po5(D) LACP Eth1/1(S) Eth1/33(I)
3 Po10(U) LACP Eth1/49(P) Eth1/50(P) Eth1/51(S) Eth1/52(S)

Related Commands

Notes

© Copyright 2023, NVIDIA. Last updated on May 23, 2023.