Port Mirroring

NVIDIA Onyx User Manual v3.10.4006

Port mirroring enables data plane monitoring functionality which allows the user to send an entire traffic stream for testing. Port mirroring sends a copy of packets of a port’s traffic stream, called “mirrored port”, into an analyzer port. Port mirroring is used for network monitoring. It can be used for intrusion detection, security breaches, latency analysis, capacity and performance matters, and protocol analysis.

The following figure provides an overview of the mirroring functionality.

images/download/attachments/2654380703/Overview_of_Mirroring_Functionalitiy-version-1-modificationdate-1709536799983-api-v2.bmp

There is no limitation on the number of mirroring sources and more than a single source can be mapped to a single analyzer destination.

Port mirroring is performed by configuring mirroring sessions. A session is an association of a mirror port (or more) and an analyzer port.

image2019-9-16_11-18-39-version-1-modificationdate-1709536802617-api-v2.png

A mirroring session is a monitoring configuration mode that has the following parameters:

Parameter

Description

Access

Source interface(s)

List of source interfaces to be mirrored.

RW

Destination interface

A single analyzer port through which all mirrored traffic egress.

RW

Header format

The format and encapsulation of the mirrored traffic when sent to analyzer.

RW

Truncation

Enabling truncation segments each mirrored packet to 64 bytes.

RW

Congestion control

Controls the behavior of the source port when destination port is congested.

RW

Admin state

Administrative state of the monitoring session.

RW

Source Interface

The source interface (mirror port) refers to the interface from which the traffic is monitored. Port mirroring does not affect the switching of the original traffic. The traffic is simply duplicated and sent to the analyzer port. Traffic in any direction (either ingress, egress or both) can be mirrored.

There is no limitation on the number of the source interfaces mapped to a mirroring session.

Warning

Ingress and egress traffic flows of a specific source interface can be mapped to two different sessions.

Warning

There is an option to filter out the specific traffic that needs to be mirrored from the source port by using an ACL action of "monitor session" type. For more details, see "monitor session" command in the ACL Commands section.

LAG

The source interface can be a physical interface or a LAG.

Port mirroring can be configured on a LAG interface but not on a LAG member. When a port is added to a mirrored LAG it inherits the LAG’s mirror configuration. However, if port mirroring configuration is set on a port, that configuration must be removed prior to adding the port to a LAG interface.

When a port is removed from a LAG, the mirror property is switched off for that port.

Control Protocols

All control protocols captured on the mirror port are forwarded to the analyzer port in addition to their normal treatment. For example LACP, STP, and LLDP are forwarded to the analyzer port in addition to their normal treatment by the CPU.

Exceptions to the behavior above are the packets that are being handled by the MAC layer, such as pause frames.

Destination Interface

The destination interface is an analyzer port to which mirrored traffic is directed. The mirrored packets are duplicated, optionally modified, and sent to the analyzer port. Spectrum platforms support up to only 3 analyzer ports, where any mirror port can be mapped to any analyzer port and more than a single mirror port can be mapped to a single analyzer port.

Packets can be forwarded to any destination using the command "destination interface".

The analyzer port supports status and statistics as any other port.

LAG

The destination interface cannot be a member of LAG when the header format is local.

Control Protocols

The destination interface may also operate in part as a standard port, receiving and sending out non-mirrored traffic. When the header format is configured as a local port, ingress control protocol packets that are received by the local analyzer port get discarded.

Advanced MTU Considerations

The analyzer port, like its counterparts, is subject to MTU configuration. It does not send packets longer than configured.

When the analyzer port sends encapsulated traffic, the analyzer traffic has additional headers and therefore longer frame. The MTU must be configured to support the additional length, otherwise, the packet is truncated to the configured MTU.

The system on the receiving end of the analyzer port must be set to handle the egress traffic. If it is not, it might discard it and indicate this in its statistics (packet too long).

Header Format

Ingress traffic from the source interface can be manipulated in several ways depending on the network layout using the command header-format.

If the analyzer system is directly connected to the destination interface, then the only parameters that can be configured on the port are the MTU, speed and port based flow control. Priority flow control is not supported is this case. However, if the analyzer system is indirectly connected to the destination interface, there are two options for switching the mirrored data to the analyzer system:

  • A VLAN tag may be added to the Ethernet header of the mirrored traffic

  • An Ethernet header can be added with include a new destination address and VLAN tag

Warning

It must be taken into account that adding headers increases packet size.

Header_Format_Options-version-1-modificationdate-1709536803853-api-v2.png


Congestion Control

The destination ports might receive pause frames that lead to congestion in the switch port. In addition, too much traffic directed to the analyzer port (for example 40GbE mirror port is directed into 10GbE analyzer port) might also lead to congestion.

In case of congestion:

  • When best effort mode is enabled on the analyzer port, Spectrum drops excessive traffic headed to the analyzer port using tail drop mechanism, however, the regular data (mirrored data heading to its original port) does not suffer from a delay or drops due to the analyzer port congestion.

  • When the best effort mode on the analyzer port is disabled, the Spectrum does not drop the excessive traffic. This might lead to buffer exhaustion and data path packet loss.

The default behavior in congestion situations is to drop any excessive frames that may clog the system.

Warning

ETS, PFC and FC configurations do not apply to the destination port.


Truncation

When enabled, the system can truncate the mirrored packets into smaller 64-byte packets (default) which is enough to capture the packets’ L2 and L3 headers.

Warning

The size of the original mirrored packet (before adding the encapsulation headers, and including the 4 bytes frame check sequence (FCs)) is truncated to 64 bytes.

The following figure presents two network scenarios with direct and remote connectivity to the analyzer equipment. Direct connectivity is when the analyzer is connected to the analyzer port of the switch. In this case there is no need for adding an L2 header to the mirrored traffic. Remote connectivity is when the analyzer is indirectly connected to the analyzer port of the switch. In this situation, adding an L2 header may be necessary depending on the network’s setup.

Mirroring_Session-version-1-modificationdate-1709536801253-api-v2.png

To configure a mirroring session:

  1. Create a session. Run:

    Copy
    Copied!
                

    switch (config) # monitor session 1

    Warning

    This command enters a monitor session configuration mode. Upon first implementation the command also creates the session.

  2. Add source interface(s). Run:

    Copy
    Copied!
                

    switch (config monitor session 1) # add source interface ethernet 1/1 direction both

  3. Add destination interface. Run:

    Copy
    Copied!
                

    switch (config monitor session 1) # destination interface ethernet 1/2

  4. (Optional) Set header format. Run:

    Copy
    Copied!
                

    switch (config monitor session 1) # header-format add-ethernet-header destination-mac 00:0d:ec:f1:a9:c8 add-vlan 10 priority 5

    Warning

    For remote connectivity use the header formats “add-vlan” or “add-ethernet-header”. For local connectivity, use “local”.

  5. (Optional) Truncate the mirrored traffic to 64-byte packets. Run:

    Copy
    Copied!
                

    switch (config monitor session 1) # truncate

  6. (Optional) Set congestion control. Run:

    Copy
    Copied!
                

    switch (config monitor session 1) # congestion pause-excessive-frames

    Warning

    The default for this command is to drop excessive frames. The “pause-excessive-frames” parameter uses flow control to regulate the traffic from the source interfaces.

    Warning

    If the parameter “pause-excessive-frame” is selected, make sure that flow control is enabled on all source interfaces on the ingress direction of the monitoring session using the command “flowcontrol” in the interface configuration mode.

  7. Enable the session. Run:

    Copy
    Copied!
                

    switch (config monitor session 1) # no shutdown

To verify the attributes of a specific mirroring session:

Copy
Copied!
            

switch (config) # show monitor session 1 Session 1: Admin: Enable Status: Up Truncate: Enable Destination interface: eth1/2 Congestion type: pause-excessive-frames Header format: add-ethernet-header -switch priority: 5   Source interfaces -------------------- Interface Direction -------------------- eth1/1 both

To verify the attributes of running mirroring sessions:

Copy
Copied!
            

switch (config) # show monitor session summary Flags: i ingress, e egress, b both   ------------------------------------------------------------- Session Admin Status Mode Destination Source ------------------------------------------------------------- 1 Enable Up add-eth eth1/2 eth1/1(b) 2 Disable Down add-vlan eth1/2 eth1/8(i), po1(e) 3 Enable Up add-eth eth1/5 eth1/18(e) 7 Disable Down local

For more information about this feature and its potential applications, please refer to the following community post:

monitor session

monitor session <session-id>

no monitor session <session-id>

Creates session and enters monitor session configuration mode upon using this command for the first time.

The no form of the command deletes the session.

Syntax Description

session-id

The monitor session ID

Range in Spectrum: 1-3

Range in Spectrum-2: 1-8

Default

N/A

Configuration Mode

config

History

3.3.3500

3.8.1000

Updated syntax

3.9.1000

Updated notes and "session-id" range

Example

switch (config)# monitor session 1

switch (config monitor session 1)#

Related Commands

recirculation

what-just-happened buffer enable

Notes

  • On Spectrum systems, the maximum number of monitor sessions that can be configured is 2 if a recirculation port is configured, and 3 if not.

  • On Spectrum-2 systems, the maximum number of monitor sessions that can be configured is 7 if what-just-happened buffer is enabled, and 8 if not.


destination interface

destination interface <type> <number> [force]

no destination interface

Sets the egress interface number.

The no form of the command deletes the destination interface.

Syntax Description

interface

Sets the interface type and number (e.g. ethernet 1/2)

force

Eliminates the need to shutdown the port prior to the operation

Default

no destination interface

Configuration Mode

config monitor session

History

3.3.3500

3.3.4100

Added force parameter

3.6.4006

Added note

Example

switch (config monitor session 1) # destination interface ethernet 1/2

Related Commands

Notes

  • Port cannot be used as destination port in monitor session when storm-control is configured on port

  • Force command cannot remove storm-control configuration. Error output: “Configuration error, storm control is configured on port”.

  • When removing an interface from a monitor session it gains the default attributes of Ethernet ports


shutdown

shutdown

no shutdown

Disables the session.

The no form of the command enables the session.

Syntax Description

interface

Sets the interface type and number (e.g. ethernet 1/2)

force

Eliminates the need to shutdown the port prior to the operation

Default

Disabled

Configuration Mode

config monitor session

History

3.3.3500

3.3.4100

Added force parameter

3.6.4006

Added note

Example

switch (config monitor session 1) # no shutdown

Related Commands

Notes


add source interface direction

add source interface <type> <number> direction <d-type>

no source interface <type> <number>

Adds a source interface to the mirrored session.

The no form of the command deletes the source interface.

Syntax Description

interface

Sets the interface type and number (e.g. ethernet 1/2)

direction

Configures the direction of the mirrored traffic. The options are as follows:

  • egress - monitors egress traffic

  • ingress - monitors ingress traffic

  • both - monitors egress and ingress traffic

Default

N/A

Configuration Mode

config monitor session

History

3.3.3500

Example

switch (config monitor session 1) # add source interface ethernet 1/1 direction ingress

Related Commands

Notes

  • If mirroring is configured in one direction (e.g. ingress) on an interface and then is configured in the other direction (e.g. egress), then the ultimate setting is “both”


header-format

header-format {local [switch-priority <sp>] | add-vlan <vlan-id> [priority <prio>] [switch-priority <sp>] | add-ethernet-header destination-mac <mac-address> [add-vlan <vlan-id> [priority <prio>]] [switch-priority <sp>]}

no header-format

Sets the header format of the mirrored traffic.

The no form of the command resets the parameter values back to default.

Syntax Description

local

The mirrored header of the frame is not changed

switch-priority

Changes the egress switch priority of the frame

Range: 0-7

add-vlan

An 802.1q VLAN tag is added to the frame

priority

The priority to be added to the Ethernet header

Range: 0-7

add-ethernet-header

Adds an Ethernet header to the mirrored frame

destination-mac

The destination MAC address of the added Ethernet frame

Default

no-change

vlan 1

priority 0

traffic-class 0

Configuration Mode

config monitor session

History

3.3.3500

3.5.1000

Added switch-priority parameter

3.8.2000

Updated switch-priority

Example

switch (config monitor session 1) # header-format add-ethernet-header destination-mac 00:0d:ec:f1:a9:c8 add-vlan 10 priority 5 switch-priority 2

Related Commands

Notes

If add-ethernet-header is used, the source MAC address is the one of the outgoing Ethernet port.


truncate

truncate

no truncate

Truncates the mirrored frames to 64-byte packets.

The no form of the command disables truncation.

Syntax Description

N/A

Default

no truncate

Configuration Mode

config monitor session

History

3.3.3500

3.9.0500

Added note

Example

switch (config monitor session 1) # truncate

Related Commands

Notes

  • This command applies for all sessions on the same analyzer port

  • The size of the original mirrored packet (before adding the encapsulation headers, and including the 4 bytes frame check sequence (FCs)) is truncated to 64 bytes


congestion

congestion [drop-excessive-frames | pause-excessive-frames]

no congestion

Sets the system’s behavior when congested.

The no form of the command disables truncation.

Syntax Description

drop-excessive-frames

Drops excessive frames

pause-excessive-frames

Pauses excessive frames

Default

drop-excessive-frames

Configuration Mode

config monitor session

History

3.3.3500

Example

switch (config monitor session 1) # congestion pause-excessive-frames

Related Commands

Notes

This command applies for all sessions on the same analyzer port


show monitor session

show monitor session <session-id>

Displays monitor session configuration and status.

Syntax Description

session-id

The monitor session ID

Range: 1-7

Default

N/A

Configuration Mode

Any command mode

History

3.3.3500

3.6.5000

Updated Example

Example

switch (config) # show monitor session 1

Session 1:

Admin: Disable

Status: Down

Truncate: Disable

Destination interface: N/A

Congestion type: drop-excessive-frames

Header format: local

-switch priority: 0

Source interfaces

--------------------

Interface Direction

--------------------

eth1/1 both

Related Commands

Notes


show monitor session summary

show monitor session summary

Displays monitor session configuration and status summary.

Syntax Description

session-id

The monitor session ID

Range: 1-7

Default

N/A

Configuration Mode

Any command mode

History

3.3.3500

3.6.5000

Updated Example

Example

switch (config) # show monitor session summary

Flags: i ingress, e egress, b both

-------------------------------------------------------------

Session Admin Status Mode Destination Source

-------------------------------------------------------------

1 Disable Down local N/A eth1/1(b)

2 Disable Down add-vlan eth1/2 eth1/8(i)

Related Commands

Notes


© Copyright 2023, NVIDIA. Last updated on Mar 5, 2024.