Buffer Histograms Monitoring

As it is becoming increasingly complex to manage networks, and network administrators need more tools to understand network behavior, it is necessary to provide basic information about network performance, identify network bottlenecks, and provide information for the purposes of network optimization and future planning.

Therefore, network administrators are required to constantly review network port behavior, record port buffer consumption, and identify shortage in buffer resources and record flows which lead to the excessive buffer consumption. NVIDIA Onyx provides the following mechanisms to perform these tasks:

  • Sampling (histograms)—a network administrator can enable a sampling of the port buffer occupancy, record occupancy changes over time, and provide information for different levels of buffer occupancy, and amount of time the buffer has been occupied during the observation period.

  • Thresholds—thresholds may be enabled per port to record the network time when port buffer occupancy crosses the defined threshold and when buffer occupancy drops below it.

  • Flow recording—a record of the most active flows which cause an excessive usage of the port buffers may be kept. Once enabled, the system may identify flow patterns and present a user with a list of flows, based on which a network administrator can rearrange distribution of the data flows in the network and minimize data loss.

For more information about this feature and its potential applications, please refer to the following community post:

protocol telemetry

protocol telemetry no protocol telemetry

Unhides telemetry config CLIs. The no form of the command hides telemetry config CLIs.

Syntax Description

N/A

Default

Hidden

Configuration Mode

config

History

3.6.3004

Example

switch (config) # protocol telemetry

Related Commands

Notes


telemetry shutdown

telemetry shutdown

no telemetry shutdown

Disables the telemetry protocol, threshold detection, and histogram fetching for all sampling enabled interfaces without changing any internal configuration.

The no form of the command enables telemetry protocol.

Syntax Description

N/A

Default

Disabled

Configuration Mode

config

History

3.6.3004

Example

switch (config) # no telemetry shutdown

Related Commands

protocol telemetry

Notes


telemetry sampling log

telemetry sampling log <time>

no telemetry sampling log <time>

Enables the log interval value (histogram fetching) from device.

The no form of the command disables the log interval value.

Syntax Description

time

Input range: 100-60000 (in msec)

Default

1000 millisecond

Configuration Mode

config

History

3.6.3004

Example

switch (config) # telemetry sampling log 1000

Related Commands

protocol telemetry

Notes


telemetry sampling tc

telemetry sampling tc <0-7> [mcast | ucast]

no telemetry sampling tc <0-7> [mcast | ucast]

Enables multicast sampling (histogram fetching) on a traffic class for a particular Ethernet interface.

The no form of the command disables multicast sampling on a TC for a particular Ethernet interface.

Syntax Description

mcast

Multicast traffic

ucast

Unicast traffic

Default

N/A

Configuration Mode

config interface ethernet

History

3.6.3004

Example

switch (config 1/2) # telemetry sampling tc 3 mcast

Related Commands

Notes


telemetry threshold

telemetry threshold tc <0-7> [ucast | mcast]

no telemetry threshold tc <0-7> [ucast | mcast]

Enables threshold in hardware for a particular traffic class.

The no form of the command disables threshold in hardware for a particular traffic class.

Syntax Description

ucast

Unicast traffic

mcast

Multicast traffic

Default

Disabled

Configuration Mode

config interface ethernet

config interface port-channel

config interface mlag-port-channel

History

3.6.5000

Example

switch (config 1/12) # telemetry threshold tc 0 ucast

Related Commands

Notes


telemetry threshold level

telemetry threshold level <level>

no telemetry threshold level <level>

Configures the threshold level in the hardware per port.

The no form of the command resets the parameter to its default.

Syntax Description

level

For Spectrum-based systems:

Range: 96-1,000,000

Level is set in bytes and in increments of 96

For Spectrum-2 and Spectrum-3-based systems:

Range: 144-1,000,000

Level is set in bytes and in increments of 144

Default

69984

Configuration Mode

config interface ethernet

config interface port-channel

config interface mlag-port-channel

History

3.6.5000

3.9.0900

Added minimum level value for Spectrum-2 and Spectrum-3

Example

switch (config 1/12) # telemetry threshold level 288

Related Commands

Notes


telemetry threshold log

telemetry threshold log

no telemetry threshold log

Enables logging of threshold events in syslog.

The no form of the command disables logging.

Syntax Description

N/A

Default

Disabled

Configuration Mode

config

History

3.6.4006

Example

switch (config) # telemetry threshold log

Related Commands

Notes


telemetry threshold syslog

telemetry threshold syslog <time>

no telemetry threshold syslog <time>

The command sets threshold events logging rate on per hour basis.

The no form of the command sets the logging rate back to default.

Syntax Description

time

Max rate per hour

Range: 1-3600

Default

100

Configuration Mode

config

History

3.6.4006

Example

switch (config) # telemetry threshold syslog 400

Related Commands

Notes


clear telemetry

clear telemetry {threshold | sampling} [interface <type> <port-id>] [tc <0-7> [ucast | mcast]]

Clears telemetry data.

Syntax Description

type

Possible values: ethernet, port-channel, mlag-port-channel

tc

Traffic class

mcast

Multicast traffic

ucast

Unicast traffic

Default

N/A

Configuration Mode

config interface ethernet

config interface port-channel

config interface mlag-port-channel

History

3.6.5000

Example

switch (config interface ethernet 1/12) # clear telemetry threshold level 288

Related Commands

Notes


clear telemetry threshold

clear telemetry threshold [interface <type> <if>]

Clears threshold and top talker data.

Syntax Description

type

Available values:ethernet, port-channel, mlag-port-channel

Default

N/A

Configuration Mode

config

History

3.6.6105

Example

switch (config) # clear telemetry threshold interface ethernet 1/34-1/36

Related Commands

Notes


stats export csv telemetry

stats export csv telemetry <slot>/<port>[/<subport>]/<tc>-[mcast | ucast][filename <name>] [after * *] [before * *]

Exports histograms collected by stats to a csv file.

Syntax Description

slot/port

Port number

subport

Subport number to be used if a port is split

Default

N/A

Configuration Mode

Any command mode

History

3.6.3004

3.9.0500

Updated example

Example

    
switch (config) # stats export csv telemetry 1/1/4-ucast after 2020/03/16 10:54:58 before 2020/03/16 11:16:24    

Generated report file: telemetry-20200316-111704.csv

Related Commands

Notes


file stats telemetry delete

file stats telemetry delete <filename>

Deletes the given .csv file created by “stats export” command to user directory.

Syntax Description

N/A

Default

N/A

Configuration Mode

config

History

3.6.3004

Example

switch (config) # file stats telemetry delete telemetry-20171006-102158.csv

Related Commands

Notes


file stats telemetry delete latest

file stats telemetry delete latest

Delete the latest stats telemetry file.

Syntax Description

N/A

Default

N/A

Configuration Mode

Configure terminal

History

3.8.1000

Example

(config) # file stats telemetry delete latest

Related Commands

file stats telemetry delete <file_name>

file stats telemetry delete all

Notes


file stats telemetry delete all

file stats telemetry delete all

Deletes all stats telemetry files from machine.

Syntax Description

N/A

Default

N/A

Configuration Mode

Configure terminal

History

3.8.1000

Example

(config) # file stats telemetry delete all

Related Commands

file stats telemetry delete <file_name>

file stats telemetry delete latest

Notes


file stats telemetry upload

file stats telemetry upload <filename> [vrf <vrf-name>] <upload-url>

Uploads .csv file created by “stats export” command to user directory.

Syntax Description

N/A

Default

N/A

Configuration Mode

config

History

3.6.3004

3.9.2000—Added VRF option

Example

switch (config) # file stats telemetry upload telemetry-20170119-102715.csv scp://username:password@server//directory

Password (if required): ******

Related Commands

Notes


file stats telemetry upload latest

file stats telemetry upload latest [vrf <vrf-name>] <upload-url>

Upload the latest stats telemetry file to a remote host.

Syntax Description

file stats telemetry upload latest [vrf <vrf-name>] <upload-url>

Default

N/A

Configuration Mode

Configure terminal

History

3.8.1000

3.9.2000—Added VRF option

Example

(config) # file stats telemetry upload latest scp://user:pass@10.135.155.8/tmp

Related Commands

file stats telemetry upload <file_name>

file stats telemetry upload all

Notes


file stats telemetry upload all

file stats telemetry upload all [vrf <vrf-name>] <upload_url>

Upload all stats telemetry files to a remote host.

Syntax Description

vrf-name—VRF to be affected. If "vrf-name" parameter is not specified, "default" VRF will be used.

Default

N/A

Configuration Mode

Configure terminal

History

3.8.1000

3.9.2000—Added VRF option

Example

(config) # file stats telemetry upload all scp://user:pass@10.135.155.8/tmp

Related Commands

file stats telemetry upload <file_name>

file stats telemetry upload latest

Notes


show telemetry

show telemetry

Displays the global configuration of telemetry properties.

Syntax Description

N/A

Default

N/A

Configuration Mode

config

History

3.6.4000

Example

Telemetry Status                   : Enabled

H/W Sampling Interval(nsec) : 512

S/W Sampling Interval(ms) : 1000

Threshold Logging : Disabled

Threshold Logging(rate per hour) : 100

-----------------------------------------------------------------------------------------------------

Interface TC Sampling Threshold Level (bytes)

-----------------------------------------------------------------------------------------------------

Eth1/1 N/A Disabled Disabled N/A

Eth1/2 N/A Disabled Disabled N/A

Eth1/3 N/A Disabled Disabled N/A

Eth1/4 N/A Disabled Disabled N/A

Eth1/5 N/A Disabled Disabled N/A

Related Commands

Notes


show telemetry sampling tc mcast

show telemetry sampling <slot>/<port>[/<subport>] tc <tc_id> mcast

Displays fetched multicast histogram details for a given tc_id of the Ethernet interface.

Syntax Description

slot/port

Ethernet port number

subport

Ethernet subport number to be used if a port is split

tc_id

Range: 0-7

Default

N/A

Configuration Mode

Any command mode

History

3.6.3004

Example

switch (config) # show telemetry sampling 1/2 tc 3 mcast
-----------------------------------------------------------------------------------------------------------------------------------------

Telemetry histogram: Eth1/2 traffic-class 3 - mcast

Time Bin sizes (nsec buffer was occupied in bytes range)

-----------------------------------------------------------------------------------------------------------------------------------------

01/16/17 2976< 27552 52128 76704 101280 125856 150432 175008 199584 199584>

04:09:07.79936 1000000000 0 0 0 0 0 0 0 0 0

04:09:08.80096 1000000000 0 0 0 0 0 0 0 0 0

04:09:09.80355 1000000000 0 0 0 0 0 0 0 0 0

04:09:10.80518 1000000000 0 0 0 0 0 0 0 0 0

04:09:11.80682 1000000000 0 0 0 0 0 0 0 0 0

Related Commands

Notes


show telemetry sampling tc mcast last

show telemetry sampling <slot>/<port>[/<subport>] tc <tc_id> mcast last <num_of_entries>

Displays last num of fetched multicast histogram details for the given tc_id of the ethernet interface.

Syntax Description

slot/port

Ethernet port number

subport

Ethernet subport number to be used if a port is split

tc_id

Range: 0-7

num_of_entries

Range: 0-1000

Default

N/A

Configuration Mode

Any command mode

History

3.6.3004

Example

switch (config) # show telemetry sampling 1/2 tc 3 mcast last 4
------------------------------------------------------------------------------------------------------------------------------------------------------

Telemetry histogram: Eth1/2 traffic-class 3 - mcast

Time Bin sizes (nsec buffer was occupied in bytes range)

------------------------------------------------------------------------------------------------------------------------------------------------------

01/16/17 2976< 27552 52128 76704 101280 125856 150432 175008 199584 199584>

04:23:38.28864 1000000000 0 0 0 0 0 0 0 0 0

04:23:39.28977 1000000000 0 0 0 0 0 0 0 0 0

04:23:40.29111 1000000000 0 0 0 0 0 0 0 0 0

04:23:41.29259 1000000000 0 0 0 0 0 0 0 0 0

Related Commands

Notes

If the requested entries are more than what the DB contains, it prints the amount in the table.


show telemetry sampling tc ucast

show telemetry sampling <slot>/<port>[/<subport>] tc <tc_id> ucast

Displays fetched unicast histogram details for a given TC ID of the Ethernet interface.

Syntax Description

slot/port

Ethernet port number

subport

Ethernet subport number to be used if a port is split

tc_id

Range: 0-7

Default

N/A

Configuration Mode

Any command mode

History

3.6.3004

Example

switch (config) # show telemetry sampling 1/2 tc 6 ucast
------------------------------------------------------------------------------------------------------------------------------------------------------

Telemetry histogram: Eth1/2 traffic-class 6 - ucast

Time Bin sizes (nsec buffer was occupied in bytes range)

------------------------------------------------------------------------------------------------------------------------------------------------------

01/13/17 2976< 27552 52128 76704 101280 125856 150432 175008 199584 199584>

08:18:09.67745 1000000000 0 0 0 0 0 0 0 0 0

08:18:10.67850 1000000000 0 0 0 0 0 0 0 0 0

08:18:11.67953 1000000000 0 0 0 0 0 0 0 0 0

Related Commands

Notes


show telemetry sampling tc ucast last

show telemetry sampling <slot>/<port>[/<subport>] tc <tc_id> ucast last <num_of_entries>

Displays last number of fetched unicast histogram details for the given traffic class ID of the Ethernet interface.

Syntax Description

slot/port

Ethernet port number

subport

Ethernet subport number to be used if a port is split

tc_id

Range: 0-7

num_of_entries

Range: 0-1000

Default

N/A

Configuration Mode

Any command mode

History

3.6.3004

Example

Related Commands

Notes

If the requested entries are more than what the DB contains, it prints the amount in the table.


show telemetry threshold

show telemetry threshold [interface <type> <port-id>] [tc <0-7> [ucast | mcast]]

Displays threshold data for either all interfaces or single interface or per interface per traffic class.

Syntax Description

type

  • ethernet

  • port-channel

  • mlag-port-channel

tc

Traffic class

mcast

Multicast traffic

ucast

Unicast traffic

Default

N/A

Configuration Mode

Any command mode

History

3.6.5000

3.6.6105

Updated example

Example

 switch (config) # show telemetry threshold 1/10-1/13
------------------------------------------------------------------------------------------------

Event-id Date Time Port TC Level Duration(100 usec) Repeated

------------------------------------------------------------------------------------------------

1 09/21/17 10:11:48 Eth 1/10 0 100 102497.61 1

2 09/21/17 10:12:06 Eth 1/10 3 100 85714.76 1

switch (config) # show telemetry threshold interface port-channel 20 tc 2 mcast

------------------------------------------------------------------------------------------------------

Event-id Date Time Port TC Level Duration(100 usec) Repeated

------------------------------------------------------------------------------------------------------

1 09/21/17 10:11:48 Po20 (Eth 1/1) 2 (mcast) 100 102497.61 1

2 09/21/17 10:12:06 Po20 (Eth 1/1) 2 (mcast) 100 85714.76 1

Related Commands

Notes

The command supports displaying up to 1000 threshold events. As a result, if more than 1000 thresholds configured in total, some interfaces may not be displayed. Therefore, to query thresholds for a specific interface, please use the command “show telemetry threshold interface <type> <id>”.


show files stats telemetry

show files stats telemetry [filename]

Displays all files created by the command “stats export csv telemetry”.

Syntax Description

filename

Displays stats for the specified file

Default

N/A

Configuration Mode

Any command mode

History

3.6.3004

3.6.8008

Updated example

Example

switch (config) # show files stats telemetry telemetry-20180527-102715.csv

Hostname :test-switch

Report :telemetry histogram

Time lower bound(UTC) :2018/05/28 05:58:10

Time upper bound(UTC) :2018/05/28 05:58:25

Export time(UTC) :2018/05/28 06:00:06

Time lower bound :2018/05/28 08:58:10 +0300

Time upper bound :2018/05/28 08:58:25 +0300

Export time :2018/05/28 09:00:06 +0300

System version :X86_64 sys_test 2018-05-15 04:02:13 x86_64

Related Commands

stats export csv telemetry

Notes


© Copyright 2023, NVIDIA. Last updated on Nov 15, 2023.