gNMI Streaming

You can use gRPC Network Management Interface (gNMI) to collect system metrics and export the data to a gNMI client.

Cumulus Linux supports:

  • gNMI with NetQ, where the netq-agent package includes the gNMI agent that listens over port 9339.
  • gNMI with Cumulus Linux, where Cumulus Linux includes the gNMI agent that listens over port 9339.

To use both gNMI streaming with NetQ and gNMI streaming with Cumulus Linux, you must use different ports.

gNMI with NetQ

This section discusses how to configure and use gNMI with NetQ.

To configure and use gNMI with Cumulus Linux, see gNMI with Cumulus Linux.

Configure the gNMI Agent

The netq-agent package includes the gNMI agent, which it disables by default. To enable the gNMI agent:

cumulus@switch:~$ sudo systemctl enable netq-agent.service
cumulus@switch:~$ sudo systemctl start netq-agent.service
cumulus@switch:~$ netq config add agent gnmi-enable true

The gNMI agent listens over port 9339. You can change the default port in case you use that port in another application. The /etc/netq/netq.yml file stores the configuration.

Use the following commands to adjust the settings:

  1. Disable the gNMI agent:

    cumulus@switch:~$ netq config add agent gnmi-enable false
    
  2. Change the default port over which the gNMI agent listens:

    cumulus@switch:~$ netq config add agent gnmi-port <gnmi_port>
    
  3. Restart the NetQ Agent to incorporate the configuration changes:

    cumulus@switch:~$ netq config restart agent
    

The gNMI agent relies on the data it collects from the NVUE service. For complete data collection with gNMI, you must enable the NVUE service. To check the status of the nvued service, run the sudo systemctl status nvued.service command:

cumulus@switch:mgmt:~$ sudo systemctl status nvued.service
● nvued.service - NVIDIA User Experience Daemon
   Loaded: loaded (/lib/systemd/system/nvued.service; enabled; vendor preset: enabled)
   Active: active (running) since Thu 2023-03-09 20:00:17 UTC; 6 days ago

If necessary, enable and start the service:

cumulus@switch:mgmt:~$ sudo systemctl enable nvued.service
cumulus@switch:mgmt:~$ sudo systemctl start nvued.service

Use the gNMI Agent Only

NVIDIA recommends that you collect data with both the gNMI and NetQ agents. However, if you do not want to collect data with both agents or you are not streaming data to NetQ, you can disable the NetQ agent. Cumulus Linux then sents data only to the gNMI agent.

To disable the NetQ agent:

cumulus@switch:~$ netq config add agent opta-enable false

You cannot disable both the NetQ and gNMI agent. If you enable both agents on Cumulus Linux and a NetQ server is unreachable, the switch does not send the data to gNMI from the following models:

  • openconfig-interfaces
  • openconfig-if-ethernet
  • openconfig-if-ethernet-ext
  • openconfig-system
  • nvidia-if-ethernet-ext

WJH, openconfig-platform, and openconfig-lldp data continue streaming to gNMI in this state. If you are only using gNMI and a NetQ telemetry server does not exist, disable the NetQ agent by setting opta-enable to false.

Supported Subscription Modes

Cumulus Linux supports the following gNMI subscription modes:

  • POLL mode
  • ONCE mode
  • STREAM mode, supported for ON_CHANGE subscriptions only

Supported Models

Cumulus Linux supports the following OpenConfig models:

ModelSupported Data
openconfig-interfacesName, Operstatus, AdminStatus, IfIndex, MTU, LoopbackMode, Enabled, Counters (InPkts, OutPkts, InOctets, InUnicastPkts, InDiscards, InMulticastPkts, InBroadcastPkts, InErrors, OutOctets, OutUnicastPkts, OutMulticastPkts, OutBroadcastPkts, OutDiscards, OutErrors)
openconfig-if-ethernetAutoNegotiate, PortSpeed, MacAddress, NegotiatedPortSpeed, Counters (InJabberFrames, InOversizeFrames,​ InUndersizeFrames)
openconfig-if-ethernet-extFrame size counters (InFrames_64Octets, InFrames_65_127Octets, InFrames_128_255Octets, InFrames_256_511Octets, InFrames_512_1023Octets, InFrames_1024_1518Octets)
openconfig-systemMemory, CPU
openconfig-platformPlatform data (Name, Description, Version)
openconfig-lldpLLDP data (PortIdType, PortDescription, LastUpdate, SystemName, SystemDescription, ChassisId, Ttl, Age, ManagementAddress, ManagementAddressType, Capability)
ModelSupported Data
nvidia-if-wjh-drop-aggregateAggregated WJH drops, including layer 1, layer 2, router, ACL, tunnel, and buffer drops
nvidia-if-ethernet-extExtended Ethernet counters (AlignmentError, InAclDrops, InBufferDrops, InDot3FrameErrors, InDot3LengthErrors, InL3Drops, InPfc0Packets, InPfc1Packets, InPfc2Packets, InPfc3Packets, InPfc4Packets, InPfc5Packets, InPfc6Packets, InPfc7Packets, OutNonQDrops, OutPfc0Packets, OutPfc1Packets, OutPfc2Packets, OutPfc3Packets, OutPfc4Packets, OutPfc5Packets, OutPfc6Packets, OutPfc7Packets, OutQ0WredDrops, OutQ1WredDrops, OutQ2WredDrops, OutQ3WredDrops, OutQ4WredDrops, OutQ5WredDrops, OutQ6WredDrops, OutQ7WredDrops, OutQDrops, OutQLength, OutWredDrops, SymbolErrors, OutTxFifoFull)

The client can use the following YANG models as a reference:

nvidia-if-ethernet-ext
nvidia-if-wjh-drop-aggregate

Collect WJH Data with gNMI

You can export What Just Happened (WJH) data from the NetQ agent to your own gNMI client. Refer to the nvidia-if-wjh-drop-aggregate reference YANG model, above.

The gNMI Agent supports Capabilities and STREAM subscribe requests for WJH events.

WJH Drop Reasons

The data that NetQ sends to the gNMI agent is in the form of WJH drop reasons. The SDK generates the drop reasons and Cumulus Linux stores them in the /usr/etc/wjh_lib_conf.xml file. Use this file as a guide to filter for specific reason types (L1, ACL, and so on), reason IDs, or event severeties.

Layer 1 Drop Reasons

Reason IDReasonDescription
10021Port admin downValidate port configuration
10022Auto-negotiation failureSet port speed manually, disable auto-negotiation
10023Logical mismatch with peer linkCheck cable or transceiver
10024Link training failureCheck cable or transceiver
10025Peer is sending remote faultsReplace cable or transceiver
10026Bad signal integrityReplace cable or transceiver
10027Cable or transceiver is not supportedUse supported cable or transceiver
10028Cable or transceiver is unpluggedPlug cable or transceiver
10029Calibration failureCheck cable or transceiver
10030Cable or transceiver bad statusCheck cable or transceiver
10031Other reasonOther L1 drop reason

Layer 2 Drop Reasons

Reason IDReasonSeverityDescription
201MLAG port isolationNoticeExpected behavior
202Destination MAC is reserved (DMAC=01-80-C2-00-00-0x)ErrorBad packet received from the peer
203VLAN tagging mismatchErrorValidate the VLAN tag configuration on both ends of the link
204Ingress VLAN filteringErrorValidate the VLAN membership configuration on both ends of the link
205Ingress spanning tree filterNoticeExpected behavior
206Unicast MAC table action discardErrorValidate MAC table for this destination MAC
207Multicast egress port list is emptyWarningValidate why IGMP join or multicast router port does not exist
208Port loopback filterErrorValidate MAC table for this destination MAC
209Source MAC is multicastErrorBad packet received from peer
210Source MAC equals destination MACErrorBad packet received from peer

Router Drop Reasons

Reason IDReasonSeverityDescription
301Non-routable packetNoticeExpected behavior
302Blackhole routeWarningValidate routing table for this destination IP
303Unresolved neighbor or next hopWarningValidate ARP table for the neighbor or next hop
304Blackhole ARP or neighborWarningValidate ARP table for the next hop
305IPv6 destination in multicast scope FFx0:/16NoticeExpected behavior - packet is not routable
306IPv6 destination in multicast scope FFx1:/16NoticeExpected behavior - packet is not routable
307Non-IP packetNoticeDestination MAC is the router, packet is not routable
308Unicast destination IP but multicast destination MACErrorBad packet received from the peer
309Destination IP is loopback addressErrorBad packet received from the peer
310Source IP is multicastErrorBad packet received from the peer
311Source IP is in class EErrorBad packet received from the peer
312Source IP is loopback addressErrorBad packet received from the peer
313Source IP is unspecifiedErrorBad packet received from the peer
314Checksum or IPver or IPv4 IHL too shortErrorBad cable or bad packet received from the peer
315Multicast MAC mismatchErrorBad packet received from the peer
316Source IP equals destination IPErrorBad packet received from the peer
317IPv4 source IP is limited broadcastErrorBad packet received from the peer
318IPv4 destination IP is local network (destination=0.0.0.0/8)ErrorBad packet received from the peer
320Ingress router interface is disabledWarningValidate your configuration
321Egress router interface is disabledWarningValidate your configuration
323IPv4 routing table (LPM) unicast missWarningValidate routing table for this destination IP
324IPv6 routing table (LPM) unicast missWarningValidate routing table for this destination IP
325Router interface loopbackWarningValidate the interface configuration
326Packet size is larger than router interface MTUWarningValidate the router interface MTU configuration
327TTL value is too smallWarningActual path is longer than the TTL

Tunnel Drop Reasons

Reason IDReasonSeverityDescription
402Overlay switch - Source MAC is multicastErrorThe peer sent a bad packet
403Overlay switch - Source MAC equals destination MACErrorThe peer sent a bad packet
404Decapsulation errorErrorThe peer sent a bad packet

ACL Drop Reasons

Reason IDReasonSeverityDescription
601Ingress port ACLNoticeValidate Access Control List configuration
602Ingress router ACLNoticeValidate Access Control List
603Egress router ACLNoticeValidate Access Control List
604Egress port ACLNoticeValidate Access Control List

Buffer Drop Reasons

Reason IDReasonSeverityDescription
503Tail dropWarningMonitor network congestion
504WREDWarningMonitor network congestion
505Port TC congestion threshold crossedNoticeMonitor network congestion
506Packet latency threshold crossedNoticeMonitor network congestion

gNMI Client Requests

You can use your gNMI client on a host to request capabilities and data to which the Agent subscribes. The examples below use the gNMIc client..

The following example shows a gNMIc STREAM request for WJH data:

gnmic -a 10.209.37.121:9339 -u cumulus -p ****** --skip-verify subscribe --path "wjh/aggregate/l2/reasons/reason[id=209][severity=error]/state/drop" --mode stream --prefix "/interfaces/interface[name=swp8]/" --target netq

{
  "source": "10.209.37.121:9339",
  "subscription-name": "default-1677695197",
  "timestamp": 1677695102858146800,
  "time": "2023-03-01T18:25:02.8581468Z",
  "prefix": "interfaces/interface[name=swp8]/wjh/aggregate/l2/reasons/reason[severity=error][id=209]",
  "target": "netq",
  "updates": [
    {
      "Path": "state/drop",
      "values": {
        "state/drop": "[{\"AggCount\":283,\"Dip\":\"0.0.0.0\",\"Dmac\":\"1c:34:da:17:93:7c\",\"Dport\":0,\"DropType\":\"L2\",\"EgressPort\":\"\",\"EndTimestamp\":1677695102,\"FirstTimestamp\":1677695072,\"Hostname\":\"neo-switch01\",\"IngressLag\":\"\",\"IngressPort\":\"swp8\",\"Proto\":0,\"Reason\":\"Source MAC is multicast\",\"ReasonId\":209,\"Severity\":\"Error\",\"Sip\":\"0.0.0.0\",\"Smac\":\"01:00:5e:00:00:01\",\"Sport\":0}]"
      }
    }
  ]
}
{
  "source": "10.209.37.121:9339",
  "subscription-name": "default-1677695197",
  "timestamp": 1677695132988218890,
  "time": "2023-03-01T18:25:32.98821889Z",
  "prefix": "interfaces/interface[name=swp8]/wjh/aggregate/l2/reasons/reason[severity=error][id=209]",
  "target": "netq",
  "updates": [
    {
      "Path": "state/drop",
      "values": {
        "state/drop": "[{\"AggCount\":287,\"Dip\":\"0.0.0.0\",\"Dmac\":\"1c:34:da:17:93:7c\",\"Dport\":0,\"DropType\":\"L2\",\"EgressPort\":\"\",\"EndTimestamp\":1677695132,\"FirstTimestamp\":1677695102,\"Hostname\":\"neo-switch01\",\"IngressLag\":\"\",\"IngressPort\":\"swp8\",\"Proto\":0,\"Reason\":\"Source MAC is multicast\",\"ReasonId\":209,\"Severity\":\"Error\",\"Sip\":\"0.0.0.0\",\"Smac\":\"01:00:5e:00:00:01\",\"Sport\":0}]"
      }
    }
  ]
}

The following example shows a gNMIc ONCE mode request for interface port speed:

gnmic -a 10.209.37.121:9339 -u cumulus -p ****** --skip-verify subscribe --path "ethernet/state/port-speed" --mode once --prefix "/interfaces/interface[name=swp1]/" --target netq
{
  "source": "10.209.37.123:9339",
  "subscription-name": "default-1677695151",
  "timestamp": 1677256036962254134,
  "time": "2023-02-24T16:27:16.962254134Z",
  "target": "netq",
  "updates": [
    {
      "Path": "interfaces/interface[name=swp1]/ethernet/state/port-speed",
      "values": {
        "interfaces/interface/ethernet/state/port-speed": "SPEED_1GB"
      }
    }
  ]
}

The following example shows a gNMIc POLL mode request for interface status:

gnmic -a 10.209.37.121:9339 -u cumulus -p ****** --skip-verify subscribe --path "state/oper-status" --mode poll --prefix "/interfaces/interface[name=swp1]/" --target netq
{
  "timestamp": 1677644403153198642,
  "time": "2023-03-01T04:20:03.153198642Z",
  "prefix": "interfaces/interface[name=swp1]",
  "target": "netq",
  "updates": [
    {
      "Path": "state/oper-status",
      "values": {
        "state/oper-status": "UP"
      }
    }
  ]
}
received sync response 'true' from '10.209.37.123:9339'
{
  "timestamp": 1677644403153198642,
  "time": "2023-03-01T04:20:03.153198642Z",
  "prefix": "interfaces/interface[name=swp1]",
  "target": "netq",
  "updates": [
    {
      "Path": "state/oper-status",
      "values": {
        "state/oper-status": "UP"
      }
    }
  ]
}

gNMI with Cumulus Linux

This section discusses how to configure and use gNMI with Cumulus Linux.

To configure and use gNMI with NetQ, see gNMI with NetQ.

When you enable gNMI with Cumulus Linux, do not enable and use Open Telemetry.

Cumulus Linux supports both gNMI dial-in mode, where a collector can start a connection with the switch to collect available statistics, and gNMI dial-out mode, where the switch streams statistics and exports them to a collector.

Configure gNMI Dial-in Mode

In dial-in telemetry mode, the data collector initiates the gRPC connection, the Cumulus Linux switch assumes the role of the gRPC server and the receiver (collector) is the client. The switch pushes data to the collector.

To configure gNMI dial-in mode, you must:

  • Specify the gNMI server listening address
  • Enable the gNMI server.

To configure optional settings for gNMI dial-in mode:

  • Specify the listening port. The default port is 9339.
  • Enable a TLS certificate for validation.
    • Cumulus Linux uses a self-signed certificate. You can generate your own TLS server certificate and bind it with the gNMI server application.
    • If you need to use mTLS on the gNMI RPC, import the certificate of the CA that signed the gNMI client keys (or the client certificate itself) to the switch and configure the gNMI server to use the certificate. You can also apply a CRL.

The following example sets the gNMI server listening address to 10.10.10.1 and the port to 443, and enables the gNMI server:

cumulus@switch:~$ nv set system gnmi-server listening-address 10.10.10.1
cumulus@switch:~$ nv set system gnmi-server port 443
cumulus@switch:~$ nv set system gnmi-server state enabled
cumulus@switch:~$ nv config apply

The following example imports and enables the CA certificate CERT1 and the CRL crl.crt for mTLS:

cumulus@switch:~$ nv action import system security ca-certificate CERT1 passphrase mypassphrase uri-bundle scp://user@pass:1.2.3.4:/opt/certs/cert.p12
cumulus@switch:~$ nv set system gnmi-server mtls ca-certificate CERT1
cumulus@switch:~$ nv action import system security scp:////user@pass:1.2.3.4:/path/to/your/crl.crt. 
cumulus@switch:~$ nv set system gnmi-server mtls crl /etc/ssl/certs/crl.crt
cumulus@switch:~$ nv config apply

Configure gNMI Dial-Out Mode

In dial-out telemetry mode, the Cumulus Linux switch initiates the gRPC connection to the collector through a gRPC tunnel server and assumes the role of the gRPC client.

To configure gNMI dial-out mode, you must:

  • Specify the listening address for each tunnel server to which you want to connect. Cumulus Linux supports a maximum of 10 tunnel servers.
  • Enable the tunnel server.

To configure optional settings for each tunnel server:

  • Specify the target name and target application you want to access. The default target application is GNMI-GNOI.
  • Specify the retry interval. The default retry interval is 30 seconds.
  • Import and enable a TLS or mTLS certificate for validation. You can also apply a CRL. For information about importing certificates and CRLs, refer to Security with Certificates and CRLs.

The following example sets the listening address for tunnel server SERVER1 to 10.1.1.10, and enables the tunnel server:

cumulus@switch:~$ nv set system grpc-tunnel server SERVER1 address 10.1.1.10 
cumulus@switch:~$ nv set system grpc-tunnel server SERVER1 state enabled 
cumulus@switch:~$ nv config apply

The following example sets the listening address for tunnel server SERVER1 to 10.1.1.10 and the port to 443, the target name to TARGET1, the retry interval to 40, the CA certificate to CACERT1, and enables the tunnel server:

cumulus@switch:~$ nv set system gnmi-server listening-address localhost 
cumulus@switch:~$ nv set system grpc-tunnel server SERVER1 address 10.1.1.10 
cumulus@switch:~$ nv set system grpc-tunnel server SERVER1 port 443 
cumulus@switch:~$ nv set system grpc-tunnel server SERVER1 target-name TARGET1 
cumulus@switch:~$ nv set system grpc-tunnel server SERVER1 retry-interval 40
cumulus@switch:~$ nv set system grpc-tunnel server SERVER1 ca-certificate CACERT1 uri scp://user@pass:1.2.3.4:/opt/certs/cert.p12
cumulus@switch:~$ nv set system grpc-tunnel server SERVER1 state enabled 
cumulus@switch:~$ nv config apply

Show gNMI Configuration and Status Information

To show gNMI server configuration and connection information, such the number of active subscriptions, received and rejected subscription requests, and received capability requests, run the nv show system gnmi-server command.

cumulus@switch:~$ nv show system gnmi-server 
                                  operational  applied    
--------------------------------  -----------  -----------
state                             disabled     enabled   
certificate                       self-signed  self-signed
port                              9339         9339
[listening-address]               10.1.1.100   10.1.1.100        
version                                                   
status                                                    
  total-active-subscriptions      0                       
  received-subscription-requests  0                       
  rejected-subscriptions          0                       
  received-capabilities-requests  0                       
  [client]

To show the listening address of the gNMI server, run the nv show system gnmi-server listening-address command:

cumulus@switch:~$ nv show system gnmi-server listening-address
----------
10.1.1.100

To show gNMI server mTLS information, run the nv show system gnmi-server mtls command:

cumulus@switch:~$ nv show system gnmi-server mtls
                operational  applied  pending         
--------------  -----------  -------  ----------------
ca-certificate  CACERT1       CACERT1   CACERT          
crl                                   abcdefghijklmnop

To show only gNMI server connection information, run the nv show system gnmi-server status command:

cumulus@switch:~$ nv show system gnmi-server status
                                operational
------------------------------  -----------
total-active-subscriptions      0          
received-subscription-requests  0          
rejected-subscriptions          0          
received-capabilities-requests  0

To show gRPC tunnel server configuration and connection information, run the nv show system grpc-tunnel server <server> command:

cumulus@switch:~$ nv show system grpc-tunnel server SERVER1
nv show system grpc-tunnel server SERVER1
                 operational           applied  
---------------  --------------------  ---------
state            disabled              enabled  
target-name      TARGET1               TARGET1  
address          10.1.1.10             10.1.1.10
port             443                   443      
target-type      gnmi-gnoi             gnmi-gnoi
retry-interval   40                    40       
status                                          
  local-port     0                              
  remote-port    0                              
  connection                                    
    established  1970-01-01T00:00:00Z           
    register     no                             
    tunnel       no

To show the local and remote port, and connection information, run the nv show system grpc-tunnel server SERVER1 status command:

cumulus@switch:~$ nv show system grpc-tunnel server SERVER1 status
               operational         
-------------  --------------------
local-port     0                   
remote-port    0                   
connection                         
  established  1970-01-01T00:00:00Z
  register     no                  
  tunnel       no

To show only connection information, run the nv show system grpc-tunnel server SERVER1 status connection command:

cumulus@switch:~$ nv show system grpc-tunnel server SERVER1 status connection 
             operational         
-----------  --------------------
established  1970-01-01T00:00:00Z
register     no                  
tunnel       no

RPC Methods

Cumulus Linux supports the following RPC methods: 

  • Capabilities
  • Subscription types and options:
    • STREAM (sample_interval, updates_only, suppress_redundant, and heartbeat_interval)
    • ON_CHANGE (updates_only and heartbeat_interval)
  • Notification and update types:
    • sync_response
    • update
    • delete

Cumulus Linux does not support GET or SET RPC events.

Encoding Types

Cumulus Linux supports the Protobuf and JSON data formats.

Wildcard Support

Cumulus Linux supports wildcard matching of keys. For example:

qos/interfaces/interface[interface-id=*]/output/queues/queue[name=*]/state/transmit-octets

You can use a combination of wildcard and specific keys; for example, to collect a metric for all queues on a specific interface:

/qos/interfaces/interface[interface-id=<name>]/output/queues/queue[*]/state/transmit-octets.

Regex for specific keys (such as “interface-id=swp*”) is not supported.

Metrics

Cumulus Linux supports the following metrics:

NameDescription
/interfaces/interface/state/admin-statusAdmin state of an interface.
/interfaces/interface/state/counters/in-broadcast-pktsTotal number of broadcast packets received on an interface.
/interfaces/interface/state/counters/in-multicast-pktsTotal number of multicast packets received on an interface.
/interfaces/interface/state/counters/in-octetsTotal number of octets received on an interface, including framing characters.
/interfaces/interface/ethernet/state/counters/in-fcs-errorsTotal number of frames received on an interface that are an integral number of octets in length but do not pass the FCS check. This count does not include frames received with frame-too-long or frame-too-short error.
/interfaces/interface/ethernet/state/counters/in-oversize-framesTotal number of packets received longer than 1518 octets (excluding framing bits, but including FCS octets).
/interfaces/interface/ethernet/state/counters/in-distribution/in-frames-1024-1518-octetsTotal number of packets (including bad packets) received between 1024 and 1518 octets in length inclusive (excluding framing bits but including FCS octets).
/interfaces/interface/ethernet/state/counters/in-distribution/ in-frames-128-255-octetsTotal number of packets (including bad packets) received between 128 and 255 octets in length inclusive (excluding framing bits but including FCS octets).
/interfaces/interface/ethernet/state/counters/in-distribution/in-frames-256-511-octetsTotal number of packets (including bad packets) received between 256 and 511 octets in length inclusive (excluding framing bits but including FCS octets).
/interfaces/interface/ethernet/state/counters/in-distribution/in-frames-512-1023-octetsTotal number of packets (including bad packets) received between 512 and 1023 octets in length inclusive (excluding framing bits but including FCS octets).
/interfaces/interface/ethernet/state/counters/in-distribution/in-frames-64-octetsTotal number of packets (including bad packets) received that are 64 octets in length (excluding framing bits but including FCS octets).
/interfaces/interface/ethernet/state/counters/in-distribution/in-frames-65-127-octetsTotal number of packets (including bad packets) received between 65 and 127 octets in length inclusive (excluding framing bits but including FCS octets).
/interfaces/interface/state/counters/out-broadcast-pktsTotal number of broadcast packets transmitted out of an interface.
/interfaces/interface/state/counters/out-octetsTotal number of octets transmitted out of an interface, including framing characters.
/interfaces/interface/state/counters/out-unicast-pktsTotal number of unicast packets transmitted out of an interface.
/interfaces/interface/ethernet/state/port-speedAn estimate of the interface current bandwidth in units of 1,000,000 bits per second.
/interfaces/interface/state/counters/in-bits-rateInbound bits per second on an interface.
/interfaces/interface/state/ifindexA unique value, greater than zero, for each interface.
/interfaces/interface/state/counters/in-discardsNumber of inbound packets discarded even though no errors are detected to prevent them from being deliverable to a higher-layer protocol.
/interfaces/interface/state/counters/in-errorsFor packet-oriented interfaces, the number of inbound packets that contained errors preventing them from being deliverable to a higher-layer protocol.
/interfaces/interface/state/counters/in-pktsNumber of packets discarded from the egress queue of an interface.
/interfaces/interface/state/counters/in-pkts-rateInbound packets per second on an interface.
/interfaces/interface/ethernet/state/counters/in-crc-errorsTotal number of frames received with a length (excluding framing bits, but including FCS octets) of between 64 and 1518 octets, inclusive, but had either a bad Frame Check Sequence (FCS) with an integral number of octets (FCS Error) or a bad FCS with a non-integral number of octets (Alignment Error).
/interfaces/interface/ethernet/state/negotiated-duplex-modeWhen auto-negotiate is set to TRUE, and the interface has completed auto-negotiation with the remote peer, this value shows the negotiated duplex mode.
/interfaces/interface/state/counters/in-acl-dropsNumber of inbound packets dropped because of an Access Control List (ACL).
/interfaces/interface/ethernet/state/counters/in-mac-pause-framesInbound MAC pause frames on an interface.
/interfaces/interface/state/mtuSize of the largest packet that can be sent or received on the interface, specified in octets. For interfaces used for transmitting network datagrams, this is the size of the largest network datagram that the interface can send.
/interfaces/interface/state/oper-statusCurrent operational state of an interface.
/interfaces/interface/state/counters/out-bits-rateOutbound bits per second on an interface.
/interfaces/interface/state/counters/out-discardsNumber of outbound packets discarded even though no errors are detected to prevent them from being transmitted.
/interfaces/interface/state/counters/out-errorsFor packet-oriented interfaces, the number of outbound packets not transmitted because of errors. For character-oriented or fixed-length interfaces, the number of outbound transmission units not transmitted because of errors.
/interfaces/interface/state/counters/out-pktsTotal number of packets transmitted out of the interface, including all unicast, multicast, broadcast, and bad packets.
/interfaces/interface/state/counters/out-pkts-rateOutbound packets per second on an interface.
/interfaces/interface/state/counters/out-multicast-pktsTotal number of packets that higher-level protocols requested be transmitted, and which were addressed to a multicast address at this sub-layer, including those that were discarded or not sent. For a MAC layer protocol, this includes both Group and Functional addresses.
/interfaces/interface/ethernet/state/counters/carrier-transitionsNumber of times since system boot that ifOperStatus changed.
/interfaces/interface/ethernet/phy/state/rs-fec-uncorrectable-blocksNumber of RS FEC uncorrectable blocks of an interface.
/interfaces/interface/ethernet/phy/state/rs-fec-single-error-blocksNumber of RS FEC uncorrectable blocks of an interface.
/interfaces/interface/ethernet/phy/state/rs-fec-no-error-blocksNumber of RS FEC no errors blocks of an interface.
/interfaces/interface/ethernet/phy/state/lane/fc-fec-corrected-blocksNumber FC FEC corrected blocks for a given lane of an interface.
/interfaces/interface/ethernet/phy/state/lane/fc-fec-uncorrected-blocksNumber of FC FEC uncorrectable blocks for a given lane of an interface.
/interfaces/interface/ethernet/phy/state/lane/rs-fec-corrected-symbolsNumber of RS FEC corrected symbols for a given lane of an interface.
/interfaces/interface/ethernet/phy/state/corrected-bitsNumber of phy corrected bits of an interface by the FEC engine.
/interfaces/interface/ethernet/phy/state/effective-errorsNumber of phy effective errors of an interface.
/interfaces/interface/ethernet/phy/state/effective-berPhy effective BER of an interface.
/interfaces/interface/ethernet/phy/state/lane/raw-errorsNumber of phy error bits identified for a given lane of an interface.
/interfaces/interface/ethernet/phy/state/received-bitsNumber of phy total bits received for an interface.
/interfaces/interface/ethernet/phy/state/symbol-errorsNumber of phy symbol errors for an interface.
/interfaces/interface/ethernet/phy/state/symbol-berPhy symbol BER for an interface.
/interfaces/interface/ethernet/phy/state/lane/raw-berNumber of phy bit error rates for a given lane of an interface.
/interfaces/interface/ethernet/phy/state/fec-time-since-last-clearTime after last clear of FEC stats(phy layer).
/interfaces/interface/ethernet/phy/state/ber-time-since-last-clearTime after last clear of BER stats(phy layer).
NameDescription
/lldp/state/chassis-idThe chassis component of the endpoint identifier associated with the transmitting LLDP agent.
/lldp/state/chassis-id-typeThe format and source of the chassis identifier string.
/lldp/state/system-descriptionDescription of the network entity including the full name and version identification of the system’s hardware type, software operating system, and networking software.
/lldp/state/system-nameAdministratively assigned name for the system.
/lldp/interfaces/interface[name=<name>]/neighbors/neighbor[id=<id>]/state/ageLLDP neighbor age after discovery.
/lldp/interfaces/interface[name=<name>]/neighbors/neighbor[id=<id>]/state/management-address/typeEnumerated value for the network address type identified in this TLV.
/lldp/interfaces/interface[name=<name>]/neighbors/neighbor[id=<id>]/state/chassis-idChassis component of the endpoint identifier associated with the transmitting LLDP agent.
/lldp/interfaces/interface[name=<name>]/neighbors/neighbor[id=<id>]/state/chassis-id-typeFormat and source of the chassis identifier string.
/lldp/interfaces/interface[name=<name>]/neighbors/neighbor[id=<id>]/state/system-nameAdministratively assigned name of the system associated with the transmitting LLDP agent.
/lldp/interfaces/interface[name=<name>]/neighbors/neighbor[id=<id>]/state/system-descriptionDescription of the network entity associated with the transmitting LLDP agent.
/lldp/interfaces/interface[name=<name>]/neighbors/neighbor[id=<id>]/state/port-idPort component of the endpoint identifier associated with the transmitting LLDP agent.
/lldp/interfaces/interface[name=<name>]/neighbors/neighbor[id=<id>]/state/port-descriptionBinary string containing the actual port identifier for the port from which this LLDP PDU was transmitted.
/lldp/interfaces/interface[name=<name>]/neighbors/neighbor[id=<id>]/state/port-id-typeFormat and source of the remote port ID string.
/lldp/interfaces/interface[name=<name>]/neighbors/neighbor[id=<id>]/state/ttlIndicates how long information from the neighbor is considered valid.
/lldp/interfaces/interface[name=<name>]/neighbors/neighbor[id=<id>]/capabilities/capability[name=<capability>]/state/enabledIf the corresponding system capability is enabled on the neighbor.
NameDescription
/components/component[name='PSU1']/state/namePSU Name.
/components/component[name='PSU1']/state/oper-statusPSU Status.
/components/component[name='PSU1']/state/descriptionPSU description.
/components/component[name='PSU1']/power-supply/state/capacityPSU capacity in watts.
/components/component[name='PSU1']/power-supply/state/output-currentPSU current in amperes.
/components/component[name='PSU1']/power-supply/state/output-voltagePSU voltage in volts.
/components/component[name='PSU1']/power-supply/state/output-powerPSU power in watts.
/components/component[name='fan0']/state/nameFan name.
/components/component[name='fan0']/state/oper-statusFan Status.
/components/component[name='fan0']/state/descriptionFan Description.
/components/component[name='fan0']/fan/state/speedCurrent (instantaneous) fan speed.
/components/component[name='temp-sensor0']/state/nameTemperature sensor name.
/components/component[name='temp-sensor0'']/state/oper-statusTemperature sensor operational status.
/components/component[name='temp-sensor0'']/state/descriptionTemperature sensor description.
/components/component[name='temp-sensor0']/state/temperature/instantInstant temperature.
/components/component[name='temp-sensor0']/state/temperature/alarm-statusTemperature sensor alarm status.
/components/component[name='transceiver+panelport#']/transceiver/physical-channels/channel[no]/state/input-power/instantInput optical power of a physical channel in units of 0.01dBm, which may be associated with individual physical channels or an aggregate of multiple physical channels.
/components/component[name='transceiver+panelport#']/transceiver/physical-channels/channel[no]/state/laser-bias-current/instantCurrent applied by the system to the transmit laser to achieve the output power. The current is expressed in mA with up to two decimal precision.
/components/component[name='transceiver+panelport']/transceiver/physical-channels/channel[no]/state/output-power/instantOutput optical power of a physical channel in units of 0.01dBm, which might be associated with individual physical channels or an aggregate of multiple physical channels.
NameDescription
/qos/interfaces/interface/state/switch-priority/counters/out-pause-pktsNumber of pause packets for the priority class in the egress queue.
/qos/interfaces/interface/state/priority-group/counters/watermark-maxHigh watermark of cells used in a priority group since last time watermarks were reset.
/qos/interfaces/interface/output/queues/queue/state/watermark-maxHigh watermark of cells used in a queue since last time watermarks were reset.
qos/interfaces/interface/output/queues/queue/state/ecn-marked-pktsNumber of ECN marked packets from this egress queue. If the ECN counter is not enabled, the counter value is 0.
qos/interfaces/interface/output/queues/queue/state/transmit-octetsNumber of transmitted bytes in the egress queue of an interface.
qos/interfaces/interface/output/queues/queue/state/transmit-pktsNumber of transmitted packets in the egress queue of an interface.
/qos/interfaces/interface/output/queues/queue/state/wred-dropped-pktsNumber of packets discarded from this egress queue of an interface.
/qos/interfaces/interface/output/queues/queue/state/no-buffer-uc-dropped-pktsNumber of packets discarded from this egress queue when there is no buffer left in the interface.
/qos/interfaces/interface[interface-id]/output/queues/queue[name]/state/time-since-last-clearTime since last clear of watermarks in a queue.
/qos/interfaces/interface[interface-id]/state/priority-group[priority_group]/counters/time-since-last-clearTime since last clear of watermarks in a priority group.
NameDescription
/network-instances/network-instance[name=<vrf>]/protocols/protocol[identifier=BGP][name=BGP]/bgp/neighbors/neighbor[neighbor-address=<address>]/state/session-stateOperational state of the BGP peer.
/network-instances/network-instance[name=<vrf>]/protocols/protocol[identifier=BGP][name=BGP]/bgp/neighbors/neighbor[neighbor-address=<address>]/state/established-transitionsNumber of transitions to the established state for the neighbor session.
/network-instances/network-instance[name=<vrf>]/protocols/protocol[identifier=BGP][name=BGP]/bgp/neighbors/neighbor[neighbor-address=<address>]/state/messages/sent/UPDATENumber of BGP UPDATE messages announcing, withdrawing, or modifying paths exchanged.
/network-instances/network-instance[name=<vrf>]/protocols/protocol[identifier=BGP][name=BGP]/bgp/neighbors/neighbor[neighbor-address=<address>]/messages/received/UPDATENumber of BGP UPDATE messages announcing, withdrawing, or modifying paths exchanged.
/network-instances/network-instance[name=<vrf>]/protocols/protocol[identifier=BGP][name=BGP]/bgp/neighbors/neighbor[neighbor-address=<address>]/state/queues/inputNumber of messages received from the peer currently queued.
/network-instances/network-instance[name=<vrf>]/protocols/protocol[identifier=BGP][name=BGP]/bgp/neighbors/neighbor[neighbor-address=<address>]/state/queues/outputNumber of messages queued to be sent to the peer.
/network-instances/network-instance[name=<vrf>]/protocols/protocol[identifier=BGP][name=BGP]/bgp/neighbors/neighbor[neighbor-address=<address>]/afi-safis/afi-safi[afi-safi-name=<afi-safi-name>]/state/prefixes/receivedNumber of prefixes received from the neighbor after applying policies (the number of prefixes present in the post-policy Adj-RIB-In for the neighbor).
/network-instances/network-instance[name=<vrf>]/protocols/protocol[identifier=BGP][name=BGP]/bgp/neighbors/neighbor[neighbor-address=<address>]/afi-safis/afi-safi[afi-safi-name=<afi-safi-name>]/state/prefixes/sentNumber of prefixes advertised to the neighbor after applying policies (the number of prefixes present in the post-policy Adj-RIB-Out for the neighbor).
/network-instances/network-instance[name=<vrf>]/protocols/protocol[identifier=BGP][name=BGP]/bgp/neighbors/neighbor[neighbor-address=<address>]/afi-safis/afi-safi[afi-safi-name=<afi-safi-name>]/state/prefixes/installedNumber of prefixes received from the neighbor that are installed in the network instance RIB and actively used for forwarding.
NameDescription
/system/state/hostnameSystem hostname.
/system/state/software-versionSystem software version.
/system/state/boot-timeSystem boot time.
/system/state/current-datetimeCurrent system date and time.
/system/control-plane-traffic/ingress/ipv4/counters/
/system/control-plane-traffic/ingress/ipv6/counters/
Number of input IP datagrams discarded in software including those received in error.
/system/cpus/cpu[name=<cpu_id>]/state/user/secondsCPU user Seconds
/system/cpus/cpu[name=<cpu_id>]/state/kernel/secondsCPU kernel seconds.
/system/cpus/cpu[name=<cpu_id>]/state/nice/secondsCPU Nice seconds.
/system/cpus/cpu[name=<cpu_id>]/state/idle/secondsCPU idle seconds.
/system/cpus/cpu[name=<cpu_id>]/state/wait/secondsCPU wait seconds.
/system/cpus/cpu[name=<cpu_id>]/state/hardware-interrupt/secondsCPU hardware interrupt seconds.
/system/cpus/cpu[name=<cpu_id>]/state/software-interrupt/secondsCPU software interrupt seconds.
/system/memory/state/freeFree memory.
/system/memory/state/physicalPhysical memory.
/system/memory/state/reservedMemory reserved for system use.
/system/mount-points/mount-point[name='filesystem0']/state/nameMount point name.
/system/mount-points/mount-point[name='filesystem0']/state/storage-componentA reference to the hosting component within the hierarchy.
/system/mount-points/mount-point[name='filesystem0']/state/sizeTotal size of the initialized filesystem.
/system/mount-points/mount-point[name='filesystem0']/state/availableAmount of unused space on the filesystem.
/system/mount-points/mount-point[name='filesystem0']/state/typeFilesystem type used for storage such flash, hard disk, tmpfsor or ramdisk, or remote or network based storage.

User Credentials and Authentication

User authentication is enabled by default. gNMI subscription requests must include an HTTP Basic Authentication header according to RFC7617 containing the username and password of a user with NVUE API access permissions. You can enable this setting in the standard gNMI client (gNMIc) by setting the auth-scheme parameter to basic. Refer to https://gnmic.openconfig.net/global_flags/ - auth-scheme.

Cumulus Linux ignores credentials in RPC metadata.

gNMI Client Requests

You can use your gNMI client on a host to request capabilities and data to which the Agent subscribes. The examples below use the gNMIc client.

Dial-in Mode Example

The following example shows a Dial-in Mode Subscribe request with TLS:

gnmic subscribe --mode stream --path "/qos/interfaces/interface[interface-id=swp1]/output/queues/queue[name=1]/state/transmit-octets" -i 10s --tls-cert gnmi_client.crt --tls-key gnmi_client.key -u cumulus -p ******* --auth-scheme Basic --skip-verify -a 10.188.52.108:9339

The following example shows a Dial-in Mode Subscribe request without TLS:

NVIDIA recommends using TLS. To test without TLS, you must also edit the NGINX configuration file on the switch.

gnmic subscribe --mode stream --path "/qos/interfaces/interface[interface-id=swp1]/output/queues/queue[name=1]/state/transmit-octets" -i 10s --insecure -u cumulus -p ******* --auth-scheme Basic -a 10.188.52.108:9339

Subscription Example

The following example shows a subscription response:

{ 
  "sync-response": true 
} 
{ 
  "source": "10.188.52.108:9339", 
  "subscription-name": "default-1737725382", 
  "timestamp": 1737725390247535267, 
  "time": "2025-01-24T13:29:50.247535267Z", 
  "updates": [ 
    { 
      "Path": "qos/interfaces/interface[interface-id=swp1]/output/queues/queue[name=1]/state/transmit-octets", 
      "values": { 
        "qos/interfaces/interface/output/queues/queue/state/transmit-octets": 0 
      } 
    } 
  ] 
} 
...

Capabilities Example

The following example shows a capabilities request:

gnmic capabilities --tls-cert gnmic-cert.pem --tls-key gnmic-key.pem -u cumulus -p ****** --auth-scheme Basic --skip-verify -a 10.188.52.108:9339

The following example shows the expected response to a capabilities request:

gNMI version: 0.10.0 
supported models: 
  - openconfig-ospf-types, OpenConfig working group, 0.1.3 
 
...
 
  - openconfig-platform-fabric, OpenConfig working group, 0.1.0 
  - openconfig-platform-healthz, OpenConfig working group, 0.1.1 
supported encodings: 
  - JSON 
  - JSON_IETF 
  - PROTO 

Considerations

When using gNMI with Cumulus Linux:

  • The minimum sampling interval is 1 second. If you configure a shorter sampling interval, the switch might not behave as expected.
  • ModelData, Origin, and Extensions fields are ignored in requests and not set in responses.