What can I help you with?
NMX Telemetry (NMX-T) Documentation v1.0.0

Troubleshooting

There are several endpoint that can be reached out using an http client (curl) to get the health status of the NMX-T application and its components:

  • Health check callback

    • curl http://0.0.0.0:9350/healthcheck

    • normal response is {"status":0,"message":"OK"}

  • Telemetry statistics (Prometheus endpoint)

    • curl http://0.0.0.0:9352/management/statistics

    • lists some of the internal operations indicators

  • Telemetry management status (Prometheus endpoint)

    • curl http://0.0.0.0:9352/management/check_status

    • gives overall status and uptime of the IB telemetry instance

  • Log messages are written to /var/log/nmx/nmx-t directory n the local host running the NMX-T application

Enable dump payload

Copy
Copied!
            

curl http://0.0.0.0:9350/telemetry/config --data-binary $'connector.log_data = true'

Log file name is /var/log/nmx/nmx-t/connector.log.

Copy
Copied!
            

2025/01/26 06:41:27 INFO [{"timestamp":1737873650560325000,"fields":"node_guid,port_guid,port_num,port_rcv_ibg1_nvl_pkts,port_rcv_ibg1_non_nvl_pkts,port_rcv_ibg2_pkts,port_xmit_ibg1_nvl_pkts,port_xmit_ibg1_non_nvl_pkts,port_xmit_ibg2_pkts","values":["0x1070fd030058c216,0x1070fd030058c216,9,0,0,0,0,0,0"]}] 2025/01/26 06:41:27 INFO [{"timestamp":1737873650560325000,"fields":"node_guid,sensor_index,mvcr_sensor_name,feed_managed,voltage,current,voltage_managed,power_managed,current_managed,capacity_managed","values":["0xb8cef60300fbf210,8,,,12.51,0,0,0,0,0"]}]

If there are no telemetry rules configured, perform the below:

Copy
Copied!
            

nv set acl ACL1_NMX_TELEMETRY nv set acl ACL1_NMX_TELEMETRY type ipv4 nv set acl ACL1_NMX_TELEMETRY rule 5 match ip tcp dest-port 9351 nv set acl ACL1_NMX_TELEMETRY rule 5 match ip tcp dest-port 9352 nv set acl ACL1_NMX_TELEMETRY rule 5 match ip connection-state new nv set acl ACL1_NMX_TELEMETRY rule 5 match ip connection-state established nv set acl ACL1_NMX_TELEMETRY rule 5 action permit nv set interface eth0 acl ACL1_NMX_TELEMETRY inbound control-plane nv set interface eth0 acl ACL1_NMX_TELEMETRY outbound control-plane nv config apply nv show acl nv show interface eth0 acl

© Copyright 2025, NVIDIA. Last updated on Apr 30, 2025.