ClusterMinder Plugin

The ClusterMinder plugin collects telemetry data from multiple data sources and aggregates, streams and visualizes the backend data. The plugin can cluster/group aggregated data from multiple machines that allows operational anomaly and misconfiguration detection. The plugin provides Cluster-wide histograms of hardware telemetry which details compute node configuration and inventory, PCIe bus, hardware information (SN and FW version) and health alerts of all relevant devices.

The plugin can be deployed as a container and supports multiple data sources

1.1 Data Source Supported Types and Devices

SATEC PM 130,135,175

The plugin can be deployed using the following methods:

1. On the UFM Appliance

2. On the UFM Software

To deploy the plugin, follow these steps:

• The plugin is included in the default plugin bundle available at NVIDIA's Licensing Portal .

• Load the downloaded image onto the UFM server. This can be done either by using the UFM GUI by navigating to the Settings -> Plugins Management tab or by loading the image via the following instructions:

  • Log in to the UFM server terminal.

  • Run:

    Copy

    Copied!

                
    
                
    docker load < <path_to_image> 
            

  • After successfully loading the plugin image, the plugin should become visible within the plugins management table within the UFM GUI. To initiate the plugin’s execution, simply right-click on the respective in the table.

After the successful deployment of the plugin, a new item is shown in the UFM side menu for the ClusterMinder plugin

Managing data sources with ClusterMinder is intuitive and efficient. Users can easily add new data sources, update existing ones, or remove those that are no longer needed. This feature ensures your data is always current and relevant.

4.1 Example of Adding Data Source

Adding hosts is done through the "Data Sources" section. To add a data source, you start by selecting the appropriate tab, clicking , and filling out the information on the endpoint. Then you must first test the endpoint and if the endpoint status is "up," the add button will become clickable, allowing you to add the data source.

Note: we support adding hosts in hostlist format for exmaple: agx[01-10].

for example:

4.2 Example of Adding a Redfish Host

After pressing the button you will be presented with a form for inputting the following fields: " BMC IP", "Protocol","Username" and "Password". after inputting the needed information, pressing the button tests the connection to the host. You will then be presented with a window notifying if the connection was successful. if successful you can click the button to add the data source.

For example:

Note: when adding multiple redfish hosts, they should all be of the same model and vendor. If this is not done it can cause issues with data collection and presentation.

4.3 Example of Adding a Switch Host

After pressing the button you will be presented with a form for inputting the following fields: "Switch IP", "Username", "Password". after inputting the needed information, pressing the button tests the connection to the host. You will then be presented with a window notifying if the connection was successful. if successful you can click the button to add the data source.

Note: when adding multiple switch hosts, they should all be of the same OS type. If this is not done it can cause issues with data collection and presentation.

4.4 Example of Adding a User Generated Data Source Type

Pressing the button in the row of data types allows the user to create custom user generated data source, you will be presented with a form for inputting the following fields: "Type", "Name". After inputting the needed information, you can click the button to add the data source.

4.5 Example of Removing Data Source Endpoint

Removing hosts is done through the "Data Sources" section. here you can right click any available host and click the remove option.

You can also use the button

to open the Remove Data Sources form that has the IP field. After choosing the appropriate IP you can click the button to remove those endpoints

4.6 Example of Updating Data Source Endpoint

Updating hosts is done through the "Data Sources" section. here you can right click any available host and click the update option.

You can also use the button to open the Update Data Sources form that thar has the IP field. After filling the appropriate fields, pressing the button tests the connection to the host. You will then be presented with a window notifying if the connection was successful. if successful you can click the button to update the endpoint.

5.1 Data Tab

The Data tab provides a comprehensive view of all your data in one place, organized in an easy-to-navigate tree or table view. Additionally, there is a group view tab that allows users to see differences in host groups, helping to identify and understand misconfigurations within their data. This feature allows users to quickly access and navigate through their data, making it easier to perform analyses and derive insights.

5.1.1 Redfish Data Example

5.1.2 Switch Data Example

5.1.3 Additional Features

Filter Bar

The filter allows to choose specific API based on frequency level. The search box allows to search for substring in the API or a specific data field. You can also choose to only show APIs with a minimum or maximum number of groups .

Group View

The group view inner tab allows to view endpoints divided to groups based on differences in data.

5.2 Group Differences Tab

The “Group differences” tab helps users identify and understand misconfigurations within their data. By comparing different data groups, users can easily spot discrepancies and take corrective actions. At the start of the report, there is a summary table that lists the most problematic hosts in descending order based on the number of appearances. This table provides a quick overview of the hosts that require the most attention, allowing users to prioritize their troubleshooting efforts effectively. Additionally, each table in the report (besides the summary table) includes the API that provided the data, a column for the number of hosts per group, and the fields where their values differed, ensuring transparency and traceability of the information presented.

5.2.1 Redfish Group Differences Example

5.2.2 Switch Group Differences Example

5.3 Suspected Errors Tab

The “Suspected Errors” tab provides a comprehensive report on APIs that have returned values flagged as potential issues. This report is crucial for maintaining the reliability and integrity of your data analysis. At the start of the report, a summary table lists the number of hosts with errors compared to the total number of hosts, giving users a quick snapshot of the overall health of their data environment. This summary helps prioritize troubleshooting efforts and allocate resources effectively. Each entry in the report details the host ID, the specific fields where values were problematic, and the problematic values themselves. These values are highlighted in red or orange to indicate the severity of the issue, with red denoting more critical problems and orange indicating less severe ones. This color-coding allows users to quickly assess the urgency of each issue and take appropriate corrective actions.

5.3.1 Redfish Suspected Errors Example

5.3.2 Switch Suspected Errors Example

5.4 Alerts and Alerts Rules Tab

The Alert tab displays all fields whose values meet or violate specific thresholds defined by the user. The Alert Rules tab provides an overview of all the rules the user has created. Currently only supported on CDU.

5.5 Additional Features

5.5.1 Historical Change log Tab

Displays fields that have had their values changed, sorted by SRC ID and API.

5.5.2 API Response Duration Tab

Displays the duration of each APIs data collection.

5.5.3 Log Tab

The log page in Cluster Minder provides a comprehensive view of all product logging activities. This feature is essential for tracking changes, monitoring system performance, and diagnosing issues.

Visualizing data is made simple with ClusterMinder’s histogram feature. Histograms are premade depending on the type of data source as well as the make and model. The histograms range from rpm speeds, voltages, wattage, amperage, temperatures (Celsius) and more. By clicking the histograms, users can filter specific hosts or values, allowing for a more targeted analysis. Additionally, using the dropdown menu, users can further filter the histogram bars to hone in on specific data subsets and attributes, enhancing their ability to derive meaningful insights.

6.1 Redfish Histogram Example

6.2 Switch Histogram Example

ClusterMinder’s telemetry page allows users to create custom graphs based on their data. Additionally, there are premade graphs available depending on the telemetry data the data source provides. This feature provides flexibility in how data is displayed and analyzed, enabling users to tailor their graphs to meet specific needs.

7.1 Redfish Telemetry Example

7.2 Switch Telemetry Example

7.3 Additional Features

7.3.1 Graphs Tab

Allows user to create a custom graph. To do so users should navigate to the “Graphs” tab. Here, they can press the button to initiate new graph creation. Next, users select a provider, device, and counter to add to the graph. Finally, they name the graph and complete the process by pressing the finish button. Graphs offer various options such as time frame and resolution, as well as built-in filtering by pressing the specific counter, allowing for more precise and customizable data visualization.

7.3.2 Dashboards

Dashboards in this project enable users to collect and organize specific graphs they create, grouping related visualizations within a single dashboard for easier access and comparison. This allows users to save sets of graphs that serve a common purpose and effortlessly switch between multiple dashboards to manage different groups of visualizations tailored to various needs.

Dashboard example

ClusterMinder’s analytics page allows users to view suspected errors and group differences reports in one aggregated location as well as viewing common suspected errors and common group differences. Analytics page also provides alerts (currently only CDU supported) for various user determined scenarios. This page provides ease of use viewing of all the text reports in a singular location as well as important user tailored alerts.