Install NetQ for Ethernet and NVLink

Follow these steps to set up and configure your VMs in a cluster of servers. First configure the VM on the master node, and then configure the VM on each additional node. NVIDIA recommends installing the virtual machines on different servers to increase redundancy in the event of a hardware failure.

System Requirements

NetQ for Ethernet and NVLink supports 3-node clusters with the following system requirements. Verify that each node in your cluster meets the VM requirements:

Port Requirements

Confirm that the required ports are open for communications.

Additionally, for internal cluster communication, you must open these ports:

Installation and Configuration

1. Download the NetQ image.

   a. Log in to your NVIDIA Application Hub account.
   b. Select NVIDIA Licensing Portal.
   c. Select Software Downloads from the menu.
   d. In the search field above the table, enter NetQ.
   e. For deployments using KVM, download the NetQ SW 5.0.0 KVM image. For deployments using VMware, download the NetQ SW 5.0.0 VMware image
   f. If prompted, read the license agreement and proceed with the download.
   

2. Open your hypervisor and configure your VM. You can use the following examples for reference or use your own hypervisor instructions.

3. Log in to the VM and change the password.

Use the default credentials to log in the first time:

• Username: nvidia
• Password: nvidia

$ ssh nvidia@<ipaddr>
Warning: Permanently added '<ipaddr>' (ECDSA) to the list of known hosts.
Ubuntu 24.04 LTS
nvidia@<ipaddr>'s password:
You are required to change your password immediately (root enforced)
System information as of Thu Dec  3 21:35:42 UTC 2024
System load:  0.09              Processes:           120
Usage of /:   8.1% of 61.86GB   Users logged in:     0
Memory usage: 5%                IP address for eth0: <ipaddr>
Swap usage:   0%
WARNING: Your password has expired.
You must change your password now and login again!
Changing password for nvidia.
(current) UNIX password: nvidia
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
Connection to <ipaddr> closed.

Log in again with your new password.

$ ssh nvidia@<ipaddr>
Warning: Permanently added '<ipaddr>' (ECDSA) to the list of known hosts.
Ubuntu 24.04 LTS
nvidia@<ipaddr>'s password:
  System information as of Thu Dec  3 21:35:59 UTC 2024
  System load:  0.07              Processes:           121
  Usage of /:   8.1% of 61.86GB   Users logged in:     0
  Memory usage: 5%                IP address for eth0: <ipaddr>
  Swap usage:   0%
Last login: Thu Dec  3 21:35:43 2024 from <local-ipaddr>
nvidia@ubuntu:~$

4. Verify that the master node is ready for installation. Fix any errors before installing the NetQ software.

nvidia@hostname:~$ sudo opta-check-scale

5. Change the hostname for the VM from the default value.

The default hostname for the NetQ virtual machines is ubuntu. Change the hostname to fit your naming conventions while meeting Internet and Kubernetes naming standards.

Set the new hostname using the following command. Replace NEW_HOSTNAME with the name you chose.

nvidia@hostname:~$ sudo hostnamectl set-hostname NEW_HOSTNAME

Add the same NEW_HOSTNAME value to /etc/hosts on your VM for the localhost entry. For example:

127.0.0.1 localhost NEW_HOSTNAME

6. Open your hypervisor and set up the VM for the additional nodes in the same manner as for the master node.

7. Run the following command on each node to verify that the node is ready for a NetQ software installation. Fix any errors indicated before installing the software.

nvidia@hostname:~$ sudo opta-check-scale

8. Install and activate the NetQ software using the CLI.

Run the following command on your master node to initialize the cluster. Copy the output of the command which includes the SSH key. You will use it in the next step. 

nvidia@<hostname>:~$ netq install cluster master-init
    Please run the following command on all worker nodes:
    netq install cluster worker-init c3NoLXJzYSBBQUFBQjNOemFDMXljMkVBQUFBREFRQUJBQUFCQVFDM2NjTTZPdVM3dQN9MWTU1a

9. Run the netq install cluster worker-init <ssh-key> command on each non-master node.

10. Create a JSON template using the installation command for your deployment model. Run the netq install cluster config generate command on your master node to generate a template for the cluster configuration JSON file: 

nvidia@netq-server:~$ netq install cluster config generate
2024-10-28 17:29:53.260462: master-node-installer: Writing cluster installation configuration template file @ /tmp/combined-cluster-config.json

11. Edit the cluster configuration JSON file with the values for each attribute.

nvidia@netq-server:~$ vim /tmp/combined-cluster-config.json 
{
        "version": "v2.0",
        "interface": "<INPUT>",
        "cluster-vip": "<INPUT>",
        "master-ip": "<INPUT>",
        "is-ipv6": "<INPUT>",
        "ha-nodes": [
                {
                        "ip": "<INPUT>"
                },
                {
                        "ip": "<INPUT>"
                }
                ],
        "shared-cluster-install": "<INPUT>"
        "storage-path": "/var/lib/longhorn"
        "alertmanager_webhook_url": "<INPUT>"

}

nvidia@netq-server:~$ vim /tmp/combined-cluster-config.json 
{
        "version": "v2.0",
        "interface": "eth0",
        "cluster-vip": "10.176.235.101",
        "master-ip": "10.176.235.51",
        "is-ipv6": false,
        "ha-nodes": [
                {
                        "ip": "10.176.235.52"
                },
                {
                        "ip": "10.176.235.53"
                },
                ],
        "shared-cluster-install": false,
        "storage-path": "/var/lib/longhorn",
        "alertmanager_webhook_url": "http://alert.example.com:9093/webhook"

}

12. Run the installation command on your master node using the JSON configuration file that you created in the previous step.

nvidia@<hostname>:~$ netq install cluster combined bundle /mnt/installables/NetQ-5.0.0.tgz /tmp/combined-cluster-config.json

Verify Installation Status

To view the status of the installation, use the netq show status [verbose] command. The following example shows a successful 3-node installation:

State: Active
    NetQ Live State: Active
    Installation Status: FINISHED
    Version: 5.0.0
    Installer Version: 5.0.0
    Installation Type: Cluster
    Installation Mode: Combined
    Activation Key: EhVuZXRxLWVuZHBvaW50LWdhdGV3YXkYsagDIixPSUJCOHBPWUFnWXI2dGlGY2hTRzExR2E5aSt6ZnpjOUvpVVTaDdpZEhFPQ==
    Master SSH Public Key: c3NoLXJzYSBBQUFBQjNOemFDMXljMkVBQUFBREFRQUJBQUFCZ1FDNW9iVXB6RkczNkRC
    Is Cloud: False
    
    Kubernetes Cluster Nodes Status:
    IP Address      Hostname       Role    NodeStatus    Virtual IP
    ------------    -----------    ------  ------------  ------------
    10.176.235.53   10.176.235.53  Worker  Ready         10.176.235.56
    10.176.235.52   10.176.235.52  Worker  Ready         10.176.235.55
    10.176.235.51   10.176.235.51  Master  Ready         10.176.235.54
    
    In Summary, Live state of the NetQ is... Active

Run the netq show opta-health command to verify that all applications are operating properly. Allow at least 15 minutes for all applications to come up and report their status.

Next Steps

After NetQ is installed, you can log in to NetQ from your browser using the virtual cluster IP address.

To access NVLink data, perform a system bringup to connect to telemetry and controller services.