BCM HA#

We will be configuring BCM head node high availability next by provisioning the second BCM head-node

Verify that the head node has power control over the cluster nodes.

% device
% power -c dgx-h100 status
[-head1->device]% power -c dgx-h100 status
ipmi0 .................... [ ON ] bcm-dgx-h100-01
ipmi0 .................... [ ON ] bcm-dgx-h100-02
ipmi0 .................... [ ON ] bcm-dgx-h100-03
ipmi0 .................... [ ON ] bcm-dgx-h100-04
[bcm-head-01->device]%

Power off the cluster nodes.

The cluster nodes must be powered off before configuring HA.

% power -c dgx-h100 off
ipmi0 .................... [ OFF ] bcm-dgx-h100-01
ipmi0 .................... [ OFF ] bcm-dgx-h100-02
ipmi0 .................... [ OFF ] bcm-dgx-h100-03
ipmi0 .................... [ OFF ] bcm-dgx-h100-04

Start the cmha-setup CLI wizard as the root user on the primary head node.

#cmha-setup

Choose Setup and then select SELECT.

image

Choose Configure and then select NEXT.

image

Verify that the cluster license information/MAC found in cmha-setup screen is correct and then select CONTINUE.

image

Configure an external Virtual IP address (obtained from Site Survey) that will be used by the active head node in the HA configuration and then select NEXT.

Note: This will be the IP that should always be used for accessing the active head nodes once HA configuration is complete.

_images/bcm-ha-image15.png

Provide the name of the secondary head node and then select NEXT.

image

Populate the failover network details as prescribed in the Site Survey

There are two options for the failover network:

  1. Using dedicated interfaces on the BCM head nodes or

  2. Utilizing the existing managementnet (internalnet).

If you are using a dedicated network interface, select the chosen failover interface and provide the IP information for the dedicated network.

In this example, we are using the internal network as the failover network by skipping dedicated failover interface configuration.

Refer to the BCM admin guide for BCM HA failover network configuration options.

_images/bcm-ha-image19.png

Configure the IP addresses for the secondary head node that the wizard is about to create and then select NEXT.

_images/bcm-ha-image11.png

The wizard shows a summary of the information that it has collected. The VIP will be assigned to the internal and external interfaces, respectively.

_images/bcm-ha-image1.png

Select Yes to proceed with the failover configuration.

_images/bcm-ha-image18.png

Enter the BCM root password and then select OK.

image

The wizard implements the first steps in the HA configuration. If all the steps show OK, press ENTER to continue. The progress is shown here.

Initializing failover setup on master.............. [ OK ]
Updating shared internal interface................. [ OK ]
Updating shared external interface................. [ OK ]
Updating extra shared internal interfaces.......... [ OK ]
Cloning head node.................................. [ OK ]
Updating secondary master interfaces............... [ OK ]
Updating Failover Object........................... [ OK ]
Restarting cmdaemon................................ [ OK ]
Press any key to continue

When the failover setup installation on the primary master is complete, select OK to exit the wizard to the main HA setup screen

We will come back to the HA setup screen once the secondary headnode is added to the cluster.

image

Power up and PXE boot the secondary head node and then select RESCUE from the grub menu.

_images/bcm-ha-image13.png

After the secondary head node has booted into the rescue environment, run the

/cm/cm-clone-install --failover

command, then enter YES when prompted.

The secondary head node will be cloned from the primary.

image

Specify the inband interface on the secondary node

Press c to save the disk layout when prompted

When cloning is completed, enter y to reboot the secondary head node. The secondary must be set to boot from its hard drive. PXE boot should not be enabled. Use the appliance BMC/BIOS to change the boot order. Wait for the secondary head node to reboot.

image

Then continue the HA setup procedure on the primary head node.

On the primary headnode, in the HA setup screen, select Finalize from the cmha-setup menu and then select NEXT.

This will clone the MySQL database from the primary to the secondary head node.

image

Select CONTINUE on the confirmation screen.

image

Enter the root password and then select OK.

image

The cmha-setup wizard continues. Press ENTER to continue when prompted.

image

The progress is shown in the console during the process.

Updating secondary master mac address.............. [ OK ]
Initializing failover setup on bcm-head-02......... [ OK ]
Stopping cmdaemon.................................. [ OK ]
Cloning cmdaemon database.......................... [ OK ]
Checking database consistency...................... [ OK ]
Starting cmdaemon, chkconfig services.............. [ OK ]
Cloning workload manager databases................. [ OK ]
Cloning additional databases....................... [ OK ]
Update DB permissions.............................. [ OK ]
Checking for dedicated failover network............ [ OK ]
Press any key to continue

The Finalize step is now completed. Select REBOOT and wait for the secondary head node to reboot.

image

You can verify the secondary node’s status from the primary head node using cmsh

[bcm10-headnode1]% device list -f
hostname:20,category:12,ip:20,status:15
hostname (key) category ip status
-----------------------------------------------------------------
bcm-head-01 10.130.122.254 [ UP ]
bcm-head-02 10.130.122.253 [ UP ]
bcm-dgx-h100-01 dgx-h100 10.130.122.5 [ DOWN ]
bcm-dgx-h100-02 dgx-h100 10.130.122.6 [ DOWN ]
bcm-dgx-h100-03 dgx-h100 10.130.122.7 [ DOWN ]
bcm-dgx-h100-04 dgx-h100 10.130.122.8 [ DOWN ]