Appendix - Software Components Upgrade
It is recommended to upgrade all UFM Enterprise appliance software components as listed in UFM Enterprise Appliance Upgrade.
This section includes optional instructions on how to upgrade specific software components.
Upgrading UFM Enterprise Appliance Operating System: Involves UFM Enterprise appliance operating system upgrade only.
Upgrading All UFM-Related Software Components: Involves all UFM-related software components, including UFM Enterprise, Docker Container and UFM HA. The upgrade is done on all software components at once.
Upgrading Specific UFM-Related Software Component: Involves upgrading specific UFM-related software components separately.
This section provides a step-by-step guide for UFM Enterprise Appliance Operating System upgrade.
Each UFM Enterprise Appliance software has an additional tar file with a -omu.tar suffix (OMU stands for OS Manufacture and Upgrade). This tar file can be used to re-manufacture the server and to upgrade the operating system/software on the server.
Extracting the Software
Copy the OMU tar file to a temporary directory on the server.
UFM-APPLIANCE - ufm-appliance<version>-<revision>-omu.tar
Extract the contents of the tar file to /tmp.
tar
vxf ./ufm-appliance-<version>-<revision>-omu.tar
-C /tmp/Change to the extracted directory.
cd
/tmp/ufm-appliance-<version>-<revision>-omuAn upgrade script and an ISO file are included in the extracted directory.
ls
-1 ./# ls -1 ./
./ufm-os-upgrade.sh ufm-appliance-<version>-<revision>.isoThe following flags are available in the upgrade script help.
# ufm-os-upgrade.sh --help
ufm-os-upgrade.sh will upgrade andinstall
OS packages. IMPORTANT!!! a reboot is mandatory after the finalization of this script, kernel and kernel models will not work properlyuntil
the server is rebooted. Additional SW installations will be automatically invoked after reboot, a message will pop on allopen
terminals with the installation status:"UFM-OS-FIRSTBOOT-FAILURE"
-if
installation is failed."UFM-OS-FIRSTBOOT-SUCCESS"
-if
installation succeeded. Additional info will be availablein
"/var/log/ufm_os_upgrade_@@UFM-OS-VERSION@@.log"
logfile
. Upgrade steps status information can be viewedin
"/var/log/ufm_os_upgrade_@@UFM-OS-VERSION@@_status.log"
logfile
. Syntax: ufm-os-upgrade.sh [options] options -d,--debug debug info will be visible on thescreen
. -r,--reboot Automatically reboot the server when upgrade is finished. P.S.if
secure boot is enabled and a new certificate is enrolled the server will not automatically reboot evenif
this flag isset
. -y,--yes
Will not promptfor
user acknowledgements, use with CAUTION user prompts will be assumed as answeredyes
. -h,--help print this help message.WarningIMPORTANT!!! System reboot is mandatory once the upgrade procedure is completed. The -r flag can be used to automatically reboot the server at the end of the upgrade. Note that some kernel modules may not work properly until server reboot is performed.
Standalone Mode Upgrade
Stop UFM service by running the following command:
systemctl stop ufm-enterprise.service
Run the upgrade script.
NoteSystem reboot is mandatory once the upgrade procedure is completed. The -r flag can be used to automatically reboot the server.
The --appliance-sw-upgrade flag CAN NOT !!! be supplied to upgrade the UFM Enterprise Appliance SW.
The -y flag can be supplied to skip user questions (the flag does not automatically reboot the server on its own. For auto reboot, combine with the -r flag)
Once a secure boot certificate is updated/installed, the script will not auto reboot even if -y and -r flags are provided. That is because the addition of certificates require manual user intervention at boot (after the upgrade).
There is a 10 seconds window to press any button when prompted during the boot procedure and insert the server root password in order to import the certificate. Further details are available in Appendix - Secure Boot Activation and Deactivation.
In the following example the server will auto reboot when upgrade is finished.
./ufm-os-upgrade.sh -y -r
In case a secure-boot certificate is installed/upgraded, the following warning is presented:
In that case the server does not reboot automatically, a manual configuration is required at boot (a 10 second prompt appears during the boot. For more information, refer to Appendix - Secure Boot Activation and Deactivation.
To continue with the upgrade procedure, manually reboot the server from as instructed in Appendix - Secure Boot Activation and Deactivation.
After the reboot procedure is complete, a systemd service (ufm-os-firstboot.service) runs the remainder of the upgrade procedure. Once completed, a message is prompted to all open terminals including the status:
"UFM-OS-FIRSTBOOT-FAILURE" - if installation is failed.
"UFM-OS-FIRSTBOOT-SUCCESS" - if installation succeeded.
Example:
To manually check the status, run systemctl status ufm-os-firstboot.service. If it is already finished, an error message is prompted stating that there is no such service. In that case, the log /var/log/ufm-os-firstboot.log can be checked instead.
systemctl status ufm-os-firstboot.service
Example:
High-Availability Mode Upgrade
Upgrade on HA should be done first on the stand-by node and after that on the master node, each node upgrade is similar to the SA instructions.
In case the Standby node is unavailable, the upgrade can be run on the Master node only, however, some additional steps will be required after the appliance is upgraded.
In case a secure boot certificate needs to be updated/installed, the script will stop execution and request the user to install the secure-boot certificate, secure-boot does not have to be active (although it is highly recommended), but the certificate must be installed/updated by the user before proceeding to the upgrade.
The upgrade script will verify that the certificate is up to date and will stop execution if it needs to be installed/updated (this happens at the start of the script)
[On the stand-by Node]: Copy and extract the OMU tar file to a temporary directory.
[On the stand-by Node]: Run the upgrade script.
NoteSystem reboot is mandatory once the upgrade procedure is completed. The -r flag can be used to automatically reboot the server.
The flag CAN NOT !!! be supplied to upgrade the UFM Enterprise Appliance SW.
The -y flag can be supplied to skip user questions (the flag does not automatically reboot the server on its own. For auto reboot, combine with the -r flag).
In the following example the server auto reboots once the upgrade procedure is completed:
cd
/tmp/ufm-appliance-<version>-<revision>-omu ./ufm-os-upgrade.sh -y -rIf -r flag was not provided reboot the server when the script will finish (a question will show on the screen that will ask to reboot if No was answered a manual reboot is required)
to manually reboot the server:
reboot now
After the reboot procedure is complete, a systemd service (ufm-os-firstboot.service) runs the remainder of the upgrade procedure. Once completed, a message is prompted to all open terminals including the status:
"UFM-OS-FIRSTBOOT-FAILURE" - if installation is failed.
"UFM-OS-FIRSTBOOT-SUCCESS" - if installation succeeded.
Example:
To manually check the status, run systemctl status ufm-os-firstboot.service. If it is already finished, an error message is prompted stating that there is no such service. In that case, the log /var/log/ufm-os-firstboot.log can be checked instead.
systemctl status ufm-os-firstboot.service
Example:
After the stand-by node have finished the upgrade check the HA cluster status
ufm_ha_cluster status
All the nodes in the cluster should be online and the current node should remain a stand-by (Secondary in DRBD_ROLE)
[On the Master Node]: Fail-over the UFM to the stand-by node (upgraded node will become master and current node will become stand-by).
ufm_ha_cluster failover
wait for all the resource of UFM are up and running on the upgraded node.
repeat the procedure on the un-upgraded node (which is now acting as stand-by).
The installation process consists of replacing the containers/packages with the new version and upgrading the UFM data.
Copy the tarball file of UFM Enterprise Appliance software to the /tmp folder.
Connect to the UFM Enterprise Appliance via SSH.
Stop the UFM service/cluster before upgrading.
In SA mode, run:
#systemctl stop ufm-enterprise.service
In HA mode, run:
# ufm_ha_cluster stop
Extract the tarball file and run the installer for the upgrade. Run:
# cd /tmp # tar xvf ufm-appliance-sw-<version>.tar # cd ufm-appliance-sw-<version> # ./install.sh
Installer Options:
-q|--quiet Upgrade UFM without prompt
NoteIn HA mode, this step should be performed on both servers.
After the upgrade, start the UFM service/cluster.
In SA mode, run:
# systemctl start ufm-enterprise.service
In HA mode, run:
# ufm_ha_cluster start
Wait one minute for the service to come up.
Ensure the service health. Run:
# ufm_enterprise_sanity.sh Checking Service... Done Checking Images... Done Checking Containers... Done Checking ufm REST server... Done Sanity tests completed successfully!
Upgrading UFM Docker in SA Mode
For detailed information on upgrading the UFM docker in standalone mode, please refer to Upgrading UFM on Docker Container.
Upgrading UFM Docker in HA Mode
For detailed information on upgrading the UFM docker in high availability mode, please refer to Upgrading UFM Container in High Availability Mode.
Upgrading UFM HA Package
Stop the UFM cluster before upgrading. Run:
ufm_ha_cluster stop
Download the UFM-HA package on both servers using the following command:
wget https:
//www.mellanox.com/downloads/UFM/ufm_ha_5.6.0-4.tgz
For Sha256:
wget https:
//download.nvidia.com/ufm/ufm_ha/5.6.0/ufm_ha_5.6.0-4.sha256
On both servers, extract the downloaded UFM-HA package under /tmp/
On both servers, go to the extracted directory /tmp/ufm_ha_XXX and run the installation script:
./install.sh --upgrade
After the upgrade, start the UFM HA Cluster. Run:
ufm_ha_cluster start
Upgrading UFM Enterprise Appliance CLI Package
Copy the tarball of the UFM CLI package to the /tmp folder.
Extract the tarball file and run the installer. Example:
# cd /tmp # tar xvf ufmcli_<version>.tgz # cd ufmcli_<version> # ./install.sh Creating the UFM3 CLI repository file /etc/apt/sources.list.d/ufmcli.list Refreshing the UFM3 CLI packages information... Installing the UFM3 CLI
package
... Removing the UFM3 CLI local repository /etc/apt/sources.list.d/ufmcli.list Done.Once the upgrade procedure is completed, connect to the UFM Enterprise Appliance via SSH with admin. Run:
ssh admin@<hostname>