Manual Addition of NVLink Switch Rack Entries#

This section provides step-by-step instructions for manually adding NVLink Switches to BCM with full Zero Touch Provisioning (ZTP) settings. The process includes:

Configuring the NVLink Switch in BCM
Installing the NVOS software
Applying a startup configuration file
Installing the cm-lite-daemon package

Prerequisites#

For adding NVLink Switches to BCM, ensure the following information is available:

MAC address for eth0/COMe0 of the NVLink Switch
IP address for eth0/COMe0 of the NVLink Switch
Management network for eth0/COMe0 of the NVLink Switch
Username and password for SSH access to the NVLink Switch
If eth1/COMe1 is connected instead, obtain the same information for eth1/COMe1

For setting up BCM to perform ZTP for NVLink Switches, the following information will also be needed:

NVOS firmware image files
Startup configuration files or templates

Adding NVLink Switch Entries to BCM#

This section covers the basic steps to add NVLink switch entries to BCM’s device database and can be done instead of the steps outlined in the manual rack import process. These steps create the switch device entry and configure basic network connectivity.

Add the switch in BCM, enable cm-lite-daemon, create a new interface using the MAC address and an assigned IP.

cmsh -c "device; add switch <switch_hostname>; set mac <00:00:00:00:00:00>; set hasclientdaemon yes; set kind nvlink; set disablesnmp yes; commit"

Add eth0 and rf0 interfaces and set network configuration.

cmsh -c "device; use <switch_hostname>; interfaces; add physical eth0; set ip <switch_ip>; set network <ipminet>; add physical rf0; set ip <rf0_ip>; set network <ipminet>; commit"

Note

The COMe0/1 network (for eth0/eth1) and the rf0 network are usually the same out-of-band (OOB) management network, just running at different speeds.

Set management network and verify interface configuration.

cmsh -c "device; use <switch_hostname>; set managementnetwork <management_network>; commit"

Assign the username and password for SSH access.

cmsh -c "device; use <switch_hostname>; accesssettings; set username admin; set password <password>; set restport 443; commit"

Manual ZTP Settings Configuration in BCM#

This section covers how to manually configure ZTP configuration in BCM without bcm-netautogen.

To configure Zero Touch Provisioning (ZTP) to automatically update NVOS software, firmware, and apply startup configurations, follow these steps to configure the ZTP settings in BCM manually. These settings will be applied to all cloned switch entries.

Enable the API on the switch and create the necessary directory structure.
```
cmsh -c "device; use <switch_hostname>; ztpsettings; set scripttemplate nvos-ztp.sh; set jsontemplate nvlink-nvos.json; set enableapi yes; set installlitedaemon yes; commit; exit; initialize"
```
To enable the API on the switch and create the necessary directory structure:
cmsh device use <switch_hostname> ztpsettings set scripttemplate nvos-ztp.sh set jsontemplate nvlink-nvos.json set enableapi yes set installlitedaemon yes show commit exit initialize quit
Note
- Setting installlitedaemon yes will automatically install the cm-lite-daemon on the NVLink switches during the setup process. If it is set to no, manually install cm-lite-daemon on the switches after they have successfully completed the ZTP process. For detailed instructions on manually installing cm-lite-daemon, see the section Install cm-lite-daemon.
- The initialize command creates a directory for every NVLink switch entry at /cm/local/apps/cmd/etc/htdocs/switch/<switch_name>/. This directory will be used to store startup configuration files and other switch-specific files.
Copy the startup configuration and image files into the BCM repositories.
```
# Copy image files to image directory
cp <nvos_image_file> /cm/local/apps/cmd/etc/htdocs/switch/image/

# Copy startup configuration to switch-specific directory
# (Created after the initialize step)
cp <startup_config_file> /cm/local/apps/cmd/etc/htdocs/switch/<switch_name>/
```
File locations:
- Image directory: /cm/local/apps/cmd/etc/htdocs/switch/image
- Startup configuration directory: /cm/local/apps/cmd/etc/htdocs/switch/<switch_name>
Note

The startup configuration directory is created automatically after running the initialize command in the previous step.

Modify the startup file to include the password field with hashed password from accesssettings.

Edit the startup YAML file to include the following in the password field:

- set:
      system:
      aaa:
         user:
            admin:
            hashed-password: '${CMD_ACCESS_HASHED_PASSWORD}'

Sample complete startup.yaml configuration:

Add additional parameters to the device in BCM to apply the startup file and image updates during the ZTP process.

cmsh -c "device; use <switch_hostname>; set nvconfigurationmode file; set nvconfigurationfile <startup_yaml>; ztpsettings; set jsontemplate nvlink-nvos.json; set fm_config_file <fm_config_file>; set image <NVOS_image_name>; set checkimageonboot yes; exit; commit; initialize"

Note

You need to keep the following in mind when you manually configure ZTP settings:

The minimum version of NVOS required for the NVLink Switch is 25.02.2134 to support custom scripts that will do the cm-lite-daemon installation.
The Run ZTP on each boot option, when set to Yes, will enable cm-lite-daemon to enable ZTP if it is disabled.

Clone NVLink Switch Entries#

After completing the ZTP settings configuration for the “golden” switch, clone additional NVLink Switch entries for the rack. The ZTP settings will be inherited by all cloned entries. The “golden” switch is the switch that will be used to clone the other switches.

For 9 NVLink switches in a rack:

foreach -o <golden_switch_hostname> -n <switch_hostname_02>..<switch_hostname_09> --next-ip ()

Example:

foreach -o a05-18-p1-nvsw-01 -n a05-19-p1-nvsw-02..a05-26-p1-nvsw-09 --next-ip ()
commit

Note

IPs and MACs will have to be updated manually for each cloned entry
The ZTP settings configured in the previous section will be applied to all cloned switches
Each switch will automatically receive the same NVOS firmware and startup configurations during ZTP
Ensure that the directory structure is created for each cloned switch. If it is not, run the initialize command again.

Once all the switches are added to BCM, the following commands can be used to verify the configuration.

Verification Commands#

After completing the configuration for all switches:

List all NVLink switches and check their status:

cmsh -c "device; list -t switch |grep -i nvsw"

Verify ZTP completion:

cmsh -c "device; use <switch_hostname>; ssh"
sudo ztp status

Check lite daemon status:

cmsh -c "device; use <switch_hostname>; latesthealthdata"

Troubleshooting#

Common issues and solutions:

ZTP fails: Check network connectivity, verify image and configuration file paths
SSH access fails: Verify accesssettings configuration and network connectivity
Lite daemon installation fails: Check network access to BCM head node, verify package availability
Switch not responding: Verify switch is powered on, check BMC connectivity for remote restart

Log files to check:

/var/log/cmdaemon - BCM daemon logs
/var/log/switch/<switch_name> - Switch-specific ZTP logs