Getting Started

NVIDIA Onyx User Manual v3.10.4006

The procedures described in this page assume that you have already installed and powered on your switch according to the instructions in the Hardware Installation Guide, which was shipped with the product.

Warning

Due to California Senate Bill No. 327,starting from software version 3.8.2000, Admin and Monitor passwords will need to be typed in manually—no automatic passwords will be created by default.

When the reset button is held for 15 seconds, the management module is reset and the password is deleted. You will then be able to enter without a password and make a new password for the user admin.

Warning

Any account created with admin privileges can change all passwords of other user accounts, including other user accounts with admin privileges.

To initialize the switch do the following:

  1. Connect the host PC to the console (RJ-45) port of the switch system using the supplied cable.

    Warning

    DHCP is enabled by default over the MGT port. Therefore, if you have configured your DHCP server and connected an RJ-45 cable to the MGT port, simply log in using the designated IP address.

  2. Configure a serial terminal with the settings described below.

    Warning

    This step may be skipped if the DHCP option is used and an IP is already configured for the MGT port.

    Parameter

    Setting

    Baud Rate

    115200

    Data bits

    8

    Stop bits

    1

    Parity

    None

    Flow Control

    None

  3. The boot menu is prompted.

    Copy
    Copied!
                

    NVIDIA Onyx Boot Menu:   1: <image #1> 2: <image #2> u: USB menu (if USB device is connected) (password required) c: Command prompt (password required)   Choice:

    Warning

    Select “0” to boot with software version installed on partition #1.
    Select “1” to boot with software version installed on partition #2.

    The boot menu features a countdown timer. It is recommended to allow the timer to run out by not selecting any of the options.

  4. Login as admin and use admin as password. If the machine is still initializing, you might not be able to access the CLI until initialization completes. As an indication that initialization is ongoing, a countdown of the number of remaining modules to be configured is displayed in the following format: “<no. of modules> Modules are being configured”.

  5. Go through the Switch Management configuration wizard.

    IP configuration by DHCP:

    Wizard Session Display (Example)

    Comments

    Do you want to use the wizard for initial configuration? yes

    You must perform this configuration the first time you operate the switch or after resetting the switch to the factory defaults. Type “yes” and then press <Enter>.

    Step 1: Hostname? [switch-1]

    If you wish to accept the default hostname, then press <Enter>. Otherwise, type a different hostname and press <Enter>.

    Step 2: Use DHCP on mgmt0 interface? [yes]

    Perform this step to obtain an IP address for the switch. (mgmt0 is the management port of the switch.)

    - If you wish the DHCP server to assign the IP address, type “yes” and press <Enter>.

    If you type “no” (no DHCP), then you will be asked whether you wish to use the “zeroconf” configuration or not. If you enter “yes” (yes Zeroconf), the session will continue as shown in the "IP zeroconf configuration" table.

    If you enter “no” (no Zeroconf), then you need to enter a static IP, and the session will continue as shown in the "Static IP configuration" table.

    Step 3: Enable IPv6 [yes]

    Perform this step to enable IPv6 on management ports. The default is "yes" (enabled).

    If you enter “no” (no IPv6), then you will automatically be referred to Step 5.

    Step 4: Enable IPv6 autoconfig (SLAAC) on mgmt0 interface? [no]

    Perform this step to enable stateless address autoconfig on external management port. The default is "no" (disabled).

    If you wish to enable it, type “yes” and press <Enter>.

    Step 5: Use DHCPv6 on mgmt0 interface? [yes]

    Perform this step to enable DHCPv6 on the MGMT0 interface.

    Step 6: Update time?

    Perform this step to change the time configured. Press <enter> to leave the current time.

    Step 7: Enable password hardening? [yes]

    Perform this step to enable/disable password hardening on your machine. If enabled, new passwords will be checked upon configured restrictions. The default is "yes" (enabled).

    If you wish to disable it, enter “no”.

    Step 8: Admin password (Must be typed)? <new_password>

    To avoid illegal access to the machine, please type a password and then press <Enter>.

    Starting from the 3.8.2000 release, the user must

    type in the admin password upon initial configuration. Due to Senate Bill No. 327, this stage is required and cannot be skipped.

    Step 9: Confirm admin password? <new_password>

    Confirm the password by re-entering it. Note that password characters are not printed.

    Step 10: Monitor password (Must be typed)? <new_password>

    To avoid illegal access to the machine, please type a password and then press <Enter>.

    Starting from the 3.8.2000 release, the user must

    type in the admin password upon initial configuration. Due to Senate Bill No. 327, this stage is required and cannot be skipped.

    Step 11: Confirm monitor password? <new_password>

    Confirm the password by re-entering it. Note that password characters are not printed.

    You have entered the following information:

    Hostname: <switch name>

    Use DHCP on mgmt0 interface: yes

    Enable IPv6: yes

    Enable IPv6 autoconfig (SLAAC) on mgmt0 interface: yes

    Enable DHCPv6 on mgmt0 interface: no

    Update time: <current time>

    Enable password hardening: yes

    Admin password (Enter to leave unchanged): (CHANGED)

    To change an answer, enter the step number to return to.

    Otherwise hit <enter> to save changes and exit.

    Choice: <Enter>

    Configuration changes saved.

    To return to the wizard from the CLI, enter the “configuration jump-start” command

    from configuration mode. Launching CLI...

    <switch name> [standalone: master] >

    The wizard displays a summary of your choices and then asks you to confirm the choices or to re-edit them.

    Either press <Enter> to save changes and exit, or enter the configuration step number that you wish to return to.

    To run the command “configuration jump-start” you must be in Config mode.

    Static IP configuration:

    Wizard Session Display (Example)

    Do you want to use the wizard for initial configuration? y
    Step 1: Hostname? [switch-112126]

    Step 2: Use DHCP on mgmt0 interface? [yes] n

    Step 3: Use zeroconf on mgmt0 interface? [no]

    Step 4: Primary IP address? 192.168.10.4

    Mask length may not be zero if address is not zero (interface mgmt0)

    Step 5: Netmask? [0.0.0.0] 255.255.255.0

    Step 6: Default gateway? 192.168.10.1

    Step 7: Primary DNS server?

    Step 8: Domain name?

    Step 9: Enable IPv6? [yes] yes

    Step 10: Enable IPv6 autoconfig (SLAAC) on mgmt0 interface? [no] no

    Step 11: Update time? [yyyy/mm/dd hh:mm:ss]

    Step 12: Enable password hardening? [yes] yes

    Step 13: Admin password (Enter to leave unchanged)?

    You have entered the following information:
    Hostname: switch-112126

    Use DHCP on mgmt0 interface: no

    Use zeroconf on mgmt0 interface: no

    Primary IP address: 192.168.10.4

    Netmask: 255.255.255.0

    Default gateway: 192.168.10.1

    Primary DNS server:

    Domain name:

    Enable IPv6: yes

    Enable IPv6 autoconfig (SLAAC) on mgmt0 interface: no

    Update time: yyyy/mm/dd hh:mm:ss

    Enable password hardening: yes

    Admin password (Enter to leave unchanged): (unchanged)

    To change an answer, enter the step number to return to.

    Otherwise hit <enter> to save changes and exit.

    Choice:

    Configuration changes saved.

    To return to the wizard from the CLI, enter the “configuration jump-start” command from configure mode. Launching CLI...

    <hostname>[standalone: master] >

    IP zeroconf configuration:

    Wizard Session Display (Example)

    Configuration wizard

    Do you want to use the wizard for initial configuration? y

    Step 1: Hostname? [switch-112126]

    Step 2: Use DHCP on mgmt0 interface? [no]

    Step 3: Use zeroconf on mgmt0 interface? [no] yes

    Step 4: Default gateway? [192.168.10.1]

    Step 5: Primary DNS server?

    Step 6: Domain name?

    Step 7: Enable IPv6? [yes] yes

    Step 8: Enable IPv6 autoconfig (SLAAC) on mgmt0 interface? [no] no

    Step 9: Update time? [yyyy/mm/dd hh:mm:ss]

    Step 10: Admin password (Enter to leave unchanged)?

    You have entered the following information:

    Hostname: switch-112126

    Use DHCP on mgmt0 interface: no

    Use zeroconf on mgmt0 interface: yes

    Default gateway: 192.168.10.1

    Primary DNS server:

    Domain name:

    Enable IPv6: yes

    Enable IPv6 autoconfig (SLAAC) on mgmt0 interface: yes

    Update time: yyyy/mm/dd hh:mm:ss

    Enable password hardening: yes

    Admin password (Enter to leave unchanged): (unchanged)

    To change an answer, enter the step number to return to.

    Otherwise hit <enter> to save changes and exit.

    Choice:

    Configuration changes saved.

    To return to the wizard from the CLI, enter the “configuration jump-start”

    command from configure mode. Launching CLI...

    <hostname> [standalone: master] >

  6. Check the mgmt0 interface configuration before attempting a remote (for example, SSH) connection to the switch. Specifically, verify the existence of an IP address.

    Copy
    Copied!
                

    switch # show interfaces mgmt0   Interface mgmt0 status: Comment : Admin up : yes Link up : yes DHCP running : yes IP address : 10.12.67.34 Netmask : 255.255.0.0 IPv6 enabled : yes Autoconf enabled: no Autoconf route : yes Autoconf privacy: no DHCPv6 running : no IPv6 addresses : 1   IPv6 address: fe80::268a:7ff:fe53:3d8e/64   Speed : 1000Mb/s (auto) Duplex : full (auto) Interface type : ethernet Interface source: physical MTU : 1500 HW address : 00:02:c9:11:a1:b2   Rx: 11700449 bytes 55753 packets 0 mcast packets 0 discards 0 errors 0 overruns 0 frame   Tx: 5139846 bytes 28452 packets 0 discards 0 errors 0 overruns 0 carrier 0 collisions 1000 queue len

Configuring the Switch with ZTP

Zero-touch Provisioning (ZTP) automates initial configuration of switch systems at boot time. It helps minimize manual operation and reduce customer initial deployment cost.

For more information, please refer to section “Zero-touch Provisioning”.

Rerunning the Wizard

To rerun the wizard:

  1. Enter Config mode. Run:

    Copy
    Copied!
                

    switch > enable switch # config terminal

  2. Rerun the wizard. Run:

    Copy
    Copied!
                

    switch (config) # configuration jump-start

  1. Set up an Ethernet connection between the switch and a local network machine using a standard RJ-45 connector.

  2. Start a remote secured shell (SSH) to the switch using the command “ssh -l <username> <switch ip address>”.

    Copy
    Copied!
                

    rem_mach1 > ssh -l <username> <ip address>

  3. Log into the switch (default username is admin, password admin).

  4. Read and accept the EULA when prompted.

  5. Once the following prompt appears, the system is ready to use.

    Copy
    Copied!
                

    NVIDIA Onyx Switch Management   Password: Last login: <time> from <ip-address>     NVIDIA Switch Please read and accept the End User License Agreement located at: https://www.mellanox.com/related-docs/prod_management_software/MLNX_Onyx_EULA.pdf switch >

To start a WebUI connection to the switch platform, follow the steps below:

Warning

WebUI access is enabled by default. To disable web access, run the command “no web http enable” or “no web https enable” on the CLI.

  1. Set up an Ethernet connection between the switch and a local network machine using a standard RJ-45 connector.

  2. Open a web browser that is Firefox, Chrome, Internet Explorer, or Safari.

    Warning

    Make sure the screen resolution is set to 1024*768 or higher.

    Warning

    In order to access WebUI through Sarafi 5.3, enable http:

    Copy
    Copied!
                

    no web https ssl secure-cookie enable web http enable

  3. Type the IP address of the switch or its DNS name in the following format: https://<switch_IP_address>.

  4. Log into the switch (default user name is admin, password admin).

    image2021-7-28_16-3-40-version-1-modificationdate-1709536690447-api-v2.png

  5. Read and accept the EULA, if prompted.
    The prompt will only occur if the switch has never been accessed through the CLI before.

    image2021-7-28_16-4-39-version-1-modificationdate-1709536689997-api-v2.png

  6. The Welcome popup appears. After reading through the content, click OK to continue.
    To reach the OS documentation, click on the links under the Documentation heading.
    The link under What’s New takes leads to the Changes and New Features section of the switch OS Release Notes. You may also tick the box to not show this popup again. To see this window again, click “Product Documents” on the upper right corner of the WebUI.

    image2021-7-28_16-5-29-version-1-modificationdate-1709536689633-api-v2.png

  7. A default status summary is displayed.

    image2021-7-28_16-7-54-version-1-modificationdate-1709536688963-api-v2.png

Zero-Touch Provisioning (ZTP) automates initial configuration of switch systems at boot time. It helps minimize manual operation and reduce customer initial deployment cost. ZTP allows for automatic upgrade of the switch with a specified OS image, setting up initial configuration database, and to load and run a container image file.

The initial configuration is applied using a regular text file. The user can create such a configuration file by editing the output of a “show running-config” command.

Warning

Only a textual configuration file is supported.

The user-defined docker image can be used by customers to run their own applications in a sandbox on their platform. They can therefore also be used for automating initial configuration.

Warning

Only one docker container can be launched in ZTP.

Running DHCP-ZTP

There is no explicit command to enable ZTP. It is enabled by default. Disabling it is performed by a user-initiated configuration save (using the command “configuration write”). The only way to re-enable ZTP is to run a “reset factory” command, clearing the configuration of the switch and rebooting the system.

ZTP is based on DHCP. For ZTP to work, the software enables DHCP by default on all its management interfaces. The switch OS requests option 66 (tftp-server-name) and 67 (bootfile-name) from the DHCPv4 server or option 58 (bootfile-url) from the DHCPv6 server, and waits for the DHCP responses containing file URLs. The DHCP server must be configured to send back the URLs for the software image, configuration file, and docker container image via these two options. Option 66 would contain the URL prefix to the location of the files, option 67 would contain the name of files, and option 58 would contain the complete URLs of files. The format of these two options is a string list separated by commas. The list items are placed in a fixed order:

Copy
Copied!
            

<image file>, <config file>, <docker container file>

The item value can be empty, but the comma shall not be omitted.

To have DHCP server discern the proper files based on switch-specific information, the OS must provide identifying information for the server to classify the switches. In addition, the OS attaches option 43 (vendor-specific information) and option 60 (vendor class identifier) in DHCPv4 requests and option 17 (vendor-opts) in DHCPv6. Option 60 is set as string “Mellanox” and options 17 and 43 contain the following specific sub-options:

  • System Model

  • Chassis Part Number

  • Chassis Serial Number

  • Management MAC

  • System Profile

  • NVIDIA Onyx Release Version

The corresponding subtypes respectively are defined as:

Copy
Copied!
            

DHCP_VENDOR_ENCAPSULATED_SUBOPTION_TLV_TYPE_MODEL 1 DHCP_VENDOR_ENCAPSULATED_SUBOPTION_TLV_TYPE_PARTNUM 2 DHCP_VENDOR_ENCAPSULATED_SUBOPTION_TLV_TYPE_SERIAL 3 DHCP_VENDOR_ENCAPSULATED_SUBOPTION_TLV_TYPE_MAC 4 DHCP_VENDOR_ENCAPSULATED_SUBOPTION_TLV_TYPE_PROFILE 5 DHCP_VENDOR_ENCAPSULATED_SUBOPTION_TLV_TYPE_RELEASE 6

Upon receiving such DHCP requests from a client, the server should be able to map the switch-specific information to the target file URLs according to predefined rules.

Once the OS receives the URLs from the DHCP server, it executes ZTP as follows:

  1. If the software image URL is not specified, this step is skipped. Otherwise:

    1. Perform disk space cleanup if necessary and fetch the image if it does not exist locally

    2. Resolve the image version:

    3. If it is already installed on active partition, proceed to step 2

    4. If it is installed on a standby partition, switch partition and reboot

    5. If it is not installed locally, install it and switch to the new image and then reboot

    6. If a reboot occurs, ZTP performs step 1 again and no image upgrade will occur

  2. If configuration file URL is not specified, skip this step. Otherwise:

    1. Fetch the configuration file

    2. Apply the configuration file

  3. Skip these steps if a docker image file URL is not specified. Otherwise:

    1. Fetch the docker image file

    2. Load the docker image

    3. Clean up the docker images with the same name and different tag.

    4. Start the container based on the image

    5. Remove the downloaded docker image file

Warning

While performing file transfer via HTTP, the same information as DHCP option 43 is expected to be carried in a HTTP GET request. This switch software supports the following proprietary HTTP headers:

  • MlnxSysProfile

  • MlnxMgmtMac

  • MlnxSerialNumber

  • MlnxModelName

  • MlnxPartNumber

  • MlnxReleaseVersion

If some sort of failure occurs, the switch waits a random number of seconds between 1 and 20 and reattempts the operation. The switch attempts this up to 10 times.
ZTP progress is printed to terminals including console and active SSH sessions.

ZTP and OS Upgrade

Software upgrade from non-ZTP versions to ZTP versions and vice versa is supported. When upgrading from a non-ZTP version, ZTP is disabled because ZTP is always assumed to start with an empty configuration, otherwise the final configuration becomes a mixture of the existing configuration from the stored database and new configuration from the server and hence not deterministic.

DHCPv4 Configuration Example

The following is a URL configuration example for ISC DHCPv4 server:

Copy
Copied!
            

host master { hardware ethernet E4:1D:2D:5B:72:80; fixed-address 3.1.2.13; option tftp-server-name "scp://<user>:<password>@3.1.3.100/ztp/,scp:// <user>:<password>@3.1.3.100/ztp/,scp:// <user>:<password>@3.1.3.100/ztp/"; option bootfile-name "image-X86_64-3.6.4612.img, switch-1.conf, ubuntu.img.gz"; }

DHCPv4 request is made out of the following components:

  • Option 43 (vendor-encapsulated-options) and option 60 (vendor-class-identifier) are added in the DHCPv4 request packet

  • Option 66 (tftp-server-name) and option 67 (bootfile-name) are added in the parameter request list of DHCPv4 request packet

DHCPv6 Configuration Example

The following is a DHCPv6 configuration example:

Copy
Copied!
            

host master { ...... option dhcp6.bootfile-url "scp://<user>:<password>@[2000::1]/ztp/image-X86_64- 3.6.4612.img, scp://<user>:<password>@[2000::1]/ztp/ switch.conf, scp://<user>:<password>@[2000::1]/ztp/ ubuntu.img.gz"; }

Copy
Copied!
            

host master { ...... option dhcp6.bootfile-url "scp://<user>:<password>@[2000::1]/ztp/image-X86_64- 23.01.0100.img, scp://<user>:<password>@[2000::1]/ztp/ switch.conf, scp://<user>:<password>@[2000::1]/ztp/ ubuntu.img.gz"; }

DHCPv6 request is made out of the following components:

  • Option 17 (vendor-opts) is added in the DHCPv6 request packet

  • Option 59 (bootfile-url) is added in the parameter request list of DHCPv6 request packet

ZTP Commands

no zero-touch suppress-write

no zero-touch suppress-write

Disables suppression of configuration write.

Syntax Description

N/A

Default

Enabled

Configuration Mode

config

History

3.6.5000

3.9.2400: Added note

Example

switch (config) # no zero-touch suppress-write

Related Commands

show zero-touch

Notes

  • When ZTP is active, “configuration write” is suppressed because it may interfere with ZTP operation. Therefore, after running “no zero-touch suppress-write” if “configuration write” is performed, then ZTP is disabled as a consequence of the database save.

  • To automatically save the configuration at the end of applying a configuration via ZTP, append the following two commands to the end of the config files. The first command will turn off the ZTP suppress-write, then the configuration write command should work.

    • no zero-touch suppress-write
    • configuration write

zero-touch abort

zero-touch abort

Aborts on-going zero-touch process.

Syntax Description

N/A

Default

Enabled

Configuration Mode

config

History

3.6.5000

Example

switch (config) # zero-touch abort

Zero-touch failed [Zero-touch is aborted by operator]

Zero-touch provisioning will be aborted

Related Commands

show zero-touch

Notes


show zero-touch

show zero-touch

Displays zero-touch status.

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

3.6.5000

Example

switch (config) # show zero-touch

Zero-Touch status:

Active: yes

Status: Waiting for zero-touch start

Suppress-write: no

Configured by zero-touch: no

Configuration changed after zero-touch: no

Related Commands

zero-touch abort

zero-touch suppress-write

Notes

© Copyright 2023, NVIDIA. Last updated on Mar 5, 2024.