NVIDIA Onyx User Manual v3.10.3100
NVIDIA MLNX-GW User Manual for NVIDIA Skyway Appliance v8.2.2200

Virtualization

NVIDIA Onyx allows the user to run their own applications on a Linux docker image embedded in the switch software. The container is a pure application sandbox with resource isolation of both memory and compute from the system code/NOS.

Docker container implementation in the OS enhances its VM support to provide a new set of capabilities:

  • Network traffic access
    Docker containers are implemented in the OS in the same name-space as the network devices allowing the software to send and receive packets from the switch ports by opening a standard Linux socket over the network devices and using an IP address assigned to the device via the legacy management interface (e.g., JSON over HTTP).

    Warning

    It is recommended to assign a unique port number to the Linux socket to prevent ambiguity of applications between the container and the OS.

  • Calling the SDK interfaces
    Applications running in the docker container are able to implement a set of tools pertaining only to the container such as telemetry features within the network devices. By calling the switch SDK APIs, it can also read data that is not exposed in the OS user interface, or register to receive events that occur in the system (e.g., port up/down).

    Warning

    The container implementation does not limit the container developer from calling the SDK to set parameters. However this is strongly discouraged as it may cause unexpected system behavior where the OS and the container application manage the same resources.

  • Query the Linux tables provisioned by OS such as neighbor cache, routing tables, L3 interfaces attributes etc.

It is possible to configure multiple containers in dockers, however, they would compete for the same memory and compute resources allocated by the switch software (varies for different systems). To ensure system stability and that no random process is killed to free up memory, it is strongly recommended that all resource configurations done in the container utilize OS user interfaces such as JSON/SNMP and take advantage of the internal loopback interface.

Memory Resources Allocation Protocol

The Linux docker supports a hard limit to control memory resource allocation which limits the container to a given amount of user/system memory.

To set the amount of memory allocated to the container, run the following command:

Copy
Copied!
            

switch (config) # docker start imagename latestver containername init memory 25 label newlabel privileged sdk network docker usb-mount

CPU Resource Allocation Protocol

Containers have unrestricted access to the host machine’s CPU cycles but it is possible to set a number of constraints to limit the containers’ access.

To set up limitations or regulate the containers access to CPU resources, run the following command:

Copy
Copied!
            

docker start imagename latestver containername init cpus 0.2 label new_label privileged sdk network

Changing Docker Storage Driver

As a result of the upgrade, the docker’s storage driver changes, which may cause a few additional changes:

  • The containers and docker images become inaccessible to the user (the docker process will not run)

  • The user can reach their old containers after a rollback procedure

  • The “no docker” command erases all containers and images, including those that were reachable after rollback. Rollbacking after running the “no docker” command may result in failure to create configured containers from unknown images.

  • The user is advised to execute the “no docker” command at some point in order to clear unused disk space

  • It is possible to reload the Docker images after upgrade with the command: docker load <image_name>_<image_version>.img.gz

  • The images are presented with tab-tab after “docker load “ (in cli)

  • It is also possible to load the images after rollback after "no docker” was execute. That means that containers can be restarted after upgrade/rollback if their images are loaded (with “docker load”).

It is possible to move containers from the current version to the updated one by executing the following steps:

Before upgrade:

  1. Save the container as an image—run the command: “docker commit <container_name> <new_image_name> <new_image_version>”. For example: docker commit my_name my_image my_version. You can see the new image by running: “show docker images”.

  2. Save the image—run the command: “docker save <image_name> <image_version> <file_name-optional>”. For example: docker save my_image my_version.

  3. Upload the image—save the image to a local repository by running: “image upload <image_file_name> <destination_path>”. For example: image upload my_image_my_version.img.gz scp://username:password@fit150/auto/my_dir. The <image_file_name> is presented after clicking tab-tab.

After upgrade:

  1. Start docker—run the “no docker shutdown” command.

  2. Fetch the restored image—run the “image fetch <file_name>” command. For example: image fetch scp://username:password@fit150/auto/my_dir/my_image_my_version.img.gz

  3. Load the image—run the “docker load <image_file_name>” command. For example: docker load my_image_my_version.img.gz

  4. Start a container with the defined image—now that the image with all the content from the container is available in the new environment, start a container with this image. Run the command: “docker start <image_name> <image version> <docker_name> <starting_point>| privileged | label | memory | cpus | usb-mount”. For example: docker start my_image my_version new_container now

Warning

After an upgrade operation there is a need to rerun copy-sdk command (in case in use).

For more information about this feature and its potential applications, please refer to the following community post:

docker

docker [vrf <vrf-name>] [logging-level <log-level>] [force [keep-images]]
no docker [vrf <vrf-name>]

Enables dockers then enters docker configuration context.
The no form of the command disables dockers, removes configuration, and deletes all containers and docker images.

Syntax Description

  • vrf name—Describes docker daemon VRF context, impacts fetching images and running containers. If "vrf" parameter is not specified, the "default" VRF will be used.

  • force—Restarts docker using past VRF context even if it was already enabled in other VRF context.

  • keep-images—Will not remove docker images while switching VRF context.

  • log-level—logging-level for docker. Possible levels: debug error, fatal info, warn

Default

N/A

Configuration Mode

config

History

3.6.2940
3.9.2000—Added VRF option

3.9.2300—Added log-level
and keep-images

option

Example

switch (config) # docker

Related Commands

Notes

  • Logging-level parameter is applicable when docker is "not-started" state or with "force" flag. If not specified, set warning level.

  • Only one configured instance of docker can be in the system. Moving docker between VRFs leads to restarting the docker daemon and a loss of running, cached containers and images. Pulled image can be preserved with the command "docker save".

docker login

docker login <username> <cleartext password> [server <server address>]

Logs in to remote docker repositories.

Syntax Description

username

Username

cleartext password

There are 2 options to enter password using the above command:

  1. In command—cleartext

  2. Using interactive shell—entering all needed input except the password will prompt the user to provide a password which will not be visible while typing. (masked by *)

server

The "server" field is not mandatory. In case it is not present, the docker will try to login into docker hub repository.

Default

N/A

Configuration Mode

config

History

3.9.1600

Example

switch (config) # docker login abcd 1234

Related Commands

show docker login

Notes

docker logout

docker logout [server <server address>]

Logs out from remote server.

Syntax Description

N/A

Default

N/A

Configuration Mode

config

History

3.9.1600

Example

switch (config) # docker logout

Related Commands

Notes

  • There is no need to provide username as only a single user can be connected to a specific server in any given time

commit

commit <container-name> <image-name> <image-version>

Creates a new image from a running container.

Syntax Description

container-name

Name of the running container to commit (limited to 180 characters)

image-name

Name of the new image to be created

image-version

Version of the new image to be created

Default

N/A

Configuration Mode

config docker

History

3.6.2940
3.6.8008: Added new character limitation for container-name

Example

switch (config docker) # commit mycontainer test latest

Related Commands

Notes

copy-sdk

copy-sdk

The command provides access to the switch SDK APIs giving applications running on docker access to the switch hardware.

Syntax Description

N/A

Default

N/A

Configuration Mode

config docker

History

3.6.4110
3.8.1000: Updated notes
3.8.2100: Updated notes

Example

switch (config docker) # copy-sdk

Related Commands

Notes

  • Copying SDK files to a USB mounted folder is not allowed

  • After an upgrade operation there is a need to rerun copy-sdk command (in case in use).

remove image

remove image <image-name> <image-version>

Removes an image from the Linux docker service.

Syntax Description

image-name

Name of the new image to be deleted

image-version

Version of the new image to be deleted

Default

N/A

Configuration Mode

config docker

History

3.6.3520
3.6.2940

Example

switch (config docker) # remove image test latest

Related Commands

docker

Notes

exec

exec <container-name> <program-executable>

Executes a program within a running container.

Syntax Description

container-name

Name of the running container to commit (limited to 180 characters)

program-executable

Linux command

Default

N/A

Configuration Mode

config docker

History

3.6.3520
3.6.2940

Example

switch (config docker) # exec mycontainer "ls -la"

Related Commands

docker

Notes

label

label <label name>
no label <label name>

Creates a label which can be used as a shared storage between containers.
The no form of the command removes the label.

Syntax Description

N/A

Default

N/A

Configuration Mode

config docker

History

3.6.4110

Example

switch (config docker) # label new_label

Related Commands

Notes

load

load <image-name>

Loads an image from a TAR archive.

Syntax Description

image-name

Name of the TAR image to be loaded

Default

N/A

Configuration Mode

config docker

History

3.6.2940

Example

switch (config docker) # load test

Related Commands

docker

Notes

pull

pull <image-name>[:<version>]

Pulls a docker image from a docker repository.

Syntax Description

image-name

Image name
Format: Name:Version
If only “Name” is provided, “version” defaults to latest

Default

N/A

Configuration Mode

config docker

History

3.6.2940

Example

switch (config docker) # pull test
Using default tag: latest
latest: Pulling from library/test
45a2e645736c: Pull complete
Digest: sha256:c577af3197aacedf79c5a204cd7f493c8e07ffbce7f88f7600bf19c688c38799
Status: Downloaded newer image for test:latest

Related Commands

docker

Notes

save

save <image-name> <image-version> <filename>

Saves an image to a TAR archive.

Syntax Description

image-name

Image name

image-version

Image version

filename

Name of the file in which to save the image

Default

N/A

Configuration Mode

config docker

History

3.6.2940
3.6.8008: Updated command syntax

Example

switch (config docker) # save busybox latest my_image

Saving and compressing image: busybox version: latest
this could take a while...

switch (config docker) #

Related Commands

docker
docker load

Notes

After the file is created, the filename gets appended a *.gz suffix.

shutdown

shutdown
no shutdown

Stops all docker containers, and deletes all non-auto containers.
The no form of the command enables the docker Linux service and runs all configured auto-start containers

Syntax Description

N/A

Default

N/A

Configuration Mode

config docker

History

3.6.2940

Example

switch (config docker) # no shutdown

Related Commands

docker

Notes

start

start <image-name> <image-version> <container-name> <starting-point> [privileged {network | sdk}] [cpus <max-cpu-resources>] [memory <max-memory>] [usb-mount] [host-trust [user <username>]] [logging-facility <logging-facility-level>] [user-env <env-string>]
no start <container-name>

Starts a new container from an image.
The no form of the command stops a running docker container.

Syntax Description

image-name

Name of the new image to start.

image-version

Version of the image to start.

container-name

Name of the running container to commit (limited to 180 characters).

privileged

  • network—adds network privileges to the container (--privilege flag)

  • sdk—adds required mounts to use the switch SDK from the container

starting-point

  • init—persistent, start the container after boot, when system initialization is done

  • data-path-ready—persistent, start the container after boot, when data-path is ready to be configured

  • now—start the container now, this is not persistent

  • now-and-data-path-ready—starts the container now and after boot, when data-path is ready to be configured

  • now-and-init—starts the container now and after boot, when system configuration is done

  • ptp-ready—persistent, start the container after boot, when protocol PTP is ready to be configured

cpus

Sets how much of the available CPU resources a container can use (e.g., “cpus 1.5” guarantees at most one and a half of the available CPUs for the container).

memory

Sets the maximum amount of memory the container can use in MB.
The minimum amount of memory to configure is 4MB.

usb-mount

Enables USB mount to the docker container.

host-trust

Allows SSH operation from within the container to localhost without the need to supply password.

logging-facility-level

Available Parameters: auth, authpriv daemon, ftp, kern, local0, local1, local2, local3, local4, local5, local6, local7, lpr, mail, news, syslog, user, uucp

env-string

Up to 16 user-defined environment variables. User-defined environment variable are separated by a comma (e.g., key1=value1,key2=value2)

Default

N/A

Configuration Mode

config docker

History

3.6.2940
3.6.3520: Added “privileged” parameter
3.6.8008: Added the options “now-and-data-path-ready” and “now-and-init”, new character limitation for container-name, and updated the description of the parameter “memory”
3.7.0000—Added “ptp-ready” option

3.8.1000; Updated syntax description
3.9.2000: Added host-trust option which adds support for SSH operation from within the container to localhost without the need to supply password (when activating host-trust without supplying user, user admin will be used).
3.9.2300: Added logging-facility and user-env options

Example

switch (config docker) # start centos latest test now

Starting docker container. Please wait (this can take a minute)...

switch (config) # docker start imagename latestver containername init cpus 0.2 memory 25

Related Commands

docker

Notes

  • The no form of the command removes the container if it is not persistent.

  • If trust is set and username is not given, user admin will be chosen by default.

image upload

image upload <filename> [vrf <vrf-name>] <upload_url>

Uploads an image file to a remote host.

Syntax Description

filename

Name of file

vrf-name—Describes VRF context that should be used for this transfer. If not specified, the “default” VRF is used.

upload_url

FTP, TFTP, SCP and SFTP are supported (e.g., scp://username[:password]@hostname-or-ip/path/filename)

Default

N/A

Configuration Mode

config

History

3.6.29403.9.2000—Added VRF option

Example

switch (config) # image upload centos.img.gz scp://username:password@192.168.10.125/var/www/html/<image_name>

Related Commands

Notes

file image upload

file image upload <filename> [vrf <vrf-name>] <upload_url>

Uploads a file to a remote host.

Syntax Description

filename

Name of file

vrf-name—Describes VRF context that should be used for this transfer. If not specified, the “default” VRF is used.

upload_url

FTP, TFTP, SCP and SFTP are supported (e.g., scp://username[:password]@hostname/path/filename)

Default

N/A

Configuration Mode

config

History

3.6.29403.9.2000—Added VRF option

Example

switch (config) # file image upload centos.img.gz scp://username:password@192.168.10.125/var/www/html/<image_name>

Related Commands

Notes

show docker

show docker

Displays docker running state and VRF context.

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

3.9.2000
3.9.2300—Added new output example

Example

switch (config) # show docker
Dockers state: started
Docker hub VRF: default
Docker log-level: warn

Related Commands


Notes

show docker containers

show docker containers <container_name>

Displays set parameters on containers already
running, and containers planned to run in the future.

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

3.6.8008
3.8.1000: Updated example
3.9.2000: Updated example, adding host-trust option
3.9.2300: Updated example, adding "user-defined variables" and "log-facility" fields

Example

switch (config) # show docker containers
cont_example:
image : busybox
version : latest
status : running
start point : data-path-ready
cpu limit : 0.2
memory limit: 10m
labels : -
privileges : network, sdk
usb mount : enabled
host trust : admin
log-facility: kern
user-defined variables:
name1: value1
name2: value2

another_container:
image : busybox
version : latest
status : -
start point : init
cpu limit : 0.2
memory limit: 10m
labels : my_label
privileges : network, sdk
usb mount : disabled
host trust : admin
log-facility: kern
user-defined variables:
name1: value1
name2: value2

 OS_SYSTEM_TYPE : MSN2410
OS_VERSION : 3.9.2300

3OS_DOCKERD_VRF_CONTEXT : vrf-default
OS_DOCKERD_LINUX_VRF_CONTEXT: vrf_vrf-default

switch (config) # show docker containers cont_example
cont_example:
image : busybox
version : latest
status : running
start point : data-path-ready
cpu limit : 0.2
memory limit: 10m
labels : -
privileges : network, sdk
usb mount : enabled
host trust : admin
log-facility: kern

user-defined variables:
name1: value1
name2: value2


 OS_SYSTEM_TYPE : MSN2410
 OS_VERSION : 3.9.2300

  
OS_DOCKERD_VRF_CONTEXT : vrf-default
OS_DOCKERD_LINUX_VRF_CONTEXT: vrf_vrf-default

Related Commands

Notes

  • If a container is already started, the status field displays its current status

  • If a container is configured to run on the next boot, the start point field displays when it will start

  • If there is a mismatch between the configuration of a running container and its next-boot configuration, two entries for the container are shown with both of the configurations

  • For running containers, environment variables that are automatically passed to docker container are revealed (i.e., OS_SYSTEM_TYPE, OS_VERSION, OS_DOCKERD_VRF_CONTEXT, OS_DOCKERD_LINUX_VRF_CONTEXT)

  • If no user-defined variables were configured, "user-defined variables" field is hidden

show docker images

show docker images

Display docker images.

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

3.6.3520
3.6.2940: Updated example

Example

switch (config) # show docker images
-------------------------------------------------------------
Image Version Created Size
-------------------------------------------------------------
ubuntu latest Less than a secon 117MB
d ago
ubuntu-sdk v1 41 seconds ago 215MB

Related Commands

Notes

show docker ps

show docker ps

Display docker containers.

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

3.6.3520
3.6.2940: Updated example

Example

switch (config) # show docker ps
---------------------------------------------------------------------------------
Container Image:Version Created Status
---------------------------------------------------------------------------------
my_ubuntu_app ubuntu:latest 56 seconds ago Up 50 seconds

Related Commands

Notes

This command is available only after Linux dockers are enabled (“no dockers shutdown”)

show docker labels

show docker labels

Displays docker labels.

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

3.6.4110

Example

switch (config) # show docker labels
Storage label : label_name1
configured containers list : cont_name2
active containers list : cont_name1

Storage label : label_name2

Related Commands

Notes

show docker login

show docker login

Displays docker login.

Syntax Description

N/A

Default

N/A

Configuration Mode

Any command mode

History

3.9.1600

Example

switch (config) # show docker login

Servers:
https://index.docker.io/v1/
nvcr.io

Related Commands

docker login

Notes

show docker stats

show docker stats [<name>]

Displays Linux docker statistics.

Syntax Description

name

Docker whose stats to display

Default

N/A

Configuration Mode

Any command mode

History

3.6.8008
2.9.2300: Added example

Example

switch (config) # show docker stats
----------------------------------------------------------------------------------------
Container CPU % Memory Memory Memory % Block Block Pids
Usage Limit IN OUT
----------------------------------------------------------------------------------------
container1 0.00% 952K 1000M 0.09% 0B 0B 1

Related Commands

Notes

This command is available only after Linux dockers are enabled (“no dockers shutdown”)

© Copyright 2023, NVIDIA. Last updated on May 23, 2023.