Virtualization

NVIDIA Docs Hub NVIDIA Networking Networking Software Switch Software NVIDIA Onyx User Manual v3.10.2202 Virtualization

NVIDIA Onyx allows the user to run their own applications on a Linux docker image embedded in the switch software. The container is a pure application sandbox with resource isolation of both memory and compute from the system code/NOS.

Docker container implementation in the OS enhances its VM support to provide a new set of capabilities:

Network traffic access
Docker containers are implemented in the OS in the same name-space as the network devices allowing the software to send and receive packets from the switch ports by opening a standard Linux socket over the network devices and using an IP address assigned to the device via the legacy management interface (e.g., JSON over HTTP).

Warning

It is recommended to assign a unique port number to the Linux socket to prevent ambiguity of applications between the container and the OS.
Calling the SDK interfaces
Applications running in the docker container are able to implement a set of tools pertaining only to the container such as telemetry features within the network devices. By calling the switch SDK APIs, it can also read data that is not exposed in the OS user interface, or register to receive events that occur in the system (e.g., port up/down).

Warning

The container implementation does not limit the container developer from calling the SDK to set parameters. However this is strongly discouraged as it may cause unexpected system behavior where the OS and the container application manage the same resources.
Query the Linux tables provisioned by OS such as neighbor cache, routing tables, L3 interfaces attributes etc.

Limiting the Container’s Resources

It is possible to configure multiple containers in dockers, however, they would compete for the same memory and compute resources allocated by the switch software (varies for different systems). To ensure system stability and that no random process is killed to free up memory, it is strongly recommended that all resource configurations done in the container utilize OS user interfaces such as JSON/SNMP and take advantage of the internal loopback interface.

Memory Resources Allocation Protocol

The Linux docker supports a hard limit to control memory resource allocation which limits the container to a given amount of user/system memory.

To set the amount of memory allocated to the container, run the following command:

Copy
Copied!

            
            switch (config) # docker start imagename latestver containername init memory 25 label newlabel privileged sdk network docker usb-mount

CPU Resource Allocation Protocol

Containers have unrestricted access to the host machine’s CPU cycles but it is possible to set a number of constraints to limit the containers’ access.

To set up limitations or regulate the containers access to CPU resources, run the following command:

Copy
Copied!

            
            docker start imagename latestver containername init cpus 0.2 label new_label privileged sdk network

Upgrade Ramifications

Changing Docker Storage Driver

As a result of the upgrade, the docker’s storage driver changes, which may cause a few additional changes:

The containers and docker images become inaccessible to the user (the docker process will not run)
The user can reach their old containers after a rollback procedure
The “no docker” command erases all containers and images, including those that were reachable after rollback. Rollbacking after running the “no docker” command may result in failure to create configured containers from unknown images.
The user is advised to execute the “no docker” command at some point in order to clear unused disk space
It is possible to reload the Docker images after upgrade with the command: docker load <image_name>_<image_version>.img.gz
The images are presented with tab-tab after “docker load “ (in cli)
It is also possible to load the images after rollback after "no docker” was execute. That means that containers can be restarted after upgrade/rollback if their images are loaded (with “docker load”).

It is possible to move containers from the current version to the updated one by executing the following steps:

Before upgrade:

Save the container as an image—run the command: “docker commit <container_name> <new_image_name> <new_image_version>”. For example: docker commit my_name my_image my_version. You can see the new image by running: “show docker images”.
Save the image—run the command: “docker save <image_name> <image_version> <file_name-optional>”. For example: docker save my_image my_version.
Upload the image—save the image to a local repository by running: “image upload <image_file_name> <destination_path>”. For example: image upload my_image_my_version.img.gz scp://username:password@fit150/auto/my_dir. The <image_file_name> is presented after clicking tab-tab.

After upgrade:

Start docker—run the “no docker shutdown” command.
Fetch the restored image—run the “image fetch <file_name>” command. For example: image fetch scp://username:password@fit150/auto/my_dir/my_image_my_version.img.gz
Load the image—run the “docker load <image_file_name>” command. For example: docker load my_image_my_version.img.gz
Start a container with the defined image—now that the image with all the content from the container is available in the new environment, start a container with this image. Run the command: “docker start <image_name> <image version> <docker_name> <starting_point>| privileged | label | memory | cpus | usb-mount”. For example: docker start my_image my_version new_container now

Warning

After an upgrade operation there is a need to rerun copy-sdk command (in case in use).

Additional Reading and Use Cases

For more information about this feature and its potential applications, please refer to the following community post:

HowTo Deploy Docker Container with DHCP Service

Docker Containers Commands

docker

	docker [vrf <vrf-name>] [logging-level <log-level>] [force [keep-images]] no docker [vrf <vrf-name>] Enables dockers then enters docker configuration context. The no form of the command disables dockers, removes configuration, and deletes all containers and docker images.
Syntax Description	vrf name—Describes docker daemon VRF context, impacts fetching images and running containers. If "vrf" parameter is not specified, the "default" VRF will be used. force—Restarts docker using past VRF context even if it was already enabled in other VRF context. keep-images—Will not remove docker images while switching VRF context. log-level—logging-level for docker. Possible levels: debug error, fatal info, warn
Default	N/A
Configuration Mode	config
History	3.6.2940 3.9.2000—Added VRF option 3.9.2300—Added log-level and keep-images option
Example	switch (config) # docker
Related Commands
Notes	Logging-level parameter is applicable when docker is "not-started" state or with "force" flag. If not specified, set warning level. Only one configured instance of docker can be in the system. Moving docker between VRFs leads to restarting the docker daemon and a loss of running, cached containers and images. Pulled image can be preserved with the command "docker save".

docker login

	docker login <username> <cleartext password> [server <server address>] Logs in to remote docker repositories.
Syntax Description	username	Username
	cleartext password	There are 2 options to enter password using the above command: In command—cleartext Using interactive shell—entering all needed input except the password will prompt the user to provide a password which will not be visible while typing. (masked by *)
	server	The "server" field is not mandatory. In case it is not present, the docker will try to login into docker hub repository.
Default	N/A
Configuration Mode	config
History	3.9.1600
Example	switch (config) # docker login abcd 1234
Related Commands	show docker login
Notes

docker logout

	docker logout [server <server address>] Logs out from remote server.
Syntax Description	N/A
Default	N/A
Configuration Mode	config
History	3.9.1600
Example	switch (config) # docker logout
Related Commands
Notes	There is no need to provide username as only a single user can be connected to a specific server in any given time

commit

	commit <container-name> <image-name> <image-version> Creates a new image from a running container.
Syntax Description	container-name	Name of the running container to commit (limited to 180 characters)
	image-name	Name of the new image to be created
	image-version	Version of the new image to be created
Default	N/A
Configuration Mode	config docker
History	3.6.2940 3.6.8008: Added new character limitation for container-name
Example	switch (config docker) # commit mycontainer test latest
Related Commands
Notes

copy-sdk

	copy-sdk The command provides access to the switch SDK APIs giving applications running on docker access to the switch hardware.
Syntax Description	N/A
Default	N/A
Configuration Mode	config docker
History	3.6.4110 3.8.1000: Updated notes 3.8.2100: Updated notes
Example	switch (config docker) # copy-sdk
Related Commands
Notes	Copying SDK files to a USB mounted folder is not allowed After an upgrade operation there is a need to rerun copy-sdk command (in case in use).

remove image

	remove image <image-name> <image-version> Removes an image from the Linux docker service.
Syntax Description	image-name	Name of the new image to be deleted
Syntax Description	image-version	Version of the new image to be deleted
Default	N/A
Configuration Mode	config docker
History	3.6.3520 3.6.2940
Example	switch (config docker) # remove image test latest
Related Commands	docker
Notes

exec

	exec <container-name> <program-executable> Executes a program within a running container.
Syntax Description	container-name	Name of the running container to commit (limited to 180 characters)
Syntax Description	program-executable	Linux command
Default	N/A
Configuration Mode	config docker
History	3.6.3520 3.6.2940
Example	switch (config docker) # exec mycontainer "ls -la"
Related Commands	docker
Notes

label

	label <label name> no label <label name> Creates a label which can be used as a shared storage between containers. The no form of the command removes the label.
Syntax Description	N/A
Default	N/A
Configuration Mode	config docker
History	3.6.4110
Example	switch (config docker) # label new_label
Related Commands
Notes

load

	load <image-name> Loads an image from a TAR archive.
Syntax Description	image-name	Name of the TAR image to be loaded
Default	N/A
Configuration Mode	config docker
History	3.6.2940
Example	switch (config docker) # load test
Related Commands	docker
Notes

pull

	pull <image-name>[:<version>] Pulls a docker image from a docker repository.
Syntax Description	image-name	Image name Format: Name:Version If only “Name” is provided, “version” defaults to latest
Default	N/A
Configuration Mode	config docker
History	3.6.2940
Example	switch (config docker) # pull test Using default tag: latest latest: Pulling from library/test 45a2e645736c: Pull complete Digest: sha256:c577af3197aacedf79c5a204cd7f493c8e07ffbce7f88f7600bf19c688c38799 Status: Downloaded newer image for test:latest
Related Commands	docker
Notes

save

	save <image-name> <image-version> <filename> Saves an image to a TAR archive.
Syntax Description	image-name	Image name
	image-version	Image version
	filename	Name of the file in which to save the image
Default	N/A
Configuration Mode	config docker
History	3.6.2940 3.6.8008: Updated command syntax
Example	switch (config docker) # save busybox latest my_image Saving and compressing image: busybox version: latest this could take a while... switch (config docker) #
Related Commands	docker docker load
Notes	After the file is created, the filename gets appended a *.gz suffix.

shutdown

	shutdown no shutdown Stops all docker containers, and deletes all non-auto containers. The no form of the command enables the docker Linux service and runs all configured auto-start containers
Syntax Description	N/A
Default	N/A
Configuration Mode	config docker
History	3.6.2940
Example	switch (config docker) # no shutdown
Related Commands	docker
Notes

start

	start <image-name> <image-version> <container-name> <starting-point> [privileged {network \| sdk}] [cpus <max-cpu-resources>] [memory <max-memory>] [usb-mount] [host-trust [user <username>]] [logging-facility <logging-facility-level>] [user-env <env-string>] no start <container-name> Starts a new container from an image. The no form of the command stops a running docker container.
Syntax Description	image-name	Name of the new image to start.
	image-version	Version of the image to start.
	container-name	Name of the running container to commit (limited to 180 characters).
	privileged	network—adds network privileges to the container (--privilege flag) sdk—adds required mounts to use the switch SDK from the container
	starting-point	init—persistent, start the container after boot, when system initialization is done data-path-ready—persistent, start the container after boot, when data-path is ready to be configured now—start the container now, this is not persistent now-and-data-path-ready—starts the container now and after boot, when data-path is ready to be configured now-and-init—starts the container now and after boot, when system configuration is done ptp-ready—persistent, start the container after boot, when protocol PTP is ready to be configured
	cpus	Sets how much of the available CPU resources a container can use (e.g., “cpus 1.5” guarantees at most one and a half of the available CPUs for the container).
	memory	Sets the maximum amount of memory the container can use in MB. The minimum amount of memory to configure is 4MB.
	usb-mount	Enables USB mount to the docker container.
	host-trust	Allows SSH operation from within the container to localhost without the need to supply password.
	logging-facility-level	Available Parameters: auth, authpriv daemon, ftp, kern, local0, local1, local2, local3, local4, local5, local6, local7, lpr, mail, news, syslog, user, uucp
	env-string	Up to 16 user-defined environment variables. User-defined environment variable are separated by a comma (e.g., key1=value1,key2=value2)
Default	N/A
Configuration Mode	config docker
History	3.6.2940 3.6.3520: Added “privileged” parameter 3.6.8008: Added the options “now-and-data-path-ready” and “now-and-init”, new character limitation for container-name, and updated the description of the parameter “memory” 3.7.0000—Added “ptp-ready” option 3.8.1000; Updated syntax description 3.9.2000: Added host-trust option which adds support for SSH operation from within the container to localhost without the need to supply password (when activating host-trust without supplying user, user admin will be used). 3.9.2300: Added logging-facility and user-env options
Example	switch (config docker) # start centos latest test now Starting docker container. Please wait (this can take a minute)... switch (config) # docker start imagename latestver containername init cpus 0.2 memory 25
Related Commands	docker
Notes	The no form of the command removes the container if it is not persistent. If trust is set and username is not given, user admin will be chosen by default.

image upload

	image upload <filename> [vrf <vrf-name>] <upload_url> Uploads an image file to a remote host.
Syntax Description	filename	Name of file
	vrf-name—Describes VRF context that should be used for this transfer. If not specified, the “default” VRF is used.
	upload_url	FTP, TFTP, SCP and SFTP are supported (e.g., scp://username[:password]@hostname-or-ip/path/filename)
Default	N/A
Configuration Mode	config
History	3.6.29403.9.2000—Added VRF option
Example	switch (config) # image upload centos.img.gz scp://username:password@192.168.10.125/var/www/html/<image_name>
Related Commands
Notes

file image upload

	file image upload <filename> [vrf <vrf-name>] <upload_url> Uploads a file to a remote host.
Syntax Description	filename	Name of file
	vrf-name—Describes VRF context that should be used for this transfer. If not specified, the “default” VRF is used.
	upload_url	FTP, TFTP, SCP and SFTP are supported (e.g., scp://username[:password]@hostname/path/filename)
Default	N/A
Configuration Mode	config
History	3.6.29403.9.2000—Added VRF option
Example	switch (config) # file image upload centos.img.gz scp://username:password@192.168.10.125/var/www/html/<image_name>
Related Commands
Notes

show docker

	show docker Displays docker running state and VRF context.
Syntax Description	N/A
Default	N/A
Configuration Mode	Any command mode
History	3.9.2000 3.9.2300—Added new output example
Example	switch (config) # show docker Dockers state: started Docker hub VRF: default Docker log-level: warn
Related Commands
Notes

show docker containers

	show docker containers <container_name> Displays set parameters on containers already running, and containers planned to run in the future.
Syntax Description	N/A
Default	N/A
Configuration Mode	Any command mode
History	3.6.8008 3.8.1000: Updated example 3.9.2000: Updated example, adding host-trust option 3.9.2300: Updated example, adding "user-defined variables" and "log-facility" fields
Example	switch (config) # show docker containers cont_example: image : busybox version : latest status : running start point : data-path-ready cpu limit : 0.2 memory limit: 10m labels : - privileges : network, sdk usb mount : enabled host trust : admin log-facility: kern user-defined variables: name1: value1 name2: value2 another_container: image : busybox version : latest status : - start point : init cpu limit : 0.2 memory limit: 10m labels : my_label privileges : network, sdk usb mount : disabled host trust : admin log-facility: kern user-defined variables: name1: value1 name2: value2 OS_SYSTEM_TYPE : MSN2410 OS_VERSION : 3.9.2300 3OS_DOCKERD_VRF_CONTEXT : vrf-default OS_DOCKERD_LINUX_VRF_CONTEXT: vrf_vrf-default switch (config) # show docker containers cont_example cont_example: image : busybox version : latest status : running start point : data-path-ready cpu limit : 0.2 memory limit: 10m labels : - privileges : network, sdk usb mount : enabled host trust : admin log-facility: kern user-defined variables: name1: value1 name2: value2 OS_SYSTEM_TYPE : MSN2410 OS_VERSION : 3.9.2300 OS_DOCKERD_VRF_CONTEXT : vrf-default OS_DOCKERD_LINUX_VRF_CONTEXT: vrf_vrf-default
Related Commands
Notes	If a container is already started, the status field displays its current status If a container is configured to run on the next boot, the start point field displays when it will start If there is a mismatch between the configuration of a running container and its next-boot configuration, two entries for the container are shown with both of the configurations For running containers, environment variables that are automatically passed to docker container are revealed (i.e., OS_SYSTEM_TYPE, OS_VERSION, OS_DOCKERD_VRF_CONTEXT, OS_DOCKERD_LINUX_VRF_CONTEXT) If no user-defined variables were configured, "user-defined variables" field is hidden

show docker images

	show docker images Display docker images.
Syntax Description	N/A
Default	N/A
Configuration Mode	Any command mode
History	3.6.3520 3.6.2940: Updated example
Example
switch (config) # show docker images ------------------------------------------------------------- Image Version Created Size ------------------------------------------------------------- ubuntu latest Less than a secon 117MB d ago ubuntu-sdk v1 41 seconds ago 215MB
Related Commands
Notes

show docker ps

	show docker ps Display docker containers.
Syntax Description	N/A
Default	N/A
Configuration Mode	Any command mode
History	3.6.3520 3.6.2940: Updated example
Example
switch (config) # show docker ps --------------------------------------------------------------------------------- Container Image:Version Created Status --------------------------------------------------------------------------------- my_ubuntu_app ubuntu:latest 56 seconds ago Up 50 seconds
Related Commands
Notes	This command is available only after Linux dockers are enabled (“no dockers shutdown”)

show docker labels

	show docker labels Displays docker labels.
Syntax Description	N/A
Default	N/A
Configuration Mode	Any command mode
History	3.6.4110
Example	switch (config) # show docker labels Storage label : label_name1 configured containers list : cont_name2 active containers list : cont_name1 Storage label : label_name2
Related Commands
Notes

show docker login

	show docker login Displays docker login.
Syntax Description	N/A
Default	N/A
Configuration Mode	Any command mode
History	3.9.1600
Example	switch (config) # show docker login Servers: https://index.docker.io/v1/ nvcr.io
Related Commands	docker login
Notes

show docker stats

	show docker stats [<name>] Displays Linux docker statistics.
Syntax Description	name	Docker whose stats to display
Default	N/A
Configuration Mode	Any command mode
History	3.6.8008 2.9.2300: Added example
Example	switch (config) # show docker stats ---------------------------------------------------------------------------------------- Container CPU % Memory Memory Memory % Block Block Pids Usage Limit IN OUT ---------------------------------------------------------------------------------------- container1 0.00% 952K 1000M 0.09% 0B 0B 1
Related Commands
Notes	This command is available only after Linux dockers are enabled (“no dockers shutdown”)

On This Page