NVIDIA BlueField-3 SNAP for NVMe and Virtio-blk v4.1.0
1.0

Appendix - Building SNAP Container with Custom SPDK

The SNAP source package contains the files necessary for building a container with a custom SPDK.

To build the container:

  1. Download and install the SNAP sources package:

    Copy
    Copied!
                

    [dpu] # dpkg -i /path/snap-sources_<version>_arm64.deb

  2. Navigate to the src folder and use it as the development environment:

    Copy
    Copied!
                

    [dpu] # cd /opt/nvidia/nvda_snap/src

  3. Copy the following to the container folder:

    • SNAP source package – required for installing SNAP inside the container

    • Custom SPDK – to container/spdkFor example:

      Copy
      Copied!
                  

      [dpu] # cp /path/snap-sources_<version>_arm64.deb container/ [dpu] # git clone -b v23.01.1 --single-branch --depth 1 --recursive --shallow-submodules https://github.com/spdk/spdk.git container/spdk

  4. Modify the spdk.sh file if necessary as it is used to compile SDPK.

  5. To build the container:

    • For Ubuntu, run:

      Copy
      Copied!
                  

      [dpu] # ./container/build_public.sh --snap-pkg-file=snap-sources_<version>_arm64.deb

    • For CentOS, run:

      Copy
      Copied!
                  

      [dpu] # rpm -i snap-sources-<version>.el8.aarch64.rpm [dpu] # cd /opt/nvidia/nvda_snap/src/ [dpu] # cp /path/snap-sources_<version>_arm64.deb container/ [dpu] # git clone -b v23.01.1 --single-branch --depth 1 --recursive --shallow-submodules https://github.com/spdk/spdk.git container/spdk [dpu] # yum install docker-ce docker-ce-cli [dpu] # ./container/build_public.sh --snap-pkg-file=snap-sources_<version>_arm64.deb

  6. Transfer the created image from the Docker tool to the crictl tool. Run:

    Copy
    Copied!
                

    [dpu] # docker save doca_snap:<version> doca_snap.tar [dpu] # ctr -n=k8s.io images import doca_snap.tar

    Warning

    To transfer the container image to other setups, refer to appendix "Appendix – Deploying Container on Setups Without Internet Connectivity".

  7. To verify the image, run:

    Copy
    Copied!
                

    [DPU] # crictl images IMAGE TAG IMAGE ID SIZE docker.io/library/doca_snap <version> 79c503f0a2bd7 284MB

  8. Edit the image filed in the container/doca_snap.yaml file. Run:

    Copy
    Copied!
                

    image: doca_snap:<version>

  9. Use the YAML file to deploy the container. Run:

    Copy
    Copied!
                

    [dpu] # cp doca_snap.yaml /etc/kubelet.d/

    Warning

© Copyright 2023, NVIDIA. Last updated on Aug 28, 2023.