Helm Chart Customization Options

There are various customizations you can do to tailor the deployment of the Network Operator to your cluster needs. You can find those below.

• General Parameters

  • ImagePullSecrets customization

• NFD labels

• SR-IOV Network Operator

• Container Resources

• MLNX_OFED Driver

• MLNX_OFED Driver Environment Variables

• RDMA Shared Device Plugin

• RDMA Device Plugin Resource Configurations

• SR-IOV Network Device Plugin

• SR-IOV Network Device Plugin Resource Configuration

• IB Kubernetes

• UFM Secret

• Secondary Network

• CNI Plugin

• Multus CNI

• IPoIB CNI

• IPAM CNI Plugin

• NVIDIA IPAM Plugin

• NVIDIA NIC Feature Discovery

• DOCA Telemetry Service

• Helm customization file

ImagePullSecrets customization

To provide imagePullSecrets` object references, you need to specify them using a following structure:

            

            
imagePullSecrets:
  - image-pull-secret1
  - image-pull-secret2
        

The NFD labels required by the Network Operator and GPU Operator:

SR-IOV Network Operator Helm chart customization options can be found here. Following is a list of overriden values by NVIDIA Operator Helm Chart:

Optional requests and limits can be configured for each container of the sub-resources deployed by the Network Operator by setting the parameter containerResources.

For example:

            

            
containerResources:
- name: "mofed-container"
  requests:
    cpu: "200m"
    memory: "150Mi"
  limits:
    cpu: "300m"
    memory: "300Mi"
        

The following are special environment variables supported by the MLNX_OFED container to configure its behavior:

In addition, it is possible to specify any environment variables to be exposed to the MLNX_OFED container, such as the standard “HTTP_PROXY”, “HTTPS_PROXY”, “NO_PROXY”.

To set these variables, change them into Helm values. For example:

            

            
ofedDriver:
  env:
  - name: RESTORE_DRIVER_ON_POD_TERMINATION
    value: "true"
  - name: UNLOAD_STORAGE_MODULES
    value: "true"
  - name: CREATE_IFNAMES_UDEV
    value: "true"
        

The variables can also be configured directly via the NicClusterPolicy CRD.

These configurations consist of a list of RDMA resources, each with a name and a selector of RDMA capable network devices to be associated with the resource. Refer to RDMA Shared Device Plugin Selectors for supported selectors.

            

            
resources:
- name: rdma_shared_device_a
  vendors: [15b3]
  deviceIDs: [1017]
  ifNames: [enp5s0f0]
  rdmaHcaMax: 63
- name: rdma_shared_device_b
  vendors: [15b3]
  deviceIDs: [1017]
  ifNames: [ib0, ib1]
  rdmaHcaMax: 63
        

Consists of a list of RDMA resources, each with a name and a selector of RDMA capable network devices to be associated with the resource. Refer to SR-IOV Network Device Plugin Selectors for supported selectors.

            

            
resources:
- name: hostdev
  vendors: [15b3]
- name: ethernet_rdma
  vendors: [15b3]
  linkTypes: [ether]
- name: sriov_rdma
  vendors: [15b3]
  devices: [1018]
  drivers: [mlx5_ib]
        

ib-kubernetes provides a daemon that works in conjunction with the SR-IOV Network Device Plugin. It acts on Kubernetes pod object changes (Create/Update/Delete), reading the pod’s network annotation, fetching its corresponding network CRD and reading the PKey. This is done in order to add the newly generated GUID or the predefined GUID in the GUID field of the CRD cni-args to that PKey for pods with mellanox.infiniband.app annotation.

IB Kubernetes must access NVIDIA UFM in order to manage pods’ GUIDs. To provide its credentials, the secret of the following format should be deployed in advance:

            

            
apiVersion: v1
kind: Secret
metadata:
  name: ib-kubernetes-ufm-secret
  namespace: nvidia-network-operator
stringData:
  UFM_USERNAME: "admin"
  UFM_PASSWORD: "123456"
  UFM_ADDRESS: "ufm-hostname"
  UFM_HTTP_SCHEMA: ""
  UFM_PORT: ""
data:
  UFM_CERTIFICATE: ""
        

Specifies components to deploy in order to facilitate a secondary network in Kubernetes. It consists of the following optionally deployed components:

• Multus-CNI: Delegate CNI plugin to support secondary networks in Kubernetes

• CNI plugins: Currently only containernetworking-plugins is supported

• IPAM CNI: Currently only Whereabout IPAM CNI is supported as a part of the secondaryNetwork section. NVIDIA-IPAM is configured separately.

• IPoIB CNI: Allows the user to create IPoIB child link and move it to the pod

NVIDIA IPAM Plugin is recommended to be used on large-scale deployments of the NVIDIA Network Operator.

NVIDIA NIC Feature Discovery leverages Node Feature Discovery to advertise NIC specific labels on K8s Node objects.

DOCA Telemetry Service exports metrics from NVIDIA NICs on K8s Nodes.