NVIDIA Fleet Command is a hybrid-cloud platform for securely and remotely deploying, managing, and scaling AI across dozens or thousands of servers or edge devices. Fleet Command provides the capability to easily provision systems at the edge, using a software stack maintained and updated by NVIDIA, and to manage and access the systems and applications deployed on that system.
Azure IoT Edge and Azure IoT Central are fully managed services built on Azure IoT Hub. It allows you to deploy cloud workloads, third-party services or business logic to run on Internet of Things (IoT) edge devices via standard containers and monitor it all from the cloud.
Although Azure IoT focuses on container management, users must create applications that can self-heal or must take the optional step to add Kubernetes. Azure IoT features creating a secure channel to the cloud but not system provisioning, management, and security. Fleet Command provides these features through integration with Azure IoT.
There are three methods of integration:
Using Fleet Command to deploy applications integrated with Azure IoT.
With this method, you use Fleet Command to deploy your applications enabled with Azure IoT but leverage Azure for other cloud services. For example, you can have your application send telemetry data to your Azure environment while Fleet Command manages your applications and systems.
Deploying Azure IoT Edge runtime and managing the Azure IoT Edge node from Fleet Command.
With this method, you use Fleet Command to provision and manage systems and deploy the Azure Edge runtime to connect to the Azure IoT Hub. The connection enables Azure to take over the rest of the application deployment and management process using its runtime.
Deploying Azure IoT Central runtime and managing the Azure IoT Central node from Fleet Command.
With this method, you use Fleet Command to deploy the Azure IoT Edge runtime and connect it to your IoT Central instance via DPS. You can then use IoT Central’s device templates and business logic to deploy and manage the rest of your application and solution directly from your Azure IoT Central instance.
Using these integration methods, Fleet Command can easily and securely provision and manage your systems and applications integrated with Azure IoT or the Azure IoT runtimes. Azure IoT can incorporate and gather telemetry data from the applications or fully manage the rest of the application lifecycle using its runtime.
Before you start your integration, make sure you have the following:
Access to an Azure account with permissions to create Azure IoT Hub or IoT Central resources.
Access to an NVIDIA GPU Cloud (NGC) organization enabled for Fleet Command.
A location and system created in Fleet Command. Consult the Getting Started section for additional information.
A Linux machine configured to run the commands below, with the NGC Command Line Interface (CLI) and Helm installed.
Azure SDK may also be installed for generating certificates (optional).
This document describes three ways to integrate Azure IoT with Fleet Command:
Deploying applications integrated with Azure IoT on Fleet Command.
Deploying Azure IoT and managing Azure IoT Edge node from Fleet Command.
Deploying Azure IoT and managing Azure IoT Central node from Fleet Command.
Visit the NVIDIA GitHub repo for more information, sample applications, and Helm charts.
For the first integration method, applications can be enabled for Azure IoT and then deployed on Fleet Command. Users will use Fleet Command for provisioning and deploying applications but then benefit from the constructs in Azure IoT Hub for receiving telemetry and building web applications. This section explains how to do that.
Preparing Azure IoT
Create an Azure IoT Hub resource on Azure. For more information, refer to Create an IoT hub.
Create an Azure IoT device on IoT Hub to send and receive messages from Fleet Command edge devices. For more information, refer to Register new device in the IoT hub.
Obtain a Primary Connection String to communicate with the IoT Hub from Fleet Command edge devices. Refer to Check device authentication to get a Primary Connection String.
Once you have completed these steps, proceed to the next section.
Preparing an Application
To create an Azure IoT application that will send telemetry to Azure IoT Hub, you need to integrate Azure IoT SDKs with your application. For more information, refer to the links below.
You can find Samples for the Azure IoT Hub Device SDK to help integrate Azure IoT SDK into your application.
Once you integrate Azure IoT SDK into your application, create a Helm chart to deploy on the Fleet Command system. Refer to the Fleet Command Application Development Guide for more information.
It’s recommended to configure the Azure IoT Primary Connection String as a configurable option in the Helm chart. Refer to the Example Application section below.
Example Application
An example application Helm chart is provided in the NVIDIA GitHub repo, which integrates the NVIDIA IVA demo application with Azure IoT. This integrated application aims to send the DeepStream container logs to Azure IoT as telemetry data.
To deploy this application, first upload this Helm chart to your NGC Private Registry, following the instructions here. Then follow the instructions in Adding an Application to add the application to Fleet Command.
You will also want to create an IoT Device instance in Azure IoT corresponding to the application to receive the telemetry data. Note the primary connection string for this device since you’ll reference this later.
Once the application is added, create a deployment in Fleet Command with a Location and System you have already provisioned, selecting the application that you have added to Fleet Command. Enter the Azure IoT Primary connection string in the Application Configuration field for the Deployment, then deploy the application.
This sample application includes a sample script to read the telemetry data from Azure IoT on your local machine. Refer to Azure documentation on how to view telemetry data within Azure IoT.
The second integration method uses Fleet Command for provisioning and managing the Azure IoT Edge node, but then benefit from the constructs in the Azure IoT Hub for managing the application life cycle. This allows users to continue to use their existing Azure resources and applications but simplify systems management at the Edge using Fleet Command.
This section discusses creating a Helm chart for deploying the Azure IoT Edge Runtime on Fleet Command systems, connecting these systems to Azure, and deploying an application from Azure.
Preparing Azure IoT Edge
Create an Azure IoT Hub resource on Azure. For more information, refer to create an IoT Hub.
Create an Azure IoT edge device on IoT Hub to manage the application lifecycle from Fleet Command edge devices. For more information, refer to register new device on Azure IoT Hub.
Obtain a Primary Connection String to communicate with IoT Hub from Fleet Command edge devices. Refer to device authentication to get a Primary Connection String.
Once the above steps have completed successfully, proceed to the next section.
Deploying Azure IoT Runtime on Fleet Command
Once you have a Fleet Command system up and running, follow the steps below.
To add the Azure IoT runtime Helm chart to Fleet Command, first, upload this Helm chart to your NGC Private Registry following the instructions here, then follow the instructions in Adding an Application to add the application to Fleet Command.
Next, follow the Creating a Deployment steps to deploy the Azure IoT Runtime on the Fleet Command system. This will create an Azure IoT Edge node using the Fleet Command system.
During the deployment creation, you need to provide an Azure IoT Primary Connection string in the Application Configuration field of the Deployment, to connect the runtime to Azure IoT Edge.
provisioning:
deviceConnectionString: "HostName=egx-iot.azure-devices.net;DeviceId=fc-node;SharedAccessKey=EOVGtm2b3fFzXXxxXXaaXXXxXaaaXaXxxAxAa="
Once Azure IoT Runtime is deployed on Fleet Command, wait at least five minutes to see the status below in Azure IoT.
Deploy and Manage Azure IoT Application
Now you can deploy an Azure IoT application/module from Azure on the Fleet Command system and manage its lifecycle in Azure IoT. For more information about deploying Azure IoT Applications, refer to Deploy Azure IoT Edge Modules.
Another integration method uses Fleet Command to provision and manage the Azure IoT Central node.
Azure IoT Central focuses on reducing the cost of developing an application, managing, and maintaining Internet of Things (IoT) solutions. IoT Central supports both IoT and IoT Edge devices. It allows you to deploy your cloud workloads and third-party services, or your business logic to run on IoT edge devices via device templates and monitor it all from the cloud. Device templates contain applications and configurations to deploy on devices at the edge. IoT Central also has dashboards, jobs, and the ability to integrate with other Azure and third-party services via REST API.
This section describes how to create a Helm chart for deploying the Azure IoT Central Runtime on Fleet Command systems, connecting these systems to Azure, and deploying an application from Azure.
Preparing Azure IoT Central
Azure IoT Edge devices can connect to IoT Central application using Shared Access Signature (SAS) primary keys or X.509 certificates. This section describes the X.509 certificate-based provisioning of IoT Edge devices from Fleet Command to IoT Central. For more information about device authentication using primary keys, refer to SAS enrollment group in the Azure IoT documentation.
Ensure you have created an Azure IoT Central instance for use with this deployment.
Within IoT Central, create a device enrollment group with X.509 attestation on IoT Central, following the steps described in device authentication of the Azure documentation.
After creating a device enrollment group, you should have corresponding certificates and private keys, which we will incorporate into the Helm chart for system provisioning. For information on creating certificates with the Azure SDK, refer to How to connect devices with X.509 certificates to IoT Central Application.
Once the device enrollment group with certificates is created, move on to the next section.
Preparing Azure IoT Runtime
The following steps are based on Azure documentation for how to Automatically register devices, adapted for deployment from Fleet Command. Using the Microsoft Azure IoT Runtime Helm chart, we will integrate the certificates generated for the device enrollment group created from the previous section, package the Helm chart, and then upload the Helm chart to the NGC Private Registry.
Download the Azure IoT Runtime 0.2.11 Helm Chart that is already available.
NoteBefore this version of the Helm chart is publicly available, download the Helm chart from your own NGC Private Registry to support the X.509 DPS provisioning method.
Run the following commands to include your generated certificates into the Azure IoT Runtime Helm chart.
$ tar xvfz azure-iot-runtime-0.2.11.tgz $ cat <your identity certificate>.pem | base64 > azure-iot-runtime/config/ <your identity certificate>.pem $ cat <your identity certificate key>.pem | base64 > azure-iot-runtime/config/ <your identity certificate key>.pem
Now repackage your customized Azure IoT Runtime Helm chart containing your certificate using the instructions below.
$ cat > azure-iot-runtime/Chart.yaml << 'EOF' apiVersion: v1 appVersion: '1.0' description: A Helm chart for running Azure IoT Edge on Kubernetes name: azure-iot-runtime version: 0.2.12 EOF $ helm package azure-iot-runtime
Next, upload this Helm chart to your NGC Private Registry using the NGC CLI command where
<NGC-API-KEY>is your NGC API Key and<NGC-LaunchPad-Org>is your NGC Organization.$ ngc config set Enter API key [no-apikey]. Choices: [<VALID_APIKEY>, 'no-apikey']: <NGC-API-KEY> Enter CLI output format type [ascii]. Choices: [ascii, csv, json]: Enter org ['ck8h0oxa8dtv']. Choices: ['ck8h0oxa8dtv']: <NGC-LaunchPad-Org> Enter team [no-team]. Choices: ['no-team']: no-team Enter ace [no-ace]. Choices: ['no-ace']: no-ace Successfully saved NGC configuration to /home/nvidia/.ngc/config
You can skip the step below if you’re using the same private registry that previously downloaded the Azure IoT Runtime Helm chart. Otherwise, issue the following command where
<NGC-LaunchPad-Org>is your NGC Organization.$ ngc registry chart create <NGC-LaunchPad-Org>/azure-iot-runtime --short-desc "Azure IoT Runtime Helm Chart" Successfully created chart '<NGC-LaunchPad-Org>/azure-iot-runtime'. -------------------------------------------------- Chart Information Name: azure-iot-runtime Short Description: Azure IoT Runtime Helm Chart Display Name: Team: Publisher: Built By: Labels: Logo: Created Date: 2022-03-22 18:48:36 UTC Updated Date: 2022-03-22 18:48:36 UTC Read Only: False Latest Version ID: Latest Version Size (bytes): Overview: --------------------------------------------------
Run the command below to push the Helm chart to NGC Private Registry, where
<NGC-LaunchPad-Org>is your NGC Organization.$ ngc registry chart push <NGC-LaunchPad-Org>/azure-iot-runtime:0.2.12
Deploying Azure IoT Runtime on Fleet Command
The following instructions assume you have a Fleet Command location and system already up and running.
Once you have uploaded your Helm chart containing your certificates to the NGC Private Registry, follow the instructions in the Adding an Application to add the application to Fleet Command.
Next, follow the Getting Started steps to deploy the Azure IoT Runtime on the Fleet Command system. When creating a Deployment, enter the following in the Application Configuration field.
<ID scope>: Retrieve this value from your IoT Central device connection group.<device_cert_name>and<device_key_name>will be your IoT device certificate and key, which were added to the Helm chart in the steps above.
Once you have entered these details, click Deploy. This will create an Azure IoT Edge node using the Fleet Command system, effectively connecting the Fleet Command System to Azure IoT.
provisioning: source: "dps" globalEndpoint: "https://global.azure-devices-provisioning.net" scopeId: "<ID scope>" attestation: method: "x509" identitySecret: "edge-authentication" identity_cert: "file:///etc/edge-attestation/identity_cert" identity_pk: "file:///etc/edge-attestation/identity_pk" identity_cert_name: <your identity certificate>.pem identity_key_name: <your identity certificate key>.pem dynamicReprovisioning: false
Once Azure IoT Runtime is deployed on Fleet Command, wait at least five minutes until the device status shows as Provisioned in Azure IoT Central. If the Device status says Registered, IoT Edge Runtime could not connect to Azure IoT Central, and additional troubleshooting is needed.
Creating an Azure IoT Central Device Template
This section describes how to create an Azure IoT Central device template.
Use the following CUDA sample JSON to create a device template in the steps that follow:
{
"modulesContent": {
"$edgeAgent": {
"properties.desired": {
"schemaVersion": "1.0",
"runtime": {
"type": "docker",
"settings": {
"minDockerVersion": "v1.25",
"loggingOptions": "",
"registryCredentials": {}
}
},
"systemModules": {
"edgeAgent": {
"type": "docker",
"settings": {
"image": "mcr.microsoft.com/azureiotedge-agent:1.0.9",
"createOptions": "{}"
}
},
"edgeHub": {
"type": "docker",
"status": "running",
"restartPolicy": "always",
"settings": {
"image": "mcr.microsoft.com/azureiotedge-hub:1.0.9",
"createOptions": "{}"
}
}
},
"modules": {
"CudaSample": {
"version": "1.0",
"type": "docker",
"env": {
"NVIDIA_VISIBLE_DEVICES": {
"value": "0"
}
},
"status": "running",
"restartPolicy": "always",
"settings": {
"image": "nvcr.io/nvidia/k8s/cuda-sample:nbody-cuda11.2.1",
"createOptions": "{\"Entrypoint\":[\"/bin/sh\"],\"Cmd\":[\"-c\",\"/tmp/nbody -benchmark -i=1000; while true; do echo no-op; sleep 10000;done\"],\"HostConfig\":{\"IpcMode\":\"host\",\"PidMode\":\"host\"}}"
}
}
}
}
},
"$edgeHub": {
"properties.desired": {
"schemaVersion": "1.0",
"routes": {
"route": "FROM /* INTO$upstream"
},
"storeAndForwardConfiguration": {
"timeToLiveSecs": 7200
}
}
},
"SimulatedTemperatureSensor": {
"properties.desired": {
"SendData": true,
"SendInterval": 10
}
}
}
}
Navigate to your IoT Central instance, then select Device templates from the left navigation menu.
Create a new device template, select Azure IoT Edge type, and then upload the device template JSON from above.
Follow the instructions in Assign a device to a device template to assign the device template to IoT Edge devices.
Once the device template is assigned to an IoT device, you can see the modules deployed on the Fleet Command edge node with running status.
