Microsoft Azure Setup Guide

Overview

In this guide, we will go through the steps needed to:

• Understand the architecture of the infrastructure we will be setting up to host Metropolis applications on the Microsoft Azure.

• Perform the necessary steps to procure the pre-requisite access and information to use the automated deployment scripts.

• Create one or more deployments of Metropolis applications using the automated deployment scripts.

• Verify the deployment.

• Tear down the created infrastructure when no longer required.

Infrastructure Layout

Metropolis application setup on Azure requires several Azure resources to be created such as Virtual Machines, Security Groups, Application Gateways, FrontDoor CDN for hosting UI content, etc., While there are several patterns that can be followed to bring up infrastructure needed for Metropolis, here is one way we will be working to achieve.

In addition to bringing up Azure resources, we also will have to work on downloading the Metropolis application and its dependency artifacts, configure them and install. These automation scripts will help you simplify that by abstracting out the complexity and allowing user to work with majorly 2 files viz., deploy-template.yml and secrets.sh.

deploy-template.yml is an abstraction of infra specification we need for bringing up Metropolis applications. At a high level, we define the base infrastructure specifications (e.g. VirtualNetwork); then add COTURN infrastructure (e.g. Virtual Machine) and Application infrastructure specifications (e.g. GPU Virtual Machine). COTURN infrastructure and Application infrastructure will be established on base infrastructure specified on this deploy-template.yml.

To manage multiple environments with single deploy-template.yml, you could add multiple base infrastructure elements and respective COTURN and application infrastructure elements for each base. However, need to map each of COTURN and Application infrastructure elements to base using correct reference mapping as discussed in later part of documentation.

secrets.sh will be used as mechanism for user to provide secrets of two categories. Secrets such as ‘Azure resource manager client secret’ so that automation program can interact with Azure tenant. It also takes in secrets needed for application configuration such as ‘twilio auth token’ etc., and use them while installing application.

Note

We will skip some optional features such as Auto Scaling in this reference setup.

Important

Many of the resources in this setup may not fall in Free tier, you can check Azure Cost Management pages for understanding cost implications.

Prerequisites

This setup guide assumes you have the following conditions met:

• Access to the Azure portal via a user with Admin access to at least one Subscription.

• Ubuntu 20.04/22.04 based machine, on a user with sudo privileges to run the automated deployment scripts.

• An Azure service principal to enable the automated deployment scripts to authenticate themselves with.

• An Azure storage account and container to host the state of the automated deployment scripts, so that the created infrastructure may be modified or torn down at a later date or time.

• Registration of a domain on which the Metropolis application can be hosted.

• An Azure app certificate to enable SSL support for the deployed application.

• Setup Nvidia GPU Cloud (NGC) API Key by following instructions here.

• SSH key pair to access the instances we are going to setup.

  • You may use existing SSH Key pair for this access or create a new pair.

  • Reference documentation to create a public private ssh key pair is available here.

Note

The same pre-requisites provisioned here can be used for multiple projects, and can be considered as a one time setup for most scenarios unless the parameters are not acceptable for any deployment.

Login to the Azure portal

1. Log into azure portal as user with admin access.

2. Click on More Services to get to the page listing all services.

3. For all subsequent steps, navigate back to this page to find and create a new resource.

Service Principal Setup

App Registration Service Principal

From the All Services page:

• Select the Identity from the category (on the left).

• Select App registrations in the service configuration panel (on the left).

• Click on the +New registration button to create a new registration.

• In the wizard:

  • Name: Provide an appropriate name. (eg: <my-org>-mdx-automation)

  • Supported account types: Select Accounts in this organizational directory only.

  • Ignore all other fields.

  • Click Register.

• You will be automatically taken to the created App registration. If not:

  • From All Services, navigate to Azure Active Directory under Identity category.

  • Select App registrations from the service configuration panel.

  • Select the created App Registration.

• Select Certificated & secrets from the resource configuration panel (on the left).

• Click on the +New client secret button to create a new client secret.

• In the wizard:

  • Description: Provide an appropriate description.

  • Expires: Provide the period for which this secret will be valid.

  • Click Add.

  • Copy the Value of the created client secret.

    Note

    Once you exit this screen, the value will no longer be visible.

Subscription Access grant to App Registration

From the All Services page:

• Select the General from the category (on the left).

• Select the Subscriptions service.

• Select the name of the subscription under which the Metropolis application will be deployed.

• Select Access control (IAM) from the resource configuration panel (on the left).

• Select the Role Assignments tab.

• Click on the +Add and then the Add role assignment option to add a new role assignment.

• In the wizard:

  • Role: Select Owner.

  • Members:

    • Assign access to: Select User, group, or service principal.

    • Members: Click +Select members and select the name of the App Registration.

• Click on the +Add and then the Add role assignment option to add a new role assignment.

• In the wizard:

  • Role: Select Contributor.

  • Members:

    • Assign access to: Select User, group, or service principal.

    • Members: Click +Select members and select the name of the App Registration.

Resource Group

From the All Services page:

• Select the General from the category (on the left).

• Select the Resource groups service.

• Click on the +Create button to create a new resource group.

• In the wizard:

  • Subscription: Choose the subscription under which resources will be created.

  • Resource group: Provide an appropriate name (eg: <<my-org>-mdx-automation-pre-requisites>).

  • Region: Choose a region (preferably closest to the users of the application) in which to create the Resource Group.

  • Navigate Next to optionally configure tags (we will be skipping this through the remainder of this setup).

  • Click Review + create > Create.

Deployment State Storage

From the All Services page:

• Select the Storage from the category (on the left).

• Select the Storage accounts service.

• Click on the +Create button to create a new storage account.

• In the wizard:

  • In the Basics section:

    • Subscription: Subscription of the Resource Group created earlier.

    • Resource Group: The Resource Group created earlier.

    • Storage account name: Provide an appropriate name (eg: <myorg>mdxdeploymentstate).

    • Region: Region chosen for the Resource Group created earlier.

    • Performance: Choose Standard.

    • Redundancy: Choose Locally-redundant storage (LRS).

    • Leave the remaining fields as is.

  • Leave all other sections as is.

  • Optionally add tags under the Tags section.

  • Click Review + create > Create.

• Navigate to the created storage account by clicking on Go to resource or:

  • From All Services, navigate to Storage accounts under Storage category.

  • Select the created Storage account.

• Select Containers from the resource configuration panel (on the left).

• Click on the +Container to create a new container.

• In the wizard:

  • Name: Provide an appropriate name (eg: deployment-state).

  • Public access level: Select Private.

  • Click Create.

Base Domain

From the All Services page:

• Select the Web from the category (on the left).

• Select the App Service Domains service..

• Click on the +Create button to create a new domain.

• In the wizard:

  • In the Basics section:

    • Subscription: Subscription of the Resource Group created earlier.

    • Resource Group: The Resource Group created earlier.

    • Domain: Base domain which will be used for subsequent app deployments.

  • In the Contact information section - Provide the relevant Contact information. Not the email needs to be a valid email ID.

  • Leave the Advanced section as is.

  • Optionally add tags under the Tags section.

  • Click Review + create > Create.

Certificates

Key Vault

From the All Services page:

• Select the Security from the category (on the left).

• Select the Key Vaults service.

• Click on the +Create button to create a new key vault.

• In the wizard:

  • In the Basics section:

    • Subscription: Subscription of the Resource Group created earlier.

    • Resource Group: The Resource Group created earlier.

    • Key vault name: Provide an appropriate name (eg: <my-org>-certificates-vault).

    • Region: Region chosen for the Resource Group created earlier.

    • Leave remaining fields as is.

  • In the Access policy section:

    • Add an additional Access Policy.

    • Key Permissions: Select Select All.

    • Secret Permissions: Select Select All.

    • Certificate Permissions: Select Select All.

    • Rotation Policy Operations: Select Select All.

    • Add the App Registration Service Principal created earlier as Principal.

    • Let all other configurations remain as is.

    • Click Review + create > Create.

  • Leave the Networking section as is.

  • Optionally add tags under the Tags section.

  • Click Review + create > Create.

Wildcard Certificate

From the All Services page:

• Select the Web from the category (on the left).

• Select the App Service Certificates service..

• Click on the +Create button to create a new certificate.

• In the wizard:

  • In the Basics section:

    • Subscription: Subscription of the Resource Group created earlier.

    • Resource Group: The Resource Group created earlier.

    • SKU: Wildcard.

    • Naked domain hostname: Enter *.<base-domain> where <base-domain> is the name of the Base Domain created earlier.

    • Certificate name: Provide an appropriate name (eg: <my-base-domain>-wildcard-certificate).

    • Enable auto renewal: Optionally select Disable.

  • Optionally add tags under the Tags section.

  • Click Review + create > Create.

• Navigate to the created certificate by clicking on Go to resource or:

  • From All Services, navigate to App Service Certificates under Web category

  • Select the created Certificate.

• Select Certificate Configuration from the resource configuration panel (on the left).

  • Select Store and then the Select from Key Vault link.

  • In the wizard:

    • Subscription: Subscription of the Resource Group created earlier.

    • Key vault: Name of the Key Vault created earlier.

• Navigate back to the created certificate by:

  • From All Services, navigate to App Service Certificates under Web category

  • Select the created Certificate.

• Select Certificate Configuration from the resource configuration panel (on the left).

  • Select Verify and then click on Verify.

  • Wait for Domain Verification to complete.

Increase quota for GPU VM type

From the All Services page:

• Select the General from the category (on the left).

• Select Quotas service.

• Click on Compute.

Validate sufficient quota is present to create the application virtual machine by doing the following:

• Update the Region filter to match the region where the virtual machines will be created.

• In the search box, enter NC96ads.

• Check if available usage is at least 64 times the number of instance you wish to run under this setup, if not edit to request for additional quota.

• For example if provisioning Azure VM of type Standard_NC96ads_A100_v4 we will need 96 vCPUs of type Standard NCADS_A100_v4 Family vCPUs.

• Wait for confirmation that the quota has increased before proceeding.

Prepare deployment config

Download & extract deployment artifact

• Setting up NGC CLI

• Setup ngc cli tool on Ubuntu 20.04/22.04 machine by following instructions from this page.

• Select ‘AMD64 Linux Install’ tab for Ubuntu installation.

• During ngc config set command, select ‘nfgnkvuikvjm’ as Org and ‘mdx-v2-0’ as team.

• Download Oneclick Setup Scripts

Important

• Download of artifact metropolis-azure-one-click-script-with-openvpn is optional and is only required when egde to cloud deployment needs to be tested with securing edge-to-cloud connection using OpenVPN Tunnel.

• More details on edge-to-cloud deployment can be found here.

• Using below commands, download and extract the contents of deployment artifact and navigate to the deployment directory:

  # download the artifact
  $ ngc registry resource download-version "nfgnkvuikvjm/mdx-v2-0/metropolis-azure-one-click-script:0.0.5"
  
  $ cd metropolis-azure-one-click-script_v0.0.5/
  
  
  # verify necessary files required for Installing Infra on Azure CSP
  $ ls
  README.md                              deploy-spec          examples         modules     working-deploy-template.yml
  deploy-template.yml  mdx-apps-deploy  secrets.sh
  $
  
  
  Note: You can skip download of below artifact if edge to cloud deployment mode is not needed for testing.
  
  # download the artifact to deploy metropolis app with openvpn for edge connectivity
  $ ngc registry resource download-version "nfgnkvuikvjm/mdx-v2-0/metropolis-azure-one-click-script-with-openvpn:0.0.5"
  
  $ cd metropolis-azure-one-click-script-with-openvpn_v0.0.5/
  
  
  # verify necessary files required for Installing Infra on Azure CSP
  $ ls
  README.md                              deploy-spec          examples         modules     working-deploy-template.yml
  deploy-template.yml  mdx-apps-deploy  secrets.sh
  $
  

Download & extract application helm values

Refer to this section to download and extract the contents of reference application helm values & sample data.

Prepare secrets

The file secrets.sh can be setup as below so as not have to commit and push sensitive data as part of deploy-template.yml

secrets.sh

 #!/bin/bash

 # SPDX-FileCopyrightText: Copyright (c) 2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 # SPDX-License-Identifier: LicenseRef-NvidiaProprietary
 #
 # NVIDIA CORPORATION, its affiliates and licensors retain all intellectual
 # property and proprietary rights in and to this material, related
 # documentation and any modifications thereto. Any use, reproduction,
 # disclosure or distribution of this material and related documentation
 # without an express license agreement from NVIDIA CORPORATION or
 # its affiliates is strictly prohibited.

 # ARM_CLIENT_SECRET -> client secret value of the azure app registration with access to the resources
 export ARM_CLIENT_SECRET='<replace_content_between_quotes_with_your_value>'
 # _ssh_public_key -> Your public ssh key's content
 export _ssh_public_key='<replace_content_between_quotes_with_your_value>'
 # _ngc_api_key -> Your ngc api key value
 export _ngc_api_key='<replace_content_between_quotes_with_your_value>'
 # _turnserver_password -> Password for turn server
 # This is not required if you are using 'use_twilio: true' in deploy-template.yml
 export _turnserver_password='<replace_content_between_quotes_with_your_value>'

Important

You may want to be careful on whether or not to commit this file to your version control system as it contains secrets.

Prepare deploy template

Deploy Template Schema & Configuration:

Deploy template deploy-template.yml is used to compile the infrastructure needed to setup your project/environment(s). It has separate sections to capture details for different needs such as provider config, coturn-infra etc. As shown in below layout diagram, you can choose to create one or more environments and infrastructure(s) under single project name.

Override the content of deploy-template.yml file with your environment/application specific values. This will drive the configuration of Infrastructure and application being installed.

deploy-template.yml

# SPDX-FileCopyrightText: Copyright (c) 2024 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: LicenseRef-NvidiaProprietary
#
# NVIDIA CORPORATION, its affiliates and licensors retain all intellectual
# property and proprietary rights in and to this material, related
# documentation and any modifications thereto. Any use, reproduction,
# disclosure or distribution of this material and related documentation
# without an express license agreement from NVIDIA CORPORATION or
# its affiliates is strictly prohibited.

# NOTE: Refer to examples for various configuration options

project_name: '<replace-with-unique-name-to-identify-your-project>'
description: '<add-a-brief-description-about-this-project>'
template_version: '0.1.1'
csp: 'azure'
backend:
  tenant_id: '<replace-with-your-azure-tenant-id>'
  subscription_id: '<replace-with-your-azure-subscription-id>'
  client_id: '<replace-with-your-azure-app-registration-client-id>'
  resource_group_name: '<replace-with-your-state-storage-account-resource-group-name>'
  storage_account_name: '<replace-with-your-state-storage-account-name>'
  container_name: '<replace-with-your-state-storage-container-name>'
provider:
  tenant_id: '<replace-with-your-azure-tenant-id>'
  subscription_id: '<replace-with-your-azure-subscription-id>'
  client_id: '<replace-with-your-azure-app-registration-client-id>'
base_infra:
  # NOTE: Repeat below section for as many base setups as necessary
  <your-base-env-name>:
    spec:
      region: '<replace-with-azure-region-to-create-resources-in>'
      virtual_network_address_space: '<replace-with-an-available-cidr-range>'
      ssh_public_key: '${_ssh_public_key}' # NOTE: value of _ssh_public_key assumed to be provided in secrets.sh
      dev_source_address_prefixes:
        - '<replace-with-list-of-dev-ip-cidrs>'
      user_source_address_prefixes:
        - '<replace-with-list-of-user-ip-cidrs>'
      dns_and_certs_configs:
        resource_group: '<replace-with-your-resource-group-name-containing-the-domain-and-certs>'
        dns_zone: '<replace-with-the-dns-zone-under-which-apps-will-be-registered>'
        wildcard_cert: '<replace-with-the-name-of-the-wildcard-certificate-of-the-above-base-domain>'
coturn_infra: # set as {} in case no COTURN environment is needed
  # NOTE: Repeat below section for as many COTURN environments as necessary
  <your-coturn-env-name>:
    base_ref: '<replace-with-your-base-env-name>' # NOTE: should match the name of a base env defined in the above section
    spec:
      turnserver_realm: '<replace-with-a-realm-to-use-for-the-turnserver>'
      turnserver_username: '<replace-with-a-username-to-use-for-the-turnserver>'
      turnserver_password: '${_turnserver_password}' # NOTE: value of _turnserver_password assumed to be provided in secrets.sh
app_infra:
  # NOTE: Repeat below section for as many app environments as necessary
  <your-application-env-name>:
    base_ref: '<replace-with-your-base-env-name>' # NOTE: should match the name of a base env defined in the above section
    # NOTE: Uncomment below line in case app environment should use one of the setup COTURN environments
    #coturn_ref: '<replace-with-your-coturn-env-name>' # NOTE: should match the name of a COTURN env with same base ref defined in the above section
    spec:
      ngc_api_key: '${_ngc_api_key}' # NOTE: value of _ngc_api_key assumed to be provided in secrets.sh
      # NOTE: Uncomment any of the below lines based on the need to override
      # --- OPTIONAL CONFIGURATION START ---
      api_vm_size: 'Standard_NC96ads_A100_v4' ### defaults to Standard_NC96ads_A100_v4(4x A100 GPU VM)
      gpu_driver_version: "535.54.03" ### GPU Driver Version to install specific version using CNC ###
      enable_gpu_slicing: "false"     ### Flag to control time-slicing feature for gpu-operator, deafult is set to false ###
      gpu_slice_replica: "2"   #### Slice Count for a single GPU to be partitioned into ###
      api_sub_domain: '<replace-with-the-subdomain-to-be-used-for-the-api>'
      api_vm_zone: '1' ### Default is 1 change it as per requirement
      api_app_gw_app_port: 31080 ##### port for App UI to be available, ideally ingress port, defaults to port 31080 ###
      api_app_gw_app_health_path: '/api/v2.1/analytics/livez'   ### MDX App healthcheck path
      api_vm_data_disk_size_gb: 1024
      foundational_chart:
        org: "nfgnkvuikvjm"
        team: "mdx-v2-0"
        name: "mdx-foundation-sys-svcs"
        version: "v1.3"
      foundational_override_values_file:
      app_chart:
        org: "nfgnkvuikvjm"
        team: "mdx-v2-0"
        name: "mdx-mtmc-app"
        version: "1.0.36"
      app_override_values_file: <abs_path>/mtmc_app_values.yaml
      nvstreamer_app_chart:
        org: "rxczgrvsg8nx"
        team: "vst-1-0"
        name: "nvstreamer"
        version: "0.2.23"
      nvstreamer_app_override_values_file: <abs_path>/nvstreamer_app_values.yaml
      vst_app_chart:
        org: "rxczgrvsg8nx"
        team: "vst-1-0"
        name: "vst"
        version: "1.0.24"
      vst_app_override_values_file: <abs_path>/vst_app_values.yaml
      ds_app_chart:
        org: "nfgnkvuikvjm"
        team: "mdx-v2-0"
        name: "mdx-wdm-ds-app"
        version: "0.0.32"
      wdm_ds_app_override_values_file: <abs_path>/wdm_ds_app_values.yaml
      additional_ports:
      #- port: "31560"
      #    health_port: "31080"
      #    health_path: "/elastic"
      #    health_status_codes: ["200"]
      # ---- OPTIONAL CONFIGURATION END ----

Explanation of each and every entry of this yml file are explained in below table:

Deploy Template

Parameter name	Type	Optional	Description
project_name	string		A unique name to identify the project. This is important to tear down resources later.
description	string		A brief description of the project.
backend	map		Backend configuration.
backend > tenant_id	string		Azure tenant id of the state storage account.
backend > subscription_id	string		Azure subscription id of the state storage account.
backend > client_id	string		Azure client id of the app registration with access to subscription.
backend > resource_group_name	string		Azure resource group name of the state storage account.
backend > storage_account_name	string		Azure storage account name of the state storage account.
backend > container_name	string		Azure storage account container name of the container for state storage in the state storage account.
provider	map		Provider configuration.
provider > tenant_id	string		Azure tenant id where application will be deployed.
provider > subscription_id	string		Azure subscription id where application will be deployed.
provider > client_id	string		Azure client id of the app registration with access to subscription.
base_infra	map		Base for app configuration.
base_infra > KEY	map		An instance of base configuration. There can be 1 or more instances.
base_infra > KEY > spec	map		Configuration specifications of this base instance.
base_infra > KEY > spec > region	string		Azure region in which base, coturn and app resources will be created.
base_infra > KEY > spec > virtual_network_address_space	string		Private CIDR range in which base, coturn and app resources will be created.
base_infra > KEY > spec > ssh_public_key	string		Content of the public key of the ssh key-pair used for instance access. Prefer to provide via variable in secrets.sh.
base_infra > KEY > spec > dev_source_address_prefixes	array		CIDR ranges from where SSH access should be allowed.
base_infra > KEY > spec > user_source_address_prefixes	array		CIDR ranges from where application UI and APP will be allowed access.
base_infra > KEY > spec > dns_and_certs_configs	map		DNS & Certs Configs for DNS records and Certs for https Azure App GW.
base_infra > KEY > spec > dns_and_certs_configs > resource_group	string		Resource Group Name for certs and DNS Zone are maintained.
base_infra > KEY > spec > dns_and_certs_configs > dns_zone	string		The dns-zone under which apps will be registered.
base_infra > KEY > spec > dns_and_certs_configs > wildcard_cert	string		The wildcard certificate of the above base domain.
coturn_infra	map		COTURN instance used in app configuration.
coturn_infra > KEY	map		An instance of COTURN configuration. Coturn Instance is must for Metropolis App since Application UI uses WebRTC connections for playing video snippets.
coturn_infra > KEY > base_ref	string		The key name of the base instance that should be used to set up this COTURN instance.
coturn_infra > KEY > spec > turnserver_realm	string		Realm name used during COTURN setup.
coturn_infra > KEY > spec > turnserver_username	string		Username used to connect to COTURN server.
coturn_infra > KEY > spec > turnserver_password	string		Password used to connect to COTURN server. Prefer to provide via variable in secrets.sh.
app_infra	map		Application configuration.
app_infra > KEY	map		An instance of application configuration. There can be 1 or more instances.
app_infra > KEY > base_ref	string		The key name of the base instance that should be used to set up this app instance.
app_infra > KEY > coturn_ref	string	yes	The key name of the COTURN instance that should be used to set up this app instance. Not required if using Twilio.
app_infra > KEY > spec > ngc_api_key	string		NGC API key with access to deployment artifacts. Prefer to provide via variable in secrets.sh.
app_infra > KEY > spec > api_vm_size	string	yes	The Azure Virtual Machine size on which the APP would run. Defaults to Standard_NC96ads_A100_v4(4x A100 GPU VM).
app_infra > KEY > spec > gpu_driver_version	string	yes	GPU Driver Version to install using GPU-Operator for k8s. Defaults to 535.104.12.
app_infra > KEY > spec > enable_gpu_slicing	string	yes	This will enable GPU Slicing if set to true which will slice a single GPU into partition and partition size can be defined in params “gpu_slice_replica” It is useful to enable when using high-end GPU type. Defaults to false.
app_infra > KEY > spec > gpu_slice_replica	string	yes	Number of partitions of a single GPU to be created for example “gpu_slice_replica: 2” will divide a single T4 GPU into 2 partitions. Defaults to 2 if “enable_gpu_slicing is set to true”.
app_infra > KEY > spec > api_sub_domain	string		The Subdomain to be used for the app. This will be an endpoint where APP UI will load for metropolis apps.
app_infra > KEY > spec > api_vm_zone	string	yes	The Zone in which APP VM to be provisioned. This is required because certain VM sizes are available in specific zone. “”Defaults to 1””.
app_infra > KEY > spec > api_app_gw_app_port	string	yes	Ingress port to connect app gw backend settings. Defaults to deployed foundational chart nginx ingress port as “”31080””. MDX foundational chart uses nginx ingress port as “”31080””.
app_infra > KEY > spec > api_app_gw_app_health_path	string	yes	App GW requires healthcheck. Web API follows release version in API Path for MDX releases making it configurable. “”Defaults to “”/api/v2.1/analytics/livez””.
app_infra > KEY > spec > api_vm_data_disk_size_gb	number	yes	The data disk size in GB on which the APP would run. Defaults to 1024.
app_infra > KEY > spec > foundational_chart	map	yes	Configuration to change the default APP chart used.
app_infra > KEY > spec > foundational_chart > org	string		NGC Org of the foundational_chart to be used.
app_infra > KEY > spec > foundational_chart > team	string		NGC Team of the foundational_chart to be used.
app_infra > KEY > spec > foundational_chart > name	string		NGC Resource Name of the foundational_chart to be used.
app_infra > KEY > spec > foundational_chart > version	string		NGC Resource Version of the foundational_chart to be used.
app_infra > KEY > spec > foundational_override_values_file	string		absolute path to foundational_override_values_file.
app_infra > KEY > spec > app_chart	map	yes	Configuration to change the default APP chart used.
app_infra > KEY > spec > app_chart > org	string		NGC Org of the APP chart to be used.
app_infra > KEY > spec > app_chart > team	string		NGC Team of the APP chart to be used
app_infra > KEY > spec > app_chart > name	string		NGC Resource Name of the APP chart to be used
app_infra > KEY > spec > app_chart > version	string		NGC Resource Version of the APP chart to be used
app_infra > KEY > spec > app_override_values_file	string		absolute path to app_override_values_file
app_infra > KEY > spec > nvstreamer_app_chart	map	yes	Configuration to change the default NvStreamer App Chart used
app_infra > KEY > spec > nvstreamer_app_chart > org	string		NGC Org of the NvStreamer App Chart to be used
app_infra > KEY > spec > nvstreamer_app_chart > team	string		NGC Team of the NvStreamer App Chart to be used
app_infra > KEY > spec > nvstreamer_app_chart > name	string		NGC Resource Name of the NvStreamer App Chart to be used
app_infra > KEY > spec > nvstreamer_app_chart > version	string		NGC Resource Version of the NvStreamer App Chart to be used
app_infra > KEY > spec > nvstreamer_app_override_values_file	string		absolute path to nvstreamer_app_override_values_file
app_infra > KEY > spec > vst_app_chart	map	yes	Configuration to change the default VST App chart used
app_infra > KEY > spec > vst_app_chart > org	string		NGC Org of the VST App chart to be used
app_infra > KEY > spec > vst_app_chart > team	string		NGC Team of the VST App chart to be used
app_infra > KEY > spec > vst_app_chart > name	string		NGC Resource Name of the VST App chart to be used
app_infra > KEY > spec > vst_app_chart > version	string		NGC Resource Version of the VST App chart to be used
app_infra > KEY > spec > vst_app_override_values_file	string		absolute path to vst_app_override_values_file
app_infra > KEY > spec > app_chart	map	yes	Configuration to change the default WDM-DS App chart used
app_infra > KEY > spec > app_chart > org	string		NGC Org of the WDM-DS App chart to be used
app_infra > KEY > spec > app_chart > team	string		NGC Team of the WDM-DS App chart to be used
app_infra > KEY > spec > app_chart > name	string		NGC Resource Name of the WDM-DS App chart to be used
app_infra > KEY > spec > app_chart > version	string		NGC Resource Version of the WDM-DS App chart to be used
app_infra > KEY > spec > wdm_ds_app_override_values_file	string		absolute path to wdm_ds_app_override_values_file
app_infra > KEY > spec > additional_ports	map	yes	Additional Ports for app-gw for different app specific dashboard
app_infra > KEY > spec > additional_ports > port	string		additional node-port for app-gw backend settings
app_infra > KEY > spec > additional_ports > health_port	string		Health Check Port for additional app-gw
app_infra > KEY > spec > additional_ports > health_path	string		Health probe path for healthcheck for additional port
app_infra > KEY > spec > additional_ports > health_status_codes	string		Health Probe status Code

Setup logs backup

Audit logs for any changes made via the script will be captured in a directory named logs at the same level as the deploy-template.yml. Take necessary measures to ensure these are backed up in the event they are needed for debugging.

Note

Any values defined in secrets.sh will be masked in the logs

Deploy infrastructure and application

Use the below commands to Install / Update / Uninstall Metropolis applications along with its infrastructure as per specs provided in deploy-template.

# To view available options
bash mdx-apps-deploy

# To preview changes based on deploy-template.yml without actually applying the changes
bash mdx-apps-deploy preview

# To install changes showed in preview option based on deploy-template.yml
bash mdx-apps-deploy install

# To show results/information about the project installed
bash mdx-apps-deploy show-results

# To uninstall the deployed infra and application
bash mdx-apps-deploy uninstall

Important

Both install and uninstall Options needs to be run with care. We recommend preview option to see the changes before install. If you are looking for an option to print the details of your past installation, use show-results option.

Warning

Any attempts to suspend Ctrl + Z the running deployment will result in an inability to make changes to the project via the scripts as well as the need to manually cleanup resources created via the web console. Prefer terminating the process using Ctrl + C in case it has to absolutely be exited.

Known issues

• Occasionally the install step might fail with an error reading something similar to one of the below:

  • Error: updating Azure Storage Account `static_website` "####": accounts.Client#SetServiceProperties: Failure responding to request: StatusCode=404

  • Error: failed creating container: failed creating container: containers.Client#Create: Failure responding to request: StatusCode=404

  When this occurs, running install again one or more times usually resolves the issue.

Verify Deployment

On successful deployment of Infra, you will be displayed output in a format as shown below.

Apply complete! Resources: <nn> added, <nn> changed, <nn> destroyed.

Outputs:

app_gw        = {
  "<app_infra key>" = {
    additional_ports_app_gw_public_ips = {}
    "api_endpoint" = "https://<api_domain_defined_in_deploy-template>" ### APP UI will be accessible on this Endpoint 
    app_gw_public_ip                   = {
    "<project_name>-<app_infra key>-api-app-gw" = "<app-gw_public_ip>"
            }
    "private_ips" = [
      "<private_ip_of_app_instace>",
    ]
  }
}
app_infra     = {
  "<app_infra key>"  = {
    "private_ips" = [
      "<private_ip_of_app_instace>",
    ]
  }
}
bastion_infra = {
  "<bastion_infra key>" = {
    "private_ip" = "<bastion_instance_private_ip>"
    "public_ip" = "<bastion_instance_public_ip>"
  }
}
coturn_infra = {
  "<coturn_infra key" = {
    "port" = 3478
    "private_ip" = "<coturn_instance_private_ip>"
    "public_ip" = "<coturn_instance_public_ip>"
  }
}

Use ssh command in below format to log into Application virtual machine.

Replace content between '<' and '>' with its appropriate values.
#pem file refered here must the key associated to the public key used in initial steps of setup.
ssh -i <path-to-pem-file> -o StrictHostKeyChecking=no -o ProxyCommand="ssh -i <path-to-pem-file> -W %h:%p -o StrictHostKeyChecking=no ubuntu@<bastion-vm-public-ip>" ubuntu@<app-vm-private-ip>

Once logged into the terminal, run below command to see the Kubernetes Pods’ statuses. All the pods should turn into Running state eventually.

$ kubectl get pods

## If, for some reason, pods are failing to start or run healthily, please check the logs to identify failure or pod crash issues. The `-f` flag can be used to follow the logs.

$ kubectl logs <pod-name> or kubectl logs -f <pod-name> 

Note

• Based on several conditions, Pods may take up to 30-40 mins to turn into Running state.

• To check the logs for pods that are not in the Running state, use kubectl logs <pod-name>.

Once all the pods are in Running state, try accessing the App UI with the help of output attribute api_endpoint. Example: https://<api_endpoint>/ui/<app-name> (supported app names are mtmc, rtls and people-analytics). When you try your URL on browser, you should be able to see the Metropolis application coming up at this point.

Teardown infrastructure and application

To teardown all the infrastructure along with application that we created thru above scripts, run bash mdx-apps-deploy uninstall command.

Important

Both install and uninstall Options needs to be run with care. We recommend preview option to see the changes before install. If you are looking for an option to print the details of your past installation, use show-results option.