Environment Variables And Prerequisites Setup#

Caution

While this document provides an overview of certain Cloud Service Provider (CSP) procedures, it is strongly advised to consult the official, publicly accessible CSP documentation for the most up-to-date and comprehensive instructions. CSPs frequently update their services and interfaces, making it crucial to rely on their current guidelines to ensure accuracy and optimal implementation.

Using lookup Function In Config Template#

The config template yml file contains several inputs about the infrastructure and application’s needs. For ease of use, some of these are wired to lookup environment variables. For example {{ lookup(‘env’, ‘NGC_CLI_API_KEY’) }} is expanding it to the NGC_CLI_API_KEY environment variable. What this means is, we can set an environment variable for NGC_CLI_API_KEY with its value and Deployment Script can access it automatically.

Environment Variable Definitions#

By default the sample environment variable files examples folder provides look like below. Based on need you can make more environment variables and use lookup function in config-template yaml file.

cat my-config.env
export OPENAI_API_KEY="<replace-with-actual-value>"
export NGC_CLI_API_KEY="<replace-with-actual-value>"
export NVIDIA_API_KEY="<replace-with-actual-value>"
export ARM_TENANT_ID="<replace-with-actual-value>"
export ARM_SUBSCRIPTION_ID="<replace-with-actual-value>"
export ARM_CLIENT_ID="<replace-with-actual-value>"
export ARM_CLIENT_SECRET="<replace-with-actual-value>"

Below table explains the purpose of each environment variable used in the examples folder.

Azure Environment Variables#

Environment Variable

Definition

OPENAI_API_KEY

Key to access OpenAI’s models through the API

NGC_CLI_API_KEY

Key to access NGC resources thru command line. Such as Helm charts, Models, Container images etc.

NVIDIA_API_KEY

Key to access NGC resources thru command line. Such as Helm charts, Models, Container images etc. This one is specifically used to access Nvidia NIMs

ARM_TENANT_ID

The unique identifier of your Azure Active Directory tenant.

ARM_SUBSCRIPTION_ID

The ID of the Azure subscription where resources will be deployed.

ARM_CLIENT_ID

The application (client) ID of the Azure service principal used for authentication.

ARM_CLIENT_SECRET

The client secret (essentially a password) associated with an Azure service principal. This secret is created when you set up a service principal in Azure Active Directory.

Caution

If you modify your <my-env-file.env> file or start a new shell, you will have to run source <my-env-file.env> again before running ./envbuild.sh.

Setting up Azure Pre-requisites#

Note

The pre-requisites provisioned here can be used for multiple projects, and can be considered as a one time setup for most scenarios unless the parameters are not acceptable for any deployment.

Login to the Azure portal#

  • Log into azure portal as a user with admin access.

  • Click on More Services to get to the page listing all services.

  • For all subsequent steps, navigate back to this page to find and create a new resource.

Service Principal Setup#

App Registration Service Principal#

From the All Services page:

  • Select the Identity from the category (on the left).

  • Select the Azure Active Directory service.

  • Select App registrations in the service configuration panel (on the left).

  • Click on the +New registration button to create a new registration.

  • In the wizard:

    • Name: Provide an appropriate name. (eg., <my-org>-tokkio-automation)

    • Supported account types: Select Accounts in this organizational directory only.

    • Ignore all other fields.

    • Click Register.

  • You will be automatically taken to the created App registration. If not:

    • From All Services, navigate to Azure Active Directory under Identity category

    • Select App registrations from the service configuration panel

    • Select the created App Registration.

  • Select Certificates & secrets from the resource configuration panel (on the left).

  • Click on the +New client secret button to create a new client secret.

  • In the wizard:

    • Description: Provide an appropriate description.

    • Expires: Provide the period for which this secret will be valid.

    • Click Add.

    • Copy the Value of the created client secret.

    Note

    Once you exit this screen, the value will no longer be visible.

Subscription Access grant to App Registration#

From the All Services page:

  • Select the General from the category (on the left).

  • Select the Subscriptions service.

  • Select the name of the subscription under which the Tokkio Application will be deployed.

  • Select Access control (IAM) from the resource configuration panel (on the left).

  • Select the Role Assignments tab.

  • Click on the +Add and then the Add role assignment option to add a new role assignment.

  • In the wizard:

    • Role: Select Owner.

    • Members:

      • Assign access to: Select User, group, or service principal.

      • Members: Click +Select members and select the name of the App Registration.

  • Click on the +Add and then the Add role assignment option to add a new role assignment.

  • In the wizard:

    • Role: Select Contributor.

    • Members:

      • Assign access to: Select User, group, or service principal.

      • Members: Click +Select members and select the name of the App Registration.

Resource Group#

From the All Services page:

  • Select the General from the category (on the left).

  • Select the Resource groups service.

  • Click on the +Create button to create a new resource group.

  • In the wizard:

    • Subscription: Choose the subscription under which resources will be created.

    • Resource group: Provide an appropriate name (eg., <<my-org>-tokkio-automation-pre-requisites>)

    • Region: Choose a region (preferably closest to the users of the application) in which to create the Resource Group.

    • Navigate Next to optionally configure tags (we will be skipping this through the remainder of this setup).

    • Click Review + create > Create.

Deployment State Storage#

From the All Services page:

  • Select the Storage from the category (on the left).

  • Select the Storage accounts service.

  • Click on the +Create button to create a new storage account.

  • In the wizard:

    • In the Basics section:

      • Subscription: Subscription Access grant to App Registration created earlier.

      • Resource Group: The Resource Group created earlier.

      • Storage account name: Provide an appropriate name (eg., <myorg>tokkiodeploymentstate)

      • Region: Region chosen for the Resource Group created earlier.

      • Performance: Choose Standard.

      • Redundancy: Choose Locally-redundant storage (LRS).

      • Leave the remaining fields as is.

    • Leave all other sections as is.

    • Optionally add tags under the Tags section.

    • Click Review + create > Create.

  • Navigate to the created storage account by clicking on Go to resource or:

    • From All Services, navigate to Storage accounts under Storage category

    • Select the created Storage account.

  • Select Containers from the resource configuration panel (on the left).

  • Click on the +Container to create a new container.

  • In the wizard:

    • Name: Provide an appropriate name (eg: deployment-state)

    • Public access level: Select Private.

    • Click Create.

Base Domain#

From the All Services page:

  • Select the Web from the category (on the left).

  • Select the App Service Domains service.

  • Click on the +Create button to create a new domain.

  • In the wizard:

    • In the Basics section:

      • Subscription: Subscription of the Resource Group created earlier.

      • Resource Group: The Resource Group created earlier.

      • Domain: Base domain which will be used for subsequent app deployments.

    • In the Contact information section - Provide the relevant Contact information. Not the email needs to be a valid email ID.

    • Leave the Advanced section as is.

    • Optionally add tags under the Tags section.

    • Click Review + create > Create.

Certificates#

Key Vault#

From the All Services page:

  • Select the Security from the category (on the left).

  • Select the Key Vaults service.

  • Click on the +Create button to create a new key vault.

  • In the wizard:

    • In the Basics section:

      • Subscription: Subscription of the Resource Group created earlier.

      • Resource Group: The Resource Group created earlier.

      • Key vault name: Provide an appropriate name (eg: <my-org>-certificates-vault).

      • Region: Region chosen for the Resource Group created earlier.

      • Leave remaining fields as is.

    • In the Access policy section:

      • Add an additional Access Policy.

      • Key Permissions: Select Select All.

      • Secret Permissions: Select Select All.

      • Certificate Permissions: Select Select All.

      • Rotation Policy Operations: Select Select All.

      • Add the App Registration Service Principal created earlier as Principal.

      • Let all other configurations remain as is.

      • Click Review + create > Create.

    • Leave the Networking section as is.

    • Optionally add tags under the Tags section.

    • Click Review + create > Create.

Wildcard Certificate#

From the All Services page:

  • Select the Web from the category (on the left).

  • Select the App Service Certificates service..

  • Click on the +Create button to create a new certificate.

  • In the wizard:

    • In the Basics section:

      • Subscription: Subscription of the Resource Group created earlier.

      • Resource Group: The Resource Group created earlier.

      • SKU: Wildcard.

      • Naked domain hostname: Enter *.<base-domain> where <base-domain> is the name of the Base Domain created earlier.

      • Certificate name: Provide an appropriate name (eg., <my-base-domain>-wildcard-certificate).

      • Enable auto renewal: Optionally select Disable.

    • Optionally add tags under the Tags section.

    • Click Review + create > Create.

  • Navigate to the created certificate by clicking on Go to resource or:

    • From All Services, navigate to App Service Certificates under Web category

    • Select the created Certificate.

  • Select Certificate Configuration from the resource configuration panel (on the left).

    • Select Store and then the Select from Key Vault link.

    • In the wizard:

      • Subscription: Subscription of the Resource Group created earlier.

      • Key vault: Name of the Key Vault created earlier.

  • Navigate back to the created certificate by:

    • From All Services, navigate to App Service Certificates under Web category

    • Select the created Certificate.

  • Select Certificate Configuration from the resource configuration panel (on the left).

    • Select Verify and then click on Verify.

    • Wait for Domain Verification to complete.

Increase Quota for GPU VM type#

From the All Services page:

  • Select the Other from the category (on the left).

  • Select Quotas service.

  • Click on Compute.

Validate sufficient quota is present to create the application virtual machine by doing the following:

  • Update the Region filter to match the region where the virtual machines will be created.

  • In the search box, enter NCASv3_T4.

  • Check if available usage is at least 64 times the number of instances you wish to run under this setup, if not edit to request for additional quota.

  • Wait for confirmation that the quota has increased before proceeding.

On this page