Environment Variables and Prerequisites Setup#

Caution

While this document provides an overview of certain Cloud Service Provider (CSP) procedures, it is strongly advised to consult the official, publicly accessible CSP documentation for the most up-to-date and comprehensive instructions. CSPs frequently update their services and interfaces, making it crucial to rely on their current guidelines to ensure accuracy and optimal implementation.

Environment Variable Definitions#

By default the sample environment variable files examples folder provides look like below. Based on need you can make more environment variables and use lookup function in config-template yaml file.

cat my-config.env
export OPENAI_API_KEY="<replace-with-actual-value>"
export NGC_CLI_API_KEY="<replace-with-actual-value>"
export NVIDIA_API_KEY="<replace-with-actual-value>"
export ELEVENLABS_API_KEY="<replace-with-actual-value>"
export ARM_TENANT_ID="<replace-with-actual-value>"
export ARM_SUBSCRIPTION_ID="<replace-with-actual-value>"
export ARM_CLIENT_ID="<replace-with-actual-value>"
export ARM_CLIENT_SECRET="<replace-with-actual-value>"

Below table explains the purpose of each environment variable used in the examples folder.

Azure Environment Variables#

Environment Variable

Definition

OPENAI_API_KEY

Key to access OpenAI’s models through the API

NGC_CLI_API_KEY

Key to access NGC resources thru command line. Such as Helm charts, Models, Container images etc.

NVIDIA_API_KEY

Key to access NGC resources thru command line. Such as Helm charts, Models, Container images etc. This one is specifically used to access NVIDIA NIMs

ELEVENLABS_API_KEY

Key to access ElevenLabs API

ARM_TENANT_ID

The unique identifier of your Azure Active Directory tenant.

ARM_SUBSCRIPTION_ID

The ID of the Azure subscription where resources will be deployed.

ARM_CLIENT_ID

The application (client) ID of the Azure service principal used for authentication.

ARM_CLIENT_SECRET

The client secret (essentially a password) associated with an Azure service principal. This secret is created when you set up a service principal in Azure Active Directory.

Caution

If you modify your <my-env-file.env> file or start a new shell, you will have to run source <my-env-file.env> again before running ./envbuild.sh.

Using lookup Function in Config Template#

The config template yml file contains several inputs about the infrastructure and application needs. For ease of use, some of these are wired to look up environment variables. For example {{ lookup(‘env’, ‘NGC_CLI_API_KEY’) }} is expanding it to the NGC_CLI_API_KEY environment variable. What this means is we can set an environment variable for NGC_CLI_API_KEY with its value, and the Deployment Script can access it automatically.

Setting up Azure Pre-Requisites#

Note

The pre-requisites provisioned here can be used for multiple projects and can be considered as a one time setup for most scenarios unless the parameters are not acceptable for any deployment.

Login to the Azure Portal#

  • Log into azure portal as a user with admin access.

  • Click on More Services to get to the page listing all services.

  • For all subsequent steps, navigate back to this page to find and create a new resource.

Service Principal Setup#

App Registration Service Principal#

From the All Services page:

  • Select the Identity from the category (on the left).

  • Select the Azure Active Directory service.

  • Select App registrations in the service configuration panel (on the left).

  • Click on the +New registration button to create a new registration.

  • In the wizard:

    • Name: Provide an appropriate name. (e.g., <my-org>-tokkio-automation)

    • Supported account types: Select Accounts in this organizational directory only.

    • Ignore all other fields.

    • Click Register.

  • You will be automatically taken to the created App registration. If not:

    • From All Services, navigate to Azure Active Directory under Identity category

    • Select App registrations from the service configuration panel

    • Select the created App Registration.

  • Select Certificates & secrets from the resource configuration panel (on the left).

  • Click on the +New client secret button to create a new client secret.

  • In the wizard:

    • Description: Provide an appropriate description.

    • Expires: Provide the period for which this secret will be valid.

    • Click Add.

    • Copy the Value of the created client secret.

    Note

    Once you exit this screen, the value will no longer be visible.

Subscription Access Grant to App Registration#

From the All Services page:

  • Select the General from the category (on the left).

  • Select the Subscriptions service.

  • Select the name of the subscription under which the Tokkio Application will be deployed.

  • Select Access control (IAM) from the resource configuration panel (on the left).

  • Select the Role Assignments tab.

  • Click on the +Add and then the Add role assignment option to add a new role assignment.

  • In the wizard:

    • Role: Select Owner.

    • Members:

      • Assign access to: Select User, group, or service principal.

      • Members: Click +Select members and select the name of the App Registration.

  • Click on the +Add and then the Add role assignment option to add a new role assignment.

  • In the wizard:

    • Role: Select Contributor.

    • Members:

      • Assign access to: Select User, group, or service principal.

      • Members: Click +Select members and select the name of the App Registration.

Resource Group#

From the All Services page:

  • Select the General from the category (on the left).

  • Select the Resource groups service.

  • Click on the +Create button to create a new resource group.

  • In the wizard:

    • Subscription: Choose the subscription under which resources will be created.

    • Resource group: Provide an appropriate name (e.g., <<my-org>-tokkio-automation-pre-requisites>)

    • Region: Choose a region (preferably closest to the users of the application) in which to create the Resource Group.

    • Navigate Next to optionally configure tags (we will be skipping this through the remainder of this setup).

    • Click Review + create > Create.

Deployment State Storage#

From the All Services page:

  • Select the Storage from the category (on the left).

  • Select the Storage accounts service.

  • Click on the +Create button to create a new storage account.

  • In the wizard:

    • In the Basics section:

      • Subscription: Subscription Access grant to App Registration created earlier.

      • Resource Group: The Resource Group created earlier.

      • Storage account name: Provide an appropriate name (eg., <myorg>tokkiodeploymentstate)

      • Region: Region chosen for the Resource Group created earlier.

      • Performance: Choose Standard.

      • Redundancy: Choose Locally-redundant storage (LRS).

      • Leave the remaining fields as is.

    • Leave all other sections as is.

    • Optionally add tags under the Tags section.

    • Click Review + create > Create.

  • Navigate to the created storage account by clicking on Go to resource or:

    • From All Services, navigate to Storage accounts under Storage category

    • Select the created Storage account.

  • Select Containers from the resource configuration panel (on the left).

  • Click on the +Container to create a new container.

  • In the wizard:

    • Name: Provide an appropriate name (eg: deployment-state)

    • Public access level: Select Private.

    • Click Create.

Base Domain#

From the All Services page:

  • Select the Web from the category (on the left).

  • Select the App Service Domains service.

  • Click on the +Create button to create a new domain.

  • In the wizard:

    • In the Basics section:

      • Subscription: Subscription of the Resource Group created earlier.

      • Resource Group: The Resource Group created earlier.

      • Domain: Base domain which will be used for subsequent app deployments.

    • In the Contact information section - Provide the relevant Contact information. Not the email needs to be a valid email ID.

    • Leave the Advanced section as is.

    • Optionally add tags under the Tags section.

    • Click Review + create > Create.

Certificates#

Key Vault#

From the All Services page:

  • Select the Security from the category (on the left).

  • Select the Key Vaults service.

  • Click on the +Create button to create a new key vault.

  • In the wizard:

    • In the Basics section:

      • Subscription: Subscription of the Resource Group created earlier.

      • Resource Group: The Resource Group created earlier.

      • Key vault name: Provide an appropriate name (eg: <my-org>-certificates-vault).

      • Region: Region chosen for the Resource Group created earlier.

      • Leave remaining fields as is.

    • In the Access policy section:

      • Add an additional Access Policy.

      • Key Permissions: Select Select All.

      • Secret Permissions: Select Select All.

      • Certificate Permissions: Select Select All.

      • Rotation Policy Operations: Select Select All.

      • Add the App Registration Service Principal created earlier as Principal.

      • Let all other configurations remain as is.

      • Click Review + create > Create.

    • Leave the Networking section as is.

    • Optionally add tags under the Tags section.

    • Click Review + create > Create.

Wildcard Certificate#

From the All Services page:

  • Select the Web from the category (on the left).

  • Select the App Service Certificates service..

  • Click on the +Create button to create a new certificate.

  • In the wizard:

    • In the Basics section:

      • Subscription: Subscription of the Resource Group created earlier.

      • Resource Group: The Resource Group created earlier.

      • SKU: Wildcard.

      • Naked domain hostname: Enter *.<base-domain> where <base-domain> is the name of the Base Domain created earlier.

      • Certificate name: Provide an appropriate name (eg., <my-base-domain>-wildcard-certificate).

      • Enable auto renewal: Optionally select Disable.

    • Optionally add tags under the Tags section.

    • Click Review + create > Create.

  • Navigate to the created certificate by clicking on Go to resource or:

    • From All Services, navigate to App Service Certificates under Web category

    • Select the created Certificate.

  • Select Certificate Configuration from the resource configuration panel (on the left).

    • Select Store and then the Select from Key Vault link.

    • In the wizard:

      • Subscription: Subscription of the Resource Group created earlier.

      • Key vault: Name of the Key Vault created earlier.

  • Navigate back to the created certificate by:

    • From All Services, navigate to App Service Certificates under Web category

    • Select the created Certificate.

  • Select Certificate Configuration from the resource configuration panel (on the left).

    • Select Verify and then click on Verify.

    • Wait for Domain Verification to complete.

Increase Quota for GPU VM Type#

From the All Services page:

  • Select the Other from the category (on the left).

  • Select Quotas service.

  • Click on Compute.

Validate sufficient quota is present to create the application virtual machine by doing the following:

  • Update the Region filter to match the region where the virtual machines will be created.

  • In the search box, enter NCASv3_T4.

  • Check if available usage is at least 64 times the number of instances you wish to run under this setup, if not edit to request for additional quota.

  • Wait for confirmation that the quota has increased before proceeding.