Environment Variables And Prerequisites Setup#
Caution
While this document provides an overview of certain Cloud Service Provider (CSP) procedures, it is strongly advised to consult the official, publicly accessible CSP documentation for the most up-to-date and comprehensive instructions. CSPs frequently update their services and interfaces, making it crucial to rely on their current guidelines to ensure accuracy and optimal implementation.
Using lookup Function In Config Template#
The config template yml file contains several inputs about the infrastructure and application’s needs. For ease of use, some of these are wired to lookup environment variables. For example {{ lookup(‘env’, ‘NGC_CLI_API_KEY’) }}
is expanding it to the NGC_CLI_API_KEY environment variable. What this means is, we can set an environment variable for NGC_CLI_API_KEY with its value and Deployment Script can access it automatically.
Environment Variable Definitions#
By default the sample environment variable files examples folder provides look like below. Based on need you can make more environment variables and use lookup function in config-template yaml file.
cat my-config.env
export OPENAI_API_KEY="<replace-with-actual-value>"
export NGC_CLI_API_KEY="<replace-with-actual-value>"
export NVIDIA_API_KEY="<replace-with-actual-value>"
export ARM_TENANT_ID="<replace-with-actual-value>"
export ARM_SUBSCRIPTION_ID="<replace-with-actual-value>"
export ARM_CLIENT_ID="<replace-with-actual-value>"
export ARM_CLIENT_SECRET="<replace-with-actual-value>"
Below table explains the purpose of each environment variable used in the examples folder.
Environment Variable |
Definition |
---|---|
OPENAI_API_KEY |
Key to access OpenAI’s models through the API |
NGC_CLI_API_KEY |
Key to access NGC resources thru command line. Such as Helm charts, Models, Container images etc. |
NVIDIA_API_KEY |
Key to access NGC resources thru command line. Such as Helm charts, Models, Container images etc. This one is specifically used to access Nvidia NIMs |
ARM_TENANT_ID |
The unique identifier of your Azure Active Directory tenant. |
ARM_SUBSCRIPTION_ID |
The ID of the Azure subscription where resources will be deployed. |
ARM_CLIENT_ID |
The application (client) ID of the Azure service principal used for authentication. |
ARM_CLIENT_SECRET |
The client secret (essentially a password) associated with an Azure service principal. This secret is created when you set up a service principal in Azure Active Directory. |
Caution
If you modify your <my-env-file.env> file or start a new shell, you will have to run source <my-env-file.env> again before running ./envbuild.sh.
Setting up Azure Pre-requisites#
Note
The pre-requisites provisioned here can be used for multiple projects, and can be considered as a one time setup for most scenarios unless the parameters are not acceptable for any deployment.
Login to the Azure portal#
Log into azure portal as a user with admin access.
Click on
More Services
to get to the page listing all services.For all subsequent steps, navigate back to this page to find and create a new resource.
Service Principal Setup#
App Registration Service Principal#
From the All Services page:
Select the
Identity
from the category (on the left).Select the
Azure Active Directory
service.Select
App registrations
in the service configuration panel (on the left).Click on the
+New registration
button to create a new registration.In the wizard:
Name: Provide an appropriate name. (eg.,
<my-org>-tokkio-automation
)Supported account types: Select Accounts in this organizational directory only.
Ignore all other fields.
Click
Register
.
You will be automatically taken to the created App registration. If not:
From All Services, navigate to
Azure Active Directory
underIdentity
categorySelect
App registrations
from the service configuration panelSelect the created App Registration.
Select
Certificates & secrets
from the resource configuration panel (on the left).Click on the
+New client secret button
to create a new client secret.In the wizard:
Description: Provide an appropriate description.
Expires: Provide the period for which this secret will be valid.
Click
Add
.Copy the Value of the created client secret.
Note
Once you exit this screen, the value will no longer be visible.
Subscription Access grant to App Registration#
From the All Services page:
Select the
General
from the category (on the left).Select the
Subscriptions
service.Select the name of the subscription under which the Tokkio Application will be deployed.
Select
Access control (IAM)
from the resource configuration panel (on the left).Select the
Role Assignments
tab.Click on the
+Add
and then the Add role assignment option to add a new role assignment.In the wizard:
Role: Select Owner.
Members:
Assign access to: Select User, group, or service principal.
Members: Click
+Select
members and select the name of the App Registration.
Click on the
+Add
and then the Add role assignment option to add a new role assignment.In the wizard:
Role: Select Contributor.
Members:
Assign access to: Select User, group, or service principal.
Members: Click
+Select
members and select the name of the App Registration.
Resource Group#
From the All Services page:
Select the
General
from the category (on the left).Select the
Resource groups
service.Click on the
+Create
button to create a new resource group.In the wizard:
Subscription: Choose the subscription under which resources will be created.
Resource group: Provide an appropriate name (eg.,
<<my-org>-tokkio-automation-pre-requisites>
)Region: Choose a region (preferably closest to the users of the application) in which to create the Resource Group.
Navigate Next to optionally configure tags (we will be skipping this through the remainder of this setup).
Click
Review
+create
>Create
.
Deployment State Storage#
From the All Services page:
Select the
Storage
from the category (on the left).Select the
Storage accounts
service.Click on the
+Create
button to create a new storage account.In the wizard:
In the Basics section:
Subscription: Subscription Access grant to App Registration created earlier.
Resource Group: The Resource Group created earlier.
Storage account name: Provide an appropriate name (eg.,
<myorg>tokkiodeploymentstate
)Region: Region chosen for the Resource Group created earlier.
Performance: Choose Standard.
Redundancy: Choose Locally-redundant storage (LRS).
Leave the remaining fields as is.
Leave all other sections as is.
Optionally add tags under the Tags section.
Click
Review
+create
>Create
.
Navigate to the created storage account by clicking on
Go to resource
or:From All Services, navigate to Storage accounts under Storage category
Select the created Storage account.
Select
Containers
from the resource configuration panel (on the left).Click on the
+Container
to create a new container.In the wizard:
Name: Provide an appropriate name (eg: deployment-state)
Public access level: Select Private.
Click Create.
Base Domain#
From the All Services page:
Select the
Web
from the category (on the left).Select the
App Service Domains
service.Click on the
+Create
button to create a new domain.In the wizard:
In the Basics section:
Subscription: Subscription of the Resource Group created earlier.
Resource Group: The Resource Group created earlier.
Domain: Base domain which will be used for subsequent app deployments.
In the Contact information section - Provide the relevant Contact information. Not the email needs to be a valid email ID.
Leave the Advanced section as is.
Optionally add tags under the Tags section.
Click
Review
+create
>Create
.
Certificates#
Key Vault#
From the All Services page:
Select the
Security
from the category (on the left).Select the
Key Vaults
service.Click on the
+Create
button to create a new key vault.In the wizard:
In the Basics section:
Subscription: Subscription of the Resource Group created earlier.
Resource Group: The Resource Group created earlier.
Key vault name: Provide an appropriate name (eg: <my-org>-certificates-vault).
Region: Region chosen for the Resource Group created earlier.
Leave remaining fields as is.
In the Access policy section:
Add an additional Access Policy.
Key Permissions: Select Select All.
Secret Permissions: Select Select All.
Certificate Permissions: Select Select All.
Rotation Policy Operations: Select Select All.
Add the App Registration Service Principal created earlier as Principal.
Let all other configurations remain as is.
Click
Review
+create
>Create
.
Leave the Networking section as is.
Optionally add tags under the Tags section.
Click
Review
+create
>Create
.
Wildcard Certificate#
From the All Services page:
Select the Web from the category (on the left).
Select the App Service Certificates service..
Click on the
+Create
button to create a new certificate.In the wizard:
In the Basics section:
Subscription: Subscription of the Resource Group created earlier.
Resource Group: The Resource Group created earlier.
SKU: Wildcard.
Naked domain hostname: Enter
*.<base-domain>
where<base-domain>
is the name of the Base Domain created earlier.Certificate name: Provide an appropriate name (eg.,
<my-base-domain>-wildcard-certificate
).Enable auto renewal: Optionally select Disable.
Optionally add tags under the Tags section.
Click
Review
+create
>Create
.
Navigate to the created certificate by clicking on
Go to resource
or:From All Services, navigate to App Service Certificates under Web category
Select the created Certificate.
Select
Certificate Configuration
from the resource configuration panel (on the left).Select
Store
and then the Select from Key Vault link.In the wizard:
Subscription: Subscription of the Resource Group created earlier.
Key vault: Name of the Key Vault created earlier.
Navigate back to the created certificate by:
From All Services, navigate to
App Service Certificates
under Web categorySelect the created Certificate.
Select
Certificate Configuration
from the resource configuration panel (on the left).Select
Verify
and then click onVerify
.Wait for Domain Verification to complete.
Increase Quota for GPU VM type#
From the All Services page:
Select the
Other
from the category (on the left).Select
Quotas
service.Click on
Compute
.
Validate sufficient quota is present to create the application virtual machine by doing the following:
Update the Region filter to match the region where the virtual machines will be created.
In the search box, enter
NCASv3_T4
.Check if available usage is at least 64 times the number of instances you wish to run under this setup, if not edit to request for additional quota.
Wait for confirmation that the quota has increased before proceeding.