Google Cloud Platform (GCP)#

Steps to use OneClick script to deploy on GCP.

The cloud deployment uses the same helm chart and the default topology as detailed at Default Deployment Topology and Models in Use.

Note

This release of the OneClick scripts supports single-node deployments to GCP as documented in this page.

Prerequisites#

Host system Prerequisites#

  • Ubuntu 22.04

  • No GPU required

The below tools need to be installed on the OS of the host system for the scripts to execute

  • Install jq

sudo apt update
sudo apt-get install -y jq

  • Install yq

sudo wget https://github.com/mikefarah/yq/releases/download/v4.34.1/yq_linux_amd64 -O /usr/bin/yq
sudo chmod +x /usr/bin/yq

  • Install python3, venv and pip

sudo apt update
sudo apt-get install python3.10 python3-venv python3-pip -y

  • Install OpenTofu

sudo snap install --classic opentofu

Note

Minimum version of OpenTofu must be 1.9.0.

GCP pre-requisites#

We will need an GCP service account to be able to run the installations via one-click scripts.

We will also need an GCP cloud storage bucket to be able to capture the metadata (state) of the deployment. This allows the script to track infrastructure created and be able to destroy it, and consequently save on running costs when it is no longer needed. Not having a storage for this metadata can result in users having to manually delete created resources.

One requires access to a GCP project with admin level privileges to configure the pre-requisites. One can confirm the same by navigating to the Google Cloud Console.

Service Account Setup#

  • Select the IAM & Admin from the left navigation menu.

  • Select the Service Accounts.

  • Click on the Create Service Account button at the top of the Service Accounts page to create new service account.

  • In the wizard, go to section Service account details:

    • Service account name: Provide an appropriate name.

    • Service account ID: It will be auto populated based on the service account name.

    • Service account description: Provide an appropriate description for service account.

    • Click on the Create and Continue button.

  • Go to section Grant this service account access to project.

  • Select an Owner role and click on Continue.

  • Grant users access to this service account section is optional.

  • Click on DONE.

  • You will be automatically taken to the Service accounts pages. If not:

    • Select the IAM & Admin from the left navigation menu.

    • Select the Service Accounts from the left menu.

  • On Service accounts page, it will show all the service accounts available in this project.

  • Identify the service account created using Email or Name field.

  • Click on created Service accounts from the list. It will open page showing all details about service account.

  • Click on KEYS option available in middle of page.

  • Click on ADD KEY and select Create new key option and select JSON as Key type.

  • The key file will be automatically downloaded to your local machine. This Key file contains the private key needed to authenticate as the service account with GCP.

State Storage Setup#

From the Navigation Menu page:

  • Select the Cloud Storage from the category (on the left).

  • Click on the +Create button to create a new storage account.

  • In the wizard:

    • In the Name your bucket section:

      • Provide globally unique name for the bucket. (We are creating this bucket to store deployment state.)

      • Optionally add Labels.

    • In the Choose where to store your data section:

      • Select location type as region.

      • Select appropriate region from drop-down list.

    • Leave all other sections as is.

    • Click CREATE.

1. Get the OneClick deployment package#

1.1 Clone the NVIDIA AI Blueprint: Video Search and Summarization repository.

git clone https://github.com/NVIDIA-AI-Blueprints/video-search-and-summarization.git

1.2 Untar the package

cd video-search-and-summarization/deploy/one_click
tar -xvf deploy-gcp-cns.tar.gz

2. Prepare env variables#

2.1 Prepare a file, via-env-template.txt, to hold required env variables and their values:

via-env-template.txt

#Env: GCP secrets
export CLOUDSDK_CORE_PROJECT='project-id'
export CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE='/path to/service-account-key.json'

#Env: Nvidia Secrets:
export NGC_API_KEY=

#Env: OpenAI Secrets:
export OPENAI_API_KEY=

#Env: App secrets
export VIA_DB_PASSWORD=password

#Non secrets:

#GCP Resources created above in Section: GCP pre-requisites
export VIA_DEPLOY_GCP_GCSB='gcp-bucket-name'

#Unique name for the VSS deployment
export VIA_DEPLOY_ENV='vss-deployment'

Note

  • NGC_API_KEY: refer to Obtain NGC API Key.

  • OPENAI_API_KEY: API key from https://platform.openai.com/api-keys.

  • You may consider updating VIA_DEPLOY_ENV to something other than default to identify the deployment. E.g. VIA_DEPLOY_ENV='vss-demo'

  • Make sure to update CLOUDSDK_CORE_PROJECT, CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE and VIA_DEPLOY_GCP_BUCKET to reflect the GCP Resource created above in Section: GCP pre-requisites.

2.2 Load these env variables into your current shell session:

source via-env-template.txt

3. Prepare config file#

Make a copy of config-template.yml of your own choice, eg config.yml. Or you can populate the config file as based on definition of each attribute.

example config.yml

schema_version: '0.0.10'
name: "via-gcp-cns-{{ lookup('env', 'VIA_DEPLOY_ENV') }}"
spec:
  infra:
    csp: 'gcp'
    backend:
      bucket: "{{ lookup('env', 'VIA_DEPLOY_GCP_GCSB') }}"
      credentials: "{{ lookup('env', 'CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE') }}"
    provider:
      project: "{{ lookup('env', 'CLOUDSDK_CORE_PROJECT') }}"
      credentials: "{{ lookup('env', 'CLOUDSDK_AUTH_CREDENTIAL_FILE_OVERRIDE') }}"
    configs:
      cns:
        version: 12.2
        git_ref: ebf03a899f13a0adaaa3d9f31a299cff6ab33eb3
        override_values:
          cns_nvidia_driver: yes
          gpu_driver_version: '535.216.03'
      access_cidrs:
      - 'my-org-ip-cidr'   ### Make sure to update this e.g. 'xxx.xxx.xxx.xxx/32' for a single IP address.
      region: 'us-west1'   ### Update this to change the deployment region
      ssh_public_key: "{{ lookup('file', lookup('env', 'HOME') + '/.ssh/id_rsa.pub') }}"
      ssh_private_key_path: "{{ lookup('env', 'HOME') + '/.ssh/id_rsa' }}"
      additional_ssh_public_keys:
      ### Add public key of another system e.g. your colleague's if you are working in a team else add your own.
      - "{{ lookup('file', lookup('env', 'HOME') + '/.ssh/id_rsa.pub') }}"
      clusters:
        app:
          private_instance: false
          master:
            ### Modify this to change the GPU type.
            type: 'a3-megagpu-8g'
            labels: {}
            taints: []
          ports:
            backend:
              port: 30081
            frontend:
              port: 30082
          features:
            cns: true
            platform: true
            app: true
  platform:
    configs:
      namespace: 'default'
  app:
    configs:
      namespace: 'default'
      backend_port: 'backend'
      frontend_port: 'frontend'
      ngc_api_key: "{{ lookup('env', 'NGC_API_KEY') }}"
      openai_api_key: "{{ lookup('env', 'OPENAI_API_KEY') }}"
      db_username: 'neo4j'
      db_password: "{{ lookup('env', 'VIA_DB_PASSWORD') | default('password') }}"
      vss_chart:
        repo:
          name: 'nvidia-blueprint'
          url: 'https://helm.ngc.nvidia.com/nvidia/blueprint'
        chart: 'nvidia-blueprint-vss' # repo should be removed/commented-out when using local charts
        version: '2.3.1'
        #override_values_file_absolute_path: '/home/user/vss-values.yml'

Note

The above is just a reference. In case you face issues, please make a copy of config-template.yml and update it as required.

Note

Make sure to update config sections:

region (spec/infra/configs/), the machine type,

access_cidrs : run echo `curl ifconfig.me`/32 to get user machine’s IP range.

override_values_file_absolute_path (optional) in the config file: To use the overrides.yaml file as discussed in sections like Configuration Options.

To use a helm overrides values file to customize the various parts of the VSS blueprint deployment:

  • Uncomment and update override_values_file_absolute_path shown above in config.yml to set the actual path to the overrides file.

  • Uncomment line - "{{ configs.vss_chart.override_values_file_absolute_path }}" in dist/app-tasks.yml near the end of the file.

  • More information on the VSS helm overrides file can be found in Section Configuration Options.

Attributes of the config-template.yml

Attribute

Optional

Description

name

A unique name to identify the infrastructure resources being created by.

spec > infra > backend > bucket

Name of the GCP Cloud Storage bucket in which state of the resources provisioned is stored.

spec > infra > backend > credentials

Path of the GCP service account credential used to access the backend bucket and table.

spec > infra > provider > project

The name of the GCP project used to provision resources.

spec > infra > provider > credentials

Path of the GCP service account credential used to access the backend bucket and table.

spec > infra > configs > cns

yes

CNS configurations.

spec > infra > configs > cns > version

yes

The version of CNS to install on the clusters. Defaults to 12.2.

spec > infra > configs > cns > override_values

yes

CNS values to override while setting up a cluster.

spec > infra > configs > cns > override_values > cns_value

yes

The value of the cns_value found in cns_values.yaml.

spec > infra > configs > access_cidrs

List of CIDRs from which app will be accessible.

spec > infra > configs > region

GCP region in which to bring up the resources.

spec > infra > configs > ssh_private_key_path

Absolute path of the private key to be used to SSH the hosts.

spec > infra > configs > ssh_public_key

Content of the public counterpart of the private key used to SSH the hosts.

spec > infra > configs > additional_ssh_public_keys

yes

List of contents of public counterparts to the additional keys that will be used to SSH the hosts.

spec > infra > configs > bastion

yes

Details of the GCP instance to be used as a bastion host in case of private clusters.

spec > infra > configs > bastion > type

yes

GCP instance type for the bastion node (if required). Defaults to e2-medium.

spec > infra > configs > bastion > zone

yes

GCP availability zone in the region for the bastion node (if required). Defaults to the first (alphabetically) AZ of the region.

spec > infra > configs > bastion > disk_size_gb

yes

Root volume disk size for the bastion node. Defaults to 128.

spec > infra > configs > clusters

Definitions of clusters to be created.

spec > infra > configs > clusters > cluster

Unique key to identify a cluster. There can be 1 or more clusters.

spec > infra > configs > clusters > cluster > private_instance

yes

If true, creates the cluster instances within a private subnet. Defaults to false

spec > infra > configs > clusters > cluster > zone

yes

GCP availability zone in the region for the master and nodes of the cluster. Defaults to the first (alphabetically) AZ of the region.

spec > infra > configs > clusters > cluster > master

Definitions of the master node of the cluster.

spec > infra > configs > clusters > cluster > master > type

yes

GCP instance type for the master node. Defaults to a2-ultragpu-8g.

spec > infra > configs > clusters > cluster > master > guest_accelerators

yes

List of guest accelerators to attach to the GCP instance type for the master node. Defaults to none.

spec > infra > configs > clusters > cluster > master > guest_accelerators > index

Details of one of the guest accelerators to attach to the GCP instance type for the master node.

spec > infra > configs > clusters > cluster > master > guest_accelerators > index > type

The type of the index guest accelerators for the master node.

spec > infra > configs > clusters > cluster > master > guest_accelerators > index > count

The count of the index guest accelerators for the master node.

spec > infra > configs > clusters > cluster > master > disk_size_gb

yes

Root volume disk size for the master node. Defaults to 1024.

spec > infra > configs > clusters > cluster > master > labels

yes

Labels to apply to the master node. Defaults to {}.

spec > infra > configs > clusters > cluster > master > taints

yes

Taints to apply to the master node. Defaults to [].

spec > infra > configs > clusters > cluster > nodes

yes

Definitions of nodes of the cluster. Set to {} if no extra nodes other than master needed.

spec > infra > configs > clusters > cluster > nodes > node

Unique key to identify a node. There can be 0 or more nodes.

spec > infra > configs > clusters > cluster > nodes > node > type

yes

GCP instance type for the node node. Defaults to a2-ultragpu-8g.

spec > infra > configs > clusters > cluster > nodes > node > guest_accelerators

yes

List of guest accelerators to attach to the GCP instance type for the node node. Defaults to none.

spec > infra > configs > clusters > cluster > nodes > node > guest_accelerators > index

Details of one of the guest accelerators to attach to the GCP instance type for the node node.

spec > infra > configs > clusters > cluster > nodes > node > guest_accelerators > index > type

The type of the index guest accelerators for the node node.

spec > infra > configs > clusters > cluster > nodes > node > guest_accelerators > index > count

The count of the index guest accelerators for the node node.

spec > infra > configs > clusters > cluster > nodes > node > disk_size_gb

yes

Root volume disk size for the node node. Defaults to 1024.

spec > infra > configs > clusters > cluster > nodes > node > labels

yes

Labels to apply to the node node. Defaults to {}.

spec > infra > configs > clusters > cluster > nodes > node > taints

yes

Taints to apply to the node node. Defaults to [].

spec > infra > configs > clusters > cluster > ports

yes

Definitions of ports of the cluster. Set to {} if no ports are exposed by the cluster.

spec > infra > configs > clusters > cluster > ports > port

Unique key to identify a port. There can be 0 or more ports.

spec > infra > configs > clusters > cluster > ports > port > port

The port number of the port.

spec > infra > configs > clusters > cluster > ports > port > protocol

yes

The protocol of the port. Defaults to http.

spec > infra > configs > clusters > cluster > ports > port > path

yes

The path of the application on the port for the landing URL. Defaults to /.

spec > infra > configs > clusters > cluster > features

yes

Definitions of features of the cluster. Set to {} if no features defined for the cluster.

spec > infra > configs > clusters > cluster > features > feature

Key to identify a feature and value represents enabled/disabled by setting it to true/false. There can be 0 or more features.

spec > platform > configs > namespace

yes

Namespace to deploy the platform components in. Defaults to default.

spec > app > configs > namespace

yes

Namespace to deploy the app in. Defaults to default.

spec > app > configs > backend_port

Identifier of the port in the cluster to expose the api over.

spec > app > configs > frontend_port

Identifier of the port in the cluster to expose the ui over.

spec > app > configs > ngc_api_key

NGC API key used to download application charts, models and containers.

spec > app > configs > openai_api_key

OPENAI API key used by the application.

spec > app > configs > db_username

The username used to access the DB.

spec > app > configs > db_password

The password used to access the DB.

spec > app > configs > vss_chart

Configuration details of the VSS chart.

spec > app > configs > vss_chart > repo

yes

Helm repo details of the chart. Can be ignored if using a local chart.

spec > app > configs > vss_chart > repo > name

Name provided to refer the added helm repo.

spec > app > configs > vss_chart > repo > url

Url of the helm repo containing the chart.

spec > app > configs > vss_chart > chart

The name of the chart in case of a remote repo source. The absolute path of a local chart.

spec > app > configs > vss_chart > version

The version of the chart.

4. Run OneClick script to deploy on GCP#

4.1 Make sure the host machine to run OneClick script has rsa keys generated. If not, use the following command to generate

sudo apt-get install -y openssh-client
ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa

4.2 Deployment

Use config from 2. Prepare env variables. Place it in the dist directory.

4.2.1 Choose machine type

Default machine type is: H100 GPU. Users may change this to other configurations. This change can be made in the config.yaml file available in Section: 2. Prepare env variables.

### Default is: H100 GPU
### Modify this to change the GPU type.
### Alternatives:
### - a2-ultragpu-8g (8 x A100 80GB)
### - a3-megagpu-8g (8 x H100 80GB)
type: 'a3-megagpu-8g'

More info on the machine types available can be found in the GCP Compute Engine documentation here.

4.2.2 Deploy

cd dist/
./envbuild.sh install -f config.yml -c all

Note

In case you face an error like could not process config file: ... while restarting/redeploying, try removing the temporary directory that is shown in the error logs. Example: rm -rf <dist-directory>/tmp.dZM7is5HUC

Please also confirm SSH key was generated/present on the path mentioned in the config for keys - ssh_public_key, ssh_private_key_path. If there is no need for additional_ssh_public_keys, please comment that out in the config file.

Note

This project downloads and installs additional third-party open source software projects. Review the license terms of these open source projects before use.

5. Access the deployment#

Once successful, the above command will provide the logs similar to the following.

access_urls:
app:
    backend: http://<NODE-IP>:30081/
    frontend: http://<NODE-IP>:30082/
ssh_command:
app:
    master: ssh -i $HOME/.ssh/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null ubuntu@<NODE-IP>

Note

You must wait till the deployment installation is fully complete before trying to access the nodes.

We can also get this info after successful deployment on demand using command:

cd dist/
./envbuild.sh -f config.yml info

Next, we need to wait for all pods and services to be up. Log in to the node using the ssh command shown above and check pod status using kubectl get pod.

ssh -i $HOME/.ssh/id_rsa -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null ubuntu@<NODE-IP>
kubectl get pod

Make sure all pods are in Running or Completed STATUS and shows 1/1 as READY as shown below.

Create new secret key

Note

The terraform scripts will install the kubectl utility. Users must not install kubernetes or kubectl manually.

Additionally, to make sure the VSS API and UI are ready and accessible, please check logs for deployment using command:

kubectl logs vss-vss-deployment-POD-NAME

Please make sure the below logs are present and user does not see any errors:

VIA Server loaded
Backend is running at http://0.0.0.0:8000
Frontend is running at http://0.0.0.0:9000

The VSS API and UI are now ready to be accessed at http://<NODE-IP>:30081 and http://<NODE-IP>:30082 respectively. Test the deployment by summarizing a sample video.

6. Teardown#

Un-installing

$ cd dist/
$ ./envbuild.sh uninstall -f config.yml -c all

Common Issues#

VSS pod is failing and restarting on L4 node#

The VSS container startup might be timing out on an L4 node when VILA1.5 is used as the VLM (default). Try increasing the startup timeout by using an overides file with following values:

vss:
  applicationSpecs:
    vss-deployment:
      containers:
        vss:
          startupProbe:
            failureThreshold: 360
On this page