Creating an Application

Using Textual Representation and CLI Tools

Set the NGC API Key

Set the NGC API key to be able to sync microservices from NGC:

Get the NGC API Key - Refer to Generating Your NGC API Key.

$ ucf_app_builder_cli registry repo set-api-key -a <API-KEY>
Repository 'UCF' added

The command will print the repositories that are accessible using the API key.

List Available Microservices

To get the list of available microservices:

$ ucf_app_builder_cli service list
List of available services in local repository:
==============================================
ucf.svc.myservice

List of available services in UCF repository:
================================================
ucf.svc.botmaker.dialog-manager
ucf.svc.botmaker.speech-controller
ucf.svc.botmaker.speech-web-app
ucf.svc.botmaker.text-web-app
ucf.svc.riva.speech-skills

The command lists available microservices built locally as well as in NGC repository.

View Information of Microservices

To view information about microservices, run:

$ ucf_app_builder_cli service info -n ucf.svc.botmaker.speech-web-app
name: ucf.svc.botmaker.speech-web-app
specVersion: 1.5.0
chart: https://helm.ngc.nvidia.com/eevaigoeixww/ucf-11-ea-release/charts/botmaker-speech-web-app-2.0.0.tgz
description: BotMaker Speech Web App
type: msapplication
tags: []
keywords: []
publish: false
ciTrigger: false
egress:
-   name: speech-controller
    description: Speech Controller gRCPC endpoint
    protocol: TCP
    scheme: grpc
    mandatory: true
    data-flow: in-out
ingress:
-   name: webapp-ui
    description: Web App UI Endpoint
    scheme: http
    data-flow: in-out
    service: botmaker-speech-web-app-deployment-service
    port: 5001
    protocol: TCP
metadata: {}
version: 2.0.0
displayName: BotMaker Speech Web App
category:
    functional: Speech AI
    industry: General
secrets: []
buildToolVersion: 1.1.1

Parameters:
-------------
               debug: (boolean), Start bot in debug mode [Mandatory:False]
            feedback: (boolean), Enables feedback buttons in web mode [Mandatory:False]
     gunicornWorkers: (string ), Number of Gunicorn Workers for FM Server [Mandatory:False]
     gunicornTimeout: (string ), Gunicorn Timeout for FM Server [Mandatory:False]

Compliance Info:
---------------------
Report Generated on 2022-11-22 18:24:51 (UTC) using MSBuilder v1.1.1
Development compliance (Mandatory) 100.0%
Development compliance (Optional) 66.66%

Based on this information, users can choose the microservices to use and how to use them in the applications - setting parameters, connecting endpoints etc.,

Adding --show-docs to the above command, will print detailed documentation for the microservice.

$ ucf_app_builder_cli service info -n ucf.svc.botmaker.speech-web-app --show-docs
...
Documentation:
--------------
╔═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╗
║                                              ucf.svc.botmaker.speech-web-app                                              ║
╚═══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╝


                                                         Description

BotMaker Speech Web App

Sample web application that communicates with Speech Controller and can be used to interact with speech based bot using web
browser.


                                                            Usage

                                                           Params:

┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ botmaker-webapp:                                                                                                          │
│   debug: false                                                                                                            │
│   # Show debug info for bot in Web App                                                                                    │
│   feedback: true                                                                                                          │
│   # Enables feedback buttons in web mode                                                                                  │
│   gunicornWorkers: "1"                                                                                                    │
│   # Number of Gunicorn Workers for Web App                                                                                │
│   gunicornTimeout: "120"                                                                                                  │
│   # Gunicorn Timeout for Web App                                                                                          │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

                                                        Connections:

┌───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ connections:                                                                                                              │
│   botmaker-webapp/speech-controller: speech-controller/grpc-api                                                           │
└───────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘


                                                         Performance

NA


                                                     Supported Platforms

x86 dGPU


                                                   Deployment requirements

 1 Make sure k8s foundational services are running


                                                           License

NVIDIA Proprietary License


                                                 Known Issues / Limitations

NA


                                                         References

NA

Create Application Boilerplate

Now that the microservices that will be required have been chosen, we can start creating the application.

We will first create a boilerplate and then start updating it to match the reference application Update Application.

Run the following command to generate a application boilerplate:

$ ucf_app_builder_cli app create my-application
UCF Application graph boilerplate generated at my-application.yaml (Doc: README.md)

This will generate a new file my-application.yaml along with README.md for the application documentation.

The file is a template with contents:

specVersion: '1.5.0'

version: 0.0.1

doc: README.md

name: my-application

description: Description for my-application

dependencies:
- ucf.svc.microserviceA:0.0.1
- ucf.svc.microserviceB:0.0.1

components:
- name: microserviceA
  type: ucf.service.microserviceA
  parameters:
    paramA1: 100
    paramA2:
    - name: test
  secrets:
    some-secret-name: some-secret
- name: microserviceB
  type: ucf.service.microserviceB
  parameters:
    paramB1: false
    paramB2:
    - name: test

connections:
  microserviceA/http: microserviceB/http

vaultAgent:
  auth:
    path: auth/jwt/authMountPath
    type: jwt
    jwt:
      audience: https://vault-service:443
      role: auth-role
  role: auth-role
  namespace: vault-namespace
  service: https://vault-service/

# certificates:
#   devCerts:
#     file: certificates.yaml
#     addToHelmChart: false

secrets:
  some-secret:
    vaultAgent:
      path: secret/mount/path
      template:
        type: kv
        key: KEY_IN_SECRET

  # some-secret:
  #   k8sSecret:
  #     secretName: <k8sSecretName>
  #     key: <keyInK8sSecret>

  # some-secret:
  #   secretsStoreCsi:
  #     providerClassName: <providerClassName>
  #     objectName: <objectName>

  # some-secret:
  #   certificate:
  #     certName: <certificateName>
  #     type: <certificate/privateKey/issuingCA>

The various parts of the application YAML file are explained in Sample Application - Textual Format.

Note

App name shouldn’t contain spaces

Update App Information

Update the basic information for the application like name, short description etc. For more information on the fields refer to Basic Information.

After updating, the section would look like:

specVersion: 1.5.0

version: 1.13.1

doc: README.md

name: ucf-bot-app

description: UCF application for a bot

Update Dependencies

Add the microservices to be used in the application to the dependencies section. This information is required to fetch desired version of microservices when building the application.

A list of microservices and their desired versions must be added to this section. The format is a list of strings <microservice-type>:<microservice-version>. The microservice version can be obtained from the service info command described above.

To view all available versions of a microservice, run:

$ ucf_app_builder_cli service versions -n ucf.svc.botmaker.speech-web-app
List of versions for service in UCF repository:
===============================================
2.0.0

<microservice-version> can be a fixed version or in the format of NPM version range spec https://github.com/npm/node-semver#ranges

After updating the section would look like:

dependencies:
- ucf.svc.riva.speech-skills:2.0.0
- ucf.svc.botmaker.speech-web-app:>=2.0.0 && < 3.0.0
- ucf.svc.botmaker.speech-controller:2.0.x
- ucf.svc.botmaker.dialog-manager:2.0.0

In the above example, when the application is built, the tools will fetch the highest version that matches the version specification for that microservice.

• For ucf.svc.riva.speech-skills and ucf.svc.botmaker.dialog-manager, version 2.0.0 will be picked

• For ucf.svc.botmaker.speech-web-app, if the available versions are 1.0.0, 2.0.0, and 3.0.0, version 3.0.0 will be picked

• For ucf.svc.botmaker.speech-controller, if the available versions are 1.0.0, 2.0.0 and 2.1.0, version 2.0.0 will be picked

This allows latest compatible versions of microservices to be picked when building applications without having to update the application graph file.

UCF Microservices follow semantic versioning - <MAJOR>.<MINOR>.<PATCH>

• Changes in <MAJOR> part of version indicate major changes with possible breakage in backward compatibility.

• Changes in <MINOR> part of version indicate enhancements done to the microservice with backward compatibility preserved.

• Changes in <PATCH> part of version indicate minor changes like bug fixes done to the microservice with backward compatibility preserved.

Add and Configure Microservices

Next, we will add microservices to the application and configure them i.e set their parameters and set secrets. This is done by adding microservices to the components section.

For example, ucf.svc.botmaker.speech-web-app can be added to the application, using:

- name: botmaker-webapp
  type: ucf.svc.botmaker.speech-web-app
  parameters:
    debug: false
    gunicornTimeout: "60"
    imagePullSecrets:
    - name: ngc-docker-reg-secret

The parameters must be set according to the parameter schema for the microservice which can be obtained from the service info command output. For example, for the ucf.svc.botmaker.speech-web-app:

$ ucf_app_builder_cli service info -n ucf.svc.botmaker.speech-web-app
...
Parameters:
-------------
              speech: (boolean), Speech Web App or Text Web App [Mandatory:False]
               debug: (boolean), Start bot in debug mode [Mandatory:False]
            feedback: (boolean), Disables feedback buttons in web mode [Mandatory:False]
     gunicornWorkers: (string ), Number of Gunicorn Workers for FM Server [Mandatory:False]
     gunicornTimeout: (string ), Gunicorn Timeout for FM Server [Mandatory:False]
...

Here, debug is a boolean parameter and gunicornTimeout is a string parameter. Parameters that are marked mandatory must be specified. Some microservices have nested parameters as seen in the example below.

UCF application build tools will validate these parameters. During application build, the build tools will throw an error if mandatory parameters are not set or they do not match parameter schema. For example if debug is set to "123":

Validating application ...
debug:'123' is not of type 'boolean'
Warning!!! parameter validation failed for microservice instance botmaker-webapp
Application validation failed

After adding all the required microservices for the application, the components section looks like:

components:
- name: riva-speech
  type: ucf.svc.riva.speech-skills
  parameters:
    buildModelEngines: 'true'
    ngcModelConfigs:
    - nvidia/riva/rmir_asr_citrinet_1024_en_us_str:2.6.0
    - nvidia/riva/rmir_tts_fastpitch_hifigan_en_us:2.6.0
    - eevaigoeixww/ucf-11-ea-release/misty_text_classification:1.7.3-ea
    - eevaigoeixww/ucf-11-ea-release/intent_slot_weather:1.7.3-ea
    - eevaigoeixww/ucf-11-ea-release/intent_slot_poi:1.7.3-ea
    - eevaigoeixww/ucf-11-ea-release/intent_slot_smalltalk:1.7.3-ea
    - nvidia/riva/rmir_nlp_question_answering_bert_base:2.6.0
    - nvidia/riva/rmir_nlp_named_entity_recognition_bert_base:2.6.0
    imagePullSecrets:
    - name: ngc-docker-reg-secret

- name: speech-web-app
  type: ucf.svc.botmaker.speech-web-app
  parameters:
    debug: false
    gunicornTimeout: "60"
    imagePullSecrets:
    - name: ngc-docker-reg-secret

- name: speech-controller
  type: ucf.svc.botmaker.speech-controller
  parameters:
    pipeline: speech_lite
    botNgcPath: eevaigoeixww/ucf-11-ea-release/misty_bot:2.0.0-ea-x86_64
    wordBoostFilePath: ''
    imagePullSecrets:
    - name: ngc-docker-reg-secret

- name: dialog-manager
  type: ucf.svc.botmaker.dialog-manager
  parameters:
    botNgcPath: eevaigoeixww/ucf-11-ea-release/misty_bot:2.0.0-ea-x86_64
    botConfigName: misty_bot_config.yaml
    botConfig:
      fulfillments:
        weather:
          parameters:
            api-key: XXXXXXXXXXXXXXXXXXX
        poi:
          parameters:
            api-key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            here-api-key: XXXXXXXXXXXXXXXXXXXXXX
        map:
          parameters:
            api-key: XXXXXXXXXXXXXXXXXXXXXX
    imagePullSecrets:
    - name: ngc-docker-reg-secret

For microservices whose helm charts are built using UCF MS Builder Tool, some parameters are implicitly added. These parameters are not shown in the service info output. For example, imagePullSecrets is an implicit parameter. Implicitly added parameters are described in the next section.

Implicitly Added Parameters

As mentioned in previous section, microservices whose helm charts are generated using the UCF MS Builder Tool have some parameters added implicitly.

Following is an example of how to set these parameters along with their details:

- name: <svc-name>
  type: <svc-ucf-type>
  parameters:
    imagePullSecrets:        # List of K8S secrets to be used for pulling images required by the MS (Since 1.0 EA)
    - name: ngc-docker-reg-secret

    ucfVisibleGpus: [1, 2]   # List of GPUs that should be made visible to the containers in the MS. Internally sets NVIDIA_VISIBLE_GPUS env var (Since 1.0 EA)

    resources:               # Resource requests and limits to set on the containers in the MS. (Since 1.0 EA)
      requests:              # Refer to - https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#resourcerequirements-v1-core
        cpu: 1
      limits:
        nvidia.com/gpu: 1

    serviceAccountName: name-of-sa   # Name of the service account to use to run the pods belonging to the MS (Since 1.1 EA)

    podAnnotations:          # Annotations to set on the pods belonging to the MS (Since 1.1 EA)
        annotation-name: annotation-value

    podLabels:               # Labels to set on the pods belonging to the MS (Since 1.1 EA)
        label-name: label-value

    extraPodVolumes:         # List of extra volumes to add to all the pods belonging to the MS (Since 1.1 EA)
    - name: extra-vol1       # Refer to https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.22/#volume-v1-core
      emptyDir: {}

    extraPodVolumeMounts:    # List of extra volume mounts to add to all the containers of all the pods belonging to the MS (Since 1.1 EA)
    - name: extra-vol1:
      mountPath: /tmp/extra-vol-dir

    svcAnnotations:          # Annotations to set on the services belonging to the MS (Since 1.1 EA)
        annotation-name: annotation-value

    svcLabels:               # Labels to set on the services belonging to the MS (Since 1.1 EA)
        label-name: label-value

    global:
        ucfGlobalEnv:        # List of environment variables to set in all containers of all the pods belonging to the MS (Since 1.1 EA)
        - name: SOME_ENV_VAR_NAME
          value: someEnvVarValue

Configuring Secrets

All mandatory secrets for each microservice must be specified. The secrets requirement can be viewed using the service info command.

Refer to Secrets Management for more information on how secrets work in UCF and how to configure them.

Connect Microservice Endpoints

Next we connect microservices together. Microservices that have mandatory Egress endpoints must be connected to another meaningful microservice that have corresponding and compatible Ingress endpoints that provide the necessary functionality required.

The endpoints of a microservice can be obtained from the service info command output. For example, for the ucf.svc.botmaker.speech-controller:

$ ucf_app_builder_cli service info -n ucf.svc.botmaker.speech-controller
...
egress:
-   name: redis
    description: Redis message broker
    protocol: TCP
    scheme: asyncio
    mandatory: false
    data-flow: in-out
-   name: riva
    description: Riva Speech Skills API
    scheme: grpc
    protocol: TCP
    mandatory: true
    data-flow: in-out
-   name: dialog-manager
    description: Dialog Manager API
    scheme: http
    protocol: TCP
    mandatory: true
    data-flow: in-out
-   name: ov-http
    description: OV A2F service HTTP API
    scheme: http
    protocol: TCP
    mandatory: false
    data-flow: out
-   name: ov-a2f-grpc
    description: OV A2F service GRPC API
    scheme: grpc
    protocol: TCP
    mandatory: false
    data-flow: out
ingress:
-   name: grpc-api
    description: Speech Controller GRPC API
    scheme: grpc
    data-flow: in-out
    service: botmaker-speech-controller-deployment-svc
    port: 50055
    protocol: TCP
...

From the output above, riva is mandatory Egress endpoint. The corresponding required functionality is provided by the ucf.svc.riva.speech-skills microservice.

$ ucf_app_builder_cli service info -n ucf.svc.riva.speech-skills
...
ingress:
-   name: riva-speech-api
    description: Riva Speech GRPC API
    scheme: grpc
    data-flow: in-out
    service: riva-speech-riva-speech-svc
    port: 50051
    protocol: TCP
...

So we can connect ucf.svc.botmaker.speech-controller’s (component name- speech-controller) riva Egress endpoint to ucf.svc.riva.speech-skills’s (component name - riva-speech) riva-speech-api Ingress endpoint.

Connections are specified as an object with top-level key connections in the application yaml. The connections are in the format:

connections:
  <egress-endpoint-component-name1>/<egress-endpoint-name1>: <ingress-endpoint-component-name1>/<ingress-endpoint-name1>
  <egress-endpoint-component-name2>/<egress-endpoint-name2>: <ingress-endpoint-component-name2>/<ingress-endpoint-name2>

So the above connection can be added as:

connections:
  speech-controller/riva: riva-speech/riva-speech-api

In case the egress endpoint supports multiple connections (multi: true), multiple ingress endpoints can be connected to it using array style format:

connections:
  <egress-endpoint-component-name>/<egress-endpoint-name>: [
      <ingress-endpoint-component-name1>/<ingress-endpoint-name1>,
      <ingress-endpoint-component-name2>/<ingress-endpoint-name2>
  ]

UCF application build tools will validate the endpoints and connections. If a mandatory Egress endpoint is not connected or an incase of an invalid connection, the tools will throw an error like:

Validating application ...
Warning!!! Invalid connection 'dialog-manager/redis' (scheme: asyncio, protocol: TCP) -> 'riva-speech/riva-speech-api' (scheme: grpc, protocol: TCP)
Warning!!! Mandatory endpoint 'speech-controller' not set for microservice instance speech-web-app
Application validation failed

And after adding all the connections, the section would look like:

connections:
  speech-web-app/speech-controller: speech-controller/grpc-api
  dialog-manager/riva-speech: riva-speech/riva-speech-api
  dialog-manager/triton-grpc: riva-speech/triton-grpc
  speech-controller/riva: riva-speech/riva-speech-api
  speech-controller/dialog-manager: dialog-manager/dialog-manager

Add Detailed Documentation

Add any detailed documentation required for building the application and then deploying the built helm chart to the README file accompanying the application graph file.

For example the reference application README contains instructions such as:

## Test environment
Instructions in this document were tested with the following environment.

- Ubuntu: 20.04
- Kubernetes: microk8s
- NVIDIA GPU Operator running
- MDX Local Path Provisioner running

...

## Deploy app

One time step to create the k8s secret for NGC docker registry and NGC CLI API KEY
```
kubectl create secret docker-registry ngc-docker-reg-secret --docker-server=nvcr.io --docker-username='$oauthtoken' --docker-password=<NGC_API_KEY>

kubectl create secret generic ngc-api-key-secret --from-literal=NGC_API_KEY=<NGC_API_KEY>

...

## Interacting with the Bot

Get Nodeport for `botmaker-speech-web-app-deployment-service` using `kubectl get svc` command.
You can access web agent using browser at **https://<Node_IP>:<Nodeport_>/bot**.

Final Application Graph File

After all the changes are done, the final application graph file should look like:

specVersion: 1.5.0

version: 1.13.1

doc: README.md

name: ucf-bot-app

description: UCF application for a bot

dependencies:
- ucf.svc.riva.speech-skills:2.0.0
- ucf.svc.botmaker.speech-web-app:2.0.0
- ucf.svc.botmaker.speech-controller:2.0.0
- ucf.svc.botmaker.dialog-manager:2.0.0

components:
- name: riva-speech
  type: ucf.svc.riva.speech-skills
  parameters:
    buildModelEngines: 'true'
    ngcModelConfigs:
    - nvidia/riva/rmir_asr_citrinet_1024_en_us_str:2.6.0
    - nvidia/riva/rmir_tts_fastpitch_hifigan_en_us:2.6.0
    - eevaigoeixww/ucf-11-ea-release/misty_text_classification:1.7.3-ea
    - eevaigoeixww/ucf-11-ea-release/intent_slot_weather:1.7.3-ea
    - eevaigoeixww/ucf-11-ea-release/intent_slot_poi:1.7.3-ea
    - eevaigoeixww/ucf-11-ea-release/intent_slot_smalltalk:1.7.3-ea
    - nvidia/riva/rmir_nlp_question_answering_bert_base:2.6.0
    - nvidia/riva/rmir_nlp_named_entity_recognition_bert_base:2.6.0
    imagePullSecrets:
    - name: ngc-docker-reg-secret
  secrets:
    ngc-api-key-secret: my-ngc-api-key

- name: speech-web-app
  type: ucf.svc.botmaker.speech-web-app
  parameters:
    debug: false
    gunicornTimeout: "60"
    imagePullSecrets:
    - name: ngc-docker-reg-secret

- name: speech-controller
  type: ucf.svc.botmaker.speech-controller
  parameters:
    pipeline: speech_lite
    botNgcPath: eevaigoeixww/ucf-11-ea-release/misty_bot:2.0.0-ea-x86_64
    wordBoostFilePath: ''
    imagePullSecrets:
    - name: ngc-docker-reg-secret
  secrets:
    ngc-api-key-secret: my-ngc-api-key

- name: dialog-manager
  type: ucf.svc.botmaker.dialog-manager
  parameters:
    botNgcPath: eevaigoeixww/ucf-11-ea-release/misty_bot:2.0.0-ea-x86_64
    botConfigName: misty_bot_config.yaml
    botConfig:
      fulfillments:
        weather:
          parameters:
            api-key: XXXXXXXXXXXXXXXXXXX
        poi:
          parameters:
            api-key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
            here-api-key: XXXXXXXXXXXXXXXXXXXXXX
        map:
          parameters:
            api-key: XXXXXXXXXXXXXXXXXXXXXX
    imagePullSecrets:
    - name: ngc-docker-reg-secret
  secrets:
    ngc-api-key-secret: my-ngc-api-key

connections:
  speech-web-app/speech-controller: speech-controller/grpc-api
  dialog-manager/riva-speech: riva-speech/riva-speech-api
  dialog-manager/triton-grpc: riva-speech/triton-grpc
  speech-controller/riva: riva-speech/riva-speech-api
  speech-controller/dialog-manager: dialog-manager/dialog-manager

secrets:
  my-ngc-api-key:
    k8sSecret:
      secretName: ngc-api-key-secret
      key: NGC_API_KEY

Parameters File

The parameters of the microservices can be extracted into a separate file and can be used during application build time. This allows the same application graph file to be used but with different configurations of microservices e.g different set of parameters for development and production environments.

The format of the the parameters file is a set of key-value pairs where key is the name of the microservice in the application and value is an object of the microservice parameters to set. For example, for the reference application, the parameters file would look like:

riva-speech:
  buildModelEngines: 'true'
  ngcModelConfigs:
  - nvidia/riva/rmir_asr_citrinet_1024_en_us_str:2.6.0
  - nvidia/riva/rmir_tts_fastpitch_hifigan_en_us:2.6.0
  - eevaigoeixww/ucf-11-ea-release/misty_text_classification:1.7.3-ea
  - eevaigoeixww/ucf-11-ea-release/intent_slot_weather:1.7.3-ea
  - eevaigoeixww/ucf-11-ea-release/intent_slot_poi:1.7.3-ea
  - eevaigoeixww/ucf-11-ea-release/intent_slot_smalltalk:1.7.3-ea
  - nvidia/riva/rmir_nlp_question_answering_bert_base:2.6.0
  - nvidia/riva/rmir_nlp_named_entity_recognition_bert_base:2.6.0
  imagePullSecrets:
  - name: ngc-docker-reg-secret

speech-web-app:
  debug: false
  gunicornTimeout: "60"
  imagePullSecrets:
  - name: ngc-docker-reg-secret

speech-controller:
  pipeline: speech_lite
  botNgcPath: eevaigoeixww/ucf-11-ea-release/misty_bot:2.0.0-ea-x86_64
  wordBoostFilePath: ''
  imagePullSecrets:
  - name: ngc-docker-reg-secret

dialog-manager:
  botNgcPath: eevaigoeixww/ucf-11-ea-release/misty_bot:2.0.0-ea-x86_64
  botConfigName: misty_bot_config.yaml
  botConfig:
    fulfillments:
      weather:
        parameters:
          api-key: XXXXXXXXXXXXXXXXXXX
      poi:
        parameters:
          api-key: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
          here-api-key: XXXXXXXXXXXXXXXXXXXXXX
      map:
        parameters:
          api-key: XXXXXXXXXXXXXXXXXXXXXX
  imagePullSecrets:
  - name: ngc-docker-reg-secret

Using UCF Studio

Building an Application

UCF Applications can be built using either the UCF Application Builder CLI tool or the UCF Studio GUI.

Using CLI Tools

Existing application graph files can be built using the following command:

$ ucf_app_builder_cli app build app.yaml
2022-12-05 14:44:52,464 - AppBuilder - INFO - Syncing any missing service versions to cache...
2022-12-05 14:44:52,507 - AppBuilder - INFO - Validating application ...
2022-12-05 14:44:52,517 - AppBuilder - INFO - Building application ucf-bot-app-1.13.1 ...
2022-12-05 14:45:00,122 - AppBuilder - INFO - Application Information written to ucf-bot-app-1.13.1/app_info.yaml
2022-12-05 14:45:00,122 - AppBuilder - INFO - Application compliance report generated at ucf-bot-app-1.13.1/compliance_report.json
2022-12-05 14:45:00,123 - AppBuilder - INFO - Application Helm Chart generated in ucf-bot-app-1.13.1
2022-12-05 14:45:00,123 - AppBuilder - INFO - App built

Parameter files can be applied using the following command:

$ ucf_app_builder_cli app build app.yaml params1.yaml params2.yaml

Application build performs several validations including any syntax errors, incorrect parameters, invalid connections and fails in case of any issues:

$ ucf_app_builder_cli app build app.yaml
2022-12-05 14:46:16,846 - AppBuilder - INFO - Syncing any missing service versions to cache...
2022-12-05 14:46:16,888 - AppBuilder - INFO - Validating application ...
2022-12-05 14:46:16,891 - AppBuilder - ERROR - Mandatory endpoint 'speech-controller' not set for microservice instance speech-web-app
2022-12-05 14:46:16,899 - AppBuilder - ERROR - Application validation failed

Using UCF Studio

Refer to Create Application for an example of creating an application using UCF Studio.

Build Output

Building an application generates the following:

• A deployable helm chart at the location mentioned by the command output e.g. ucf-bot-app-1.13.1

• Application information file with some information useful for deploying the application e.g. ucf-bot-app-1.13.1/app_info.yaml. Currently this includes:

  • List of secrets used in the application and their details. This can be used to setup all the secrets prior to deploying the application.

  • List of metrics and their details exposed by microservices used in the application. This can be used to create any Prometheus rules or alerts or HPA configurations.

  • Sample for this file:

    metrics:
    - name: nv_inference_count
      description: No. of inference requests executed by the server
      labels: [ model, version ]
      microservice: ucf.svc.riva.speech-skills v2.0.0
    
    secrets:
    - name: my-ngc-api-key
      description: Secret for NGC API key
      type: Kubernetes Secret
      secretName: ngc-api-key-secret
      secretKey: NGC_API_KEY
      usedBy:
      - ucf.svc.riva.speech-skills v2.0.0
      - ucf.svc.botmaker.speech-controller v2.0.0
      - ucf.svc.botmaker.dialog-manager v2.0.0
    

• Combined compliance report of all the microservices added to the application e.g. ucf-bot-app-1.13.1/compliance_report.json. Sample for this file:

  {
      "name": "ucf-bot-app",
      "version": "1.13.1",
      "time-utc": "2023-03-13 13:21:33",
      "appbuilder-version": "1.5.0",
      "compliance-development-mandatory-overall": 100.0,
      "compliance-development-optional-overall": 49.99,
      "compliance-ms": [
          {
              "name": "ucf.svc.riva.speech-skills",
              "version": "2.0.0",
              "compliance-development-mandatory": 100.0,
              "compliance-development-optional": 66.66
          },
          {
              "name": "ucf.svc.botmaker.speech-web-app",
              "version": "2.0.0",
              "compliance-development-mandatory": 100.0,
              "compliance-development-optional": 66.66
          },
          {
              "name": "ucf.svc.botmaker.speech-controller",
              "version": "2.0.0",
              "compliance-development-mandatory": 100.0,
              "compliance-development-optional": 33.33
          },
          {
              "name": "ucf.svc.botmaker.dialog-manager",
              "version": "2.0.0",
              "compliance-development-mandatory": 100.0,
              "compliance-development-optional": 33.33
          }
      ]
  }
  

Deploy an Application

Building an UCF application generates a deployable helm chart. A simple way to deploy it is using the helm install command:

$ helm install <release-name> <app-output-helm-chart>

Some microservices expose their functionality to external users using the NodePort service type. Exposing NodePort services may not be possible or maybe tricky to expose in Kubernetes provided by some cloud service providers. In this case global.useLoadBalancerInsteadOfNodePort parameter maybe used to override the NodePort service type to LoadBalancer. This is only possible for microservices that are built using UCF 1.5 or later. An example of using the option is:

$ helm install <release-name> <app-output-helm-chart> –set=global.useLoadBalancerInsteadOfNodePort=true

Users may also have their own of deploying the application charts.

The application README which gets copied to the output helm chart may have additional pre-requisites/steps required to deploy the application.

Some applications depend on core services like Vault Agent Injector, Secrets Store CSI Driver. Users must deploy these services before deploying the applciation. The application information generated during app build might be helpful here.