Creating a Microservice

In this section, we will go through the steps required to create a UCF Microservice. UCF Microservices are built using the UCF Microservice Builder CLI (msbuilder) tool. The input to the tool is a directory with a specific structure and files with specific names and formats. The output of the tool is UCF microservice specification for the microservice and optionally the Container Images used by the microservice and Helm Chart for the microservice based on the use case and type of input to the UCF Microservice Builder CLI tool.

For more information, on what a UCF Microservice is refer to - UCF Microservice.

For the msbuilder input directory specification refer to - Microservice Builder Input Specification.

Generate scaffolding

UCF Microservice Builder CLI service create generates a scaffolding for the new service:

$ ucf_ms_builder_cli service create [-h] [-d DESCRIPTION] [-o OUTPUT_DIR] -n NAME -i {container-image,container-config,helm-chart} [-c CHART]

The tool supports creating microservices for three types of inputs:

• Pre-built container images

• Container build configs

• Pre-built helm charts

Pre-built Container Images

This option can be used when container images are already available for the microservice to be built. With this method, the tool will generate:

• Helm chart for the microservice

• UCF Microservice specification for the microservice

To use this method, run:

$ ucf_ms_builder_cli service create -n myservice -d "my service description" -i container-image

This command will generate a template microservice in the following output directory structure:

myservice
├── changelog.txt
├── configs
├── endpoints
│   ├── http-api.yaml
│   └── myservice-endpoint-name.yaml
├── LICENSE.txt
├── manifest.yaml
├── manual_compliance_test_results.yaml
├── README.md
├── scripts
└── tests
    └── dev
        ├── app.yaml
        └── params1.yaml

Container Build Config

This option can be used when container images must be built along with the microservice. With this method, the tool will generate:

• Container Images for the microservice

• Helm chart for the microservice

• UCF Microservice specification for the microservice

To use this method, run:

$ ucf_ms_builder_cli service create -n myservice -d "my service description" -i container-config

This command will generate a template microservice in the following output directory structure:

myservice
├── changelog.txt
├── configs
├── containers
│   └── cb_config.yaml
├── endpoints
│   ├── http-api.yaml
│   └── myservice-endpoint-name.yaml
├── LICENSE.txt
├── manifest.yaml
├── manual_compliance_test_results.yaml
├── README.md
├── scripts
├── target_x86_64.yaml
└── tests
    └── dev
        ├── app.yaml
        └── params1.yaml

Pre-built Helm Chart

This option can be used when helm chart for the microservice is already available but the UCF microservice specification needs to be created. With this method, the tool will generate:

• UCF Microservice specification for the microservice

To use this method, run:

$ ucf_ms_builder_cli service create -n myservice -d "my service description" -i helm-chart -c <helm chart link>

This command will generate a template microservice in the following output directory structure:

myservice
├── changelog.txt
├── endpoints
│   ├── http-api.yaml
│   └── myservice-endpoint-name.yaml
├── LICENSE.txt
├── manifest.yaml
├── manual_compliance_test_results.yaml
├── README.md
├── tests
│   └── dev
│       ├── app.yaml
│       └── params1.yaml
└── values.yaml

Updating Microservice Builder Input

As demonstrated in the previous section, the input will differ slightly based on the use case and type of the input to the tool. This section will describe how to update the files in input directory structure for all the use cases.

Update configs

Add any configuration files required by the microservice to this directory. Files in this directory can container placeholders which will get overridden by a UCF application.

The files in this directory are NOT used when using a Pre-built Helm Chart.

Example of a file app_cfg.yaml in configs directory containing placeholders:

redis:
  host: $egress.redis.address
  port: $egress.redis.port

log_level: 5

timeout: $params.timeout

The file app_cfg.yaml in the configs directory will get mounted at /opt/configs/app_cfg.yaml in containers that are part of the microservice with placeholders updated with actual values.

When an application containing this microservice (with redis egress endpoint connected and timeout param set to 10) is built and deployed, a file /opt/configs/app_cfg.yaml will be mounted to containers that are part of the microservice with following contents:

redis:
  host: redis-svc
  port: 6739

log_level: 5

timeout: 10

Update scripts

Add any script files required by the microservice to this directory.

The files in this directory are NOT used when using a Pre-built Helm Chart.

For example a file named run.sh in the scripts directory will get mounted at /opt/scripts/run.sh in containers that are part of the microservice with exactly the same contents.

Adding Container Build config files to containers

Add and update Container Build config files to containers directory for any containers that must be built along with the microservice.

The files in this directory are NOT used when using a Pre-built Helm Chart or when using Pre-built Container Images.

Update endpoints

Endpoint definition file for each ingress/egress endpoint as per the UCF MS Specification can be added under the endpoints directory. The filename must be <endpoint-name>.<ext>. Extension and content of the file will depend on the scheme of the endpoint.

For more details on the contents, formats and file names for the endpoint definition files, refer to UCF Endpoint Schemes.

Update manual_compliance_test_results.yaml

Developers must run manual compliance checks as specified in UCF Microservice Compliance and update the results (boolean) as true / false for individual compliance rules in manual_compliance_test_results.yaml.

Add tests

The tests directory contains UCF applications that are basically tests for the microservice. The test apps follow the UCF application graph file specification - UCF Application.

These tests should ideally test the microservice as a whole i.e. if the helm chart that for the microservice is working correctly rather than unit tests for the applications running inside the microservice.

It could test for example if accessing APIs through the microservice endpoints works, if the application works correctly when containerized and deployed through Kubernetes by calling APIs with known inputs and corresponding known outputs.

If the microservice being built depends on other microservices, these other microservices may be replaced by a stub microservice or simpler functional equivalent microservice to remove dependency on other microservices when testing.

Update values.yaml

This file is used ONLY when using a Pre-built Helm Chart.

It is in the form of a standard Helm values.yaml and contains values of the pre-built helm chart that must be overridden. The file can contain the $params, $egress and $secrets placeholders which will get substituted with the actual values when the application is built and deployed.

When the scaffolding is generated with Pre-built Helm Chart type - values.yaml is pulled from the helm chart and placed here. The file can be trimmed and updated by the microservice developer to keep only those values that need to be overridden by the microservice or the application using the microservice.

For example, when a scaffolding is generated for a helm chart, the values.yaml may look like this:

applicationSpecs:
  myservice-deployment:
    apptype: stateless
    containers:
      myservice-container:
        env:
        - name: REDIS_ENDPOINT
          value: redis-svc:6379
        command:
        - sleep
        - '1000'
        image:
          repository: ubuntu
          tag: latest
        ports:
        - containerPort: 1000
          name: http
    restartPolicy: Always
    securityContext:
      runAsGroup: 1000
      runAsUser: 1000
    services:
      myservice-service:
        ports:
        - name: http-api
          port: 1000
image:
  pullPolicy: IfNotPresent
imagePullSecrets: []
ingress:
  enabled: false

This can be updated by the microservice developer to remove values not to be overridden, and add placeholders like $params and $egress:

applicationSpecs:
  myservice-deployment:
    apptype: stateless
    containers:
      myservice-container:
        env:
        - name: REDIS_ENDPOINT
          value: $egress.redis.address:$egress.redis.port
        command:
        - sleep
        - '1000'
        image:
          repository: ubuntu
          tag: $params.stringToEcho

Update Documentation

The microservice builder tool accepts four pieces of documentation:

• README.md - Detailed information about the microservice. The template generated by service create command has multiple sections like detailed description, usage information, performance KPIs, supported platforms and deployment requirements, known issues etc. Microservice developer must fill in the information in these sections as applicable.

• LICENSE.txt - Contains License text for any 3rdparty software used in the microservice. This includes the licenses for the 3rdparty software used in the applications running inside the containers as well as any 3rdparty packages added to the container file system.

• changelog.txt - A list of changes in the current version as compared to the previous version.

Updating Manifest file manifest.yaml

The manifest.yaml contains information for describing the microservice, the helm chart implementation and container build config details.

The template manifests generated for each of the three use cases differ slightly. This section will describe how to update the manifest file for the first use case i.e. Pre-built Container Images and then describe the differences for other use cases where applicable.

Following is an example of the template manifest.yaml file generated for Pre-built Container Images use case:

type: msapplication
specVersion: 2.0.0
name: ucf.svc.myservice
chartName: myservice
description: default description
version: 0.0.1
tags: []
keywords: []

publish: false

egress-endpoints:
  - name: myservice-endpoint-name
    description: Short description of endpoint
    protocol: TCP # Or UDP
    scheme: asyncio # Or grpc / rtsp / asyncio / none
    mandatory: True # Or False
    data-flow: in-out # Or in or out

ingress-endpoints:
  - name: http-api
    description: Short description of http-api ingress endpoint
    scheme: http
    data-flow: in-out # Or in or out

secrets:
  - name: some-secret-name
    description: Description for the secret
    mandatory: True
    mountPath: /secrets
    fileName: someSecretFileName

params:
  stringToEcho: someString
  #> type: string
  #> enum_values: someString, someOtherString
  #> description: String to echo in init container
  timeToSleep: 1000000
  #> type: integer
  #> maximum: 2000000
  #> minimum: 1000000
  #> description: String to echo in init container
  #> flags: mandatory

metrics:
- portName: metrics # Name of pod container port which implements prometheus metrics
  path: /metrics    # HTTP path where metrics are available

  # List of metrics exported by the microservice endpoint and their details
  details:
  - name: latency
    description: Average latency of handling HTTP requests in milliseconds
    labels: [request-type]
    # recommendedMin: 5  # Optional
    # recommendedMax: 50  # Optional

  # Optional. Selector for pods that implement the metrics endpoint
  # podSelectorLabels:
  #   app: myservice-myservice-deployment

# Files and directories will be mounted with prefix /opt/ext-files/
externalFiles:
- name: some-config.yaml # File will be available in containers at /opt/ext-files/some-config.yaml
  description: Some Configuration file
  mandatory: True
  isDirectory: False

tests:
  - name: dev-params1
    app: tests/dev/app.yaml
    params: tests/dev/params1.yaml
    ciTrigger: false
    timeout: 10
    duration: 10
    installPreReqs: true  # Wether to install foundational services
    namespace: default  # Kubernetes namespace
    gpuNodeLabels: ""
    watchAllPods: true # OR set to false and set list of pods to watch below
    watchPods:
    - <pod-name-regex>
    testerPods:  # At least one tester pod is required
    - name: testpod1  # Name of the test pod
      startSignature: <START>  # Signature to look for in the logs indicating start of tests. Regex is accepted
      endSignature: <END>  # Signature to look for in the logs indicating end of tests. Regex is accepted
      errorSignatures:  # Signatures that indicate test failures.  Regex is accepted
      - <REGEX1>
      - <REGEX2>


---
spec:
  - name: myservice-deployment
    type: ucf.k8s.app.deployment
    parameters:
      apptype: stateless

  - name: myservice-init-container
    type: ucf.k8s.initcontainer
    parameters:
      image: ubuntu
      imagePullPolicy: IfNotPresent
      args: [echo, $params.stringToEcho]
  - name: "myservice-container"
    type: ucf.k8s.container
    parameters:
      image:
        repository: ubuntu
        tag: "latest"
      command: ["sleep", "$params.timeToSleep"]
      ports:
      - containerPort: 0 # <PORT>
        name: http
      startupProbe:
        httpGet:
          path: /healthz
          port: http
        failureThreshold: 30
        periodSeconds: 10
      livenessProbe:
        httpGet:
          path: /healthz
          port: http
      readinessProbe:
        exec:
          command:
          - cat
          - /tmp/healthy
        initialDelaySeconds: 5
        periodSeconds: 5

  - name: myservice-service
    type: ucf.k8s.service
    parameters:
      annotations:
        test-svc-annotation: annotation-value
      labels:
        test-svc-label: label-value
      ports:
      - port: 0 # <OUT_PORT>
        name: http-api
      #externalTrafficPolicy: # Cluster / Local. Allowed when service type is NodePort or LoadBalancer

Update the following sections based on the manifest specification:

• Basic Information

• Ingress Endpoints

• Egress Endpoints

• Parameters

• Secrets

• External Files

• Metrics

• Tests along with test applications in tests directory - Add tests

Spec

Implementation of the microservice (i.e. it’s helm chart) can be specified under the spec field. As part of the UCF Microservice Builder Tool, the microservice implementation must be done using composable components.

When using Pre-built Helm Chart, this section must be omitted. MSBuilder ignores this field since it does not build the microservice helm chart.

Components

Components define a small part of the microservice implementation such as a container, a service, a volume. The list of available components can be viewed using:

$ ucf_ms_builder_cli component list
List of available components:
=============================
ucf.appspec.app.workload
ucf.appspec.defaultVolumeMount
ucf.k8s.app.deployment
ucf.k8s.configmap
ucf.k8s.container
ucf.k8s.imagepullsecret
ucf.k8s.initcontainer
ucf.k8s.podAnnotations
ucf.k8s.podLabels
ucf.k8s.podSecurityContext
ucf.k8s.pvc
ucf.k8s.restartPolicy
ucf.k8s.service
ucf.k8s.volume
...

To view details about a component, run:

$ ucf_ms_builder_cli component info -c ucf.k8s.service
type:        ucf.k8s.service
description: Service
properties:
                 pod: (string ), Service pod. [Mandatory:False]
               ports: (array  ), An explanation about the purpose of this instance. [Mandatory:True]
                        port: (integer), Service port. [Mandatory:True]
                        name: (string ), Name for service port (this is used as endpoint name) [Mandatory:False]
                  targetPort: (['string', 'integer']), Target port number or name. [Mandatory:False]
                    nodePort: (integer), Target port number or name. [Mandatory:False]
                       range: (integer), Range of ports. 0-<range - 1> will be added added to port/targetPort/nodePort [Mandatory:False]
                type: (string ), Service type [Mandatory:False, Allowed Values:{ ClusterIP, NodePort }]
externalTrafficPolicy: (string ), Route external traffic to node-local or cluster-wide endpoints [Mandatory:False, Allowed Values:{ Cluster, Local }]
        nameOverride: (boolean), Enable to set service name as set as <appname>-<service-name> [Mandatory:False]
    fullNameOverride: (boolean), Enable to set service name as set as <service-name> [Mandatory:False]
          extraSpecs: (object ), Extra specs to set on the service [Mandatory:False]
         annotations: (object ), Annotations to set on the service [Mandatory:False]
              labels: (object ), Labels to set on the service [Mandatory:False]

Commonly used components

The following table lists the commonly used component and their purpose:

Component	Purpose
ucf.appspec.defaultVolumeMount	Add a default volumeMount. This volumeMount will get added to all the containers / initContainers in the pod
ucf.appspec.ingress	Add an ingress resource
ucf.k8s.app.deployment	Defines a deployment workload along with it’s type (e.g. stateless, stateful, job, cronjob, daemonset, static-pod)
ucf.k8s.configmap	Add a configmap to the microservice
ucf.k8s.container	Add a container to the microservice pod/pod template
ucf.k8s.dnsPolicy	Set a DNS policy for the microservice pod/pod template
ucf.k8s.initcontainer	Add an initContainer to the microservice pod/pod template
ucf.k8s.podAnnotations	Set annotations on the pod/pod template
ucf.k8s.podLabels	Set custom labels on the pod/pod template
ucf.k8s.podSecurityContext	Set a securityContext for the pod/pod template
ucf.k8s.pvc	Add a PVC resource for the microservice
ucf.k8s.restartPolicy	Set a restart policy for the pod/pod template
ucf.k8s.service	Add a kubernetes service to the
ucf.k8s.volume	Add a volume to the pod/pod template

Adding components to the spec

The components must be added under spec as a list along with their parameters. The parameter values must follow the parameter schema for the component which can be viewed using the component info command as shown above. Placeholders $params, $egress and $secrets can be used in parameter values to refer to the parameters, egress endpoint and secret respectively. These placeholders remain in the built microservice helm chart but are substituted during application build.

An example of adding components with placeholders is:

spec:
  - name: myservice-deployment
    type: ucf.k8s.app.deployment
    parameters:
      apptype: stateless

  - name: "myservice-container"
    type: ucf.k8s.container
    parameters:
      image:
        repository: ubuntu
        tag: "latest"
      env:
      - name: HTTP_API_CA_CERT_PATH
        value: $secrets.ca-cert.path
      command: ["curl", "$egress.http-api.address:$egress.http-api.port",
        "--connect-timeout", $params.connectTimeout, "--cacert", $(HTTP_API_CA_CERT_PATH)]
      startupProbe:
        httpGet:
          path: /healthz
          port: http
        failureThreshold: 30
        periodSeconds: 10
      livenessProbe:
        httpGet:
          path: /healthz
          port: http
      readinessProbe:
        exec:
          command:
          - cat
          - /tmp/healthy
        initialDelaySeconds: 5
        periodSeconds: 5

Note

For the components defined in spec, parameters must adhere to the component schema specific to each type as shown above.

Only one workload type (e.g stateless, stateful, job, cronjob, daemonset, static-pod) can be specified in one yamldoc. To add multiple workload types (e.g. two stateless (i.e. Deployment) ) to the microservice:

---
spec:
- name: app-deployment-1
  type: ucf.k8s.app.deployment
  parameters:
    apptype: stateless
...
---
spec:
- name: app-deployment-2
  type: ucf.k8s.app.deployment
  parameters:
    apptype: stateless
...

Using container builder config

For using the Container Build option to build containers along with the microservice, modify the image parameter of the ucf.k8s.container and ucf.k8s.initcontainer components to specify the container builder config file path. The path is relative to the manifest.yaml file. For example:

spec:
- name: myservice-init-container
  type: ucf.k8s.initcontainer
  parameters:
    image:
      config_path: containers/init-container.yaml
    ...
- name: myservice-container
  type: ucf.k8s.container
  parameters:
    image:
      config_path: containers/app-container.yaml
    ...

A template container builder config file is generated at containers/cb_config.yaml. The following changes must be done at a minimum for the config to work correctly.

• Update the base_image

• Update local_copy_files and stage_copy_files or remove them completely

• Update http_archives or remove completely

• Update image_name under docker_build to where the image will be uploaded.

Building a microservice

Once the input for the microservice builder tool is ready, the microservice can be built using:

$ ucf_ms_builder_cli service build -d <msbuilder-input-dir>

$ ucf_ms_builder_cli service build -d myservice/
2022-12-05 14:54:13,927 - Registry - INFO - Building: myservice/
2022-12-05 14:54:15,943 - MsBuilder - INFO - Helm chart generated in myservice/output/helm
2022-12-05 14:54:15,962 - MsBuilder - INFO - MS spec generated in myservice/output/msspec
2022-12-05 14:54:17,034 - MsBuilder - WARNING - Mandatory compliance checks failed. Check myservice/output/compliance_test_logs.txt for more information
2022-12-05 14:54:17,176 - MsBuilder - INFO - MS spec generated in myservice/output/msspec
2022-12-05 14:54:17,185 - MsBuilder - INFO - Building test application 'dev-params1'
2022-12-05 14:54:17,327 - MsBuilder - INFO - Syncing any missing service versions to cache...
2022-12-05 14:54:17,329 - MsBuilder - INFO - Validating application ...
2022-12-05 14:54:17,335 - MsBuilder - INFO - Building application myservice-dev-params1-0.0.1 ...
2022-12-05 14:54:17,671 - MsBuilder - INFO - Application compliance report generated at myservice/output/tests/dev-params1/compliance_report.json
2022-12-05 14:54:17,758 - MsBuilder - INFO - Application Helm Chart generated in myservice/output/tests/dev-params1
2022-12-05 14:54:17,759 - MsBuilder - INFO - Tests generated in myservice/output/tests
2022-12-05 14:54:17,902 - Registry - INFO - Generating service helm chart package...
2022-12-05 14:54:17,993 - Registry - INFO - Generating test helm chart packages...
2022-12-05 14:54:18,152 - MsBuilder - INFO - Added microservice 'ucf.svc.myservice:0.0.1' to local repository

If using the Container Build option, an additional argument -t <target-file> must be added.

$ ucf_ms_builder_cli service build -d myservice/ -t myservice/target_x86_64.yaml

The target file describes the configuration of the target the container is being built for. A sample target file <msbuilder-input-dir>/target_x86_64.yaml with following contents is added as part of the template microservice generation:

---
platform:
  arch: x86_64
  os: linux
  distribution: ubuntu_22.04
compute:
  cuda: 11.8
  tensorrt: 8.5.1
  cudnn: 8.6.0
  deepstream: 6.2
  triton: null
  vpi: 2.1.6

Container image built by the tool can be manually uploaded if required using

.. code-block:: text

$ docker images # List all images and find the image that was built with tool $ docker push <image>

Note

Some Kubernetes configurations will not be able to pull images if the images are in local docker cache since Kubernetes might have it’s own container runtime instance running and does not use the local docker cache. In this case, the container images must be pushed to a container registry where Kubernetes can pull images from. Corresponding changes must be done in the manifest file if required to point to the new repository. When using Microk8s, its built-in registry may be used: https://microk8s.io/docs/registry-built-in.

Build Output

Building the microservice generates the following output based on the use case:

• UCF Microservice Specification for the microservice

• Helm Chart - for Pre-built Container Image and Container Build use cases

• Container Images - for Container Build use case

• Test Application Helm Chart - for each of the tests mentioned in the manifest.yaml

• Compliance Report for the microservice

• Compliance Check Logs are generated at <msbuilder-input-dir>/output/compliance_test_logs.txt

Once the microservice is built successfully, it gets automatically added to the local repository and is ready to use in UCF applications.

The information for the built microservice can be viewed using Viewing Microservice Information.

For more information on compliance refer to UCF Microservice Compliance. Microservice developers can refer to the log file at <msbuilder-input-dir>/output/compliance_test_logs.txt for the reasons for any compliance failures.