> This page is for version 11 (default).
> For other versions, use one of these documentation indexes:
> - 11 (default): https://docs.nvidia.com/base-command-manager/product-manuals/11/llms.txt
> - 10: https://docs.nvidia.com/base-command-manager/product-manuals/10/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/base-command-manager/product-manuals/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/base-command-manager/product-manuals/_mcp/server.

# Kubernetes Apps

Kubernetes add-ons were introduced in an older version of NVIDIA Base Command Manager (at that time still called Bright Cluster Manager) version 8.1. Add-ons could be managed in that version as part of the `addons` submode of the `kubernetes` mode in `cmsh`. The feature later (in version 8.2) was expanded into the *Kubernetes Applications & Groups* feature. Kubernetes Applications & Groups, less formally called app groups, can be accessed via the `appgroups` submode of `cmsh`:

**Example**

```bash
root@basecm11 ~# cmsh
[basecm11]% kubernetes
[basecm11->kubernetes[default]]% appgroups
[basecm11->kubernetes[default]->appgroups]% list
Name (key)   Applications
------------ ------------------------------
system       <13 in submode>
[basecm11->kubernetes[default]->appgroups]%
```

Versions from 9.2 onward started deploying operators as Helm charts, such as for the NVIDIA GPU operator, Run:ai, and NetQ. The remaining apps have since been migrated to equivalent Helm charts except for Flannel.

The legacy version 8.1 `addons` mode parameters are now accessed from version 8.2 onward via a default `system` app group instance. The `system` instance is accessed in the `appgroups` submode.

**Example**

```bash
[basecm11->kubernetes[default]->appgroups]% use system
[basecm11->kubernetes[default]->appgroups[system]]% show
Parameter                        Value
-------------------------------- ------------------------------------------------
Name                             system
Revision
Enabled                          yes
applications                     <13 in submode>
[basecm11->kubernetes[default]->appgroups[system]]% applications
[basecm11->kubernetes[default]->appgroups[system]->applications]% list
Name (key)          Format Enabled
------------------- ------ -------
bootstrap           Yaml   yes
calico              Yaml   yes
flannel             Yaml   no
kubestatemetrics    Yaml   yes
metrics_server      Yaml   yes
root                Yaml   yes
```

A Kubernetes application can span multiple namespaces. A name in `appgroups` therefore only exists to group logically-related applications. Each application contains a YAML configuration file, which BCM synchronizes to the Kubernetes API.

The default `system` app group is pre-defined. Other app groups can be created as needed. For example, an app group called `monitoring` could be created to group applications for running Prometheus, node exporters, and anything else related to exposing or viewing Prometheus metrics.

Toggling the `Enable` parameter of an app group enables or disables all of its application components in Kubernetes. Finer-grained control is possible within the `applications` mode level, by toggling the `enabled` parameter per application component instance. For example, within the `calico` application component instance:

**Example**

```bash
[basecm11->kubernetes[default]->appgroups[system]->applications]% use calico
[basecm11->kubernetes[default]->appgroups[system]->applications[calico]]% show
Parameter                        Value
-------------------------------- ------------------------------------------------
Name                             calico
Revision
Format                           Yaml
Enabled                          yes
Config                           <244KiB>
Environment                      <3 in submode>
Exclude list snippets            <2 in submode>
```

A large YAML configuration file for each application component instance can be configured via the `Config` parameter property, using the `set` option of `cmsh`. This opens up a text editor and allows the environment variables in the YAML configuration file to be managed.

*Exclude list snippets* are short exclude lists that can be set up for Kubernetes apps computing within the `excludelistsnippets` submode. They are used to prevent BCM software image updates from overwriting the provisioned files or directories of the container image that are important to the associated Kubernetes application.

Using exclude list snippets within an `excludelistsnippets` submode is discussed in detail in [section 4.3.1 of the *Cloudbursting Manual*](/cloudbursting/clicloud#431-setting-exclude-lists-with-excludelistsnippets). Similar to the case of Kubernetes apps images, in cloud computing exclude list snippets are used to prevent overwriting of the provisioned files and directories of cloud images.

*Environment entries* can be set via the `Environment` submode. Environment entries are similar to environment variables, and are used to replace variables inside the YAML configuration file. Environment entries can be added to the environment as well, if the `Nodes environment` value inside the `Environment` submode is set to `yes`.

**Example**

```bash
[basecm11->kubernetes[default]->appgroups[system]->applications[calico]]% environment
[basecm11->kubernetes[default]->appgroups[system]->applications[calico]->environment]% list
Name (key)             Value            Nodes environment
---------------------- ---------------- ------------------
calico_typha_replicas  0                no
calico_typha_service   none             no
head_node_internal_ip  10.141.255.254   no
```