> This page is for version 10. > For other versions, use one of these documentation indexes: > - 11 (default): https://docs.nvidia.com/base-command-manager/product-manuals/11/llms.txt > - 10: https://docs.nvidia.com/base-command-manager/product-manuals/10/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/base-command-manager/product-manuals/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/base-command-manager/product-manuals/_mcp/server. # CMDaemon Configuration File Directives This appendix lists all configuration file directives that may be used in the cluster management daemon configuration file. If a change is needed, then the directives are normally changed on the head node, or on both head nodes in the high availability configuration, in: ```bash /cm/local/apps/cmd/etc/cmd.conf ``` The directives can also be set in some cases for the regular nodes, via the software image in `/cm/images/default-image/cm/local/apps/cmd/etc/cmd.conf` on the head node. Changing the defaults already there is however not usually needed, and is not recommended. Only one directive is valid per `cmd.conf` file. To activate changes in a `cmd.conf` configuration file, the `cmd` service associated with it must be restarted. * For the head node this is normally done with the command: ```bash systemctl restart cmd ``` * For regular nodes, `cmd` running on the nodes is restarted. Often, the image should be updated before `cmd` is restarted. How to carry out these procedures for a directive is described with an example where the FrozenFile directive is activated on a regular node [here](#c6911-frozenfile-directive-for-regular-nodes-for-cmdaemon). ## C.1 Master directive **Syntax:** `Master = ` *hostname*\ **Default:** `Master = master` The cluster management daemon treats the host specified in the `Master` directive as the head node. A cluster management daemon running on a node specified as the head node starts in *head* mode. On a regular node, it starts in *node* mode. ## C.2 Port directive **Syntax:** `Port = ` *number*\ **Default:** `Port = 8080` The *number* used in the syntax above is a number between 0 and 65535. The default value is 8080. The `Port` directive sets the value of the port of the cluster management daemon to listen for non-SSL HTTP calls. By default, this happens only during init. All other communication with the cluster management daemon is carried out over the SSL port. [Pre-init port adjustment can be carried out in the node-installer.conf configuration. Shorewall may need to be modified to allow traffic through for a changed port.](https://enterprise-support.nvidia.com/s/article/how-do-i-change-the-default-port-cmdaemon-uses) ## C.3 SSLPort directive **Syntax:** `SSLPort = ` *number*\ **Default:** `SSLPort = 8081` The *number* used in the syntax above is a number between 0 and 65535. The default value is 8081. The `SSLPort` directive sets the value of the SSL port of the cluster management daemon to listen for SSL HTTP calls. By default, it is used for all communication of CMDaemon with Base View and `cmsh`, except for when CMDaemon is started up from init. This directive does not change the firewall port settings to match the value of the SSL port for CMDaemon communication. ## C.4 SSLPortOnly directive **Syntax:** `SSLPortOnly = yes`|`no`\ **Default:** `SSLPortOnly = no` The `SSLPortOnly` directive allows the non-SSL port to be disabled. By default, during normal running, both SSL and non-SSL ports are listening, but only the SSL port is used. Also by default, the non-SSL port is only used during CMDaemon start up. If `bootloaderprotocol` (section [5.1.6](/admin/provisioning#516-network-booting-protocol)) is set to `HTTP`, then `SSLPortOnly` must be set to `no`. The HTTPS protocol is unsupported by most bootloaders. #### C.4.0.1 Changing CMDaemon Ports The `cm-cmd-ports` utility changes the HTTP and HTTPS ports on which CMDaemon listens. The default ports are 8080 (HTTP) and 8081 (HTTPS), as set by the `Port` ([here](#c2-port-directive)) and `SSLPort` directives in `cmd.conf` ([here](#c3-sslport-directive)). The current port configuration can be listed with: ```bash [root@basecm10 ~]# cm-cmd-ports -l type http https path -------------- ------ ------- ------------------------------------------------------- image 8080 8081 /cm/images/default-image/cm/local/apps/cmd/etc/cmd.conf image 8080 8081 /cm/local/apps/cmd/etc/cmd.conf node-installer 8081 /cm/node-installer/scripts/node-installer.conf ``` To change the HTTPS port, for example to port 8082: ```bash [root@basecm10 ~]# cm-cmd-ports --https 8082 Successfully updated ports Changes take effect after restarting CMDaemon. ``` The firewall configuration is not automatically updated by `cm-cmd-ports`. The firewall rules file must be updated manually as part of the procedure. This procedure should be performed during a maintenance window. **Example** An example session is as follows: On the active head node, the port is changed with: ```bash [root@basecm10 ~]# cm-cmd-ports --https 8082 ``` The software images are updated on the provisioners with the `updateprovisioners` command ([here](/admin/provisioning#5244-provisioning-role-change-notification-with-updateprovisioners)), and an update is triggered for all non-image fsparts with the `trigger` command ([here](/admin/provisioning#5247-synchronization-of-fspart-subdirectories-to-provisioning-nodes)): ```bash [basecm10]% softwareimage updateprovisioners ... [basecm10]% fspart; trigger -a ... ``` The provisioning status can be checked with: ```bash [basecm10]% softwareimage provisioningstatus -r ``` An empty output indicates that provisioning is complete, and that the procedure may continue. CMDaemon can then be restarted on the active head node: ```bash [root@basecm10 ~]# systemctl restart cmd ``` In a high availability configuration, CMDaemon on the passive head node is restarted as well, but only after the `CMDaemon` on the active head node has successfully restarted. This order keeps the passive in sync with the active. Updating the passive first can cause issues later on. The updated `cmd.conf` is then copied from the software images to the compute nodes: ```bash [root@basecm10 ~]# pdcp -w -p /cm/images//cm/local/apps/cmd/etc/cmd.conf /cm/local/apps/cmd/etc/cmd.conf ``` The command is run for each node image in use on the cluster. CMDaemon on the compute nodes is then restarted: ```bash [root@basecm10 ~]# pdsh -g computenode systemctl restart cmd ``` The firewall rules on the head nodes are updated by editing `/etc/shorewall/rules` to replace the old port number with the new port number, after which Shorewall is reloaded: ```bash [root@basecm10 ~]# shorewall reload ``` The port change can be verified with: ```bash [root@basecm10 ~]# cm-cmd-ports -l type http https path -------------- ------ ------- ------------------------------------------------------- image 8080 8082 /cm/images/default-image/cm/local/apps/cmd/etc/cmd.conf image 8080 8082 /cm/local/apps/cmd/etc/cmd.conf node-installer 8082 /cm/node-installer/scripts/node-installer.conf [root@basecm10 ~]# cmsh -c ds basecm10 ............................. [ UP ] node001 .............................. [ UP ] ``` From BCM 12 onwards, Shorewall is superseded by nftables. ## C.5 CertificateFile directive **Syntax:** `CertificateFile = ` *filename*\ **Default:** `CertificateFile = "/cm/local/apps/cmd/etc/cert.pem"` The `CertificateFile` directive specifies the PEM-format certificate which is to be used for authentication purposes. On the head node, the certificate used also serves as a software license. ## C.6 PrivateKeyFile directive **Syntax:** `PrivateKeyFile = ` *filename*\ **Default:** `PrivateKeyFile = "/cm/local/apps/cmd/etc/cert.key"` The `PrivateKeyFile` directive specifies the PEM-format private key which corresponds to the certificate that is being used. ## C.7 CACertificateFile directive **Syntax:** `CACertificateFile = ` *filename*\ **Default:** `CACertificateFile = "/cm/local/apps/cmd/etc/cacert.pem"` The `CACertificateFile` directive specifies the path to the BCM PEM-format root certificate. It is normally not necessary to change the root certificate. ## C.8 ClusterCertificateFile directive **Syntax:** `ClusterCertificateFile = ` *filename*\ **Default:** `ClusterCertificateFile = "/cm/local/apps/cmd/etc/cluster.pem"` The `ClusterCertificateFile` directive specifies the path to the BCM PEM-format cluster certificate file used as a software license, and to sign all client certificates ## C.9 ClusterPrivateKeyFile directive **Syntax:** `ClusterPrivateKeyFile = ` *filename*\ **Default:** `ClusterPrivateKeyFile = "/cm/local/apps/cmd/etc/cluster.key"` The `ClusterPrivateKeyFile` directive specifies the path to the BCM PEM-format private key which corresponds to the cluster certificate file. ## C.10 RandomSeedFile directive **Syntax:** `RandomSeedFile = ` *filename*\ **Default:** `RandomSeedFile = "/dev/urandom"` The `RandomSeedFile` directive specifies the path to a source of randomness for a random seed. ## C.11 RandomSeedFileSize directive **Syntax:** `RandomSeedFileSize = ` *number*\ **Default:** `RandomSeedFileSize = 8192` The `RandomSeedFileSize` directive specifies the size of the random seed. ## C.12 DHParamFile directive **Syntax:** `DHParamFile = ` *filename*\ **Default:** `DHParamFile = "/cm/local/apps/cmd/etc/dh1024.pem"` The `DHParamFile` directive specifies the path to the Diffie-Hellman parameters. ## C.13 SSLHandshakeTimeout directive **Syntax:** `SSLHandshakeTimeout = ` *number*\ **Default:** `SSLHandshakeTimeout = 10` The `SSLHandShakeTimeout` directive controls the time-out period (in seconds) for SSL handshakes. ## C.14 SSLSessionCacheExpirationTime directive **Syntax:** `SSLSessionCacheExpirationTime = ` *number*\ **Default:** `SSLSessionCacheExpirationTime = 300` The `SSLSessionCacheExpirationTime` directive controls the period (in seconds) for which SSL sessions are cached. Specifying the value `0` can be used to disable SSL session caching. ## C.15 DBHost directive **Syntax:** `DBHost = ` *hostname*\ **Default:** `DBHost = "localhost"` The `DBHost` directive specifies the hostname of the MySQL database server. ## C.16 DBPort directive **Syntax:** `DBPort = ` *number*\ **Default:** `DBHost = 3306` The `DBPort` directive specifies the TCP port of the MySQL database server. ## C.17 DBUser directive **Syntax:** `DBUser = ` *username*\ **Default:** `DBUser = cmdaemon` The `DBUser` directive specifies the username used to connect to the MySQL database server. ## C.18 DBPass directive **Syntax:** `DBPass = ` *password*\ **Default:** `DBPass = "`*``*" The `DBPass` directive specifies the password used to connect to the MySQL database server. ## C.19 DBName directive **Syntax:** `DBName = ` *database*\ **Default:** `DBName = "cmdaemon"` The `DBName` directive specifies the database used on the MySQL database server to store CMDaemon related configuration and status information. ## C.20 DBUnixSocket directive **Syntax:** `DBUnixSocket = ` *filename*\ **Default:** `DBUnixSocket = ` `"/var/lib/mysql/mysql.sock"` The `DBUnixSocket` directive specifies the named pipe used to connect to the MySQL database server if it is running on the same machine. ## C.21 DBUpdateFile directive **Syntax:** `DBUpdateFile = ` *filename*\ **Default:** `DBUpdateFile = "/cm/local/apps/cmd/etc/cmdaemon_upgrade.sql"` The `DBUpdateFile` directive specifies the path to the file that contains information on how to upgrade the database from one revision to another. ## C.22 EventBucket directive **Syntax:** `EventBucket = ` *filename*\ **Default:** `EventBucket = "/var/spool/cmd/eventbucket"` The `EventBucket` directive (section [12.10.3](/admin/monitoring#12103-using-the-event-bucket-from-the-shell-for-events-and-for-tagging-device-states)) specifies the path to the named pipe that is created to listen for incoming events from a user. ## C.23 EventBucketFilter directive **Syntax:** `EventBucketFilter = ` *filename*\ **Default:** `EventBucketFilter = "/cm/local/apps/cmd/etc/eventbucket.filter"` The `EventBucketFilter` directive (section [12.10.3](/admin/monitoring#12103-using-the-event-bucket-from-the-shell-for-events-and-for-tagging-device-states)) specifies the path to the file that contains regular expressions used to filter out incoming messages on the event-bucket. ## C.24 LDAPHost directive **Syntax:** `LDAPHost = ` *hostname*\ **Default:** `LDAPHost = "localhost"` The `LDAPHost` directive specifies the hostname of the LDAP server to connect to for user management. ## C.25 LDAPUser directive **Syntax:** `LDAPUser = ` *username*\ **Default:** `LDAPUser = "root"` The `LDAPUser` directive specifies the username used when connecting to the LDAP server. ## C.26 LDAPPass directive **Syntax:** `LDAPPass = ` *password*\ **Default:** `LDAPPass = "`*``*`"` The `LDAPPass` directive specifies the password used when connecting to the LDAP server. It can be changed following the procedure described in Appendix [I](/admin/ldapchangepassword). ## C.27 LDAPReadOnlyUser directive **Syntax:** `LDAPReadOnlyUser = ` *username*\ **Default:** `LDAPReadOnlyUser = "readonlyroot"` The `LDAPReadOnlyUser` directive specifies the username that will be used when connecting to the LDAP server during LDAP replication. The user is a member of the "rogroup" group, whose members have a read-only access to the whole LDAP directory. ## C.28 LDAPReadOnlyPass directive **Syntax:** `LDAPReadOnlyPass = ` *password*\ **Default:** `LDAPReadOnlyPass = "`*``*`"` The `LDAPReadOnlyPass` directive specifies the password that will be used when connecting to the LDAP server during LDAP replication. ## C.29 LDAPSearchDN directive **Syntax:** `LDAPSearchDN = ` *dn*\ **Default:** `LDAPSearchDN = "dc=cm,dc=cluster"` The `LDAPSearchDN` directive specifies the Distinguished Name (DN) used when querying the LDAP server. ## C.30 LDAPProtocol directive **Syntax:** `LDAPProtocol = ` `ldap|ldaps`\ **Default:** `LDAPProtocol = "ldaps"` The `LDAPProtocol` directive specifies the LDAP protocol to be used when querying the LDAP server. ## C.31 LDAPPort directive **Syntax:** `LDAPPort = ` *number*\ **Default:** `LDAPPort = 636` The `LDAPPort` directive specifies the port to be used when querying the LDAP server. ## C.32 LDAPCACertificate directive **Syntax:** `LDAPCACertificate = ` *filename*\ **Default:** `LDAPCACertificate = "/cm/local/apps/openldap/etc/certs/ca.pem"` The `LDAPCACertificate` directive specifies the CA certificate to be used when querying the LDAP server. ## C.33 LDAPCertificate directive **Syntax:** `LDAPCertificate = ` *filename*\ **Default:** `LDAPCertificate = "/cm/local/apps/openldap/etc/certs/ldap.pem"` The `LDAPCertificate` directive specifies the LDAP certificate to be used when querying the LDAP server. ## C.34 LDAPPrivateKey directive **Syntax:** `LDAPPrivateKey= ` *filename*\ **Default:** `LDAPPrivateKey= "/cm/local/apps/openldap/etc/certs/ldap.key"` The `LDAPPrivateKey` directive specifies the LDAP key to be used when querying the LDAP server. ## C.35 HomeRoot directive **Syntax:** `HomeRoot = ` *path*\ **Default:** `HomeRoot = "/home"` The `HomeRoot` directive specifies the default user home directory used by CMDaemon. It is used for automatic mounts, exports, and when creating new users. ## C.36 DocumentRoot directive **Syntax:** `DocumentRoot = ` *path*\ **Default:** `DocumentRoot = "/cm/local/apps/cmd/etc/htdocs"` The `DocumentRoot` directive specifies the directory mapped to the web-root of the CMDaemon. The CMDaemon acts as a HTTP-server, and can therefore in principle also be accessed by web-browsers. ## C.37 SpoolDir directive **Syntax:** `SpoolDir = ` *path*\ **Default:** `SpoolDir = "/var/spool/cmd"` The `SpoolDir` directive specifies the directory which is used by the CMDaemon to store temporary and semi-temporary files. ## C.38 EnableShellService directive **Syntax:** `EnableShellService = true`|`false`\ **Default:** `EnableShellService = true` The `EnableShellService` directive allows shells to be started from Base View. The connection runs over CMDaemon, which is running over SSL, which means that between Base View and the device, the connection is encrypted. The directive does not affect `cmsh`’s `rshell`, `rconsole` `telnet`, and `ssh` commands. ## C.39 DisableRemoteShell directive **Syntax:** `AdvancedConfig = {"DisableRemoteShell=0|1", ...}`\ **Default:** `DisableRemoteShell=0`\ `DisableRemoteShell` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. By default CMDaemon provides access to devices via `rshell`, `rconsole` (SOL), `telnet`, and `ssh`. Setting the directive to `1` disables all RPC shell access. A more fine-grained disabling is possible with the following `AdvancedConfig` directives, which follow the same syntax: * **DisableRemoteRShell directive**: disables a remote shell to the head node. * **DisableRemoteRShellImage directive**: disables a remote shell to the chrooted software image. * **DisableRemoteRShellNode directive**: disables a remote shell to the regular nodes. * **DisableRemoteRShellTelnet directive**: disables a telnet shell to a device. * **DisableRemoteRShellSSH directive**: disables SSH to a device. * **DisableRemoteSOL directive**: disables a console via SOL to a device. ## C.40 EnableWebSocketService directive **Syntax:** `EnableWebSocketService = true`|`false`\ **Default:** `EnableWebSocketService = true` The `EnableWebSocketService` directive allows the use of CMDaemon Lite (section [2.6.7](/admin/clustermanagement#267-cmdaemon-lite)). ## C.41 EnablePrometheusMetricService directive **Syntax:** `EnablePrometheusMetricService = true`|`false`\ **Default:** `EnablePrometheusMetricService = true` If true, the `EnablePrometheusMetricService` directive creates an HTTP endpoint for Prometheus-style exporters that do not have their own HTTP endpoint. ## C.42 PrometheusMetricServicePath directive **Syntax:** `PrometheusMetricServicePath = ` *path*\ **Default:** `PrometheusMetricServicePath = SCRIPTS_DIR"metrics/prometheus"` The `PrometheusMetricServicePath` directive is the path from which CMDaemon can serve Prometheus metrics. `SCRIPTS_DIR` is the stem path `/cm/local/apps/cmd/scripts` by default. ## C.43 EnablePrometheusExporterService directive **Syntax:** `EnablePrometheusExporterService = true`|`false`\ **Default:** `EnablePrometheusExporterService = false` If true, the `EnablePrometheusExporterService` directive allows a Prometheus server to pull data from CMDaemon. To collect all data, the external Prometheus YAML configuration ([https://prometheus.io/docs/prometheus/latest/configuration/configuration/](https://prometheus.io/docs/prometheus/latest/configuration/configuration/)) should have all nodes with a monitoring role added to it: **Example** ```bash # # - job_name: cmdaemon-head01 # metrics_path: /exporter # static_configs: # - targets: ['https://head01:8081'] # ``` Configuring CMDaemon to sample metrics from a Prometheus exporter instead is covered in section [3.17](/admin/configuring#317-configuring-sampling-from-a-prometheus-exporter). ## C.44 PrometheusExporterRequireCertificate directive **Syntax:** `AdvancedConfig = {"PrometheusExporterRequireCertificate=0|1", ...}`\ **Default:** `PrometheusExporterRequireCertificate = 1` `PrometheusExporterRequireCertificate` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. * If set to `0`, then all users can use the Prometheus service to pull data using CMDaemon. * If set to `1`, then only users using certificate-based access can use the Prometheus service to pull data using CMDaemon. Certificate-based access can be configured as follows: A custom profile with the name `prometheus` can be created for BCM users with `profile` mode (section [6.4](/admin/usermanagement#64-tokens-and-profiles)) as follows: ```bash [basecm10->profile]% add prometheus [basecm10->profile*[prometheus*]]% append tokens PROMETHEUS_EXPORTER_TOKEN [basecm10->profile*[prometheus*]]% set nonuser yes [basecm10->profile*[prometheus*]]% commit ``` A 2048-bit SSL certificate called `mypromcert` can be created for a non-privileged user `prometheus` with 3650 days validity using the `createcertificate` command (section [6.4.2](/admin/usermanagement#642-creation-of-custom-certificates-with-profiles-for-users-managed-by-bcms-internal-ldap)) as follows: ```bash [basecm10->profile[prometheus]]% cert [basecm10->cert]% createcertificate 2048 mypromcert "" "" "" "" "" prometheus "" 3650 /tmp/prometheus.key \ /tmp/prometheus.pem Certificate key written to file: /tmp/prometheus.key Certificate pem written to file: /tmp/prometheus.pem ``` The PEM certificate is the public part of the key pair. The private key must be owned and be readable by the Prometheus instance that is scraping from the node exporter. To configure SSL authentication between Prometheus server and the Prometheus exporter node: * On the Prometheus server: * A directive value of: * `0` means that the session key is a temporary key that is auto-generated for any user * `1` means that the user (the Prometheus server) must provide a certificate * CMDaemon in a default cluster manager installation uses a self-signed certificate. A copy of certificate can be picked up on the head node with: **Example** ```bash [root@basecm10 ~]# ca_cert="/etc/prometheus/cmdaemon-$(hostname)-ca.pem" [root@basecm10 ~]# cat /cm/local/apps/cmd/etc/{cacert,cluster}.pem > ${ca_cert} [root@basecm10 ~]# chmod 600 ${ca_cert} ``` The Prometheus server should be configured to specify this CA certificate. If the certificate used by CMDaemon is not a self-signed certificate—that is to say, it is signed by a recognized CA—then the certificate issued by CMDaemon is legitimate, and the Prometheus server does not need to have that CA certificate explicitly specified. * The Prometheus exporter always uses TLS, which is automatically managed by CMDaemon. ## C.45 EnablePrometheusPushService directive **Syntax:** `EnablePrometheusPushService = true`|`false`\ **Default:** `EnablePrometheusPushService = false` If true, the `EnablePrometheusPushService` directive allows a Prometheus server to push data to CMDaemon: **Example** ```bash myhead # cat < "$file" myhead # test_prometheus_push{hello="world"} 42 myhead # EOF myhead # curl -k -X POST --data-binary "@$file" https://localhost:8081/prometheuspush/job/test ``` ## C.46 CMDaemonAudit directive **Syntax:** `CMDaemonAudit = true`|`false`\ **Default:** `CMDaemonAudit = true` When the `CMDaemonAudit` directive is set to `true`, and a value is set for the CMDaemon auditor file with the `CMDaemonAuditorFile` directive, then CMDaemon actions are time-stamped and logged in the CMDaemon auditor file. ## C.47 CMDaemonAuditorFile directive **Syntax:** `CMDaemonAuditorFile = ` *path to* `audit.log `*file*\ or\ **Syntax:** `CMDaemonAuditorFile = ` *path to* `audit.json `*file*\ or\ **Syntax:** `CMDaemonAuditorFile = ` *http(s) path to JSON server* **Default:** `CMDaemonAuditorFile = "/var/spool/cmd/audit.log"` The `CMDaemonAuditorFile` directive sets where the audit logs for CMDaemon actions are logged. The log format for a `.log` file in a standard directory path is:\ (*time stamp*) *profile* \[*IP-address*] *action* (*unique key*) **Example** ```bash (Mon Jan 31 12:41:37 2011) Administrator [127.0.0.1] added Profile: arbitprof(4294967301) ``` The directive can be set in the following kinds of formats: * `CMDaemonAuditorFile = "/var/spool/cmd/audit.log"` * `CMDaemonAuditorFile = "/var/spool/cmd/audit.json"` * `CMDaemonAuditorFile = "http://` \<*`IP address`*>`:<`*`port`*>/*`some`*`/`*`path`*`"` * `CMDaemonAuditorFile = "https://` \<*`IP address`*>`:<`*`port`*>/*`some`*`/`*`path`*`"` A simple POST web service can be faked using `netcat`: **Example** ```bash nc -l 1234 -k ``` The JSON file always contains a valid array. An RPC call looks like this: **Example** ```bash ... { "entity": "node001", "rpc": { "address": "127.0.0.1", "call": "updateDevice", "service": "cmdevice", "timestamp": 1579185696, "user": "Administrator.root" }, "task_id": 0, "updated": true }, ... ``` ## C.48 DisableAuditorForProfiles directive **Syntax:** `DisableAuditorForProfiles = {`*`profile`*` [,`*` profile`*`] ...}`\ **Default:** `DisableAuditorForProfiles = {NODE}` The `DisableAuditorForProfiles` directive sets the profile for which an audit log for CMDaemon actions is disabled. A profile (section [2.3.4](/admin/clustermanagement#234-profiles)) is what allows or denies what CMDaemon may do for that profile user. More than one profile can be set as a comma-separated list. New profiles can also be created via the `profile` mode of `cmsh` or via the navigation path `Identity Management` > `Profiles` > of Base View, thus making it possible to disable auditing for arbitrary groups of CMDaemon services. ## C.49 EventLogger directive **Syntax:** `EventLogger = true`|`false`\ **Default:** `EventLogger = true` The `EventLogger` directive sets whether to log events. If active, then by default it logs events to `/var/spool/cmd/events.log` on the active head. If a failover takes place, then the event logs on both heads should be checked and merged for a complete list of events. The location of the event log on the filesystem can be changed using the `EventLoggerFile` directive ([here](#c50-eventloggerfile-directive)). Whether events are logged in files or not, events are cached and accessible using `cmsh` or Base View. The number of events cached by CMDaemon is determined by the parameter `MaxEventHistory` ([here](#c51-maxeventhistory-directive)). ## C.50 EventLoggerFile directive **Syntax:** `EventLoggerFile = ` *filename*\ **Default:** `EventLogger = "/var/spool/cmd/events.log"` The `EventLogger` directive sets where the events seen in the event viewer (section [12.10](/admin/monitoring#1210-event-viewer)) are logged. ## C.51 MaxEventHistory directive **Syntax:** `AdvancedConfig = {"MaxEventHistory=`*number*`", ...}`\ **Default:** `MaxEventHistory=8192`\ `MaxEventHistory` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. By default, when not explicitly set, the maximum number of events that is retained by CMDaemon is 8192. Older events are discarded. The parameter can take a value from 0 to 1000000. However, CMDaemon is less responsive with larger values, so in that case, setting the `EventLogger` directive ([here](#c49-eventlogger-directive)) to `true`, to activate logging to a file, is advised instead. ## C.52 TimingOverview directive **Syntax:** `TimingOverview = ` *filename*\ **Default:** `TimingOverview = true`|`false` If set to `true`, the `TimingOverview` directive records timing information for CMDaemon. ## C.53 TimingOverviewFile directive **Syntax:** `TimingOverviewFile = ` *filename*\ **Default:** `TimingOverviewFile = "/var/spool/cmd/timing.overview.log"` The `TimingOverviewFile` directive sets the file where the timing data values for CMDaemon go. ## C.54 PublicDNS directive **Syntax:** `PublicDNS = true`|`false`\ **Default:** `PublicDNS = false` By default, internal hosts are resolved only if requests are from the internal network. Setting `PublicDNS` to true allows the head node name server to resolve internal network hosts for any network, including networks that are on other interfaces on the head node. Separate from this directive, port 53/UDP must also be opened up in Shorewall ([section 7.2 of the *Installation Manual*](/installation/thirdparty#72-shorewall)) if DNS is to be provided for queries from an external network. ## C.55 MaximalSearchDomains directive **Syntax:** `GlobalConfig = {"MaximalSearchDomains = `*`number`*`", ...}`\ **Default:** `32` The `MaximalSearchDomains` directive is a parameter of the `GlobalConfig` ([here](#c81-globalconfig-directive)) directive. It sets the maximum number of search domains that BCM configures for the cluster, and can be set to any value from 1 to 1024. The default was 6 before NVIDIA Base Command Manager point release 10.23.09, 32 from point release 10.23.09 onward, and 64 from point release 11.35.0 onward. On older versions of glibc (before 2.26), only the first 6 configured search domains are used by the operating system, regardless of this directive’s setting. For example, to raise the limit to 100 domains on a cluster running glibc 2.26 or later, the directive setting would be: `GlobalConfig = { "MaximalSearchDomains=100" } ` ## C.56 LockDownDhcpd directive **Syntax:** `LockDownDhcpd = ` `true`|`false`\ **Default:** `LockDownDhcpd = false` `LockDownDhcpd` is a deprecated legacy directive. If set to true, a global DHCP “deny unknown-clients” option is set. This means no new DHCP leases are granted to unknown clients for all networks. Unknown clients are nodes for which BCM has no MAC addresses associated with the node. The directive `LockDownDhcpd` is deprecated because its globality affects clients on all networks managed by BCM, which is contrary to the general principle of segregating the network activity of networks. The recommended way now to deny letting new nodes boot up is to set the option for specific networks by using `cmsh` or Base View (section [3.2.1](/admin/configuring#321-configuring-networks): figure [3.5](/admin/configuring#321-configuring-networks) and table [3.1](/admin/configuring#tablenetwork-configuration-settings)). Setting the `cmd.conf` `LockDownDhcpd` directive overrides `lockdowndhcpd` values set by `cmsh` or Base View. ## C.57 MaxNumberOfProvisioningThreads directive **Syntax:** `MaxNumberOfProvisioningThreads = ` *number*\ **Default:** `MaxNumberOfProvisioningThreads = 10000` The `MaxNumberOfProvisioningThreads` directive specifies the cluster-wide total number of nodes that can be provisioned simultaneously. Individual provisioning servers typically define a much lower bound on the number of nodes that may be provisioned simultaneously. ## C.58 SetupBMC directive **Syntax:** `SetupBMC = ` `true`|`false`\ **Default:** `SetupBMC = true` Automatically configure the username and password for the BMC interface of the head node. This may also be valid for regular nodes. The `SetupBMC` directive should not be confused with the `setupBmc` field of the node-installer configuration file, described in section [5.8.8](/admin/provisioning#588-node-installer-cannot-start-bmc-ipmiilo-interface). The node-installer normally takes care of BMC interface configuration on regular nodes by acting on the node-installer configuration file field `setupBmc`. The CMDaemon directive `SetupBMC` can only work on regular nodes if the node-installer is not configuring the regular nodes. If the boolean parameters `installbootrecord` and `allownetworkingrestart` for a regular node are set to `yes`, then the `SetupBMC` directive is able to work for that regular node. Setting the boolean parameters at category level also makes the directive work for the associated nodes. ## C.59 BMCSessionTimeout directive **Syntax:** `BMCSessionTimeout = ` *number*\ **Default:** `BMCSessionTimeout = 2000` The `BMCSessionTimeout` specifies the time-out for BMC calls in milliseconds. ## C.60 BMCIdentifyScript directive **Syntax:** `AdvancedConfig = {"BMCIdentify=`*filename*`", ...}`\ **Default:** *unset*\ `BMCIdentifyScript` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. The parameter takes a full file path to a script that can be used for [identification with a BMC (section 3.7.4)](/admin/configuring#374-identification-with-a-bmc). ## C.61 BMCIdentifyScriptTimeout directive **Syntax:** `AdvancedConfig = {"BMCIdentifyScriptTimeout=`*number from 1 to 360*`", ...}`\ **Default:** `60`\ `BMCIdentifyScriptTimeout` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. CMDaemon waits at the most `BMCIdentifyScriptTimeout` seconds for the script used by the `BMCIdentify` directive to complete. ## C.62 BMCIdentifyCache directive **Syntax:** `AdvancedConfig = {"BMCIdentifyCache=0|1", ...}`\ **Default:** `1`\ `BMCIdentifyCache` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. If set to `1`, then CMDaemon remembers the last value of the output of the script used by the `BMCIdentify` directive. ## C.63 SnmpSessionTimeout directive **Syntax:** `SnmpSessionTimeout = ` *number*\ **Default:** `SnmpSessionTimeout = 500000` The `SnmpSessionTimeout` specifies the time-out for SNMP calls in microseconds. ## C.64 PowerOffPDUOutlet directive **Syntax:** `PowerOffPDUOutlet = ` `true`|`false`\ **Default:** `PowerOffPDUOutlet = false` Enabling the `PowerOffPDUOutlet` directive allows PDU ports to be powered off for clusters that have both PDU and IPMI power control. Section [4.1.3](/admin/power#413-combining-pdu--and-ipmi-based-power-control) has more on this. ## C.65 PowerThreadPoolSize directive **Syntax:** `AdvancedConfig = {"PowerThreadPoolSize=` \<*`integer`*>`", ...}`\ **Default:** `PowerThreadPoolSize=32` `PowerThreadPoolSize` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. The parameter can take positive integer values. Increasing its value increases the number of threads that are used to power up the nodes in a cluster (section [4.2.3](/admin/power#423-power-operations-through-cmsh)), so that the cluster is fully operational quicker. The administrator should however take into account the power surge due to increasing the number of threads (number of subprocesses ) before increasing the value beyond its default. ## C.66 DisableBootLogo directive **Syntax:** `DisableBootLogo = ` `true`|`false`\ **Default:** `DisableBootLogo = false` When `DisableBootLogo` is set to `true`, the BCM logo is not displayed on the first boot menu when nodes PXE boot. ## C.67 StoreBIOSTimeInUTC directive **Syntax:** `StoreBIOSTimeInUTC = ` `true`|`false`\ **Default:** `StoreBIOSTimeInUTC = false` When `StoreBIOSTimeInUTC` is set to `true`, the system relies on the time being stored in BIOS as being UTC rather than local time. ## C.68 FreezeChangesTo\<*wlm*>Config directives: FreezeChangesToPBSPro directive FreezeChangesToUGEConfig directive FreezeChangesToSlurmConfig directive FreezeChangesToLSFConfig directive **Syntax:** `FreezeChangesTo` \<*wlm*>`Config= ` `true`|`false`\ **Default:** `FreezeChangesTo` \<*wlm*>`Config = false` When `FreezeChangesTo` \<*wlm*>`Config` is set to `true`, the CMDaemon running on that node does not make any modifications to the workload manager configuration for that node. Workload managers for which this value can be set are: * PBSPro * GE * Slurm * LSF Monitoring of jobs, and workload accounting and reporting continues for frozen workload managers. Upgrades to newer workload manager versions may still require some manual adjustments of the configuration file, typically if a newer version of the workload manager configuration changes the syntax of one of the options in the file. ## C.69 FrozenFile directive **Syntax:** `FrozenFile = { `*`filename`*`[, `*`filename`*`]... }`\ **Example:** `FrozenFile = { "/etc/dhcpd.conf", "/etc/postfix/main.cf" }`\ The `FrozenFile` directive is used to prevent the CMDaemon-maintained sections of configuration files from being automatically generated. This is useful when site-specific modifications to configuration files have to be made. The `filewriteinfo` command (section [A.2](/admin/generatedfiles#a2-system-configuration-files-created-or-modified-directly-on-the-node)) can be run from `device` mode to see which files are frozen. The output in the `Frozen` column then shows which files are protected from CMDaemon-managed changes. To avoid problems, the file that is frozen should not be a symlink, but should be the ultimate destination file. The `readlink -f` \<*symlinkname*> command returns the ultimate destination file of a symlink called \<*symlinkname*>. This is also the case for an ultimate destination file that is reached via several chained symlinks. ### C.69.1 FrozenFile directive for regular nodes #### C.69.1.1 FrozenFile directive for regular nodes for CMDaemon The `FrozenFile` directive can be used within the `cmd.conf` file of the regular node. **Example** To freeze the file `/etc/named.conf` on the regular nodes running with the image `default-image`, the file: `/cm/images/default-image/cm/local/apps/cmd/etc/cmd.conf` can have the following directive set in it: `FrozenFile = { "/etc/named.conf" }` The path of the file that is to be frozen on the regular node must be specified relative to the root of the regular node. The running node should then have its image updated. This can be done with the `imageupdate` command in `cmsh` (section [5.6.2](/admin/provisioning#562-updating-running-nodes-with-cmsh-using-imageupdate)), or the `Update node` button in Base View (section [5.6.3](/admin/provisioning#563-updating-running-nodes-with-base-view-using-the-update-node-option)). After the update, CMDaemon should be restarted within that category of nodes: **Example** ```bash [root@basecm10 ~]# pdsh -v -g category=default systemctl restart cmd node002: Waiting for CMDaemon (25129) to terminate... node001: Waiting for CMDaemon (19645) to terminate... node002: [ OK ] node001: [ OK ] node002: Waiting for CMDaemon to start...[ OK ] node001: Waiting for CMDaemon to start...[ OK ] ``` #### C.69.1.2 FrozenFile regex specification The `FrozenFile` directive allows regexes to be used for a path, if the path begins with the `|` character: **Example** ```bash [root@head current]# egrep -e '^FrozenFile' /cm/local/apps/cmd/etc/cmd.conf FrozenFile = { "/etc/postfix/main.cf", "|/cm/images/.*?/etc/postfix/main.cf" } ``` In the preceding entry, all image directories under `/cm/images/` are matched for the path `etc/postfix/main.cf`. #### C.69.1.3 FrozenFile directive for regular nodes for the node-installer CMDaemon directives only affect files on a regular node after CMDaemon starts up on the node during the init stage. So files frozen by the CMDaemon directive stay unchanged by CMDaemon after this stage, but they may still be changed before this stage. Freezing files so that they also stay unchanged during the pre-init stage—that is during the node-installer stage—is possible with node-installer directives. Node-installer freezing is independent of CMDaemon freezing, which means that if a file freeze is needed for the entire startup process as well as beyond, then both a node-installer as well as a CMDaemon freeze are sometimes needed. Node-installer freezes can be done with the node-installer directives in `/cm/node-installer/scripts/node-installer.conf`, introduced in section [5.4](/admin/provisioning#54-node-installer): * `frozenFilesPerNode` * `frozenFilesPerCategory` For the `node-installer.conf` file in multidistro and multiarch (section [11.7](/admin/softwareimagemanagement#117-creating-images-for-other-distributions-and-architectures-multidistro-and-multiarch)) configurations, the directory path `/cm/node-installer` takes the form:\ `/cm/node-installer-`*\-\*\ The values for \<*distribution*> and \<*architecture*> can take the values outlined [here](/admin/softwareimagemanagement#1171-the-cm-image-tool). **Example** Per node: ```bash frozenFilesPerNode = { "*:/localdisk/etc/ntp.conf", "node003:/localdisk/etc/hosts" } ``` Here, the `*` wildcard means that no restriction is set. Setting `node003` means that only node003 is frozen. **Example** Per category: ```bash frozenFilesPerCategory = { "mycategory:/localdisk/etc/sysconfig/network-scripts/ifcfg-eth1" } ``` Here, the nodes in the category `mycategory` are prevented from being changed by the node-installer. #### C.69.1.4 Is A `FrozenFile` Directive Always Needed? A `FrozenFile` directive is not always needed because in some cases a workaround is possible. In a configuration file after a node is fully up, the effect of a statement earlier on can often be overridden by a statement later in the file. So, the following useful behavior is independent of whether `FrozenFile` is being used for a configuration file or not: A configuration file, for example `/etc/postfix/main.cf`, with a configuration statement in an earlier CMDaemon-maintained part of the file, for example: ```bash mydomain = eth.cluster ``` can often be overridden by a statement later on outside the CMDaemon-maintained part of the file: ```bash mydomain = eth.gig.cluster ``` Using `FrozenFile` in CMDaemon or the node-installer can thus sometimes be avoided by the use of such overriding statements later on. Whether overriding later on is possible depends on the software being configured. It is true for Postfix configuration files, for example, but it may not be so for the configuration files of other applications. ## C.70 EaseNetworkValidation directive **Syntax:** `EaseNetworkValidation = ` `0`|`1`|`2`\ **Default:** `EaseNetworkValidation = 0` CMDaemon enforces certain requirements on network interfaces and management/node-booting networks by default. In heavily customized setups, such as is common in Type 3 networks ([section 3.3.9 of the *Installation Manual*](/installation/installing#339-network-topology)), the user may wish to disable these requirements. * `0` enforces all requirements. * `1` allows violation of the requirements, with validation warnings. This value should never be set except under instructions from BCM support. * `2` allows violation of the requirements, without validation warnings. This value should never be set except under instructions from BCM support. ## C.71 CustomUpdateConfigFileScript directive **Syntax:** `CustomUpdateConfigFileScript = ` *filename*\ **Default:** *commented out in the default `cmd.conf` file* Whenever one or more entities have changed, the custom script at *filename*, specified by a full path, is called 30s later. Python bindings can be used to get information on the current setup. ## C.72 ConfigDumpPath directive **Syntax:** `ConfigDumpPath = ` *filename*\ **Default:** `ConfigDumpPath = /var/spool/cmd/cmdaemon.config.dump` The `ConfigDumpPath` directive sets a dump file for dumping the configuration used by the power control script `/cm/local/apps/cmd/scripts/pctl/pctl`. The `pctl` script is a fallback script to allow power operations if CMDaemon is not running. * If no directive is set (`ConfigDumpPath = ""`), then no dump is done. * If a directive is set, then the administrator must match the variable `cmdconfigfile` in the powercontrol configuration file `/cm/local/apps/cmd/scripts/pctl/config.py` to the value of `ConfigDumpPath`. By default, the value of `cmdconfigfile` is set to `/var/spool/cmd/cmdaemon.config.dump`. ## C.73 SyslogHost directive **Syntax:** `SyslogHost = ` *hostname*\ **Default:** `SyslogHost = "localhost"` The `SyslogHost` directive specifies the hostname of the syslog host. ## C.74 SyslogFacility directive **Syntax:** `SyslogFacility = ` *facility*\ **Default:** `SyslogFacility = "LOG_LOCAL6"` The default value of `LOG_LOCAL6` is set in: * `/etc/rsyslog.conf` in RHEL, Ubuntu. * `/etc/syslog-ng/syslog-ng.conf` in SLES versions These are the configuration files for the default syslog daemons `syslog`, `rsyslog`, and `syslog-ng`, respectively, that come with the distribution. BCM redirects messages from CMDaemon to `/var/log/cmdaemon` only for the default syslog daemon that the distribution provides. So, if another syslog daemon other than the default is used, then the administrator has to configure the non-default syslog daemon facilities manually. The value of *facility* must be one of: `LOG_KERN`, `LOG_USER`, `LOG_MAIL`, `LOG_DAEMON`, `LOG_AUTH`, `LOG_SYSLOG` or `LOG_LOCAL0..7` ## C.75 NameServerLocalhostLocation directive **Syntax:** `AdvancedConfig = {"NameServerLocalhostLocation=0|1", ...}`\ **Default:** `NameServerLocalhostLocation=0` `NameServerLocalhostLocation` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. When set to `1`, the location of the localhost as specified by the nameserver directive in `/etc/resolv.conf` is moved to the bottom of the list of nameserver entries. The default value of `0` places it at the top of those entries. ## C.76 ResolveToExternalName directive **Syntax:** `ResolveToExternalName = ` `true`|`false`\ **Default:** `ResolveToExternalName = false` The value of the `ResolveToExternalName` directive determines under which domain name the primary and secondary head node hostnames are visible from within the head nodes, and to which IP addresses their hostnames are resolved. Enabling this directive resolves the head nodes’ hostnames to the IP addresses of their external interfaces. Thus, on head nodes and regular nodes in both single-head and failover clusters * with `ResolveToExternalName` disabled, the `master` hostname and the actual hostname of the head node (e.g. `head1`, `head2`) by default always resolve to the internal IP address of the head node. * with `ResolveToExternalName` enabled, the `master` hostname and the actual hostname of the head node (e.g. `head1`, `head2`) by default always resolve to the external IP address of the head node. The resolution behavior can be summarized by the following table:
*ResolveToExternalName Directive Effects*
**on simple head,** **on failover head,** **on regular node,** **Using**
**resolving:** **resolving:** **resolving:** **the**
**master** **head** **master** **head1** **head2** **master** **head(s)** **DNS?**
**ResolveToExternalName = False**
I I I I I I I No
I I I I I I I Yes
**ResolveToExternalName = True**
E E E E E E E No
E E E E E E E Yes
Key: I: resolves to internal IP address of head
E: resolves to external IP address of head
The system configuration files on the head nodes that get affected by this directive include `/etc/hosts` and, on SLES systems, also the `/etc/HOSTNAME`. Also, the DNS zone configuration files get affected. Additionally, in both the single-head and failover clusters, using the “`hostname -f`” command on a head node while `ResolveToExternalName` is enabled results in the host’s Fully Qualified Domain Name (FQDN) being returned with the host’s external domain name. That is, the domain name of the network that is specified as the "External network" in the base partition in `cmsh` (the output of “`cmsh -c "partition use base; get externalnetwork"`”). Modifying the value of the `ResolveToExternalName` directive and restarting the CMDaemon while important system services (e.g. Slurm) are running should not be done. Doing so is likely to cause problems with accessing such services due to them then running with a different domain name than the one with which they originally started. On a tangential note that is closely, but not directly related to the `ResolveToExternalName` directive: the cluster can be configured so that the “`hostname -f`” command executed on a regular node returns the FQDN of that node, and so that the FQDN in turn resolves to an external IP for that regular node. The details on how to do this are in the Knowledge Articles of the NVIDIA Enterprise Support Portal at [https://enterprise-support.nvidia.com/](https://enterprise-support.nvidia.com/). A search query for FQDN leads to the relevant entry. ## C.77 ResolveMasterToExternalName directive **Syntax:** `AdvancedConfig = {"ResolveMasterToExternalName=0|1", ...}`\ **Default:** `ResolveMasterToExternalName = 1` `ResolveMasterToExternalName` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. `ResolveMasterToExternalName` can only be used if `ResolveToExternalName` ([here](#c76-resolvetoexternalname-directive)) is active. If set to `1` (the default), then the master head node, as specified by the name `master`, resolves to the IP address as set by `ResolveToExternalName`. If set to `0` then the master head node resolves to the internal shared IP address. ## C.78 ResolveMasterToExternalDomainName directive **Syntax:** `AdvancedConfig = {"ResolveMasterToExternalDomainName=0|1", ...}`\ **Default:** `ResolveMasterToExternalDomainName = 1` `ResolveMasterToExternalDomainName` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. `ResolveMasterToExternalDomainName` can only be used if `ResolveToExternalName` ([here](#c76-resolvetoexternalname-directive)) is active. The external domain name as defined for the external network, in the form `master.`\<*external domain*>, can be used in `/etc/hosts` during name resolution. If set to `1` (the default), then the external domain name is used in `/etc/hosts`. If set to `0`, then the external domain name is not used in `/etc/hosts`. Resolution occurs as shown in the following table:
*ResolveMasterToExternalDomainName Directive Effects*
**on simple head,** **on failover head,** **on regular node,** **Using**
**resolving:** **resolving:** **resolving:** **the**
**master** **head** **master** **head1** **head2** **master** **head(s)** **DNS?**
**ResolveToExternalName = False**
I I I I I I I No
I I I I I I I Yes
**ResolveToExternalName = True**
E E I E E I I No
E E I E E I E Yes
Key: I: resolves to internal IP address of head
E: resolves to external IP address of head
## C.79 DisableLua directive **Syntax:** `DisableLua = ` `true`|`false`\ **Default:** `DisableLua = false` The value of the `DisableLua` directive determines if Lua code (section [L.6](/admin/filtermultiplex#l6-lua-node-execution-filters)) used in monitoring expressions can be executed. ## C.80 AdvancedConfig directive **Syntax:** `AdvancedConfig = ` `{ "=", "=", ...` }\ **Default:** *Commented out in the default `cmd.conf` file*\ The `AdvancedConfig` directive is not part of the standard directives. It takes a set of key/value pairs as parameters, with each key/value pair allowing a particular functionality, and is quite normal in that respect. However, the functionality of a parameter to this directive is often applicable only under restricted conditions, or for non-standard configurations. The `AdvancedConfig` parameters are therefore generally not recommended for use by the administrator, nor are they generally documented. Like for the other directives, only one `AdvancedConfig` directive line is used. This means that whatever functionality is to be enabled by this directive, its corresponding parameters must be added to that one line. These key/value pairs are therefore added by appending them to any existing `AdvancedConfig` key/value pairs, which means that the directive line can be a long list of key/value pairs to do with a variety of configurations. #### C.80.0.1 Managing Key/Value Pairs With The `cm-manipulate-advanced-config.py` Utility The `cm-manipulate-advanced-config.py` utility can be used to make it easier to manage `AdvancedConfig` key/value pairs. For example, to add a key/value pair `key8=value8`: ```bash [root@basecm10 ~]# cm-manipulate-advanced-config.py key8=value8 Updated: /cm/local/apps/cmd/etc/cmd.conf ``` To show the current state of the `AdvancedConfig`, the `-s|--show` option can be used: ```bash [root@basecm10 ~]# cm-manipulate-advanced-config.py -s === /cm/local/apps/cmd/etc/cmd.conf === VirtualCluster=1 key8=value8 ``` A key/value pair can be removed by specifying its key with the `-r|--remove` option: ```bash [root@basecm10 ~]# cm-manipulate-advanced-config.py -r key8 Updated: /cm/local/apps/cmd/etc/cmd.conf [root@basecm10 ~]# cm-manipulate-advanced-config.py -s === /cm/local/apps/cmd/etc/cmd.conf === VirtualCluster=1 ``` The utility can be used on `cmd.conf` in node images too, using the `-i|--image` option. ```bash [root@basecm10 ~]# cm-manipulate-advanced-config.py -i /cm/images/default-image Updated: /cm/images/default-image/cm/local/apps/cmd/etc/cmd.conf ``` The `-q` option causes the utility to exit with code 1 if `cmd.conf` has changed. Further options can be seen with the `-h|--help` option. ## C.81 GlobalConfig directive **Syntax:** `GlobalConfig = ` `{ "=", "=", ...` }\ **Default:** *not in the default `cmd.conf` file*\ The `GlobalConfig` directive is not part of the standard directives. It takes a set of key/value pairs as parameters, with each key/value pair allowing a particular functionality, and is quite normal in that respect. However, the parameter to this directive only needs to be specified on the head node. The non-head node CMDaemons take this value upon connection, which means that the `cmd.conf` file on the non-head nodes do not need to have this specified. This allows nodes to set up, for example, their search domains using the `MaximalSearchDomains` `GlobalConfig` directive ([here](#c55-maximalsearchdomains-directive)). Like for the other directives, only one `GlobalConfig` directive line is used. This means that whatever functionality is to be enabled by this directive, its corresponding parameters must be added to that one line. These key/value pairs are therefore added by appending them to any existing `GlobalConfig` key/value pairs, which means that the directive line can be a long list of key/value pairs to do with a variety of configurations. ## C.82 ScriptEnvironment directive **Syntax:** `ScriptEnvironment = ` `{ "CMD_ENV1=", "CMD_ENV2=", ...` }\ **Default:** *Commented out in the default `cmd.conf` file* The `ScriptEnvironment` directive sets extra environment variables for CMDaemon and child processes. For example, if CMDaemon is running behind a web proxy, then the environment variable `http_proxy` may need to be set for it. If, for example, the proxy is the host `brawndo`, and it is accessed via port 8080 using a username/password pair of `joe`/`electrolytes`, then the directive becomes: ```bash ScriptEnvironment = { "http_proxy=joe:electrolytes@brawndo:8080" } ``` ## C.83 BurnSpoolDir directive **Syntax:** `BurnSpoolDir = ` *path*\ **Default:** `BurnSpoolDir = "/var/spool/burn/"` The `BurnSpoolDir` directive specifies the directory under which node burn log files are placed ([Chapter 11 of the *Installation Manual*](/installation/burn)). The log files are logged under a directory named after the booting MAC address of the NIC of the node. For example, for a MAC address of 00:0c:29:92:55:5e the directory is `/var/spool/burn/00-0c-29-92-55-5e`. ## C.84 IdleThreshold directive **Syntax:** `IdleThreshold = ` *number*\ **Default:** `IdleThreshold = 1.0` The `IdleThreshold` directive sets a threshold value for `loadone`. If `loadone` exceeds this value, then data producers that have `Only when idle` ([here](/admin/monitoring#1241-monitoring-configuration-data-producers)) set to `true` (`enabled`), will not run. If the data producer is sampled on a regular node rather than on the head node, then `cmd.conf` on the regular node should be modified and its CMDaemon restarted. ## C.85 MonitoringPath directive **Syntax:** `AdvancedConfig = {"MonitoringPath=`*path*`", ...}`\ **Default:** Implicit value: `"MonitoringPath=/var/spool/cmd/monitoring/"` `MonitoringPath` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. Its value determines the path of the directory in which monitoring data is saved (section [16.8](/admin/administration#168-managing-raw-monitoring-data)). ## C.86 MaxServiceFailureCount directive **Syntax:** `AdvancedConfig = {"MaxServiceFailureCount=`*number*`", ...}`\ **Default:** Implicit value: `"MaxServiceFailureCount=10"` `MaxServiceFailureCount` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. Its value determines the number of times a service failure event is logged ([here](/admin/configuring#31424-monitoring-a-service-with-cmsh-and-base-view)). Restart attempts on the service still continue when this number is exceeded. ## C.87 InitdScriptTimeout directive **Syntax:** `AdvancedConfig = {"InitdScriptTimeout[.`*service*`]=`*timeout*`", ...}`\ **Default:** Implicit value: `"InitdScriptTimeout=30"` `InitdScriptTimeout` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. It can be set globally or locally: * **Global (all services)**\ `InitdScriptTimeout` can be set as a global value for init scripts, by assigning *timeout* as a period in seconds. If an init script fails to start up its service within this period, then CMDaemon kills the service and attempts to restart it. * If `InitdScriptTimeout` has a value for *timeout* set, then all init scripts have a default timeout of *timeout* seconds. * If `InitdScriptTimeout` has no *timeout* value set, then all init scripts have a default timeout of 30 seconds. * **Local (for a specific service)**\ If `InitdScriptTimeout.`*service* is assigned a *timeout* value, then the init script for that *service* times out in *timeout* seconds. This timeout overrides, for that service only, any existing global default timeout. When a timeout happens for an init script attempting to start a service, the event is logged. If the number of restart attempts exceeds the value determined by the `MaxServiceFailureCount` directive ([here](#c86-maxservicefailurecount-directive)), then the event is no longer logged, but the restart attempts continue. **Example** An fhgfs startup takes a bit longer than 30 seconds, and therefore times out with the default timeout value of 30s. This results in the following logs in `/var/log/cmdaemon`: ```bash cmd: [SERVICE] Debug: ProgramRunner: /etc/init.d/fhgfs-client restart [DONE] 0 9 cmd: [SERVICE] Debug: /etc/init.d/fhgfs-client restart, exitcode = 0, signal = 9 ``` Here, *service* is `fhgfs-client`, so setting the parameter can be done with: ```bash AdvancedConfig = { ..., "initdScriptTimeout.fhgfs-client=60", ...} ``` This allows a more generous timeout of 60 seconds instead. Restarting CMDaemon then should allow the fhgs startup to complete ```bash # systemctl restart cmd ``` A more refined approach that avoids a complete CMDaemon restart would be to execute a `reset` ([here](/admin/configuring#31424-monitoring-a-service-with-cmsh-and-base-view)) on the `fhgfs-client` from within CMDaemon, as follows: ```bash [basecm10->category[default]->services[fhgfs-client]]% reset fhgfs-client Successfully reset service fhgfs-client on: node001,node002 [basecm10->category[default]->services[fhgfs-client]]% ``` ## C.88 CMDaemonListenOnInterfaces directive **Syntax:** `AdvancedConfig = {"CMDaemonListenOnInterfaces=` \<*interfaces*>`", ...}`\ **Default:** *all interfaces listening to port 8081* `CMDaemonListenOnInterfaces` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. When set explicitly, CMDaemon listens only to the interfaces listed in \<*interfaces*>. The form of \<*interfaces*> is a comma-separated list of interface device names: **Example** ```bash CMDaemonListenOnInterfaces=eth0,eth1,eth0:0,eth0:1 ``` If the interface list item `lo` is omitted from the list of names, it will still be listened to. This is because CMDaemon must always be able to talk to itself on the loopback interface. ## C.89 DisableInotifyInterface directive **Syntax:** `AdvancedConfig = {"DisableInotifyInterface =0|1"}`\ **Default:** `0` `DisableInotifyInterface` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. When set to `1` CMDaemon ignores inotify events from all interfaces. When set to `0` CMDaemon does not ignore inotify events from all interfaces. ## C.90 IgnoreInotifyInterface directive **Syntax:** `AdvancedConfig = {"IgnoreInotifyInterface =` \<*interfaces*>`", ...}`\ **Syntax:** `GlobalConfig = {"IgnoreInotifyInterface =` \<*interfaces*>`", ...}`\ **Default:** `veth*,lxc*,fg-*,qg-*,qr-*,sg-*,tbr-*,qbr*,qvb*,qvo*,cali*,flannel*,ecu_*,chassis_*, enx*,az*,tap*,rdma*,cni0,/[a-f0-9]12``_[hc]/` *In the preceding lines showing the value for* `Default`*, the line ending with* `chassis_*,` *must be concatenated to the line starting with* `enx*,` `IgnoreInotifyInterface` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive, as well as the `GlobalConfig` ([here](#c81-globalconfig-directive)) directive. `IgnoreInotifyInterface` is active if `DisableInotifyInterface` is `0`. By default, the directive has CMDaemon ignore the normally harmless messages that are typically generated by Kubernetes as Calico creates and removes interfaces. CMDaemon ignores events from the interfaces listed in \<*interfaces*>. The form of \<*interfaces*> is a comma-separated list of interface device names, with some globbing allowed. ## C.91 CookieCooldownTime directive **Syntax:** `AdvancedConfig = {"CookieCooldownTime=`*number from 60 to 86400*`", ...}`\ **Default:** `900` `CookieCooldownTime` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. It defines the number of seconds until the Base View connection to CMDaemon times out, if there is no user activity at the Base View client side. ## C.92 DHCPMaxleaseTime directive **Syntax:** `AdvancedConfig = {"DHCPMaxleaseTime=`*integer from 60 to 604800*`", ...}`\ **Default:** `100800` `DHCPMaxleaseTime` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. `DHCPMaxleaseTime`, which is the time in seconds, sets both `default-lease-time` and `max-lease-time` in the DHCP server configuration on the head node. The maximum lease time is the longest time that the DHCP server allows to the DHCP client on the node. The default value of 100800 corresponds to 28 hours. The minimum value for this directive is `60`, and the maximum value is `604800`, which corresponds to 7 days. Values from `0` to `59` are treated as `60`, and a value greater than `604800` is treated as `604800`. A value of `-1` is not treated as an unlimited lease time, and is treated as `604800`. From BCM point release 10.32.0 onward, a warning is logged in the CMDaemon log file when a value is treated in this way. ## C.93 SlurmStraightExtraTopology directive **Syntax:** `AdvancedConfig = {"SlurmStraightExtraTopology=0|1", ...}`\ **Default:** `1` `SlurmStraightExtraTopology` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. If set to `0`, then the order of switches is reversed when setting extra values for `SlurmTopology` ([here](/admin/workloadmanagement#7514-configuring-slurm)). ## C.94 SlurmConcatTopologySwitchName directive **Syntax:** `AdvancedConfig = {"SlurmConcatTopologySwitchName=0|1", ...}`\ **Default:** `0` `SlurmConcatTopologySwitchName` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. If set to `1`, then it allows concatenation of the switch names in the topology defined via the `SlurmTopology` extra values ([here](/admin/workloadmanagement#7514-configuring-slurm)). ## C.95 SlurmDrainReasonPolicy directive **Syntax:** `AdvancedConfig = {"SlurmDrainReasonPolicy=` \<*policy*>`", ...}`\ **Default:** `SlurmDrainReasonPolicy=replace` The `SlurmDrainReasonPolicy` directive is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. The `drain` command (section [7.7.3](/admin/workloadmanagement#773-nodes-drainage-status-and-handling-in-cmsh)) can be used by a user to drain a node, or a monitoring trigger (section [12.4.5](/admin/monitoring#1245-monitoring-configuration-triggers)) can make CMDaemon carry out a drain action if, for example, a health check fails. Outside of CMDaemon control, Slurm’s own `scontrol` command (`man(1) scontrol`) can be used. A drain reason can be specified. By default, the policy is that if a node is already drained, then a second drain command with a new drain reason replaces the old drain reason. The policy can be modified by changing \<*policy*> to one of the following values: * `replace`: the old drain reason(s) is replaced by a new one * `append`: the new drain reason is separated by a comma, and appended to the existing one(s). * `skip`: * If a drain reason already exists, then setting the new drain reason is skipped. * If no drain reason already exists, then the new drain reason is applied. Capitalization does not matter for the value of \<*policy*>. Beyond the BCM version 10 release, the advanced configuration directive `SlurmDrainReasonPolicy` is replaced by a `cmsh` Slurm WLM parameter, `drainreasonpolicy`. ## C.96 SlurmTopologyGenerator directive **Syntax:** `AdvancedConfig = {"SlurmTopologyGenerator=0|1"}`\ **Default:** `0` `SlurmTopologyGenerator` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. The directive is used only in later versions of BCM 10. If the parameter is set to `1`, then it allows the Topograph service ([here](/admin/workloadmanagement#slurm-topology-generator)) to generate a Slurm topology configuration for the Slurm `topology/tree` plugin ([https://slurm.schedmd.com/topology.conf.html](https://slurm.schedmd.com/topology.conf.html)). The generation happens if a Slurm compute node is affected by: * poweroff, shutdown, or power up * creation or termination in cloud nodes * a Slurm-related configuration change due to a BCM configuration change The directive is added to the nodes that run the `slurmserver` role, along with the `TopographHostname` directive that follows. ## C.97 TopographHostname directive **Syntax:** `AdvancedConfig = {"TopographHostname=` \<*hostname or IP address*>`"}`\ **Default:** `localhost` `TopographHostname` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. The directive is used only in later versions of BCM 10. The directive must be added to the same nodes as the preceding `SlurmTopologyGenerator` directive. The parameter must be set to the hostname that the Topograph service ([here](/admin/workloadmanagement#slurm-topology-generator)) runs on. If the Topograph service is running on a node without the `slurmserver` role, then the value must be manually adjusted from its default setting of `localhost`. ## C.98 JobsSamplingMetricsInterval directive **Syntax:** `AdvancedConfig = {"JobsSamplingMetricsInterval=` \<*number*>`", ...}`\ **Default:** `60` `JobsSamplingMetricsInterval` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. Its value is a time period, in seconds, and it applies only to metrics associated with job queues. Such metric sampling is carried out with this time period if job queues are added, or if job queues are re-created after disappearing. ## C.99 MembershipQueryInterval directive **Syntax:** `AdvancedConfig = {"MembershipQueryInterval=` \<*number*>`", ...}`\ **Default:** `4`\ `MembershipQueryInterval` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. Its value is a time period, in seconds. This time period value elapses between the checks that CMDaemon makes to determine the node states (section [5.5](/admin/provisioning#55-node-states)) in a cluster. If the network is very congested, then a larger value can be used to reduce the network load caused by these checks. ## C.100 AddUserScript directive **Syntax:** `AdvancedConfig = {"AddUserScript=` \<*path*>`", ...}`\ **Default:** *none* `AddUserScript` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. If this parameter is set to a path leading to a script, and if a new user is added using `cmsh` or Base View, then the script is automatically run by CMDaemon, with the username of the new user automatically passed as the first argument to the script. The script has a default timeout of 5 seconds. ## C.101 AddUserScriptPasswordInEnvironment directive **Syntax:** `AdvancedConfig = {"AddUserScriptPasswordInEnvironment=0|1", ...}`\ **Default:** `0` `AddUserScriptPasswordInEnvironment` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. If this parameter is set to `1`, then CMDaemon passes the `CMD_USER_PASSWORD` environment variable to the script defined by the `AddUserScript` directive. ## C.102 RemoveUserScript directive **Syntax:** `AdvancedConfig = {"RemoveUserScript=` \<*path*>`", ...}`\ **Default:** *none* `RemoveUserScript` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. If this parameter is set to a path leading to a script, and if an existing user is removed using `cmsh` or Base View, then the script is automatically run by CMDaemon. The script has a default timeout of 5 seconds. ## C.103 AddUserScriptTimeout directive **Syntax:** `AdvancedConfig = {"AddUserScriptTimeout=` \<*number*>`", ...}`\ **Default:** `5` `AddUserScriptTimeout` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. It sets the timeout value in seconds, for the script set by `AddUserScript`. ## C.104 RemoveUserScriptTimeout directive **Syntax:** `AdvancedConfig = {"RemoveUserScriptTimeout=` \<*number*>`", ...}`\ **Default:** `5` `RemoveUserScriptTimeout` is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. It sets the timeout value in seconds, for the script set by `RemoveUserScript`. ## C.105 AdditionalSubmitHosts directive AdditionalExecHosts directive **Syntax:**\ `AdvancedConfig = {"AdditionalSubmitHosts=` \<*`host1`*>`,<`*`host2`*>,...", ...}\ or\ `AdvancedConfig = {"AdditionalExecHosts=` \<*`host1`*>`,<`*`host2`*>,...", ...}\ **Default:** *none* The `AdditionalSubmitHosts` and `AdditionalExecHosts` directives are parameters of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. These directives can be used to make BCM aware of the existence of a submit host or execution host that is outside of BCM control. The directives can be used with the GE workload manager only, so that CMDaemon does not remove such hosts from the workload manager configuration file during a configuration file maintenance run. An example of their use is given [here](/admin/workloadmanagement#7523-ge-submission-administrative-and-execution-hosts). ## C.106 AutomaticMountAll directive **Syntax:** `AdvancedConfig = {"AutomaticMountAll=0|1", ...}`\ **Default:** `1` The `AutomaticMountAll` directive is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. If set to the default of `1`, then a `mount -a` operation is carried out when a mount change is carried out by CMDaemon. The `mount -a` operation has to do with attempting to mount devices listed in `/etc/fstab`. It should not be confused with auto-mounting of filesystems, which has to do with mounting an arbitrary device to a filesystem automatically. If the `AutomaticMountAll` directive is set to `0`, then `/etc/fstab` is written, but the `mount -a` command is not run by CMDaemon. However, the administrator should be aware that since `mount -a` is run by the distribution during booting, a node reboot implements the mount change. The directive is only set in the `cmd.conf` file of the compute nodes, and not the head node(s). ## C.107 AllowImageUpdateWithAutoMount directive **Syntax:** `AdvancedConfig = {"AllowImageUpdateWithAutoMount=0|1|2|3", ...}`\ **Default:** `0` The `AllowImageUpdateWithAutoMount` directive is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. The values it takes decide how an auto-mounted filesystem should be dealt with during image updates (section [5.6.2](/admin/provisioning#562-updating-running-nodes-with-cmsh-using-imageupdate)) or grab to image (syncing node-to-image, section [11.5.2](/admin/softwareimagemanagement#1152-managing-the-software-directly-on-a-node-then-syncing-node-to-image)). It must be set per software image or per node.
**Value** **Description**
`0` If auto-mount is running, abort provisioning (default)
`1` If auto-mount is running, warn but continue
`2` Do not check auto-mount status. This saves a little time, but it risks data loss, unless the automounted filesystem has been added to `excludelistupdate` . If the automounted filesystem is not added to `excludelistupdate` , then if the automounted filesystem happens to be unavailable at the time that an image update is carried out, then the rsync process can end up deleting the automounted filesystem contents during the rsync, because it assumes that content should not be there.
`3` Pretend auto-mount is running. This prevents an image update
How an auto-mounted filesystem can be configured using the `autofs` service in BCM is discussed in section [3.13](/admin/configuring#313-configuring-nfs-volume-exports-and-mounts). The need for adding the automounted filesystem to `excludelistupdate` is discussed [here](/admin/provisioning#automounterbeware). ## C.108 DNS::options_allow-query directive **Syntax:** `AdvancedConfig = {"DNS::options_allow-query=` \<*subnet1*>`,<`*subnet2*>,...",...}\ **Default:** *unset* The `DNS::options_allow-query` directive is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. If a subnet value is specified, in CIDR notation, then that subnet can query the DNS running on the head node. Setting a subnet places an entry within the `allow-query` section of `/etc/named.conf`. Extra subnets must specified with comma-separation, without whitespace. The standard directive `PublicDNS` ([here](#c54-publicdns-directive)) simply adds the entry 0.0.0.0/0 to the `allow-query` section and can be used if no specific subnet needs to be added. ## C.109 CipherList directive **Syntax:** `AdvancedConfig = {"CipherList=` \<*ciphers*>`", ...}`\ **Default:** `CipherList=ALL:!aNULL:!eNULL` The `CipherList` directive is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. It sets the cipher list of the OpenSSL suite that CMDaemon negotiates with clients. Ciphers in the cipher list can be viewed with: **Example** ```bash [root@basecm10 ~]# openssl ciphers -v ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESGCM(256) Mac=AEAD ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AES(256) Mac=SHA384 ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AES(256) Mac=SHA384 ECDHE-RSA-AES256-SHA SSLv3 Kx=ECDH Au=RSA Enc=AES(256) Mac=SHA1 ... ``` The columns are: the cipher name, SSL protocol, key exchange used, authentication mechanism, encryption used, and MAC digest algorithm. Ciphers can be specified for the `CipherList` directive according to the specification described in `man ciphers.1`. For example, as: **Example** ```bash AdvancedConfig = {"CipherList=ALL:!aNULL:!ADH:!eNULL:!LOW:!EXP:RC4+RSA:+HIGH:+MEDIUM"}\\ ``` An analogous non-CMDaemon directive is found in LDAP client-server negotiation, where `TLSCipherSuite` can be set in the `slapd.conf` file, described in `man slapd.conf.5`. ## C.110 SSLServerMethod directive (with TLS masking) **Syntax:** `AdvancedConfig = {"SSLServerMethod=TLS` \<*versionnumber*>`", ...}`\ **Syntax:** `GlobalConfig = {"SSLServerMethod=TLS` \<*versionnumber*>`", ...}`\ **Masking syntax:** `AdvancedConfig = {"TLS` \<*versionnumber*>`=<`*boolean*>", ...}\ **Masking syntax:** `GlobalConfig = {"TLS` \<*versionnumber*>`=<`*boolean*>", ...}\ **Default:** `GlobalConfig = { "TLS 1.2=1", "TLS 1.3=1", ... }` The `SSLServerMethod` directive is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive, as well as the `GlobalConfig` ([here](#c81-globalconfig-directive)) directive. It sets the SSL server method for the OpenSSL suite that CMDaemon negotiates with clients. Possible values for \<*versionnumber*> are: * `1` * `1.1` * `1.2` * `1.3` By default, with no `SSLServerMethod` directive, TLS versions 1.2 and 1.3 are enabled. This is because there are still some SSL clients that require a TLS 1.2 cipher and have no TLS 1.3 cipher negotiation ability. If the `SSLServerMethod` directive is specified, then only that TLS version is negotiated. For example, to allow only TLS 1.2 negotiation: **Example** ```bash GlobalConfig = { "SSLServerMethod=TLS 1.2", ... } ``` The `SSLServerMethod` directive is implied by the `TLS` masking syntax. The TLS masking syntax allows TLS negotiation methods to be set concurrently. For example, TLS 1 and TLS1.1 could be disabled, and TLS 1.2 and TLS 1.3 could be enabled, with: **Example** ```bash GlobalConfig = { "TLS 1=0", "TLS 1.1=0", "TLS 1.2=1", "TLS 1.3=1", ... } ``` The `GlobalConfig` value is the directive that the cluster administrator should set in most cases. It only needs to be set on `cmd.conf` on the head node (or head nodes in a high availability configuration), in order to configure the same TLS negotiation settings for the regular nodes. The `GlobalConfig` value is overridden by the `AdvancedConfig` value. The `AdvancedConfig` value needs to be set on `cmd.conf` on the head node(s), but also needs to be set on the non-head nodes that are to be disabled. Setting the `AdvancedConfig` directive allows custom configuration of the TLS negotiations to be carried out per node, and if the negotiations do not match with the other end, then other settings are tried. This does mean that sometimes nodes with the extra settings end up running extra processes for no reason. Running the directive as an `AdvancedConfig` is therefore suboptimal for most reasonable use cases. Reverting to the TLS 1.1 fallback cipher availability of earlier versions of CMDaemon is possible by setting:\ `AdvancedConfig = { "TLS 1.1=1", "TLS 1.2=1", "TLS 1.3=1" }`\ Setting this brings with it the risk of allowing cryptographic downgrade attacks. It is therefore not recommended, and is also why it is disabled by default. ## C.111 JobInformationDisabled directive **Syntax:** `AdvancedConfig = {"JobInformationDisabled=0|1", ...}`\ **Syntax:** `GlobalConfig = {"JobInformationDisabled=0|1", ...}`\ **Default:** `JobInformationDisabled=0` The `JobInformationDisabled` directive is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive, as well as the `GlobalConfig` ([here](#c81-globalconfig-directive)) directive. If set to `1` it disables job-based monitoring (Chapter [13](/admin/monitoringjob)). The details of how this is done are discussed shortly. For a cluster that is running millions of jobs at a time, job-based monitoring can typically consume significant resources. The monitoring of data for so many small jobs is typically not useful and not of interest. For such a case, setting this directive improves cluster performance by not having to deal with the information on the current jobs running on the cluster. The `GlobalConfig` value is the directive that the cluster administrator should set in most cases. It only needs to be set on `cmd.conf` on the head node (or head nodes in a high availability configuration). Setting it disables the collection of job information by the nodes running the jobs, and no collection of the job information from those nodes is done by the head (monitoring) node. The `GlobalConfig` value is overridden by the `AdvancedConfig` value. The `AdvancedConfig` value needs to be set on `cmd.conf` on the head node(s), but also needs to be set on the non-head nodes that are to be disabled. Setting the `AdvancedConfig` directive still allows job information to be collected by the nodes running the jobs, but the information is not collected from the nodes by the head (monitoring) node. This means that nodes are running extra processes for probably no reason. Running the directive as an `AdvancedConfig` is therefore suboptimal for most use cases. ## C.112 JobInformationKeepDuration directive **Syntax:** `AdvancedConfig = {"JobInformationKeepDuration=` \<*number*>`", ...}`\ **Default:** `JobInformationKeepDuration=2419200` The `JobInformationKeepDuration` directive is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. It takes on a value in seconds. If a job has finished more than that many seconds ago, then it will be removed along with all its monitoring data from the database. By default it is set to 28 days (24×3600×28 seconds). If `persistent` is set to `yes` (`cmsh:wlm>jobs`), then job information is not removed even after `JobInformationKeepDuration` has been exceeded. ## C.113 JobInformationChargeBackKeepDuration directive **Syntax:** `AdvancedConfig = {"JobInformationChargeBackKeepDuration=` \<*number*>`", ...}`\ **Default:** `JobInformationChargeBackKeepDuration=158112000` The `JobInformationChargeBackKeepDuration` directive is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. It takes on a value in seconds. If a job has finished more than that many seconds ago, then it will be removed along with all its monitoring data from the database. By default it is set to a little more than 5 years (24×3600×366×5 seconds). ## C.114 JobInformationKeepCount directive **Syntax:** `AdvancedConfig = {"JobInformationKeepCount=` \<*number*>`", ...}`\ **Default:** `JobInformationKeepCount=8192` The `JobInformationKeepCount` directive is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. If the total number of jobs is greater than `(JobInformationKeepCount + 10%)`, then the job record and its data content are discarded, starting from the oldest job first, until the total number of jobs remaining becomes `JobInformationKeepCount`. If it is set to 0, then none of the job records and content are removed. The maximum value for this directive is `65536`. A larger value is treated as `65536`. ## C.115 JobInformationChargeBackKeepCount directive **Syntax:** `AdvancedConfig = {"JobInformationChargeBackKeepCount=` \<*number*>`", ...}`\ **Default:** `JobInformationChargeBackKeepCount=1048576` The `JobInformationChargeBackKeepCount` directive is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. If the total number of jobs is greater than `(JobInformationChargeBackKeepCount + 10%)`, then the job record and its data content are discarded, starting from the oldest job first, until the total number of jobs remaining becomes `JobInformationChargeBackKeepCount`. If it is set to 0, then none of the job records and content are removed. The default value of about 1 million corresponds to about 0.5 GB of storage. The maximum value for this directive is about 1 billion (1,073,741,824), which corresponds to about 500 GB of storage. ## C.116 JobInformationMinimalJobDuration directive **Syntax:** `AdvancedConfig = {"JobInformationMinimalJobDuration=` \<*number*>`", ...}`\ **Default:** `JobInformationMinimalJobDuration=0` The `JobInformationMinimalJobDuration` directive is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. If set, then jobs that run for less than this number of seconds are not stored in the cache. Its default value of `0` seconds means that all jobs are handled. ## C.117 JobInformationFlushInterval directive **Syntax:** `AdvancedConfig = {"JobInformationFlushInterval=` \<*number*>`", ...}`\ **Default:** `JobInformationFlushInterval=600` The `JobInformationFlushInterval` directive is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. If this interval, in seconds, is set, then the cache is flushed to the database with that interval. Its default value is 10 minutes (10×60 seconds). Values of around `30` seconds or less will conflict with the default CMDaemon maintenance timeout value of `30` seconds, and will mostly simply add load. ## C.118 JobInformationChargeBackRemoveInterval directive **Syntax:** `AdvancedConfig = {"JobInformationChargeBackRemoveInterval=` \<*number*>`", ...}`\ **Default:** `JobInformationChargeBackRemoveInterval=600` The `JobInformationChargeBackRemoveInterval` directive is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. If this interval, in seconds, is set, then the cache is flushed to the database with that interval. Its default value is 10 minutes (10×60 seconds). Values of around `30` seconds or less will conflict with the default CMDaemon maintenance timeout value of `30` seconds, and will mostly simply add load. The maximum remove interval is 1 day (86400 seconds). ## C.119 ActionLoggerFile directive **Syntax:** `AdvancedConfig = {"ActionLoggerFile=`*filename*`", ...}`\ **Default:** `/var/spool/cmd/actions.log` The directive is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. Its value overrides the default path. The directive needs to be implemented per node or image. ## C.120 ActionLoggerOnSuccess directive **Syntax:** `AdvancedConfig = {"ActionLoggerOnSuccess=0|1", ...}`\ **Default:** `ActionLoggerOnSuccess=0` The directive is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. By default, only failed actions are logged. Successful actions are also logged if setting `ActionLoggerOnSuccess=1` The directive needs to be implemented per node or image. The log file shows timestamped output with one line per run for an action script, with the script response. **Example** ```bash (time) /path/to/action/script [ timeout ] (time) /path/to/action/script [ failed ] (exit code: 1) (time) /path/to/action/script [ success ] ``` The `success` line only appears if `ActionLoggerOnSuccess=1`. ## C.121 FailoverPowerRetries directive **Syntax:** `AdvancedConfig = {"FailoverPowerRetries=` \<*number*>`", ...}`\ **Default:** `FailoverPowerRetries=5` The `FailoverPowerRetries` directive is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. After a decision to carry out the failover has been made, CMDaemon sends a power off command to the BMC of the head node that is meant to be powered off. If the power off command fails, then on getting the fail response, CMDaemon waits for 1 second. After that second, it sends out a power off command again. The value of `FailoverPowerRetries` is the number of times that CMDaemon retries sending the power off command to the BMC of the head node that is intended to be powered off during the failover, if the response from the power off command remains a fail response. The power down attempts cease, either when the BMC reports that the head node is OFF, or when the number of attempts reaches the `FailoverPowerRetries` value. `FailoverPowerRetries` takes a maximum value of 120. A value of 0 means that 1 attempt to power off is carried out during failover, but no retry is attempted after the first attempt. Because CMDaemon waits for a period of 1s before checking for an OFF reponse, it means that the number of retries is about the same as the number of seconds before CMDaemon decides that powering off has failed, unless the OFF response also takes some time to get to CMDaemon. Increasing the value for this directive can be useful for some BMC cards that take longer than about 5s to report their power status, because the power off attempt may otherwise time out. ## C.122 AddUserDefaultGroupID directive **Syntax:** `AdvancedConfig = { "AddUserDefaultGroupID =` \<*number*>`", ...}`\ **Default:** *none* The `AddUserDefaultGroupID` directive is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. If the `AddUserDefaultGroupID` is unset, then the default group ID of a new user is the same as the User ID. If the `AddUserDefaultGroupID` is set, then the set value becomes the default GUID of new users, when new users are created via the `cmsh` or Base View front ends to CMDaemon. The directive is not intended to set a non-default group occasionally during user creation. In that case, a non-default group ID can be set from the `cmsh` or Base View front ends, by setting the `groupid` value for the user. To set a default group for a directory, while retaining the default group for the user, it may be possible to use the setgid bit for a directory. ## C.123 MaxMeasurablesPerProducer directive **Syntax:** `AdvancedConfig = { "MaxMeasurablesPerProducer =` \<*number*>`", ...}`\ **Default:** `500` The `MaxMeasurablesPerProducer` directive is a parameter of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. By default there is a software limit of 500 measurables per data producer. If this limit is exceeded, then the CMDaemon monitoring info logs show complaints about “too many measurables”. In that case, if the cluster hardware is not too slow, and if the measurables produced are not some kind of hardware garbage data values, then increasing the value of this directive should allow more measurables to be dealt with. ## C.124 HeadNodeCertificateNameAlternatives directive **Syntax:** `AdvancedConfig = {"HeadNodeCertificateNameAlternatives=<`*name*,…`>",...}`\ **Default:** *none* The `HeadNodeCertificateNameAlternatives` directive is a key of the `AdvancedConfig` ([here](#c80-advancedconfig-directive)) directive. It takes on a comma-separated list of alternative DNS names as values to add to the SSL certificate that it generates for CMDaemon. Extra names must specified with comma-separation, without whitespace. The old certificates must be removed before the new ones are used. **Example** ```bash [root@basecm10 ~]# cm-manipulate-advanced-config.py HeadNodeCertificateNameAlternatives=myname1,myname2 [root@basecm10 ~]# openssl x509 -in /cm/local/apps/cmd/etc/cert.pem -noout -text | grep -o DNS:myname1.*$ [root@basecm10 ~]# rm /cm/local/apps/cmd/etc/cert.pem,key [root@basecm10 ~]# systemctl restart cmd #new certificates auto-generated [root@basecm10 ~]# openssl x509 -in /cm/local/apps/cmd/etc/cert.pem -noout -text | grep -o DNS:myname1.*$ DNS:myname1, DNS:myname1:8081, DNS:myname2, DNS:myname2:8081 ```