> This page is for version 10. > For other versions, use one of these documentation indexes: > - 11 (default): https://docs.nvidia.com/base-command-manager/product-manuals/11/llms.txt > - 10: https://docs.nvidia.com/base-command-manager/product-manuals/10/llms.txt > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://docs.nvidia.com/base-command-manager/product-manuals/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/base-command-manager/product-manuals/_mcp/server. # BeeGFS ## 10.1 BeeGFS Introduction Integration of BeeGFS with BCM is a legacy option that is no longer supported. It is discussed in this chapter for legacy cluster purposes only. ### 10.1.1 BeeGFS Concepts BeeGFS is a high-performance parallel file system, developed by the Fraunhofer Competence Center for High Performance Computing, and optimized for intensive I/O. It uses a distributed metadata architecture designed for scalability and flexibility. More information about BeeGFS can be found at the official online documentation at [https://www.beegfs.io/content/documentation/](https://www.beegfs.io/content/documentation/). NVIDIA Base Command Manager provides packages to allow BeeGFS to be deployed, managed, and monitored on a cluster running RHEL and derivatives. The deployment tool provided by BCM is `cm-beegfs-setup` (section [10.2](#102-deployment-and-uninstallation-of-beegfs-with-cm-beegfs-setup)). A BeeGFS cluster consists of a management server, one or more metadata servers and one or more storage servers. Clients should have client servers running on them. If high-availability is required, then the administrator needs to arrange for it separately, for the BeeGFS management servers. Since BeeGFS is a parallel filesystem, it means that adding more nodes increases not only the storage capacity, but also system performance. BeeGFS can work with any local file system for data storage as long as it is POSIX compliant. By default, the latest version of BeeGFS is supported. However, at the time of writing (November 2022), the latest BeeGFS version is 7.3, which is supported for RHEL8 and variants. The BeeGFS Mon monitoring service is on the roadmap for future BCM integration. ### 10.1.2 BeeGFS Installation Notes And Options By default all logging is done to the systemd journal. The installation logs can be viewed using: `journalctl -u 'beegfs-*'`. Log locations can be changed using `cmsh` or Base View at any moment after installation, by editing BeeGFS roles for the corresponding configuration overlays (section [10.3.2](#1032-beegfs-objects)). Authentication is possible for the BeeGFS cluster via a shared secret file. The shared secret file can be installed during `cm-beegfs-setup` installation (section [10.2](#102-deployment-and-uninstallation-of-beegfs-with-cm-beegfs-setup)). ## 10.2 Deployment And Uninstallation Of BeeGFS With `cm-beegfs-setup` Deployment and uninstallation for BCM can be carried out with a TUI utility, `cm-beegfs-setup`. The utility is a part of a `cluster-tools` package that comes with BCM. If the BeeGFS packages are not installed, then the utility installs them. The `cm-beegfs-setup` utility can be run as root from the head node. The first screen that shows up is the main menu for operations (figure [10.1](#fig-beegfs-urwid-main-menu)): ![BeeGFS Operations Via cm-beegfs-setup Main Menu](/base-command-manager/product-manuals/_fern-files/nvidia-bcm-manuals.docs.buildwithfern.com/51c36f3f728e7597331030adae1091470bafb2af008a334f651b00ff8d408697/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/tmp/tmp-465-AR0SE6FbhiT1/latextofern/fern/assets/v10/figures/beegfs-urwid-main-menu.png) *Figure 10.1: BeeGFS Operations Via cm-beegfs-setup Main Menu* In the main menu, the administrator can choose to: * Deploy BeeGFS (section [10.2.2](#1022-deployment-of-beegfs)) * Modify an existing BCM BeeGFS configuration (the options are a subset of the deployment as described in section [10.2.2](#1022-deployment-of-beegfs)) * Uninstall BeeGFS (section [10.2.3](#1023-uninstalling-beegfs)) * Simply exit the TUI session ### 10.2.1 Preparation Of The Cluster For BeeGFS While `cm-beegfs-setup` can configure many aspects of BeeGFS installation, the optimization and disk layout aspects are expected to be managed by the system administrator. This section discusses the preparations needed before running `cm-beegfs-setup`. #### 10.2.1.1 Creation Of A New Software Image For BeeGFS An existing default image is first cloned as follows: **Example** ```bash [{basecm10} \textasciitilde]# cmsh [basecm10]% softwareimage [basecm10->softwareimage]% clone default-image beegfs-image [basecm10->softwareimage*[beegfs-image*]]% commit [basecm10->softwareimage]% task wait --last ``` The software image is now modified to store additional configuration files to optimize performance. The modifications made to this image may be unnecessary, or even harmful in the default image. #### 10.2.1.2 Preparation Of The Software Image Directories Directories are created for storage and metadata in the image. The directories are to be used later in the disk setup layout and when running `cm-beegfs-setup`. **Example** ```bash [{basecm10} \textasciitilde] cm-chroot-sw-img /cm/images/beegfs-image [root@default-image /]# mkdir -p /beegfs/{storage,metadata} ``` #### 10.2.1.3 Creation Of Disk Setup XML For Storage And Metadata VMs The XML examples that follow illustrate the connection between the storage and the metadata directories of BeeGFS. The examples are simplified for illustration. A more realistic scenario would typically involve a hardware or a software RAID configuration. The XML disk setup is saved as a file, `/root/beegfs-storage.xml`, on the head node, and may look similar to: **Example** ```bash /dev/sda 100M linux fat /boot/efi defaults,noatime,nodiratime 500M linux ext4 /boot defaults,noatime,nodiratime max linux ext4 / defaults,noatime,nodiratime /dev/sdb max linux xfs -d su=128k,sw=8 -l version=2,su=128k -isize=512 /beegfs/storage noatime,nodiratime,logbufs=8,logbsize=256k,largeio,inode64,\ swalloc,allocsize=131072k ``` While the setup for `/dev/sda` is rather generic, there are some important modifications in the `/dev/sdb` section: * The value for `filesystem` is set to `XFS`, which is as recommended at [https://doc.beegfs.io/latest/advanced_topics/storage_tuning.html](https://doc.beegfs.io/latest/advanced_topics/storage_tuning.html). * `mkfsFlags` has settings designed to help optimize for large I/O. In particular, many large files are assumed per directory. This assumption is based on how BeeGFS saves its own files rather than the users’ files: BeeGFS stores the data aggregated and split into large-sized chunks. * `mountPoint` is important later when running `cm-beegfs-setup`. This directory must be later specified as a BeeGFS storage directory. * `mountOptions`, just like the `mkfs` flags, are intended to optimize for I/O in large-sized chunks. It also assumes many large files per directory. The BeeGFS metadata XML file disk setup is saved as a file, `/root/beegfs-metadata.xml`, on the head node, and may look similar to: **Example** ```bash /dev/sda 100M linux fat /boot/efi defaults,noatime,nodiratime 500M linux ext4 /boot defaults,noatime,nodiratime max linux ext4 / defaults,noatime,nodiratime /dev/sdb max linux ext4 -i 2048 -I 512 -J size=400 -Odir_index,filetype /beegfs/metadata noatime,nodiratime ``` Similar to the storage setup XML, the `mkfs` options are based on BeeGFS recommendations, in this case as found at [https://doc.beegfs.io/latest/advanced_topics/metadata_tuning.html](https://doc.beegfs.io/latest/advanced_topics/metadata_tuning.html). The setting for `mountPoint` here too is the one used when running `cm-beegfs-setup`. #### 10.2.1.4 Creation Of Categories For Storage And Metadata Nodes Categories are necessary to tie images and disk layout together (output truncated): **Example** ```bash [{basecm10} \textasciitilde]# cmsh [basecm10]% category [basecm10->category]% clone default beegfs-metadata [basecm10->category*[beegfs-metadata*]]% set softwareimage beegfs-image [basecm10->category*[beegfs-metadata*]]% set disksetup /root/beegfs-metadata.xml [basecm10->category*[beegfs-metadata*]]% clear roles [basecm10->category*[beegfs-metadata*]]% commit [basecm10->category*[beegfs-metadata]]% clone beegfs-metadata beegfs-storage [basecm10->category*[beegfs-storage*]]% set softwareimage beegfs-image [basecm10->category*[beegfs-storage*]]% set disksetup /root/beegfs-storage.xml [basecm10->category*[beegfs-storage*]]% commit [basecm10->category*[beegfs-storage]]% clone beegfs-metadata beegfs-management [basecm10->category*[beegfs-management*]]% set softwareimage beegfs-image [basecm10->category*[beegfs-management*]]% commit ``` #### 10.2.1.5 Tuning Kernel Parameters The BeeGFS Wiki at [https://www.beegfs.io/wiki/StorageServerTuning](https://www.beegfs.io/wiki/StorageServerTuning) suggests enabling transparent huge tables. Further information about huge pages may be found at [https://www.kernel.org/doc/html/latest/admin-guide/mm/transhuge.html](https://www.kernel.org/doc/html/latest/admin-guide/mm/transhuge.html). Kernel parameters can be set for the image to permanently enable this feature: **Example** ```bash [{basecm10} \textasciitilde]# cmsh [basecm10]% softwareimage [basecm10->softwareimage]% use beegfs-image [basecm10->softwareimage[beegfs-image]]% append kernelparameters transparent_hugepages=always [basecm10->softwareimage[beegfs-image*]]% commit [basecm10->softwareimage[beegfs-image*]]% task wait --last ``` #### 10.2.1.6 Tuning Device Configuration Assuming the preceding disk setup layout, with BeeGFS using `/dev/sdb`, the following `udev` rules can be added, to improve large and highly-parallel I/O. **Example** ```bash [{basecm10} \textasciitilde]# cm-chroot-sw-img /cm/images/beegfs-image [root@default-image /]# cat < /etc/udev/rules.d/90-beegfs.rules ACTION=="add|change", KERNEL=="sdb", ATTR{queue/scheduler}="deadline" # To improve throughput for sequential reads, # increase the maximum amount of read-ahead data. # The actual amount of read-ahead is adaptive, # so using a high value here won't # harm performance for small random access. ACTION=="add|change", KERNEL=="sdb", ATTR{queue/read_ahead_kb}="4096" # Now give the IO scheduler more flexibility by increasing the size # of the request queue: ACTION=="add|change", KERNEL=="sdb", ATTR{queue/nr_requests}="4096" EOF ``` It is always a good idea to test the preceding optimization suggestions. Benchmarking the system before and after making the adjustments is therefore recommended. Benchmarking BeeGFS is discussed at [https://www.beegfs.io/wiki/Benchmark](https://www.beegfs.io/wiki/Benchmark). #### 10.2.1.7 Tuning VM Parameters Similarly, VFS caching can be optimized by setting a lower latency: ```bash [{basecm10} \textasciitilde]# cm-chroot-sw-img /cm/images/beegfs-image [root@default-image /]# cat < /etc/sysctl.d/90-beegfs.conf ##################################################################### # BeeGFS-related settings ##################################################################### # To avoid long IO stalls (latencies) for write cache flushing in # a production environment with very different workloads, you will # typically want to limit the kernel dirty (write) cache size: vm.dirty_ratio = 10 vm.dirty_background_ratio = 5 # Most likely already the default # Buffering of file system data requires frequent memory allocation. # Raising the amount of reserved kernel memory will enable faster and # more reliable memory allocation in critical situations. # Raise the corresponding value to 64 MB if you have less than 8GB # of memory, otherwise, raise it to at least 256 MB vm.min_free_kbytes = 262144 # Assigning slightly higher priority to inode caching helps to # avoid disk seeks for inode loading: vm.vfs_cache_pressure = 50 EOF ``` #### 10.2.1.8 Preparation Of Storage And Metadata Nodes For BeeGFS Setup The categories created earlier on must now be connected to the actual nodes. In the following illustrative cluster `node002` is used for metadata, `node001` is used for management, and `node003..node005` are used for storage: **Example** ```bash [basecm10]% device [basecm10->device]% use node002 [basecm10->device[node002]]% set category beegfs-metadata [basecm10->device*[node002*]]% commit [basecm10->device[node002]]% .. [basecm10->device]% use node001 [basecm10->device[node001]]% set category beegfs-management [basecm10->device*[node001*]]% commit [basecm10->device]% foreach -n node003..node005 ( set category beegfs-storage; commit ) [basecm10->device]% reboot -n node001..node005 ``` #### 10.2.1.9 Running `cm-beegfs-setup` During the `cm-beegfs-setup` run, when prompted to select categories for BeeGFS metadata, the `beegfs-metadata` category should be selected. Similarly, when prompted to select categories for BeeGFS storage, the tt beegfs-storage category should be selected. The client nodes are the nodes where users access the storage. However, the client *service* is deployed to other BeeGFS nodes as well. The BeeGFS helper is needed to help other BeeGFS services with DNS and other functionality that is not part of the BeeGFS kernel module. When prompted to select nodes for BeeGFS helper, all cluster nodes that are intended to run BeeGFS-related services should be selected. When prompted to configure the BeeGFS metadata role, the `/beegfs/metadata` directory should be set in the `Data directories` field. This directory should already be configured in the disk setup XML step which sets up the mount point for data disks for storage and metadata. Similarly, when prompted to configure the BeeGFS storage role, the `/beegfs/storage` directory should be set in the `Data directories` field. ### 10.2.2 Deployment Of BeeGFS If the `Deploy` option is chosen, then the deployment options are presented by the wizard. #### 10.2.2.1 Name, Mount Point, Authentication File, Multi Mode The first BeeGFS settings options that can be configured in the wizard are the BeeGFS cluster name, BeeGFS client mount point, BeeGFS authentication file name, and BeeGFS Multi Mode (figure [10.2](#fig-beegfs-urwid-name-mountpoint-auth-multimode)): ![BeeGFS Cluster Name, Mount Point, BeeGFS Authentication File, And Multi Mode Options](/base-command-manager/product-manuals/_fern-files/nvidia-bcm-manuals.docs.buildwithfern.com/d41c9c65dff38737bb2674113984cba42f66bd828ba4169209e631103e13bfab/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/tmp/tmp-465-AR0SE6FbhiT1/latextofern/fern/assets/v10/figures/beegfs-urwid-name-mountpoint-auth-multimode.png) *Figure 10.2: BeeGFS Cluster Name, Mount Point, BeeGFS Authentication File, And Multi Mode Options* A suggested mount point name and authentication file name are generated automatically as the cluster name file is typed in. The suggestions can be modified if needed. #### 10.2.2.2 Select Management Node The next screen (figure [10.3](#fig-beegfs-urwid-select-management-node)) then asks for one node to be selected to be the BeeGFS management server: ![Select Management Node](/base-command-manager/product-manuals/_fern-files/nvidia-bcm-manuals.docs.buildwithfern.com/96736e8b74bd72640c9c11a83be6b37bb4ba8556f8980215d4fadde5eb24c60b/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/tmp/tmp-465-AR0SE6FbhiT1/latextofern/fern/assets/v10/figures/beegfs-urwid-select-management-node.png) *Figure 10.3: Select Management Node* #### 10.2.2.3 Select Metadata Nodes The next screen then asks for node categories to be selected for the BeeGFS metadata servers. Zero or more categories can be chosen. The next screen (figure [10.4](#fig-beegfs-urwid-select-metadata-nodes)) in any case displays the remaining nodes that have not already been selected by category, and allows these remaining nodes to be individually set to be metadata servers. One or more metadata servers are needed for a BeeGFS instance. ![Select Metadata Nodes](/base-command-manager/product-manuals/_fern-files/nvidia-bcm-manuals.docs.buildwithfern.com/18de12c1c57c2e5d93dd86fe05c81fd844317c9b3b45d710b098405a3be52dd0/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/tmp/tmp-465-AR0SE6FbhiT1/latextofern/fern/assets/v10/figures/beegfs-urwid-select-metadata-nodes.png) *Figure 10.4: Select Metadata Nodes* #### 10.2.2.4 Select Storage Nodes The next screen then asks for node categories to be selected for the BeeGFS storage servers. Zero or more categories can be chosen. The next screen (figure [10.5](#fig-beegfs-urwid-select-storage-nodes)) in any case displays the remaining nodes that have not already been selected by category, and allows these remaining nodes to be individually set to be storage servers. One or more storage servers are needed for a BeeGFS instance. ![Select Storage Nodes](/base-command-manager/product-manuals/_fern-files/nvidia-bcm-manuals.docs.buildwithfern.com/0207aa9243ae3f9487b0750d4c4c637a4512bf461390907dd0f6b2e576d598e5/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/tmp/tmp-465-AR0SE6FbhiT1/latextofern/fern/assets/v10/figures/beegfs-urwid-select-storage-nodes.png) *Figure 10.5: Select Storage Nodes* #### 10.2.2.5 Select Client Nodes The next screen then asks for node categories to be selected for the BeeGFS clients. Zero or more categories can be chosen. The next screen (figure [10.6](#fig-beegfs-urwid-select-client-nodes)) in any case displays the remaining nodes that have not already been selected by category, and allows these remaining nodes to be individually set to be BeeGFS clients. One or more BeeGFS client servers are needed for a BeeGFS instance. ![Select Client Nodes](/base-command-manager/product-manuals/_fern-files/nvidia-bcm-manuals.docs.buildwithfern.com/ca7104e221bf98e7d927c49b2114b8a7267b0abb37f1e5a40b1e8030e403f4a5/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/tmp/tmp-465-AR0SE6FbhiT1/latextofern/fern/assets/v10/figures/beegfs-urwid-select-client-nodes.png) *Figure 10.6: Select Client Nodes* #### 10.2.2.6 Select BeeGFS Helper Nodes The next screen then asks for node categories to be selected for the BeeGFS helper service. Zero or more categories can be chosen. The next screen (figure [10.7](#fig-beegfs-urwid-select-helper-nodes)) in any case displays the remaining nodes that have not already been selected by category, and allows these remaining nodes to be individually set to be BeeGFS helper nodes. By default, the selection of the helper nodes, which provide logging and DNS lookup functions to the BeeGFS clients, matches the selection that has been made earlier on of the BeeGFS client servers. ![Select Helper Nodes](/base-command-manager/product-manuals/_fern-files/nvidia-bcm-manuals.docs.buildwithfern.com/f5f53007992bb21069d15e863f09ffaa7cbe38f666670c0d00e485118eb0de20/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/tmp/tmp-465-AR0SE6FbhiT1/latextofern/fern/assets/v10/figures/beegfs-urwid-select-helper-nodes.png) *Figure 10.7: Select Helper Nodes* #### 10.2.2.7 Select BeeGFS Version The next screen asks for a BeeGFS version to be selected. By default the latest stable compatible version is selected. #### 10.2.2.8 Management Server Configuration The next screen (figure [10.8](#fig-beegfs-urwid-configure-management-node)) asks for some configuration parameters for the BeeGFS management service to be set: ![Configure Management Service](/base-command-manager/product-manuals/_fern-img/979cf61d111dcef11514804f54a8e0a810e44b6404f6e7f107e6de717da39dff.webp) *Figure 10.8: Configure Management Service* Default suggestions are presented. #### 10.2.2.9 Metadata Server Configuration The next screen (figure [10.9](#fig-beegfs-urwid-configure-metadata)) asks for input on configuring the BeeGFS metadata service: ![Configure Metadata Servce](/base-command-manager/product-manuals/_fern-img/71d0404605c9fcc947644ddc16232f4355b9597f3da02b46e3408f8e6dc2fc8e.webp) *Figure 10.9: Configure Metadata Servce* Default suggestions are presented. #### 10.2.2.10 Storage Server Configuration The next screen (figure [10.10](#fig-beegfs-urwid-configure-storage)) asks for input on setting some configuration parameters for the BeeGFS storage service: ![Configure Storage Servce](/base-command-manager/product-manuals/_fern-files/nvidia-bcm-manuals.docs.buildwithfern.com/c2181b2a476dd6870b9daac9efbf7b49e2500ccd9d263614236843c8e017e35c/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/tmp/tmp-465-AR0SE6FbhiT1/latextofern/fern/assets/v10/figures/beegfs-urwid-configure-storage.png) *Figure 10.10: Configure Storage Servce* #### 10.2.2.11 Client Configuration The next screen (figure [10.11](#fig-beegfs-urwid-configure-client)) asks for input on configuring the BeeGFS clients: ![Configure Client Servce](/base-command-manager/product-manuals/_fern-files/nvidia-bcm-manuals.docs.buildwithfern.com/9a202fe70eb87f26d32d1babf55da046623d99e26333a23a88468f3e6b8ca559/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/tmp/tmp-465-AR0SE6FbhiT1/latextofern/fern/assets/v10/figures/beegfs-urwid-configure-client.png) *Figure 10.11: Configure Client Servce* #### 10.2.2.12 Helper Configuration The next screen (figure [10.12](#fig-beegfs-urwid-configure-helper)) asks for input on configuring the BeeGFS helper: ![Configure Helper Servce](/base-command-manager/product-manuals/_fern-files/nvidia-bcm-manuals.docs.buildwithfern.com/7bec96e734cf50cb34b677b93973a5f5c777b251cc89aa6992bd0316434aee35/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/tmp/tmp-465-AR0SE6FbhiT1/latextofern/fern/assets/v10/figures/beegfs-urwid-configure-helper.png) *Figure 10.12: Configure Helper Servce* #### 10.2.2.13 Deployment Summary Finally, the wizard displays a deployment summary (figure [10.13](#fig-beegfs-urwid-deploy-summary)): ![Deployment Summary](/base-command-manager/product-manuals/_fern-files/nvidia-bcm-manuals.docs.buildwithfern.com/4271563d7b0921aefac322e92be148dbd692348672b6c9c9673a366183ade0e2/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/_dot_dot_/tmp/tmp-465-AR0SE6FbhiT1/latextofern/fern/assets/v10/figures/beegfs-urwid-deploy-summary.png) *Figure 10.13: Deployment Summary* From this screen the deployment process can be started, the deployment configuration can be viewed and saved, or the wizard can simply be exited. #### 10.2.2.14 Command Line Installation If a configuration file is saved from a `cm-beegfs-setup` run, then it can be used to install a BeeGFS cluster automatically: **Example** ```bash cm-beegfs-setup -c cm-beegfs-setup.conf ``` ### 10.2.3 Uninstalling BeeGFS If the `Uninstall` option is chosen from the main menu (figure [10.1](#fig-beegfs-urwid-main-menu)), then the uninstallation process starts. An extra `Are you sure?` prompt is thrown in the way first, to help avoid a mistaken removal, and after confirmation, a further confirmation is needed to remove an authentication file that was installed with the BeeGFS instance. The uninstallation process then starts. It can take several minutes in order to ensure that services are stopped properly, so that errors and junk files are avoided. ## 10.3 Managing The Deployed BeeGFS Instance A BeeGFS cluster consists of: * One Management node * Some Metadata nodes * Some Storage nodes Information about the right number of nodes to use can be found at [https://www.beegfs.io/docs/whitepapers/Picking_the_right_Number_of_Targets_per_Server_for_BeeGFS_by_ThinkParQ.pdf](https://www.beegfs.io/docs/whitepapers/Picking_the_right_Number_of_Targets_per_Server_for_BeeGFS_by_ThinkParQ.pdf) BeeGFS clients should have client servers running on them. ### 10.3.1 Setup Associated with BeeGFS are the following BeeGFS-related packages, most of which provide services: 1. `beegfs-mgmtd` 2. `beegfs-meta` 3. `beegfs-storage` 4. `beegfs-client` 5. `beegfs-helperd` 6. `beegfs-utils` The repository from which these packages are picked up is added to the repositories list when BeeGFS is installed by BCM (section [10.2](#102-deployment-and-uninstallation-of-beegfs-with-cm-beegfs-setup)), and the packages are then installed on all of the cluster nodes. The services are assignable by the cluster administrator to cluster devices as roles. In `cmsh` these BeeGFS roles are treated as objects, and the configuration of these roles is determined by role parameters (object attributes). The roles are described in detail section [10.3.2](#1032-beegfs-objects). ### 10.3.2 BeeGFS Objects The BeeGFS configuration settings are updated by BCM in instance-specific configuration files under `/etc/beegfs/`. #### 10.3.2.1 BeeGFS Instances Properties In section [10.2.2](#1022-deployment-of-beegfs), an example session is run with the BCM utility `cm-beegfs-setup`. During the session, an example of a BeeGFS instance, `mybeegfs`, is created. BeeGFS instances such as `mybeegfs`, that have been created by BCM, can be listed, and can have their main properties viewed in the `beegfs` main mode of `cmsh`. * The `list` command lists the instances and the allocation of nodes to the BeeGFS servers: **Example** ```bash [root@basecm10 ~]# cmsh [basecm10]% beegfs [basecm10->beegfs]% list Name (key) Management nodes Client nodes Storage nodes Metadata nodes ---------- ---------------- ----------------- ---------------- ---------------- mybeegfs basecm10 node005,basecm10 node003,node004 node001,node002 ``` * The main properties of a particular BeeGFS instance, such as `mybeegfs` can be shown with: **Example** ```bash [basecm10->beegfs]% show mybeegfs Parameter Value -------------------------------- ------------------------------------------------ Name mybeegfs Revision Multi mode yes Authentication file /etc/beegfs/mybeegfs.d/auth Mountpoint /mnt/beegfs-mybeegfs Management Nodes basecm10 Client Nodes node005,basecm10 Storage Nodes node003,node004 Metadata Nodes node001,node002 Helper Nodes node005,basecm10 [basecm10->beegfs]% ``` The properties that are specific to the BeeGFS services can be viewed via the associated `cmsh` roles. The tree layout for the submodes associated with BeeGFS roles is according to the following scheme: ```bash |--roles | |--[beegfs::management] | | `--configurations | | |--logsettings | | `--connectionsettings | |--[beegfs::client] | | `--configurations | | `--connectionsettings | |--[beegfs::helper] | | `--configurations | | |--logsettings | | `--connectionsettings | |--[beegfs::metadata] | | `--configurations | | |--logsettings | | `--connectionsettings | `--[beegfs::storage] | `--configurations | |--logsettings | `--connectionsettings | ``` #### 10.3.2.2 BeeGFSManagement Role The BeeGFSManagement role is used to configure BeeGFS management services. The role parameters are described in table [10.1](#beegfsmanagement-role-parameters-table):
**Parameter** **Description** **Option to `beegfs-mgmtd`**
`Data directory` Path to data directory `storeMgmtdDirectory`
`Allow new servers` Allow new servers registration `sysAllowNewServers`
`Allow new storage` Allow new storage targets registration `sysAllowNewTargets`
`targets`
`Backlog TCP` TCP listen backlog size `connBacklogTCP`
`Interfaces file` Path to file with interfaces list `connInterfacesFile`
`Management port TCP` TCP port for management service `connMgmtdPortTCP`
`Management port UDP` UDP port for management service `connMgmtdPortUDP`
`Net filter file` Path to file with list of allowed IP `connNetFilterFile`
subnets
`Log level` Log level `logLevel`
`No log date` Do not show date along with time in log `logNoDate`
`Number of log lines` Number of lines in log file, after which `logNumLines`
it will be rotated
`Number of log` Number of old log files to keep `logNumRotatedFiles`
`rotated files`
`Log file` Path to log file, empty means logs go `logStdFile`
to the journal
`Enable Quota` Enable quota `quotaEnableEnforcement`
`Quota query GID file` Path to file with GIDs to be checked by quota `quotaQueryGIDFile`
`Quota query GID` GID range to be checked by quota `quotaQueryGIDRange`
`range`
`Quota query UID file` Path to file with UIDs to be checked by `quotaQueryUIDFile`
quota
`Quota query UID` UID range to be checked by quota `quotaQueryUIDRange`
`range`
`Quota query type` Query type for quota `quotaQueryType`
`Quota query with` Allow also system users/groups to be `quotaQueryWithSystemUsersGroups`
`system users groups` checked by quota
`Quota update` Quota update interval `quotaUpdateIntervalMin`
`interval`
`Target offline` Timeout for targets on storage server `sysTargetOfflineTimeoutSecs`
`timeout` to be considered offline when no target
status received
`Client auto remove` Time after which an unreachable node `tuneClientAutoRemoveMins`
`time` is considered dead
`Number of workers` Number of worker threads `tuneNumWorkers`
`Meta dynamic pools` Raise lower limits if difference in `tuneMetaDynamicPools`
capacity becomes too large between
targets
`Meta inodes low` Metadata inode free space pool `tuneMetaInodesLowLimit`
`limit` threshold
`Meta inodes` Metadata inode free space pool `tuneMetaInodesEmergencyLimit`
`emergency limit` threshold
`Meta space low limit` Metadata space free space pool `tuneMetaSpaceLowLimit`
threshold
`Meta space emergency` Metadata space free space pool `tuneMetaSpaceEmergencyLimit`
`limit` threshold
`Storage dynamic` Raise lower limits if difference in `tuneStorageDynamicPools`
`pools` capacity becomes too large between
targets
`Storage inodes low` Storage inode free space pool `tuneStorageInodesLowLimit`
`limit` threshold
`Storage inodes` Storage inode free space pool `tuneStorageInodesEmergencyLimit`
`emergency limit` threshold
`Storage space low` Storage space free space pool `tuneStorageSpaceLowLimit`
`limit` threshold
`Storage space` Storage space free space pool `tuneStorageSpaceEmergencyLimit`
`emergency limit` threshold
*Table 10.1: BeeGFSManagement role parameters and `beegfs-mgmtd` options* The settings can be managed in `cmsh` via the `beegfs::management` role. As is usual for roles, they can be managed by `device` mode, `category` mode, or `configurationoverlay` mode. The parameters of the role that correspond to the parameters in the preceding table can then be set. If the management role is assigned to the head node, as suggested in the example session in section [10.2.2](#1022-deployment-of-beegfs), then the role can be viewed from the `cmsh` session as follows: **Example** ```bash [root@basecm10 ~]# cmsh [basecm10]% device use basecm10 [basecm10->device[basecm10]]% roles [basecm10->device[basecm10]->roles]% use bee beegfs::client beegfs::helper beegfs::management [basecm10->device[basecm10]->roles]% use beegfs::management [basecm10->device[basecm10]->roles[BeeGFS::Management]]% show Parameter Value -------------------------------- ------------------------------------------------ Name BeeGFS::Management Revision Type BeeGFSManagementRole Add services yes Configurations <1 in submode> ``` The configuration settings can be viewed within the submodes of the role for a particular BeeGFS instance: **Example** ```bash [basecm10->device[basecm10]->roles[BeeGFS::Management]]% configurations [basecm10->device[basecm10]->roles[BeeGFS::Management]->configurations]% list BeeGFS cluster (key) -------------------- mybeegfs [basecm10->device[basecm10]->roles[BeeGFS::Management]->configurations]% use mybeegfs [basecm10->device[basecm10]->roles[BeeGFS::Management]->configurations[mybeegfs]]% show Parameter Value -------------------------------------- ------------------------------------------------ BeeGFS cluster mybeegfs Revision Data directory /var/lib/beegfs/management-mybeegfs Allow new servers yes Allow new storage targets yes Target offline timeout 3m Client auto remove time 30m Number of workers 4 Meta dynamic pools yes Meta inodes low limit 10M Meta inodes emergency limit 1M Meta space low limit 9.3GiB Meta space emergency limit 2.79GiB Storage dynamic pools yes Storage inodes low limit 9.5MiB Storage inodes emergency limit 976KiB Storage space low limit 931GiB Storage space emergency limit 18.6GiB Enable Quota no Quota query GID file Quota GIDs Quota query GID range Quota query UID file Quota UIDs Quota query UID range Quota query type system Quota query with system users groups no Quota update interval 10m Connection settings Log settings ``` The `Connection settings` submode for the management server in the BeeGFS instance has the ports and interface settings for the management service: Similarly, the `Log settings` submode for the management server in the BeeGFS instance has log configuration settings for the management service: **Example** ```bash [basecm10->...[BeeGFS::Management]->configurations[mybeegfs]]% connectionsettings [basecm10->...[BeeGFS::Management]->configurations[mybeegfs]->connectionsettings]% show Parameter Value -------------------------------- ------------------------------------------------ Revision Port TCP 8018 Port UDP 8018 Backlog TCP 128 Interfaces file Interfaces list Net filter file Use RDMA yes ``` Similarly, the `Log settings` submode for the management server in the BeeGFS instance has log configuration settings for the management service: **Example** ```bash [basecm10->...[BeeGFS::Management]->configurations[mybeegfs]->connectionsettings]% ..; logsettings; show Parameter Value -------------------------------- ------------------------------------------------ Revision Log type Syslog Level 2 No date no Number of lines 50000 Number of rotated files 5 File [basecm10->device[basecm10]->roles[BeeGFS::Management]->configurations[mybeegfs]->logsettings]% ``` #### 10.3.2.3 BeeGFSMetadata Role The BeeGFSMetadata role is used for configuring BeeGFS metadata services. The role parameters are described in Table [10.2](#beegfsmetadata-role-parameters-table):
**Parameter** **Description** **Option to `beegfs-meta`**
`Data directory` Path to the data directory `storeMetaDirectory`
`Management node` Node with a management role `sysMgmtdHost`
`Backlog TCP` TCP listen backlog `connBacklogTCP`
`Fallback expiration` Time after which a connection to a `connFallbackExpirationSecs`
`time` fallback interface expires
`Interfaces file` Path to the file with a list of interfaces `connInterfacesFile`
for communication
`Max internode number` Max number of simultaneous `connMaxInternodeNum`
connections to the same node
`Meta port TCP` TCP port for metadata service `connMetaPortTCP`
`Meta port UDP` UDP port for metadata service `connMetaPortUDP`
`Net filter file` Path to a file with a list of allowed `connNetFilterFile`
IP subnets
`Use RDMA` Use RDMA `connUseRDMA`
`RDMA type of service` RDMA type of service `connRDMATypeOfService`
`Tcp only filter file` Path to a file with a list of no DRMA `connTcpOnlyFilterFile`
IP ranges
`Log level` Log level `logLevel`
`No log date` Do not show date along with time in log `logNoDate`
`Number of log lines` Number of lines in log file, after which `logNumLines`
it will be rotated
`Number of log` Number of old log files to keep `logNumRotatedFiles`
`rotated files`
`Log file` Path to the log file, empty means logs `logStdFile`
go to the journal
`Client xattrs` Enable client-side extended attributes `storeClientXAttrs`
`Client ACLs` Enable handling and storage of client-side ACLs `storeClientACLs`
client-side ACLs
`Use extended` Store metadata as extended attributes `storeUseExtendedAttribs`
`attributes` or not
`Target attachment` File with a list of targets to be grouped `sysTargetAttachmentFile`
`file` within the same domain for
randominternode
`Target offline` Timeout for targets on a storage `sysTargetOfflineTimeoutSecs`
`timeout` server to be considered offline when no
target status is received
`Allow user set` Allow non-privileged users to modify `sysAllowUserSetPattern`
`pattern` stripe pattern settings for directories
they own
`Bind to numa zone` Zero-based NUMA zone number to `tuneBindToNumaZone`
which all threads of metadata process
should be bound
`Number of stream` The number of threads waiting for `tuneNumStreamListeners`
`listeners` incoming data events
`Number of workers` Number of worker threads `tuneNumWorkers`
`Target chooser` The algorithm to choose storage targets `tuneTargetChooser`
for file creation
`Use aggressive` Actively poll for events instead of `tuneUseAggressiveStreamPoll`
`stream poll` sleeping until an event occurs
`Use per user msg` Use per-user queues for pending `tuneUsePerUserMsgQueues`
`queues` requests
*Table 10.2: BeeGFSMetadata role parameters and `beegfs-meta` options* The settings for BeeGFS metadata can be managed in `cmsh` via the `beegfs::metadata` role. Roles are, as usual, assigned via `device`, `category`, or `configurationoverlay` mode. The parameters of the role that correspond to the parameters in the preceding table can then be set. **Example** ```bash [basecm10 ~]# cmsh [root@basecm10 ~]# cmsh [basecm10]% device use node001 [basecm10->device[node001]]% roles [basecm10->device[node001]->roles]% use beegfs::metadata [basecm10->device[node001]->roles[BeeGFS::Metadata]]% configurations [basecm10->device[node001]->roles[BeeGFS::Metadata]->configurations]% use mybeegfs [basecm10->device[node001]->roles[BeeGFS::Metadata]->configurations[mybeegfs]]% show Parameter Value -------------------------------- ------------------------------------------------ BeeGFS cluster mybeegfs Revision Data directory /var/lib/beegfs/metadata-mybeegfs Bind to numa zone Run daemonized yes Client xattrs no Client ACLs no Use extended attributes yes Allow user set pattern no Use aggressive stream poll no Use per user msg queues no Target chooser Randomized Target offline timeout 3m Target attachment file Number of stream listeners 1 Number of workers 0 Start by CMDaemon yes Connection settings Log settings [basecm10->...::Metadata]->configurations[mybeegfs]]% connectionsettings; show; .. Parameter Value -------------------------------- ------------------------------------------------ Revision Port TCP 8015 Port UDP 8015 Backlog TCP 128 Max internode number 12 Fallback expiration time 15m Interfaces file Interfaces list Net filter file Tcp only filter file Use RDMA yes RDMA type of service 0 [basecm10->...::Metadata]->configurations[mybeegfs]]% logsettings; show; .. Parameter Value -------------------------------- ------------------------------------------------ Revision Log type Syslog Level 2 No date no Number of lines 50000 Number of rotated files 5 File [basecm10->device[node001]->roles[BeeGFS::Metadata]->configurations[mybeegfs]]% ``` #### 10.3.2.4 BeeGFSStorage Role The BeeGFSStorage role is used for configuring BeeGFS storage services. The role parameters are described in Table [10.3](#beegfsstorage-role-parameters-table):
**Parameter** **Description** **Option to `beegfs-storage`**
`Data directories` Path to the data directories `storeStorageDirectory`
`Management node` Node with a management role `sysMgmtdHost`
`Authentification` Path to the shared secret `connAuthFile`
`file` authentification file
`Backlog TCP` TCP listen backlog `connBacklogTCP`
`Interfaces file` Path to the file with a list of interfaces `connInterfacesFile`
for communication
`Max internode number` Max number of simultaneous `connMaxInternodeNum`
connections to the same node
`Storage port TCP` TCP port for storage service `connStoragePortTCP`
`Storage port UDP` UDP port for storage service `connStoragePortUDP`
`Net filter file` Path to a file with a list of allowed `connNetFilterFile`
IP subnets
`Use RDMA` Use RDMA `connUseRDMA`
`RDMA type of service` RDMA type of service `connRDMATypeOfService`
`Tcp only filter file` Path to a file with a list of subnets `connTcpOnlyFilterFile`
with no RDMA
`Log level` Log level `logLevel`
`No log date` Do not show date along with time in log `logNoDate`
`Number of log lines` Number of lines in log file, after which `logNumLines`
it will be rotated
`Number of log` Number of old log files to keep `logNumRotatedFiles`
`rotated files`
`Log file` Path to the log file, empty means logs `logStdFile`
go to the journal
`Resync safety` Add an extra amount of time to the last `sysResyncSafetyThresholdMins`
`threshold` successful communication timestamp
in case of a potential cache loss
`Target offline` Timeout until targets on a storage `sysTargetOfflineTimeoutSecs`
`timeout` server are considered offline when no
target status is received
`Bind to numa zone` Zero-based NUMA zone number to `tuneBindToNumaZone`
which all threads of metadata process
should be bound
`File read ahead size` Byte range submitted to the kernel for `tuneFileReadAheadSize`
read-ahead after number of bytes was
already read from target
`File read ahead` Number of bytes read after which `tuneFileReadAheadTriggerSize`
`trigger size` the read-ahead is triggered
`File read size` Maximum amount of data server `tuneFileReadSize`
should read in a single operation
`File write size` Maximum amount of data server `tuneFileWriteSize`
should write in a single operation
`File write sync size` Number of bytes after which kernel `tuneFileWriteSyncSize`
advised to commit data
`Number of resync` Number of threads to gather filesystem `tuneNumResyncGatherSlaves`
`gather slaves` information for a buddy mirror resync
`Number of resync` Number of threads to sync filesystem `tuneNumResyncSlaves`
`slaves` information for a buddy mirror resync
`Number of stream` Number of threads waiting for `tuneNumStreamListeners`
`listeners` incoming data events
`Number of workers` Number of worker threads `tuneNumWorkers`
`Use aggressive` Actively poll for events instead of `tuneUseAggressiveStreamPoll`
`stream poll` sleeping until an event occur
`Use per target` Create a separate set of workers and `tuneUsePerTargetWorkers`
`workers` attach it for each storage target
`Use per user msg` Use per-user queues for pending `tuneUsePerUserMsgQueues`
`queues` requests
`Worker buffer size` Size of network and io buffers, allocated for each worker `tuneWorkerBufSize`
*Table 10.3: BeeGFSStorage role parameters and `beegfs-storage` options* The `beegfs-storage` settings are updated by BCM in `/etc/beegfs/beegfs-storage.conf`. The settings can be managed in `cmsh` via the `beegfs::storage` role. Roles are, as usual, assigned via `device`, `category`, or `configurationoverlay` mode. The parameters of the role that correspond to the parameters in the preceding table can then be set. **Example** ```bash [root@basecm10 ~]# cmsh [basecm10]% device use node003 [basecm10->device[node003]]% roles [basecm10->device[node003]->roles]% use beegfs::storage [basecm10->device[node003]->roles[BeeGFS::Storage]]% show Parameter Value -------------------------------- ------------------------------------------------ Name BeeGFS::Storage Revision Type BeeGFSStorageRole Add services yes Configurations <1 in submode> [basecm10->device[node003]->roles[BeeGFS::Storage]]% configurations [basecm10->device[node003]->roles[BeeGFS::Storage]->configurations]% use mybeegfs [basecm10->device[node003]->roles[BeeGFS::Storage]->configurations[mybeegfs]]% show Parameter Value -------------------------------- ------------------------------------------------ BeeGFS cluster mybeegfs Revision Data directories /var/lib/beegfs/storage-mybeegfs Target offline timeout 3m Use aggressive stream poll no Use per target workers yes Use per user msg queues no Run daemonized yes Bind to numa zone Resync safety threshold 10m File read ahead size 0B File read ahead trigger size 3.8MiB File read size 122MiB File write size 122MiB File write sync size 122MiB Worker buffer size 3.8MiB Number of resync gather slaves 6 Number of resync slaves 12 Number of stream listeners 1 Number of workers 12 Connection settings Log settings Start by CMDaemon yes [basecm10->...->roles[BeeGFS::Storage]->configurations[mybeegfs]]% connectionsettings; show; .. Parameter Value -------------------------------- ------------------------------------------------ Revision Port TCP 8013 Port UDP 8013 Backlog TCP 128 Max internode number 12 Interfaces file Interfaces list Net filter file Tcp only filter file Use RDMA yes RDMA type of service 0 [basecm10->...->roles[BeeGFS::Storage]->configurations[mybeegfs]]% logsettings; show; .. Parameter Value -------------------------------- ------------------------------------------------ Revision Log type Syslog Level 2 No date no Number of lines 50000 Number of rotated files 5 File [basecm10->device[node003]->roles[BeeGFS::Storage]->configurations[mybeegfs]]% ``` #### 10.3.2.5 BeeGFSClient Role The BeeGFSClient role is used for configuring BeeGFS client services. The role parameters are described in table [10.4](#beegfsclient-role-parameters-table).
**Parameter** **Description** **Option to `beegfs-client`**
`Management node` Node with a management role `sysMgmtdHost`
`Client port UDP` UDP port for client service `connClientPortUDP`
`Communication retry time` Time for retries in case of a network failure `connCommRetrySecs`
`Fallback expiration time` Time after which a connection to a fallback interface expires `connFallbackExpirationSecs`
`Interfaces file` Path to file with list of communication interfaces `connInterfacesFile`
`Max internode number` Maximum simultaneous connections to the same node `connMaxInternodeNum`
`Net filter file` Path to a file with a list of allowed IP subnets `connNetFilterFile`
`Use RDMA` Use RDMA `connUseRDMA`
`RDMA buffers number` Number of RDMA buffers `connRDMABufNum`
`RDMA buffer size` Maximum size of a buffer that will be sent over the network `connRDMABufSize`
`RDMA type of service` RDMA type of service `connRDMATypeOfService`
`Tcp only filter file` Path to a file with a list of no RDMA IP ranges `connTcpOnlyFilterFile`
`Log level` Log level `logLevel`
`Enable Quota` Enable quota `quotaEnabled`
`Create hardlinks as symlinks` Create a symlink when an application tries to create a hardlink `sysCreateHardlinksAsSymlinks`
`Mount sanity check` Time in ms that server has to respond after mount sanity check `sysMountSanityCheckMS`
`Session check on close` Check for valid sessions on storage server when a file is closed `sysSessionCheckOnClose`
`Sync on close` Sync file content on close `sysSyncOnClose`
`Target offline timeout` Timeout until all storage targets are considered offline when no target state updates can be fetched from management server `sysTargetOfflineTimeoutSecs`
`Update target states time` Interval for storage targets states check `sysUpdateTargetStatesSecs`
`Enable xattrs` Enable xattrs `sysXAttrsEnabled`
`Enable ACLs` Enable ACLs `sysACLsEnabled`
`File cache type` File read/write cache type `tuneFileCacheType`
`Preferred meta file` Path to a file with preferred metadata servers `tunePreferredMetaFile`
`Preferred storage file` Path to a file with preferred storage targets `tunePreferredStorageFile`
`Remote fsync` Should fsync be executed on server to flush cached file `tuneRemoteFSync`
`Use global append locks` Should files, opened in append mode, be protected by locks on local machine (false) or on servers (true) `tuneUseGlobalAppendLocks`
`Use global file locks` Should advisory locks be checked on local machine (false) or on servers (true) `tuneUseGlobalFileLocks`
*Table 10.4: BeeGFSClient role parameters and `beegfs-client` options* The settings can be managed in `cmsh` via the `beegfs::client` role: **Example** ```bash [root@basecm10 ~]# cmsh [basecm10]% device use node005; roles; use beegfs::client; configurations; use mybeegfs [basecm10->device[node005]->roles[BeeGFS::Client]->configurations[mybeegfs]]% show Parameter Value -------------------------------- ------------------------------------------------ BeeGFS cluster mybeegfs Revision Enable Quota no Create hardlinks as symlinks no Mount sanity check 11s Session check on close no Sync on close no Target offline timeout 15m Update target states time 1m Enable xattrs no Enable ACLs no File cache type buffered Preferred meta file Preferred storage file Preferred metadata server IDs Preferred storage server IDs Remote fsync yes Use global append locks no Use global file locks no Connection settings Log type Syslog Level 2 Add clientid no Helper IP 0.0.0.0 [basecm10->device[node005]->roles[BeeGFS::Client]->configurations[mybeegfs]]% connectionsettings; show; .. Parameter Value -------------------------------------------------------- ------------------------------------------------ Revision Port UDP 8014 Max internode number 12 Communication retry time 10m Fallback expiration time 15m Interfaces file Interfaces list The maximum number of simultaneous connection attempts 0 Net filter file Tcp only filter file Use RDMA yes RDMA buffers number 70 RDMA buffer size 8192 RDMA type of service 0 [basecm10->device[node005]->roles[BeeGFS::Client]->configurations[mybeegfs]]% ``` #### 10.3.2.6 BeeGFS Helper Role The BeeGFS helper role is used for configuring BeeGFS helper services. The role parameters are described in table [10.5](#beegfshelperd-role-parameters-table):
**Parameter** **Description** **Option to `beegfs-helperd`**
`Helper workers number` Number of worker threads for helper service `tuneNumWorkers`
`Helper port TCP` TCP port for helper service `connHelperdPortTCP`
`No log date` Do not show date along with time in log `logNoDate`
`Number of log lines` Number of lines in log file, after which it is rotated `logNumLines`
`Number of log rotated files` Number of old log files to keep `logNumRotatedFiles`
`Log file` Path to the log file, empty means logs go to the journal `logStdFile`
*Table 10.5: BeeGFS Helper role parameters and `beegfs-helperd` options* The role settings can be accessed as follows in `cmsh`: **Example** ```bash [root@basecm10 ~]# cmsh [basecm10]% device use node005; roles; use beegfs::helper; configurations; use mybeegfs [basecm10->device[node005]->roles[BeeGFS::Helper]->configurations[mybeegfs]]% show Parameter Value -------------------------------- ------------------------------------------------ BeeGFS cluster mybeegfs Revision Run daemonized yes Helper workers number 2 Connection settings Log settings [basecm10->device[node005]->roles[BeeGFS::Helper]->configurations[mybeegfs]]% connectionsettings; show; .. Parameter Value -------------------------------- ------------------------------------------------ Revision Port TCP 8016 [basecm10->device[node005]->roles[BeeGFS::Helper]->configurations[mybeegfs]]% logsettings; show; .. Parameter Value -------------------------------- ------------------------------------------------ Revision Log type Syslog Level 2 No date no Number of lines 50000 Number of rotated files 5 File [basecm10->device[node005]->roles[BeeGFS::Helper]->configurations[mybeegfs]]% ``` ### 10.3.3 Usage #### 10.3.3.1 Accessing The Filesystem After the installation process finishes, the BeeGFS filesystem can be accessed on the client nodes or on the head node. The default mount point for a BeeGFS instance with the name \<*name*> is `/mnt/beegfs-` \<*name*>, as set early on during BeeGFS installation, when using `cm-beegfs-setup` (figure [10.2](#fig-beegfs-urwid-name-mountpoint-auth-multimode)).