|Configuration Option (default)||cuFile IO Behavior|
|"allow_compat_mode": true||If true, falls back to using compatibility mode when the library detects that the buffer file descriptor opened cannot use GPUDirect Storage.|
|"GPUDirect Storage_rdma_write_support": true||If false, forces compatibility mode to be used for writes even when the underlying file system is capable of performing GPUDirect Storage writes.|
|"posix_unaligned_writes" : false||
If true, forces compatibility mode to be used for writes where the file offset and/or IO size is not aligned to Page Boundary (4KB).
|“lustre:posix_GPUDirect Storage_min_kb" : 0||
If greater than 0, compatibility mode is used for IO sizes between [1 - posix_GPUDirect Storage_min_kb] specified in kB.Note: This option will force posix mode even if “allow_compat_mode” is set to “false”.
|"weka:rdma_write_support" : false||
If this option is false, WekaFs will use compatibility mode for all writes to the filesystem.Note: if the option is set to “false”, cuFile library will use the posix path even if the allow_compat_mode option is true or false.
"rdma_dynamic_routing_order": [ " "SYS_MEM" ]
|If rdma_dynamic_routing is set to true and rdma_dynamic_routing_order is set to [“SYS_MEM”] , then all IO for DFS will use compatibility mode.|
In addition to the above configuration options, compatibility mode will be used as a fallback option for following use cases.
|Use Case||cuFile IO Behavior|
|IBM Spectrum Scale File System writes.||Note: All IBM Spectrum Scale in GPUDirect Storage 1.0.0.x release use posix path for writes even if the allow_compat_mode option is set to false.|
|No BAR1 memory in GPU.||Use compatibility mode.|
For wekaFS or IBM Spectrum Scale mounts: If there are no rdma_dev_addr_list specified, or failure to register MR with ib device.
|Use compatibility mode.|
|Bounce buffers cannot be allocated in GPU memory.||Use compatibility mode.|
|For WekaFS and IBM Spectrum Scale: If the kernel returns -ENOTSUP for GPUDirect Storage read/write.||Retry the IO operation internally using compatibility mode.|
|The nvidia_fs.ko driver is not loaded.||All IO operations will use compatibility mode.|