Resources

Environment Variables

View as Markdown

Overview

NIXL uses environment variables for runtime configuration. This page documents all recognized variables, grouped by subsystem. Environment variables are read at agent initialization time and cannot be changed after an agent is created.

Variables are organized into the following categories:

  • Core — Logging and plug-in discovery that apply to all NIXL deployments
  • etcd — Distributed metadata exchange configuration
  • Telemetry — Performance monitoring, event collection, and export
  • Backend-Specific — Transport-level settings for individual backends (UCX, Libfabric, Mooncake, S3, Azure Blob, GDS)

Boolean environment variables follow the convention: set to y/yes/on/true/enable/1 (case-insensitive) to enable, or n/no/off/false/disable/0 to disable. Presence-check variables are activated by setting them to any value.

Core Variables

These variables control fundamental NIXL behavior across all backends.

VariableTypeDefaultDescription
NIXL_LOG_LEVELStringWARNControls log verbosity. Values: ERROR, WARN, INFO, DEBUG, TRACE.
NIXL_PLUGIN_DIRString (path)System defaultCustom directory to search for backend plug-in shared libraries.
NIXL_DISABLE_CUDA_ADDR_WABoolean (presence)Not set (workaround enabled)Disables CUDA address workaround in the Libfabric backend. Set this variable to any value to disable the workaround.

etcd Variables

These variables configure NIXL’s distributed metadata exchange via etcd.

VariableTypeDefaultDescription
NIXL_ETCD_ENDPOINTSString (URL)None (etcd disabled)etcd server endpoint(s). Setting this activates etcd metadata exchange mode. Example: http://localhost:2379.
NIXL_ETCD_NAMESPACEString (path)/nixl/agents/Key prefix namespace for agent metadata stored in etcd.

For detailed etcd setup and usage, see the Metadata Exchange with etcd guide.

Telemetry Variables

These variables control NIXL’s built-in telemetry collection and export system.

VariableTypeDefaultDescription
NIXL_TELEMETRY_ENABLEBooleanfalseEnable telemetry collection. Accepts y/yes/on/true/enable/1 (case-insensitive) to enable.
NIXL_TELEMETRY_ENABLED_METRICSStringNoneOptional comma-separated fnmatch allowlist of event names. Unset or empty exports all events.
NIXL_TELEMETRY_BUFFER_SIZEInteger4096Number of events in the cyclic telemetry buffer. Must be a power of 2.
NIXL_TELEMETRY_RUN_INTERVALInteger (ms)100Flush interval in milliseconds for the telemetry exporter.
NIXL_TELEMETRY_EXPORTERStringNoneName of the telemetry exporter plug-in to load (e.g., prometheus).
NIXL_TELEMETRY_DIRString (path)NoneDirectory for shared memory telemetry files used by the cyclic buffer exporter. If telemetry is enabled but this is not set, no telemetry file is generated.
NIXL_TELEMETRY_PROMETHEUS_PORTInteger9090HTTP listen port for the Prometheus telemetry exporter.
NIXL_TELEMETRY_PROMETHEUS_LOCALBooleanfalseWhen true, binds the Prometheus exporter to localhost only (127.0.0.1) instead of all interfaces (0.0.0.0).

For the full telemetry architecture and usage examples, see the Telemetry Guide.

Backend-Specific Variables

UCX

These variables apply when using the UCX transport backend.

VariableTypeDefaultDescription
NIXL_UCX_WARNING_TIMEOUTInteger (ms)5000Timeout in milliseconds for UCX memory registration warning. If a memory registration takes longer than this threshold, a warning is logged.

Libfabric

These variables apply when using the Libfabric transport backend.

VariableTypeDefaultDescription
NIXL_LIBFABRIC_MAX_BW_PER_DRAM_SEGInteger (Gbps)Auto-computed from PCIe topologyBandwidth limit in Gbps for NUMA-aware rail selection on DRAM segments. Override this when the auto-detected PCIe topology does not accurately reflect your network configuration.

Additional NIXL_LIBFABRIC_* variables may exist for provider-specific configuration. The Libfabric backend constructs environment variable names dynamically using the NIXL_LIBFABRIC_ prefix followed by an uppercase key name. Consult the Libfabric backend README for provider-specific options.

Mooncake

These variables apply when using the Mooncake transport backend.

VariableTypeDefaultDescription
NIXL_MOONCAKE_IP_ADDRString (IP)First non-loopback interface IPOverride IP address for the Mooncake transport engine. Set this when auto-detection selects the wrong network interface.

S3 / Object Storage

These variables apply when using the S3/Object Storage backend.

VariableTypeDefaultDescription
AWS_DEFAULT_BUCKETStringNoneDefault S3 bucket name. Used as fallback when bucket is not specified in backend parameters.
AWS_ENDPOINT_OVERRIDEString (URL)NoneCustom S3 endpoint URL. Used as fallback when endpoint_override is not specified in backend parameters. Set this for S3-compatible storage services (e.g., MinIO, Ceph).

The OBJ backend also recognizes standard AWS SDK environment variables for authentication:

VariableTypeDescription
AWS_ACCESS_KEY_IDStringAWS access key for authentication.
AWS_SECRET_ACCESS_KEYStringAWS secret key for authentication.
AWS_SESSION_TOKENStringAWS session token for temporary credentials.
AWS_REGIONStringAWS region for the S3 endpoint.

Azure Blob Storage

These variables apply when using the Azure Blob Storage backend.

VariableTypeDefaultDescription
AZURE_STORAGE_ACCOUNT_URLString (URL)NoneAzure Storage account URL (e.g., https://<account>.blob.core.windows.net).
AZURE_STORAGE_CONTAINER_NAMEStringNone (required)Azure Storage container name for blob operations.
AZURE_STORAGE_CONNECTION_STRINGStringNoneAzure Storage connection string. Used for local development with Azurite or when connection string auth is preferred.
AZURE_CA_BUNDLEString (path)NoneCustom CA certificate bundle path for Azure HTTPS connections.

Backend parameters passed to createBackend() take precedence over these environment variables. Supported override keys are account_url, container_name, connection_string, and ca_bundle.

GDS (GPUDirect Storage)

These variables apply when using the GDS backend.

VariableTypeDefaultDescription
CUFILE_ENV_PATH_JSONString (path)Default cuFile configPath to the cuFile configuration JSON file. This is a cuFile SDK variable, not NIXL-specific. Override to customize GPUDirect Storage behavior.

Quick Reference

Set environment variables before launching your NIXL application:

# Enable debug logging and telemetry with Prometheus export
export NIXL_LOG_LEVEL=DEBUG
export NIXL_TELEMETRY_ENABLE=true
export NIXL_TELEMETRY_EXPORTER=prometheus
export NIXL_TELEMETRY_PROMETHEUS_PORT=9090
# Configure etcd for distributed metadata exchange
export NIXL_ETCD_ENDPOINTS=http://localhost:2379
# Run your NIXL application
./my_nixl_app