AIStoreAIStore (AIS) supports three logical HTTP networks, each with a distinct role, lifecycle, and performance profile:
These are logical networks. They may map to:
NOTE: It is strongly recommended to (a) configure three logical networks, and (b) provision each one with separate, dedicated bandwidth (for example, to avoid head-of-line blocking that can impact latency-sensitive control-plane traffic).
Separately, AIS runs an additional distinct intra-cluster data plane:
(A) HTTP intra-data network (DataNet)
(B) Transport/bundle streams (data movers)
See also: transport package: README
Rest of this document is structured as follows:
Table of Contents
AIS separates traffic across three logical networks: a user-facing public network, and two intra-cluster networks for control and data. Each can be placed on a dedicated NIC or VLAN. The most impactful use is isolating the intra-cluster data network - which carries global rebalance, get-batch, and other bulk data movement - from the public network serving foreground I/O.
In fact, on loaded clusters there’s also a sufficiently high motivation to isolate latency-sensitive (small-bandwidth) control plane from all of the above. Logical network separation is a designed-in capability that pays off most at high drive counts and high utilization, and is worth configuring explicitly rather than relying on shared bandwidth of 100GbE links.
Having said all of the above - the presence of three logical networks does not necessarily require three physical networks:
If multiple logical networks resolve to the same address and port, AIS aliases them internally and operates correctly over a single interface.
Example: single-network deployment
In this case:
Compare with (preferred) configuration with 3 logical networks:
NOTE: It is strongly recommended to (a) configure three logical networks, and (b) provision each one with separate, dedicated bandwidth.
The public network is user-facing and client-oriented:
ais), SDKs, notebooks, browsersCharacteristics:
The control network carries cluster coordination traffic:
Characteristics:
The data network carries HTTP-based object movement between targets:
Important distinction:
High-volume streaming operations such as rebalance, resilver, and get-batch do not use net/http and are not part of the intra-cluster data HTTP network. They use AIStore’s transport/bundle subsystem (and fasthttp), which is a separate data-mover layer.
Thus, “data network” here means HTTP object paths, not transport streams.
Public networking supports multi-homing to utilize multiple NICs without LACP:
Effect:
Multi-homing currently applies to the public network only.
AIStore supports IPv6 for all three logical networks and the transport streaming layer.
IPv6 is a cluster-wide setting:
To take effect, the change requires a cluster restart. On restart, each node discovers local IPv6 addresses and binds all listeners (public, control, data) using IPv6 addresses.
Operational note: As with any cluster-wide configuration change that rebuilds cluster metadata (e.g., cluster map (Smap)), it is good practice to periodically back up cluster-level metadata (BMD, Smap, configuration). This is not specific to IPv6.
The use_ipv6 flag expresses preference, not a hard requirement. At startup, each node:
fe80::/10)This means use_ipv6: true on a host with no routable IPv6 will not prevent the node from starting - it will run on IPv4.
An existing cluster can switch address families without changing cluster identity or affecting stored data in any shape or form:
On restart, the primary builds a fresh Smap. Each node discovers its new addresses (e.g., [::1] instead of 127.0.0.1) and joins with current net-info. Node IDs are persistent, so the cluster retains its identity - same UUID, same BMD, same data.
The reverse switch (true => false) works the same way.
Precondition: the primary_url in each node’s local config must be a hostname (e.g., localhost, an FQDN), not a literal IP address. This allows DNS resolution to return the appropriate address for the current IP family. With a hardcoded IP like 127.0.0.1, nodes cannot find the primary after switching to IPv6.
IPv6 addresses in URLs use bracket notation per RFC 2732:
This applies to all AIS-internal URLs (Smap entries, intra-cluster communication) and to the CLI endpoint:
AIS sets low-latency traffic class on control-plane sockets (IP_TOS for IPv4, IPV6_TCLASS for IPv6). The correct socket option is selected automatically based on the actual socket family - not the configuration flag. This handles edge cases where a dual-stack dialer creates an IPv4 socket despite an IPv6 preference.
No operator action is required.
In Kubernetes deployments, the address family is determined by the CNI and cluster networking configuration - not by AIStore’s use_ipv6 flag. AIStore nodes accept whatever addresses the infrastructure assigns.
For K8s-specific network configuration (ports, CNI requirements, bandwidth tuning), see ais-k8s network configuration.
The configuration option:
expresses intent, not a guarantee.
At startup, each node performs runtime network resolution:
This decision is made once, early in initialization, and applies uniformly to:
IP_TOS vs IPV6_TCLASS)The original configuration is not mutated.
Instead, the resolved choice becomes part of the node’s effective runtime networking state.
Once a node determines that IPv6 is not usable, it must not attempt IPv6 connections elsewhere, even if the configuration requested it.
This avoids mixed-family failures where a node listens on IPv4 but dials IPv6.
AIS initialization separates networking concerns into two phases:
resolve local IPs
decide effective IP family
construct HTTP servers and muxers:
build Snode network descriptors
No sockets are opened yet.
This split guarantees:
net.use_ipv6; switching between IPv4 and IPv6 requires only a config change and restartFurther, it is generally recommended: