Deployment Scenarios

NVIDIA MLNX-GW User Manual for NVIDIA Skyway Appliance v8.2.2200

The gateway can be used in various deployment topologies, each with their particular strength.

Option #1:

image2020-10-25_2-7-39-version-1-modificationdate-1709139098767-api-v2.png

The Skyway appliances are connected to the Spine switches of the Fat Tree. This topology requires fewer hops to reach the Skyway, though, on the other hand, the Spine ports are occupied instead of keeping them available for future expansion of the cluster.

Option #2:

image2020-10-25_2-10-16-version-1-modificationdate-1709139099423-api-v2.png

The Skyway appliances are connected to the Leaf switches. This topology makes Spine ports available for future expansion, though, on the other hand, the hop count from cluster nodes to the Skyway are not even—there may be nodes with a fewer hop count than others.

Option #1:

image2020-10-25_2-10-52-version-1-modificationdate-1709139099696-api-v2.png

The Skyway appliances are connected to a dedicated cell for services (e.g., storage and login services). This topology provides fairness and symmetry among all nodes on all other cells, though, on the other hand, it requires having an additional cell (or “services island”).

Option #2:

image2020-10-25_2-12-27-version-1-modificationdate-1709139099945-api-v2.png

The Skyway appliances are connected directly to leaf switches on the compute islands. An additional cell (or “service island”) is not required.

image2022-3-22_15-32-52-version-1-modificationdate-1709139100184-api-v2.png

The clearest advantage of LAG/MLAG is that it is a simple and standard topology. While the topology provides a good load distribution and good resiliency, it is limited in scale. For more information on configuring MLAG, see the following community post.

Warning

Please consider the following while implementing LAG/MLAG using Skyway appliances:

  • There is no Inter Peer Link (IPL) across Skyway appliances.

  • Skyway-based MLAG cannot be VLAN enabled. Ports are always access.

  • All Skyway appliances in an MLAG domain will share the same Virtual IP.

  • Any encapsulation/tunneling must start and terminate at the Ethernet switch

Multiple IP Subnets

image2022-4-3_10-46-2-version-1-modificationdate-1709139100460-api-v2.png

Multiple IP subnets can be configured over the InfiniBand network. In such cases, every IPoIB subnet will be served by dedicated Skyway appliances that are configured in High Availability (HA) Domain.

The specific IP configuration (e.g., Default Gateway, Next Hop Router, and so forth) will have to be configured separately per HA domain of the Skyway appliances.

Warning
  • InfiniBand network is assumed to be a single InfiniBand subnet encompassing several IPoIB subnets.


© Copyright 2023, NVIDIA. Last updated on Feb 29, 2024.