Configuring Partition Keys (PKEYs)

NVIDIA Skyway InfiniBand-to-Ethernet Gateway User Manual
  1. Configure partition keys on the Skyway appliance.

    Copy
    Copied!
                

    #### pkey_id = [1, 7ffe] (hex)

  2. Configuring partition keys on the subnet manager.

    Copy
    Copied!
                

    ib switch (config) # ib partition <partition name> pkey 0x<pkey_id> ib switch (config) # ib partition <partition name> ipoib ib switch (config) # ib partition <partition name> member ALL type full

    Example:

    Copy
    Copied!
                

    ib switch (config) # ib partition pkey_0x1 pkey 0x1 ib switch (config) # ib partition pkey_0x1 ipoib ib switch (config) # ib partition pkey_0x1 member ALL type full

  3. Configure PKEY on the InfiniBand host.

    Copy
    Copied!
                

    #### pkey_full = pkey_id (hex) + 8000 (hex) echo "0x<pkey_full>" > /sys/class/net/ib<interface_number>/create_child ifconfig ib<interface_number>.< pkey_full> up ifconfig ib<interface_number>.< pkey_full> <Ib host pkey port ip> netmask <netmask> ip route add <eth subnet> via <Gw pkey port virtual ip> ip route add <gw eth subnet> via <Gw pkey port virtual ip>

    Example:

    Copy
    Copied!
                

    echo "0x8001" > /sys/class/net/ib1/create_child ifconfig ib1.8001 up ifconfig ib1.8001 111.222.62.11 netmask 255.255.255.0 ip route add 2.2.2.0/24 via 111.222.62.1 ip route add 192.168.1.0/24 via 111.222.62.1

    Warning

    Note that this example shows how to configure PKEY that has an ID of 0x1 (0x8001 in hex === 8000+pkeyID). When configuring other PKEYs on the InfiniBand host, make sure to add 8000 to the PKEY ID.

  4. Configure the Skyway appliance.

    Copy
    Copied!
                

    gateway (config) # interface ib port-channel 1 pkey 0x<pkey_id> gateway (config interface ib port-channel 1 pkey 0x<pkey_id>) # exit gateway (config) # interface ib port-channel 1 pkey 0x<pkey_id> ip address <Gw pkey port ip> <netmask> gateway (config) # interface ib port-channel 1 pkey 0x<pkey_id> virtual ip address <Gw pkey port virtual ip> <netmask> gateway (config) # configuration write

    Example:

    Copy
    Copied!
                

    gateway (config) # interface ib port-channel 1 pkey 0x1 gateway (config interface ib port-channel 1 pkey 0x1) # exit gateway (config) # interface ib port-channel 1 pkey 0x1 ip address 111.222.62.2 255.255.255.0 gateway (config) # interface ib port-channel 1 pkey 0x1 virtual ip address 111.222.62.1 255.255.255.0 gateway (config) # configuration write

  5. Add routes on the Ethernet host.

    Copy
    Copied!
                

    ip route add <pkey subnet>/<netmask> via <eth router port ip>

    Example:

    Copy
    Copied!
                

    ip route add 111.222.62.0/24 via <eth router port ip>

  6. Add routes on the Ethernet router.

    Copy
    Copied!
                

    ip route <pkey subnet>/<netmask> <gw eth pc ip>

    Example:

    Copy
    Copied!
                

    ip route 111.222.62.0 /24 <gw eth pc>

© Copyright 2024, NVIDIA. Last updated on Apr 17, 2024.