Aerial RAN CoLab Over-the-Air - ARC-OTA 1.3
v1.3

Part 2. Configure the Network Hardware

Note

Refer to the NVIDIA SDK Manager resources for setup and installation of ARC-OTA.

Tip

Refer to the tutorials for help with these installation steps.

The network hardware is configured in the following steps.

  1. Setup the GrandMaster

  2. Setup the switch

  3. Setup PTP

  4. Setup Foxconn O-RU

Step 1.

Follow the Qulsar User Guide to set up the MGMT connection.

image11.png


Step 2.

Set the operating mode to GNSS Only, and other fields as such, then run Start Engine.

image2.png


Step 3.

Enable the ports on the GrandMaster with the 8275.1 Profile configurations.

image3.png


Step 4.

Configure the clock configs as follows:

image4.png


Step 5.

Ensure the GPS configuration values are unchanged from the QG2 default settings.

image5.png


Step 6.

Verify that the GPS Signal reaches the GrandMaster.

image6.png


Chapter 2.2.1 Dell Switch

The following example uses these VLAN 2 settings:

  • RUs are on ports 1 and 7

  • GrandMaster is on port 5

  • CN is on ports 11 and 12

  • gNB ports are connected to ports 49 and 51

  1. Set up MGMT access to the switch (in this case 172.168.20.67):

    Copy
    Copied!
                

    OS10# configure terminal OS10(config)# interface mgmt1/1/1 no shutdown no ip address dhcp ip address 172.16.204.67/22 exit

  2. Use SSH to access admin@172.168.204.67.

  3. Set the speed to 10G for port groups 1 and 2.

    Copy
    Copied!
                

    OS10(config)# port-group 1/1/1 mode Eth 10g-4x exit port-group 1/1/2 mode Eth 10g-4x exit

  4. Enable PTP on the switch.

    Copy
    Copied!
                

    OS10# configure terminal OS10(config)# ptp clock boundary profile g8275.1 ptp domain 24 ptp system-time enable !

  5. Configure the GrandMaster port.

    Copy
    Copied!
                

    OS10(config)# interface ethernet 1/1/5:1 no shutdown no switchport ip address 169.254.2.1/24 flowcontrol receive off ptp delay-req-min-interval -4 ptp enable ptp sync-interval -4 ptp transport layer2 exit

    After some time, the following values will print:

    Copy
    Copied!
                

    <165>1 2023-05-09T07:49:22.625584+00:00 OS10 dn_alm 1021 - - Node.1-Unit.1:PRI [event], Dell EMC (OS10) %PTP_SYSTEM_TIME_NOT_SET: System time is not set. System time will be set when the clock is. <165>1 2023-05-09T07:51:22.312557+00:00 OS10 dn_alm 1021 - - Node.1-Unit.1:PRI [event], Dell EMC (OS10) %PTP_CLOCK_PHASE_LOCKED: Clock servo is phase locked. <165>1 2023-05-09T07:51:22.313081+00:00 OS10 dn_alm 1021 - - Node.1-Unit.1:PRI [event], Dell EMC (OS10) %PTP_SYSTEM_TIME_UPDATE_STARTED: System time update service is started. Update interval: 60 minutes. <165>1 2023-05-09T07:51:59.334346+00:00 OS10 dn_alm 1021 - - Node.1-Unit.1:PRI [event], Dell EMC (OS10) %ALM_CLOCK_UPDATE: Clock changed MESSAGE=apt-daily.timer: Adding 6h 36min 18.719270s random time. <165>1 2023-05-09T07:57:27.254181+00:00 OS10 dn_alm 1021 - - Node.1-Unit.1:PRI [event], Dell EMC (OS10) %ALM_CLOCK_UPDATE: Clock changed MESSAGE=apt-daily.timer: Adding 4h 31mi

  6. Configure the Fronthaul Network Configuration by creating a VLAN.

    Note

    If you choose to use a different VLAN, you must modify the Aerial YAML file and O-RU configuration. C- and U-planes use the same VLAN.

    Create “VLAN 2”.

    Copy
    Copied!
                

    OS10(config)# interface vlan 2 OS10(conf-if-vl-2)# <165>1 2023-03-16T16:51:36.458730+00:00 OS10 dn_alm 813 - - Node.1-Unit.1:PRI [event], Dell EMC (OS10) %IFM_ASTATE_UP: Interface admin state up :vlan2 OS10(conf-if-vl-2)# show configuration ! interface vlan2 no shutdown OS10(conf-if-vl-2)# exit

  7. Configure the RU, gNB, CN, and MEC ports.

    Interfaces that are configured to be slower than their maximum speed have a :1 appended to their name. This applies to ports in port groups 1 and 2.

    Copy
    Copied!
                

    no shutdown switchport mode trunk switchport trunk allowed vlan 2 mtu 8192 flowcontrol receive off ptp enable ptp transport layer2 ptp role timeTransmitter exit

  8. Check the PTP status.

    Copy
    Copied!
                

    OS10# show ptp | no-more PTP Clock : Boundary Clock Identity : b0:4f:13:ff:ff:46:63:5f GrandMaster Clock Identity : fc:af:6a:ff:fe:02:bc:8d Clock Mode : One-step Clock Quality Class : 135 Accuracy : <=100ns Offset Log Scaled Variance : 65535 Domain : 24 Priority1 : 128 Priority2 : 128 Profile : G8275-1(Local-Priority:-128) Steps Removed : 1 Mean Path Delay(ns) : 637 Offset From Master(ns) : 1 Number of Ports : 8 ---------------------------------------------------------------------------- Interface State Port Identity ---------------------------------------------------------------------------- Ethernet1/1/1:1 Master b0:4f:13:ff:ff:46:63:5f:1 Ethernet1/1/3:1 Master b0:4f:13:ff:ff:46:63:5f:3 Ethernet1/1/5:1 Slave b0:4f:13:ff:ff:46:63:5f:5 Ethernet1/1/7:1 Master b0:4f:13:ff:ff:46:63:5f:8 Ethernet1/1/11 Master b0:4f:13:ff:ff:46:63:5f:4 Ethernet1/1/49 Master b0:4f:13:ff:ff:46:63:5f:9 Ethernet1/1/51 Master b0:4f:13:ff:ff:46:63:5f:10 Ethernet1/1/54 Master b0:4f:13:ff:ff:46:63:5f:2 ---------------------------------------------------------------------------- Number of slave ports :1 Number of master ports :7

  9. Save the switch configuration:

    Copy
    Copied!
                

    copy running-configuration startup-configuration

Chapter 2.2.2 Fibrolan Falcon RX Setup

Although the Fibrolan switch has not been qualified in the NVIDIA lab, OAI labs incorporate the following configuration and switch for interoperability.

fibrolan_1.png

To get started, follow the Fibrolan Getting Started Guide.

In this setup, the Qulsar GrandMaster is connected to port 4, the Aerial cuBB to port 17, and the Foxconn O-RU to port 16 (C/U plane) and port 15 (S/M plane). You can ignore all other ports in the figures[A][B] below.

VLAN Setup

The following assumes that the VLAN tag is 2 for both the control plane and the user plane of the O-RAN CU plane. VLAN tag 80 is used for everything else.

Open the configuration page of the Fibrolan switch, then go to Configuration > VLANs. Port 4 (the Qulsar GrandMaster) needs to be set to “Access” mode, with the port VLAN set to 80.

fibrolan_2.png

Figure A - VLAN Setup

Use the same configuration for port 15 (RU S/M plane).

Configure ports 16 and 17 as follows:

  • Mode: “Trunk”

  • Port: VLAN 80

  • Untag Port VLAN

  • Allowed VLANs: 2, 80

fibrolan_3.png

Figure B - VLAN Setup

DHCP Setup

The RU M-plane requires you to set up a DHCP server. Go to Configuration > DHCP > Server > Pool and create a new DHCP server with the following settings:

fibrolan_4.png


PTP Setup

For the PTP setup, follow the Fibrolan PTP Boundary Clock Configuration guide and use the following settings:

  • Device Type: “Ord-Bound”

  • Profile: “G8275.1”

  • Clock domain: 24

  • VLAN: 80

Also make sure you enable the used ports (in this case, 4, 15, 16, and 17).

Hybrid mode is recommended as the sync mode.

If everything is configured correctly, the SyncCenter should show green.

fibrolan_5.png

These commands assume that PTP4L runs on the ens6f0 NIC interface and uses CPU core 20. Core clash can cause problems, so if a different core is being used, it must not be used by L1 or L2+.

Verify Inbound PTP Packets

Typically, you should see packets with ethertype 0x88f7 on the selected interface.

Copy
Copied!
            

sudo tcpdump -i ens6f0 -c 5 | grep ethertype tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on ens6f1, link-type EN10MB (Ethernet), capture size 262144 bytes 13:27:41.291503 48:b0:2d:63:83:ac (oui Unknown) > 01:1b:19:00:00:00 (oui Unknown), ethertype Unknown (0x88f7), length 60: 13:27:41.291503 48:b0:2d:63:83:ac (oui Unknown) > 01:1b:19:00:00:00 (oui Unknown), ethertype Unknown (0x88f7), length 60: 13:27:41.296727 c4:5a:b1:14:1a:c6 (oui Unknown) > 01:1b:19:00:00:00 (oui Unknown), ethertype Unknown (0x88f7), length 78: 13:27:41.296784 c4:5a:b1:14:1a:c6 (oui Unknown) > 01:1b:19:00:00:00 (oui Unknown), ethertype Unknown (0x88f7), length 60: 13:27:41.306316 08:c0:eb:71:e7:d5 (oui Unknown) > 01:1b:19:00:00:00 (oui Unknown), ethertype Unknown (0x88f7), length 58:


Create ptp4l Configuration File

Paste these commands into the shell to create the three configuration files:

Copy
Copied!
            

cat <<EOF | sudo tee /etc/ptp.conf [global] priority1 128 priority2 128 domainNumber 24 tx_timestamp_timeout 30 dscp_event 46 dscp_general 46 logging_level 6 verbose 1 use_syslog 0 logMinDelayReqInterval 1 [ens6f0] logAnnounceInterval -3 announceReceiptTimeout 3 logSyncInterval -4 logMinDelayReqInterval -4 delay_mechanism E2E network_transport L2 EOF cat <<EOF | sudo tee /lib/systemd/system/ptp4l.service [Unit] Description=Precision Time Protocol (PTP) service Documentation=man:ptp4l [Service] Restart=always RestartSec=5s Type=simple ExecStart=/usr/bin/taskset -c 9 /usr/sbin/ptp4l -f /etc/ptp.conf [Install] WantedBy=multi-user.target EOF


Create phc2sys Configuration File

Copy
Copied!
            

# If more than one instance is already running, kill the existing # PHC2SYS sessions. # Command used can be found in /lib/systemd/system/phc2sys.service # Update the ExecStart line to the following, assuming ens6f0 interface is used. sudo nano /lib/systemd/system/phc2sys.service [Unit] Description=Synchronize system clock or PTP hardware clock (PHC) Documentation=man:phc2sys After=ntpdate.service Requires=ptp4l.service After=ptp4l.service [Service] Restart=always RestartSec=5s Type=simple ExecStart=/bin/sh -c "taskset -c 9 /usr/sbin/phc2sys -s /dev/ptp$(ethtool -T ens6f0 | grep PTP | awk '{print $4}')-c CLOCK_REALTIME -n 24 -O 0 -R 256 -u 256" [Install] WantedBy=multi-user.target


Enable and Start phc2sys and ptp4l

After changing the configuration files, they need to be reloaded, enabled, and restarted. These services can be restarted if they don’t sync.

Copy
Copied!
            

sudo systemctl daemon-reload sudo systemctl enable ptp4l.service sudo systemctl enable phc2sys.service sudo systemctl restart phc2sys.service ptp4l.service # check that the service is active and has low rms value (<30): systemctl status ptp4l.service phc2sys.service ● ptp4l.service - Precision Time Protocol (PTP) service Loaded: loaded (/lib/systemd/system/ptp4l.service; enabled; vendor preset: enabled) Active: active (running) since Tue 2023-05-09 13:21:12 UTC; 14s ago Docs: man:ptp4l Main PID: 6962 (ptp4l) Tasks: 1 (limit: 94588) Memory: 544.0K CGroup: /system.slice/ptp4l.service └─6962 /usr/sbin/ptp4l -f /etc/ptp.conf May 09 13:21:17 aerial-rf-gb-gnb taskset[6962]: ptp4l[15552.609]: rms 15 max 32 freq -639 +/- 25 delay 211 +/- 1 May 09 13:21:18 aerial-rf-gb-gnb taskset[6962]: ptp4l[15553.609]: rms 21 max 29 freq -583 +/- 12 delay 210 +/- 1 May 09 13:21:19 aerial-rf-gb-gnb taskset[6962]: ptp4l[15554.609]: rms 11 max 21 freq -576 +/- 8 delay 211 +/- 1 May 09 13:21:20 aerial-rf-gb-gnb taskset[6962]: ptp4l[15555.609]: rms 6 max 13 freq -579 +/- 8 delay 211 +/- 1 May 09 13:21:21 aerial-rf-gb-gnb taskset[6962]: ptp4l[15556.609]: rms 4 max 7 freq -578 +/- 6 delay 212 +/- 0 May 09 13:21:22 aerial-rf-gb-gnb taskset[6962]: ptp4l[15557.609]: rms 5 max 11 freq -589 +/- 6 delay 213 +/- 1 May 09 13:21:23 aerial-rf-gb-gnb taskset[6962]: ptp4l[15558.609]: rms 6 max 12 freq -593 +/- 8 delay 210 +/- 1 May 09 13:21:24 aerial-rf-gb-gnb taskset[6962]: ptp4l[15559.609]: rms 3 max 7 freq -587 +/- 5 delay 211 +/- 1 May 09 13:21:25 aerial-rf-gb-gnb taskset[6962]: ptp4l[15560.609]: rms 5 max 12 freq -582 +/- 7 delay 212 +/- 1 May 09 13:21:26 aerial-rf-gb-gnb taskset[6962]: ptp4l[15561.609]: rms 4 max 7 freq -587 +/- 7 delay 213 +/- 1 ● phc2sys.service - Synchronize system clock or PTP hardware clock (PHC) Loaded: loaded (/lib/systemd/system/phc2sys.service; enabled; vendor preset: enabled) Active: active (running) since Tue 2023-05-09 13:21:12 UTC; 14s ago Docs: man:phc2sys Main PID: 6963 (phc2sys) Tasks: 1 (limit: 94588) Memory: 572.0K CGroup: /system.slice/phc2sys.service └─6963 /usr/sbin/phc2sys -a -r -n 24 -R 256 -u 256 May 09 13:21:17 aerial-rf-gb-gnb phc2sys[6963]: [15553.320] CLOCK_REALTIME rms 42 max 79 freq +8240 +/- 368 delay 1762 +/- 16 May 09 13:21:18 aerial-rf-gb-gnb phc2sys[6963]: [15554.336] CLOCK_REALTIME rms 35 max 64 freq +8091 +/- 303 delay 1754 +/- 13 May 09 13:21:19 aerial-rf-gb-gnb phc2sys[6963]: [15555.352] CLOCK_REALTIME rms 27 max 52 freq +8218 +/- 224 delay 1752 +/- 13 May 09 13:21:20 aerial-rf-gb-gnb phc2sys[6963]: [15556.368] CLOCK_REALTIME rms 21 max 49 freq +8153 +/- 152 delay 1758 +/- 16 May 09 13:21:21 aerial-rf-gb-gnb phc2sys[6963]: [15557.384] CLOCK_REALTIME rms 17 max 39 freq +8149 +/- 125 delay 1761 +/- 16 May 09 13:21:22 aerial-rf-gb-gnb phc2sys[6963]: [15558.400] CLOCK_REALTIME rms 14 max 33 freq +8185 +/- 101 delay 1750 +/- 14 May 09 13:21:23 aerial-rf-gb-gnb phc2sys[6963]: [15559.416] CLOCK_REALTIME rms 12 max 32 freq +8138 +/- 63 delay 1752 +/- 13 May 09 13:21:24 aerial-rf-gb-gnb phc2sys[6963]: [15560.431] CLOCK_REALTIME rms 11 max 43 freq +8171 +/- 54 delay 1756 +/- 15 May 09 13:21:25 aerial-rf-gb-gnb phc2sys[6963]: [15561.447] CLOCK_REALTIME rms 10 max 32 freq +8163 +/- 38 delay 1762 +/- 16 May 09 13:21:26 aerial-rf-gb-gnb phc2sys[6963]: [15562.463] CLOCK_REALTIME rms 9 max 23 freq +8162 +/- 17 delay 1761 +/- 16


Disable NTP

Use these commands to turn off NTP:

Copy
Copied!
            

sudo timedatectl set-ntp false timedatectl Local time: Thu 2022-02-03 22:30:58 UTC Universal time: Thu 2022-02-03 22:30:58 UTC RTC time: Thu 2022-02-03 22:30:58 Time zone: Etc/UTC (UTC, +0000) System clock synchronized: no NTP service: inactive RTC in local TZ: no


Verify System Clock Synchronization

Make NTP inactive and synchronize the system clock:

Copy
Copied!
            

timedatectl Local time: Thu 2022-02-03 22:30:58 UTC Universal time: Thu 2022-02-03 22:30:58 UTC RTC time: Thu 2022-02-03 22:30:58 Time zone: Etc/UTC (UTC, +0000) System clock synchronized: yes NTP service: inactive RTC in local TZ: no


Tip

There is a tutorial video for setting up the Foxconn ORU.

image7.png

Foxconn RPQN-7801E

Connections and Settings

image8.png
Connections:
  • 10SFP: C/U plane (will support S/M plane after firmware upgrade)
  • 1G RJ45: S/M plane
  • 10G RJ45: POE only
  • Micro-USB: USB to serial for debugging (115200, 8, 1, none, flow control off)

GrandMaster settings (Qulsar):

  • PTP timing port: Disable VLAN
  • Two steps: OFF
  • Domain number: 24 (needs to configured on O-RU)
  • IPv4, Unicast, etc.

/home/root/sdcard/RRHconfig_xran.xml:

  • RRH_PTPV2_GRAND_MASTER_IP = 20.0.0.8
  • RRH_PTPV2_SUB_DOMAIN_NUM = 24
  • C/U plane VLAN tag
  • RRH_LO_FREQUENCY_KHZ = 3750000

Configure VLAN and IP Address on the gNB Server

  1. Add these commands to the server startup script (/etc/rc.local) so they are automatically run on reboot.

  2. Configure these settings on the fronthaul port.

  3. You must use IP addresses that do not match those in the example below:

Copy
Copied!
            

sudo ip link add link ens6f0 name ens6f0.2 type vlan id 2 sudo ip addr add 169.254.1.103/24 dev ens6f0.2 sudo ip link set up ens6f0.2


O-RU M-Plane Setup

  1. Add the following to the bottom of /etc/profile and comment out the line with set_qse.sh if it already exists. Set the interface initially to eth0 for firmware version 1, and to qse-eth after upgrading to firmware version 2 or greater.

    Copy
    Copied!
                

    interface=eth0 vlanid=2 ipLastOctet=20 ip link add link ${interface} name ${interface}.$vlanid type vlan id $vlanid ip addr flush dev ${interface} ip addr add 169.254.0.0/24 dev ${interface} ip addr add 169.254.1.${ipLastOctet}/24 dev ${interface}.$vlanid ip link set up ${interface}.$vlanid

  2. Reboot the O-RU using the command ./reboot.sh and check the network configuration:

    Copy
    Copied!
                

    # ip r 169.254.1.0/24 dev eth0.2 src 169.254.1.20

Update O-RU Configuration

  1. Update the O-RU configuration in /home/root/sdcard/RRHconfig_xran.xml.

    Copy
    Copied!
                

    root@arria10:~/test# grep -v '<!-' ../sdcard/RRHconfig_xran.xml RRH_DST_MAC_ADDR = 08:c0:eb:71:e7:d4 # To match fronthaul interface of DU RRH_SRC_MAC_ADDR = 6C:AD:AD:00:04:6C # To match qse-eth of RU RRH_EN_EAXC_ID = 0 RRH_EAXC_ID_TYPE1 = 0x0, 0x1, 0x2, 0x3 RRH_EAXC_ID_TYPE3 = 0x8, 0x9, 0xA, 0xB RRH_EN_SPC = 1 RRH_RRH_LTE_OR_NR = 1 RRH_TRX_EN_BIT_MASK = 0x0f RRH_RF_EN_BIT_MASK = 0x0f RRH_CMPR_HDR_PRESENT = 0 RRH_CMPR_TYPE = 1, 1 RRH_CMPR_BIT_LENGTH = 9, 9 RRH_UL_INIT_SYM_ID = 0 RRH_TX_TRUNC_BITS = 4 RRH_RX_TRUNC_BITS = 4 RRH_MAX_PRB = 273 RRH_C_PLANE_VLAN_TAG = 0x0002 #To match vlan id set in cuphycontroller yaml file RRH_U_PLANE_VLAN_TAG = 0x0002 #To match vlan id set in cuphycontroller yaml file RRH_SLOT_TICKS_IN_SEC = 2000 RRH_SLOT_PERIOD_IN_SAMPLE = 61440 RRH_LO_FREQUENCY_KHZ = 3750000, 0 RRH_TX_POWER = 24, 24 RRH_TX_ATTENUATION = 12.0, 12.0, 12.0, 12.0 RRH_RX_ATTENUATION = 0.0, 0.0, 0.0, 0.0 RRH_BB_GENERAL_CTRL = 0x0, 0x0, 0x0, 0x0 RRH_RF_GENERAL_CTRL = 0x3, 0x1, 0x0, 0x0 RRH_PTPV2_GRAND_MASTER_MODE = 3 RRH_PTPV2_JITTER_LEVEL = 0 RRH_PTPV2_VLAN_ID = 0 RRH_PTPV2_IP_MODE = 4 RRH_PTPV2_GRAND_MASTER_IP = 192.167.27.150 RRH_PTPV2_SUB_DOMAIN_NUM = 24 RRH_PTPV2_ACCEPTED_CLOCK_CLASS = 135 RRH_TRACE_PERIOD = 10

    Note

    In Foxconn firmware version 3.1.15 and later, the configuration file is located in /home/root/test.

    Note

    The above configuration was taken from an ORU running firmware 2.6.9; for the latest configuration, see the full stack ARC-OTA resources.

  2. Reboot O-RU.

    Copy
    Copied!
                

    cd /home/root/test/ ./reboot

  3. Run the following to enable the configuration:

    Copy
    Copied!
                

    cd /home/root/test/ ./init_rrh_config_enable_cuplane

At this point, the console becomes unresponsive and fills with prints related to PTP, AFE initialization, and packet counters.

Previous Part 1. Procure the Hardware
Next Part 3. Configure the gNB Server
© Copyright 2024, NVIDIA.. Last updated on Jul 24, 2024.