Aerial RAN CoLab Over-the-Air (ARC-OTA) v1.2

Part 2. Configure the Network Hardware

Note

Refer to ASDK resources for setup and installation of the SDK Developer Kit. https://docs.nvidia.com/sdk-manager/install-with-sdkm-aerial/index.html

Refer to the tutorials for help with these installation steps.

Configuration Steps

  1. Setup the GrandMaster

  2. Setup the switch

  3. Setup PTP

  4. Setup Foxconn O-RU

Step 1.

Follow the Qulsar User Guide to setup the MGMT connection.

image1.png


Step 2.

Set the operating mode to GNSS Only, and other fields as such, then run Start Engine.

image2.png


Step 3.

Enable the ports on the GrandMaster with the 8275.1 Profile configurations.

image3.png


Step 4.

Configure the clock configs as such:

image4.png


Step 5.

GPS configuration values are unchanged from the default settings of QG2.

image5.png


Step 6.

Verify that the GPS Signal reaches the GrandMaster.

image6.png


Chapter 2.2.1 Dell Switch

In the following example on VLAN 2 the:

  • RUs are on ports 1 and 7

  • GrandMaster is on port 5

  • CN is on ports 11 and 12

  • gNB ports are connected to ports 49 and 51

Set up MGMT access to the switch, in this case 172.168.20.67.

Copy
Copied!
            

OS10# configure terminal OS10(config)# interface mgmt1/1/1 no shutdown no ip address dhcp ip address 172.16.204.67/22 exit

SSH to admin@172.168.204.67.

Set the speed to 10G for port groups 1 and 2.

Copy
Copied!
            

OS10(config)# port-group 1/1/1 mode Eth 10g-4x exit port-group 1/1/2 mode Eth 10g-4x exit

Enable PTP on the switch.

Copy
Copied!
            

OS10# configure terminal OS10(config)# ptp clock boundary profile g8275.1 ptp domain 24 ptp system-time enable !

Configure the GrandMaster port.

Copy
Copied!
            

OS10(config)# interface ethernet 1/1/5:1 no shutdown no switchport ip address 169.254.2.1/24 flowcontrol receive off ptp delay-req-min-interval -4 ptp enable ptp sync-interval -4 ptp transport layer2 exit

After some time, the following prints:

Copy
Copied!
            

<165>1 2023-05-09T07:49:22.625584+00:00 OS10 dn_alm 1021 - - Node.1-Unit.1:PRI [event], Dell EMC (OS10) %PTP_SYSTEM_TIME_NOT_SET: System time is not set. System time will be set when the clock is. <165>1 2023-05-09T07:51:22.312557+00:00 OS10 dn_alm 1021 - - Node.1-Unit.1:PRI [event], Dell EMC (OS10) %PTP_CLOCK_PHASE_LOCKED: Clock servo is phase locked. <165>1 2023-05-09T07:51:22.313081+00:00 OS10 dn_alm 1021 - - Node.1-Unit.1:PRI [event], Dell EMC (OS10) %PTP_SYSTEM_TIME_UPDATE_STARTED: System time update service is started. Update interval: 60 minutes. <165>1 2023-05-09T07:51:59.334346+00:00 OS10 dn_alm 1021 - - Node.1-Unit.1:PRI [event], Dell EMC (OS10) %ALM_CLOCK_UPDATE: Clock changed MESSAGE=apt-daily.timer: Adding 6h 36min 18.719270s random time. <165>1 2023-05-09T07:57:27.254181+00:00 OS10 dn_alm 1021 - - Node.1-Unit.1:PRI [event], Dell EMC (OS10) %ALM_CLOCK_UPDATE: Clock changed MESSAGE=apt-daily.timer: Adding 4h 31mi

Confgure Fronthaul Network Configuration by creating a VLAN.

Create VLAN 2.

Copy
Copied!
            

OS10(config)# interface vlan 2 OS10(conf-if-vl-2)# <165>1 2023-03-16T16:51:36.458730+00:00 OS10 dn_alm 813 - - Node.1-Unit.1:PRI [event], Dell EMC (OS10) %IFM_ASTATE_UP: Interface admin state up :vlan2 OS10(conf-if-vl-2)# show configuration ! interface vlan2 no shutdown OS10(conf-if-vl-2)# exit

Configure RU, gNB, CN, and MEC ports.

Intefaces that are configured to be slower than their maximum speed have a :1 appended to their name. This applies to ports in port groups 1 and 2.

Copy
Copied!
            

no shutdown switchport mode trunk switchport trunk allowed vlan 2 mtu 8192 flowcontrol receive off ptp enable ptp transport layer2 ptp role timeTransmitter exit

Check the PTP status.

Copy
Copied!
            

OS10# show ptp | no-more PTP Clock : Boundary Clock Identity : b0:4f:13:ff:ff:46:63:5f GrandMaster Clock Identity : fc:af:6a:ff:fe:02:bc:8d Clock Mode : One-step Clock Quality Class : 135 Accuracy : <=100ns Offset Log Scaled Variance : 65535 Domain : 24 Priority1 : 128 Priority2 : 128 Profile : G8275-1(Local-Priority:-128) Steps Removed : 1 Mean Path Delay(ns) : 637 Offset From Master(ns) : 1 Number of Ports : 8 ---------------------------------------------------------------------------- Interface State Port Identity ---------------------------------------------------------------------------- Ethernet1/1/1:1 Master b0:4f:13:ff:ff:46:63:5f:1 Ethernet1/1/3:1 Master b0:4f:13:ff:ff:46:63:5f:3 Ethernet1/1/5:1 Slave b0:4f:13:ff:ff:46:63:5f:5 Ethernet1/1/7:1 Master b0:4f:13:ff:ff:46:63:5f:8 Ethernet1/1/11 Master b0:4f:13:ff:ff:46:63:5f:4 Ethernet1/1/49 Master b0:4f:13:ff:ff:46:63:5f:9 Ethernet1/1/51 Master b0:4f:13:ff:ff:46:63:5f:10 Ethernet1/1/54 Master b0:4f:13:ff:ff:46:63:5f:2 ---------------------------------------------------------------------------- Number of slave ports :1 Number of master ports :7

Save switch configuration

Copy
Copied!
            

copy running-configuration startup-configuration


Chapter 2.2.2 Fibrolan Falcon RX Setup

Although the Fibrolan switch has not been qualified in the NVIDIA lab, OAI labs incorporate the following configuration and switch for interoperability.

fibrolan_1.png

To get started follow the Fibrolan Getting Started Guide.

In our setup the Qulsar GrandMaster is connected to port 4, the Aerial SDK to port 17, and the Foxconn O-RU to port 16 (C/U plane) and port 15 (S/M plane). You can ignore all other ports in the figures[A][B] below.

VLAN Setup

The following assumes that the VLAN tag for both the control plane and the user plane of the O-RAN CU plane is 2. VLAN 80 is used for everything else.

fibrolan_2.png

Figure A - VLAN Setup

Open the configuration page of the Fibrolan switch, go to configuration -> VLANs. Port 4 (the Qulsar GrandMaster) needs to be configured in Access mode using and setting the port VLAN to 80.

fibrolan_3.png

Figure B - VLAN Setup

Use the same configuration for port 15 (RU S/M plane).

Ports 16 and 17 need to be configured in Trunk mode, port VLAN 80, Untag Port VLAN, Allowed VLANs 80,2.

DHCP Setup

The RU M-plane requires to setup a DHCP server. Go to Configuration -> DHCP -> server -> pool and create a new DHCP server with the following settings:

fibrolan_4.png


PTP Setup

For the PTP setup, follow the Fibrolan PTP Boundary Clock Configuration guide and use the following settings:

  • Device Type “Ord-Bound”

  • Profile “G8275.1”

  • Clock domain 24

  • VLAN 80

Also make sure you enable the used ports (4,15,16,17 in our case).

Hybrid mode is recommended for the sync mode.

If everything is configured correctly, the SyncCenter shows green.

fibrolan_5.png

These commands assume that PTP4L runs on the ens6f0 NIC interface and uses the CPU core 20. Core clash can cause problems, so if a different core is to be used, it must not be used by L1 or L2+.

Verify Inbound PTP Packets

Typically, you see packets with ethertype 0x88f7 on the selected interface.

Copy
Copied!
            

$ sudo tcpdump -i ens6f0 -c 5 | grep ethertype tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on ens6f1, link-type EN10MB (Ethernet), capture size 262144 bytes 13:27:41.291503 48:b0:2d:63:83:ac (oui Unknown) > 01:1b:19:00:00:00 (oui Unknown), ethertype Unknown (0x88f7), length 60: 13:27:41.291503 48:b0:2d:63:83:ac (oui Unknown) > 01:1b:19:00:00:00 (oui Unknown), ethertype Unknown (0x88f7), length 60: 13:27:41.296727 c4:5a:b1:14:1a:c6 (oui Unknown) > 01:1b:19:00:00:00 (oui Unknown), ethertype Unknown (0x88f7), length 78: 13:27:41.296784 c4:5a:b1:14:1a:c6 (oui Unknown) > 01:1b:19:00:00:00 (oui Unknown), ethertype Unknown (0x88f7), length 60: 13:27:41.306316 08:c0:eb:71:e7:d5 (oui Unknown) > 01:1b:19:00:00:00 (oui Unknown), ethertype Unknown (0x88f7), length 58:


Create ptp4l Configuration File

Paste these commands into the shell to create the three configuration files:

Copy
Copied!
            

cat <<EOF | sudo tee /etc/ptp.conf [global] priority1 128 priority2 128 domainNumber 24 tx_timestamp_timeout 30 dscp_event 46 dscp_general 46 logging_level 6 verbose 1 use_syslog 0 logMinDelayReqInterval 1 [ens6f0] logAnnounceInterval -3 announceReceiptTimeout 3 logSyncInterval -4 logMinDelayReqInterval -4 delay_mechanism E2E network_transport L2 EOF cat <<EOF | sudo tee /lib/systemd/system/ptp4l.service [Unit] Description=Precision Time Protocol (PTP) service Documentation=man:ptp4l [Service] Restart=always RestartSec=5s Type=simple ExecStart=/usr/bin/taskset -c 9 /usr/sbin/ptp4l -f /etc/ptp.conf [Install] WantedBy=multi-user.target EOF


Create phc2sys Configuration File

Copy
Copied!
            

# If more than one instance is already running, kill the existing # PHC2SYS sessions. # Command used can be found in /lib/systemd/system/phc2sys.service # Update the ExecStart line to the following, assuming ens6f0 interface is used. $ sudo nano /lib/systemd/system/phc2sys.service [Unit] Description=Synchronize system clock or PTP hardware clock (PHC) Documentation=man:phc2sys After=ntpdate.service Requires=ptp4l.service After=ptp4l.service [Service] Restart=always RestartSec=5s Type=simple ExecStart=/bin/sh -c "taskset -c 9 /usr/sbin/phc2sys -s /dev/ptp$(ethtool -T ens6f0 | grep PTP | awk '{print $4}')-c CLOCK_REALTIME -n 24 -O 0 -R 256 -u 256" [Install] WantedBy=multi-user.target


Enable and Start phc2sys and ptp4l

After changing those configuration files they need to be reloaded, enabled, and restarted. These services can be restarted if they don’t sync.

Copy
Copied!
            

$ sudo systemctl daemon-reload $ sudo systemctl enable ptp4l.service $ sudo systemctl enable phc2sys.service $ sudo systemctl restart phc2sys.service ptp4l.service # check that the service is active and has low rms value (<30): ~$ systemctl status ptp4l.service phc2sys.service ● ptp4l.service - Precision Time Protocol (PTP) service Loaded: loaded (/lib/systemd/system/ptp4l.service; enabled; vendor preset: enabled) Active: active (running) since Tue 2023-05-09 13:21:12 UTC; 14s ago Docs: man:ptp4l Main PID: 6962 (ptp4l) Tasks: 1 (limit: 94588) Memory: 544.0K CGroup: /system.slice/ptp4l.service └─6962 /usr/sbin/ptp4l -f /etc/ptp.conf May 09 13:21:17 aerial-rf-gb-gnb taskset[6962]: ptp4l[15552.609]: rms 15 max 32 freq -639 +/- 25 delay 211 +/- 1 May 09 13:21:18 aerial-rf-gb-gnb taskset[6962]: ptp4l[15553.609]: rms 21 max 29 freq -583 +/- 12 delay 210 +/- 1 May 09 13:21:19 aerial-rf-gb-gnb taskset[6962]: ptp4l[15554.609]: rms 11 max 21 freq -576 +/- 8 delay 211 +/- 1 May 09 13:21:20 aerial-rf-gb-gnb taskset[6962]: ptp4l[15555.609]: rms 6 max 13 freq -579 +/- 8 delay 211 +/- 1 May 09 13:21:21 aerial-rf-gb-gnb taskset[6962]: ptp4l[15556.609]: rms 4 max 7 freq -578 +/- 6 delay 212 +/- 0 May 09 13:21:22 aerial-rf-gb-gnb taskset[6962]: ptp4l[15557.609]: rms 5 max 11 freq -589 +/- 6 delay 213 +/- 1 May 09 13:21:23 aerial-rf-gb-gnb taskset[6962]: ptp4l[15558.609]: rms 6 max 12 freq -593 +/- 8 delay 210 +/- 1 May 09 13:21:24 aerial-rf-gb-gnb taskset[6962]: ptp4l[15559.609]: rms 3 max 7 freq -587 +/- 5 delay 211 +/- 1 May 09 13:21:25 aerial-rf-gb-gnb taskset[6962]: ptp4l[15560.609]: rms 5 max 12 freq -582 +/- 7 delay 212 +/- 1 May 09 13:21:26 aerial-rf-gb-gnb taskset[6962]: ptp4l[15561.609]: rms 4 max 7 freq -587 +/- 7 delay 213 +/- 1 ● phc2sys.service - Synchronize system clock or PTP hardware clock (PHC) Loaded: loaded (/lib/systemd/system/phc2sys.service; enabled; vendor preset: enabled) Active: active (running) since Tue 2023-05-09 13:21:12 UTC; 14s ago Docs: man:phc2sys Main PID: 6963 (phc2sys) Tasks: 1 (limit: 94588) Memory: 572.0K CGroup: /system.slice/phc2sys.service └─6963 /usr/sbin/phc2sys -a -r -n 24 -R 256 -u 256 May 09 13:21:17 aerial-rf-gb-gnb phc2sys[6963]: [15553.320] CLOCK_REALTIME rms 42 max 79 freq +8240 +/- 368 delay 1762 +/- 16 May 09 13:21:18 aerial-rf-gb-gnb phc2sys[6963]: [15554.336] CLOCK_REALTIME rms 35 max 64 freq +8091 +/- 303 delay 1754 +/- 13 May 09 13:21:19 aerial-rf-gb-gnb phc2sys[6963]: [15555.352] CLOCK_REALTIME rms 27 max 52 freq +8218 +/- 224 delay 1752 +/- 13 May 09 13:21:20 aerial-rf-gb-gnb phc2sys[6963]: [15556.368] CLOCK_REALTIME rms 21 max 49 freq +8153 +/- 152 delay 1758 +/- 16 May 09 13:21:21 aerial-rf-gb-gnb phc2sys[6963]: [15557.384] CLOCK_REALTIME rms 17 max 39 freq +8149 +/- 125 delay 1761 +/- 16 May 09 13:21:22 aerial-rf-gb-gnb phc2sys[6963]: [15558.400] CLOCK_REALTIME rms 14 max 33 freq +8185 +/- 101 delay 1750 +/- 14 May 09 13:21:23 aerial-rf-gb-gnb phc2sys[6963]: [15559.416] CLOCK_REALTIME rms 12 max 32 freq +8138 +/- 63 delay 1752 +/- 13 May 09 13:21:24 aerial-rf-gb-gnb phc2sys[6963]: [15560.431] CLOCK_REALTIME rms 11 max 43 freq +8171 +/- 54 delay 1756 +/- 15 May 09 13:21:25 aerial-rf-gb-gnb phc2sys[6963]: [15561.447] CLOCK_REALTIME rms 10 max 32 freq +8163 +/- 38 delay 1762 +/- 16 May 09 13:21:26 aerial-rf-gb-gnb phc2sys[6963]: [15562.463] CLOCK_REALTIME rms 9 max 23 freq +8162 +/- 17 delay 1761 +/- 16


Disable NTP

Enter these commands to turn off NTP:

Copy
Copied!
            

$ sudo timedatectl set-ntp false $ timedatectl Local time: Thu 2022-02-03 22:30:58 UTC Universal time: Thu 2022-02-03 22:30:58 UTC RTC time: Thu 2022-02-03 22:30:58 Time zone: Etc/UTC (UTC, +0000) System clock synchronized: no NTP service: inactive RTC in local TZ: no


Verfiy System Clock Synchronization

Make NTP inactive and sychronise the system clock:

Copy
Copied!
            

$ timedatectl Local time: Thu 2022-02-03 22:30:58 UTC Universal time: Thu 2022-02-03 22:30:58 UTC RTC time: Thu 2022-02-03 22:30:58 Time zone: Etc/UTC (UTC, +0000) System clock synchronized: yes NTP service: inactive RTC in local TZ: no


Note

There is a Tutorial video for setting up the Foxconn ORU.

image7.png

Foxconn RPQN-7801E

Connections and Settings

image8.png
Connections:
  • 10SFP: C/U plane (will support S/M plane after FW upgrade)
  • 1G RJ45: S/M plane
  • 10G RJ45: POE only
  • Micro-USB: USB to serial for debugging (115200, 8, 1, none, flow control off)

GrandMaster settings (Qulsar):

  • PTP timing port: Disable VLAN
  • Two steps: OFF
  • Domain number: 24 <- need to config on O-RU
  • IPv4, Unicast, etc.

/home/root/sdcard/RRHconfig_xran.xml:

  • RRH_PTPV2_GRAND_MASTER_IP = 20.0.0.8
  • RRH_PTPV2_SUB_DOMAIN_NUM = 24
  • C/U plane VLAN tag
  • RRH_LO_FREQUENCY_KHZ = 3750000

Configure VLAN and IP Address on the gNB Server

  1. Add these instructions to the server startup script ‘/etc/rc.local’ so they are automatically run on reboot.

  2. Configure this on the fronthaul port.

  3. You must use IP addresses that do not match those in the example below:

Copy
Copied!
            

sudo ip link add link ens6f0 name ens6f0.2 type vlan id 2 sudo ip addr add 169.254.1.103/24 dev ens6f0.2 sudo ip link set up ens6f0.2


O-RU M-Plane Setup

Add the following to the bottom of /etc/profile and comment out the line with set_qse.sh if it exists. Set the interface initially to eth0 for firmware version 1, and to qse-eth after upgrading to firmware version ≥ 2.

Copy
Copied!
            

interface=eth0 vlanid=2 ipLastOctet=20 ip link add link ${interface} name ${interface}.$vlanid type vlan id $vlanid ip addr flush dev ${interface} ip addr add 169.254.0.0/24 dev ${interface} ip addr add 169.254.1.${ipLastOctet}/24 dev ${interface}.$vlanid ip link set up ${interface}.$vlanid

Reboot the O-RU using the command ./reboot.sh and check the network configuration:

Copy
Copied!
            

# ip r 169.254.1.0/24 dev eth0.2 src 169.254.1.20


Update O-RU Configuration

Update configurations in /home/root/sdcard/RRHconfig_xran.xml.

Copy
Copied!
            

root@arria10:~/test# grep -v '<!-' ../sdcard/RRHconfig_xran.xml RRH_DST_MAC_ADDR = 08:c0:eb:71:e7:d4 # To match fronthaul interface of DU RRH_SRC_MAC_ADDR = 6C:AD:AD:00:04:6C # To match qse-eth of RU RRH_EN_EAXC_ID = 0 RRH_EAXC_ID_TYPE1 = 0x0, 0x1, 0x2, 0x3 RRH_EAXC_ID_TYPE3 = 0x8, 0x9, 0xA, 0xB RRH_EN_SPC = 1 RRH_RRH_LTE_OR_NR = 1 RRH_TRX_EN_BIT_MASK = 0x0f RRH_RF_EN_BIT_MASK = 0x0f RRH_CMPR_HDR_PRESENT = 0 RRH_CMPR_TYPE = 1, 1 RRH_CMPR_BIT_LENGTH = 9, 9 RRH_UL_INIT_SYM_ID = 0 RRH_TX_TRUNC_BITS = 4 RRH_RX_TRUNC_BITS = 4 RRH_MAX_PRB = 273 RRH_C_PLANE_VLAN_TAG = 0x0002 #To match vlan id set in cuphycontroller yaml file RRH_U_PLANE_VLAN_TAG = 0x0002 #To match vlan id set in cuphycontroller yaml file RRH_SLOT_TICKS_IN_SEC = 2000 RRH_SLOT_PERIOD_IN_SAMPLE = 61440 RRH_LO_FREQUENCY_KHZ = 3750000, 0 RRH_TX_POWER = 24, 24 RRH_TX_ATTENUATION = 12.0, 12.0, 12.0, 12.0 RRH_RX_ATTENUATION = 0.0, 0.0, 0.0, 0.0 RRH_BB_GENERAL_CTRL = 0x0, 0x0, 0x0, 0x0 RRH_RF_GENERAL_CTRL = 0x3, 0x1, 0x0, 0x0 RRH_PTPV2_GRAND_MASTER_MODE = 3 RRH_PTPV2_JITTER_LEVEL = 0 RRH_PTPV2_VLAN_ID = 0 RRH_PTPV2_IP_MODE = 4 RRH_PTPV2_GRAND_MASTER_IP = 192.167.27.150 RRH_PTPV2_SUB_DOMAIN_NUM = 24 RRH_PTPV2_ACCEPTED_CLOCK_CLASS = 135 RRH_TRACE_PERIOD = 10


Note

In Foxconn firmware version 3.1.15 or later, the configuration file is located in /home/root/test.

Note

The above configuration was taken from an ORU that is running firmware 2.6.9, for the latest configuration, see the full stack ARC resources.

Reboot O-RU

Copy
Copied!
            

cd /home/root/test/ ./reboot

Run the following to enable the config

Copy
Copied!
            

cd /home/root/test/ ./init_rrh_config_enable_cuplane

At this point the console becomes unresponsive and fills with prints related to PTP, AFE initialization, and packet counters.

Previous Part 1. Procure the Hardware
Next Part 3. Configure gNB Server
© Copyright 2023, NVIDIA.. Last updated on Feb 9, 2024.