Refer to the NVIDIA SDK Manager resources for setup and installation of ARC-OTA.
Refer to the tutorials for help with these installation steps.
The network hardware is configured in the following steps.
Setup the GrandMaster
Setup the switch
Setup PTP
Setup Foxconn O-RU
Step 1.
Follow the Qulsar User Guide to set up the MGMT connection.
Step 2.
Set the operating mode to GNSS Only, and other fields as such, then run Start Engine.
Step 3.
Enable the ports on the GrandMaster with the 8275.1 Profile configurations.
Step 4.
Configure the clock configs as follows:
Step 5.
Ensure the GPS configuration values are unchanged from the QG2 default settings.
Step 6.
Verify that the GPS Signal reaches the GrandMaster.
Chapter 2.2.1 Dell Switch
The following example uses these VLAN 2 settings:
RUs are on ports 1 and 7
GrandMaster is on port 5
CN is on ports 11 and 12
gNB ports are connected to ports 49 and 51
Set up MGMT access to the switch (in this case 172.168.20.67):
OS10# configure terminal OS10(config)# interface mgmt1/1/1 no shutdown no ip address dhcp ip address 172.16.204.67/22 exit
Use SSH to access
admin@172.168.204.67.Set the speed to 10G for port groups 1 and 2.
OS10(config)# port-group 1/1/1 mode Eth 10g-4x exit port-group 1/1/2 mode Eth 10g-4x exit
Enable PTP on the switch.
OS10# configure terminal OS10(config)# ptp clock boundary profile g8275.1 ptp domain 24 ptp system-time enable !
Configure the GrandMaster port.
OS10(config)# interface ethernet 1/1/5:1 no shutdown no switchport ip address 169.254.2.1/24 flowcontrol receive off ptp delay-req-min-interval -4 ptp enable ptp sync-interval -4 ptp transport layer2 exit
After some time, the following values will print:
<165>1 2023-05-09T07:49:22.625584+00:00 OS10 dn_alm 1021 - - Node.1-Unit.1:PRI [event], Dell EMC (OS10) %PTP_SYSTEM_TIME_NOT_SET: System time is not set. System time will be set when the clock is. <165>1 2023-05-09T07:51:22.312557+00:00 OS10 dn_alm 1021 - - Node.1-Unit.1:PRI [event], Dell EMC (OS10) %PTP_CLOCK_PHASE_LOCKED: Clock servo is phase locked. <165>1 2023-05-09T07:51:22.313081+00:00 OS10 dn_alm 1021 - - Node.1-Unit.1:PRI [event], Dell EMC (OS10) %PTP_SYSTEM_TIME_UPDATE_STARTED: System time update service is started. Update interval: 60 minutes. <165>1 2023-05-09T07:51:59.334346+00:00 OS10 dn_alm 1021 - - Node.1-Unit.1:PRI [event], Dell EMC (OS10) %ALM_CLOCK_UPDATE: Clock changed MESSAGE=apt-daily.timer: Adding 6h 36min 18.719270s random time. <165>1 2023-05-09T07:57:27.254181+00:00 OS10 dn_alm 1021 - - Node.1-Unit.1:PRI [event], Dell EMC (OS10) %ALM_CLOCK_UPDATE: Clock changed MESSAGE=apt-daily.timer: Adding 4h 31mi
Configure the Fronthaul Network Configuration by creating a VLAN.
NoteIf you choose to use a different VLAN, you must modify the Aerial YAML file and O-RU configuration. C- and U-planes use the same VLAN.
Create “VLAN 2”.
OS10(config)# interface vlan 2 OS10(conf-if-vl-2)# <165>1 2023-03-16T16:51:36.458730+00:00 OS10 dn_alm 813 - - Node.1-Unit.1:PRI [event], Dell EMC (OS10) %IFM_ASTATE_UP: Interface admin state up :vlan2 OS10(conf-if-vl-2)# show configuration ! interface vlan2 no shutdown OS10(conf-if-vl-2)# exit
Configure the RU, gNB, CN, and MEC ports.
Interfaces that are configured to be slower than their maximum speed have a
:1appended to their name. This applies to ports in port groups 1 and 2.no shutdown switchport mode trunk switchport trunk allowed vlan 2 mtu 8192 flowcontrol receive off ptp enable ptp transport layer2 ptp role timeTransmitter exit
Check the PTP status.
OS10# show ptp | no-more PTP Clock : Boundary Clock Identity : b0:4f:13:ff:ff:46:63:5f GrandMaster Clock Identity : fc:af:6a:ff:fe:02:bc:8d Clock Mode : One-step Clock Quality Class : 135 Accuracy : <=100ns Offset Log Scaled Variance : 65535 Domain : 24 Priority1 : 128 Priority2 : 128 Profile : G8275-1(Local-Priority:-128) Steps Removed : 1 Mean Path Delay(ns) : 637 Offset From Master(ns) : 1 Number of Ports : 8 ---------------------------------------------------------------------------- Interface State Port Identity ---------------------------------------------------------------------------- Ethernet1/1/1:1 Master b0:4f:13:ff:ff:46:63:5f:1 Ethernet1/1/3:1 Master b0:4f:13:ff:ff:46:63:5f:3 Ethernet1/1/5:1 Slave b0:4f:13:ff:ff:46:63:5f:5 Ethernet1/1/7:1 Master b0:4f:13:ff:ff:46:63:5f:8 Ethernet1/1/11 Master b0:4f:13:ff:ff:46:63:5f:4 Ethernet1/1/49 Master b0:4f:13:ff:ff:46:63:5f:9 Ethernet1/1/51 Master b0:4f:13:ff:ff:46:63:5f:10 Ethernet1/1/54 Master b0:4f:13:ff:ff:46:63:5f:2 ---------------------------------------------------------------------------- Number of slave ports :1 Number of master ports :7
Save the switch configuration:
copy running-configuration startup-configuration
Chapter 2.2.2 Fibrolan Falcon RX Setup
Although the Fibrolan switch has not been qualified in the NVIDIA lab, OAI labs incorporate the following configuration and switch for interoperability.
To get started, follow the Fibrolan Getting Started Guide.
In this setup, the Qulsar GrandMaster is connected to port 4, the Aerial cuBB to port 17, and the Foxconn O-RU to port 16 (C/U plane) and port 15 (S/M plane). You can ignore all other ports in the figures[A][B] below.
VLAN Setup
The following assumes that the VLAN tag is 2 for both the control plane and the user plane of the O-RAN CU plane. VLAN tag 80 is used for everything else.
Open the configuration page of the Fibrolan switch, then go to Configuration > VLANs. Port 4 (the Qulsar GrandMaster) needs to be set to “Access” mode, with the port VLAN set to 80.
Figure A - VLAN Setup
Use the same configuration for port 15 (RU S/M plane).
Configure ports 16 and 17 as follows:
Mode: “Trunk”
Port: VLAN 80
Untag Port VLAN
Allowed VLANs: 2, 80
Figure B - VLAN Setup
DHCP Setup
The RU M-plane requires you to set up a DHCP server. Go to Configuration > DHCP > Server > Pool and create a new DHCP server with the following settings:
PTP Setup
For the PTP setup, follow the Fibrolan PTP Boundary Clock Configuration guide and use the following settings:
Device Type: “Ord-Bound”
Profile: “G8275.1”
Clock domain: 24
VLAN: 80
Also make sure you enable the used ports (in this case, 4, 15, 16, and 17).
Hybrid mode is recommended as the sync mode.
If everything is configured correctly, the SyncCenter should show green.
These commands assume that PTP4L runs on the ens6f0 NIC interface and uses CPU core 20.
Core clash can cause problems, so if a different core is being used, it must not be used by L1 or L2+.
Verify Inbound PTP Packets
Typically, you should see packets with ethertype 0x88f7 on the selected interface.
sudo tcpdump -i ens6f0 -c 5 | grep ethertype
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens6f1, link-type EN10MB (Ethernet), capture size 262144 bytes
13:27:41.291503 48:b0:2d:63:83:ac (oui Unknown) > 01:1b:19:00:00:00 (oui Unknown), ethertype Unknown (0x88f7), length 60:
13:27:41.291503 48:b0:2d:63:83:ac (oui Unknown) > 01:1b:19:00:00:00 (oui Unknown), ethertype Unknown (0x88f7), length 60:
13:27:41.296727 c4:5a:b1:14:1a:c6 (oui Unknown) > 01:1b:19:00:00:00 (oui Unknown), ethertype Unknown (0x88f7), length 78:
13:27:41.296784 c4:5a:b1:14:1a:c6 (oui Unknown) > 01:1b:19:00:00:00 (oui Unknown), ethertype Unknown (0x88f7), length 60:
13:27:41.306316 08:c0:eb:71:e7:d5 (oui Unknown) > 01:1b:19:00:00:00 (oui Unknown), ethertype Unknown (0x88f7), length 58:
Create ptp4l Configuration File
Paste these commands into the shell to create the three configuration files:
cat <<EOF | sudo tee /etc/ptp.conf
[global]
priority1 128
priority2 128
domainNumber 24
tx_timestamp_timeout 30
dscp_event 46
dscp_general 46
logging_level 6
verbose 1
use_syslog 0
logMinDelayReqInterval 1
[ens6f0]
logAnnounceInterval -3
announceReceiptTimeout 3
logSyncInterval -4
logMinDelayReqInterval -4
delay_mechanism E2E
network_transport L2
EOF
cat <<EOF | sudo tee /lib/systemd/system/ptp4l.service
[Unit]
Description=Precision Time Protocol (PTP) service
Documentation=man:ptp4l
[Service]
Restart=always
RestartSec=5s
Type=simple
ExecStart=/usr/bin/taskset -c 9 /usr/sbin/ptp4l -f /etc/ptp.conf
[Install]
WantedBy=multi-user.target
EOF
Create phc2sys Configuration File
# If more than one instance is already running, kill the existing
# PHC2SYS sessions.
# Command used can be found in /lib/systemd/system/phc2sys.service
# Update the ExecStart line to the following, assuming ens6f0 interface is used.
sudo nano /lib/systemd/system/phc2sys.service
[Unit]
Description=Synchronize system clock or PTP hardware clock (PHC)
Documentation=man:phc2sys
After=ntpdate.service
Requires=ptp4l.service
After=ptp4l.service
[Service]
Restart=always
RestartSec=5s
Type=simple
ExecStart=/bin/sh -c "taskset -c 9 /usr/sbin/phc2sys -s /dev/ptp$(ethtool -T ens6f0 | grep PTP | awk '{print $4}')-c CLOCK_REALTIME -n 24 -O 0 -R 256 -u 256"
[Install]
WantedBy=multi-user.target
Enable and Start phc2sys and ptp4l
After changing the configuration files, they need to be reloaded, enabled, and restarted. These services can be restarted if they don’t sync.
sudo systemctl daemon-reload
sudo systemctl enable ptp4l.service
sudo systemctl enable phc2sys.service
sudo systemctl restart phc2sys.service ptp4l.service
# check that the service is active and has low rms value (<30):
systemctl status ptp4l.service phc2sys.service
● ptp4l.service - Precision Time Protocol (PTP) service
Loaded: loaded (/lib/systemd/system/ptp4l.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2023-05-09 13:21:12 UTC; 14s ago
Docs: man:ptp4l
Main PID: 6962 (ptp4l)
Tasks: 1 (limit: 94588)
Memory: 544.0K
CGroup: /system.slice/ptp4l.service
└─6962 /usr/sbin/ptp4l -f /etc/ptp.conf
May 09 13:21:17 aerial-rf-gb-gnb taskset[6962]: ptp4l[15552.609]: rms 15 max 32 freq -639 +/- 25 delay 211 +/- 1
May 09 13:21:18 aerial-rf-gb-gnb taskset[6962]: ptp4l[15553.609]: rms 21 max 29 freq -583 +/- 12 delay 210 +/- 1
May 09 13:21:19 aerial-rf-gb-gnb taskset[6962]: ptp4l[15554.609]: rms 11 max 21 freq -576 +/- 8 delay 211 +/- 1
May 09 13:21:20 aerial-rf-gb-gnb taskset[6962]: ptp4l[15555.609]: rms 6 max 13 freq -579 +/- 8 delay 211 +/- 1
May 09 13:21:21 aerial-rf-gb-gnb taskset[6962]: ptp4l[15556.609]: rms 4 max 7 freq -578 +/- 6 delay 212 +/- 0
May 09 13:21:22 aerial-rf-gb-gnb taskset[6962]: ptp4l[15557.609]: rms 5 max 11 freq -589 +/- 6 delay 213 +/- 1
May 09 13:21:23 aerial-rf-gb-gnb taskset[6962]: ptp4l[15558.609]: rms 6 max 12 freq -593 +/- 8 delay 210 +/- 1
May 09 13:21:24 aerial-rf-gb-gnb taskset[6962]: ptp4l[15559.609]: rms 3 max 7 freq -587 +/- 5 delay 211 +/- 1
May 09 13:21:25 aerial-rf-gb-gnb taskset[6962]: ptp4l[15560.609]: rms 5 max 12 freq -582 +/- 7 delay 212 +/- 1
May 09 13:21:26 aerial-rf-gb-gnb taskset[6962]: ptp4l[15561.609]: rms 4 max 7 freq -587 +/- 7 delay 213 +/- 1
● phc2sys.service - Synchronize system clock or PTP hardware clock (PHC)
Loaded: loaded (/lib/systemd/system/phc2sys.service; enabled; vendor preset: enabled)
Active: active (running) since Tue 2023-05-09 13:21:12 UTC; 14s ago
Docs: man:phc2sys
Main PID: 6963 (phc2sys)
Tasks: 1 (limit: 94588)
Memory: 572.0K
CGroup: /system.slice/phc2sys.service
└─6963 /usr/sbin/phc2sys -a -r -n 24 -R 256 -u 256
May 09 13:21:17 aerial-rf-gb-gnb phc2sys[6963]: [15553.320] CLOCK_REALTIME rms 42 max 79 freq +8240 +/- 368 delay 1762 +/- 16
May 09 13:21:18 aerial-rf-gb-gnb phc2sys[6963]: [15554.336] CLOCK_REALTIME rms 35 max 64 freq +8091 +/- 303 delay 1754 +/- 13
May 09 13:21:19 aerial-rf-gb-gnb phc2sys[6963]: [15555.352] CLOCK_REALTIME rms 27 max 52 freq +8218 +/- 224 delay 1752 +/- 13
May 09 13:21:20 aerial-rf-gb-gnb phc2sys[6963]: [15556.368] CLOCK_REALTIME rms 21 max 49 freq +8153 +/- 152 delay 1758 +/- 16
May 09 13:21:21 aerial-rf-gb-gnb phc2sys[6963]: [15557.384] CLOCK_REALTIME rms 17 max 39 freq +8149 +/- 125 delay 1761 +/- 16
May 09 13:21:22 aerial-rf-gb-gnb phc2sys[6963]: [15558.400] CLOCK_REALTIME rms 14 max 33 freq +8185 +/- 101 delay 1750 +/- 14
May 09 13:21:23 aerial-rf-gb-gnb phc2sys[6963]: [15559.416] CLOCK_REALTIME rms 12 max 32 freq +8138 +/- 63 delay 1752 +/- 13
May 09 13:21:24 aerial-rf-gb-gnb phc2sys[6963]: [15560.431] CLOCK_REALTIME rms 11 max 43 freq +8171 +/- 54 delay 1756 +/- 15
May 09 13:21:25 aerial-rf-gb-gnb phc2sys[6963]: [15561.447] CLOCK_REALTIME rms 10 max 32 freq +8163 +/- 38 delay 1762 +/- 16
May 09 13:21:26 aerial-rf-gb-gnb phc2sys[6963]: [15562.463] CLOCK_REALTIME rms 9 max 23 freq +8162 +/- 17 delay 1761 +/- 16
Disable NTP
Use these commands to turn off NTP:
sudo timedatectl set-ntp false
timedatectl
Local time: Thu 2022-02-03 22:30:58 UTC
Universal time: Thu 2022-02-03 22:30:58 UTC
RTC time: Thu 2022-02-03 22:30:58
Time zone: Etc/UTC (UTC, +0000)
System clock synchronized: no
NTP service: inactive
RTC in local TZ: no
Verify System Clock Synchronization
Make NTP inactive and synchronize the system clock:
timedatectl
Local time: Thu 2022-02-03 22:30:58 UTC
Universal time: Thu 2022-02-03 22:30:58 UTC
RTC time: Thu 2022-02-03 22:30:58
Time zone: Etc/UTC (UTC, +0000)
System clock synchronized: yes
NTP service: inactive
RTC in local TZ: no
There is a tutorial video for setting up the Foxconn ORU.
Foxconn RPQN-7801E |
Connections and Settings |
|---|---|
|
Connections:
GrandMaster settings (Qulsar):
|
Configure VLAN and IP Address on the gNB Server
Add these commands to the server startup script (
/etc/rc.local) so they are automatically run on reboot.Configure these settings on the fronthaul port.
You must use IP addresses that do not match those in the example below:
sudo ip link add link ens6f0 name ens6f0.2 type vlan id 2
sudo ip addr add 169.254.1.103/24 dev ens6f0.2
sudo ip link set up ens6f0.2
O-RU M-Plane Setup
Add the following to the bottom of
/etc/profileand comment out the line withset_qse.shif it already exists. Set the interface initially toeth0for firmware version 1, and toqse-ethafter upgrading to firmware version 2 or greater.interface=eth0 vlanid=2 ipLastOctet=20 ip link add link ${interface} name ${interface}.$vlanid type vlan id $vlanid ip addr flush dev ${interface} ip addr add 169.254.0.0/24 dev ${interface} ip addr add 169.254.1.${ipLastOctet}/24 dev ${interface}.$vlanid ip link set up ${interface}.$vlanid
Reboot the O-RU using the command
./reboot.shand check the network configuration:# ip r 169.254.1.0/24 dev eth0.2 src 169.254.1.20
Update O-RU Configuration
Update the O-RU configuration in
/home/root/sdcard/RRHconfig_xran.xml.root@arria10:~/test# grep -v '<!-' ../sdcard/RRHconfig_xran.xml RRH_DST_MAC_ADDR = 08:c0:eb:71:e7:d4 # To match fronthaul interface of DU RRH_SRC_MAC_ADDR = 6C:AD:AD:00:04:6C # To match qse-eth of RU RRH_EN_EAXC_ID = 0 RRH_EAXC_ID_TYPE1 = 0x0, 0x1, 0x2, 0x3 RRH_EAXC_ID_TYPE3 = 0x8, 0x9, 0xA, 0xB RRH_EN_SPC = 1 RRH_RRH_LTE_OR_NR = 1 RRH_TRX_EN_BIT_MASK = 0x0f RRH_RF_EN_BIT_MASK = 0x0f RRH_CMPR_HDR_PRESENT = 0 RRH_CMPR_TYPE = 1, 1 RRH_CMPR_BIT_LENGTH = 9, 9 RRH_UL_INIT_SYM_ID = 0 RRH_TX_TRUNC_BITS = 4 RRH_RX_TRUNC_BITS = 4 RRH_MAX_PRB = 273 RRH_C_PLANE_VLAN_TAG = 0x0002 #To match vlan id set in cuphycontroller yaml file RRH_U_PLANE_VLAN_TAG = 0x0002 #To match vlan id set in cuphycontroller yaml file RRH_SLOT_TICKS_IN_SEC = 2000 RRH_SLOT_PERIOD_IN_SAMPLE = 61440 RRH_LO_FREQUENCY_KHZ = 3750000, 0 RRH_TX_POWER = 24, 24 RRH_TX_ATTENUATION = 12.0, 12.0, 12.0, 12.0 RRH_RX_ATTENUATION = 0.0, 0.0, 0.0, 0.0 RRH_BB_GENERAL_CTRL = 0x0, 0x0, 0x0, 0x0 RRH_RF_GENERAL_CTRL = 0x3, 0x1, 0x0, 0x0 RRH_PTPV2_GRAND_MASTER_MODE = 3 RRH_PTPV2_JITTER_LEVEL = 0 RRH_PTPV2_VLAN_ID = 0 RRH_PTPV2_IP_MODE = 4 RRH_PTPV2_GRAND_MASTER_IP = 192.167.27.150 RRH_PTPV2_SUB_DOMAIN_NUM = 24 RRH_PTPV2_ACCEPTED_CLOCK_CLASS = 135 RRH_TRACE_PERIOD = 10
NoteIn Foxconn firmware version 3.1.15 and later, the configuration file is located in
/home/root/test.NoteThe above configuration was taken from an ORU running firmware 2.6.9; for the latest configuration, see the full stack ARC-OTA resources.
Reboot O-RU.
cd /home/root/test/ ./reboot
Run the following to enable the configuration:
cd /home/root/test/ ./init_rrh_config_enable_cuplane
At this point, the console becomes unresponsive and fills with prints related to PTP, AFE initialization, and packet counters.