Using pyAerial for decoding PUSCH transmissions from multiple cells using Aerial Data Lake data#
This example shows how to use the pyAerial bindings to run cuPHY GPU accelerated PUSCH decoding for 5G NR PUSCH. The 5G NR PUSCH data is read from an example over the air captured PUSCH dataset collected from two cells and stored using Aerial Data Lake. A complete PUSCH receiver using individual separate Python function calls to individual PUSCH receiver components is used so that channel estimates can be shown.
In this setup there are two cells:
Cell 51 is an active cell being used with the OAI L2+ using slot pattern DDDSU and commercial UEs.
Cell 41 is a passive “listener” cell using the same slot pattern as cell 51 but is being driven by testMAC to request IQ samples for every uplink slot.
There are multiple UEs connected to cell 51, as is shown in the nUEs field of the fapi table. Both cells receive the same PUSCH data, and we attempt decoding from both cells. Note: As the UEs are connected to cell 51, it is expected that the decoding does not succeed in all cases from cell 41.
The first two plots show the power in all of the resource elements for the given slot on both cells. The plots after that show just the resource elements scheduled for a given UE (RNTI) across both cells, as well pre- and post- equalized samples, then channel estimates, for IQ samples from each cell, followed by text indicating whether the transmission decoded successfully.
Note: This example requires that the clickhouse server is running and that the example data has been stored in the database. Refer to the Aerial Data Lake documentation on how to do this.
Imports#
[1]:
import math
import os
os.environ["CUDA_VISIBLE_DEVICES"] = "0"
import numpy as np
import pandas as pd
from IPython.display import Markdown
from IPython.display import display
# Connecting to clickhouse on remote server
import clickhouse_connect
# Plotting with Matplotlib.
import matplotlib.pyplot as plt
from matplotlib import dates as mdates
# pyAerial imports
from aerial.phy5g.config import PuschConfig
from aerial.phy5g.config import PuschUeConfig
from aerial.phy5g.algorithms import ChannelEstimator
from aerial.phy5g.algorithms import ChannelEqualizer
from aerial.phy5g.algorithms import NoiseIntfEstimator
from aerial.phy5g.ldpc import LdpcDeRateMatch
from aerial.phy5g.ldpc import LdpcDecoder
from aerial.phy5g.ldpc import CrcChecker
from aerial.util.cuda import CudaStream
from aerial.util.fapi import dmrs_fapi_to_bit_array
# Hide log10(10) warning
_ = np.seterr(divide='ignore', invalid='ignore')
pd.set_option('display.max_rows', 500)
pd.set_option('display.max_columns', 500)
pd.set_option('display.width', 1000)
np.set_printoptions(threshold=100) # Control the number of elements to display
np.set_printoptions(edgeitems=100) # Control the number of edge items to display
np.set_printoptions(linewidth=200) # Control the width of the display
plt.rcParams['figure.figsize'] = [10, 4]
Create the PUSCH pipelines#
This is a PUSCH receiver pipeline made up of separately called pyAerial PUSCH receiver components.
[2]:
# Whether to plot intermediate results within the PUSCH pipeline, such as channel estimates and equalized symbols.
plot_figures = True
num_ues = 1
num_tx_ant = 2 # UE antennas
num_rx_ant = 4 # gNB antennas
enable_pusch_tdi = 0 # Enable time interpolation for equalizer coefficients
eq_coeff_algo = 1 # Equalizer algorithm
# The PUSCH receiver chain built from separately called pyAerial Python components is defined here.
class PuschRxSeparate:
"""PUSCH receiver class.
This class encapsulates the whole PUSCH receiver chain built using
pyAerial components.
"""
def __init__(self,
num_rx_ant,
enable_pusch_tdi,
eq_coeff_algo,
plot_figures):
"""Initialize the PUSCH receiver."""
self._stream = CudaStream()
# Build the components of the receiver.
self.channel_estimator = ChannelEstimator(
num_rx_ant=num_rx_ant,
cuda_stream=self._stream)
self.channel_equalizer = ChannelEqualizer(
num_rx_ant=num_rx_ant,
enable_pusch_tdi=enable_pusch_tdi,
eq_coeff_algo=eq_coeff_algo,
cuda_stream=self._stream)
self.noise_intf_estimator = NoiseIntfEstimator(
num_rx_ant=num_rx_ant,
eq_coeff_algo=eq_coeff_algo,
cuda_stream=self._stream)
self.derate_match = LdpcDeRateMatch(
enable_scrambling=True,
cuda_stream=self._stream)
self.decoder = LdpcDecoder(cuda_stream=self._stream)
self.crc_checker = CrcChecker(cuda_stream=self._stream)
# Whether to plot the intermediate results.
self.plot_figures = plot_figures
def run(
self,
rx_slot,
slot,
pusch_configs,
cell_id
):
"""Run the receiver."""
# Channel estimation.
ch_est = self.channel_estimator.estimate(
rx_slot=rx_slot,
slot=slot,
pusch_configs=pusch_configs
)
# Noise and interference estimation.
lw_inv, noise_var_pre_eq = self.noise_intf_estimator.estimate(
rx_slot=rx_slot,
channel_est=ch_est,
slot=slot,
pusch_configs=pusch_configs
)
# Channel equalization and soft demapping. The first return value are the LLRs,
# second are the equalized symbols. We only want the LLRs now.
llrs, sym = self.channel_equalizer.equalize(
rx_slot=rx_slot,
channel_est=ch_est,
lw_inv=lw_inv,
noise_var_pre_eq=noise_var_pre_eq,
pusch_configs=pusch_configs
)
if self.plot_figures:
fig, axs = plt.subplots(1, 4)
for ant in range(4):
axs[ant].imshow(10*np.log10(np.abs(rx_slot[:, :, ant] ** 2)), aspect='auto')
axs[ant].set_ylim([(pusch_record.rbStart + pusch_record.rbSize) * 12, pusch_record.rbStart * 12])
axs[ant].set_title('Ant ' + str(ant))
axs[ant].set(xlabel='Symbol', ylabel='Resource Element')
axs[ant].label_outer()
fig.suptitle('Power in PUSCH REs cell {} for RNTI {}'.format(cell_id, pusch_record.rnti))
fig, axs = plt.subplots(1, 2)
axs[0].scatter(rx_slot.reshape(-1).real, rx_slot.reshape(-1).imag)
axs[0].set_title("Pre-Equalized samples")
axs[0].set_aspect('equal')
axs[1].scatter(np.array(sym).reshape(-1).real, np.array(sym).reshape(-1).imag)
axs[1].set_title("Post-Equalized samples")
axs[1].set_aspect('equal')
fig, axs = plt.subplots(1)
axs.set_title("Channel estimates from the PUSCH pipeline")
for ant in range(4):
axs.plot(np.abs(ch_est[0][ant, 0, :, 0]))
axs.legend(["Rx antenna 0, estimate",
"Rx antenna 1, estimate",
"Rx antenna 2, estimate",
"Rx antenna 3, estimate"])
axs.grid(True)
plt.show()
coded_blocks = self.derate_match.derate_match(
input_llrs=llrs,
pusch_configs=pusch_configs
)
code_blocks = self.decoder.decode(
input_llrs=coded_blocks,
pusch_configs=pusch_configs
)
decoded_tbs, _ = self.crc_checker.check_crc(
input_bits=code_blocks,
pusch_configs=pusch_configs
)
return decoded_tbs
pusch_rx_separate = PuschRxSeparate(
num_rx_ant=num_rx_ant,
enable_pusch_tdi=enable_pusch_tdi,
eq_coeff_algo=eq_coeff_algo,
plot_figures=plot_figures
)
Querying the database#
Below shows how to connect to the clickhouse database and querying the data from it.
[3]:
# Connect to the local database
import clickhouse_connect
client = clickhouse_connect.get_client(host='localhost')
# Pick a packet from the database,
pusch_records = client.query_df('select * from fapi order by TsTaiNs,rbStart limit 10')
Extract the PUSCH parameters and run the pipelines#
In this section we use the timestamp of the start of a slot to query the IQ sample database for both cells and demonstrate that the transmission can be decoded in the IQ samples of both cells.
[4]:
run_receiver = True
# Only show the full slot pattern the first time through:
shown_timestamp = -1
pusch_records = client.query_df('select * from fapi order by TsTaiNs,rbStart limit 8')
print(pusch_records[['TsTaiNs','SFN','Slot','CellId','nUEs','rnti','rbStart','rbSize','StartSymbolIndex','NrOfSymbols','nrOfLayers','TBSize','CQI']])
for index, pusch_record in pusch_records.iterrows():
query = f"""select TsTaiNs,CellId,fhData from fh where TsTaiNs == toDateTime64(\'{pusch_record.TsTaiNs.timestamp()}\', 9)"""
fh = client.query_df(query)
# Extract all the needed parameters from the PUSCH record and create the PuschConfig.
pusch_ue_config = PuschUeConfig(
scid=int(pusch_record.SCID),
layers=pusch_record.nrOfLayers,
dmrs_ports=pusch_record.dmrsPorts,
rnti=pusch_record.rnti,
data_scid=pusch_record.dataScramblingId,
mcs_table=pusch_record.mcsTable,
mcs_index=pusch_record.mcsIndex,
code_rate=pusch_record.targetCodeRate,
mod_order=pusch_record.qamModOrder,
tb_size=pusch_record.TBSize
)
slot = int(pusch_record.Slot)
tb_input = np.array(pusch_record.pduData)
# Note that this is a list. One UE group only in this case.
pusch_configs = [PuschConfig(
ue_configs=[pusch_ue_config],
num_dmrs_cdm_grps_no_data=pusch_record.numDmrsCdmGrpsNoData,
dmrs_scrm_id=pusch_record.ulDmrsScramblingId,
start_prb=pusch_record.rbStart,
num_prbs=pusch_record.rbSize,
dmrs_syms=dmrs_fapi_to_bit_array(int(pusch_record.ulDmrsSymbPos)),
dmrs_max_len=1,
dmrs_add_ln_pos=pusch_record.ulDmrsSymbPos.bit_count() - 1,
start_sym=pusch_record.StartSymbolIndex,
num_symbols=pusch_record.NrOfSymbols
)]
num_cells = fh.index.size
if shown_timestamp != pusch_record.TsTaiNs.timestamp():
shown_timestamp = pusch_record.TsTaiNs.timestamp()
display(Markdown("### SFN.Slot {}.{}, RNTI {} at time {}"
.format(pusch_record.SFN, pusch_record.Slot, pusch_record.rnti, pusch_record.TsTaiNs
)))
fig = plt.figure(constrained_layout=True)
if num_cells > 1:
outer = fig.subfigures(1, num_cells)
for cell_num, cell_plot in enumerate(outer.flat):
fh_samp = (np.array(fh['fhData'].iloc[cell_num], dtype=np.int16).view(np.float16)).astype(np.float32)
rx_slot = np.swapaxes(fh_samp.view(np.complex64).reshape(4, 14, 273 * 12), 2, 0)
axs = cell_plot.subplots(1, 4)
for ant, ax in enumerate(axs.flat):
ax.imshow(10 * np.log10(np.abs(rx_slot[:, :, ant] ** 2)), aspect='auto')
ax.set_title('Ant ' + str(ant))
ax.set(xlabel='Symbol', ylabel='Resource Element')
ax.label_outer()
cell_plot.suptitle('Power in RU Antennas Cell ' + str(fh['CellId'].iloc[cell_num]))
plt.show()
else:
cell_num = 0
fig, axs = plt.subplots(1, 4)
fh_samp = (np.array(fh['fhData'].iloc[cell_num], dtype=np.int16).view(np.float16)).astype(np.float32)
rx_slot = np.swapaxes(fh_samp.view(np.complex64).reshape(4, 14, 273 * 12), 2, 0)
for ant in range(4):
axs[ant].imshow(10 * np.log10(np.abs(rx_slot[:, :, ant] ** 2)), aspect='auto')
axs[ant].set_ylim([pusch_record.rbStart * 12, (pusch_record.rbStart+pusch_record.rbSize) * 12])
axs[ant].set_title('Ant ' + str(ant))
axs[ant].set(xlabel='Symbol', ylabel='Resource Element')
axs[ant].label_outer()
fig.suptitle('Power in RU Antennas')
fig, axs = plt.subplots(1,2)
axs[0].scatter(rx_slot.reshape(-1).real, rx_slot.reshape(-1).imag)
axs[0].set_title("Pre-Equalized samples")
axs[0].set_aspect('equal')
axs[1].scatter(np.array(sym).reshape(-1).real, np.array(sym).reshape(-1).imag)
axs[1].set_title("Post-Equalized samples")
axs[1].set_aspect('equal')
fig, axs = plt.subplots(1)
axs.set_title("Channel estimates from the PUSCH pipeline")
for ant in range(4):
axs.plot(np.abs(ch_est[0][ant, 0, :, 0]))
axs.legend(["Rx antenna 0, estimate",
"Rx antenna 1, estimate",
"Rx antenna 2, estimate",
"Rx antenna 3, estimate"])
axs.grid(True)
plt.show()
if run_receiver:
for cell_num in range(0, num_cells):
cell_id = fh['CellId'].iloc[cell_num]
fh_samp = (np.array(fh['fhData'].iloc[cell_num], dtype=np.int16).view(np.float16)).astype(np.float32)
rx_slot = np.swapaxes(fh_samp.view(np.complex64).reshape(4, 14, 273 * 12), 2, 0)
tbs = pusch_rx_separate.run(
rx_slot=rx_slot,
slot=slot,
pusch_configs=pusch_configs,
cell_id=cell_id
)
if np.array_equal(tbs[0][:tb_input.size], tb_input):
display(Markdown("**PUSCH decoding success** for SFN.Slot {}.{} RNTI {} originally on cell {} using IQ data from cell {} "
.format(pusch_record.SFN, pusch_record.Slot, pusch_record.rnti, pusch_record.CellId, cell_id)))
else:
display(Markdown("**PUSCH decoding failure** for SFN.Slot {}.{} RNTI {} originally on cell {} using IQ data from cell {} "
.format(pusch_record.SFN, pusch_record.Slot, pusch_record.rnti, pusch_record.CellId, cell_id)))
print("Output bytes:")
print(tbs[0][:tb_input.size])
print("Expected output:")
print(tb_input)
TsTaiNs SFN Slot CellId nUEs rnti rbStart rbSize StartSymbolIndex NrOfSymbols nrOfLayers TBSize CQI
0 2024-07-19 10:42:46.272 391 4 41 7 20000 0 8 0 10 1 63 -7.352562
1 2024-07-19 10:42:46.272 391 4 51 7 62290 0 5 0 13 1 185 31.755341
2 2024-07-19 10:42:46.272 391 4 51 7 53137 5 5 0 13 1 185 30.275444
3 2024-07-19 10:42:46.272 391 4 51 7 1624 10 5 0 13 1 185 31.334328
4 2024-07-19 10:42:46.272 391 4 51 7 47905 15 5 0 13 1 185 30.117304
5 2024-07-19 10:42:46.272 391 4 51 7 57375 20 5 0 13 1 185 29.439499
6 2024-07-19 10:42:46.272 391 4 51 7 20216 25 248 0 13 1 19985 25.331459
7 2024-07-19 10:42:47.292 493 4 41 6 20000 0 8 0 10 1 63 -7.845479
SFN.Slot 391.4, RNTI 20000 at time 2024-07-19 10:42:46.272000#
PUSCH decoding success for SFN.Slot 391.4 RNTI 20000 originally on cell 41 using IQ data from cell 41
PUSCH decoding success for SFN.Slot 391.4 RNTI 20000 originally on cell 41 using IQ data from cell 51
PUSCH decoding success for SFN.Slot 391.4 RNTI 62290 originally on cell 51 using IQ data from cell 41
PUSCH decoding success for SFN.Slot 391.4 RNTI 62290 originally on cell 51 using IQ data from cell 51
PUSCH decoding success for SFN.Slot 391.4 RNTI 53137 originally on cell 51 using IQ data from cell 41
PUSCH decoding success for SFN.Slot 391.4 RNTI 53137 originally on cell 51 using IQ data from cell 51
PUSCH decoding failure for SFN.Slot 391.4 RNTI 1624 originally on cell 51 using IQ data from cell 41
Output bytes:
[ 2 32 217 29 5 72 230 133 18 7 249 217 29 192 195 211 75 91 138 229 10 7 106 121 229 69 70 88 148 140 106 242 236 205 149 11 103 225 17 212 96 72 128 196 211 223 93 129 42
105 190 143 210 13 147 99 66 229 82 113 100 198 131 100 212 179 133 51 81 125 173 25 253 57 122 193 19 201 180 38 145 112 250 74 205 147 91 32 160 193 175 111 193 82 178 204 90 0
212 53 45 107 195 85 225 208 202 11 84 211 36 217 30 144 187 18 198 75 137 227 186 151 55 252 186 113 95 89 183 132 145 63 138 125 202 148 178 113 98 112 3 126 87 31 114 115 75
161 51 215 185 137 116 136 134 1 57 218 125 214 80 59 132 79 104 77 219 137 213 217 33 13 55 7 65 251 239 146 219 35 250 17 0 14 60]
Expected output:
[61 0 57 63 51 63 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33
33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33
33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33]
PUSCH decoding success for SFN.Slot 391.4 RNTI 1624 originally on cell 51 using IQ data from cell 51
PUSCH decoding failure for SFN.Slot 391.4 RNTI 47905 originally on cell 51 using IQ data from cell 41
Output bytes:
[191 131 42 160 120 171 5 122 183 48 200 46 220 202 78 207 17 8 250 235 190 59 64 14 105 130 35 209 70 51 14 157 223 29 184 127 228 88 7 175 67 103 113 194 28 97 168 251 19
207 117 170 102 35 17 226 165 243 249 218 239 31 57 8 28 244 146 107 2 89 180 202 164 70 140 134 78 10 138 214 228 136 36 179 154 136 94 187 167 204 70 127 33 96 176 116 127 246
24 81 21 227 134 231 188 8 151 72 1 250 48 44 178 192 113 141 11 85 172 118 182 217 8 182 99 26 161 187 160 23 160 8 86 167 64 86 54 217 151 205 208 5 95 15 156 7 96
252 211 183 113 214 194 111 75 79 70 114 139 39 203 54 25 91 224 135 42 253 47 105 118 123 122 41 27 104 151 76 176 133 55 95 18 179 105]
Expected output:
[61 0 57 63 51 63 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33
33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33
33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33]
PUSCH decoding success for SFN.Slot 391.4 RNTI 47905 originally on cell 51 using IQ data from cell 51
PUSCH decoding failure for SFN.Slot 391.4 RNTI 57375 originally on cell 51 using IQ data from cell 41
Output bytes:
[165 153 165 3 61 136 161 91 85 2 26 149 119 112 106 42 211 37 32 49 199 135 82 105 162 70 176 163 253 91 187 141 45 88 42 36 38 11 22 2 81 114 219 8 27 199 181 168 139
25 220 25 76 135 114 88 140 103 152 101 78 6 146 8 148 167 82 161 13 137 195 186 129 75 91 218 215 51 225 194 97 174 178 174 190 94 61 30 3 128 108 127 225 180 84 201 36 196
91 241 9 51 58 25 112 49 232 151 34 242 250 250 133 126 143 243 105 60 70 102 70 79 40 151 168 109 133 106 66 102 189 123 231 127 10 76 80 250 44 190 162 205 12 110 70 179 103
105 208 102 69 171 184 187 195 190 169 235 124 218 36 253 29 89 246 121 36 32 191 58 235 74 80 66 153 73 84 127 59 231 174 46 197 9 116]
Expected output:
[ 4 3 0 0 80 61 0 57 63 51 63 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33
33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33
33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33 33]
PUSCH decoding success for SFN.Slot 391.4 RNTI 57375 originally on cell 51 using IQ data from cell 51
PUSCH decoding failure for SFN.Slot 391.4 RNTI 20216 originally on cell 51 using IQ data from cell 41
Output bytes:
[129 110 210 96 4 183 250 172 186 215 150 23 114 98 223 24 93 251 134 128 106 6 130 139 237 194 193 24 217 180 109 76 218 57 36 16 215 138 143 180 158 192 60 49 73 133 249 120 147
105 189 181 202 4 92 174 16 1 253 128 138 195 58 243 144 215 209 185 156 167 72 40 151 29 127 220 51 83 120 114 250 224 52 71 13 101 186 80 229 239 202 85 96 202 213 96 46 87
166 142 ... 113 143 157 254 1 27 247 25 255 249 59 38 4 213 83 30 111 25 78 205 102 73 100 144 43 57 151 47 197 79 155 124 142 205 113 138 181 185 116 84 50 148 137 88 36 15
167 142 45 136 202 179 110 254 4 148 237 153 250 16 237 252 75 178 255 194 176 0 114 46 201 223 128 121 11 86 216 79 83 42 25 37 164 14 206 115 10 38 193 131 160 117 44 15 17
147 118 80 7 33]
Expected output:
[ 68 1 133 160 13 151 4 71 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52
53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51
52 53 ... 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54
55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53 54 55 56 57 48 49 50 51 52 53
54 55 56 57 48]
PUSCH decoding success for SFN.Slot 391.4 RNTI 20216 originally on cell 51 using IQ data from cell 51
SFN.Slot 493.4, RNTI 20000 at time 2024-07-19 10:42:47.292000#
PUSCH decoding success for SFN.Slot 493.4 RNTI 20000 originally on cell 41 using IQ data from cell 41
PUSCH decoding success for SFN.Slot 493.4 RNTI 20000 originally on cell 41 using IQ data from cell 51