DOCA Sensitive Information Detection Example

DOCA Support is in early access and may only be used via the Morpheus DOCA Container found in NGC. Please speak to your NVIDIA Morpheus contact for more information.

The container must be run in privileged mode and mount in hugepages as configured according to the DOCA GPUNetIO documentation.

Copy
Copied!
            

docker run -v /dev/hugepages:/dev/hugepages --privileged --rm -ti --runtime=nvidia --net=host --gpus=all --cap-add=sys_nice ${MORPHEUS_DOCA_IMAGE} bash

The DOCA example requires specifying the PCIe Address of both the GPU and NIC explicitly. Determining the correct GPU and NIC PCIe Addresses is non-trivial and requires coordinating with those who have configured the physical hardware and firmware according to the DOCA GPUNetIO documentation, but the following commands can help find a NIC and GPU situation on the same NUMA node.

Copy
Copied!
            

$ lspci -tv | grep -E "NVIDIA|ella|(^\+)|(^\-)" -+-[0000:ff]-+-00.0 Intel Corporation Device 344c | \-02.0-[ca-cf]----00.0-[cb-cf]--+-00.0-[cc]--+-00.0 Mellanox Technologies MT42822 BlueField-2 integrated ConnectX-6 Dx network controller | | +-00.1 Mellanox Technologies MT42822 BlueField-2 integrated ConnectX-6 Dx network controller | | \-00.2 Mellanox Technologies MT42822 BlueField-2 SoC Management Interface | \-01.0-[cd-cf]----00.0-[ce-cf]----08.0-[cf]----00.0 NVIDIA Corporation Device 20b9 | \-02.0-[b1]--+-00.0 Mellanox Technologies MT42822 BlueField-2 integrated ConnectX-6 Dx network controller | +-00.1 Mellanox Technologies MT42822 BlueField-2 integrated ConnectX-6 Dx network controller | \-00.2 Mellanox Technologies MT42822 BlueField-2 SoC Management Interface

From the result we can assemble the PCIe addresses of the nearest GPU and NIC. But it will be easier to cross-reference them with the explicit PCIe addresses from these commands:

Copy
Copied!
            

$ lspci | grep ella b1:00.0 Ethernet controller: Mellanox Technologies MT42822 BlueField-2 integrated ConnectX-6 Dx network controller (rev 01) b1:00.1 Ethernet controller: Mellanox Technologies MT42822 BlueField-2 integrated ConnectX-6 Dx network controller (rev 01) b1:00.2 DMA controller: Mellanox Technologies MT42822 BlueField-2 SoC Management Interface (rev 01) ca:00.0 PCI bridge: Mellanox Technologies MT42822 Family [BlueField-2 SoC PCIe Bridge] (rev 01) cb:00.0 PCI bridge: Mellanox Technologies MT42822 Family [BlueField-2 SoC PCIe Bridge] (rev 01) cb:01.0 PCI bridge: Mellanox Technologies MT42822 Family [BlueField-2 SoC PCIe Bridge] (rev 01) cc:00.0 Ethernet controller: Mellanox Technologies MT42822 BlueField-2 integrated ConnectX-6 Dx network controller (rev 01) cc:00.1 Ethernet controller: Mellanox Technologies MT42822 BlueField-2 integrated ConnectX-6 Dx network controller (rev 01) cc:00.2 DMA controller: Mellanox Technologies MT42822 BlueField-2 SoC Management Interface (rev 01) cd:00.0 PCI bridge: Mellanox Technologies MT42822 Family [BlueField-2 SoC PCIe Bridge] (rev 01) ce:08.0 PCI bridge: Mellanox Technologies MT42822 Family [BlueField-2 SoC PCIe Bridge] (rev 01)

Copy
Copied!
            

$ lspci | grep NVIDIA cf:00.0 3D controller: NVIDIA Corporation Device 20b9 (rev a1)

We can see the GPU’s PCIe address is cf:00.0, and we can infer from the above commands that the nearest ConnectX-6 NIC’s PCIe address is cc:00.*. In this case, we have port 1 physically connected to the network, so we use PCIe Address cc:00.1.

The DOCA example is similar to the Sensitive Information Detection (SID) example in that it uses the sid-minibert model in conjunction with the TritonInferenceStage to detect sensitive information. The difference is that the sensitive information we will be detecting is obtained from a live TCP packet stream provided by a DocaSourceStage.

Prior to running the example, the rdma-core conda package needs to be removed by force from the conda environment, otherwise the environment is incompatible with the DOCA-provided packages.

Copy
Copied!
            

conda remove --force rdma-core

To run the example from the Morpheus root directory and capture all TCP network traffic from the given NIC, use the following command and replace the nic_addr and gpu_addr arguments with your NIC and GPU PCIe addresses.

Copy
Copied!
            

# python examples/doca/run.py --nic_addr cc:00.1 --gpu_addr cf:00.0

Copy
Copied!
            

====Registering Pipeline==== ====Building Pipeline==== DOCA GPUNetIO rate: 0 pkts [00:00, ? pkt====Building Pipeline Complete!==== Deserialize rate: 0 pkts [00:00, ? pktsStarting! Time: 1689110835.1106102 EAL: Detected CPU lcores: 72, ? pkts/s] EAL: Detected NUMA nodes: 200, ? pkts/s] EAL: Detected shared linkage of DPDKs/s] EAL: Multi-process socket /var/run/dpdk/rte/mp_socket EAL: Selected IOVA mode 'PA' EAL: VFIO support initialized TELEMETRY: No legacy callbacks, legacy socket not created EAL: Probe PCI driver: mlx5_pci (15b3:a2d6) device: 0000:cc:00.1 (socket 1) EAL: Probe PCI driver: gpu_cuda (10de:20b9) device: 0000:cf:00.0 (socket 1) DOCA GPUNetIO rate: 0 pkts [00:03, ? pkts/s]====Registering Pipeline Complete!==== ====Starting Pipeline====[00:02, ? pkts/s] ====Pipeline Started====0:02, ? pkts/s] ====Building Segment: linear_segment_0==== Added source: <from-doca-0; DocaSourceStage(nic_pci_address=cc:00.1, gpu_pci_address=cf:00.0)> └─> morpheus.MessageMeta Added stage: <monitor-1; MonitorStage(description=DOCA GPUNetIO rate, smoothing=0.05, unit=pkts, delayed_start=False, determine_count_fn=None, log_level=LogLevels.INFO)> └─ morpheus.MessageMeta -> morpheus.MessageMeta Added stage: <deserialize-2; DeserializeStage(ensure_sliceable_index=True)> └─ morpheus.MessageMeta -> morpheus.MultiMessage Added stage: <monitor-3; MonitorStage(description=Deserialize rate, smoothing=0.05, unit=pkts, delayed_start=False, determine_count_fn=None, log_level=LogLevels.INFO)> └─ morpheus.MultiMessage -> morpheus.MultiMessage Added stage: <preprocess-nlp-4; PreprocessNLPStage(vocab_hash_file=/workspace/models/training-tuning-scripts/sid-models/resources/bert-base-uncased-hash.txt, truncation=True, do_lower_case=True, add_special_tokens=False, stride=-1, column=data)> └─ morpheus.MultiMessage -> morpheus.MultiInferenceNLPMessage Added stage: <monitor-5; MonitorStage(description=Tokenize rate, smoothing=0.05, unit=pkts, delayed_start=False, determine_count_fn=None, log_level=LogLevels.INFO)> └─ morpheus.MultiInferenceNLPMessage -> morpheus.MultiInferenceNLPMessage Added stage: <inference-6; TritonInferenceStage(model_name=sid-minibert-onnx, server_url=localhost:8000, force_convert_inputs=True, use_shared_memory=True)> └─ morpheus.MultiInferenceNLPMessage -> morpheus.MultiResponseMessage Added stage: <monitor-7; MonitorStage(description=Inference rate, smoothing=0.05, unit=pkts, delayed_start=False, determine_count_fn=None, log_level=LogLevels.INFO)> └─ morpheus.MultiResponseMessage -> morpheus.MultiResponseMessage Added stage: <add-class-8; AddClassificationsStage(labels=None, prefix=, probs_type=TypeId.BOOL8, threshold=0.5)> └─ morpheus.MultiResponseMessage -> morpheus.MultiResponseMessage Added stage: <monitor-9; MonitorStage(description=AddClass rate, smoothing=0.05, unit=pkts, delayed_start=False, determine_count_fn=None, log_level=LogLevels.INFO)> └─ morpheus.MultiResponseMessage -> morpheus.MultiResponseMessage ====Building Segment Complete!==== Stopping pipeline. Please wait... Press Ctrl+C again to kill. DOCA GPUNetIO rate: 0 pkts [00:09, ? pkts/s] Deserialize rate: 0 pkts [00:09, ? pkts/s] Tokenize rate: 0 pkts [00:09, ? pkts/s] Inference rate: 0 pkts [00:09, ? pkts/s] AddClass rate: 0 pkts [00:09, ? pkts/s]

The output can be found in doca_output.csv

Previous ABP Detection Example Using Morpheus
Next GNN Fraud Detection Pipeline
© Copyright 2023, NVIDIA. Last updated on Feb 2, 2024.