{ "cells": [ { "cell_type": "markdown", "id": "b543072d-6c02-4664-9dfb-adb8809962be", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "library/cudf", "library/kafka", "library/morpheus" ] }, "source": [ "# Deploying End-to-End Kafka Streaming SI Detection Pipeline with cuDF, Morpheus, and Triton on EKS\n", "\n", "_June, 2025_\n", "\n", "In this example workflow, we demonstrate how to deploy an NVIDIA GPU-accelerated streaming\n", "pipeline for Sensitive Information (SI) detection using [Morpheus](https://docs.nvidia.com/morpheus/), [cuDF](https://docs.rapids.ai/api/cudf/stable/), and [Triton\n", "Inference Server](https://docs.nvidia.com/deeplearning/triton-inference-server/user-guide/docs/) on [Amazon EKS](https://docs.aws.amazon.com/eks/latest/userguide/what-is-eks.html).\n", "\n", "We build upon the existing Morpheus\n", "[NLP SI Detection example](https://docs.nvidia.com/morpheus/examples/nlp_si_detection/readme.html)\n", "and enhance it to showcase a production-style end-to-end deployment integrated with [Apache Kafka](https://kafka.apache.org/) for data streaming.\n", "\n", "The pipeline, under `pipeline-dockerfile/run_pipeline_kafka.py` in the side panel, includes the\n", "following components:\n", "\n", "- **Kafka Data Streaming Source Stage**: We introduce Apache Kafka for streaming data. A custom\n", " Kafka producer was created to continuously publish network data to a Kafka topic. Code under `producer-dockerfile/producer.py` in the side panel. \n", "\n", "- **cuDF Message Filtering Stage**: The data stream first flows through a message filtering stage\n", " that leverages `cuDF` to preprocess and filter messages based on custom logic. Code under `pipeline-dockerfile/message_filter_stage.py` in the side panel. \n", "\n", "- **SI Detection with Morpheus and Triton**: The filtered data passes through multiple stages to\n", " prepare data for inference, perform the inference and classify the data. We use Morpheus' provided NLP\n", " SI Detection model to identify potentially sensitive information in the network packet data. For more\n", " details on the model check the original example on the [Morpheus documentation](https://docs.nvidia.com/morpheus/examples/nlp_si_detection/readme.html#background)\n", "\n", "- **cuDF Network Traffic Analysis Stage**: We incorporate an additional analysis stage using `cuDF` to perform some network traffic analytics for enriched context and anomaly detection. Code under `pipeline-dockerfile/network_traffic_analyzer_stage.py` in the side panel. \n", "\n", "- **Kafka Output Sink**: Finally, the processed and enriched data, with SI detection results\n", " and traffic insights, is published to a downstream Kafka topic for further processing, alerting,\n", " or storage.\n", "\n", "The entire pipeline is containerized and deployed on **Amazon EKS**, leveraging Kubernetes\n", "for orchestration, scalability, and resiliency in a cloud-native environment." ] }, { "cell_type": "markdown", "id": "986fd0f9-5ccd-44b1-acbf-5ba437c0a18d", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "## Deployment Components\n", "\n", "The pipeline is deployed on Amazon EKS using several Kubernetes manifests:\n", "\n", "### Kafka Deployment (`k8s/kafka`)\n", "\n", "The Kafka cluster is deployed using the [Strimzi Operator](https://strimzi.io/), which simplifies Kafka deployment and management on Kubernetes. See instructions on section [Deploying on EKS](deploying-on-eks)\n", "\n", "The deployment configuration includes:\n", "\n", "- Kafka cluster setup `kafka-single-node.yaml`.\n", "\n", " A modification of the file [https://strimzi.io/examples/latest/kafka/kafka-single-node.yaml](https://strimzi.io/examples/latest/kafka/kafka-single-node.yaml) where we modify:\n", " - Cluster name to `kafka-cluster`.\n", " - Modify the volume to use `type: ephemeral` and use `sizeLimit: 5Gi` (instead of `size: 100Gi` that corresponded to `type: persistent-claim`).\n", "- Kafka topics setup.\n", "- [Kafka UI](https://github.com/provectus/kafka-ui).\n", "\n", "### Kafka Producer Deployment (`k8s/kafka-producer`)\n", "\n", "The Kafka producer is deployed as a separate Pod using the `kafka-producer.yaml` manifest. It continuously generates and publishes network data to the Kafka topic. \n", "\n", "- Uses `kafka-python` for message production.\n", "- Contains the producer script for generating network data.\n", "\n", "This producer script is containerized using a custom Docker image that is already built and public. But if you want to build and push this image yourself, you need:\n", "\n", "- Log in to docker `docker login`.\n", "- Download the `scripts` directory from the sidebar.\n", "- Navigate to the `producer-dockerfile` directory and run.\n", "\n", " ```\n", " docker build -t /kafka-producer-image:latest .\n", " ```\n", "- Push image to docker.\n", "- Replace the image link in the `kafka-producer/kafka-producer.yaml`.\n", "\n", "### Triton-Morpheus Deployment (`k8s/triton`)\n", "\n", "The inference server is deployed using the NVIDIA Morpheus-Triton Inference Server docker image\n", "`nvcr.io/nvidia/morpheus/morpheus-tritonserver-models:25.02`.\n", "\n", "### Morpheus Pipeline Deployment (`k8s/morpheus-pipeline`)\n", "\n", "The core processing pipeline is deployed as a separate Pod that, uses an image that uses a custom image we created for this purpose.\n", "\n", "- Runs the Morpheus nightly conda build.\n", "- Contains all pipeline and stage scripts `scripts/pipeline-dockerfile/*.py`.\n", "- Processes the streaming data through the various stages.\n", "\n", "This image is already built and public. But if you want to build and push this image yourself, you need:\n", "\n", "- Log in to docker `docker login`.\n", "- Download the `scripts` directory from the sidebar.\n", "- Navigate to the `pipeline-dockerfile` directory and run.\n", " ```\n", " docker build -t /morpheus-pipeline-image:latest .\n", " ```\n", "- Push image to docker \n", "- Replace the image link in the `morpheus-pipeline/morpheus-pipeline-deployment.yaml` " ] }, { "cell_type": "markdown", "id": "06998d6b-5bac-415e-ae02-deed8b5a348b", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "(deploying-on-eks)=\n", "\n", "## Deploying on EKS\n", "\n", "### Prerequisites\n", "\n", "You need to have the [`aws` CLI tool](https://aws.amazon.com/cli/) and [`eksctl` CLI tool](https://docs.aws.amazon.com/eks/latest/userguide/eksctl.html) installed along with [`kubectl`](https://kubernetes.io/docs/tasks/tools/) for managing Kubernetes.\n", "\n", "### Launch GPU enabled EKS cluster\n", "\n", "We launch a GPU enabled EKS cluster with `eksctl`.\n", "\n", "```{note}\n", "1. You will need to create or import a public SSH key to be able to execute the following command.\n", "In your aws console under `EC2` in the side panel under **Network & Security** > **Key Pairs**, you can create a key pair or import (see \"Actions\" dropdown) one you've created locally.\n", "\n", "2. If you are not using your default AWS profile, add `--profile ` to the following command.\n", "```\n", "\n", "```console\n", "$ eksctl create cluster morpheus-rapids \\\n", " --version 1.32 \\\n", " --nodes 2 \\\n", " --node-type=g4dn.xlarge \\\n", " --timeout=40m \\\n", " --ssh-access \\\n", " --ssh-public-key \\ # Name assigned during creation of your key in aws console\\\n", " --region us-east-1 \\\n", " --zones=us-east-1c,us-east-1b,us-east-1d \\\n", " --auto-kubeconfig\n", "```\n", "\n", "To access the cluster we need to pull down the credentials. Add `--profile ` if you are not using the default profile.\n", "\n", "```console\n", "$ aws eks --region us-east-1 update-kubeconfig --name morpheus-rapids\n", "```\n", "\n", "### Deploy the Strimzi Operator\n", "\n", "[Strimzi](https://strimzi.io/) is an open-source project that provides a way to run Apache Kafka on Kubernetes. It simplifies the deployment and management of Kafka clusters by providing a Kubernetes operators that handle the complex tasks of setting up and maintaining Kafka.\n", "\n", "We use `kubectl` to deploy the operator. In our case we are deploying everything on the default\n", "namespace, and the entire pipeline is design for that.\n", "\n", "```console\n", "$ kubectl create -f 'https://strimzi.io/install/latest?namespace=default'\n", "```\n", "\n", "### Deploy the pipeline\n", "\n", "Get all the files in the `k8s` directory, you should be able to download them from the sidebar, or you can find them in\n", "[https://github.com/rapidsai/deployment/source/examples/rapids-morpheus-pipeline/k8s](https://github.com/rapidsai/deployment/source/examples/rapids-morpheus-pipeline/k8s)\n", "\n", "```console\n", "$ kubectl apply -f k8s --recursive\n", "```\n", "\n", "This will take around 15 minutes to get all the Pods up and running, you will see for a while that the the `morpheus-pipeline` Pod fails and try to reconcile. This happens because the triton inference Pod takes a while to get up and running.\n", "\n", "### Kafka UI: checking the pipeline results\n", "\n", "Once all the Pods are running, you can check the input topic and the results topic in the Kafka UI by forwarding the port to your local host\n", "\n", "```console\n", "$ kubectl port-forward svc/kafka-ui 8080:80\n", "```\n", "\n", "In your browser go to `http://localhost:8080/` and you will see:\n", "\n", "![Kafka UI demo](../../images/morpheus-pipeline-KafkaUI_9MB.gif)\n" ] }, { "cell_type": "markdown", "id": "c906ee22-606c-4429-8b88-b1b43f8550a3", "metadata": {}, "source": [ "## Conclusion\n", "\n", "This example demonstrates how to build and deploy a production-like, GPU-accelerated streaming pipeline for sensitive information detection using NVIDIA RAPIDS, Morpheus, and Triton Inference Server on Amazon EKS while integrating Apache Kafka for data streaming capabilities. This architecture showcases how modern streaming technologies combine with GPU-accelerated inference to create efficient, production-grade solutions for sensitive information detection." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.8" } }, "nbformat": 4, "nbformat_minor": 5 }