Prerequisites | NVIDIA Financial Fraud NIM Documentation

Hardware Requirements

Requirement	Minimum	Recommended
GPUs	1× A6000, A100, H100, or newer	2+ NVIDIA GPUs (A100 40GB or equivalent)
GPU VRAM	32 GB per GPU	40–80 GB per GPU
System RAM	16 GB	256 GB+
Disk space (for outputs)	40 GB	100 GB+ for large graphs

Multi-GPU is required for the default config (gpu: multi, num_gpus: 2). Single-GPU runs are supported but require gpu: single in the config or -e NPROC=1 at the Docker command line.

Every docker run invocation requires the following ulimit flags for stable CUDA/NCCL operation. For example:

--ulimit memlock=-1 --ulimit stack=67108864

Software Requirements

Component	Version
Docker Engine	26+
NVIDIA Container Toolkit (`nvidia-container-toolkit`)	1.15.0+
NVIDIA Driver	560.28.03+
CUDA (host driver)	12.6+
Operating System	Ubuntu 20.04+

Verify that Docker can access all GPUs on the host:

docker run --rm --gpus all nvidia/cuda:12.6.0-base-ubuntu22.04 nvidia-smi

The output should list all available GPUs.

Technology Stack

Training

Component	Technology	Role
GNN framework	cuGraph-PyG (NVIDIA)	Distributed heterogeneous GNN with GPU-accelerated sampling
Graph storage	WholeGraph (NVIDIA)	Distributed graph feature store; nodes partitioned across GPUs with transparent remote access
GNN training	PyTorch DDP + torchrun	Gradient synchronisation across processes using NCCL
GNN model	GNNEncoder	Heterogeneous multi-layer convolution (SAGE / GAT / Transformer / General)
XGBoost training	XGBoost + RabitTracker	Distributed gradient boosted trees; data sharded across GPUs
Orchestration	`torchrun`	Spawns one process per GPU; sets `LOCAL_RANK`, `RANK`, `WORLD_SIZE`

Inference

Component	Technology	Role
Serving runtime	NVIDIA Triton Inference Server	Production model server with ensemble support
GNN inference	Triton Python backend (torch + torch_geometric)	Runs the saved GNN model on CPU/GPU tensors
XGBoost inference	Triton FIL backend (RAPIDS FIL)	Hardware-accelerated forest inference on GPU
Explainability	Triton Python backend (XGBoost + Captum)	Shapley Value Sampling feature attributions
LLM narrative	Triton Python backend (OpenAI SDK)	Calls OpenAI-compatible API; thread pool for parallel requests

NGC Authentication

The training container is hosted on NVIDIA NGC and requires authentication.

Create a free account at https://ngc.nvidia.com.
Generate an API key from Profile → Setup → Generate API Key.
Log in to the NGC registry:

docker login nvcr.io
# Username: $oauthtoken
# Password: <your NGC API key>

The username is always the literal string $oauthtoken — this is not an environment variable.

Inference Client Requirements

The inference clients (client.py, client_np.py, client_saved_emb.py, and client_production.py) run on the host (outside Docker) and communicate with Triton over gRPC:

pip install "tritonclient[grpc]" pandas numpy torch

For client_production.py --serve mode (persistent HTTP service), also install:

pip install fastapi uvicorn

tritonclient[grpc] must match the Triton server version. For Triton 26.04, use the version pinned to that release, or install the latest and verify the gRPC handshake succeeds on first use.

LLM Explainability (Optional)

The LLM explanation feature requires an API key from any OpenAI-compatible provider:

Provider	`LLM_BASE_URL`	Example `LLM_MODEL`
NVIDIA NIM (using `NVIDIA_API_KEY`)	(not needed — auto-configured to `https://inference-api.nvidia.com`)	`nvidia/meta/llama-3.2-1b-instruct`
NVIDIA NIM (using `LLM_API_KEY`)	`https://inference-api.nvidia.com`	`nvidia/meta/llama-3.1-70b-instruct`
OpenAI	`https://api.openai.com/v1`	`gpt-4o`

The server starts and scores correctly without an LLM key — only the fraud_pipeline_explained ensemble is affected.