Is this page helpful?

Quick Start — End-to-End in 5 Steps#

This section walks through the full EP (Edge Prediction) pipeline from a cold start to a running inference server. Each step depends on the previous one completing successfully. Detailed documentation for every step is in the sections below.

Prerequisites: Docker with --gpus all access, ≥ 2 NVIDIA GPUs, your graph data in the layout described in Data Layout.

Step 1 — Pull the container from NGC#

docker pull nvcr.io/nvidia/cugraph/financial-fraud-training:3.0.0

Requires an NGC account and API key. Log in first if not already authenticated:

docker login nvcr.io
# Username: $oauthtoken
# Password: <your NGC API key>

Step 2 — Train the Edge Prediction (EP) model#

docker run --rm --gpus all --shm-size=10g \
  --ulimit memlock=-1 --ulimit stack=67108864 \
  -v /your/graph/data:/data:ro \
  -v /your/ep/output:/workspace/output \
  -v /your/config_ep.yaml:/workspace/config.yaml:ro \
  nvcr.io/nvidia/cugraph/financial-fraud-training:3.0.0

Data layout: /your/graph/data must contain nodes/ and edges/ sub-directories following the conventions in Data Layout. Node and edge files can be CSV, Parquet, or ORC — the pipeline detects format by extension. Mask files (optional train/val/test splits) are always .npy.

Watch for the final metrics block in the log (AUC, AUCPR, F1, threshold). Training takes 30–90 minutes depending on dataset size. See Training for config options.

Step 3 — Serve the EP model#

Copy the full output directory before serving so the original is preserved:

cp -r /your/ep/output /your/ep/output_serving

Start Triton (scoring only):

NUM_GPUS=<num_gpus> INSTANCES_PER_GPU=<instances_per_gpu> \
  MODEL_REPO=/your/ep/output_serving/infer/triton/ \
  bash /your/ep/output_serving/infer/serve.sh

Optional — with LLM explanation:

Any OpenAI-compatible provider works (NVIDIA NIM, OpenAI, Azure OpenAI, etc.):

NUM_GPUS=<num_gpus> INSTANCES_PER_GPU=<instances_per_gpu> \
  LLM_API_KEY="<your-api-key>" \
  LLM_BASE_URL="<your-openai-compatible-endpoint>" \
  LLM_MODEL="<model-name>" \
  MODEL_REPO=/your/ep/output_serving/infer/triton/ \
  bash /your/ep/output_serving/infer/serve.sh

Example provider configurations (any OpenAI-compatible endpoint works — model names and URLs below are illustrative):

Provider	`LLM_BASE_URL`	Example `LLM_MODEL`
NVIDIA NIM (using `NVIDIA_API_KEY`)	(not needed — auto-configured to `https://inference-api.nvidia.com`)	`nvidia/meta/llama-3.2-1b-instruct`
NVIDIA NIM (using `LLM_API_KEY`)	`https://inference-api.nvidia.com`	`nvidia/meta/llama-3.1-70b-instruct`
OpenAI	`https://api.openai.com/v1`	`gpt-4o`

Poll until ready (usually < 2 minutes):

until curl -sf http://localhost:8000/v2/health/ready >/dev/null 2>&1; do sleep 5; done
echo "Triton ready"

Step 4 — Install client dependencies#

The inference clients (client.py, client_np.py, client_saved_emb.py, and client_production.py) run outside the container and communicate with Triton over gRPC.

pip install "tritonclient[grpc]" pandas numpy torch

# For client_production.py --serve mode (persistent HTTP service) also install:
pip install fastapi uvicorn

Note: tritonclient[grpc] must match the Triton server version. For Triton 26.04 use the version pinned to that release, or install the latest and verify the gRPC handshake succeeds.

Step 5 — Run the EP inference client#

python /your/ep/output_serving/infer/client.py \
  --data_path /your/graph/data/test_gnn \
  --model_config /your/ep/output_serving/infer/triton/gnn_embedder/1/model_config.json \
  --batch_size 512 \
  --explain \
  --explain_limit 10

Stop the server when done:

docker ps --filter "ancestor=nvcr.io/nvidia/tritonserver:26.04-py3" -q | xargs docker stop 2>/dev/null || true

Step 5b — (Optional) Train EP with saved node embeddings#

Set save_node_embeddings: true in your config before training to enable hybrid inference, where known nodes bypass the GNN entirely at inference time:

# In your config.yaml — models[0].hyperparameters.gnn:
models:
  - kind: GNN_XGBoost
    hyperparameters:
      gnn:
        save_node_embeddings: true
        emb_dtype: float16          # halves disk footprint at negligible accuracy cost

Re-run the same docker training command. After training completes, Stage 9 runs automatically: it extracts GNN embeddings for all training nodes and saves them. The Triton export step merges rank shards into output/node_embeddings/{type}/embeddings.mmap.

Run the hybrid inference client:

python <output>/infer/client_production.py \
  --model_config <output>/infer/triton/gnn_embedder/1/model_config.json \
  --emb_dir      <output>/node_embeddings \
  --data_path    /data/test_gnn \
  --batch_size   512

Or start a persistent HTTP service:

pip install fastapi uvicorn
python <output>/infer/client_production.py \
  --model_config <output>/infer/triton/gnn_embedder/1/model_config.json \
  --emb_dir      <output>/node_embeddings \
  --data_path    /data/test_gnn \
  --serve --port 8080

What to check at each step#

Step	Success indicator
1 — Pull	`Status: Downloaded newer image` or `Status: Image is up to date`
2 — EP train	Final log line: `Training complete!` + metrics block with `AUC`, `AUCPR`, threshold
3 — EP serve	All 6 models show `"state":"READY"` in the startup log
4 — Client deps	`pip install` completes without error
5 — EP client	Per-batch latency printed; fraud detections listed with probabilities

If any step fails, stop and check Troubleshooting before continuing.