Is this page helpful?

Data Layout — Rigorous Reference#

3.1 Top-Level Directory Structure#

your-data/                                       ← example names; use any names that fit your domain
├── nodes/
│   ├── account.csv                              ← one source node type (any name)
│   ├── merchant.csv                             ← one destination node type (any name)
│   ├── customer.csv                             ← optional additional context node (any name)
│   ├── device.csv                               ← optional (any name)
│   └── terminal.csv                             ← optional (any name)
├── edges/
│   ├── account_transacts_merchant.csv           ← connectivity edge (any name)
│   ├── account_transacts_merchant_label.csv     ← fraud labels (required, ONE type only)
│   ├── account_transacts_merchant_attr.csv      ← per-transaction features (optional)
│   ├── customer_holds_account.csv               ← additional context edges (no label, any name)
│   ├── device_initiates_account.csv             ← additional context edges (no label, any name)
│   └── account_uses_terminal.csv                ← additional context edges (no label, any name)
└── test_gnn/                                    ← held-out evaluation split
    ├── nodes/
    │   ├── account.csv
    │   └── merchant.csv
    └── edges/
        ├── account_transacts_merchant.csv
        ├── account_transacts_merchant_label.csv
        └── account_transacts_merchant_attr.csv

Design principle: The pipeline auto-discovers all node and edge types from directory contents. You do not hardcode entity or relationship names anywhere; the file names are the names.

All edge types are message-passing channels. Every edge file in edges/ is loaded into the GNN graph, not just the labeled (predicted) edge type. Context edges (e.g. customer_holds_account.csv, device_initiates_account.csv) enrich the neighbourhood signal for the predicted edge type. Adding or removing a context edge file changes the model’s receptive field, its feature dimension, and the expected Triton input schema — the model must be retrained whenever the edge set changes.

3.2 Node Files#

Location: nodes/{node_type}.csv

The file stem (name without extension) is used as the node type identifier throughout the entire pipeline — in the GNN graph schema, the Triton config, and the inference client. You can use any names that fit your domain; short, lowercase, singular names are recommended for readability.

Format requirements:

Requirement

Detail

Header row

First row must be column names

No index column

Do not include a row number or ID column. Row 0 = node ID 0.

Numeric only

All values must be finite floats or integers. No strings, nulls, or NaNs.

Categorical encoding

Encode categories numerically before writing the CSV (e.g. binary encoding). See Feature Engineering.

Consistent across splits

nodes/account.csv and test_gnn/nodes/account.csv must have identical column names in identical order.

Example — nodes/account.csv:

Balance,AvgMonthlyTxn,OverdraftCount,IsActive,AccountType_bit0,AccountType_bit1
549.19885,327.34738,3.0,1.0,1.0,1.0
572.57916,1832.1833,9.0,1.0,0.0,0.0
1792.9539,276.47278,2.0,1.0,0.0,1.0

Row 0 (account ID 0): balance $549.20, average monthly transaction $327.35, 3 overdrafts, active, account type 3 (binary 11).
Row 1 (account ID 1): balance $572.58, account type 0 (binary 00).
Row 2 (account ID 2): balance $1792.95, account type 2 (binary 01).

Example — nodes/merchant.csv:

CategoryCode_0,CategoryCode_1,...,CategoryCode_5,AvgTxnAmount,ChargebackRate,IsOnline
0.0,0.0,1.0,0.0,0.0,0.0,145.23,0.0023,0.0
1.0,0.0,0.0,0.0,0.0,0.0,23.10,0.0,1.0

3.3 Edge Files#

All edge files share a common naming stem derived from a three-part type triple:

{source_node_type}_{relationship_name}_{destination_node_type}

How the Parser Works#

The pipeline splits the filename stem on every _ and assigns:

src = parts[0]          ← first token
dst = parts[-1]         ← last token
rel = "_".join(parts[1:-1])   ← everything in between (may contain underscores)

Minimum requirement: at least three _-separated tokens — one for src, at least one for rel, one for dst.

Part

Rule

Example

src

First token. Must match a file stem in nodes/.

account

rel

All middle tokens joined with _. Can be multi-word.

transacts, pays_to

dst

Last token. Must match a file stem in nodes/.

merchant

Valid examples:

Filename stem

src

rel

dst

account_transacts_merchant

account

transacts

merchant

account_pays_to_merchant

account

pays_to

merchant

user_sent_wire_to_account

user

sent_wire_to

account

Invalid examples — these will fail validation:

Filename

Problem

transaction.csv

Only 1 token — no _ at all; cannot extract src, rel, dst

account_merchant.csv

Only 2 tokens — missing rel

b.csv

Only 1 token

Important: src and dst are always the first and last token respectively. If your node file is nodes/credit_card.csv (stem = credit_card), you cannot use it as src because the parser will take only credit as src and card will be absorbed into rel. Either rename the node file to nodes/creditcard.csv and use creditcard as the first token, or accept that credit becomes the logical node type name.

3.3.1 Connectivity File (required for every edge type)#

File: edges/{src}_{rel}_{dst}.csv

Two columns read positionally (first = source node ID, second = destination node ID). The column headers can be any names — the pipeline reads the first two columns regardless of name. The conventional names src and dst are recommended for clarity and are required by the validation commands in Section 3.6.

src,dst
495,87
182,265
311,219
432,237

Row 0: node 495 in account.csv is connected through one transaction to node 87 in merchant.csv.

Important: Edges are directed. The pipeline automatically creates reverse edges (rev_{rel}) for bidirectional message passing during GNN training. You do not need to add them yourself.

3.3.2 Label File#

Edge Prediction (EP) — File: edges/{src}_{rel}_{dst}_label.csv

A single column containing the fraud label for each edge. The column header can be any name; only the last column is read (iloc[:, -1]). For single-column files this makes no difference, but avoid appending extra columns after the label column (e.g. an ID column), as the last column would then be read as the label. Must have exactly the same number of rows as the connectivity file.

fraud
0
0
1
0

1 = fraudulent. 0 = legitimate.

The pipeline automatically detects which edge type has a label file and uses it as the prediction target. Exactly one edge type should have a label file. If zero or multiple label files are found, training will fail with a clear error.

Node Prediction (NP) — File: nodes/{node_type}_label.csv

For node-level fraud prediction (kind: GNN_XGBoost_NP), place the label file in the nodes/ directory alongside the node feature file:

nodes/
  account.csv            ← node features
  account_label.csv      ← one row per account node; 1 = fraud, 0 = legitimate

Same single-column format as the EP label file. The stem must exactly match the target node type filename (e.g. account_label.csv targets nodes in account.csv). The pipeline detects this automatically.

3.3.4 Train/Val/Test Mask Files (optional)#

By default the pipeline applies a fixed 90/10 stratified train/val split (hardcoded, no config parameters control the ratio). To use a custom, reproducible split instead, provide boolean NumPy mask arrays:

Edge Prediction (EP): place masks alongside the edge files:

edges/
  train_mask.npy    ← boolean array, length = number of labeled edges
  val_mask.npy
  test_mask.npy

Node Prediction (NP): place masks alongside the node files:

nodes/
  train_mask.npy    ← boolean array, length = number of labeled nodes
  val_mask.npy
  test_mask.npy

Each mask is a bool NumPy array saved with np.save. True means the corresponding edge or node belongs to that split. The three masks are mutually exclusive and together cover the full labeled set. If no mask files are present the pipeline falls back to the default 90/10 split. If any mask files are detected, train_mask.npy and val_mask.npy must both be provided (the pipeline raises an error if either is missing); test_mask.npy is optional.

Masks are always .npy — this is not configurable. The mask format is intentionally decoupled from the node/edge file format (CSV, Parquet, or ORC). NumPy binary arrays are compact, platform-portable, and avoid the row-count ambiguity of text formats. There is no benefit to storing a boolean vector in Parquet or ORC, and the overhead of a Parquet reader for a single 1-D boolean array would be unnecessary.

# Example: generate masks from a temporal split
import numpy as np
n = len(labels)
train_mask = timestamps < cutoff_train
val_mask   = (timestamps >= cutoff_train) & (timestamps < cutoff_val)
test_mask  = timestamps >= cutoff_val
np.save("edges/train_mask.npy", train_mask)
np.save("edges/val_mask.npy",   val_mask)
np.save("edges/test_mask.npy",  test_mask)

3.4 The Test Split#

The test_gnn/ subdirectory is a completely separate graph used only for final evaluation — the pipeline never sees its labels during training or threshold calibration.

your-data/
├── nodes/            ← training graph node features
├── edges/            ← training graph edges, labels, attributes
└── test_gnn/
    ├── nodes/        ← test graph node features (same columns, different rows)
    └── edges/        ← test graph edges, labels, attributes

Important considerations:

  • Temporal correctness: If your data has a time dimension, the test split should contain transactions from a later time period than training. Splitting randomly can leak future information and produce unrealistically optimistic metrics.

  • Node overlap is acceptable. The same account or merchant can appear in both the training and test graphs. The test set evaluates the model’s generalisation on new transactions, not new entities.

  • Minimum size: At least a few thousand edges, with a reasonable number of fraud cases (≥ 100 fraud edges recommended for stable metric estimates).

3.5 Supported File Formats#

The pipeline auto-detects format by file extension. Node, edge, label, and attribute files can use any of the following:

Format

Extension

Notes

CSV

.csv

Most portable; larger file size

Apache Parquet

.parquet

Efficient columnar format; recommended for large datasets (> 10 M edges)

Apache ORC

.orc

Similar to Parquet; requires pyarrow with ORC support

Mixing formats is fully supported. You can have nodes/user.parquet alongside edges/user_to_merchant.orc — each file is read independently based on its own extension. The label file (e.g. user_to_merchant_label.csv) must use an extension the pipeline recognises, but it does not need to match the extension of the edge file.

Mask files are always .npy regardless of data format. See Section 3.3.4.

Parquet/ORC written on a different machine: if you generate Parquet files on the host and train inside the container, ensure the pyarrow versions are compatible. When in doubt, convert inside the container:

docker run --rm --entrypoint python3 \
  -v /your/data:/src:ro -v /your/converted:/dst \
  nvcr.io/nvidia/cugraph/financial-fraud-training:3.0.0 -c "
import pandas as pd; from pathlib import Path
for f in Path('/src/nodes').glob('*.csv'):
    pd.read_csv(f).to_parquet(Path('/dst/nodes') / f.with_suffix('.parquet').name, index=False)
"

3.6 Pre-Training Data Validation Checklist#

Run these checks before starting training to avoid hard-to-diagnose errors mid-run.

Substitute your own names throughout: Replace account_transacts_merchant with your labeled edge stem, account with your source node type, and merchant with your destination node type in every command below.

#

Check

Command to verify

1

Node IDs are zero-based

python -c "import pandas as pd; df=pd.read_csv('edges/account_transacts_merchant.csv'); n=len(pd.read_csv('nodes/account.csv')); assert df.src.max() < n, f'max src={df.src.max()}, n_accounts={n}'"

2

Edge indices in range for dst

Same as above but for dst and nodes/merchant.csv

3

No NaN or Inf in node features

python -c "import pandas as pd, numpy as np; df=pd.read_csv('nodes/account.csv'); assert not df.isnull().any().any(); assert np.isfinite(df.values).all()"

4

Labels are strictly binary

python -c "import pandas as pd; df=pd.read_csv('edges/account_transacts_merchant_label.csv'); assert set(df.iloc[:,-1].unique()).issubset({0,1})"

5

Exactly one label file (EP)

python -c "from pathlib import Path; f=[p for p in Path('edges').glob('*_label.*') if p.suffix in ('.csv','.parquet','.orc')]; assert len(f)==1, f"

5b

Exactly one label file (NP)

python -c "from pathlib import Path; f=[p for p in Path('nodes').glob('*_label.*') if p.suffix in ('.csv','.parquet','.orc')]; assert len(f)==1, f"

6

Train and test have identical node columns

diff <(head -1 nodes/account.csv) <(head -1 test_gnn/nodes/account.csv) → no output

7

Train and test have identical attr columns

diff <(head -1 edges/account_transacts_merchant_attr.csv) <(head -1 test_gnn/edges/account_transacts_merchant_attr.csv)

8

Edge count matches label count

wc -l edges/account_transacts_merchant.csv edges/account_transacts_merchant_label.csv → both should print the same count

9

Attr count matches edge count (if attr file exists)

wc -l edges/account_transacts_merchant.csv edges/account_transacts_merchant_attr.csv → both should print the same count

10

Fraud rate is non-zero

python -c "import pandas as pd; df=pd.read_csv('edges/account_transacts_merchant_label.csv'); print(f'fraud rate: {df.iloc[:,-1].mean():.4f}')"