> This page is for version 3.0.0 (Latest) (default).
> For other versions, use one of these documentation indexes:
> - 3.0.0 (Latest) (default): https://docs.nvidia.com/nim/financial-fraud-training/3.0.0/llms.txt
> - 2.0.0: https://docs.nvidia.com/nim/financial-fraud-training/2.0.0/llms.txt
> - 1.0.0: https://docs.nvidia.com/nim/financial-fraud-training/1.0.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nim/financial-fraud-training/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nim/financial-fraud-training/_mcp/server.

# Data Layout — Rigorous Reference

## 3.1 Top-Level Directory Structure

```
your-data/                                       ← example names; use any names that fit your domain
├── nodes/
│   ├── account.csv                              ← one source node type (any name)
│   ├── merchant.csv                             ← one destination node type (any name)
│   ├── customer.csv                             ← optional additional context node (any name)
│   ├── device.csv                               ← optional (any name)
│   └── terminal.csv                             ← optional (any name)
├── edges/
│   ├── account_transacts_merchant.csv           ← connectivity edge (any name)
│   ├── account_transacts_merchant_label.csv     ← fraud labels (required, ONE type only)
│   ├── account_transacts_merchant_attr.csv      ← per-transaction features (optional)
│   ├── customer_holds_account.csv               ← additional context edges (no label, any name)
│   ├── device_initiates_account.csv             ← additional context edges (no label, any name)
│   └── account_uses_terminal.csv                ← additional context edges (no label, any name)
└── test_gnn/                                    ← held-out evaluation split
    ├── nodes/
    │   ├── account.csv
    │   └── merchant.csv
    └── edges/
        ├── account_transacts_merchant.csv
        ├── account_transacts_merchant_label.csv
        └── account_transacts_merchant_attr.csv
```

> **Design principle:** The pipeline auto-discovers all node and edge types from directory contents. You do not hardcode entity or relationship names anywhere; the file names *are* the names.

> **All edge types are message-passing channels.** Every edge file in `edges/` is loaded into the GNN graph, not just the labeled (predicted) edge type. Context edges (e.g. `customer_holds_account.csv`, `device_initiates_account.csv`) enrich the neighbourhood signal for the predicted edge type. Adding or removing a context edge file changes the model's receptive field, its feature dimension, and the expected Triton input schema — the model must be retrained whenever the edge set changes.

## 3.2 Node Files

**Location:** `nodes/{node_type}.csv`

The file stem (name without extension) is used as the node type identifier throughout the entire pipeline — in the GNN graph schema, the Triton config, and the inference client. You can use any names that fit your domain; short, lowercase, singular names are recommended for readability.

**Format requirements:**

| Requirement | Detail |
|---|---|
| Header row | First row must be column names |
| No index column | Do **not** include a row number or ID column. Row 0 = node ID 0. |
| Numeric only | All values must be finite floats or integers. No strings, nulls, or NaNs. |
| Categorical encoding | Encode categories numerically before writing the CSV (e.g. binary encoding). See [Feature Engineering](/nim/financial-fraud-training/data/data-prep#feature-engineering-guidelines). |
| Consistent across splits | `nodes/account.csv` and `test_gnn/nodes/account.csv` must have identical column names in identical order. |

**Example — `nodes/account.csv`:**

```
Balance,AvgMonthlyTxn,OverdraftCount,IsActive,AccountType_bit0,AccountType_bit1
549.19885,327.34738,3.0,1.0,1.0,1.0
572.57916,1832.1833,9.0,1.0,0.0,0.0
1792.9539,276.47278,2.0,1.0,0.0,1.0
```

Row 0 (account ID 0): balance $549.20, average monthly transaction $327.35, 3 overdrafts, active, account type 3 (binary `11`).  
Row 1 (account ID 1): balance $572.58, account type 0 (binary `00`).  
Row 2 (account ID 2): balance $1792.95, account type 2 (binary `01`).

**Example — `nodes/merchant.csv`:**

```
CategoryCode_0,CategoryCode_1,...,CategoryCode_5,AvgTxnAmount,ChargebackRate,IsOnline
0.0,0.0,1.0,0.0,0.0,0.0,145.23,0.0023,0.0
1.0,0.0,0.0,0.0,0.0,0.0,23.10,0.0,1.0
```

## 3.3 Edge Files

All edge files share a common naming stem derived from a three-part type triple:

```
{source_node_type}_{relationship_name}_{destination_node_type}
```

### How the Parser Works

The pipeline splits the filename stem on every `_` and assigns:

```
src = parts[0]          ← first token
dst = parts[-1]         ← last token
rel = "_".join(parts[1:-1])   ← everything in between (may contain underscores)
```

**Minimum requirement:** at least three `_`-separated tokens — one for `src`, at least one for `rel`, one for `dst`.

| Part | Rule | Example |
|------|------|---------|
| `src` | First token. Must match a file stem in `nodes/`. | `account` |
| `rel` | All middle tokens joined with `_`. Can be multi-word. | `transacts`, `pays_to` |
| `dst` | Last token. Must match a file stem in `nodes/`. | `merchant` |

**Valid examples:**

| Filename stem | src | rel | dst |
|---------------|-----|-----|-----|
| `account_transacts_merchant` | `account` | `transacts` | `merchant` |
| `account_pays_to_merchant` | `account` | `pays_to` | `merchant` |
| `user_sent_wire_to_account` | `user` | `sent_wire_to` | `account` |

**Invalid examples — these will fail validation:**

| Filename | Problem |
|----------|---------|
| `transaction.csv` | Only 1 token — no `_` at all; cannot extract src, rel, dst |
| `account_merchant.csv` | Only 2 tokens — missing rel |
| `b.csv` | Only 1 token |

<Warning>`src` and `dst` are always the **first and last** token respectively. If your node file is `nodes/credit_card.csv` (stem = `credit_card`), you cannot use it as `src` because the parser will take only `credit` as `src` and `card` will be absorbed into `rel`. Either rename the node file to `nodes/creditcard.csv` and use `creditcard` as the first token, or accept that `credit` becomes the logical node type name.</Warning>

### 3.3.1 Connectivity File (required for every edge type)

**File:** `edges/{src}_{rel}_{dst}.csv`

Two columns read positionally (first = source node ID, second = destination node ID). The column headers can be any names — the pipeline reads the first two columns regardless of name. The conventional names `src` and `dst` are recommended for clarity and are required by the validation commands in Section 3.6.

```
src,dst
495,87
182,265
311,219
432,237
```

Row 0: node 495 in `account.csv` is connected through one transaction to node 87 in `merchant.csv`.

<Warning>Edges are directed. The pipeline automatically creates reverse edges (`rev_{rel}`) for bidirectional message passing during GNN training. You do not need to add them yourself.</Warning>

### 3.3.2 Label File

**Edge Prediction (EP) — File:** `edges/{src}_{rel}_{dst}_label.csv`

A single column containing the fraud label for each edge. The column header can be any name; only the last column is read (`iloc[:, -1]`). For single-column files this makes no difference, but avoid appending extra columns after the label column (e.g. an ID column), as the last column would then be read as the label. Must have exactly the same number of rows as the connectivity file.

```
fraud
0
0
1
0
```

`1` = fraudulent. `0` = legitimate.

The pipeline automatically detects which edge type has a label file and uses it as the prediction target. **Exactly one edge type** should have a label file. If zero or multiple label files are found, training will fail with a clear error.

**Node Prediction (NP) — File:** `nodes/{node_type}_label.csv`

For node-level fraud prediction (`kind: GNN_XGBoost_NP`), place the label file in the `nodes/` directory alongside the node feature file:

```
nodes/
  account.csv            ← node features
  account_label.csv      ← one row per account node; 1 = fraud, 0 = legitimate
```

Same single-column format as the EP label file. The stem must exactly match the target node type filename (e.g. `account_label.csv` targets nodes in `account.csv`). The pipeline detects this automatically.

### 3.3.3 Edge Attribute File (optional but recommended)

**File:** `edges/{src}_{rel}_{dst}_attr.csv`

Per-transaction features that do not naturally belong to either endpoint node. Classic examples: transaction amount, time of day, day of week, payment channel (ATM / in-store / online), whether a chip was used.

```
Amount,TimeOfDay_0,TimeOfDay_1,TimeOfDay_2,TimeOfDay_3,TimeOfDay_4,TimeOfDay_5,DayOfWeek_0,...,Channel_Online
9.929522,0.0,0.0,0.0,1.0,0.0,0.0,1.0,...,1.0
17.898922,0.0,0.0,0.0,1.0,0.0,0.0,0.0,...,0.0
```

These attributes are concatenated with the GNN-derived node embeddings to form the final edge embedding passed into XGBoost. They are also exposed to the LLM explainer if a `feature_map.json` is present.

(train-val-test-mask-files-optional)=
### 3.3.4 Train/Val/Test Mask Files (optional)

By default the pipeline applies a fixed 90/10 stratified train/val split (hardcoded, no config parameters control the ratio). To use a custom, reproducible split instead, provide boolean NumPy mask arrays:

**Edge Prediction (EP):** place masks alongside the edge files:
```
edges/
  train_mask.npy    ← boolean array, length = number of labeled edges
  val_mask.npy
  test_mask.npy
```

**Node Prediction (NP):** place masks alongside the node files:
```
nodes/
  train_mask.npy    ← boolean array, length = number of labeled nodes
  val_mask.npy
  test_mask.npy
```

Each mask is a `bool` NumPy array saved with `np.save`. `True` means the corresponding edge or node belongs to that split. The three masks are mutually exclusive and together cover the full labeled set. If **no** mask files are present the pipeline falls back to the default 90/10 split. If **any** mask files are detected, `train_mask.npy` and `val_mask.npy` must both be provided (the pipeline raises an error if either is missing); `test_mask.npy` is optional.

> **Masks are always `.npy` — this is not configurable.** The mask format is intentionally decoupled from the node/edge file format (CSV, Parquet, or ORC). NumPy binary arrays are compact, platform-portable, and avoid the row-count ambiguity of text formats. There is no benefit to storing a boolean vector in Parquet or ORC, and the overhead of a Parquet reader for a single 1-D boolean array would be unnecessary.

```python
# Example: generate masks from a temporal split
import numpy as np
n = len(labels)
train_mask = timestamps < cutoff_train
val_mask   = (timestamps >= cutoff_train) & (timestamps < cutoff_val)
test_mask  = timestamps >= cutoff_val
np.save("edges/train_mask.npy", train_mask)
np.save("edges/val_mask.npy",   val_mask)
np.save("edges/test_mask.npy",  test_mask)
```

## 3.4 The Test Split

The `test_gnn/` subdirectory is a completely separate graph used **only for final evaluation** — the pipeline never sees its labels during training or threshold calibration.

```
your-data/
├── nodes/            ← training graph node features
├── edges/            ← training graph edges, labels, attributes
└── test_gnn/
    ├── nodes/        ← test graph node features (same columns, different rows)
    └── edges/        ← test graph edges, labels, attributes
```

**Important considerations:**

- **Temporal correctness:** If your data has a time dimension, the test split should contain transactions from a *later* time period than training. Splitting randomly can leak future information and produce unrealistically optimistic metrics.
- **Node overlap is acceptable.** The same account or merchant can appear in both the training and test graphs. The test set evaluates the model's generalisation on new transactions, not new entities.
- **Minimum size:** At least a few thousand edges, with a reasonable number of fraud cases (≥ 100 fraud edges recommended for stable metric estimates).

## 3.5 Supported File Formats

The pipeline auto-detects format by file extension. Node, edge, label, and attribute files can use any of the following:

| Format | Extension | Notes |
|---|---|---|
| CSV | `.csv` | Most portable; larger file size |
| Apache Parquet | `.parquet` | Efficient columnar format; recommended for large datasets (> 10 M edges) |
| Apache ORC | `.orc` | Similar to Parquet; requires `pyarrow` with ORC support |

**Mixing formats is fully supported.** You can have `nodes/user.parquet` alongside `edges/user_to_merchant.orc` — each file is read independently based on its own extension. The label file (e.g. `user_to_merchant_label.csv`) must use an extension the pipeline recognises, **but it does not need to match the extension of the edge file**.

> **Mask files are always `.npy` regardless of data format.** See [Section 3.3.4](#train-val-test-mask-files-optional).

> **Parquet/ORC written on a different machine**: if you generate Parquet files on the host and train inside the container, ensure the pyarrow versions are compatible. When in doubt, convert inside the container:
> ```bash
> docker run --rm --entrypoint python3 \
>   -v /your/data:/src:ro -v /your/converted:/dst \
>   nvcr.io/nvidia/cugraph/financial-fraud-training:3.0.0 -c "
> import pandas as pd; from pathlib import Path
> for f in Path('/src/nodes').glob('*.csv'):
>     pd.read_csv(f).to_parquet(Path('/dst/nodes') / f.with_suffix('.parquet').name, index=False)
> "
> ```

## 3.6 Pre-Training Data Validation Checklist

Run these checks before starting training to avoid hard-to-diagnose errors mid-run.

> **Substitute your own names throughout:** Replace `account_transacts_merchant` with your labeled edge stem, `account` with your source node type, and `merchant` with your destination node type in every command below.

| # | Check | Command to verify |
|---|---|---|
| 1 | Node IDs are zero-based | `python -c "import pandas as pd; df=pd.read_csv('edges/account_transacts_merchant.csv'); n=len(pd.read_csv('nodes/account.csv')); assert df.src.max() < n, f'max src={df.src.max()}, n_accounts={n}'"` |
| 2 | Edge indices in range for dst | Same as above but for `dst` and `nodes/merchant.csv` |
| 3 | No NaN or Inf in node features | `python -c "import pandas as pd, numpy as np; df=pd.read_csv('nodes/account.csv'); assert not df.isnull().any().any(); assert np.isfinite(df.values).all()"` |
| 4 | Labels are strictly binary | `python -c "import pandas as pd; df=pd.read_csv('edges/account_transacts_merchant_label.csv'); assert set(df.iloc[:,-1].unique()).issubset({0,1})"` |
| 5 | Exactly one label file (EP) | `python -c "from pathlib import Path; f=[p for p in Path('edges').glob('*_label.*') if p.suffix in ('.csv','.parquet','.orc')]; assert len(f)==1, f"` |
| 5b | Exactly one label file (NP) | `python -c "from pathlib import Path; f=[p for p in Path('nodes').glob('*_label.*') if p.suffix in ('.csv','.parquet','.orc')]; assert len(f)==1, f"` |
| 6 | Train and test have identical node columns | `diff <(head -1 nodes/account.csv) <(head -1 test_gnn/nodes/account.csv)` → no output |
| 7 | Train and test have identical attr columns | `diff <(head -1 edges/account_transacts_merchant_attr.csv) <(head -1 test_gnn/edges/account_transacts_merchant_attr.csv)` |
| 8 | Edge count matches label count | `wc -l edges/account_transacts_merchant.csv edges/account_transacts_merchant_label.csv` → both should print the same count |
| 9 | Attr count matches edge count (if attr file exists) | `wc -l edges/account_transacts_merchant.csv edges/account_transacts_merchant_attr.csv` → both should print the same count |
| 10 | Fraud rate is non-zero | `python -c "import pandas as pd; df=pd.read_csv('edges/account_transacts_merchant_label.csv'); print(f'fraud rate: {df.iloc[:,-1].mean():.4f}')"` |