Graph Data Organization for GNN-based XGBoost Training

Organize your data under a parent directory (for example, data_root) with the following subdirectories:

    data_root/
    ├── nodes/
    └── edges/

• nodes/: Contains the files for node features and labels.
• edges/: Contains the file for edges.

Nodes Directory

File Naming Convention

• Node Feature File: Name the file as node.<ext>, where <ext> is one of the supported file formats (CSV, Parquet, or ORC). Example:

  • node.csv

• Node Label File: Provide the labels for your nodes in a file named node_label.<ext>. Example:

  • node_label.csv

• Optional offset range JSON file: Provide the start and end offset of the target nodes in the node.<ext> file. If this isn’t provided, the training will be run on all the nodes.

  • offset_range_of_training_node.json

File Contents

• Feature File (node.<ext>): This file should contain a table of node attributes (features).

  • Each row corresponds to one node.
  • The number of columns is the feature dimension.

  NOTE: If you have nodes of different types, You must prepare and align their feature.ach row must contain the same number of feature entries.

• Label File (node_label.<ext>, Optional): This file should contain the ground-truth labels for the nodes.

  • There must be one column for each name, with the number of rows matching the feature file.

• Optional JSON file containing the starting and ending offsets for the nodes in the node.<ext> file that will be used for training and validation. In other words, it defines the range of node offsets that the model uses to train.

  Consider credit card fraud detection. A node.<ext> file includes features for Users, Merchants, and Transaction nodes. Your goal is to find fraudulent transactions. Here, you can only train on transaction nodes.

    {
      "start": ST #start offset of training nodes
      "end": ET #end offset of training nodes
    }

  If no file is provided, the models will be trained using the entire range.

Edges Directory

File Naming Convention

• Edge File: Name the file as node_to_node.<ext>, where <ext> is one of the supported file formats (CSV, Parquet, or ORC). Examples:
  • node_to_node.csv

File Contents

• Edge File (node_to_node.<ext>): This file must contain at least two columns:
  • src: The column containing indices of the source nodes
  • dst: The column containing indices of the destination nodes.
  • The index of a node is determined by its position in the node feature file node.<ext> and is zero-based.
  • The number of rows corresponds to the number of edges. NOTE: The columns containing source and destination nodes must be named src and dst, respectively.

Data types

node.<ext>

• Description: Contains node features.
• Data Type: All columns must contain floating point numbers in float32 format.
• Notes: Each row represents a node, and each column corresponds to a specific feature.

node_label.<ext>

• Description: Contains node labels.
• Data Type: All values are integers (0 or 1).
• Column Name: The file contains a single column named target.
• Notes: Each row corresponds to the label of a node.

node_to_node.<ext>

• Description: Contains edge indices.
• Data Type: All columns must contain integers.
• Columns:
  • src: Source node index.
  • dst: Destination node index.
• Notes: The indices must be zero-based; in other words, node numbering starts at 0.

Example Data Layout

credit_card_transaction_data/
├── nodes/
│ ├── node.csv
│ └── node_label.csv
│ └── offset_range_of_training_node.json
└── edges/
  └── node_to_node.csv

Data Organization for Standalone XGBoost Training

Your dataset is expected to be organized in a specific structure under the xgb/ folder. Follow the guidelines below to ensure your data is correctly formatted and named.

Directory Structure

Place your data files under the xgb/ directory within your main data folder. The expected files are:

• training.<ext> (Required): Contains the training data.
• validation.<ext> (Optional): Contains the validation data.
  • If this file is missing, the training data will be split into 80% training and 20% validation.
• test.<ext> (Optional): Contains the test data. If test data is provided, it will be used for testing the model performance.

Supported File Formats

The <ext> in the file names represents the file format. The supported file formats are:

• CSV: Files with the .csv extension (comma-separated values).
• Parquet: Files with the .parquet extension (columnar storage).
• ORC: Files with the .orc extension (Optimized Row Columnar format).

Important Data Format Note

• Target Column: The last column in every file (i.e.training.<ext>, validation.<ext> and test.<ext>) is assumed to be the target variable (y). All splits (training, validation, and test) are performed with this assumption in mind.

Example Directory Layout

data_root/
└── xgb/
    ├── training.csv # or training.parquet, training.orc
    ├── validation.csv # (optional) or validation.parquet, validation.orc
    └── test.csv # (optional) or test.parquet, test.orc

Data Splitting Behavior

• When only training.<ext> is provided:
  • If only the training.<ext> file exists, the code will split the data into 80% training and 20% validation.
• When validation.<ext> is provided:
  • The provided training and validation files are used directly without splitting.
• When test.<ext> is present:
  • The test file is used to evaluate the model after training.

Model Testing

If a test.<ext> file is present in the xgb/ directory, it will be used for evaluation after training.