> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/sdgm/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/sdgm/_mcp/server.

# Introduction to Graph Neural Networks

> An introduction to graphs, graph neural networks, message passing, and the move to graph transformers

Graph-structured data describes entities and the relationships between them.
This guide introduces graph neural networks (GNNs), explains how message passing learns from those relationships, and shows why graph transformers are the next step for relational machine learning.

**Originally published:** October 21, 2022\

**Authors:** Matthias Fey and Ivaylo Bahtchevanov

## What are graphs?

Most interesting phenomena can be broken down into entities, their relationships, and their interactions.
Graphs are the mathematical abstraction that captures this structure.
A graph consists of **nodes** (entities) and **edges** (connections between entities).

Graphs appear throughout real-world data:

* **Social networks:** People are nodes, while friendships, messages, and follows are edges.
* **E-commerce:** Users, products, and categories connect through purchases, reviews, and browsing sessions.
* **Biology:** Proteins, drugs, genes, pathways, atoms, and chemical bonds form interconnected networks.
* **Finance:** Accounts, transactions, merchants, and devices form a dynamic graph.
* **Infrastructure:** Roads, power grids, supply chains, and communication networks connect physical or logical entities.

**Think of a graph as a city map.** Buildings are nodes and roads are edges.
Some roads are one-way, and different buildings serve different purposes.
The map preserves the actual pattern of connections instead of forcing the city into a grid or a sequence.

### Homogeneous and heterogeneous graphs

A **homogeneous graph** has one node type and one edge type.
An author-collaboration network, where every node is a researcher and every edge means "co-authored a paper," is one example.

A **heterogeneous graph** has multiple node and edge types.
For example, a financial graph can contain `account`, `merchant`, and `device` nodes, connected by `transfer`, `purchase`, and `login` edges.
Each type can carry different attributes.

Relational databases already contain graph structure.
Rows define entities, and foreign keys define relationships.
Querying across joined tables is graph traversal, whether or not the data is called a graph.

## Why use graphs for machine learning?

Traditional machine learning models such as logistic regression, random forests, and gradient-boosted trees expect a flat table: one row per sample and one column per feature.
That works when data is naturally tabular, but many important signals live in the relationships between records.

Consider fraud detection.
A transaction has attributes such as amount, timestamp, and merchant category, but its strongest signals often come from context: the sending and receiving accounts, their other transactions, the devices involved, and the broader network pattern.
Flat-table models require teams to manually engineer features that summarize this context, such as transaction counts over a time window or the number of devices linked to an account.

![Comparison of flat-table machine learning with graph neural networks, showing benefits and trade-offs for each approach.](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/nvidia-sdgm.docs.buildwithfern.com/2cfafd68da6e61e4260eb9b6400b401a4d58dbdb9437c864dd85588658109171/img/research/tabular-ml-vs-gnn.svg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260920%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260920T164629Z&X-Amz-Expires=604800&X-Amz-Signature=61cd8d973182e3cb39b22ccd689ffb82ddc0a93b0d727817b03fda25ba1c01ba&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

### Five graph learning tasks

1. **Node classification:** Predict an attribute of a node, such as whether an account is fraudulent or a customer will churn.
2. **Link prediction:** Predict whether a relationship should exist, such as whether a user will purchase a product.
3. **Graph classification:** Classify an entire graph, such as whether a molecule is toxic.
4. **Community detection:** Find closely connected groups, such as fraud rings or customer segments.
5. **Missing-node identification:** Discover entities that should exist but are not yet observed, such as potential drug side effects.

In graph ML, connected nodes inform one another.
GNNs propagate information along edges, so topology and multi-hop patterns become model features.
A node with few attributes can still be informative when its neighbors are informative.

## How GNNs work: message passing

![Four-stage GNN message-passing pipeline: initialize node features, aggregate neighbor information, update the node representation, and repeat.](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/nvidia-sdgm.docs.buildwithfern.com/83cf1ae98fea90863c8289fd276113d281e14b627b96cbd5db5b3d28a8d15ffb/img/research/message-passing-pipeline.svg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260920%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260920T164629Z&X-Amz-Expires=604800&X-Amz-Signature=b9cb1d5ce797290e420316a508f1472fbca87006c9597f72374dfc7d0c8b061c&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

The core GNN mechanism is **message passing**.
Each node updates its representation by collecting information from its neighbors.
After multiple rounds, a node representation encodes progressively larger neighborhoods.

### A concrete example

Imagine Alice connected to Bob and Carol in a social network.
Bob is connected to Dave, and Carol is connected to Eve.
After one layer, Alice's representation includes a summary of Bob and Carol.
After two layers, Bob's and Carol's representations also contain information about Dave and Eve, so Alice indirectly encodes that two-hop context.

**Analogy:** Message passing is like a structured game of telephone.
In the first round, you learn what your direct friends know.
In the next round, you learn what their friends know.
Each round applies learned transformations that filter and compress the information.

### The simplified math

For node *v* at layer *k*, message passing follows three steps:

1. **Message:** For each neighbor *u* of *v*, compute a message from *u*'s current features.
   It can be a copy, a linear transformation, or an attention-weighted projection.
2. **Aggregate:** Combine incoming messages into one vector, commonly with a sum, mean, or maximum.
   This operation must be permutation-invariant because a graph has no inherent neighbor ordering.
3. **Update:** Combine the aggregated message with *v*'s previous features, typically using a neural network layer.

The number of message-passing layers determines the receptive field.
Two layers expose a node to its two-hop neighborhood.
In many tasks, two or three layers work well; much deeper models can over-smooth node representations until they become hard to distinguish.

## Types of GNNs

GNN architectures differ in how they compute and aggregate messages.
Their trade-offs are expressiveness, computational cost, and scalability.

### Graph Convolutional Networks (GCN)

GCN computes a degree-normalized weighted average of neighbor features.
The normalization prevents high-degree nodes from dominating an aggregation.
GCN is fast and effective for homogeneous graphs with relatively uniform structure, but it does not learn which specific neighbors matter most for a prediction.

### GraphSAGE

GraphSAGE addresses scalability by sampling a fixed number of neighbors at each layer instead of aggregating every neighbor.
It also uses learnable aggregation functions, such as mean, LSTM, or pooling.
Sampling makes GraphSAGE practical for very large graphs and supports inductive learning on unseen nodes, at the cost of sampling variance.

### Graph Attention Networks (GAT)

GAT learns attention scores between connected nodes.
Rather than weighting neighbors only by degree, it learns how relevant each neighbor is to a target node for the task.
For a citation graph, for example, a model can attend more strongly to influential references than to tangential citations.

### Other notable architectures

![Comparison of GCN, GraphSAGE, and GAT architectures, showing the strengths and trade-offs of each.](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/nvidia-sdgm.docs.buildwithfern.com/4c8c18d166b96a8a8f4e3edb3c25fd83ebad1f7fb677459a61bf97f924692fa8/img/research/gnn-architectures.svg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260920%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260920T164629Z&X-Amz-Expires=604800&X-Amz-Signature=c87d8873c3d271c07fd3ceceeec18629162a5a8d5985bbd560bdc0788c03871d&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

* **GIN (Graph Isomorphism Network):** Uses sum aggregation and learned injective functions to maximize structural expressiveness.
* **MPNN (Message Passing Neural Network):** A general framework that includes GCN, GraphSAGE, and GAT as different message and aggregation functions.
* **R-GCN (Relational GCN):** Uses edge-type-specific transformations for heterogeneous graphs, such as knowledge graphs.

There is no universally best GNN.
GCN is a useful baseline for homogeneous graphs, GraphSAGE is a practical choice when scale matters, and GAT is useful when neighbor relevance varies significantly.
The right architecture depends on graph size, heterogeneity, and whether predictions must generalize to new nodes.

## Real-world applications

### Fraud detection and financial crime

Financial networks connect accounts, transactions, devices, and merchants.
Fraud patterns can look ordinary in an isolated transaction but become visible across the two- or three-hop network of related accounts and events.

### Recommender systems

GNNs model user-item interactions as a bipartite graph, with purchases, views, or ratings as edges.
The learned representations capture behavioral similarity through the graph, extending beyond collaborative filtering on a flat matrix.

### Drug discovery and biomedicine

Molecules are graphs of atoms and chemical bonds.
GNNs can predict molecular properties such as toxicity, solubility, and binding affinity, while biological knowledge graphs can support predictions about drug-protein interactions and side effects.

### Trust, language, and 3D understanding

Trust-and-abuse systems use connections such as phone numbers, email domains, IP addresses, and payment methods to assess new accounts with little history.
Knowledge graphs add explicit relational information to language applications.
Point clouds, 3D meshes, and scene graphs let GNNs preserve geometric and topological information without forcing it into a grid.

Across these applications, relational structure provides signal that a flat representation cannot capture.
GNNs make graph topology and multi-hop patterns available to the model.

## Tools and implementation

![Four-stage graph machine learning production pipeline: raw tables, graph construction, GNN training, and prediction.](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/nvidia-sdgm.docs.buildwithfern.com/721bf7be96854c42dd31cd15315d8b63c4e4608bb3c67b2055859d4a0a7a119c/img/research/graph-ml-production-pipeline.svg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260920%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260920T164629Z&X-Amz-Expires=604800&X-Amz-Signature=acb9bb5ceaf64facdce9df718e2ab778f31547dc733e5c9fe70b787669a5cf04&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

Building GNNs requires working with sparse adjacency structures and variable-size neighborhoods.
[PyTorch Geometric](https://pytorch-geometric.readthedocs.io/) provides tools for batching graphs, sampling neighborhoods, and running sparse message passing efficiently on GPUs.
It supports many GNN architectures as well as heterogeneous and temporal graphs.

### From research to production

Production GNN systems must construct a graph from raw data, update it as nodes and edges arrive, serve low-latency predictions, and retrain as the graph changes.
These engineering concerns can outweigh the modeling work itself.

KumoRFM applies this workflow to relational data.
Given relational tables and a prediction target, it builds on graph-transformer research to reason across multi-table relationships without requiring per-task feature engineering.

## Limitations and the path to graph transformers

### Over-smoothing

As message-passing layers accumulate, node representations can converge.
With enough depth, nodes in a connected component can become nearly identical, which limits standard GNNs to relatively local patterns.

### Expressiveness and scale

Message-passing GNNs have known expressiveness limits, including limits related to the Weisfeiler-Leman graph-isomorphism test.
Full-batch training also requires the graph in memory, while neighborhood sampling introduces variance and can miss long-range connections.

### Global attention on graphs

![Comparison of standard GNNs with graph transformers, showing local message passing versus global attention and their trade-offs.](https://fdr-prod-docs-files-public.s3.us-east-1.amazonaws.com/nvidia-sdgm.docs.buildwithfern.com/90cd4863a5c9501c15b69f42d83472f3619078d7b7c1866d71e510194481f3fa/img/research/gnn-vs-graph-transformers.svg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Content-Sha256=UNSIGNED-PAYLOAD&X-Amz-Credential=AKIA6KXJSKKNFOCF7G4B%2F20260920%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260920T164629Z&X-Amz-Expires=604800&X-Amz-Signature=5cd9d0123ff901c5a4eaaf2757033399f1008e3b10d19968c235b42b280a8e4d&X-Amz-SignedHeaders=host&x-amz-checksum-mode=ENABLED&x-id=GetObject)

Graph transformers replace strictly local message passing with attention that can consider any node, weighted by learned relevance.
This helps capture long-range dependencies and avoids repeated neighborhood averaging.
The trade-off is that standard self-attention grows quadratically with the number of nodes, so practical graph transformers combine graph structure with sparse attention, positional encodings, and hybrid message-passing mechanisms.

The progression from flat tables to GNNs to graph transformers changes how models use enterprise data.
Graph transformers preserve the structural awareness of GNNs while extending the model's ability to reason across a relational database.

## Further reading

* [Graph Transformers](https://arxiv.org/abs/2107.07999)
* [Graph Convolutional Networks](https://arxiv.org/abs/1609.02907)
* [GraphSAGE](https://arxiv.org/abs/1706.02216)
* [Graph Attention Networks](https://arxiv.org/abs/1710.10903)
* [KumoRFM overview](/rfm/overview)