> This page is for RFdiffusion, version 2.3.0 (Latest) (default).
> For other versions, use one of these documentation indexes:
> - 2.3.0 (Latest) (default): https://docs.nvidia.com/nim/bionemo/rfdiffusion/2.3.0/llms.txt
> - 2.2.0: https://docs.nvidia.com/nim/bionemo/rfdiffusion/2.2.0/llms.txt
> - 2.0.0: https://docs.nvidia.com/nim/bionemo/rfdiffusion/2.0.0/llms.txt
> - 1.0.0: https://docs.nvidia.com/nim/bionemo/rfdiffusion/1.0.0/llms.txt

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://docs.nvidia.com/nim/bionemo/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://docs.nvidia.com/nim/bionemo/_mcp/server.

# Benchmarking

## Accuracy

Accuracy benchmarking for RFdiffusion involves generating long sequences.
However, given a fixed scaffolding input, there still exist hundreds of
possible configurations at the same length. Therefore, we currently only test
whether the sequences generated under fixed random number conditions in the
NIM container match those generated by the
[public version on GitHub](https://github.com/RosettaCommons/RFdiffusion).

The sequences generated by the GitHub version have undergone manual verification
beforehand. This test assesses NIM's ability to faithfully reproduce these
sequences. It is important to note that accuracy can vary depending on the GPU
microarchitecture. When executed on the same architecture, the outputs match
exactly. When evaluated across different GPU architectures, the mean RMSE (Root Mean
Square Error) between atoms in the reference dataset and the
generated dataset is below 0.64 Ångströms. Here, *mean* is the average RMSE across all benchmark runs; individual runs can vary within the min–max range.

## Performance

The run time of RFdiffusion depends on several factors, including the number of
atoms in the inputs, the number and length of chains, and how many chains need
to be generated.

To represent overall performance effectively, we measure RFdiffusion's main
performance characteristic as the average number of generated amino acids per
second per diffusion step. We calculate this by multiplying the length of the
generated sequence (number of amino acids) by the number of diffusion steps, and
dividing by the measured run time in seconds. For instance,
when considering the generation of a protein binder of a specific length using a
certain number of diffusion steps, one can estimate the time by dividing the
product of the protein length and the number of steps by the performance metric:

time (seconds) ≈ (protein length × number of diffusion steps) ÷ performance (aa/s/step)

This calculation gives an estimate of the time needed to generate one protein
structure. For example, generating a 100-amino-acid binder with 50 diffusion
steps on an H100 (up to 969.1 aa/s/step) takes an estimated (100 × 50) ÷ 969.1,
or about 5.2 seconds.

Starting with 2.0 release of RFdiffusion NIM, the model is optimized using
[NVIDIA Warp](https://developer.nvidia.com/warp-python) and
[NVIDIA TensorRT](https://developer.nvidia.com/tensorrt) frameworks.

With optimizations, RFdiffusion NIM runs up to two times faster than the non-optimized version. Performance varies by GPU architecture. The following measurements (amino acids per second per step) are from v2.3.0:

| GPU         | Microarchitecture | Performance (aa/s/step) |
|-------------|-------------------|--------------------------|
| H100        | Hopper            | up to 969.1              |
| GB200       | Blackwell         | up to 932.2              |
| L40S        | Ada Lovelace      | up to 799.6              |
| A100        | Ampere            | up to 673.4              |
| A10G-20GB   | Ampere            | up to 475.9              |
| A10G-24GB   | Ampere            | up to 448.4              |

## Run the Sample Benchmarking Script

This NIM comes with a benchmarking script that can measure both accuracy
and performance. It is useful to make sure that the neural network provides the same
results on some known proteins.

The script is already packaged in NIM's Docker image. You can view and study the
benchmark using the following command:

```bash
docker run --entrypoint cat nvcr.io/nim/ipd/rfdiffusion:2 /opt/nim/benchmarking.py
```

To execute the benchmark, follow this sequence:

1. Make sure NIM is running as described in the [Quickstart Guide](/nim/bionemo/rfdiffusion/quickstart-guide).

2. Benchmark script automatically downloads test dataset. To save time and
   bandwidth it is recommended to provide a local cache directory. This way the
   script will be able to reuse already downloaded data. Execute the following
   command to set up the cache directory.

```bash
export LOCAL_NIM_CACHE=~/.cache/nim
mkdir -p "$LOCAL_NIM_CACHE"
sudo chmod 0777 -R "$LOCAL_NIM_CACHE"
```

3. Execute the benchmark.

```bash
docker run -it --net host -v "$LOCAL_NIM_CACHE":/opt/nim/.cache --entrypoint "" \
    nvcr.io/nim/ipd/rfdiffusion:2 \
    /opt/nim/benchmarking.py --benchmark-type both
```