Performance in RFD4-Proteina NIM

Execution Backends

The RFD4-Proteina NIM can run three execution backends, selected per deployment (refer to Configuration Alternatives):

• oss_eager: the upstream reference network, uncompiled.
• oss_torch_compile: the same network under torch.compile.
• bioir: an NVIDIA-accelerated trunk built on the BioNeMo Inference Runtime, with its own fused kernels and CUDA-graph capture.

Every response reports which backend actually ran it in model.execution_backend, alongside runtime.cuda_graphs and runtime.cuda_graph_cache, which describe what the loaded runtime currently holds. Results are comparable in kind across backends but not bit-identical, since compilation and BioIR’s own kernels both change floating-point numerics.

Benchmark Configuration

The table in this section comes from a single release-candidate sweep of the synthetic latency suite, 1.0.0-rc.20261012093607, measured 2026-10-12.

Design time is the client’s end-to-end wall clock around one HTTP request. It therefore covers input parsing, sampling, metrics, and response encoding, not only the trunk forward passes. Because the cold run is excluded, these are steady-state figures; refer to Cold-Start Behavior for what the first request into a bucket costs instead.

Replicates are batched into one forward pass, so a batch-n request is a single request returning n designs. Divide by the batch size for a per-design cost. Refer to Replicates and Batching.

Five GPUs sweep batch 1 only — GB10, both 40 GB A100s, L40S, and RTX 6000 Ada Generation — so their tabs carry 15 rows rather than 45, and each one says so.

Performance Across Execution Backends

All three execution backends over the full sweep, at every token bucket and every batch size that GPU was swept at. Select a GPU to switch the table; the two rightmost columns are bioir’s speedup over each OSS baseline, where higher means bioir is faster. A dash marks a shape that did not complete.

Three things the grid shows.

Design time is super-linear in token count, because the pair track is quadratic: on H100 80GB HBM3 with bioir at batch 1, an eightfold rise in tokens, 128 to 1024, costs 15.3x the time. The spread between GPUs widens with it. At 128 tokens the datacenter parts are about 2.3x apart; at 1024 they fall into bands — 11.6 to 13.9 s on the Blackwell datacenter parts, 16.9 to 27.0 s on Hopper, 51.3 to 64.8 s on Ampere, and 76.1 to 97.9 s on the Ada-generation parts. GPU choice therefore matters far more for long designs than for short ones. The GB10 (DGX Spark) sits well outside those bands at 833.9 s, which is what a desktop part costs on the longest design this suite sends.

Batching is close to free at short lengths and close to linear at long ones. On H100 at 128 tokens, oss_eager returns four designs in 8.25 s against 6.82 s for one — 1.21x the time for 4x the output. bioir starts from a far lower per-request floor and so saturates sooner, at 2.08x. By 1024 tokens the GPU is saturated either way and batch 4 costs about 3.9x on both. A deployment serving short designs should batch; one serving long designs gains throughput from batching but little latency.

bioir leads on most parts, by a margin that narrows with both length and batch. On H100 it is 5.83x over oss_eager and 2.67x over oss_torch_compile at 128 tokens and batch 1, 3.39x and 1.61x at the same length and batch 4, and settles near 2.3x and 1.38x from 512 tokens upward regardless of batch. The short-length, batch-1 figures are the most flattering ones and should not be quoted as the general case.

Accuracy Across Execution Backends

The eval suite’s structural-quality metrics, on one H100 80GB HBM3, from the same release candidate as the performance table above. Each backend ran the same 87 conditioned designs. n is how many of those 87 the metric is defined on.

Every design completed on every backend, and the structural-quality means sit within a few thousandths of one another. Results are comparable in kind across backends but not bit-identical — compilation and BioIR’s own kernels both change floating-point numerics — so small differences between the three columns are expected and none of the gaps here is larger than that.

Cold-Start Behavior

The NIM reports ready as soon as its weights are loaded; it does not pre-warm every token-count bucket the oss_torch_compile backend compiles for. This means the first request against a given token bucket after startup pays a Triton JIT compilation cost, once per bucket per container start, and later requests against the same (or a smaller, same-bucket) size run at steady-state speed. This is a deliberate trade: a NIM that is servable in well under a minute, with the compile cost spread across a handful of early requests, versus one that is uniformly fast but takes several minutes to report ready.

Measured on one H100, thinking_medium (the checkpoint’s default recycle strategy), one design:

Two things follow from this table. First, the penalty is per bucket, not “the first request ever”: a deployment serving a wide range of input sizes pays this cost once per distinct bucket its clients exercise, scattered across whichever requests happen to land first. Second, after a bucket is warm, a smaller request that rounds up into that same bucket (694 tokens padding to the 768 bucket already warmed by a 650-token request) pays nothing extra.

If a deployment or a demo needs a specific input size to respond quickly from the first request, send one request of that size to warm its bucket before the size matters.

Memory Footprint

This NIM ships with no refolding oracle built in (refer to Oracles), so the design model’s own memory footprint is what a deployment plans capacity around. That footprint grows with the square of the token count and with the batch size, and sampling.batch_size defaults to sampling.num_replicates: a multi-replicate request is a single large forward pass unless it says otherwise. Refer to Replicates and Batching. A mounted oracle that calls out to an independently hosted structure-prediction NIM (the reference implementations this NIM ships against OpenFold3 and Boltz-2 both work this way) adds no GPU memory cost of its own here, since the model it calls lives entirely in that other service.