Performance in Boltz-2 NIM

NIM Accuracy

The Boltz-2 NIM is based on the state-of-the-art Boltz-2 architecture for biomolecular structure prediction. The NIM’s accuracy should match that of the reference implementation when using equivalent parameters and inputs.

The accuracy of the NIM is measured by structural quality metrics such as lddt. These scores help assess the reliability of the predicted structures.

Factors Affecting NIM Performance

The performance of the Boltz-2 NIM is determined by several key factors:

Hardware Factors

• Number and type of GPUs: More GPUs generally improve throughput for concurrent requests
• GPU memory: Larger proteins and complexes require more GPU memory
• Storage speed: Fast NVMe SSD storage improves model loading and caching performance

Input Complexity

• Sequence length: Runtime scales approximately quadratically with total sequence length
• Number of chains: Multi-chain complexes require more computation than single chains
• Ligands and constraints: Additional molecular components increase computational cost

Model Parameters

• Sampling steps: Higher values improve quality but significantly increase runtime
• Recycling steps: More iterations improve accuracy with modest runtime increase
• Diffusion samples: Multiple samples provide diversity but multiply computational cost

Performance Characteristics

Typical Runtimes

For reference, runtime varies with sequence length, backend selection, and template usage. Detailed benchmark metrics for the current release are reported in the tables below.

Performance Metrics (Boltz2 v1.6.0)

The tables below report benchmark results from Boltz2 v1.6.0 performance runs.

Configuration

Table 1: Performance Across the Supported NVIDIA Hardware Units

The table below reports predict_time (seconds) with TensorRT and no templates.

Table 2: Performance Across Optimization Backends

The table below compares H100 performance between OSS and TensorRT backends without templates.

Table 3: Performance Impact From Structural Templates

The table below reports H100 TensorRT performance with and without templates.

Boltz2 v1.6.0 Performance Results on H100

Typical Runtimes

For reference, approximate runtimes on NVIDIA H100 80GB HBM3:

Structure Prediction (Boltz2 v1.6.0 on H100):

• ~200 residues: 1.72 seconds (TensorRT) / 11.07 seconds (OSS)
• ~500-700 residues: 6.63-9.39 seconds (TensorRT) / 12.24-19.78 seconds (OSS)
• ~1200-1500 residues: 24.67-35.64 seconds (TensorRT) / 50.32-77.02 seconds (OSS)
• ~1500-1800 residues: 37.96-51.69 seconds (TensorRT) / 64.28-85.22 seconds (OSS)
• ~2000 residues: 79.55 seconds (TensorRT) / 123.07 seconds (OSS)

Binding Affinity Prediction (Boltz2 v1.6.0 on H100):

• ~200 residues: 6.21 seconds (TensorRT) / 18.52 seconds (OSS)
• ~500-700 residues: 10.74-14.16 seconds (TensorRT) / 26.96-34.03 seconds (OSS)
• ~1200-1500 residues: 29.69-43.99 seconds (TensorRT) / 53.13-87.57 seconds (OSS)
• ~1500-1800 residues: 43.77-54.10 seconds (TensorRT) / 80.94-93.29 seconds (OSS)
• ~2000 residues: 81.19 seconds (TensorRT) / 130.67 seconds (OSS)

Structure Prediction Performance

The following table shows runtime performance for Boltz2 v1.6.0 on NVIDIA H100 GPUs for structure prediction across different sequence lengths:

Binding Affinity Prediction Performance

The following table shows runtime performance for binding affinity prediction on NVIDIA H100 GPUs:

Performance Analysis

Key Observations:

• TensorRT Optimization: TensorRT consistently outperforms OSS for both structure and affinity prediction across all 18 H100 benchmark cases.
• Structure Prediction: Speed up ranges from 1.45x to 6.44x; H100 runtime ranges from 1.72s to 79.55s (TensorRT) vs 6.80s to 123.07s (OSS).
• Binding Affinity Prediction: Speed up ranges from 1.61x to 2.98x; H100 runtime ranges from 6.21s to 81.19s (TensorRT) vs 18.52s to 130.67s (OSS).
• Scaling Behavior: Runtime increases with sequence length for both OSS and TensorRT backends.

Recommended Configuration:

• For development/testing: Use OSS for easier debugging and TensorRT for latency validation.
• For production: Use TensorRT to maximize throughput and minimize inference latency.
• For large proteins (>1500 residues): Prefer TensorRT and provision sufficient GPU memory headroom.

Performance Testing

You can test basic performance and functionality with protein structure and binding affinity prediction:

import requests
import json
import time
def test_boltz2_structure_performance():
    """Test basic protein structure prediction performance."""
    url = "http://localhost:8000/biology/mit/boltz2/predict"
    # Test protein: Green Fluorescent Protein (GFP) - ~240 residues
    test_sequence = (
        "MSKGEELFTGVVPILVELDGDVNGHKFSVSGEGEGDATYGKLTLKFICTTGKLPVPWPTLVTTFCYGD"
        "QIQEQYKGIPLDGDQVQAVNGHEFEIEGEGEGRPYEGTQTAQLNKFCDKLPVMHYKQFFDSGNYNTLS"
        "AKAGFPFKVPHTYNNSSFVVKQKPGMVFKFIHGKDPGLNGQTVFLMVGGISQNLSGSSNLGVGYTFVQ"
        "KTSVLLESEIKKRLRGFHTRGAVTQGLHQFVNLPTLVTQVLDGDMSQLLQVT"
    )
    data = {
        "polymers": [
            {
                "id": "A",
                "molecule_type": "protein",
                "sequence": test_sequence
            }
        ],
        "recycling_steps": 3,
        "sampling_steps": 50,
        "diffusion_samples": 1,
        "step_scale": 1.638,
        "output_format": "mmcif"
    }
    print("Starting Boltz-2 structure prediction performance test...")
    start_time = time.time()
    response = requests.post(url, json=data)
    end_time = time.time()
    runtime = end_time - start_time
    if response.status_code == 200:
        result = response.json()
        confidence = result.get('confidence_scores', [0])[0]
        print(f"✓ Structure prediction successful")
        print(f"✓ Runtime: {runtime:.1f} seconds")
        print(f"✓ Confidence score: {confidence:.3f}")
        print(f"✓ Structure format: {result['structures'][0]['format']}")
        return True
    else:
        print(f"✗ Structure prediction failed: {response.status_code}")
        print(f"✗ Error: {response.text}")
        return False
def test_boltz2_affinity_performance():
    """Test binding affinity prediction performance."""
    url = "http://localhost:8000/biology/mit/boltz2/predict"
    # Smaller test protein for affinity testing - ~140 residues
    hemoglobin_alpha = (
        "MVLSPADKTNVKAAWGKVGAHAGEYGAEALERMFLSFPTTKTYFPHFDLSHGSAQVKGHGKKVADALT"
        "NAVAHVDDMPNALSALSDLHAHKLRVDPVNFKLLSHCLLVTLAAHLPAEFTPAVHASLDKFLASVSTV"
        "LTSKYR"
    )
    data = {
        "polymers": [
            {
                "id": "A",
                "molecule_type": "protein",
                "sequence": hemoglobin_alpha
            }
        ],
        "ligands": [
            {
                "id": "HEME",
                "smiles": "[Fe+2].C1=CC2=NC1=CC3=NC(=CC4=NC(=CC5=NC(=C2)C=C5)C=C4)C=C3",
                "predict_affinity": True
            }
        ],
        "recycling_steps": 3,
        "sampling_steps": 50,
        "sampling_steps_affinity": 100,  # Reduced from default (200) for faster performance testing
        "diffusion_samples_affinity": 3,  # Reduced from default (5) for faster performance testing
        "output_format": "mmcif"
    }
    print("Starting Boltz-2 binding affinity performance test...")
    start_time = time.time()
    response = requests.post(url, json=data)
    end_time = time.time()
    runtime = end_time - start_time
    if response.status_code == 200:
        result = response.json()
        confidence = result.get('confidence_scores', [0])[0]
        print(f"✓ Affinity prediction successful")
        print(f"✓ Runtime: {runtime:.1f} seconds")
        print(f"✓ Confidence score: {confidence:.3f}")
        if result.get("affinities"):
            for ligand_id, affinity_data in result["affinities"].items():
                if affinity_data.get("affinity_pic50"):
                    print(f"✓ Predicted pIC50 for {ligand_id}: {affinity_data['affinity_pic50'][0]:.2f} kcal/mol")
        return True
    else:
        print(f"✗ Affinity prediction failed: {response.status_code}")
        print(f"✗ Error: {response.text}")
        return False
if __name__ == "__main__":
    # Test both structure and affinity prediction
    structure_success = test_boltz2_structure_performance()
    print("\n" + "="*50 + "\n")
    affinity_success = test_boltz2_affinity_performance()
    if structure_success and affinity_success:
        print("\n✓ All performance tests passed!")
    else:
        print("\n✗ Some performance tests failed.")

Performance Optimization Tips

1. For development/testing: Use faster settings

   "recycling_steps": 2,
   "sampling_steps": 25

2. For production quality: Use higher quality settings

   "recycling_steps": 5,
   "sampling_steps": 100

3. For batch processing: Submit multiple concurrent requests with default settings

4. For very large proteins (>1000 residues): Consider domain-based approaches or consult the literature for handling strategies

Troubleshooting Performance Issues

Common Issues and Solutions

General Performance Issues

• Out of memory errors: Reduce sequence length, decrease sampling steps, or use fewer concurrent requests
• Slow performance: Ensure fast storage (NVMe SSD), sufficient CPU cores (12+ per GPU), and adequate system RAM (48+ GB per GPU)
• Poor quality predictions: Increase sampling steps, recycling steps, or check input sequence quality

Binding Affinity Specific Issues

• Affinity prediction timeouts: Reduce sampling_steps_affinity (default: 200) and diffusion_samples_affinity (default: 5) for faster results (e.g., 50 and 1)
• Unrealistic affinity values: Enable affinity_mw_correction for metal-containing ligands and verify SMILES format
• Memory errors with affinity: Binding affinity requires 2-3x more memory than structure prediction alone
• Inconsistent affinity results: Ensure you use default values (diffusion_samples_affinity=5) or increase it to 7-10 for even more reliable estimates
• Cannot predict affinity for multiple ligands: Only one ligand per request can have predict_affinity=True