Contents

GETTING STARTED

• Introduction
  • BioNeMo Framework
  • BioNeMo Models: Overview
• Prerequisites for Using BioNeMo Framework
  • How to Run the Example Commands?
• Quickstart Guide
  • Setup
  • Getting Started on Local Machine
• Getting Started with BCP
  • Training BioNeMo Models with BCP
• BioNeMo Framework Tutorials

BioNeMo CORE

• Fundamentals
  • Overview & Core Features
  • Creating New Models
  • Config Files
  • Command Line Configuration
  • Checkpoints
    • Save-Restore Connectors
    • Modifying a Checkpoint Dictionary
    • Fine Tuning Pre-trained Checkpoints
  • Datasets
  • Weights & Biases Integration
    • API Setup
    • Weights and Biases Charts
• Data Module
  • Overview
  • Reasons for using BioNeMo’s Data Module
  • How to use BioNeMo DataModule
• Validation with a Downstream Task
  • Overview
  • Input Data Format
  • Supported Downstream Tasks
    • Sequence-Level Regression
    • Sequence-Level Classification
    • Token-Level Classification
  • Downstream Task Model Architectures
    • MLP Model
    • ConvNet model
  • Intended Use
    • Setup
    • Model Configuration
    • Universal Parameters
    • SingleValuePredictionCallback Parameters
    • PerTokenPredictionCallback Parameters
    • Optimization Parameters
  • Logging with Weights & Biases and Tensorboard
  • Limitations
• Inference with Nvidia Triton Server
  • Start Inference Server
  • Perform Inference as a Client
  • Detailed Example of PyTriton Inference
• Example of PyTriton Inference
  • Detailed Example with ESM-1nv
  • Extending These Examples

BEST PRACTICES

• Hyperparameter Usage and Tuning
  • General Information
  • Hyperparameter Tuning Tips
  • Recommended Hyperparameter Search Method
  • General Training Tips
• Training Parallelism
  • Supported Parallelism Features
  • Model Parallelism Guidelines

MODELS

• ESM-1nv
• Model Overview
  • Description:
  • References:
  • Model Architecture:
  • Input: (Enter “None” As Needed)
  • Output: (Enter “None” As Needed)
  • Software Integration:
  • Model Version(s):
• Training & Evaluation:
  • Training Dataset:
  • Evaluation Dataset:
  • Inference:
  • Ethical Considerations:
• ESM-2nv
• Model Overview
  • Description:
  • Third-Party Community Consideration
  • References:
  • Model Architecture:
  • Input:
  • Output:
  • Software Integration:
  • Model Version(s):
• Training & Evaluation:
  • Training Dataset:
  • Evaluation Dataset:
  • Inference:
    • Accuracy Benchmarks
    • Training Performance Benchmarks
  • License
• Prott5nv
• Model Overview
  • Description:
  • References:
  • Model Architecture:
  • Input:
  • Output:
  • Software Integration:
  • Model Version(s):
• Training & Evaluation:
  • Training Dataset:
  • Evaluation Dataset:
  • Inference:
  • Ethical Considerations:
  • License
• MegaMolBART
• Model Overview
  • Description:
  • References:
  • Model Architecture:
  • Input:
  • Output:
  • Software Integration:
  • Model Version(s):
• Training & Evaluation:
  • Training Dataset:
  • Evaluation Dataset:
  • Inference:
  • Ethical Considerations:
  • License:
• MolMIM
• Model Overview
  • Description:
  • References:
  • License: TODO @Neha update licenses
  • Model Architecture:
  • Output:
  • Software Integration:
  • Model Version(s):
• Training & Evaluation:
  • Training Dataset:
  • Evaluation Dataset:
  • Inference:
  • Ethical Considerations:
    • Accuracy Benchmarks
    • Training Performance Benchmarks
• EquiDock
• Model Overview
  • Description:
  • Third-Party Community Consideration
  • References:
  • Model Architecture:
  • Input:
  • Output:
  • Software Integration:
  • Model Version(s):
• Training & Evaluation:
  • Training Dataset:
  • Evaluation Dataset:
  • Inference:
  • Ethical Considerations:
  • Benchmarks
    • Accuracy Benchmarks
    • Training Performance Benchmarks
  • License
• DiffDock
• Model Overview
  • Description:
  • Third-Party Community Consideration
  • References:
  • Model Architecture:
  • Input:
  • Output:
  • Software Integration:
  • Model Version(s):
• Evaluation:
  • Evaluation Dataset:
  • Inference:
  • Benchmarks
    • Accuracy Benchmarks
    • Training Performance Benchmarks
  • Limitations
  • License
• DNABERT
• Model Overview
  • Description:
  • References:
  • Model Architecture:
  • Input: (Enter “None” As Needed)
  • Output: (Enter “None” As Needed)
  • Software Integration:
  • Model Version(s):
• Training & Evaluation:
  • Training Dataset:
  • Evaluation Dataset:
  • Inference:
  • Accuracy Benchmarks
  • Performance Benchmarks
  • Ethical Considerations:
• OpenFold
• Model Overview
  • User guide
    • Inference
    • Training
  • Description:
  • References:
  • Model Architecture:
  • Input:
  • Output:
  • Software Integration:
  • Model Version(s):
• Training & Evaluation:
  • Training Dataset:
  • Evaluation Dataset:
  • Inference:
  • Accuracy Benchmarks
    • Cluster specs
    • Parameter settings
    • Initial Training
    • Fine tuning
  • Performance Benchmarks
    • Initial Training
    • Fine-tuning
  • Ethical Considerations:
• sc-FM
• Model Overview
  • Description:
  • References:
  • Model Architecture:
  • Input:
  • Output:
  • Software Integration:
  • Model Version(s):
• Training & Evaluation:
  • Training Dataset:
  • Evaluation Dataset:
  • Inference:
  • Performance Benchmarks:
  • Ethical Considerations:
• Model Benchmarks
  • Protein Sequence Representation
  • SMILES Representation
  • SMILES Generation

DATASETS

• UniProt Dataset
  • ESM-2nv
  • ESM-1nv
• ZINC-15 Dataset
• The Protein Data Bank (PDB)
  • PoseBusters Benchmark buttenschoen2023posebusters
  • The Docking Benchmark 5.5 (DB5.5) vreven2015updates
  • Database of Interacting Protein Structures (DIPS) townshend2019end
  • Continuous Automated Model EvaluatiOn (CAMEO) robin2023automated
  • Derivative datasets in OpenFold training Ahdritz2022.11.20.517210
• Human Reference Genome Version GRCh38.p13
• OpenProteinSet
• MoleculeNet Physical Chemistry Datasets - Lipophilicity, FreeSolv, ESOL
• Genecorpus-30M
• PertNet 2022

TUTORIALS

• DiffDoc: Preparing Workspace and Data for Pre-training
  • Step 1: Create a Workspace
  • Step 2: Create a Local Data Directory
  • Step 3: Mount the NGC Workspace To Your Local Directory
  • Step 4: Prepare Raw PDB Data
  • Step 5: Run Pre-processing of Train Data for Score Model
  • Step 6 : Verify that the data pre-processing is successful in Step 5
  • Step 7: Run Pre-processing of Train Data for Confidence Model
• ESM2: Preparing Workspace and Data for Pre-training
  • Part 1: Create a Workspace
    • Step 1.1: Add Variables to Your Environment
    • Step 1.2: Command to create a workspace
    • Step 1.3: Mount the NGC Workspace To a Directory on Your Machine
  • Part 2: Download the Dataset
    • Step 2.1: Start an Interactive Job on an NGC Compute Instance
    • Step 2.2: Configure NGC client on NGC host
    • Step 2.3: Download Raw Data
    • Step 2.4: Copy Raw Dataset to your NGC Worksapce
  • Part 3: Preprocess the Raw Dataset
    • Step 3.1: Setup
    • Step 3.2: Pre-process the Pre-train Data
    • Step 3.3: Run Pre-processing of FLIP Data
    • Step 3.4: Verify that the data pre-processing is successful
• BioNeMo - MegaMolBART Inferencing for Generative Chemistry
  • Setting up MegaMolBART
    • Prerequisites:
  • Overview
  • BioNeMo-MegaMolBART Inference Functions:
    • Setup the connection to the inference wrapper and providing example compounds
    • Going from SMILES to hidden state
    • SMILES to Embedding
    • Hidden state to SMILES
  • Molecule Generation / Chemical Space Exploration
    • First, let’s define chem_sample — the generation function
    • Generating analogous small molecules and visualizing them
  • Downstream Prediction Model using Learned Embeddings from Pre-trained Model
    • Example: Compound Water Solubility (ESOL) Prediction
    • Preprocessing dataset using cuDF
    • Generating ML models using cuML
• BioNeMo - MolMIM Inferencing for Generative Chemistry
  • Setting up MolMIM
    • Prerequisites:
  • Overview
  • BioNeMo-MolMIM Inference Functions:
    • Setup the connection to the inference wrapper and providing example compounds
    • From SMILES to hidden state
    • SMILES to Embedding
    • Hidden state to SMILES
  • Molecule generation and chemical space exploration
    • Define chem_sample - the generation function
    • Generating analogous small molecules and visualizing them
  • Downstream Prediction Model using Learned Embeddings from Pre-trained Model
    • Example: Compound Water Solubility (ESOL) Prediction
    • Preprocessing dataset using cuDF
    • Generating ML models using cuML
• MolMIM ZINC15 Training Dataset Setup
  • Preparation of a drug-like subset of ZINC15 dataset
  • 1. Download files
  • 2. Process files
    • 2.1 Compute molecular properties
    • 2.2 Get vocabulary compliance
    • 2.3 Filter out undesirable molecules
  • 3. Write filtered output to file
  • 4. Split files into chunks of 10100000
  • 5. Split data into train/val/test
  • [OPTIONAL] 6. Clustering validation and test set for faster evaluation
• Finetune pre-trained models in BioNeMo
  • Setup
  • Finetune MMB model in BioNeMo
    • Finetuning Configuration
    • Loading pretrained model
• MolMIM Property Guided Molecular Optimization Using CMA-ES
  • Setup your environment for this test
  • Load your checkpoint into the molmim inference wrapper
  • Setup user-defined molecule scoring function
  • Define starting molecules
  • Setup the optimizer and wrap the inference API for CMA-ES
  • Tune CMA-ES
  • Run a larger CMA-ES optimization with discovered parameters
  • Explore results
  • How well did our optimization perform?
• Adding the OAS Dataset: Modifying the Dataset Class
  • Setup and Assumptions
  • Configuring the CSV Memory Mapped Dataset
    • Custom YAML Config
    • Testing
    • Results
• Adding the OAS Dataset: Customizing Dataset Object and Dataloader Functions
  • Setup and Assumptions
  • Customizing a collate function
  • Injecting a custom collate object into an existing model.
  • Creating the Dataset object
  • Testing our new collate function
• DataLoader!
• Conclusion and further reading.
• Adding the OAS Dataset: Downloading and Preprocessing
  • Accessing OAS Dataset
  • Downloading and Verifying Data
    • Data Preprocessing Class
    • Custom YAML Config
    • Python Execution Script
    • Testing
    • Results
  • Decompressing OAS Sequence Files
    • Data Preprocessing Class
    • Python Execution Script
    • Testing
    • Results
  • Cleaning and Splitting OAS Sequence Files
    • Data Preprocessing Class
    • Testing
    • Results
  • Optional Variation: Process the Light Chain Data
• Encoder Fine-tuning
  • Overview
  • Setup and Assumptions
  • Getting Started
    • Data Setup
    • Setup Downstream Task Model Class
    • Config YAML
    • Training Script
• DiffDock Model Training using BioNeMo
  • Overview - DiffDock model
  • Setup and Assumptions
  • Requesting compute resources
    • Access to DGX compute resource NGC site or NGC CLI
  • Data Preprocessing for Score Model
    • Downloading and pre-processing the example dataset for score model
  • Model training: Score Model
    • Example dataset
  • Output and Results
    • Train a small score model
    • Train Confidence Model
• EquiDock Model Training using BioNeMo
  • Overview - EquiDock model
  • Setup and Assumptions
  • Requesting compute resources
    • Access to DGX compute resource NGC site or NGC CLI
  • Data Preprocessing
    • Downloading and pre-processing the dataset
  • Model training
    • Example dataset
    • Single-node or Multi-node setup
  • Output and Results
• ESM1nv Model Training using BioNeMo
  • Overview - ESM1nv model
  • Setup and Assumptions
  • Requesting compute resources
    • Access to DGX compute resource NGC site or NGC CLI
  • Data Preprocessing
    • Downloading and pre-processing the dataset
  • Model training
    • Example dataset
    • Single-node or Multi-node setup
  • Output and Results
• ESM2nv Model Training using BioNeMo
  • Overview - ESM2nv model
• Prerequisites
  • Requesting compute resources
    • Access to DGX compute resources via NGC site or NGC CLI
    • Launch an Interactive NGC Job
  • Data Pre-processing for a Down-sampled Dataset
    • Downloading and pre-processing the example dataset
  • Model training
    • Single-node or Multi-node setup
  • Output and Results
    • Data Pre-processing for the Full Dataset [Optional]
    • Alternative datasets [Optional]
• MegaMolBART Model Training using BioNeMo
  • Overview - MegaMolBART model
  • Setup and Assumptions
  • Requesting compute resources
    • Access to DGX compute resource NGC site or NGC CLI
  • Data Preprocessing
    • Downloading and pre-processing the dataset
  • Model training
    • Example dataset
    • Single-node or Multi-node setup
  • Output and Results
• MolMIM Model Training using BioNeMo
  • Overview - MolMIM model
  • Setup and Assumptions
  • Requesting compute resources
    • Access to DGX compute resource NGC site or NGC CLI
  • Data Preprocessing
    • Downloading and pre-processing the dataset
  • Model training
    • Example dataset
    • Single-node or Multi-node setup
    • Continuing training from an existing checkpoint
  • Output and Results
• Finetuning LLM in BioNeMo for a Downstream Task
  • Prerequisites
  • Example: Train Model for Compound Water Solubility (ESOL) Prediction using MMB Pretrained Model
    • Results and Logging
• Generating Embeddings for Protein Clustering
  • Prerequisites:
  • Download the Pre-trained Models
  • Run the container
  • Download the data
    • Using BioNeMo features to download UniRef50
    • Alternative datasets
  • Generating Embeddings
  • Example
    • ESM-1nv limits
    • Data Cleaning
• Conclusion
• Generating Embeddings for Protein Clustering
  • Prerequisites
    • Launch a Docker container
    • Prepare data
    • Download pre-trained Models
  • Run inference on protein sequence data
  • Protein sequence clustering example
    • ESM-2nv limits
    • Data Cleaning
• Conclusion
• Training LLM for a Custom Downstram Task: Retrosynthesis Prediction using MegaMolBART
  • Overview
  • Setup and Assumptions
  • Dataset
  • Training
    • Processing the dataset
  • Inference

APPENDIX

• Frequently Asked Questions (FAQ)
• Bibliography
• Release Notes
  • BioNeMo Framework v1.4
    • New Models
    • New Features
    • Bug fixes and Improvements
    • Known Issues
  • BioNeMo Framework v1.3
    • New Models
    • New Features
    • Bug fixes and Improvements
    • Known Issues
  • BioNeMo Framework v1.2
  • New Models
    • New Features
    • Bug fixes and Improvements
    • Known Issues
  • BioNeMo Framework v1.1
  • New Models
    • New Features
    • Bug fixes and Improvements
    • Security Notice
  • BioNeMo Framework v1.0
  • New Models
    • New Features
  • BioNeMo Framework v0.4.0
    • New Models
    • New Features
    • Known Issues
    • New APIs