InstructPix2Pix

User Guide (Latest Version)

InstructPix2Pix introduces a method for editing images based on human-written instructions. Given an input image and a textual directive, the model follows these instructions to modify the image accordingly.

NeMo Multimodal offers a training pipeline for conditional diffusion models using the edit dataset. Additionally, we provide a tool that generates modified images based on user-written instructions during the inference process.

Feature

Training

Inference

Data parallelism Yes N/A
Tensor parallelism No No
Pipeline parallelism No No
Sequence parallelism No No
Activation checkpointing No No
FP32/TF32 Yes Yes (FP16 enabled by default)
AMP/FP16 Yes Yes
AMP/BF16 Yes No
BF16 O2 No No
TransformerEngine/FP8 No No
Multi-GPU Yes Yes
Multi-Node Yes Yes
Inference deployment N/A NVIDIA Triton supported
SW stack support Slurm DeepOps/Base Command Manager/Base Command Platform Slurm DeepOps/Base Command Manager/Base Command Platform
NVfuser No N/A
Distributed Optimizer No N/A
TorchInductor Yes N/A
Flash Attention Yes N/A
Previous Performance
Next Data Preparation
© | | | | | | |. Last updated on May 30, 2024.