Dreambooth

DreamBooth is a fine-tuning technique and a solution to personalize large diffusion models like Stable Diffusion, which are powerful but lack the ability to mimic subjects of a given reference set. With DreamBooth, you only need a few images of a specific subject to fine-tune a pretrained text-to-image model, so that it learns to bind a unique identifier with a special subject. This unique identifier can then be used to synthesize fully-novel photorealistic images of the subject contextualized in different scenes.

DreamBooth provides a new prior preservation loss, which enables synthesizing the subject in diverse scenes, poses, views, and lighting conditions that do not appear in the reference images. With this new approach, DreamBooth achieves several previously-unassailable tasks, including subject recontextualization, text-guided view synthesis, appearance modification, and artistic rendering, while still preserving the subject’s key features.

Feature

Training

Inference

Data parallelism Yes N/A
Tensor parallelism No No
Pipeline parallelism No No
Sequence parallelism No No
Activation checkpointing No No
FP32/TF32 Yes Yes (FP16 enabled by default)
AMP/FP16 Yes Yes
AMP/BF16 Yes No
BF16 O2 No No
TransformerEngine/FP8 No No
Multi-GPU Yes Yes
Multi-Node Yes Yes
Inference deployment N/A NVIDIA Triton supported
SW stack support Slurm DeepOps/Base Command Manager/Base Command Platform Slurm DeepOps/Base Command Manager/Base Command Platform
NVfuser No N/A
Distributed Optimizer No N/A
TorchInductor Yes N/A
Flash Attention Yes N/A
Previous Performance
Next Data Preparation
© Copyright 2023-2024, NVIDIA. Last updated on Apr 25, 2024.