Dreambooth
DreamBooth is a fine-tuning technique and a solution to personalize large diffusion models like Stable Diffusion, which are powerful but lack the ability to mimic subjects of a given reference set. With DreamBooth, you only need a few images of a specific subject to fine-tune a pretrained text-to-image model, so that it learns to bind a unique identifier with a special subject. This unique identifier can then be used to synthesize fully-novel photorealistic images of the subject contextualized in different scenes.
DreamBooth provides a new prior preservation loss, which enables synthesizing the subject in diverse scenes, poses, views, and lighting conditions that do not appear in the reference images. With this new approach, DreamBooth achieves several previously-unassailable tasks, including subject recontextualization, text-guided view synthesis, appearance modification, and artistic rendering, while still preserving the subject’s key features.
Feature |
Training |
Inference |
|---|---|---|
Data parallelism |
Yes |
N/A |
Tensor parallelism |
No |
No |
Pipeline parallelism |
No |
No |
Sequence parallelism |
No |
No |
Activation checkpointing |
No |
No |
FP32/TF32 |
Yes |
Yes (FP16 enabled by default) |
AMP/FP16 |
Yes |
Yes |
AMP/BF16 |
Yes |
No |
BF16 O2 |
No |
No |
TransformerEngine/FP8 |
No |
No |
Multi-GPU |
Yes |
Yes |
Multi-Node |
Yes |
Yes |
Inference deployment |
N/A |
|
SW stack support |
Slurm DeepOps/Base Command Manager/Base Command Platform |
Slurm DeepOps/Base Command Manager/Base Command Platform |
NVfuser |
No |
N/A |
Distributed Optimizer |
No |
N/A |
TorchInductor |
Yes |
N/A |
Flash Attention |
Yes |
N/A |