Framework Inference
For DreamBooth, the inference script generates images from text prompts defined in the config file, similar to section
5.7.3. Note that, dreambooth is a fine-tuning model based on diffusion models to link a special token with certain
subject, so make sure the special token you trained on is included in the text prompt. For
example, a photo of sks dog sleeping
.
To enable the inference stage with DreamBooth, configure the configuration files:
In the
defaults
section ofconf/config.yaml
, update thefw_inference
field to point to the desired DreamBooth inference configuration file. For example, if you want to use thedreambooth/text2img.yaml
configuration, change thefw_inference
field todreambooth/text2img
.defaults: - fw_inference: dreambooth/text2img ...
In the
stages
field ofconf/config.yaml
, make sure thefw_inference
stage is included. For example,stages: - fw_inference ...
Configure
prompts
andnum_images_per_prompt
fields ofconf/fw_inference/dreambooth/text2img.yaml
. Setmodel.restore_from_path
to the ckpt generated from dreambooth training.
Remarks:
Please refer to DreamBooth Training , the inference stage of DreamBooth should be conducted subsequent to the DreamBooth conversion process. This conversion transforms the DreamBooth ckpt into a ‘.nemo’ format and meanwhile remapping the parameter keys into Stable Diffusion style, allowing for a consistent inference pipeline.