Tutorials

Image-to-Image Editing

View as Markdown
Run in Google Colab

🎨 Data Designer Tutorial: Image-to-Image Editing

📚 What you'll learn

This notebook shows how to chain image generation columns: first generate animal portraits from text, then edit those generated images by adding accessories and changing styles—all without loading external datasets.

  • 🖼️ Text-to-image generation: Generate images from text prompts
  • 🔗 Chaining image columns: Use ImageContext to pass generated images to a follow-up editing column
  • 🎲 Sampler-driven diversity: Combine sampled accessories and settings for varied edits

This tutorial uses an autoregressive model (one that supports both text-to-image and image-to-image generation via the chat completions API). Diffusion models (DALL·E, Stable Diffusion, etc.) do not support image context—see Tutorial 5 for text-to-image generation with diffusion models.

Prerequisites: This tutorial uses OpenRouter with the Flux 2 Pro model. Set OPENROUTER_API_KEY in your environment before running.

If this is your first time using Data Designer, we recommend starting with the first notebook in this tutorial series.

📦 Import Data Designer

  • data_designer.config provides the configuration API.
  • DataDesigner is the main interface for generation.
Python
1import base64
2from pathlib import Path
3 
4from IPython.display import Image as IPImage
5from IPython.display import display
6 
7import data_designer.config as dd
8from data_designer.interface import DataDesigner
9

⚙️ Initialize the Data Designer interface

We initialize Data Designer without arguments here—the image model is configured explicitly in the next cell.

Python
1data_designer = DataDesigner()
2

🎛️ Define an image model

We need an autoregressive model that supports both text-to-image and image-to-image generation via the chat completions API. This lets us generate images from text and then pass those images as context for editing.

  • Use ImageInferenceParams so Data Designer treats this model as an image generator.
  • Image-specific options are model-dependent; pass them via extra_body.

Note: This tutorial uses the Flux 2 Pro model via OpenRouter. Set OPENROUTER_API_KEY in your environment.

Python
1MODEL_PROVIDER = "openrouter"
2MODEL_ID = "black-forest-labs/flux.2-pro"
3MODEL_ALIAS = "image-model"
4 
5model_configs = [
6    dd.ModelConfig(
7        alias=MODEL_ALIAS,
8        model=MODEL_ID,
9        provider=MODEL_PROVIDER,
10        inference_parameters=dd.ImageInferenceParams(
11            extra_body={"height": 512, "width": 512},
12        ),
13    )
14]
15

🏗️ Build the configuration

We chain two image generation columns:

  1. Sampler columns — randomly sample animal types, accessories, settings, and art styles
  2. First image column — generate an animal portrait from a text prompt
  3. Second image column with context — edit the generated portrait using ImageContext
Python
1config_builder = dd.DataDesignerConfigBuilder(model_configs=model_configs)
2 
3# 1. Sampler columns for diversity
4config_builder.add_column(
5    dd.SamplerColumnConfig(
6        name="animal",
7        sampler_type=dd.SamplerType.CATEGORY,
8        params=dd.CategorySamplerParams(
9            values=["cat", "dog", "fox", "owl", "rabbit", "panda"],
10        ),
11    )
12)
13 
14config_builder.add_column(
15    dd.SamplerColumnConfig(
16        name="accessory",
17        sampler_type=dd.SamplerType.CATEGORY,
18        params=dd.CategorySamplerParams(
19            values=[
20                "a tiny top hat",
21                "oversized sunglasses",
22                "a red bow tie",
23                "a knitted beanie",
24                "a flower crown",
25                "a monocle and mustache",
26                "a pirate hat and eye patch",
27                "a chef hat",
28            ],
29        ),
30    )
31)
32 
33config_builder.add_column(
34    dd.SamplerColumnConfig(
35        name="setting",
36        sampler_type=dd.SamplerType.CATEGORY,
37        params=dd.CategorySamplerParams(
38            values=[
39                "a cozy living room",
40                "a sunny park",
41                "a photo studio with soft lighting",
42                "a red carpet event",
43                "a holiday card backdrop with snowflakes",
44                "a tropical beach at sunset",
45            ],
46        ),
47    )
48)
49 
50config_builder.add_column(
51    dd.SamplerColumnConfig(
52        name="art_style",
53        sampler_type=dd.SamplerType.CATEGORY,
54        params=dd.CategorySamplerParams(
55            values=[
56                "a photorealistic style",
57                "a Disney Pixar 3D render",
58                "a watercolor painting",
59                "a pop art poster",
60            ],
61        ),
62    )
63)
64 
65# 2. Generate animal portrait from text
66config_builder.add_column(
67    dd.ImageColumnConfig(
68        name="animal_portrait",
69        prompt="A close-up portrait photograph of a {{ animal }} looking at the camera, studio lighting, high quality.",
70        model_alias=MODEL_ALIAS,
71    )
72)
73 
74# 3. Edit the generated portrait
75config_builder.add_column(
76    dd.ImageColumnConfig(
77        name="edited_portrait",
78        prompt=(
79            "Edit this {{ animal }} portrait photo. "
80            "Add {{ accessory }} on the animal. "
81            "Place the {{ animal }} in {{ setting }}. "
82            "Render the result in {{ art_style }}. "
83            "Keep the animal's face, expression, and features faithful to the original photo."
84        ),
85        model_alias=MODEL_ALIAS,
86        multi_modal_context=[dd.ImageContext(column_name="animal_portrait")],
87    )
88)
89 
90data_designer.validate(config_builder)
91
Output
[13:27:10] [INFO] ✅ Validation passed

🔁 Preview: quick iteration

In preview mode, generated images are stored as base64 strings in the dataframe. Use this to iterate on your prompts, accessories, and sampler values before scaling up.

Python
1preview = data_designer.preview(config_builder, num_records=2)
2
Output
[13:27:10] [INFO] 👀 Preview generation in progress
[13:27:10] [INFO]   |-- 🔒 Jinja rendering engine: secure
[13:27:10] [INFO] ✅ Validation passed
[13:27:10] [INFO] ⛓️ Sorting column configs into a Directed Acyclic Graph
[13:27:10] [INFO] 🩺 Running health checks for models...
[13:27:10] [INFO]   |-- 👀 Checking 'black-forest-labs/flux.2-pro' in provider named 'openrouter' for model alias 'image-model'...
[13:27:18] [INFO]   |-- ✅ Passed!
[13:27:18] [INFO] ⚡ DATA_DESIGNER_ASYNC_ENGINE is enabled - using async task-queue preview
[13:27:18] [INFO] 🖼️ image model config for column 'animal_portrait'
[13:27:18] [INFO]   |-- model: 'black-forest-labs/flux.2-pro'
[13:27:18] [INFO]   |-- model alias: 'image-model'
[13:27:18] [INFO]   |-- model provider: 'openrouter'
[13:27:18] [INFO]   |-- inference parameters:
[13:27:18] [INFO]   |  |-- generation_type=image
[13:27:18] [INFO]   |  |-- max_parallel_requests=4
[13:27:18] [INFO]   |  |-- extra_body={'height': 512, 'width': 512}
[13:27:18] [INFO] 🖼️ image model config for column 'edited_portrait'
[13:27:18] [INFO]   |-- model: 'black-forest-labs/flux.2-pro'
[13:27:18] [INFO]   |-- model alias: 'image-model'
[13:27:18] [INFO]   |-- model provider: 'openrouter'
[13:27:18] [INFO]   |-- inference parameters:
[13:27:18] [INFO]   |  |-- generation_type=image
[13:27:18] [INFO]   |  |-- max_parallel_requests=4
[13:27:18] [INFO]   |  |-- extra_body={'height': 512, 'width': 512}
[13:27:18] [INFO] ⚡️ Async generation: 2 column(s) (animal_portrait, edited_portrait), 4 tasks across 1 row group(s)
[13:27:18] [INFO] 🚀 (1/1) Dispatching with 2 records
[13:27:18] [INFO] 🎲 (1/1) Preparing samplers to generate 2 records across 4 columns
[13:27:26] [INFO] 📊 Progress [7.9s]:
[13:27:26] [INFO]   |-- ⛅ animal_portrait: 1/2 (50%) 0.1 rec/s
[13:27:26] [INFO]   |-- 🥚 edited_portrait: 0/2 (0%) 0.0 rec/s
[13:27:41] [INFO] 📊 Progress [22.8s]:
[13:27:41] [INFO]   |-- ☀️ animal_portrait: 2/2 (100%) 0.1 rec/s
[13:27:41] [INFO]   |-- 🐥 edited_portrait: 1/2 (50%) 0.0 rec/s
[13:27:41] [INFO] 📊 Progress [23.2s]:
[13:27:41] [INFO]   |-- ☀️ animal_portrait: 2/2 (100%) 0.1 rec/s
[13:27:41] [INFO]   |-- 🐔 edited_portrait: 2/2 (100%) 0.1 rec/s
[13:27:41] [INFO] ✅ Async generation complete [23.2s]: 4 ok, 0 failed across 2 column(s)
[13:27:41] [INFO] 📊 Model usage summary:
[13:27:41] [INFO]   |-- model: black-forest-labs/flux.2-pro
[13:27:41] [INFO]   |-- tokens: input=9203, output=12288, reasoning=0, total=21491, tps=924
[13:27:41] [INFO]   |-- requests: success=4, failed=0, total=4, rpm=10
[13:27:41] [INFO]   |-- images: total=4
[13:27:41] [INFO] 📐 Measuring dataset column statistics:
[13:27:41] [INFO]   |-- 🎲 column: 'animal'
[13:27:41] [INFO]   |-- 🎲 column: 'accessory'
[13:27:41] [INFO]   |-- 🎲 column: 'setting'
[13:27:41] [INFO]   |-- 🎲 column: 'art_style'
[13:27:41] [INFO]   |-- 🖼️ column: 'animal_portrait'
[13:27:41] [INFO]   |-- 🖼️ column: 'edited_portrait'
[13:27:41] [INFO] 🙌 Preview complete!
Python
1for i in range(len(preview.dataset)):
2    preview.display_sample_record()
3
Output
[index: 0]
                                                                                                              
                                              Generated Columns                                               
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Name                          ┃ Value                                                                      ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ animal                        │ rabbit                                                                     │
├───────────────────────────────┼────────────────────────────────────────────────────────────────────────────┤
│ accessory                     │ a flower crown                                                             │
├───────────────────────────────┼────────────────────────────────────────────────────────────────────────────┤
│ setting                       │ a tropical beach at sunset                                                 │
├───────────────────────────────┼────────────────────────────────────────────────────────────────────────────┤
│ art_style                     │ a pop art poster                                                           │
└───────────────────────────────┴────────────────────────────────────────────────────────────────────────────┘
                                                                                                              
                                                                                                              
                                                    Images                                                    
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Name                                   ┃ Preview                                                           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ animal_portrait                        │ [0] <base64, 1802580 chars>                                       │
├────────────────────────────────────────┼───────────────────────────────────────────────────────────────────┤
│ edited_portrait                        │ [0] <base64, 2075348 chars>                                       │
└────────────────────────────────────────┴───────────────────────────────────────────────────────────────────┘
🖼️ animal_portrait[0]
🖼️ edited_portrait[0]
[index: 1]
                                                                                                              
                                              Generated Columns                                               
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Name                          ┃ Value                                                                      ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ animal                        │ owl                                                                        │
├───────────────────────────────┼────────────────────────────────────────────────────────────────────────────┤
│ accessory                     │ a chef hat                                                                 │
├───────────────────────────────┼────────────────────────────────────────────────────────────────────────────┤
│ setting                       │ a tropical beach at sunset                                                 │
├───────────────────────────────┼────────────────────────────────────────────────────────────────────────────┤
│ art_style                     │ a pop art poster                                                           │
└───────────────────────────────┴────────────────────────────────────────────────────────────────────────────┘
                                                                                                              
                                                                                                              
                                                    Images                                                    
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Name                                   ┃ Preview                                                           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ animal_portrait                        │ [0] <base64, 1589416 chars>                                       │
├────────────────────────────────────────┼───────────────────────────────────────────────────────────────────┤
│ edited_portrait                        │ [0] <base64, 1972132 chars>                                       │
└────────────────────────────────────────┴───────────────────────────────────────────────────────────────────┘
🖼️ animal_portrait[0]
🖼️ edited_portrait[0]
Python
1preview.dataset
2
Output
animal accessory setting art_style animal_portrait edited_portrait
0 rabbit a flower crown a tropical beach at sunset a pop art poster [iVBORw0KGgoAAAANSUhEUgAABAAAAAMACAIAAAA12IJaA... [iVBORw0KGgoAAAANSUhEUgAABAAAAAMACAIAAAA12IJaA...
1 owl a chef hat a tropical beach at sunset a pop art poster [iVBORw0KGgoAAAANSUhEUgAABAAAAAMACAIAAAA12IJaA... [iVBORw0KGgoAAAANSUhEUgAABAAAAAMACAIAAAA12IJaA...

🔎 Compare original vs edited

Let's display the generated animal portraits next to their edited versions.

Python
1def display_image(image_value, base_path: Path | None = None) -> None:
2    """Display an image from base64 (preview mode) or file path (create mode)."""
3    values = [image_value] if isinstance(image_value, str) else list(image_value)
4    for value in values:
5        if base_path is not None:
6            display(IPImage(filename=str(base_path / value)))
7        else:
8            display(IPImage(data=base64.b64decode(value)))
9 
10 
11def display_before_after(row, index: int, base_path: Path | None = None) -> None:
12    """Display original portrait vs edited version for a single record."""
13    print(f"\n{'=' * 60}")
14    print(f"Record {index}: {row['animal']} wearing {row['accessory']}")
15    print(f"Setting: {row['setting']}, Style: {row['art_style']}")
16    print(f"{'=' * 60}")
17 
18    print("\n📷 Generated portrait:")
19    display_image(row["animal_portrait"], base_path)
20 
21    print("\n🎨 Edited version:")
22    display_image(row["edited_portrait"], base_path)
23
Python
1for index, row in preview.dataset.iterrows():
2    display_before_after(row, index)
3
Output

============================================================
Record 0: rabbit wearing a flower crown
Setting: a tropical beach at sunset, Style: a pop art poster
============================================================

📷 Generated portrait:
Output

🎨 Edited version:
Output

============================================================
Record 1: owl wearing a chef hat
Setting: a tropical beach at sunset, Style: a pop art poster
============================================================

📷 Generated portrait:
Output

🎨 Edited version:
Output

🆙 Create at scale

In create mode, images are saved to disk in images/<column_name>/ folders with UUID filenames. The dataframe stores relative paths. ImageContext auto-detection handles this transparently—generated file paths are resolved to base64 before being sent to the model for editing.

Python
1results = data_designer.create(config_builder, num_records=5, dataset_name="tutorial-6-edited-images")
2
Output
[13:27:42] [INFO] 🎨 Creating Data Designer dataset
[13:27:42] [INFO]   |-- 🔒 Jinja rendering engine: secure
[13:27:42] [INFO] ✅ Validation passed
[13:27:42] [INFO] ⛓️ Sorting column configs into a Directed Acyclic Graph
[13:27:42] [INFO] 🩺 Running health checks for models...
[13:27:42] [INFO]   |-- 👀 Checking 'black-forest-labs/flux.2-pro' in provider named 'openrouter' for model alias 'image-model'...
[13:27:48] [INFO]   |-- ✅ Passed!
[13:27:48] [INFO] ⚡ DATA_DESIGNER_ASYNC_ENGINE is enabled - using async task-queue builder
[13:27:48] [INFO] 🖼️ image model config for column 'animal_portrait'
[13:27:48] [INFO]   |-- model: 'black-forest-labs/flux.2-pro'
[13:27:48] [INFO]   |-- model alias: 'image-model'
[13:27:48] [INFO]   |-- model provider: 'openrouter'
[13:27:48] [INFO]   |-- inference parameters:
[13:27:48] [INFO]   |  |-- generation_type=image
[13:27:48] [INFO]   |  |-- max_parallel_requests=4
[13:27:48] [INFO]   |  |-- extra_body={'height': 512, 'width': 512}
[13:27:48] [INFO] 🖼️ image model config for column 'edited_portrait'
[13:27:48] [INFO]   |-- model: 'black-forest-labs/flux.2-pro'
[13:27:48] [INFO]   |-- model alias: 'image-model'
[13:27:48] [INFO]   |-- model provider: 'openrouter'
[13:27:48] [INFO]   |-- inference parameters:
[13:27:48] [INFO]   |  |-- generation_type=image
[13:27:48] [INFO]   |  |-- max_parallel_requests=4
[13:27:48] [INFO]   |  |-- extra_body={'height': 512, 'width': 512}
[13:27:48] [INFO] ⚡️ Async generation: 2 column(s) (animal_portrait, edited_portrait), 10 tasks across 1 row group(s)
[13:27:48] [INFO] 🚀 (1/1) Dispatching with 5 records
[13:27:48] [INFO] 🎲 (1/1) Preparing samplers to generate 5 records across 4 columns
[13:27:56] [INFO] 📊 Progress [7.6s]:
[13:27:56] [INFO]   |-- 🐱 animal_portrait: 1/5 (20%) 0.1 rec/s
[13:27:56] [INFO]   |-- 🥚 edited_portrait: 0/5 (0%) 0.0 rec/s
[13:28:01] [INFO] 📊 Progress [12.9s]:
[13:28:01] [INFO]   |-- 😸 animal_portrait: 3/5 (60%) 0.2 rec/s
[13:28:01] [INFO]   |-- 🥚 edited_portrait: 0/5 (0%) 0.0 rec/s
[13:28:07] [INFO] 📊 Progress [19.0s]:
[13:28:07] [INFO]   |-- 😸 animal_portrait: 3/5 (60%) 0.2 rec/s
[13:28:07] [INFO]   |-- 🥚 edited_portrait: 1/5 (20%) 0.1 rec/s
[13:28:16] [INFO] 📊 Progress [28.0s]:
[13:28:16] [INFO]   |-- 🦁 animal_portrait: 5/5 (100%) 0.2 rec/s
[13:28:16] [INFO]   |-- 🥚 edited_portrait: 1/5 (20%) 0.0 rec/s
[13:28:22] [INFO] 📊 Progress [34.0s]:
[13:28:22] [INFO]   |-- 🦁 animal_portrait: 5/5 (100%) 0.1 rec/s
[13:28:22] [INFO]   |-- 🐤 edited_portrait: 4/5 (80%) 0.1 rec/s
[13:28:34] [INFO] 📊 Progress [45.6s]:
[13:28:34] [INFO]   |-- 🦁 animal_portrait: 5/5 (100%) 0.1 rec/s
[13:28:34] [INFO]   |-- 🐔 edited_portrait: 5/5 (100%) 0.1 rec/s
[13:28:34] [INFO] ✅ Async generation complete [45.6s]: 10 ok, 0 failed across 2 column(s)
[13:28:34] [INFO] 📊 Model usage summary:
[13:28:34] [INFO]   |-- model: black-forest-labs/flux.2-pro
[13:28:34] [INFO]   |-- tokens: input=23022, output=30720, reasoning=0, total=53742, tps=1174
[13:28:34] [INFO]   |-- requests: success=10, failed=0, total=10, rpm=13
[13:28:34] [INFO]   |-- images: total=10
[13:28:34] [INFO] 📐 Measuring dataset column statistics:
[13:28:34] [INFO]   |-- 🎲 column: 'animal'
[13:28:34] [INFO]   |-- 🎲 column: 'accessory'
[13:28:34] [INFO]   |-- 🎲 column: 'setting'
[13:28:34] [INFO]   |-- 🎲 column: 'art_style'
[13:28:34] [INFO]   |-- 🖼️ column: 'animal_portrait'
[13:28:34] [INFO]   |-- 🖼️ column: 'edited_portrait'
Python
1dataset = results.load_dataset()
2dataset.head()
3
Output
animal accessory setting art_style animal_portrait edited_portrait
0 owl a knitted beanie a sunny park a watercolor painting ['images/animal_portrait/cde22892-ac15-49be-9a... ['images/edited_portrait/bea609e9-91da-44c5-93...
1 panda a flower crown a holiday card backdrop with snowflakes a pop art poster ['images/animal_portrait/349dc99c-5cba-4b8e-8e... ['images/edited_portrait/b0dad002-c66e-4ed5-8a...
2 dog a monocle and mustache a holiday card backdrop with snowflakes a Disney Pixar 3D render ['images/animal_portrait/0302b2f0-e0f3-41a5-8b... ['images/edited_portrait/70e2a3a5-3b35-4690-92...
3 cat a red bow tie a tropical beach at sunset a Disney Pixar 3D render ['images/animal_portrait/0ff0072f-c740-4e6f-87... ['images/edited_portrait/e0ff89a9-9af5-4ad6-89...
4 rabbit a pirate hat and eye patch a tropical beach at sunset a watercolor painting ['images/animal_portrait/6086abbf-8c16-4b1e-8d... ['images/edited_portrait/73581c7c-0cd3-4662-a5...
Python
1for index, row in dataset.head(10).iterrows():
2    display_before_after(row, index, base_path=results.artifact_storage.base_dataset_path)
3
Output

============================================================
Record 0: owl wearing a knitted beanie
Setting: a sunny park, Style: a watercolor painting
============================================================

📷 Generated portrait:
Output

🎨 Edited version:
Output

============================================================
Record 1: panda wearing a flower crown
Setting: a holiday card backdrop with snowflakes, Style: a pop art poster
============================================================

📷 Generated portrait:
Output

🎨 Edited version:
Output

============================================================
Record 2: dog wearing a monocle and mustache
Setting: a holiday card backdrop with snowflakes, Style: a Disney Pixar 3D render
============================================================

📷 Generated portrait:
Output

🎨 Edited version:
Output

============================================================
Record 3: cat wearing a red bow tie
Setting: a tropical beach at sunset, Style: a Disney Pixar 3D render
============================================================

📷 Generated portrait:
Output

🎨 Edited version:
Output

============================================================
Record 4: rabbit wearing a pirate hat and eye patch
Setting: a tropical beach at sunset, Style: a watercolor painting
============================================================

📷 Generated portrait:
Output

🎨 Edited version:
Output

⏭️ Next steps

  • Experiment with different autoregressive models for image generation and editing
  • Try more creative editing prompts (style transfer, background replacement, artistic filters)
  • Combine image generation with text generation (e.g., generate captions using an LLM-Text column with ImageContext)
  • Chain more than two image columns for multi-step editing pipelines

Related tutorials: