Tutorials

Image-to-Image Editing

View as Markdown
Run in Google Colab

🎨 Data Designer Tutorial: Image-to-Image Editing

📚 What you'll learn

This notebook shows how to chain image generation columns: first generate animal portraits from text, then edit those generated images by adding accessories and changing styles—all without loading external datasets.

  • 🖼️ Text-to-image generation: Generate images from text prompts
  • 🔗 Chaining image columns: Use ImageContext to pass generated images to a follow-up editing column
  • 🎲 Sampler-driven diversity: Combine sampled accessories and settings for varied edits

This tutorial uses an autoregressive model (one that supports both text-to-image and image-to-image generation via the chat completions API). Diffusion models (DALL·E, Stable Diffusion, etc.) do not support image context—see Tutorial 5 for text-to-image generation with diffusion models.

Prerequisites: This tutorial uses OpenRouter with the Flux 2 Pro model. Set OPENROUTER_API_KEY in your environment before running.

If this is your first time using Data Designer, we recommend starting with the first notebook in this tutorial series.

📦 Import Data Designer

  • data_designer.config provides the configuration API.
  • DataDesigner is the main interface for generation.
Python
1import base64
2import os
3from pathlib import Path
4 
5from IPython.display import Image as IPImage
6from IPython.display import display
7 
8import data_designer.config as dd
9from data_designer.interface import DataDesigner
10

⚙️ Initialize the Data Designer interface

We initialize Data Designer without arguments here—the image model is configured explicitly in the next cell.

Python
1data_designer = DataDesigner()
2

🎛️ Define an image model

We need an autoregressive model that supports both text-to-image and image-to-image generation via the chat completions API. This lets us generate images from text and then pass those images as context for editing.

  • Use ImageInferenceParams so Data Designer treats this model as an image generator.
  • Image-specific options are model-dependent; pass them via extra_body.

Note: This tutorial uses the Flux 2 Pro model via OpenRouter. Set OPENROUTER_API_KEY in your environment.

Python
1MODEL_PROVIDER = "openrouter"
2MODEL_ID = "black-forest-labs/flux.2-pro"
3MODEL_ALIAS = "image-model"
4 
5model_configs = [
6    dd.ModelConfig(
7        alias=MODEL_ALIAS,
8        model=MODEL_ID,
9        provider=MODEL_PROVIDER,
10        inference_parameters=dd.ImageInferenceParams(
11            extra_body={"height": 512, "width": 512},
12        ),
13    )
14]
15

🏗️ Build the configuration

We chain two image generation columns:

  1. Sampler columns — randomly sample animal types, accessories, settings, and art styles
  2. First image column — generate an animal portrait from a text prompt
  3. Second image column with context — edit the generated portrait using ImageContext
Python
1config_builder = dd.DataDesignerConfigBuilder(model_configs=model_configs)
2 
3# 1. Sampler columns for diversity
4config_builder.add_column(
5    dd.SamplerColumnConfig(
6        name="animal",
7        sampler_type=dd.SamplerType.CATEGORY,
8        params=dd.CategorySamplerParams(
9            values=["cat", "dog", "fox", "owl", "rabbit", "panda"],
10        ),
11    )
12)
13 
14config_builder.add_column(
15    dd.SamplerColumnConfig(
16        name="accessory",
17        sampler_type=dd.SamplerType.CATEGORY,
18        params=dd.CategorySamplerParams(
19            values=[
20                "a tiny top hat",
21                "oversized sunglasses",
22                "a red bow tie",
23                "a knitted beanie",
24                "a flower crown",
25                "a monocle and mustache",
26                "a pirate hat and eye patch",
27                "a chef hat",
28            ],
29        ),
30    )
31)
32 
33config_builder.add_column(
34    dd.SamplerColumnConfig(
35        name="setting",
36        sampler_type=dd.SamplerType.CATEGORY,
37        params=dd.CategorySamplerParams(
38            values=[
39                "a cozy living room",
40                "a sunny park",
41                "a photo studio with soft lighting",
42                "a red carpet event",
43                "a holiday card backdrop with snowflakes",
44                "a tropical beach at sunset",
45            ],
46        ),
47    )
48)
49 
50config_builder.add_column(
51    dd.SamplerColumnConfig(
52        name="art_style",
53        sampler_type=dd.SamplerType.CATEGORY,
54        params=dd.CategorySamplerParams(
55            values=[
56                "a photorealistic style",
57                "a Disney Pixar 3D render",
58                "a watercolor painting",
59                "a pop art poster",
60            ],
61        ),
62    )
63)
64 
65# 2. Generate animal portrait from text
66config_builder.add_column(
67    dd.ImageColumnConfig(
68        name="animal_portrait",
69        prompt="A close-up portrait photograph of a {{ animal }} looking at the camera, studio lighting, high quality.",
70        model_alias=MODEL_ALIAS,
71    )
72)
73 
74# 3. Edit the generated portrait
75config_builder.add_column(
76    dd.ImageColumnConfig(
77        name="edited_portrait",
78        prompt=(
79            "Edit this {{ animal }} portrait photo. "
80            "Add {{ accessory }} on the animal. "
81            "Place the {{ animal }} in {{ setting }}. "
82            "Render the result in {{ art_style }}. "
83            "Keep the animal's face, expression, and features faithful to the original photo."
84        ),
85        model_alias=MODEL_ALIAS,
86        multi_modal_context=[dd.ImageContext(column_name="animal_portrait")],
87    )
88)
89 
90data_designer.validate(config_builder)
91
Output
[20:16:39] [INFO] ✅ Validation passed

🔁 Preview: quick iteration

In preview mode, generated images are stored as base64 strings in the dataframe. Use this to iterate on your prompts, accessories, and sampler values before scaling up.

Python
1preview = data_designer.preview(config_builder, num_records=2)
2
Output
[20:16:39] [INFO] 🎥 Preview generation in progress
[20:16:39] [INFO]   |-- 🔒 Jinja rendering engine: secure
[20:16:39] [INFO] ✅ Validation passed
[20:16:39] [INFO] ⛓️ Sorting column configs into a Directed Acyclic Graph
[20:16:39] [INFO] 🩺 Running health checks for models...
[20:16:39] [INFO]   |-- 👀 Checking 'black-forest-labs/flux.2-pro' in provider named 'openrouter' for model alias 'image-model'...
[20:16:48] [INFO]   |-- ✅ Passed!
[20:16:48] [INFO] ⚡ Using async task-queue preview
[20:16:48] [INFO] 🖼️ image model config for column 'animal_portrait'
[20:16:48] [INFO]   |-- model: 'black-forest-labs/flux.2-pro'
[20:16:48] [INFO]   |-- model alias: 'image-model'
[20:16:48] [INFO]   |-- model provider: 'openrouter'
[20:16:48] [INFO]   |-- inference parameters:
[20:16:48] [INFO]   |  |-- generation_type=image
[20:16:48] [INFO]   |  |-- max_parallel_requests=4
[20:16:48] [INFO]   |  |-- extra_body={'height': 512, 'width': 512}
[20:16:48] [INFO] 🖼️ image model config for column 'edited_portrait'
[20:16:48] [INFO]   |-- model: 'black-forest-labs/flux.2-pro'
[20:16:48] [INFO]   |-- model alias: 'image-model'
[20:16:48] [INFO]   |-- model provider: 'openrouter'
[20:16:48] [INFO]   |-- inference parameters:
[20:16:48] [INFO]   |  |-- generation_type=image
[20:16:48] [INFO]   |  |-- max_parallel_requests=4
[20:16:48] [INFO]   |  |-- extra_body={'height': 512, 'width': 512}
[20:16:48] [INFO] ⚡️ Async generation: 2 column(s) (column 'animal_portrait', column 'edited_portrait'), 4 tasks across 1 row group(s)
[20:16:48] [INFO] 🚀 (1/1) Dispatching with 2 records
[20:16:48] [INFO] 🎲 (1/1) Preparing samplers to generate 2 records across 4 columns
[20:16:56] [INFO] 📊 Progress [8.7s]:
[20:16:56] [INFO]   |-- 🐥 column 'animal_portrait': 1/2 (50%) 0.1 rec/s
[20:16:56] [INFO]   |-- 🐱 column 'edited_portrait': 0/2 (0%) 0.0 rec/s
[20:17:09] [INFO] 📊 Progress [21.5s]:
[20:17:09] [INFO]   |-- 🐔 column 'animal_portrait': 2/2 (100%) 0.1 rec/s
[20:17:09] [INFO]   |-- 😸 column 'edited_portrait': 1/2 (50%) 0.0 rec/s
[20:17:09] [INFO] 📊 Progress [21.6s]:
[20:17:09] [INFO]   |-- 🐔 column 'animal_portrait': 2/2 (100%) 0.1 rec/s
[20:17:09] [INFO]   |-- 🦁 column 'edited_portrait': 2/2 (100%) 0.1 rec/s
[20:17:09] [INFO] ✅ Async generation complete [21.6s]: 4 ok, 0 failed across 2 column(s)
[20:17:09] [INFO] 📊 Model usage summary:
[20:17:09] [INFO]   |-- model: black-forest-labs/flux.2-pro
[20:17:09] [INFO]   |-- tokens: input=9203, output=12288, reasoning=0, total=21491, tps=996
[20:17:09] [INFO]   |-- requests: success=4, failed=0, total=4, rpm=11
[20:17:09] [INFO]   |-- images: total=4
[20:17:09] [INFO] 📐 Measuring dataset column statistics:
[20:17:09] [INFO]   |-- 🎲 column: 'animal'
[20:17:09] [INFO]   |-- 🎲 column: 'accessory'
[20:17:09] [INFO]   |-- 🎲 column: 'setting'
[20:17:09] [INFO]   |-- 🎲 column: 'art_style'
[20:17:09] [INFO]   |-- 🖼️ column: 'animal_portrait'
[20:17:09] [INFO]   |-- 🖼️ column: 'edited_portrait'
[20:17:09] [INFO] 👏 Preview complete!
Python
1for i in range(len(preview.dataset)):
2    preview.display_sample_record()
3
Output
[index: 0]
                                                                                                              
                                              Generated Columns                                               
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Name                           ┃ Value                                                                     ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ animal                         │ fox                                                                       │
├────────────────────────────────┼───────────────────────────────────────────────────────────────────────────┤
│ accessory                      │ a red bow tie                                                             │
├────────────────────────────────┼───────────────────────────────────────────────────────────────────────────┤
│ setting                        │ a red carpet event                                                        │
├────────────────────────────────┼───────────────────────────────────────────────────────────────────────────┤
│ art_style                      │ a Disney Pixar 3D render                                                  │
└────────────────────────────────┴───────────────────────────────────────────────────────────────────────────┘
                                                                                                              
                                                                                                              
                                                    Images                                                    
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Name                                   ┃ Preview                                                           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ animal_portrait                        │ [0] <base64, 1971932 chars>                                       │
├────────────────────────────────────────┼───────────────────────────────────────────────────────────────────┤
│ edited_portrait                        │ [0] <base64, 1636712 chars>                                       │
└────────────────────────────────────────┴───────────────────────────────────────────────────────────────────┘
🖼️ animal_portrait[0]
🖼️ edited_portrait[0]
[index: 1]
                                                                                                              
                                              Generated Columns                                               
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Name                          ┃ Value                                                                      ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ animal                        │ dog                                                                        │
├───────────────────────────────┼────────────────────────────────────────────────────────────────────────────┤
│ accessory                     │ a pirate hat and eye patch                                                 │
├───────────────────────────────┼────────────────────────────────────────────────────────────────────────────┤
│ setting                       │ a red carpet event                                                         │
├───────────────────────────────┼────────────────────────────────────────────────────────────────────────────┤
│ art_style                     │ a pop art poster                                                           │
└───────────────────────────────┴────────────────────────────────────────────────────────────────────────────┘
                                                                                                              
                                                                                                              
                                                    Images                                                    
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Name                                   ┃ Preview                                                           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ animal_portrait                        │ [0] <base64, 1643424 chars>                                       │
├────────────────────────────────────────┼───────────────────────────────────────────────────────────────────┤
│ edited_portrait                        │ [0] <base64, 1554424 chars>                                       │
└────────────────────────────────────────┴───────────────────────────────────────────────────────────────────┘
🖼️ animal_portrait[0]
🖼️ edited_portrait[0]
Python
1preview.dataset
2
Output
animal accessory setting art_style animal_portrait edited_portrait
0 fox a red bow tie a red carpet event a Disney Pixar 3D render [iVBORw0KGgoAAAANSUhEUgAABAAAAAMACAIAAAA12IJaA... [iVBORw0KGgoAAAANSUhEUgAABAAAAAMACAIAAAA12IJaA...
1 dog a pirate hat and eye patch a red carpet event a pop art poster [iVBORw0KGgoAAAANSUhEUgAABAAAAAMACAIAAAA12IJaA... [iVBORw0KGgoAAAANSUhEUgAABAAAAAMACAIAAAA12IJaA...

🔎 Compare original vs edited

Let's display the generated animal portraits next to their edited versions.

Python
1def display_image(image_value, base_path: Path | None = None) -> None:
2    """Display an image from base64 (preview mode) or file path (create mode)."""
3    values = [image_value] if isinstance(image_value, str) else list(image_value)
4    for value in values:
5        if base_path is not None:
6            display(IPImage(filename=str(base_path / value)))
7        else:
8            display(IPImage(data=base64.b64decode(value)))
9 
10 
11def display_before_after(row, index: int, base_path: Path | None = None) -> None:
12    """Display original portrait vs edited version for a single record."""
13    print(f"\n{'=' * 60}")
14    print(f"Record {index}: {row['animal']} wearing {row['accessory']}")
15    print(f"Setting: {row['setting']}, Style: {row['art_style']}")
16    print(f"{'=' * 60}")
17 
18    print("\n📷 Generated portrait:")
19    display_image(row["animal_portrait"], base_path)
20 
21    print("\n🎨 Edited version:")
22    display_image(row["edited_portrait"], base_path)
23
Python
1for index, row in preview.dataset.iterrows():
2    display_before_after(row, index)
3
Output

============================================================
Record 0: fox wearing a red bow tie
Setting: a red carpet event, Style: a Disney Pixar 3D render
============================================================

📷 Generated portrait:
Output

🎨 Edited version:
Output

============================================================
Record 1: dog wearing a pirate hat and eye patch
Setting: a red carpet event, Style: a pop art poster
============================================================

📷 Generated portrait:
Output

🎨 Edited version:
Output

🆙 Create at scale

In create mode, images are saved to disk in images/<column_name>/ folders with UUID filenames. The dataframe stores relative paths. ImageContext auto-detection handles this transparently—generated file paths are resolved to base64 before being sent to the model for editing.

Python
1create_num_records = int(os.environ.get("DATA_DESIGNER_FLUX_2_PRO_CREATE_NUM_RECORDS", "5"))
2results = data_designer.create(
3    config_builder,
4    num_records=create_num_records,
5    dataset_name="tutorial-6-edited-images",
6)
7
Output
[20:17:09] [INFO] OpenTelemetry metrics available at http://127.0.0.1:9464/metrics
[20:17:09] [INFO] 🎨 Creating Data Designer dataset
[20:17:09] [INFO]   |-- 🔒 Jinja rendering engine: secure
[20:17:09] [INFO] ✅ Validation passed
[20:17:09] [INFO] ⛓️ Sorting column configs into a Directed Acyclic Graph
[20:17:09] [INFO] 🩺 Running health checks for models...
[20:17:09] [INFO]   |-- 👀 Checking 'black-forest-labs/flux.2-pro' in provider named 'openrouter' for model alias 'image-model'...
[20:17:18] [INFO]   |-- ✅ Passed!
[20:17:18] [INFO] ⚡ Using async task-queue builder
[20:17:18] [INFO] 🖼️ image model config for column 'animal_portrait'
[20:17:18] [INFO]   |-- model: 'black-forest-labs/flux.2-pro'
[20:17:18] [INFO]   |-- model alias: 'image-model'
[20:17:18] [INFO]   |-- model provider: 'openrouter'
[20:17:18] [INFO]   |-- inference parameters:
[20:17:18] [INFO]   |  |-- generation_type=image
[20:17:18] [INFO]   |  |-- max_parallel_requests=4
[20:17:18] [INFO]   |  |-- extra_body={'height': 512, 'width': 512}
[20:17:18] [INFO] 🖼️ image model config for column 'edited_portrait'
[20:17:18] [INFO]   |-- model: 'black-forest-labs/flux.2-pro'
[20:17:18] [INFO]   |-- model alias: 'image-model'
[20:17:18] [INFO]   |-- model provider: 'openrouter'
[20:17:18] [INFO]   |-- inference parameters:
[20:17:18] [INFO]   |  |-- generation_type=image
[20:17:18] [INFO]   |  |-- max_parallel_requests=4
[20:17:18] [INFO]   |  |-- extra_body={'height': 512, 'width': 512}
[20:17:18] [INFO] ⚡️ Async generation: 2 column(s) (column 'animal_portrait', column 'edited_portrait'), 10 tasks across 1 row group(s)
[20:17:18] [INFO] 🚀 (1/1) Dispatching with 5 records
[20:17:18] [INFO] 🎲 (1/1) Preparing samplers to generate 5 records across 4 columns
[20:17:27] [INFO] 📊 Progress [8.9s]:
[20:17:27] [INFO]   |-- 🥚 column 'animal_portrait': 1/5 (20%) 0.1 rec/s
[20:17:27] [INFO]   |-- 🐱 column 'edited_portrait': 0/5 (0%) 0.0 rec/s
[20:17:36] [INFO] 📊 Progress [17.8s]:
[20:17:36] [INFO]   |-- 🐤 column 'animal_portrait': 4/5 (80%) 0.2 rec/s
[20:17:36] [INFO]   |-- 🐱 column 'edited_portrait': 0/5 (0%) 0.0 rec/s
[20:17:41] [INFO] 📊 Progress [23.1s]:
[20:17:41] [INFO]   |-- 🐔 column 'animal_portrait': 5/5 (100%) 0.2 rec/s
[20:17:41] [INFO]   |-- 😺 column 'edited_portrait': 2/5 (40%) 0.1 rec/s
[20:17:48] [INFO] 📊 Progress [29.9s]:
[20:17:48] [INFO]   |-- 🐔 column 'animal_portrait': 5/5 (100%) 0.2 rec/s
[20:17:48] [INFO]   |-- 😸 column 'edited_portrait': 3/5 (60%) 0.1 rec/s
[20:17:55] [INFO] 📊 Progress [37.2s]:
[20:17:55] [INFO]   |-- 🐔 column 'animal_portrait': 5/5 (100%) 0.1 rec/s
[20:17:55] [INFO]   |-- 🦁 column 'edited_portrait': 5/5 (100%) 0.1 rec/s
[20:17:55] [INFO] ✅ Async generation complete [37.2s]: 10 ok, 0 failed across 2 column(s)
[20:17:56] [INFO] 📊 Model usage summary:
[20:17:56] [INFO]   |-- model: black-forest-labs/flux.2-pro
[20:17:56] [INFO]   |-- tokens: input=23018, output=30720, reasoning=0, total=53738, tps=1434
[20:17:56] [INFO]   |-- requests: success=10, failed=0, total=10, rpm=16
[20:17:56] [INFO]   |-- images: total=10
[20:17:56] [INFO] 📐 Measuring dataset column statistics:
[20:17:56] [INFO]   |-- 🎲 column: 'animal'
[20:17:56] [INFO]   |-- 🎲 column: 'accessory'
[20:17:56] [INFO]   |-- 🎲 column: 'setting'
[20:17:56] [INFO]   |-- 🎲 column: 'art_style'
[20:17:56] [INFO]   |-- 🖼️ column: 'animal_portrait'
[20:17:56] [INFO]   |-- 🖼️ column: 'edited_portrait'
Python
1dataset = results.load_dataset()
2dataset.head()
3
Output
animal accessory setting art_style animal_portrait edited_portrait
0 cat a knitted beanie a holiday card backdrop with snowflakes a photorealistic style ['images/animal_portrait/82d39c9b-d18b-45b8-af... ['images/edited_portrait/23650bcc-735b-4bb4-b1...
1 panda a tiny top hat a red carpet event a pop art poster ['images/animal_portrait/b0680a10-4fcf-4b70-bc... ['images/edited_portrait/d18becc2-a3f3-4036-9f...
2 panda a chef hat a red carpet event a photorealistic style ['images/animal_portrait/8a74ec22-caff-436e-ad... ['images/edited_portrait/3f4ec33c-a597-46ca-83...
3 cat a monocle and mustache a holiday card backdrop with snowflakes a watercolor painting ['images/animal_portrait/0504f482-de49-4d20-a3... ['images/edited_portrait/3053b546-1581-4827-b6...
4 owl a pirate hat and eye patch a sunny park a Disney Pixar 3D render ['images/animal_portrait/6fc87dbe-90ce-48e4-b4... ['images/edited_portrait/09e5477e-082d-478f-a1...
Python
1for index, row in dataset.head(10).iterrows():
2    display_before_after(row, index, base_path=results.artifact_storage.base_dataset_path)
3
Output

============================================================
Record 0: cat wearing a knitted beanie
Setting: a holiday card backdrop with snowflakes, Style: a photorealistic style
============================================================

📷 Generated portrait:
Output

🎨 Edited version:
Output

============================================================
Record 1: panda wearing a tiny top hat
Setting: a red carpet event, Style: a pop art poster
============================================================

📷 Generated portrait:
Output

🎨 Edited version:
Output

============================================================
Record 2: panda wearing a chef hat
Setting: a red carpet event, Style: a photorealistic style
============================================================

📷 Generated portrait:
Output

🎨 Edited version:
Output

============================================================
Record 3: cat wearing a monocle and mustache
Setting: a holiday card backdrop with snowflakes, Style: a watercolor painting
============================================================

📷 Generated portrait:
Output

🎨 Edited version:
Output

============================================================
Record 4: owl wearing a pirate hat and eye patch
Setting: a sunny park, Style: a Disney Pixar 3D render
============================================================

📷 Generated portrait:
Output

🎨 Edited version:
Output

⏭️ Next steps

  • Experiment with different autoregressive models for image generation and editing
  • Try more creative editing prompts (style transfer, background replacement, artistic filters)
  • Combine image generation with text generation (e.g., generate captions using an LLM-Text column with ImageContext)
  • Chain more than two image columns for multi-step editing pipelines

Related tutorials: