Inference Parameters

Inference parameters control how models generate responses during synthetic data generation. Data Designer provides three types of inference parameters: ChatCompletionInferenceParams for text/code/structured generation, EmbeddingInferenceParams for embedding generation, and ImageInferenceParams for image generation.

Overview

When you create a ModelConfig, you can specify inference parameters to adjust model behavior. These parameters control aspects like randomness (temperature), diversity (top_p), context size (max_tokens), and more. Data Designer supports both static values and dynamic distribution-based sampling for certain parameters.

Chat Completion Inference Parameters

The ChatCompletionInferenceParams class controls how models generate text completions (for text, code, and structured data generation). It provides fine-grained control over generation behavior and supports both static values and dynamic distribution-based sampling.

Fields

1
import data_designer.config as dd
2
3
# High reasoning effort (more thorough, slower)
4
inference_parameters = dd.ChatCompletionInferenceParams(
5
    extra_body={"reasoning_effort": "high"}
6
)
7
8
# Medium reasoning effort (balanced)
9
inference_parameters = dd.ChatCompletionInferenceParams(
10
    extra_body={"reasoning_effort": "medium"}
11
)
12
13
# Low reasoning effort (faster, less thorough)
14
inference_parameters = dd.ChatCompletionInferenceParams(
15
    extra_body={"reasoning_effort": "low"}
16
)

Temperature and Top P Guidelines

• Temperature:

  • 0.0-0.3: Highly deterministic, focused outputs (ideal for structured/reasoning tasks)
  • 0.4-0.7: Balanced creativity and coherence (general purpose)
  • 0.8-1.0: Creative, diverse outputs (ideal for creative writing)
  • 1.0+: Highly random and experimental

• Top P:

  • 0.1-0.5: Very focused, only most likely tokens
  • 0.6-0.9: Balanced diversity
  • 0.95-1.0: Maximum diversity, including less likely tokens

Distribution-Based Inference Parameters

For temperature and top_p in ChatCompletionInferenceParams, you can specify distributions instead of fixed values. This allows Data Designer to sample different values for each generation request, introducing controlled variability into your synthetic data.

Uniform Distribution

Samples values uniformly between a low and high bound:

1
import data_designer.config as dd
2
3
inference_params = dd.ChatCompletionInferenceParams(
4
    temperature=dd.UniformDistribution(
5
        params=dd.UniformDistributionParams(low=0.7, high=1.0)
6
    ),
7
)

Manual Distribution

Samples from a discrete set of values with optional weights:

1
import data_designer.config as dd
2
3
# Equal probability for each value
4
inference_params = dd.ChatCompletionInferenceParams(
5
    temperature=dd.ManualDistribution(
6
        params=dd.ManualDistributionParams(values=[0.5, 0.7, 0.9])
7
    ),
8
)
9
10
# Weighted probabilities (normalized automatically)
11
inference_params = dd.ChatCompletionInferenceParams(
12
    top_p=dd.ManualDistribution(
13
        params=dd.ManualDistributionParams(
14
            values=[0.8, 0.9, 0.95],
15
            weights=[0.2, 0.5, 0.3]  # 20%, 50%, 30% probability
16
        )
17
    ),
18
)

Concurrency Control

The max_parallel_requests parameter controls how many concurrent API calls Data Designer makes to a specific model. This directly impacts throughput and should be tuned to match your inference server’s capacity.

Embedding Inference Parameters

The EmbeddingInferenceParams class controls how models generate embeddings. This is used when working with embedding models for tasks like semantic search or similarity analysis.

Fields

Image Inference Parameters

The ImageInferenceParams class is used for image generation models, including both diffusion models (DALL·E, Stable Diffusion, Imagen) and autoregressive models (Gemini image, GPT image). Unlike text models, image-specific options are passed entirely via extra_body, since they vary significantly between providers.

Fields

Examples

1
import data_designer.config as dd
2
3
# Autoregressive model (chat completions API, supports image context)
4
dd.ModelConfig(
5
    alias="image-model",
6
    model="black-forest-labs/flux.2-pro",
7
    provider="openrouter",
8
    inference_parameters=dd.ImageInferenceParams(
9
        extra_body={"height": 512, "width": 512}
10
    ),
11
)
12
13
# Diffusion model (e.g., DALL·E, Stable Diffusion)
14
dd.ModelConfig(
15
    alias="dalle",
16
    model="dall-e-3",
17
    provider="openai",
18
    inference_parameters=dd.ImageInferenceParams(
19
        extra_body={"size": "1024x1024", "quality": "hd"}
20
    ),
21
)

See Also

• Default Model Settings: Pre-configured model settings included with Data Designer
• Custom Model Settings: Learn how to create custom providers and model configurations
• Model Configurations: Learn about configuring model settings
• Model Providers: Learn about configuring model providers
• Architecture & Performance: Understanding separation of concerns and optimizing concurrency