Configuration

View source on GitHub

FastGen uses a hierarchical Python-based configuration system built on Hydra, OmegaConf, and attrs.

Directory Structure

fastgen/configs/
├── experiments/         # Experiment configs (EDM, Flux, QwenImage, Wan, Cosmos3, LTX-2, etc.)
├── methods/             # Method-specific configs (DMD2, CM, KD, PDD, SFT, etc.)
├── callbacks.py         # Callback configurations (EMA, WandB, GradClip, etc.)
├── config_utils.py      # Utilities (import, override, serialize configs)
├── config.py            # Base config classes (BaseConfig, BaseModelConfig, BaseTrainerConfig)
├── data.py              # Dataset/dataloader configurations
├── discriminator.py     # Discriminator configs (for GAN-based methods)
├── instantiate.py       # LazyCall / instantiate helpers
├── net.py               # Network architecture configurations
└── opt.py               # Optimizer and scheduler configurations

Config Hierarchy

1. Base Configs (config.py): Core classes defining model, trainer, data, and logging settings
2. Method Configs (methods/): Extend base configs with method-specific parameters
3. Experiment Configs (experiments/): Concrete configs for specific dataset/model combinations

LazyCall Pattern

Deferred instantiation using LazyCall:

from fastgen.configs.experiments.EDM.config_dmd2_fast import create_config
from fastgen.configs.instantiate import instantiate
config = create_config()  # config.model_class is a deferred LazyCall
model = instantiate(config.model_class, config=config.model)

Custom Configs

A config is a Python file whose create_config() returns the config, usually an edited method or experiment config:

# fastgen/configs/experiments/EDM/config_dmd2_fast_small.py
from fastgen.configs.experiments.EDM.config_dmd2_fast import create_config as base_create_config
def create_config():
    config = base_create_config()
    config.dataloader_train.batch_size = 32
    return config

train.py imports the file as a module, so --config must be a relative path below the working directory; launch from the repository root or add the working directory to PYTHONPATH.

Command-Line Arguments

python train.py --config=path/to/config.py [--log_level LEVEL] [--dryrun] - key=value

Examples:

# Override training settings
python train.py --config=fastgen/configs/experiments/EDM/config_dmd2_fast.py - \
    trainer.max_iter=10000 \
    model.gan_loss_weight_gen=0. \
    log_config.name=my_experiment
# Debug config without training
python train.py --config=fastgen/configs/experiments/EDM/config_dmd2_fast.py --dryrun
# Verbose logging
python train.py --config=fastgen/configs/experiments/EDM/config_dmd2_fast.py --log_level DEBUG

Overrides use Hydra’s grammar: key=value changes a key that the config defines, +key=value adds one that it does not, ++key=value does either, and key=null sets None. To replace a dict, see config_utils.py.

Key Config Classes

Run Directory

A run writes its resolved config.yaml, W&B files, and by default its checkpoints to log_config.save_path, $FASTGEN_OUTPUT_ROOT/{project}/{group}/{name}, where name defaults to the config file’s name. With trainer.resume=True (the default), a run resumes from its latest checkpoint, so give independent runs of one config distinct names; trainer.max_iter counts the restored iterations too.

Distributed Checkpoint Storage

For multi-node runs, trainer.checkpointer.save_dir must be on shared storage (e.g. Lustre or NFS) or S3; a relative one (default checkpoints) goes in <project>/<group>/<name>/ under $FASTGEN_OUTPUT_ROOT, or under s3_container if use_s3=True. FSDP runs save sharded checkpoints and can resume only from those; to start one from a monolithic (non-FSDP) checkpoint, use trainer.checkpointer.pretrained_ckpt_path and a new run directory. The module docstring of checkpointer.py covers the formats and how they load.

Training Data

The CIFAR-10 and ImageNet recipes read the datasets that scripts/download_data.py prepares. The other recipes ship placeholder data paths that a run must override, e.g. - dataloader_train.datatags=[WDS:/data/my_video_shards]; see data.py.

Large Models, Precision, and LoRA

torchrun distributes training only when the config sets trainer.ddp or trainer.fsdp. config.py documents the sharding (trainer.fsdp*, model.meta_init), precision (model.precision*), and compilation (model.torch_compile_mode) fields where it defines them, and lora_utils.py documents LoRA training with model.net_adapter. Recipes with measured GPU-memory limits state them in their docstrings.

Environment Variables

Export these in the shell or put them in a .env file at the repository root: fastgen/__init__.py loads it with python-dotenv when the package is imported, and variables already present in the environment take precedence. To start, copy the template .env.example to .env.

Core Variables

Loading Credentials from Files

As an alternative to setting environment variables directly, credentials can be loaded automatically from files in the folder given by the config’s credentials_dir (default ./credentials/; override it with, e.g., - credentials_dir=/path/to/credentials). A non-empty file overrides the matching environment variable, and a missing or empty file leaves it untouched. Token files must hold the token on a single line.

Format for s3.json:

{
    "aws_access_key_id": "<your_access_key>",
    "aws_secret_access_key": "<your_secret_key>",
    "aws_session_token": "<optional_session_token>",
    "region_name": "<region>",
    "endpoint_url": "<s3_endpoint_url>"
}