How-To Guides#

Task-focused guides for nemotron steps run byob/mcq with the mcq family.

Start with Getting Started with Building MCQ Benchmarks if you have not produced benchmark.parquet yet.

Setup and configuration#

Prepare your data

Lay out input_dir, text or Parquet inputs, and target_source_mapping.

input_dir

Prepare Your Own Domain Data
Domain corpus files

Create per-target directories of .txt files and match them to YAML.

corpus

Using Your Own Domain Data
Model endpoints

Configure OpenAI-compatible providers for generation, judgement, expansion, validity, and filters.

yaml

Configure Model Endpoints for BYOB

Advanced workflows#

Prompt tuning

Point prompt_config at a YAML file that defines stage templates.

prompt

Prompt Tuning for Benchmarks
Skip stages

Resume with skip_until and cached Parquet files.

iteration

Skip Stages When Iterating

Workflow overview#

        flowchart LR
    A[Prepare data layout] --> B[Edit YAML]
    B --> C[uv run nemotron steps run byob/mcq]
    C --> D{Need translation?}
    D -->|yes| E[translate config + passthrough]
    D -->|no| F[Done]
    E --> F
    C -.->|iterate| G[skip_until]
    G --> C