How-To Guides#

Task-focused guides for nemotron steps run byob/bfcl with the bfcl family.

Start with Getting Started with Building Function-Calling Benchmarks if you have not produced a benchmark.parquet yet.

Get a Pack#

Everything downstream depends on the pack, so start by choosing how you will produce one. All three routes end at the same generation pipeline and the same Gold gate; refer to Authoring Flows for the trade-offs.

Starting with existing domain assets?

Follow From Domain Assets to an Oracle Pack for a compact example that shows the manual and model-assisted conventional-source paths, from tool interface and deterministic state to a reviewed Oracle Pack. MCP onboarding remains a third route below.

From domain assets to a pack

Map an executable domain onto the manual and model-assisted paths, then converge on the same Gold pack.

From Domain Assets to an Oracle Pack
Author a pack by hand

Scaffold the pack files, write the tools, templates, and assertions, then validate without generating.

Hand-Author an Oracle Pack
Draft a pack with model assistance

Certify a Python package or reviewed HTTP service, then draft, review, approve, and freeze a candidate pack.

Author a Pack with Model Assistance
Onboard an MCP server

Discover a server, expose it through the oracle gateway, and take its evidence into the same authoring flow.

Onboard an MCP Server as the Oracle

Publish, Translate, and Evaluate#

Publish a release

Choose a size target and the balancing mixes, generate at publication scale, then verify the manifest and exports.

Publish a Benchmark Release
Translate a published benchmark

Localize approved model-facing text from a committed run_manifest.json without changing oracle truth.

Translate a Published Benchmark
Run an evaluation

Point an evaluation config at a published benchmark and a candidate endpoint, then read the report.

Evaluate a Candidate Model

When Something Fails#

The step declares its failure modes with recovery guidance, and Troubleshooting indexes them by symptom. Two habits save the most time:

  • Run stage=prepare first to validate a pack without paying for generation.

  • When generation drops tasks, join the adjacent stage_cache/ tables on task_id to find the stage that dropped them, as described in Output Files.