nemo_rl.modelopt.utils

Lightweight ModelOpt helpers shared by Megatron and vLLM workers.

Module Contents

Functions

_iter_quant_ignore_suffix_variants	Yield name and, if it ends in a known quant suffix, the stripped form.
iter_quant_ignore_name_candidates	Yield name variants matched by ModelOpt real-quant ignore patterns.
matches_quant_ignore_pattern	Return whether name matches any ModelOpt real-quant ignore pattern.
build_vllm_modelopt_nvfp4_config	Build the HuggingFace quantization_config consumed by vLLM ModelOpt NVFP4.
_resolve_effective_quantizer_formats	Resolve enabled weight and input formats from ordered ModelOpt entries.
_is_float_format	Return whether a ModelOpt numeric-format value names the requested float.
_validate_nvfp4_quantizer_format	Validate the block-16 E2M1 format supported by vLLM ModelOpt NVFP4.
resolve_nvfp4_real_quant_mode	Resolve a ModelOpt training config to its supported real-quant mode.
resolve_quant_cfg	Resolve a quantization config string into a dict consumable by mtq.quantize.

Data

MODELOPT_REAL_QUANT_ZMQ_TIMEOUT_MS
_QUANT_IGNORE_NAME_SUFFIXES
DEFAULT_NVFP4_IGNORE
NVFP4RealQuantMode
_NVFP4_REAL_QUANT_MODES

API

nemo_rl.modelopt.utils.MODELOPT_REAL_QUANT_ZMQ_TIMEOUT_MS

600000

nemo_rl.modelopt.utils._QUANT_IGNORE_NAME_SUFFIXES

(‘.weight’, ‘.weight_scale’, ‘.weight_scale_2’, ‘.input_scale’)

nemo_rl.modelopt.utils.DEFAULT_NVFP4_IGNORE

[‘lm_head’, ‘output_layer’, ‘*mlp.gate’, ‘router’, ‘block_sparse_moe.gate’, ‘self_attention’…

nemo_rl.modelopt.utils.NVFP4RealQuantMode

None

nemo_rl.modelopt.utils._NVFP4_REAL_QUANT_MODES

‘frozenset(…)’

nemo_rl.modelopt.utils._iter_quant_ignore_suffix_variants(name: str) → Iterator[str]

Yield name and, if it ends in a known quant suffix, the stripped form.

nemo_rl.modelopt.utils.iter_quant_ignore_name_candidates(name: str) → Iterator[str]

Yield name variants matched by ModelOpt real-quant ignore patterns.

nemo_rl.modelopt.utils.matches_quant_ignore_pattern(name: str, patterns: list[str]) → bool

Return whether name matches any ModelOpt real-quant ignore pattern.

nemo_rl.modelopt.utils.build_vllm_modelopt_nvfp4_config(

*,

mode: nemo_rl.modelopt.utils.NVFP4RealQuantMode,

ignore: list[str] | None = None,

) → dict[str, Any]

Build the HuggingFace quantization_config consumed by vLLM ModelOpt NVFP4.

NeMo-RL’s quant_cfg recipes are ModelOpt PTQ/QAT configs consumed by mtq.quantize. vLLM expects the deployment/export-side quantization_config shape instead.

nemo_rl.modelopt.utils._resolve_effective_quantizer_formats(

quant_cfg: collections.abc.Sequence[collections.abc.Mapping[str, Any]],

*,

source: str,

) → tuple[list[object], list[object]]

Resolve enabled weight and input formats from ordered ModelOpt entries.

nemo_rl.modelopt.utils._is_float_format(

value: object,

exponent_bits: int,

mantissa_bits: int,

) → bool

Return whether a ModelOpt numeric-format value names the requested float.

nemo_rl.modelopt.utils._validate_nvfp4_quantizer_format(

format_cfg: object,

*,

quantizer_name: str,

source: str,

) → None

Validate the block-16 E2M1 format supported by vLLM ModelOpt NVFP4.

nemo_rl.modelopt.utils.resolve_nvfp4_real_quant_mode(

quant_cfg: str,

) → nemo_rl.modelopt.utils.NVFP4RealQuantMode

Resolve a ModelOpt training config to its supported real-quant mode.

The mode is derived from all effective weight and input quantizer entries, including model-specific selectors, rather than from a config name. NVFP4 weights with no enabled input quantizer resolve to W4A16; block-16 NVFP4 input quantization resolves to W4A4. FP8, W4A8, sequential, mixed, and other activation formats fail loudly because vLLM would otherwise run a kernel with incompatible semantics.

nemo_rl.modelopt.utils.resolve_quant_cfg(quant_cfg: str) → dict[str, Any]

Resolve a quantization config string into a dict consumable by mtq.quantize.

Resolution order:

1. Built-in ModelOpt config constant exposed on modelopt.torch.quantization (e.g. "NVFP4_DEFAULT_CFG", "FP8_DEFAULT_CFG").

2. A ModelOpt PTQ recipe — either the name of a built-in recipe shipped under modelopt_recipes/ (e.g. "general/ptq/nvfp4_default-fp8_kv"; the .yml / .yaml suffix is optional) or the path to a user-authored YAML recipe. Resolution is performed by modelopt.recipe.load_config, which searches the filesystem first and then the built-in recipe library. For Ray/container workers, use an absolute path for user-authored recipe files; NeMo-RL repo-relative recipe paths are not resolved here.

YAML recipes are expected to follow the standard ModelOpt PTQ recipe layout with a top-level quantize: section in the {"quant_cfg": [...], "algorithm": ...} shape that mtq.quantize expects. A bare {"quant_cfg": [...], "algorithm": ...} document (without a wrapping quantize: key) is also accepted for convenience. If algorithm is omitted, it defaults to "max" so ModelOpt’s calibration helpers see the same normalized config as mtq.quantize. The extracted dict — not the full recipe — is returned.

See modelopt_recipes/general/ptq/ in the NVIDIA/Model-Optimizer repo (https://github.com/NVIDIA/Model-Optimizer) for the canonical format and examples/modelopt/quant_configs/ for a user-authored example.