nemo_rl.models.generation.vllm.metric_names#
Names for the per-step vLLM engine metrics: vLLM’s, and the ones we log.
Split out of utils.py (which imports torch) so the declarations stay
importable on their own, and kept beside compute_engine_step_metrics, which
is what produces the keys.
Module Contents#
Data#
API#
- nemo_rl.models.generation.vllm.metric_names.PROMPT_TOKEN_COUNTERS#
(‘vllm:prompt_tokens’,)
- nemo_rl.models.generation.vllm.metric_names.GENERATION_TOKEN_COUNTERS#
(‘vllm:generation_tokens’,)
- nemo_rl.models.generation.vllm.metric_names.PROMPT_LENGTH_HISTOGRAMS#
(‘vllm:request_prompt_tokens’,)
- nemo_rl.models.generation.vllm.metric_names.GENERATION_LENGTH_HISTOGRAMS#
(‘vllm:request_generation_tokens’,)
- nemo_rl.models.generation.vllm.metric_names.REQUEST_SUCCESS_COUNTERS#
(‘vllm:request_success’,)
- nemo_rl.models.generation.vllm.metric_names.OK_FINISH_REASONS#
‘frozenset(…)’
- nemo_rl.models.generation.vllm.metric_names.FINISHED_REASON_LABEL#
‘finished_reason’
- nemo_rl.models.generation.vllm.metric_names.PROMPT_TOKENS_KEY#
‘vllm/prompt_tokens’
- nemo_rl.models.generation.vllm.metric_names.GENERATION_TOKENS_KEY#
‘vllm/generation_tokens’
- nemo_rl.models.generation.vllm.metric_names.PROMPT_LENGTH_MEAN_KEY#
‘vllm/prompt_length_mean’
- nemo_rl.models.generation.vllm.metric_names.GENERATION_LENGTH_MEAN_KEY#
‘vllm/generation_length_mean’
- nemo_rl.models.generation.vllm.metric_names.GENERATIONS_OK_KEY#
‘vllm/generations_ok’
- nemo_rl.models.generation.vllm.metric_names.GENERATIONS_FAILED_KEY#
‘vllm/generations_failed’
- nemo_rl.models.generation.vllm.metric_names.VLLM_TEED_METRICS#
()
- nemo_rl.models.generation.vllm.metric_names.BATCH_DURATION_METRIC#
‘rl.vllm.batch.duration’
- nemo_rl.models.generation.vllm.metric_names.VLLM_RECORDED_METRICS#
()
- nemo_rl.models.generation.vllm.metric_names.BATCH_DURATION_KEY#
None