nemo_rl.models.generation

Subpackages

• nemo_rl.models.generation.dynamo
  • nemo_rl.models.generation.dynamo.worker_pool
  • nemo_rl.models.generation.dynamo.dynamo_worker
  • nemo_rl.models.generation.dynamo.managed_runtime
  • nemo_rl.models.generation.dynamo.dynamo_generation
  • nemo_rl.models.generation.dynamo.venv
  • nemo_rl.models.generation.dynamo.validate_dynamo_vllm_args
  • nemo_rl.models.generation.dynamo.config
  • nemo_rl.models.generation.dynamo.token_wrapper
  • nemo_rl.models.generation.dynamo.refit
  • nemo_rl.models.generation.dynamo.metrics
  • nemo_rl.models.generation.dynamo.arguments
  • nemo_rl.models.generation.dynamo.http_client
• nemo_rl.models.generation.trtllm
  • nemo_rl.models.generation.trtllm.trtllm_http_server
  • nemo_rl.models.generation.trtllm.trtllm_worker_async
  • nemo_rl.models.generation.trtllm.config
  • nemo_rl.models.generation.trtllm.trtllm_generation
  • nemo_rl.models.generation.trtllm.trtllm_backend
• nemo_rl.models.generation.megatron
  • nemo_rl.models.generation.megatron.utils
  • nemo_rl.models.generation.megatron.megatron_generation
  • nemo_rl.models.generation.megatron.megatron_worker
  • nemo_rl.models.generation.megatron.config
• nemo_rl.models.generation.vllm
  • nemo_rl.models.generation.vllm.video_utils
  • nemo_rl.models.generation.vllm.utils
  • nemo_rl.models.generation.vllm.refit_layout
  • nemo_rl.models.generation.vllm.vllm_worker_async
  • nemo_rl.models.generation.vllm.vllm_sparse_refit
  • nemo_rl.models.generation.vllm.vllm_sparse_delta
  • nemo_rl.models.generation.vllm.config
  • nemo_rl.models.generation.vllm.refit_loader
  • nemo_rl.models.generation.vllm.collective_rpc
  • nemo_rl.models.generation.vllm.worker_utils
  • nemo_rl.models.generation.vllm.vllm_backend
  • nemo_rl.models.generation.vllm.vllm_generation
  • nemo_rl.models.generation.vllm.checkpoint_engine
  • nemo_rl.models.generation.vllm.patches
  • nemo_rl.models.generation.vllm.vllm_worker

Submodules

• nemo_rl.models.generation.generation_router
• nemo_rl.models.generation.interfaces
• nemo_rl.models.generation.constants
• nemo_rl.models.generation.openai_server_utils
• nemo_rl.models.generation.fleet_health

Package Contents

Functions

resolve_generation_class	Map generation_config to its GenerationInterface class.
configure_generation_config	Apply specific configurations to generation config.

Data

TokenizerType

API

nemo_rl.models.generation.TokenizerType

None

nemo_rl.models.generation.resolve_generation_class(

generation_config: nemo_rl.models.generation.interfaces.GenerationConfig,

) → type[nemo_rl.models.generation.interfaces.GenerationInterface]

Map generation_config to its GenerationInterface class.

nemo_rl.models.generation.configure_generation_config(

config: nemo_rl.models.generation.interfaces.GenerationConfig,

tokenizer: nemo_rl.models.generation.TokenizerType,

is_eval: bool = False,

has_refit_draft_weights: bool = False,

trains_mtp: bool = False,

) → nemo_rl.models.generation.interfaces.GenerationConfig

Apply specific configurations to generation config.