nemo_rl.models.generation

Subpackages

• nemo_rl.models.generation.trtllm
  • nemo_rl.models.generation.trtllm.trtllm_http_server
  • nemo_rl.models.generation.trtllm.trtllm_worker_async
  • nemo_rl.models.generation.trtllm.config
  • nemo_rl.models.generation.trtllm.trtllm_generation
  • nemo_rl.models.generation.trtllm.trtllm_backend
• nemo_rl.models.generation.megatron
  • nemo_rl.models.generation.megatron.utils
  • nemo_rl.models.generation.megatron.megatron_generation
  • nemo_rl.models.generation.megatron.megatron_worker
  • nemo_rl.models.generation.megatron.config
• nemo_rl.models.generation.vllm
  • nemo_rl.models.generation.vllm.utils
  • nemo_rl.models.generation.vllm.refit_layout
  • nemo_rl.models.generation.vllm.vllm_worker_async
  • nemo_rl.models.generation.vllm.vllm_sparse_refit
  • nemo_rl.models.generation.vllm.vllm_sparse_delta
  • nemo_rl.models.generation.vllm.config
  • nemo_rl.models.generation.vllm.refit_loader
  • nemo_rl.models.generation.vllm.collective_rpc
  • nemo_rl.models.generation.vllm.worker_utils
  • nemo_rl.models.generation.vllm.vllm_backend
  • nemo_rl.models.generation.vllm.vllm_generation
  • nemo_rl.models.generation.vllm.checkpoint_engine
  • nemo_rl.models.generation.vllm.patches
  • nemo_rl.models.generation.vllm.vllm_worker

Submodules

• nemo_rl.models.generation.interfaces
• nemo_rl.models.generation.constants
• nemo_rl.models.generation.openai_server_utils

Package Contents

Functions

configure_generation_config

Apply specific configurations to generation config.

Data

TokenizerType

API

nemo_rl.models.generation.TokenizerType

None

nemo_rl.models.generation.configure_generation_config(

config: nemo_rl.models.generation.interfaces.GenerationConfig,

tokenizer: nemo_rl.models.generation.TokenizerType,

is_eval: bool = False,

has_refit_draft_weights: bool = False,

trains_mtp: bool = False,

) → nemo_rl.models.generation.interfaces.GenerationConfig

Apply specific configurations to generation config.