nemo_rl

Subpackages

• nemo_rl.utils
  • nemo_rl.utils.config
  • nemo_rl.utils.checkpoint
  • nemo_rl.utils.nsys
  • nemo_rl.utils.native_checkpoint
  • nemo_rl.utils.nvml
  • nemo_rl.utils.prefetch_venvs
  • nemo_rl.utils.venvs
  • nemo_rl.utils.logger
  • nemo_rl.utils.timer
• nemo_rl.experience
  • nemo_rl.experience.rollouts
• nemo_rl.environments
  • nemo_rl.environments.utils
  • nemo_rl.environments.math_environment
  • nemo_rl.environments.interfaces
  • nemo_rl.environments.metrics
• nemo_rl.data
  • nemo_rl.data.eval_datasets
    • nemo_rl.data.eval_datasets.gpqa
    • nemo_rl.data.eval_datasets.mmlu_pro
    • nemo_rl.data.eval_datasets.mmlu
    • nemo_rl.data.eval_datasets.math
    • nemo_rl.data.eval_datasets.aime2024
    • nemo_rl.data.eval_datasets.local_math_dataset
  • nemo_rl.data.hf_datasets
    • nemo_rl.data.hf_datasets.chat_templates
    • nemo_rl.data.hf_datasets.openmathinstruct2
    • nemo_rl.data.hf_datasets.helpsteer3
    • nemo_rl.data.hf_datasets.squad
    • nemo_rl.data.hf_datasets.prompt_response_dataset
    • nemo_rl.data.hf_datasets.dpo
    • nemo_rl.data.hf_datasets.deepscaler
    • nemo_rl.data.hf_datasets.oasst
    • nemo_rl.data.hf_datasets.oai_format_dataset
  • nemo_rl.data.packing
    • nemo_rl.data.packing.algorithms
    • nemo_rl.data.packing.metrics
  • nemo_rl.data.datasets
  • nemo_rl.data.interfaces
  • nemo_rl.data.llm_message_utils
  • nemo_rl.data.processors
• nemo_rl.distributed
  • nemo_rl.distributed.ray_actor_environment_registry
  • nemo_rl.distributed.model_utils
  • nemo_rl.distributed.named_sharding
  • nemo_rl.distributed.batched_data_dict
  • nemo_rl.distributed.worker_group_utils
  • nemo_rl.distributed.virtual_cluster
  • nemo_rl.distributed.worker_groups
  • nemo_rl.distributed.collectives
• nemo_rl.algorithms
  • nemo_rl.algorithms.sft
  • nemo_rl.algorithms.utils
  • nemo_rl.algorithms.loss_functions
  • nemo_rl.algorithms.grpo
  • nemo_rl.algorithms.dpo
  • nemo_rl.algorithms.interfaces
• nemo_rl.converters
  • nemo_rl.converters.huggingface
    • nemo_rl.converters.huggingface.vllm_export
  • nemo_rl.converters.megatron
    • nemo_rl.converters.megatron.vllm_export
• nemo_rl.models
  • nemo_rl.models.dtensor
    • nemo_rl.models.dtensor.parallelize
  • nemo_rl.models.huggingface
    • nemo_rl.models.huggingface.common
  • nemo_rl.models.policy
    • nemo_rl.models.policy.utils
    • nemo_rl.models.policy.lm_policy
    • nemo_rl.models.policy.dtensor_policy_worker
    • nemo_rl.models.policy.interfaces
    • nemo_rl.models.policy.megatron_policy_worker
  • nemo_rl.models.generation
    • nemo_rl.models.generation.vllm_backend
    • nemo_rl.models.generation.interfaces
    • nemo_rl.models.generation.vllm
  • nemo_rl.models.megatron
    • nemo_rl.models.megatron.converters
    • nemo_rl.models.megatron.common
    • nemo_rl.models.megatron.refit_utils
    • nemo_rl.models.megatron.community_import
• nemo_rl.metrics
  • nemo_rl.metrics.metrics_utils
• nemo_rl.evals
  • nemo_rl.evals.eval
  • nemo_rl.evals.answer_parsing

Submodules

• nemo_rl.package_info

Package Contents

Functions

_patch_nsight_file

Patch the nsight.py file to fix the context.py_executable assignment.

Data

megatron_path

API

nemo_rl.megatron_path

None

nemo_rl._patch_nsight_file()

Patch the nsight.py file to fix the context.py_executable assignment.

Until this fix is upstreamed, we will maintain this patch here. This patching logic is only applied if the user intends to use nsys profiling which they enable with NRL_NSYS_WORKER_PATTERNS.

If enabled, will effectively apply the following patch in an idempotent manner:

https://github.com/ray-project/ray/compare/master…terrykong:ray:tk/nsight-py-exeutable-fix?expand=1

This hack works b/c the nsight plugin is not called from the main driver process, so as soon as nemo_rl is imported, the patch is applied and the source of the nsight.py module is up to date before the nsight.py is actually needed.