Skip to main content
Ctrl+K
NeMo-Export-Deploy - Home NeMo-Export-Deploy - Home

NeMo-Export-Deploy

  • GitHub
NeMo-Export-Deploy - Home NeMo-Export-Deploy - Home

NeMo-Export-Deploy
  • GitHub

Table of Contents

🚀 NeMo Export and Deploy

  • Deploy NeMo Framework Models
  • Deploy Large Language Models
  • Deploy NeMo Models Using NIM LLM Containers
  • Deploy NeMo Models by Exporting to Inference Optimized Libraries
  • Deploy NeMo Models by Exporting TensorRT-LLM
  • Deploy NeMo Models by Exporting vLLM
  • Deploy NeMo Models in the Framework
  • Deploy NeMo Models using Ray
  • Send Queries to the NVIDIA Triton Server for NeMo LLMs
  • Deploy NeMo AutoModel LLM Models in the Framework
  • Deploy Hugging Face Models by Exporting to TensorRT-LLM
  • Deploy NeMo AutoModel LLM Models using Ray
  • Deploy NeMo Multimodal Models

🛠️ Development

  • Test NeMo Export-Deploy
  • Documentation Development
  • API Reference
    • nemo_export
      • nemo_export.model_adapters
        • nemo_export.model_adapters.embedding
        • nemo_export.model_adapters.reranker
      • nemo_export.multimodal
        • nemo_export.multimodal.build
        • nemo_export.multimodal.run
      • nemo_export.trt_llm
        • nemo_export.trt_llm.nemo_ckpt_loader
        • nemo_export.trt_llm.qnemo
        • nemo_export.trt_llm.tensorrt_llm_run
        • nemo_export.trt_llm.utils
      • nemo_export.utils
        • nemo_export.utils._mock_import
        • nemo_export.utils.constants
        • nemo_export.utils.lora_converter
        • nemo_export.utils.model_loader
        • nemo_export.utils.utils
      • nemo_export.onnx_llm_exporter
      • nemo_export.package_info
      • nemo_export.sentencepiece_tokenizer
      • nemo_export.tarutils
      • nemo_export.tensorrt_llm
      • nemo_export.tensorrt_llm_deployable_ray
      • nemo_export.tensorrt_mm_exporter
      • nemo_export.tiktoken_tokenizer
      • nemo_export.vllm_exporter
    • nemo_deploy
      • nemo_deploy.multimodal
        • nemo_deploy.multimodal.nemo_multimodal_deployable
        • nemo_deploy.multimodal.query_multimodal
      • nemo_deploy.nlp
        • nemo_deploy.nlp.hf_deployable
        • nemo_deploy.nlp.hf_deployable_ray
        • nemo_deploy.nlp.megatronllm_deployable
        • nemo_deploy.nlp.megatronllm_deployable_ray
        • nemo_deploy.nlp.query_llm
        • nemo_deploy.nlp.trtllm_api_deployable
      • nemo_deploy.service
        • nemo_deploy.service.fastapi_interface_to_pytriton
      • nemo_deploy.deploy_base
      • nemo_deploy.deploy_pytriton
      • nemo_deploy.deploy_ray
      • nemo_deploy.package_info
      • nemo_deploy.ray_utils
      • nemo_deploy.triton_deployable
      • nemo_deploy.utils
  • API Reference
  • nemo_deploy
  • nemo_deploy.nlp

nemo_deploy.nlp#

Submodules#

  • nemo_deploy.nlp.hf_deployable
  • nemo_deploy.nlp.hf_deployable_ray
  • nemo_deploy.nlp.megatronllm_deployable
  • nemo_deploy.nlp.megatronllm_deployable_ray
  • nemo_deploy.nlp.query_llm
  • nemo_deploy.nlp.trtllm_api_deployable

Package Contents#

Data#

__all__

API#

nemo_deploy.nlp.__all__ = ['NemoQueryLLM', 'NemoQueryLLMHF', 'NemoQueryLLMPyTorch', 'NemoQueryTRTLLMAPI', 'NemoQueryvLLM']#

previous

nemo_deploy.multimodal.query_multimodal

next

nemo_deploy.nlp.hf_deployable
On this page
  • Submodules
  • Package Contents
    • Data
    • API
      • __all__
NVIDIA NVIDIA
Privacy Policy | Manage My Privacy | Do Not Sell or Share My Data | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2025, NVIDIA Corporation.