Deploy Megatron-Bridge LLMs by Exporting to Inference Optimized Libraries

Export-Deploy supports optimizing and deploying Megatron-Bridge checkpoints using inference-optimized libraries such as vLLM and TensorRT-LLM.

• vLLM

Note: Support for exporting and deploying Megatron-Bridge models with TensorRT-LLM is coming soon. Please check back for updates.