DeepEP Benchmark
DeepEP Benchmark workload (test_template_name is DeepEP) allows users to execute DeepEP (Deep Expert Parallelism) MoE (Mixture of Experts) benchmarks within the CloudAI framework.
Overview
DeepEP is a benchmark for measuring the performance of MoE models with distributed expert parallelism. It supports:
- Two operation modes: Standard and Low-Latency
- Multiple data types: bfloat16 and FP8
- Flexible network configurations: With or without NVLink
- Configurable model parameters: Experts, tokens, hidden size, top-k
- Performance profiling: Kineto profiler support
Usage Example
Test TOML example (Standard Mode):
Test TOML example (Low-Latency Mode):
Test Scenario example:
Test-in-Scenario example:
API Documentation
Command Arguments
DeepEPCmdArgs
Command arguments for the official DeepEP test scripts.
Test Definition
DeepEPTestDefinition
Test object for official DeepEP v1/v2 test scripts.